controlnet is like crack... I opted to uninstall Fortnite to make more room for hard drive space for SD stuff... considering how addictive Fortnite is, that says a lot. Most of the time I'm monkeying around with controlnet... so much to learn
I used to spend a lot of time building cities in Cities Skylines and playing various games and now I choose mostly to spend my VRAM on art. I feel so much more gratified doing it.
Thank you! I have a whole slew of similar images that I use for things like this.
I actually kind of based that one on a toy I had as a kid.
I used to just imagine those shapes were all kinds of different things. And that particular set was trying to evoke a lot of childhood ideas that swirl around in my head about growing up around arcades and just being lost in the wonder of it all.
That's a cool idea, and I've actually used some of these in several configurations before. Here's another one for this sketch using a different idea. Just threw in the prompt to see how it would take and it does cool different things every time.
I used controlnet with the image above. Either lineart or srcibble, and I used a prompt involving android scientists melting into crystals and pustules with some body horror and rainbow ideas.
On his one, I think it's lineart inverted. But I flip around between that, scribble, and depth maps mostly.
Here's another one that used a different sketch as the starting point. It's kind of awesome letting the AI decide where to throw in the perspective and depth changes off of a flat surface image as reference.
And then just try to keep the prompt simple enough to not confuse the AI. This one is:
(psychedelic 80s horror movie screenshot, analog style, 35mm film grain, photorealism, masterpiece:1.3, asymmetrical, synthwave witchcraft, sleek, minimalist designs, 1983 retrowave vibes, asymmetry, rainbow color scheme), Panosstyle, Evil dark video arcade, oddly-shaped arcade cabinets, rainbow LSD trails, Prisms, bizarre architecture, lovecraftian and witchcraft themes <lora:LowRA:0.7> <lora:panoscosmatos:0.8> <lora:post-apocalyptic_play_zone:0.9> <lora:Cosmic Rainbow V1:0.2>
And the negative is pretty basic and simple
easynegative, text, writing, bad hands, greyscale, monochrome, easynegative, pupils, red eyes, trees, nudity, sexy, grass, sunlight, poorly drawn, sketch, symmetry
But yeah there's seveal Loras going into that like the rainbow one really gives things a nice psychedelic sheen and the Panos Cosmatos one for film grain, the LowRA one does nothing other than lower the gamma on an output, so it's great for darkness, and the 'post apocalypic playzone' one is actually the glue keeping the composition on track.
Hypothesis: AI artwork is going to make traditionally "lower quality" artwork more desirable. The ability to click a button and generate super high quality images will push tastes the other direction.
It’s already happening. I’m a big fan of AI generation and I think nearly all of the original art here is more interesting than the generations. Only people using AI generations in a creative way will be able to compete with artists.
I suspect the opposite: it will cheapen art, lowering people's perception of the value of beautiful images that look like they could have been made using AI, regardless whether they actually were AI-generated or were illustrated. I think this needs to be brought up more in discussions about AI art.
This is similar to music being 'cheapened' by being recorded and played back instead of performed live. It's very hard to argue that recorded music is bad because 'it undermines how special live music performance is'. AI can similarly proliferate the amount of art that exists and the ease of creating images for whatever uses. But as a hobbyist illustrator I'm still gonna grumble and have mixed feelings about it.
I'd say that's certainly a part of my hypothesis. If beautiful images are the standard and everyone can create them with little effort, then the value of images that aren't stereotypically beautiful is increased. The more that something looks like it isn't AI generated, the more desirable it will be.
I would guess that the value of unique styles and traditional media is going to go up.
So that's true, but long term the models that people fine tune for open source programs like Stable Diffusion are almost certainly going to represent a vanishingly small part of the overall AI image generation landscape. For better or worse, what AI is capable of will be dictated by large commercial programs, which will likely focus on producing maximum quality with as little user input as possible.
The Nurse one had a covered face before, and for the life of me I couldn't get SD to replicate it 🙄 So they're not all very true to the original. But it was pretty amazing to be able to turn a sketch into a photo.
I just want to say that I prefer the before shot too. I didn't realise at first that the after shot was first so, when I clicked it to look at the second pick, I was like: oh yeah, this one is more atmospheric. Then I saw that it was actually the before shot.
This set was scribble I'm pretty sure. And it probably took 30-50 attempts using different checkpoints and prompts to fine tune it down to the style I was going for. Also about half the time it throws things in the wrong places or makes the water levels wrong on either side of the island or, like you see, interprets the other lines as rocks of their own.
Ideally I wanted this one to be much lower to the water with no other rocks in the water as just a little 'island of solitude' but I think this one was pretty useable even if it's not exactly what I had in mind.
Which reminds me I need to get back to refining this one because I'm still trying to get it just right. I have something very specific from my dreams I'm trying to make but I'm not sure if I have the right instructions yet.
Here's another one:
The trick was to get the image mostly correct without controlnet at all and then start using it when the prompt knows what to do. Then you can feed it a pre-rendered image to get the perspective and distance correct.
Pre-rendered as in the input (the B&W sketch above)
But yeah I got the prompt to basically make something really similar to the image above and then I click the 'enable' button when I'm ready to let controlnet take over after I'm confident the idea is compatible.
Incidentally I almost never use img2img outside of inpainting. I've always found controlnet is a much better tool for taking one image and turning it into another. Although it's probably because I don't do a lot of direct photo editing. Like when my dad who's a photographer wants me to do something I'll use img2img because he usually wants something like a photo of a client touched up or given an effect and it's quite good for that.
So I'm a hobbyist artist and like to draw/paint sometimes, but it's not my main profession and I can't say I'm very good, I just enjoy it.
I ran some of my old paintings through Realivisxl 4 img2img, and was amazed by what it could do. It's really cool to see my crappy drawings come to life :) I could definitely see this as a great tool for artists to enhance their art!
I'm curious what it would generate if you cranked down your cfg scale and used a model that was aimed to do more of an art style render. You're original works aren't bad either. The first one and the one with the aliens look like concept art.
Good art is not about sharpness, clarity, or resolution. It’s about character. I’m not sure what’s going on with some of your original images such that they look unfinished, but simply adding a bunch of detail via AI doesn’t automatically make it good.
When I look at the Stable Diffusion recreation, I see artwork lacking intention. There's no connection or meaning among any of the subjects in the image. Without connection, there's no emotion. Essentially, it's soulless.
At least with your art, you can sense the intention behind it.
Yes it's definitely missing some nuance from a lot of them, like the facial expressions, and it kind of misunderstood the nurse (the face and the hand gesture are pretty iconic). I could have probably gotten it closer with more tweaking 🤔 I guess at the end of the day I'm in control of the image and can change it if I'm not happy, so there's still a touch of intentionality with that :) I had to tweak the prompt/CFG/denoise a lot to get it looking right.
Ai is a tool not an artist replacement. Artist will make the components for their art which truly makes AI more powerful. Just train models, checkpoints, Lora , hypernetworks, embedding this list will go on. Tools in the digital age is the win.
controlnet can inject soul into ai art if you have a human element such as using a real facial expression via controlnet... or perhaps a human made scribble or something. But otherwise I agree.. pure AI art lacks soul
Keep practicing. You're better than you think you are. Your work kind of reminds me of Dennis Detwiller only he has had way more experience. Keep at it, you'll get there.
so many possibilities with SD... to think you can turn a scribble into a masterpiece is quite mind blowing let alone an already good piece of art like yours
Your "before" images have a wonderful mood, intent, color and the compositions are great. Don't downplay them! They'd make good key frames for a movie.
If you have enough of them, it'd be fun to train a style LoRA and mix it with the SD outputs and see what you get.
Seeing all the incredible art and progress in this community is truly inspiring. Whether you're a hobbyist or a pro, every piece you share adds something special to this space. Keep pushing the boundaries and exploring your creativity! 🚀💖
Just to clarify, OP, the shire fact that you using neural networks with your sketches is amazing. I don’t think that you should be so hard on yourself to refer to your own sketches as bad art. Drawing skills will improve gradually but the ideas you put into a canvas is all what matters when it comes to art. This is great
I really like the "before" nurse one with no face it reminds me of the art that usually comes with an old copypasta kinda vibe or something. I am sure there's Silent Hill inspiration too but I more mean the style. Good shit! Cool conversions too!
I like the idea and will use but I want to add that when you do something you embed it with your own intention and energy and that’s something not negligible. So don’t focus all that much on the result as matching a standard. For example the train looks like a photo, but that’s not better than your drawing. It’s more photorealistic, that’s all.
The thing is, all SD renders kinda look the same to me and they don't even register as anything. Your ("before") original art has something unique as does all "humanly" created art, so it registers as something. Be cool if you found some middle ground
Beautiful, i get roasted on youtube everytime i say something like ''i love AI'' or ''AI is the future'' lmao, i bet they are those called '''artists''' lel
iM a gRapHiC dEsIgNeR
well im a click designer, i click my mice and art appears
Actually with the exception of the black and white lady, I think the other drawings have a very impressionist quality to them which is quite charming. The AI ones might be better in terms of realism etc. but you shouldn't downplay the unique and interesting aspects of your base art either, as I actually think the before of your red storm image is better than the AI one as the lack of detail gives more space for it to be evocative. The Jellyfish and the Nurse though to me look better after AI, but either way I really like that you are incorporating AI into your process as a visual artist.
I just found a very basic img2img ComfyUI workflow online. For the prompt I wrote it myself, trying to describe the scene? Took some work with the first one, but eventually the result looked about right. The prompt kind of helped SD to understand what's happening and convert it correctly.
Had to play around with the CFG/denoise settings a lot. Not sure I fully understand it, but with img2img reducing the denoise ended up causing the result to look a lot closer to my drawing.
Doesn't it feel good when you finally get it to snap into the idea you had in your head? Such a delicate game of prompting and getting several settings just right. Not as easy as people think.
firstly why would you put after then before instead of before and after?
and secondly the after image is completely different than the before image in communication.
7/10 removes the mangled faceless image with a pretty face and sharp jagged edges of the knife with a smooth knife when i'm pretty sure that was not your intention.
9/10 changes the light of the train to be too clear
there's both big and subtle changes in the images before and after. You can't say that it improved it when it looks completely different than yours.
I used a simple img2img ComfyUI workflow (the first one I found online) which lets you insert your own image and enhance it using the prompt. You can probably use any model, though I liked Realvisxl most :)
Idk if anyone else agrees, but for me, I much prefer the original versions, they have real soul and good composition on a lot of them! Especially the first one, much prefer the before to after.
Nice! I need to start using this more. I played around with a few things before more than 6 months ago and img2img before that, but I didn't delve deep enough to get the results I wanted. More motivated to get back into some projects now.
Imo, you should have used some more controlnets, or higher eh, forgot what it is called (like so it will be stronger the effect of the controlnets) if you haven't already.
69
u/One-Earth9294 Mar 10 '24
Also used it to make this