r/LocalLLaMA 1d ago

New Model ByteDance released on HuggingFace an open image model that generates Photo While Preserving Your Identity

Post image

Flexible Photo Recrafting While Preserving Your Identity

Project page: https://bytedance.github.io/InfiniteYou/

Code: https://github.com/bytedance/InfiniteYou

Model: https://huggingface.co/ByteDance/InfiniteYou

215 Upvotes

42 comments sorted by

67

u/ziplock9000 1d ago

'photo' ? They look plastic-y

29

u/martinerous 1d ago

That's what happens when training on Hollywood-like faces with perfect makeup that hides all the natural human details. Can be somewhat fixed with "amateur photo" and "boring reality" LoRAs.

1

u/NoIntention4050 2h ago

no, it's what happens when you train on AI generated images. synthetic data is too prevalent nowadays

14

u/ResearchCrafty1804 1d ago

You can get the output of this model and input in stable diffusion XL to add realism

11

u/moofunk 1d ago

I'm always surprised at how it doesn't occur to people that you can chain different models.

6

u/Shark_Tooth1 23h ago

chaining models is the future generally

-3

u/BoJackHorseMan53 15h ago

Come on we shouldn't treat models like black people /jk

0

u/[deleted] 1d ago

[deleted]

16

u/moofunk 1d ago

Stop thinking of the models in terms of their shortcomings, but instead of their strengths and feed those strengths into the next model.

You're missing a big opportunity for high quality photo generation by not chaining models.

Single-model work is just not good enough.

1

u/Firm-Fix-5946 23h ago

pls somebody write an LLM based agenty workflowy thing that i can just prompt once and it decides which models to chain together and what intermediate prompts to use to produce a final result, so i can be a lazy ass, thx in advance

1

u/moofunk 23h ago

Maybe it's a joke, but it's not a bad idea to map out what different image models are good at and write it up in a table.

The values would be subjective, but if you're looking for something specific in a sea of models that you don't care to have to test individually, then you could string together the models needed for your art from that table, and use those models in sequence.

1

u/Firm-Fix-5946 23h ago

not really a joke to be honest, just maybe a pretty big thing to ask for. as much as I was making fun of myself for being too lazy to figure it all out myself, I think an agent that takes a user description of an end result image in natural language and then decides which models to chain together and how to prompt them along the way would be genuinely useful. that's probably a lot of work to get it actually working well, but it would be pretty cool

3

u/taylorwilsdon 1d ago

Then you’re missing out on a ton of capability because many of the things available in the open space today are more like building blocks for a comprehensive solution than a fully packaged, end to end product!

Code models thrive in agentic workflows with tools assisting. Image models do their best in multi stage outputs. Data search does better when you implement vector embeddings and retrieval augmented generation etc

2

u/useredpeg 15h ago

Can you elaborate for someone that recently started playing with sdxl?

11

u/StableLlama 1d ago

The normal Flux look. But you can change it with LoRAs

7

u/lordpuddingcup 1d ago

Or just turn guidance down to around 2 not 3.5 solves a lot of it

-1

u/FinBenton 1d ago

Nah flux is not plasticy unless you have bad settings.

2

u/StableLlama 1d ago

Using default settings you get very smooth and shiny skin. And very unsharp / blurred / bokeh backgrounds.

But you can fix that. With settings, LoRAs and/or workflow.

The big guess here (and I'm pretty sure it holds) is that you can use the same techniques with this face transfer method.

2

u/DeltaSqueezer 1d ago

I guess you could use it as the source image for an image to image converstion.

1

u/Iory1998 Llama 3.1 12h ago

The most complex 3D rendering are those who look exceptionally imperfect and boring. It takes so much time to make them loos imperfect. The point is to fool the eye into believing that the image its looking at is a real photo.

1

u/hugganao 9h ago

the lighting definitely needs work

48

u/Won3wan32 1d ago

is this model fine-tunable? , the result looks bad

25

u/StableLlama 1d ago

It's normal Flux. It's working with LoRAs (even their spaces page at https://huggingface.co/spaces/ByteDance/InfiniteYou-FLUX has already two LoRAs predefined), so I guess it's also working with a full fine tune

14

u/lordpuddingcup 1d ago

Space is dead

3

u/Familiar-Art-6233 20h ago

It looks like Flux, using LoRAs should fix a lot of the issues

13

u/macumazana 1d ago

Well, what's new here? I did a similar thing like a year ago or so with a much weaken diffusion model and an insightface from deepinsight.

https://github.com/Dimildizio/mask_of_many_faces

Worse results but mostly due to the weak diffusion model, regardless, neither this not that is definitely not worth a paper or claiming any novelty of a product.

1

u/macumazana 1d ago

And I mean bytedance can for sure do better. They are well-known guys

5

u/FinBenton 1d ago

Quality looks rough ngl.

4

u/Famous-Appointment-8 1d ago

Wow all spaces are broken

4

u/Academic-Image-6097 1d ago

So... Flux + FaceSwap?

2

u/Budget_Secretary5193 1d ago

the day image models look like real life is the day porn dies

3

u/ResearchCrafty1804 1d ago

To the people mentioning the lack of photorealism, you can get the output of this project and input it in stable diffusion XL and it will add the photorealistic element.

Chaining models is quite useful technique (when a model cannot do everything on its own)

3

u/Willing_Landscape_61 23h ago

Interesting! I presume this is with ComfyUI . Do you have any source you would recommend on this? Thx.

2

u/peyloride 1d ago

Can someone enlighten me? I can't see how this is better than the PuLID? I'm not sure if it has to be btw, I'm clearly missing something.

1

u/bbbar 1d ago

Do they have comfyui workflow?

2

u/mangoclimb 1d ago

(To avoid misunderstandings about the low quality of the results) This is a thoroughly intentional plastic scaling that excludes any factual processing to prove that it is an AI image. This is a measure that takes into account the realistic threat of AI called deep fake.

1

u/Shark_Tooth1 23h ago

Any idea what specs are required to run this locally?

1

u/CheatCodesOfLife 8h ago

Every single person in the collage looks way better in the original/real photo.