Double-post but that's because nobody else has said anything - allowed!
So here's something which I noticed in the SD3 API documentation. Search and Replace! Probably old news to others(?) but new to me and furthermore always good to see how new models can handle something like this.
So search and replace - basically inpainting by prompt rather than a mask. First example blew me away - the image on the left was a very simple prompt to SD3: "Blond man in a field". Then I sent that image back to SD3 and told it the search term was "man" and the new prompt was "blonde woman in a field". A few seconds later the machine spat out the image on the right.
Look how the cornstalks in the foreground are in the same place and the distant scenery is the same. And how neat a job it has done of replacing the dude with a lass. Dong-Gone wishes this worked in reality!
I wanted to mix things around a little and this brought to mind the old Stalin meme of someone disappearing in a photograph. So to do something a little more idiosyncratic I asked SD3 for Stalin walking along with Batman. And then I gave its image back to it and told it to replace Batman with Superman.
Oo-er, there's something not quite right with our Red Son in this one.
Still, interesting. First off this is quite different to regular in-painting by mask. It gave me an old style comic-book Superman and then extrapolated to make Stalin fit the same style! Not sure if that's good or bad. Depends what you're trying to do, really.
To be clear, I specified nothing about the style, background or anything else. And the reason I'm just trying this on SD3's own images to begin with is because I thought that might be giving it an easier time of things. Figured I'd begin simple.
But given the change of style I decided for the next attempt I would specify a style. So this time I asked for a photograph of Stalin walking by a blond man. And then much like the cornfield I asked it to replace that with a blonde woman, again specifying a photograph.
Well that's all kinds of disturbing and I hope that
@Susanna doesn't put me on a Soviet naughty list because of it. However, there's still a lot that's impressive about this. The background is unaltered, it's seamless in its replacement of one subject with another. It again has adapted Stalin's style to make it in keeping with the new subject.
I really don't know quite what to make of the way it re-interprets other elements of the image to match the new subject. What is notable is that it does it less so the more I am clear about the style and particulars of the image. Like as a more detailed photograph it doesn't change things so wholesale because both before and after are explicitly set to be similar styles. I imagine the more precise I am the less it tends to alter things outside the subject of the search and replace. Still, I found this interesting and hoped someone else would. This has some very interesting potential.
