Stained Glass generated by AI
What is Image generation by AI
2022 was the year of text-to-image AI.
What is text-to-image AI?
In simple words: it’s a software that translate a text into an image.
For example:
“Cat in an astronaut suit” ?
Here we are:
DALL·E - high quality photo of a cat astronaut
“Sea otter riding a bicycle” ?
You are welcome:
DALL·E - sea otter on a bicycle
Why stained glass?
When you are in front of a huge and beautiful glass window there is always a mix of excitement and reverentiality.
The lights from behind, the vivid colors, their awesomeness.
Last year I started an evening class of Stained Glass windows at SUPER in Milan.
As a novice stained glass hobbyist, I am slowly delving deeper into the world of this art form.
Also, as I learn more about the latest advancements in AI image generation, it occurred to me that it would be an interesting experiment to try and generate stained glass windows using this technology.
Each week, as I gain more knowledge and experience, I am becoming increasingly excited about the possibilities that this technology offers. I am eager to explore the creative potential of using AI to generate unique and intricate designs for stained glass windows.
Stained glass generation from AI
There are a lot of tools for text-to-image generation: Dalle-2, Midjourney, Stable Diffusion (and all of its derivate models)…
In this experiments I’ll use the one from OpenAI (Dalle-2) and a model of Stable Diffusion offered by lexica.art
The two prompts tested will be about a tree in the syle of LOTR and Ygritte from GOT.
Dalle-2: Starting with a really simple example and prompt.
Prompt text: “lord of the rings tree, green, stained glass window, simple”
Results:
First result: there are some impossible cuts but nothing that can be solved with little work
Second result: also here there are impossible cuts (for example the too tight branch) but this tree with leaf is a good example
Third result: in this case it also generate some cuts for the background, that is interesting because in the other two imags it's a missing feature
Dalle-2: And with a more complex text with a style and description.
Prompt text: “A stained glass window representing the game of thrones character Ygritte. In the style of Louis Comfort Tiffany. Iper realistic. Detailed.”
Results:
First result: is a good picture in a stylized stained glass window. Love the picture but it's more a drawing.
Second result: also a drawing with some resemblances of a stained glass window.
Third result: a curious example of mosaic with hexagonal cells. There are a lot of imperfections (eyes, nose, hair and the missing arm) but it's pretty interesting
Stable Diffusion (lexica.art): Same tree as before
Prompt text: “lord of the rings tree, green, stained glass window, simple”
Results:
First result: impressive. Seems too much like a 3D model printed out and added to a real stained glass window. There are some impossible details like the upper leaves but it's impressive.
Second result: from a window to a door. Not good as the first one, but I can image a drawed tree all over the door
Third result: as said before, there is too much '3D effect' in it, but that's a beautiful lamp that I would love to hang up.
Stable Diffusion (lexica.art): Same Ygritte.
Prompt text: “A stained glass window representing the game of thrones character Ygritte. In the style of Louis Comfort Tiffany. Iper realistic. Detailed.”
Results:
First result: beautiful image. Photorealistic draw that covers a realistic stained glass window. Not what I was looking for.
Second result: as before, the window in the background is really interesting but it's covered by what seems to be a draw
Third result: same as before.
Conclusions?
As we can see the images generated are pretty impressive but very far from perfection.
Dalle-2 produces more simple results.
The images seems to be drawings, without realistic patterns but sometimes it makes it easier (like the trees) to use for real examples.
The mosaic one is also super interesting. With more prompt it will be useful to find out if it’s also good on creating mosaics, that are simpler (for a software) to recreate because don’t need a lot of rework (just cut an image in a lot of squared pieces).
Instead lexica.art model is really impressive.
The tree example is astonishing. These curves and the photorealistic lights. What a surprise!
There is obviously a problem with the “style” of the images: in Dalle-2 it’s not consistent at all and seems like it tries differnt possible stylings. In Lexica.art model the results are all good but too much photorealistic results.
What’s next?
It is important to emphasize that the goal of this technology is not to completely replace our imagination, but rather to enhance and augment it. We are able to improve the synthesis of our thoughts and ideas, allowing to expand and stretch the bounds of our creativeness.
The next post will be about the creation of a custom model to try to solve that “style” problem and produce more coherent results.
Our intelligence is what makes us human, and AI is an extension of that quality.
Yann LeCun Professor, New York University