Generating stained glass windows with a custom style in stable diffusion

Previous post: Stained Glass generated by AI

WTF is Stable Diffusion

Last year there was the Perfect Storm: the common use of a technology called transformers + the availability of gpu power + the release to the public as open source software.

From OpenAI with its Dalle-2 to Midjourney and Stability AI open source version of Stable Diffusion, that generated a positive euphoria which brought to the tremendous growing of AI tools for image generation.

In particular, Stable Diffusion is the one that was one open sourced and the community started to grow rapidly.

I will not describe how technically works because there are a lot of resources on the web.

If you want a more detailed description, read this awesome blogpost. It explains very well what is and how it works!

The problem with the “style”

As seen in the last blog post, the style of the stained glass window is really hard to manage.

Some results seems to be ok, some others are more like mosaics, and others photorealist representations of people.

If you want to create some coherent visualization, based on your idea and an already existent style of a stained glass window, you really need some custom work to do.

Luckily last year the contributions to the Stable Diffusion software came also from top notch scientists all over the world. Google developed its method called Dreambooth for custom train/style transfer model into our image generator tool.

This studies (that are then extended to the software) allows to choose, select and transfer an image style (or add a new photo) to be used and syntetized into the newly generated image.

Our goal

Our goal is to generate some stained glass windows using a specific drawing style. I’ll choose this beautiful Artwork Style by Ranefea.

In this link you will find some examples.

Spoiler: yes, as beginner in the stained glass worls i know that some of these drawings cannot be realistically translated into real stained glass windows. Just take this as what it is: an experiment on style transfer for this images. Then if you want to create your design in real world, you will have to fix the template.

Let’s try it!

I will not describe each step. If you need help, follow this very extensive tutorial.

Just some tips and suggestions:

• If you have not a good GPU, use Google Colab. It’s really easy and comes with working GPU.
• Create a new Google Account. It will take some GB of space to download the existent model and create your own.
• Download 10-12 images with the style you are recreating and use “number of images * 100” steps for better results. You’ll understand when you find out about it.

After the installation of the software using this Python Notebook) I’ll just need to add these images to my Google Drive, create an HuggingFace account and run step by step the notebook.

Be sure to select the “style transfer” option!

You need to give your model a name. I’ll call it “sglass” (stained glass) style.

After 15 to 20 minutes the trained model is ready, we can start generating our images!

Results for my new “sglass style model”

Prompt text: “cat, sglass style”

Results:

---

Prompt text: “frodo baggings, sglass style”

Results:

---

Prompt text: “girl with red hair, sglass style”

Results:

---

Prompt text: “a brown bear in a forest, sglass style”

Results:

Conclusions

As you can see, the results are pretty interesting!

I know, i know. It’s not good enought for zero-work and for sure not perfect for a staind glass window.

This experiment was done in a bored afternoon after going to buy some glass sheets.

I need a software that can help me dream of new stained glass designs (that probably will never recreate on a window because it need a lot more of an afternoon), and gives me also ideas for smaller stained glass works to do.

Given an idea and a style, this can be a first step in that direction.

Also, I did this because it’s funny. Isn’t that a very good reason to try out and experiment?

All life is an experiment.

The more experiments you make the better.

Ralph Waldo Emerson