AI-generated Melee Weapons
In this article, we explore how to use stable diffusion to generate weapon sprites for you automatically. The goal is to have melee weapon sprites like swords, hammers, axes and daggers that have a consistent style and could be used in a 2d fighting game as weapons and inventory icons. We want a clear background that we can easily turn transparent in any photo editing software to get a free floating melee weapon.
Prompts
I mostly used the following prompt:
(WEAPON NAME), dnd style, rpg item, fantasy, enchanted, highly detailed, ornaments, wood, wooden handle, leather handle, gemstones
With the negative prompt:
cropped, ((out of frame)), ugly, deformed, (blur)
Input Images
Without input images it is very hard to tell the model to keep the background clear for easy editing. So I just draw the shape of the weapon I want on a clear (white or black) background. The results you see below are from the Protogen v3.4 model with 768x768 resolution, resulting from input images like these:
Pickaxes
Some of these look like Minecraft-inspired diamondpickaxes, while others have more of an egyptiantouch to them.
Battle Axes
I really like the first and second axe. Some start to grow a weird hammer like extension at the back, which does not look too good. One of the axes even has a sword attached to it's blade.
Scythe-Axe Hybrids
These look interesting, especially the blade attached to the grip, which could be used as an alternative weapon for stabbing.
Other melee weapons
Some weapons are hard to put in a category. What is this even supposed to be?
Notice how the model does not care for the palms of our hero's hands. Almost all grips are sharp metal constructions. Or even worse: the grip appears to be a blade. Good luck with that.
Some weird results
What is this? Honestly, I don't know how to describe this monstrosity:
What you have seen so far are just the usable results, but I want to show you the complete story. Many times the model generated artifacts that are sometimes just little islands of stuff next to the weapon like here:
Sometimes the blades are disconnected from the rest of the weapon in the case of axes. But without the Blades they make for great combat sticks for mages.
Using a black background.
I am not sure if a white or black background is better for this task. Here you can see some examples:
I feel like the weapons on the black background have more shinyness/lighting on them. Let's remove the background and look at them side by side:
I let the background animate on purpose, such that you can compare them against different backgrounds. I feel like the white background is better in general. In the next article we will explore swords, bows and daggers. ALso I want to try if we can use the stable diffusion image to depth models to generate 3d models of the weapons.
About Tadeo Hepperle
Hey, I am Tadeo, a Software Developer with a passion for Rust, Dart and Deep Learning. I love to explore how to make use of generative AI, especially for games.
