The most recent video I published made some use of AI art. I wanted to write some reflections on that, more to clarify my own thoughts than anything else. Feel no obligation to read it all, I know you're subscribed here for math videos. If you have opinions on the topic and feel like adding a comment, I'd love to read it.
Long story short, I don't regret experimenting with using AI art, but for a mixture of reasons I don't plan to use these tools for future artwork needs in videos, at least not until a number of thorny questions are hammered out.
If you're an artist/animator looking for contract work, feel free to reach out to me with a portfolio.
Context
Several months ago I asked an artist I was working with to handle the visuals of the opening scene of this video.
This artist, Kurt Bruns, was someone I'd worked with before, adding the occasional hand-drawn digital art piece to new videos. Visuals on 3blue1brown are usually dominated by programmatically-generated mathematical animations, but I see a lot of room to broaden that stylistic scope when human stories add to a lesson, and one of my goals recently and moving forward is to do more of that.
When I sent Kurt the ask for this particular scene, I didn't have any constraints on how I wanted it illustrated, either in terms of the medium or the concept. Kurt has good taste and I knew I'd be happy with anything he felt satisfied with.
We had both talked about AI art before, having been part of the artist beta for DALL-E earlier that summer, and generally keeping an eye on the technology. Until this point any projects we had done together were hand-drawn digital artwork, but this time Kurt opted to experiment with using Midjourney to get the initial versions of the pieces we'd use.
It's worth highlighting that usage here does not look like typing one prompt, and using whatever pops out as-is. Initially, there's a lot of brainstorming, trying many ideas, and tossing what's not working well. The pieces themselves often have numerous problems that need cleaning up. Uncanny hands need to be repainted. Mangled background characters need to be fixed either to be more abstract so as to fade into the background or more detailed so as to not look weird. Most pertinently, getting multiple pieces to cover different parts of a scene in a way that looks stylistically consistent is a constant uphill battle.
This is all to say a lot of human labor and love went into the final result, but AI was most certainly a central part of the workflow.
But is it theft?
In tandem with the rise in the capability of these AI Image generation models, there's been a strong pushback from many artist communities regarding the ethics of these tools. The short version of the pushback is that artists did not consent to have their work used in training these large models, and this feels especially unjust when those tools threaten to replace many of those same artists. This is especially true when the names of those artists are used in prompts.
Some people will push back to this push back by saying it's analogous to what human artists do. You see lots of different pieces, you learn from them and take inspiration from different styles, and incorporate it into your own work.
To this, the "AI art is theft" community often replies that yeah, no, it's not at all the same as what human artists do. Processing a massive quantity of image data while gradient descending through a pile of weights that effectively compress all that data into a single model is nothing like the way human artists pass along their craft.
I'm paraphrasing, of course, but I've seen multiple back-and-forths like this repeated in online arguments.
So which is it? Are Midjourney/DALL-E/etc more analogous to human artists, learning and taking inspiration from all the freely visible (but potentially copyrighted) work that they see online? Or are they more like lossy compressions of massive image databases, whose generations are more like a souped-up copy-paste function?
To my taste, these tools feel sufficiently novel that it's unproductive to try reasoning by analogy. It's not really like either. It's a new kind of thing!
Some use cases feel like clear-cut jerk moves, like prompting a model to create something in the style of a living artist, then using that piece instead of licensing from or commissioning that artist. In a perfect world, I'd like to see such prompts result in royalties for the artist whose name is used.
In my case, I initially felt fine if an artist I hired chose to use it as a central part of his workflow, assuming it wasn't deliberately lifting any individual's particular style. Technology at its best augments it doesn't replace, and in this case, it freed an artist to rapidly iterate with many more ideas than he otherwise could have.
But should it have felt fine? In the eyes of some, these models are tainted regardless of how you use them. Even if the prompts weren't explicitly asking for the style of a particular artist, who's to say the model wasn't heavily "inspired" by a specific piece out there drawn by someone who wouldn't appreciate their style being regurgitated by a computer?
Legal concerns
The question is not a purely academic one. There are numerous pending court cases that concern the as-of-yet unanswered legal question of whether it's okay for these models to use online copyrighted work the way they have. Furthermore, what is the copyright status for images that they generate?
Maybe the outputs will be deemed public domain. Maybe there will emerge a mechanism to appropriately credit the copyright holder of the images which most heavily influenced an output. Maybe any model trained on images without the explicit consent of the copyright holders will be deemed illegitimate in a way that effectively renders their outputs unusable. We don't know yet!
If for no other reason, the ambiguity on this question alone is enough reason to hold off using the tools for new projects.
The fundamental quality problem
Even if all the ethical and legal concerns were somehow magically resolved, there's still the question of whether the outputs are actually good.
Certainly, many of these AI generations are stunning at first glance. In a shallow sense, many of them are much more beautiful and intricate than what most humans can produce.
There are some pragmatic issues making it work in videos. It's actually very time intensive to get a series of images that feel stylistically consistent, especially if there's a single character involved. That feels like a technology problem. I'm guessing there are either already tools, or there will be soon, that solve this.
There's a deeper issue, though, which is that the quality of a piece of art is not purely a function of the image. The story of where it came from matters.
For my intended use cases, where the whole goal of incorporating more artwork into videos is to add an element of humanity and character to the otherwise abstract and ruthlessly precise world of mathematics, there's something undermining if that artwork is machine-generated.
I don't know about you, but I often find myself feeling fonder towards artwork when I get a glimpse of how it was made, say by watching a timelapse of its creation, or hearing an artist describe why they made certain decisions. That fondness grows even more if I know a bit about the artist who made it.
AI artwork tends to have noticeable artifacts when you know to look for them. Artifacts in and of themselves are not bad, though. Brush strokes, lens flares, and film grain are all artifacts of specific media, but artists, photographers, and filmmakers often lean into these. Maybe they do so to appeal to a viewer's fondness for the medium as a whole. Or maybe it's to offer some small fingerprint of where the piece came from, a reminder that there is a story behind the work.
What I've noticed in myself as soon as I realize a piece is AI-generated, either by noticing the artifacts or by seeing the attribution, is that I'm of two minds.
One half of me feels the opposite of how I feel when learning more about how a human-made piece was created. Instead of fondness I'm left feeling flat. Even if the models improve to the point where no artifacts exist, the mere knowledge that something is AI-generated means there's a missing opportunity for a piece to be imbued with that bit of story for how it was crafted. Admittedly, I don't know the story behind how 99.9% of all images I see in the world were made, but somehow knowing whether or not there is a story changes things for me.
But there's another side of me, the technologist half, that delights in the fact that algorithms can create these images. It's the same part of me that delights in generative art and emergent patterns. It's the same part of me that resonates with Feynmann's tirade about how scientific knowledge about a flower only adds to its beauty.
If I had to guess, I'd speculate that as the years go on and more people do incredible things with AI tools, that second half will start to dominate. For me, at least. In the meantime, and I'm willing to guess this is true for enough other people out there that it matters, anything AI-generated artwork has an insurmountable hollowness.
Where does this leave things?
Until all these questions have clearer answers, I feel a lot better if future artwork is done by hand.
I posted multiple times last year about the Galois project. It'll still certainly happen, but it's been sitting on the backburner while I've been more focused on the usual videos. Kurt was also working with me on that one, and for a while, we pursued what it might look like to lean heavily into AI artwork, prompting Midjourney to create scenes from Galois's life in the style of contemporary artists like Delacroix.
A lot of the initial concepts look great, but due to a combination of the concerns I've mentioned here, and the fact that getting these to work coherently in a video is actually very challenging, I've swung back to a desire for all the artwork in that project to be human-made. Even setting aside the ethical and legal questions, I think this will result in a better final piece.
For the Summer of Math Exposition, a number of people have asked about our policy on using AI-generated material in entries. We do require that anyone submitting an entry has the rights to all assets used in their entry, and considering the fact that the question of who owns the copyright for AI generations is currently unresolved, we ask you to treat it as if someone else owned the copyright. That is, if it falls under fair use, go for it, but in general, it should be avoided. Also, it's just not clear to me that AI generations would actually make your piece any better.
If you've read this far, thank you for taking the time. I'd love to hear any thoughts you have!