AI 2D Animation Challenges: Character Consistency and Multiple Characters Speaking in the Same Shot

An unlikely pair of Space Cargo Pilots.

Animation, specifically 2D animation using generative AI has really started to evolve into a more useful tool for creators however, character consistency across camera shots, within the same scene, is still an issue.

As well, another issue I've bumped up against, as someone who makes very dialogue heavy animations, is having more than one character talk within the same camera shot. 

There are a few all-in-one generative AI services that include effective tools to overcome these two issues, such as OpenArt.ai, but I wanted to see what I could achieve with completely free tools.  

For image generation I went with Google Gemini because you get limited free access to one of the best image generation models, Nano Banana. 

For video generation I went with Grok Imagine because it's very good with 2D animated characters and includes audio and lip sync. The output on the free plan is only  480p and tastefully watermarked in the lower right corner but it's good enough for experimenting. (Note that as of March 19, 2026 Grok has changed Imagine to a three day free trial and doesn't seem to let you generate anything unless you sign up to the trial).

Character Consistency

Google's Nano Banana is actually quite good with character consistency, even if you only use a single image of your character as a reference. That said creating a character reference sheet showing different angles definitely will help keep things more consistent.

 

AI Samson is a YouTuber I'm subscribed to, who helps to keep me informed about all AI developments in the creative arts, created a video on character consistency that works across different AIs. In it he demonstrates a prompt he developed (and gives away for free) that generates a character sheet of any character you can use as a reference for any AI that let's you upload images.

 

This ONE Prompt Fixes AI Character Consistency (Use This)

Using AI Samson's prompt, I modified it to generate a character sheet for my character, a space cargo pilot, that I generated with Gemini moments earlier in the same chat thread.  As you can see my pilot (who clearly doesn't get the well paid cargo runs) looks exactly like the same person across all images, including his outfit.

 

Space Cargo Pilot Character sheet generated in 
Gemini Nano Banana using AI Samson's prompt.

 

My idea was to have my space cargo pilot be sitting in a run down space port bar talking to some kind of tentacle headed alien pilot. Rather than create an alien character first, I simply prompted Gemini for the whole scene, describing what I thought the alien should look like in that prompt. This is what I got...

My space cargo pilot, sitting in a space port bar, talking to a tentacle 
headed alien pilot. This was the very first output to my prompt for Gemini.

The image was pretty much what I wanted except, in my mind, the alien might have been more tentacle headed with less body but I didn't want spend hours on getting the character exactly right for a demo.

I was interested to see that, now I had my alien, could I generate a character sheet for him, to help keep him consistent in the future. I asked Gemini to make one using the same format as the one I used for my pilot - I didn't even have to re-enter my character sheet prompt. Here is what I got...

Gemini generated this complete character turnaround of the alien character 
just based on the character it generated in the image of my scene.

My prompt didn't ask for any text so it's interesting that Gemini felt the need to not only label certain features but also give the species of alien a name.

The one thing I didn't want to compromise on was the composition of my scene. I wanted both characters sitting next to each other at the bar as if you were viewing them from the point of view of a bartender. It took me a few prompts to get the composition right but Gemini got there in the end...

This was the final image I used for the animation, except I did crop it so 
you could no longer see each character's legs.

All of these images were generated in the same chat thread so I wasn't really using those reference sheets directly in my prompts to maintain character consistency. I'd just say, as part of the prompt, to maintain character consistency, and it worked relatively well, though I do think my human pilot aged up a little.

 Multiple Characters Speaking in the Same Shot

 Having both characters talking in the same scene was surprisingly easy with Grok. I was able to 'freestyle' a conversation using simple prompts that described what was happening in the scene and who I wanted to say specific dialogue.

On the free version of Grok you only get six second clips so I simply prompted each part of the conversation in a sort of order, creating another six second clip each time. 

I didn't actually prompt for specific dialogue unless I needed a character to say something to lead the conversation forward... or to end it like I did in the clip prompt screen shot shown below. 

My final conversation ending prompt where the alien excuses 
himself in need of the bathroom.

 

Grok chose the voice for each character and seemed to maintain the same voice for each across multiple clips. This may be because everything was generated within the same conversation thread. 

 

Were I to start a new project with these characters they may end up with different voices. If that was the case I would consider voice replacement with ElevenLabs.

The Final Edit

After generating nine different clips using the same image of my two characters sitting in a bar I brought all the clips into my video editor (Shotcut). I edited them together so the whole thing wasn't just one long scene with jump cuts. The final edit is below followed by all the source clips so you can see the output Grok Imagine generated.

 

A Space Pilot Walks Into a Bar... Gemini, Grok Demo 

 

 

The final scene is nothing particularly spectacular but it does demonstrate that it is possible to animate conversations with multiple characters in a scene. It may be somewhat harder if you were sticking to a rigid script but it is fun that you can kind of freestyle a conversation together that allows for unexpected ad-libs, as well as scripted, key moments.

 

To some degree, it really feels like you're directing actors, who aren't afraid to go off script, rather than getting too frustrated because they're not saying your words exactly. For me this was a very fun bit of experimentation.

 

 
o---o--- ---o--- o---

Did you find this article useful?
Subscribe to my newsletter and get the
latest articles delivered to your inbox.