Over the years I've looked at various ways of creating and animating a digital avatar, from simply creating a character for Cartoon Animator and voicing and animating it myself, to creating a live motion capture ready Vtuber avatar puppeted and voiced by me in real-time.
In the last year or so, making photographic images talk, using AI and AI voice cloning has really progressed. To the point where I wondered if I could create a photographic AI avatar of myself, complete with my cloned voice, that I might use on some of my videos?
Creating My Avatar: Artflow
To create my avatar image, as far as I'm concerned, Artflow.ai is the best value AI site for creating consistent digital characters (or 'actors' as Artflow calls them). That's pretty much their entire focus.
 |
| Artflow's actor model training user interface. |
You get your first actor for free, 100 free credits per month (which equates to 100 still images per month - get an additional 50 credits if you sign up with my link), 4 minutes of video from their Story Studio (put your characters into video stories), and 2 minutes of video from their Video Studio (kind of like Story Studio but you can add dialogue and lip sync your characters to it).
As well you can buy additional actors for a small, one time fee, and put multiple actors into the same image using their actor director mode. The only negative for me is you need a credit or debit card.
Training a model takes a minimum of five images but I would recommend using as many as the site will let you if you can.
Once trained, Artflow will show you a selection of sample images with your actor in different situations. Text prompt any situation or use one of Artflow's templates to get you started. It's pretty addictive, especially if your first actor is of yourself or someone you know. It's just fun imagining different scenes to see what you get.
Voice Cloning: ElevenLabs/Speechify
I looked around for a good, free voice cloner and couldn't find any that actually delivered a voice that sounded anything close to mine. Most sounded like bad text to speech (TTS) voices from 5-8 years ago.
If you really don't want to spend any money then Speechify is the only site I came across that would let you train the AI on your own voice for free and create a quality model. They give you 1000 characters of TTS to try out your trained model. Use them wisely because once they're gone, there's no topping up next month, they're gone!
 |
| ElevenLabs Voice Training UI. It really is this simple. |
Unfortunately Speechify is very pricy to pay by the month, but almost a third of the monthly price if you're prepared to pay annually.
ElevenLabs on the other hand, while you can't train a model on your voice for free, is extremely cheap for a single month of their Starter plan (again credit/debit card or bank transfer only). Currently they're considered the Gold Standard in AI voices.
I decided to take the gamble and paid for a month on the Starter plan - totally worth it! I uploaded less than two minutes of sample audio and got a near perfect copy of my voice in seconds, and now I have 40,000 characters of TTS to burn through.
Animating My Avatar: Hedra
Artflow's Video Studio actually does have the ability to make your images talk but not in a completely convincing way. You'll get some head movement but mostly just mouth movement. On the plus side you can make more than one character speak in a single image.
Hedra, which is currently free to use, by comparison, is specifically designed to animate an avatar head in a more natural way. It's not perfect but in short bursts, it almost passes for actual video footage of the person. It's able to add not just lip syncing but head and upper body movement too.
Unfortunately Hedra doesn't really give you any control over the output. You get what you get. But the result is generally pretty good. It does limit you to thirty seconds of video each generation because by about that point the magic is starting to fall apart just a little.
 |
| Hedra is a three step process. 1. Upload your audio. 2. Upload your image (or prompt for a character). 3. Click Generate video. |
Hedra also tends to crop relatively close to your image's face. If you wanted the full image, there's no way to tell Hedra. Probably because Hedra's animation is centered mostly around the head and shoulders.
The Finished Clone Video
You can see my finished demonstration video of my cloned avatar below.
I didn't want my finished video to be all one camera shot so I made an attempt to stitch Hedra's cropped video image back into the full image of me in an office that doesn't exist outside of a computer (as far as I know).
You can see the wide shots don't exactly work if you look too closely. I'm sure I could make it seamless with a little more work. However if you look at the close up shot, where I've stitched the sides of the image back into the shot so it fills the frame, that section is pretty convincing that it could be the real me.
For the most part I think it's almost good enough to fool people into thinking it really is me. Though I will say, I don't move my head around nearly that much in my videos. It would be nice if Hedra would let you tone down the head movement a little.
What do you think? It may or may not be convincing now but just remember the AI mantra "This is as bad as AI will ever be. It's only going to get better from here," and I think this is pretty good.
Did you find this article useful?
Subscribe to my newsletter and get the
latest articles delivered to your inbox.
nice
ReplyDelete