Skip to main content

Creating an AI Digital Avatar and Voice Clone of Myself with Free and Low Cost AI Tools

AI Image of TET, wearing clothes he doesn't own, in an office that doesn't exist, generated from an AI model trained on photos of TET.

Over the years I've looked at various ways of creating and animating a digital avatar, from simply creating a character for Cartoon Animator and voicing and animating it myself, to creating a live motion capture ready Vtuber avatar puppeted and voiced by me in real-time.

In the last year or so, making photographic images talk, using AI and AI voice cloning has really progressed. To the point where I wondered if I could create a photographic AI avatar of myself, complete with my cloned voice, that I might use on some of my videos?


Creating My Avatar: Artflow

To create my avatar image, as far as I'm concerned, Artflow.ai is the best value AI site for creating consistent digital characters (or 'actors' as Artflow calls them). That's pretty much their entire focus.

Artflow's avatar model training user interface.
Artflow's actor model training user interface.

You get your first actor for free, 100 free credits per month (which equates to 100 still images per month - get an additional 50 credits if you sign up with my link), 4 minutes of video from their Story Studio (put your characters into video stories), and 2 minutes of video from their Video Studio (kind of like Story Studio but you can add dialogue and lip sync your characters to it).

As well you can buy additional actors for a small, one time fee, and put multiple actors into the same image using their actor director mode. The only negative for me is you need a credit or debit card.

Training a model takes a minimum of five images but I would recommend using as many as the site will let you if you can.

Once trained, Artflow will show you a selection of sample images with your actor in different situations. Text prompt any situation or use one of Artflow's templates to get you started. It's pretty addictive, especially if your first actor is of yourself or someone you know. It's just fun imagining different scenes to see what you get.


Voice Cloning: ElevenLabs/Speechify

I looked around for a good, free voice cloner and couldn't find any that actually delivered a voice that sounded anything close to mine. Most sounded like bad text to speech (TTS) voices from 5-8 years ago.

If you really don't want to spend any money then Speechify is the only site I came across that would let you train the AI on your own voice for free and create a quality model. They give you 1000 characters of TTS to try out your trained model. Use them wisely because once they're gone, there's no topping up next month, they're gone!

ElevenLabs Voice Training UI.
ElevenLabs Voice Training UI.
It really is this simple.

Unfortunately Speechify is very pricy to pay by the month, but almost a third of the monthly price if you're prepared to pay annually.

ElevenLabs on the other hand, while you can't train a model on your voice for free, is extremely cheap for a single month of their Starter plan (again credit/debit card or bank transfer only). Currently they're considered the Gold Standard in AI voices.

I decided to take the gamble and paid for a month on the Starter plan - totally worth it! I uploaded less than two minutes of sample audio and got a near perfect copy of my voice in seconds, and now I have 40,000 characters of TTS to burn through.


Animating My Avatar: Hedra

Artflow's Video Studio actually does have the ability to make your images talk but not in a completely convincing way. You'll get some head movement but mostly just mouth movement. On the plus side you can make more than one character speak in a single image.

Hedra, which is currently free to use, by comparison, is specifically designed to animate an avatar head in a more natural way. It's not perfect but in short bursts, it almost passes for actual video footage of the person. It's able to add not just lip syncing but head and upper body movement too.

Unfortunately Hedra doesn't really give you any control over the output. You get what you get. But the result is generally pretty good. It does limit you to thirty seconds of video each generation because by about that point the magic is starting to fall apart just a little.

Hedra's UI.
Hedra is a three step process. 1. Upload your audio.
2. Upload your image (or prompt for a character). 3. Click Generate video.

Hedra also tends to crop relatively close to your image's face. If you wanted the full image, there's no way to tell Hedra. Probably because Hedra's animation is centered mostly around the head and shoulders.


The Finished Clone Video

You can see my finished demonstration video of my cloned avatar below. 

I didn't want my finished video to be all one camera shot so I made an attempt to stitch Hedra's cropped video image back into the full image of me in an office that doesn't exist outside of a computer (as far as I know).

You can see the wide shots don't exactly work if you look too closely. I'm sure I could make it seamless  with a little more work. However if you look at the close up shot, where I've stitched the sides of the image back into the shot so it fills the frame, that section is pretty convincing that it could be the real me.

For the most part I think it's almost good enough to fool people into thinking it really is me. Though I will say, I don't move my head around nearly that much in my videos. It would be nice if Hedra would let you tone down the head movement a little.

What do you think? It may or may not be convincing now but just remember the AI mantra "This is as bad as AI will ever be. It's only going to get better from here," and I think this is pretty good.


o---o--- ---o--- o---

Did you find this article useful?
Subscribe to my newsletter and get the
latest articles delivered to your inbox.

Comments

Popular posts from this blog

Inochi2D - Free Open Source 2D VTuber Avatar Rigging and Puppeteering Software (Part 1)

Inochi2D Creator - Free Open Source VTuber Software. If you've been looking for a way to live perform as a 2D cartoon avatar on camera, whether it be for a live stream or for pre-recorded content like educational videos, then VTuber software is a low cost (or even no cost) option worth looking into. In my previous post, How to Become a VTuber - 2D and 3D Software for Creating and Controlling Your Avatar , I took a brief look at the relatively new but completely free and open source Inochi2D  which I thought showed great potential for my own needs of creating a live performance character rig for my own TET Avatar that I use for all my promotional materials. While it is possible to live perform my character using Cartoon Animator itself, Reallusion's MotionLive2D capture system isn't great - with lip sync in particular. More importantly though, I can't exactly teach people how to use Cartoon Animator if I'm using Cartoon Animator to control my Avatar. What is Inochi2D

Wonder Unit Storyboarder - Free Storyboarding Software for People Who Can (or Can't) Draw

Wonder Unit Storyboarder.  As an independent and solo animator I'm always tempted to try and skip storyboarding my animated shorts because they're usually only single scene sketch comedy type jokes. As a result I have many unfinished projects that kind of petered out due to having no clear finishing line. Storyboarding your productions, no matter how small, gives you a step by step guide of every shot that needs to be completed (no planning shots as you animate). It also allows you to create an animatic that gives you a rough preview of the finished production. In short, you shouldn't skip storyboards as they, generally, increase the chance of the project being completed. Disclaimer - I'm Not a Fan of Storyboarder Upfront, Wonder Unit's Storyboarder  is not my preferred storyboarding software. However it's completely free, has a number of very compelling featu

Can You Learn Reallusion's Cartoon Animator 5 for Free Using Their 137 Official YouTube Video Tutorials Sorted Into a Logical Learning Order?

Or you could just buy The Lazy Animator Beginner's Guide to Cartoon Animator . While Reallusion's Cartoon Animator is one of the easiest 2D animation studios to get up and running with quickly, learning it from all of the official, free, video tutorials can be more overwhelming than helpful. With more than 137 videos totaling more than 28 and a half hours of tutorials, spread across three generations of the software (Cartoon Animator 3 through 5) it's hard to know if what you're learning is a current or legacy feature that you either need to know or can be skipped. Many of the official tutorials only teach specific features of the software and don't relate at all to previous or later tutorials. As a result there are many features either not mentioned or are hard to find. To make your learning easier, on this page, I've collected together all of the essential, official, free video tutorials and sorted them into a learning order that makes sense. Simply start at

Dollars Mocap: Full Body Webcam Motion Capture (Including Hands and Fingers) For iClone and Cartoon Animator

Even though I should be further away from the camera Dollars Mocap MONO still does a good job of  tracking my arms, hands and fingers. Ever since I wrote my series on becoming a VTuber , discovering it was possible to do full body motion capture, including hands and fingers, with just software and a webcam, I've been on the look out for any motion capture software that can bring that functionality to Cartoon Animator. Dollars Mocap is a low cost motion capture application with a free trial that I learned about through the YouTube Channel Digital Puppets  and their test video . It can record full body, upper body, arms and hands, and facial mocap from a live video source or pre-recorded video. Investigating further, I discovered not only does Dollars Mocap have a free iClone7, iClone8 character profile file download (look for it at the bottom of the main program download page), so you can use the saved motions with iClone8, they've also got a demo video for how to convert your

Five AI Generative Image to Video Tools For Animation You Can Try Free Right Now

The Emo Girl Character created by Start Animating. A I generative video isn't new but it is the next big thing in the visual imaging space as various development teams work to perfect the generated output.  Just like generating still images AI video sometimes struggles with physics, arm and hand movement, and the general structure of things. However it is getting better and, as is the catch cry of all AI development, this is the worst it will ever be, because it's improving fast. If you're an animator one current potential use of generative AI video is to animate your key frames, as opposed to generating something entirely from a text prompt. Starting with an image helps to keep your characters and art style consistent across AI generations. With that in mind I tried five, free image to video AI generators to see what their potential might be and whether they can handle cartoon style characters well. Note all but the last entry on the list do not create any sound with the

Eric W. Schwartz: Cartoonist, Animator and Amiga Die Hard

July 1992 Edition, CU Amiga Featuring Amy the Squirrel. American Cartoonist, Eric W. Schwartz , (whose unofficial Amiga Icon, Amy the Squirrel, is pictured on the July 92 edition of CU Amiga cover on the right) is my only real animation hero. Sure there are the big names like Disney , Chuck Jones , Tex Avery and even Preston Blair whose influences can all be seen in my own cartoons but Eric did what none of the others could. He showed that really great 2D computer animation was within my reach with little more than an Amiga Computer , a copy of Deluxe Paint and Moviesetter . This was at a time when computer based animation was in its infancy (outside of computer game animation) and Flash was something that lights did. There were many great Amiga artists but Eric was really the only one consistently making very funny, traditional style animations. His humor and drawing style is heavily influenced by classic Warner Brothers and Disney cartoons but he managed to build on this,

Animation Paper V5.0 Alpha 2 Pre-release

Animation Paper V5.0 Alpha 2 Pre-release. Animation Paper strives to be the best tool for digital hand drawn animation by not only recreating the traditional animator's table in a modern user interface, but also automating many of the more tedious processes so you can focus almost entirely on drawing. At the time of writing Animation Paper V5.0 is only available as an Alpha 2 Pre-release, and still does not have all of its features implemented, such as the ability to add audio (due to be part of the Alpha 3 release scheduled for the end of May 2020). Animation Paper has been completely redesigned and updated from th hand drawn animation app that was formerly known as Plastic Animation Paper (PAP) which, at the time, was easily one of the best apps for the sole purpose of creating hand drawn digital animation roughs. You can still download PAP completely free from the Animation Paper website . If you look past the extremely dated 90's interface it's still a great tool. I wr