Skip to main content

Creating an AI Digital Avatar and Voice Clone of Myself with Free and Low Cost AI Tools

AI Image of TET, wearing clothes he doesn't own, in an office that doesn't exist, generated from an AI model trained on photos of TET.

Over the years I've looked at various ways of creating and animating a digital avatar, from simply creating a character for Cartoon Animator and voicing and animating it myself, to creating a live motion capture ready Vtuber avatar puppeted and voiced by me in real-time.

In the last year or so, making photographic images talk, using AI and AI voice cloning has really progressed. To the point where I wondered if I could create a photographic AI avatar of myself, complete with my cloned voice, that I might use on some of my videos?


Creating My Avatar: Artflow

To create my avatar image, as far as I'm concerned, Artflow.ai is the best value AI site for creating consistent digital characters (or 'actors' as Artflow calls them). That's pretty much their entire focus.

Artflow's avatar model training user interface.
Artflow's actor model training user interface.

You get your first actor for free, 100 free credits per month (which equates to 100 still images per month - get an additional 50 credits if you sign up with my link), 4 minutes of video from their Story Studio (put your characters into video stories), and 2 minutes of video from their Video Studio (kind of like Story Studio but you can add dialogue and lip sync your characters to it).

As well you can buy additional actors for a small, one time fee, and put multiple actors into the same image using their actor director mode. The only negative for me is you need a credit or debit card.

Training a model takes a minimum of five images but I would recommend using as many as the site will let you if you can.

Once trained, Artflow will show you a selection of sample images with your actor in different situations. Text prompt any situation or use one of Artflow's templates to get you started. It's pretty addictive, especially if your first actor is of yourself or someone you know. It's just fun imagining different scenes to see what you get.


Voice Cloning: ElevenLabs/Speechify

I looked around for a good, free voice cloner and couldn't find any that actually delivered a voice that sounded anything close to mine. Most sounded like bad text to speech (TTS) voices from 5-8 years ago.

If you really don't want to spend any money then Speechify is the only site I came across that would let you train the AI on your own voice for free and create a quality model. They give you 1000 characters of TTS to try out your trained model. Use them wisely because once they're gone, there's no topping up next month, they're gone!

ElevenLabs Voice Training UI.
ElevenLabs Voice Training UI.
It really is this simple.

Unfortunately Speechify is very pricy to pay by the month, but almost a third of the monthly price if you're prepared to pay annually.

ElevenLabs on the other hand, while you can't train a model on your voice for free, is extremely cheap for a single month of their Starter plan (again credit/debit card or bank transfer only). Currently they're considered the Gold Standard in AI voices.

I decided to take the gamble and paid for a month on the Starter plan - totally worth it! I uploaded less than two minutes of sample audio and got a near perfect copy of my voice in seconds, and now I have 40,000 characters of TTS to burn through.


Animating My Avatar: Hedra

Artflow's Video Studio actually does have the ability to make your images talk but not in a completely convincing way. You'll get some head movement but mostly just mouth movement. On the plus side you can make more than one character speak in a single image.

Hedra, which is currently free to use, by comparison, is specifically designed to animate an avatar head in a more natural way. It's not perfect but in short bursts, it almost passes for actual video footage of the person. It's able to add not just lip syncing but head and upper body movement too.

Unfortunately Hedra doesn't really give you any control over the output. You get what you get. But the result is generally pretty good. It does limit you to thirty seconds of video each generation because by about that point the magic is starting to fall apart just a little.

Hedra's UI.
Hedra is a three step process. 1. Upload your audio.
2. Upload your image (or prompt for a character). 3. Click Generate video.

Hedra also tends to crop relatively close to your image's face. If you wanted the full image, there's no way to tell Hedra. Probably because Hedra's animation is centered mostly around the head and shoulders.


The Finished Clone Video

You can see my finished demonstration video of my cloned avatar below. 

I didn't want my finished video to be all one camera shot so I made an attempt to stitch Hedra's cropped video image back into the full image of me in an office that doesn't exist outside of a computer (as far as I know).

You can see the wide shots don't exactly work if you look too closely. I'm sure I could make it seamless  with a little more work. However if you look at the close up shot, where I've stitched the sides of the image back into the shot so it fills the frame, that section is pretty convincing that it could be the real me.

For the most part I think it's almost good enough to fool people into thinking it really is me. Though I will say, I don't move my head around nearly that much in my videos. It would be nice if Hedra would let you tone down the head movement a little.

What do you think? It may or may not be convincing now but just remember the AI mantra "This is as bad as AI will ever be. It's only going to get better from here," and I think this is pretty good.


o---o--- ---o--- o---

Did you find this article useful?
Subscribe to my newsletter and get the
latest articles delivered to your inbox.

Comments

Popular posts from this blog

Inochi2D - Free Open Source 2D VTuber Avatar Rigging and Puppeteering Software (Part 1)

Inochi2D Creator - Free Open Source VTuber Software. If you've been looking for a way to live perform as a 2D cartoon avatar on camera, whether it be for a live stream or for pre-recorded content like educational videos, then VTuber software is a low cost (or even no cost) option worth looking into. In my previous post, How to Become a VTuber - 2D and 3D Software for Creating and Controlling Your Avatar , I took a brief look at the relatively new but completely free and open source Inochi2D  which I thought showed great potential for my own needs of creating a live performance character rig for my own TET Avatar that I use for all my promotional materials. While it is possible to live perform my character using Cartoon Animator itself, Reallusion's MotionLive2D capture system isn't great - with lip sync in particular. More importantly though, I can't exactly teach people how to use Cartoon Animator if I'm using Cartoon Animator to control my Avatar. What is Inochi2D

Wonder Unit Storyboarder - Free Storyboarding Software for People Who Can (or Can't) Draw

Wonder Unit Storyboarder.  As an independent and solo animator I'm always tempted to try and skip storyboarding my animated shorts because they're usually only single scene sketch comedy type jokes. As a result I have many unfinished projects that kind of petered out due to having no clear finishing line. Storyboarding your productions, no matter how small, gives you a step by step guide of every shot that needs to be completed (no planning shots as you animate). It also allows you to create an animatic that gives you a rough preview of the finished production. In short, you shouldn't skip storyboards as they, generally, increase the chance of the project being completed. Disclaimer - I'm Not a Fan of Storyboarder Upfront, Wonder Unit's Storyboarder  is not my preferred storyboarding software. However it's completely free, has a number of very compelling featu

Zack in Time - Support an Independent 2D Animated Series that is Creating Opportunities for New Talent

Z ack in Time is an independent, 2D animated series about a biracial 13-year-old  teen, who has trouble fitting in at his new school, until he finds a time-traveling watch developed by a secret government agency.  Created by Christian Haynes, who I featured in December of 2021 , and produced by Imhapie , the production has now launched a Kickstarter  (which runs until October 25th, 2024) to fund the development of the series pilot episode, as well as a full series, which is ready to produce. Why You Should Support Zack in Time Zack In Time is more than just another 2D animated series hoping to be successful, and is worth your consideration on two levels. Not only does the series hope to tell a fun and authentic story, showcasing unique characters but also, behind the scenes, the producers hope it continues to be a launching pad for young artists' careers.  A Relatable Story that Promotes Diversity and Representation The story of Zack in Time reflects the struggle that we all feel a

Glif (Alpha) - Make Your Own Tiny AI Powered Niche Image Generator Apps

I  first heard about Glif through a YouTube video that mentioned you could get access to Flux Pro (the latest 'game changer AI' generative image model) through the site for free. While I had a vague notion from the video of what Glif was, I wasn't expecting it to be so easy to get started with, and so good with my very first results. Glif is an easy to use, low-code platform for creating tiny AI-powered generators called Glifs. While that may not sound inspiring, what Glifs allow you to do is create a tiny app that niches down to a specific type of AI generation that the user modifies with their own inputs. The best way to really understand is with an example.  My First Glif I've recently been using VivaGo's AI platform  (free and unlimited at the time of writing) to consistently generate full body characters in a front facing T-Pose that I can rig as front facing characters in Cartoon Animator. Unfortunately it can be a bit hit or miss maintaining the T-Pose part

Krita AI Diffusion - Generative Image AI For Krita is Seriously Useful, Powerful and Free (If You Can Install it Locally)

Generative AI sequence of a woman in a business suit. From sketch to refined image using Krita AI Diffusion - by TET G enerative image AI, where you describe an image with a text prompt to an Artificial Intelligence model and it produces a new image based on your prompt, is gaining a strong hold as a tool for many artists. Krita AI Diffusion brings generative AI image tools right into your favourite free and opensource, graphics editor, Krita. Not only that, if you have a computer with decent specs (and at least 10GB of hard drive space), Krita AI Diffusion is completely free. What If I Don't Have a Powerful Computer? If you're in my situation, with a computer that was around before anyone in the mainstream had even heard of generative AI, you can still access Krita AI Diffusion for free, using a cloud based AI server, Interstice  and 300 tokens, to get you started. Once your initial tokens run out, purchase 5000 more for 10€ (approx US$11.00). Tokens never expire. I would

The Ultimate Independent Animator's App and Resource List - Animation and Video Life

Image created with Cartoon Animator 4. Being an independent animator is not like a studio animation job. There's so much more to do that is indirectly related to the actual task of animating. Over the years I've sought out many apps, tools, and services that can help me achieve that one single task, expressing myself through animation. Below is my Ultimate Independent Animator's Resource List for 2024 (last updated Oct 2024). It started out as a list of free or low cost apps that could help you in every stage of producing either 2D or 3D animation, and then just kind of grew from there. You may not have been looking for a Time Management App as much as you needed something to get you started in 3D animation but when those commissioned projects start coming in you'll have a head start on maximizing your time. All the apps and services on this list had to meet two main criteria: They had to be useful and relevant to an Indy Animator/artist. The base app/se

Using Avatar Maker with Cartoon Animator - Free Vector Cartoon Avatar Creator with Four Art Styles

I'm always on the lookout for cartoon avatar makers of any kind, whether it be ones that 'cartoonify' your photo, or ones that let you build a cartoon likeness from a library of individual features.  Free Avatar Maker  falls into the latter category and can be used for making head and shoulder cartoon avatars. While it doesn't have an extensive library of character features (you may struggle to get a good likeness), uniquely it will make your avatar in four different art styles concurrently, allowing you to save the one you like most, or even all four.  I wasn't overly impressed how my TET avatar looked in the first two styles, but style three is quite possibly the coolest looking version of my avatar I've ever seen in a third party avatar creator. It's a very contemporary style. Style four, line art, is also not too bad. Avatar Maker's User Interface. Switch between the four different art styles shown across the top at any time. I particularly like the