Skip to main content

How to Become a Virtual YouTuber (VTuber) - 2D and 3D Software for Creating and Controlling Your Avatar

TET VRoid Avatar in VSeeFace.
Young TET VRoid Avatar in VSeeFace.
In a previous YouTube video I demonstrated how you could use a webcam, Cartoon Animator and its MotionLive 2D Facial Capture system to create a virtual avatar that you could perform live during a live stream. Unfortunately the motion capture performance of this system is very average to say the least, and is even more expensive if you want to add hand motion capture via a Leap Motion Controller.

This lead me down a rabbit hole of seeing what other solutions are out there, where I bumped into the world of the VTuber or Virtual YouTuber (creators who puppet and voice an on camera avatar instead of appearing on camera themselves). Specifically VTuber software, which has been a real eye opener for how expensive and far behind Reallusion's upper body motion capture system actually is.

Free 3D VTuber Avatar Creator, VRoid Studio.
The VTuber world is heavily entrenched in the world of Anime so you'll see a lot of 2D and 3D avatars in this style. If fact, if an Anime avatar is your thing, you can be up and running with a customized 3D Avatar within an hour using VRoid studio, a Free 3D Anime character maker specifically designed for this purpose, and VSeeFace, an also free, VTuber avatar puppeteering program.

The Gold Standard in 3D VTuber Software (IMO)

One of the best examples of 3D VTuber software I've come across is Webcam Motion Capture (yes that is literally the application's name). Free to try, and a small monthly fee to remove the aggressive text watermark, it seems like the gold standard of upper body motion capture.

Using a standard webcam and some AI wizardry this software can capture facial expressions including lip sync, head and upper body movement, arms, hands, and individual finger movement, very smoothly in real time.

Webcam Motion Capture in action using my custom VRoid Avatar.
Webcam Motion Capture in action using my custom 3D VRoid, Anime Avatar. This application has everything you would want in upper body motion capture all from a single webcam. While it's completely free to try the aggressive text watermark makes the free version totally unusable for anything but testing. That said it's only US$1.99 per month or you can buy a lifetime license.

The system is so good I feel Reallusion should drop everything they're doing with upper body motion capture and use this instead - seriously. It has full hand and finger motion capture with no extra hardware to buy.

If that wasn't enough it has so many additional settings you can adjust to not just fine tune the motion capture but to also enhance your avatar's performance, like automatic eye blinking, hot key expressions, or idle motions for when you're not doing any movement yourself. As well it supports the .VRM format which is the standard avatar file format most of these systems use (so you can load your VRoid characters into it).

VSeeFace and Kalidoface 3D (Free Alternatives)

If you're looking for a free alternative to Webcam Motion Capture try VSeeFace a very popular, free app used by VTubers. Watch this very good video by Vtuber, Fofamit, to get you started: 

VSeeFace does support the addition of a Leap Motion Controller to capture hand data but it turns out you don't actually need one. Install this free hand motion capture softwareVideo here by VTuber, Lady Aska, to explain how.

If you don't want to install anything then try Kalidoface 3D VTuber app. It's completely browser based and you can use your custom VRoid Avatars with it.

Kalidoface 3D featuring my TET VRoid Avatar.
Kalidoface is free, browser based, and has full body motion capture. Here I've loaded my TET VRoid Avatar into the interface.

Most of the software I'd found so far was 3D based so I kept going to see what was out there for 2D VTubers.

The PNG 2D VTuber (The simplest way to be a 2D VTuber)

This is not the first way I discovered how to make and puppet your own 2D VTuber Avatar but it's so easy to set up using Cartoon Animator and the completely free PNG Avatar VTuber software, Veadotube Mini.

Unlike most VTuber software this one does not use a webcam or motion capture at all. Instead it works based on the sound from your microphone. In essence it's a flipbook style software that switches between four states of your PNG Avatar... 1. Eyes open, mouth closed, 2. Eyes open, mouth open, 3. Eyes closed, mouth closed, 4. Eyes closed, mouth open. This doesn't sound like much but it's enough to create very rudimentary lip syncing with random eye blinks at any time.

Creating my PNG images in Cartoon Animator for use in Veadotube mini shown in the lower right.
By setting up the maximum image size of 2048 pixels square in Cartoon Animator it's easy to create all the transparent PNG images you need to make a PNG VTuber Avatar in Veadotube Mini (shown lower right). All you need is a microphone.

In addition to that you have three different movement states you can apply depending upon what mouth state is being displayed.

For even more flexibility you can add different states of these images and movement, that you can switch with your assigned hot keys. For example I made a second state of my character talking excitedly with slightly wider eyes and hands raised up in the air.

However because this software is literally flipping images you can do almost any pose or style of character you need. Side view, talking horse... whatever you can dream up. VTuber, Syafire, has a full video on how to set Veadotube Mini up in 4 easy steps.

Also worth mentioning... If you like this style of VTuber Avatar there is a low cost, paid software called Honk that is worth checking out. It's a little more complex, supporting layered characters that allows  better lip sync capabilities.

Advanced 2D VTuber Avatar Creation

Once you start looking into creating a more advanced 2D Avatar with the ability to add facial expressions, head turning, physics and more, you may start thinking being a PNG VTuber is a great compromise (or maybe you'll just stick with a VRoid Avatar - the easiest option of all).

Live 2D Cubism Editor

Finding a free or low cost 2D avatar character rigging application is not easy. There seems to be only one name in town, Live2D Cubism. On the plus side they'll give you a 42 day full feature trial of their VTuber rigging software, which is enough time to learn the process. I'm told it's simple once you know how but, at first glance, it's overwhelming.

Live 2D Cubism VTuber Rigging Software with my TET Avatar being rigged.
The early stages of my attempt to rig my character with Live 2D Cubism. I imported the same mouth and eye sprites I used for the character in Cartoon Animator but these may not be usable as Live 2D uses deform tools to create most facial movement and expression.

Preparing your avatar for rigging is a relatively simple case of separating all the sprites into individual parts in a .PSD file with no group folders. Everything is placed in layer order from top to bottom.

While I tried to rig my TET avatar in Live 2D Cubism I wasn't able to learn it quick enough to demonstrate it here. Even with the many YouTube tutorials I was overwhelmed - and my avatar isn't nearly as complex as some of the Anime avatars many VTubers create themselves and use.

Inochi2D VTuber Rigging Editor

I was able to find a free, opensource 2D Vtuber rigging software called Inochi2D Creator. Unfortunately it's a completely new VTuber avatar format that only works with their Inochi2D Sessions puppeteering software. However it can import the same .PSD files you would use for Live2D Cubism as a starting point.

Inochi2D Creator User Interface with my TET Avatar.
As you can see Inochi2D Creator had no trouble importing my .PSD file. From what I can tell the rigging process is similar to Live2D Cubism but this is as far as I got prior to writing this post.

The other issue is Inochi2D is still in development. I wasn't able to determine if it is still an active project or not however the documentation for Creator seemed quite good, while the same can't be said for Session, which is almost non-existent. While I was able to load an example avatar into Session I wasn't able to work out how to hook up a motion capture solution to it, despite it supporting VMC motion capture that I had (and you will too if you download DSSBodyTrackor2 that is used to add hand capture to VSeeFace, mentioned earlier).

I did find an excellent YouTube Tutorial stepping you through the whole process of using both Inochi  Creator, Session, and hooking everything up to OBS by VTuber Dragon Dragonnator96 - How To Rig a VTuber Model with Inochi2D ( Creator and 0.5 Session). Definitely worth searching YouTube for more as well.

2D VTuber Puppeteering Software

Once you have your VTuber avatar rigged with Live2D (not Inochi Creator because their format only works with Session) you need an application to puppeteer it with. 

Most applications focus on facial motion capture so your avatar will lip sync and emote in line with your own facial movement. Other body motions and physics based motions are usually achieved with pre-set up auto-animated loops, or triggered with hot keys (such as extreme facial expressions to enhance your facial motion capture). Some also let you create animated environmental effects for your avatar too.

As I haven't really been able to try any of this software yet I'm going to leave this up to VTuber, Kappako Channel's video, Which is the best FREE face tracker for Live2d Vtubers? where she compares the four most commonly used applications including:

Still from Kappako's video, Which is the best FREE face tracker for Live2d Vtubers?
Still from Kappako's video, Which is the best FREE face tracker for Live2d Vtubers?

  • VTube Studio - Probably the most used application. Supports Webcam and iPhone motion tracking. Includes hand tracking. Available via Steam for PC users.
  • PrprLive - Also on Steam. Facial motion capture via webcam.
  • VUP - Steam link. Webcam face capture. Does support other motion capture devices for body capture.
  • Animaze - Works with webcam or iphone. Animaze also have their own avatar editing software that supports common 2D and 3D formats. It's purpose is to help optimize your model to take advantage of the special features of  Animaze.

How Do I Live Stream or Just Record My Performance?

If you're goal is to keep things free or low cost then OBS Studio is probably the most widely used application for live streaming. It's completely free and has so many options. 

It's beyond the scope of this article to go into detail on how to set OBS up with your avatar but the general idea is (if you want to overlay your character on something else like gameplay) to record your avatar with a transparent background or single color (usually bright green) background, and set up a Scene in OBS that contains all the sources you want on screen including your Avatar puppeteering software. If you're using a color background you'll need to add a chromakey filter to make this transparent.

Then you either set up OBS to Live stream, or record video to a file for editing later.

Coming Up

I really wanted to have my 2D TET Avatar all set up for this article so I could demonstrate a 2D VTuber Avatar that wasn't Anime based. 

I also wanted to see how useful having a character template from Cartoon Avatar might be. For example can you use all those mouth sprites in a VTuber Avatar (I think you can but sprite switching is not typically how lip sync is done with VTuber Avatars).

My plan is to write a follow up article on my experience of learning the rigging process and then demonstrating my TET Avatar with a video. For now I wanted to make sure you had everything you need to start your own journey into creating a VTuber avatar.

While you may not want to be a VTuber who live streams, a Virtual Avatar can be very useful if you want to create video tutorials that perhaps appeal to a younger audience, or maybe you'd like to do virtual presentations as an animated presenter. There are so many possibilites.


Popular posts from this blog

Inochi2D - Free Open Source 2D VTuber Avatar Rigging and Puppeteering Software (Part 1)

Inochi2D Creator - Free Open Source VTuber Software. If you've been looking for a way to live perform as a 2D cartoon avatar on camera, whether it be for a live stream or for pre-recorded content like educational videos, then VTuber software is a low cost (or even no cost) option worth looking into. In my previous post, How to Become a VTuber - 2D and 3D Software for Creating and Controlling Your Avatar , I took a brief look at the relatively new but completely free and open source Inochi2D  which I thought showed great potential for my own needs of creating a live performance character rig for my own TET Avatar that I use for all my promotional materials. While it is possible to live perform my character using Cartoon Animator itself, Reallusion's MotionLive2D capture system isn't great - with lip sync in particular. More importantly though, I can't exactly teach people how to use Cartoon Animator if I'm using Cartoon Animator to control my Avatar. What is Inochi2D

Wonder Unit Storyboarder - Free Storyboarding Software for People Who Can (or Can't) Draw

Wonder Unit Storyboarder.  As an independent and solo animator I'm always tempted to try and skip storyboarding my animated shorts because they're usually only single scene sketch comedy type jokes. As a result I have many unfinished projects that kind of petered out due to having no clear finishing line. Storyboarding your productions, no matter how small, gives you a step by step guide of every shot that needs to be completed (no planning shots as you animate). It also allows you to create an animatic that gives you a rough preview of the finished production. In short, you shouldn't skip storyboards as they, generally, increase the chance of the project being completed. Disclaimer - I'm Not a Fan of Storyboarder Upfront, Wonder Unit's Storyboarder  is not my preferred storyboarding software. However it's completely free, has a number of very compelling featu

Can You Learn Reallusion's Cartoon Animator 5 for Free Using Their 137 Official YouTube Video Tutorials Sorted Into a Logical Learning Order?

Or you could just buy The Lazy Animator Beginner's Guide to Cartoon Animator . While Reallusion's Cartoon Animator is one of the easiest 2D animation studios to get up and running with quickly, learning it from all of the official, free, video tutorials can be more overwhelming than helpful. With more than 137 videos totaling more than 28 and a half hours of tutorials, spread across three generations of the software (Cartoon Animator 3 through 5) it's hard to know if what you're learning is a current or legacy feature that you either need to know or can be skipped. Many of the official tutorials only teach specific features of the software and don't relate at all to previous or later tutorials. As a result there are many features either not mentioned or are hard to find. To make your learning easier, on this page, I've collected together all of the essential, official, free video tutorials and sorted them into a learning order that makes sense. Simply start at

Five AI Generative Image to Video Tools For Animation You Can Try Free Right Now

The Emo Girl Character created by Start Animating. A I generative video isn't new but it is the next big thing in the visual imaging space as various development teams work to perfect the generated output.  Just like generating still images AI video sometimes struggles with physics, arm and hand movement, and the general structure of things. However it is getting better and, as is the catch cry of all AI development, this is the worst it will ever be, because it's improving fast. If you're an animator one current potential use of generative AI video is to animate your key frames, as opposed to generating something entirely from a text prompt. Starting with an image helps to keep your characters and art style consistent across AI generations. With that in mind I tried five, free image to video AI generators to see what their potential might be and whether they can handle cartoon style characters well. Note all but the last entry on the list do not create any sound with the

Dollars Mocap: Full Body Webcam Motion Capture (Including Hands and Fingers) For iClone and Cartoon Animator

Even though I should be further away from the camera Dollars Mocap MONO still does a good job of  tracking my arms, hands and fingers. Ever since I wrote my series on becoming a VTuber , discovering it was possible to do full body motion capture, including hands and fingers, with just software and a webcam, I've been on the look out for any motion capture software that can bring that functionality to Cartoon Animator. Dollars Mocap is a low cost motion capture application with a free trial that I learned about through the YouTube Channel Digital Puppets  and their test video . It can record full body, upper body, arms and hands, and facial mocap from a live video source or pre-recorded video. Investigating further, I discovered not only does Dollars Mocap have a free iClone7, iClone8 character profile file download (look for it at the bottom of the main program download page), so you can use the saved motions with iClone8, they've also got a demo video for how to convert your

Animation Paper V5.0 Alpha 2 Pre-release

Animation Paper V5.0 Alpha 2 Pre-release. Animation Paper strives to be the best tool for digital hand drawn animation by not only recreating the traditional animator's table in a modern user interface, but also automating many of the more tedious processes so you can focus almost entirely on drawing. At the time of writing Animation Paper V5.0 is only available as an Alpha 2 Pre-release, and still does not have all of its features implemented, such as the ability to add audio (due to be part of the Alpha 3 release scheduled for the end of May 2020). Animation Paper has been completely redesigned and updated from th hand drawn animation app that was formerly known as Plastic Animation Paper (PAP) which, at the time, was easily one of the best apps for the sole purpose of creating hand drawn digital animation roughs. You can still download PAP completely free from the Animation Paper website . If you look past the extremely dated 90's interface it's still a great tool. I wr

Eric W. Schwartz: Cartoonist, Animator and Amiga Die Hard

July 1992 Edition, CU Amiga Featuring Amy the Squirrel. American Cartoonist, Eric W. Schwartz , (whose unofficial Amiga Icon, Amy the Squirrel, is pictured on the July 92 edition of CU Amiga cover on the right) is my only real animation hero. Sure there are the big names like Disney , Chuck Jones , Tex Avery and even Preston Blair whose influences can all be seen in my own cartoons but Eric did what none of the others could. He showed that really great 2D computer animation was within my reach with little more than an Amiga Computer , a copy of Deluxe Paint and Moviesetter . This was at a time when computer based animation was in its infancy (outside of computer game animation) and Flash was something that lights did. There were many great Amiga artists but Eric was really the only one consistently making very funny, traditional style animations. His humor and drawing style is heavily influenced by classic Warner Brothers and Disney cartoons but he managed to build on this,