Skip to main content

How to Become a Virtual YouTuber (VTuber) - 2D and 3D Software for Creating and Controlling Your Avatar

TET VRoid Avatar in VSeeFace.
Young TET VRoid Avatar in VSeeFace.
In a previous YouTube video I demonstrated how you could use a webcam, Cartoon Animator and its MotionLive 2D Facial Capture system to create a virtual avatar that you could perform live during a live stream. Unfortunately the motion capture performance of this system is very average to say the least, and is even more expensive if you want to add hand motion capture via a Leap Motion Controller.

This lead me down a rabbit hole of seeing what other solutions are out there, where I bumped into the world of the VTuber or Virtual YouTuber (creators who puppet and voice an on camera avatar instead of appearing on camera themselves). Specifically VTuber software, which has been a real eye opener for how expensive and far behind Reallusion's upper body motion capture system actually is.

Free 3D VTuber Avatar Creator, VRoid Studio.
The VTuber world is heavily entrenched in the world of Anime so you'll see a lot of 2D and 3D avatars in this style. If fact, if an Anime avatar is your thing, you can be up and running with a customized 3D Avatar within an hour using VRoid studio, a Free 3D Anime character maker specifically designed for this purpose, and VSeeFace, an also free, VTuber avatar puppeteering program.

The Gold Standard in 3D VTuber Software (IMO)


One of the best examples of 3D VTuber software I've come across is Webcam Motion Capture (yes that is literally the application's name). Free to try, and a small monthly fee to remove the aggressive text watermark, it seems like the gold standard of upper body motion capture.

Using a standard webcam and some AI wizardry this software can capture facial expressions including lip sync, head and upper body movement, arms, hands, and individual finger movement, very smoothly in real time.

Webcam Motion Capture in action using my custom VRoid Avatar.
Webcam Motion Capture in action using my custom 3D VRoid, Anime Avatar. This application has everything you would want in upper body motion capture all from a single webcam. While it's completely free to try the aggressive text watermark makes the free version totally unusable for anything but testing. That said it's only US$1.99 per month or you can buy a lifetime license.

The system is so good I feel Reallusion should drop everything they're doing with upper body motion capture and use this instead - seriously. It has full hand and finger motion capture with no extra hardware to buy.

If that wasn't enough it has so many additional settings you can adjust to not just fine tune the motion capture but to also enhance your avatar's performance, like automatic eye blinking, hot key expressions, or idle motions for when you're not doing any movement yourself. As well it supports the .VRM format which is the standard avatar file format most of these systems use (so you can load your VRoid characters into it).

VSeeFace and Kalidoface 3D (Free Alternatives)

If you're looking for a free alternative to Webcam Motion Capture try VSeeFace a very popular, free app used by VTubers. Watch this very good video by Vtuber, Fofamit, to get you started: 

VSeeFace does support the addition of a Leap Motion Controller to capture hand data but it turns out you don't actually need one. Install this free hand motion capture softwareVideo here by VTuber, Lady Aska, to explain how.

If you don't want to install anything then try Kalidoface 3D VTuber app. It's completely browser based and you can use your custom VRoid Avatars with it.

Kalidoface 3D featuring my TET VRoid Avatar.
Kalidoface is free, browser based, and has full body motion capture. Here I've loaded my TET VRoid Avatar into the interface.

Most of the software I'd found so far was 3D based so I kept going to see what was out there for 2D VTubers.

The PNG 2D VTuber (The simplest way to be a 2D VTuber)

This is not the first way I discovered how to make and puppet your own 2D VTuber Avatar but it's so easy to set up using Cartoon Animator and the completely free PNG Avatar VTuber software, Veadotube Mini.

Unlike most VTuber software this one does not use a webcam or motion capture at all. Instead it works based on the sound from your microphone. In essence it's a flipbook style software that switches between four states of your PNG Avatar... 1. Eyes open, mouth closed, 2. Eyes open, mouth open, 3. Eyes closed, mouth closed, 4. Eyes closed, mouth open. This doesn't sound like much but it's enough to create very rudimentary lip syncing with random eye blinks at any time.

Creating my PNG images in Cartoon Animator for use in Veadotube mini shown in the lower right.
By setting up the maximum image size of 2048 pixels square in Cartoon Animator it's easy to create all the transparent PNG images you need to make a PNG VTuber Avatar in Veadotube Mini (shown lower right). All you need is a microphone.

In addition to that you have three different movement states you can apply depending upon what mouth state is being displayed.

For even more flexibility you can add different states of these images and movement, that you can switch with your assigned hot keys. For example I made a second state of my character talking excitedly with slightly wider eyes and hands raised up in the air.

However because this software is literally flipping images you can do almost any pose or style of character you need. Side view, talking horse... whatever you can dream up. VTuber, Syafire, has a full video on how to set Veadotube Mini up in 4 easy steps.

Also worth mentioning... If you like this style of VTuber Avatar there is a low cost, paid software called Honk that is worth checking out. It's a little more complex, supporting layered characters that allows  better lip sync capabilities.

Advanced 2D VTuber Avatar Creation

Once you start looking into creating a more advanced 2D Avatar with the ability to add facial expressions, head turning, physics and more, you may start thinking being a PNG VTuber is a great compromise (or maybe you'll just stick with a VRoid Avatar - the easiest option of all).

Live 2D Cubism Editor

Finding a free or low cost 2D avatar character rigging application is not easy. There seems to be only one name in town, Live2D Cubism. On the plus side they'll give you a 42 day full feature trial of their VTuber rigging software, which is enough time to learn the process. I'm told it's simple once you know how but, at first glance, it's overwhelming.

Live 2D Cubism VTuber Rigging Software with my TET Avatar being rigged.
The early stages of my attempt to rig my character with Live 2D Cubism. I imported the same mouth and eye sprites I used for the character in Cartoon Animator but these may not be usable as Live 2D uses deform tools to create most facial movement and expression.

Preparing your avatar for rigging is a relatively simple case of separating all the sprites into individual parts in a .PSD file with no group folders. Everything is placed in layer order from top to bottom.

While I tried to rig my TET avatar in Live 2D Cubism I wasn't able to learn it quick enough to demonstrate it here. Even with the many YouTube tutorials I was overwhelmed - and my avatar isn't nearly as complex as some of the Anime avatars many VTubers create themselves and use.

Inochi2D VTuber Rigging Editor

I was able to find a free, opensource 2D Vtuber rigging software called Inochi2D Creator. Unfortunately it's a completely new VTuber avatar format that only works with their Inochi2D Sessions puppeteering software. However it can import the same .PSD files you would use for Live2D Cubism as a starting point.

Inochi2D Creator User Interface with my TET Avatar.
As you can see Inochi2D Creator had no trouble importing my .PSD file. From what I can tell the rigging process is similar to Live2D Cubism but this is as far as I got prior to writing this post.

The other issue is Inochi2D is still in development. I wasn't able to determine if it is still an active project or not however the documentation for Creator seemed quite good, while the same can't be said for Session, which is almost non-existent. While I was able to load an example avatar into Session I wasn't able to work out how to hook up a motion capture solution to it, despite it supporting VMC motion capture that I had (and you will too if you download DSSBodyTrackor2 that is used to add hand capture to VSeeFace, mentioned earlier).

I did find an excellent YouTube Tutorial stepping you through the whole process of using both Inochi  Creator, Session, and hooking everything up to OBS by VTuber Dragon Dragonnator96 - How To Rig a VTuber Model with Inochi2D (0.7.4.1 Creator and 0.5 Session). Definitely worth searching YouTube for more as well.

2D VTuber Puppeteering Software

Once you have your VTuber avatar rigged with Live2D (not Inochi Creator because their format only works with Session) you need an application to puppeteer it with. 

Most applications focus on facial motion capture so your avatar will lip sync and emote in line with your own facial movement. Other body motions and physics based motions are usually achieved with pre-set up auto-animated loops, or triggered with hot keys (such as extreme facial expressions to enhance your facial motion capture). Some also let you create animated environmental effects for your avatar too.

As I haven't really been able to try any of this software yet I'm going to leave this up to VTuber, Kappako Channel's video, Which is the best FREE face tracker for Live2d Vtubers? where she compares the four most commonly used applications including:

Still from Kappako's video, Which is the best FREE face tracker for Live2d Vtubers?
Still from Kappako's video, Which is the best FREE face tracker for Live2d Vtubers?

  • VTube Studio - Probably the most used application. Supports Webcam and iPhone motion tracking. Includes hand tracking. Available via Steam for PC users.
  • PrprLive - Also on Steam. Facial motion capture via webcam.
  • VUP - Steam link. Webcam face capture. Does support other motion capture devices for body capture.
  • Animaze - Works with webcam or iphone. Animaze also have their own avatar editing software that supports common 2D and 3D formats. It's purpose is to help optimize your model to take advantage of the special features of  Animaze.

How Do I Live Stream or Just Record My Performance?

If you're goal is to keep things free or low cost then OBS Studio is probably the most widely used application for live streaming. It's completely free and has so many options. 

It's beyond the scope of this article to go into detail on how to set OBS up with your avatar but the general idea is (if you want to overlay your character on something else like gameplay) to record your avatar with a transparent background or single color (usually bright green) background, and set up a Scene in OBS that contains all the sources you want on screen including your Avatar puppeteering software. If you're using a color background you'll need to add a chromakey filter to make this transparent.

Then you either set up OBS to Live stream, or record video to a file for editing later.

Coming Up

I really wanted to have my 2D TET Avatar all set up for this article so I could demonstrate a 2D VTuber Avatar that wasn't Anime based. 

I also wanted to see how useful having a character template from Cartoon Avatar might be. For example can you use all those mouth sprites in a VTuber Avatar (I think you can but sprite switching is not typically how lip sync is done with VTuber Avatars).

My plan is to write a follow up article on my experience of learning the rigging process and then demonstrating my TET Avatar with a video. For now I wanted to make sure you had everything you need to start your own journey into creating a VTuber avatar.

While you may not want to be a VTuber who live streams, a Virtual Avatar can be very useful if you want to create video tutorials that perhaps appeal to a younger audience, or maybe you'd like to do virtual presentations as an animated presenter. There are so many possibilites.
 

Comments

Popular posts from this blog

Inochi2D - Free Open Source 2D VTuber Avatar Rigging and Puppeteering Software (Part 1)

Inochi2D Creator - Free Open Source VTuber Software. If you've been looking for a way to live perform as a 2D cartoon avatar on camera, whether it be for a live stream or for pre-recorded content like educational videos, then VTuber software is a low cost (or even no cost) option worth looking into. In my previous post, How to Become a VTuber - 2D and 3D Software for Creating and Controlling Your Avatar , I took a brief look at the relatively new but completely free and open source Inochi2D  which I thought showed great potential for my own needs of creating a live performance character rig for my own TET Avatar that I use for all my promotional materials. While it is possible to live perform my character using Cartoon Animator itself, Reallusion's MotionLive2D capture system isn't great - with lip sync in particular. More importantly though, I can't exactly teach people how to use Cartoon Animator if I'm using Cartoon Animator to control my Avatar. What is Inochi2D

Moho 14 Released - Still the Best 2D Animation Software for Indy Animators on a Budget

Moho 14 Released. Regular readers know I am a Reallusion, Cartoon Animator advocate through and through. Hands down I would recommend Cartoon Animator 5 first over Lost Marble's Moho 14 to anyone who is just starting in 2D animation, is a team of one, or just needs to animate as quickly as possible. However, feature for feature, Moho is, arguably, the best 2D animation software for the rest of us who can't justify a Toon Boom Harmony , or Adobe Creative Cloud subscription (and even with their applications Moho is very competitive on features). You can get started with Moho Debut for just USD$59.99 which is a cut down version of Moho Pro but it still has the most essential features needed for 2D animation. While Moho Pro is a whopping USD$399.99 (Cartoon Animator, which only has one version, is just USD$149.00) upgrades to new version numbers come down to a quarter of the price at USD$99.00. Even though Reallusion just released features like Motion Pilot Puppet Animation and

Start Your 2D Animation Side Hustle - Sell Your Cartoon Animator Characters, Props, Scenes, and Motion Files in the Reallusion 2D/3D Marketplace

Have you thought about starting a side hustle selling your original Cartoon Animator assets in the Reallusion 2D/3D Marketplace ? In this article, the first in a series on selling in the marketplace, I'll give you an overview of what's involved, why you should give it some thought, and whether you can earn enough to quit your day job (or at least have a worthwhile side hustle). If you're an artist with any kind of drawing skills, and you're creating your own original characters, props, scenes, and even motion files for your Cartoon Animator projects, then setting up your own store in the Reallusion Marketplace should be a no brainer. You're making content already, it doesn't cost you anything to set up, and Reallusion only takes a 30% commission from each item sold. (If you think that's a lot, I'll address that further down). Don't be put off if you think your art skills aren't up to professional standards. There are plenty of artists with naïve

Wonder Unit Storyboarder - Free Storyboarding Software for People Who Can (or Can't) Draw

Wonder Unit Storyboarder.  As an independent and solo animator I'm always tempted to try and skip storyboarding my animated shorts because they're usually only single scene sketch comedy type jokes. As a result I have many unfinished projects that kind of petered out due to having no clear finishing line. Storyboarding your productions, no matter how small, gives you a step by step guide of every shot that needs to be completed (no planning shots as you animate). It also allows you to create an animatic that gives you a rough preview of the finished production. In short, you shouldn't skip storyboards as they, generally, increase the chance of the project being completed. Disclaimer - I'm Not a Fan of Storyboarder Upfront, Wonder Unit's Storyboarder  is not my preferred storyboarding software. However it's completely free, has a number of very compelling featu

Can You Learn Reallusion's Cartoon Animator 5 for Free Using Their 137 Official YouTube Video Tutorials Sorted Into a Logical Learning Order?

Or you could just buy The Lazy Animator Beginner's Guide to Cartoon Animator . While Reallusion's Cartoon Animator is one of the easiest 2D animation studios to get up and running with quickly, learning it from all of the official, free, video tutorials can be more overwhelming than helpful. With more than 137 videos totaling more than 28 and a half hours of tutorials, spread across three generations of the software (Cartoon Animator 3 through 5) it's hard to know if what you're learning is a current or legacy feature that you either need to know or can be skipped. Many of the official tutorials only teach specific features of the software and don't relate at all to previous or later tutorials. As a result there are many features either not mentioned or are hard to find. To make your learning easier, on this page, I've collected together all of the essential, official, free video tutorials and sorted them into a learning order that makes sense. Simply start at

2D Animation Side Hustle - How to Find a Niche Market Selling Digital Characters, Props, and Background Art in the Reallusion 2D Marketplace

C ontinuing my series on your 2D Animation Side Hustle, Selling in the Reallusion 2D Marketplace , in this article I'm going to take a look at niches and why they're important to becoming a successful seller. I'll also look at some indicators, specific to the Reallusion Marketplace, that can help point you in the right direction of finding niches that are in demand. What is a Niche and Why Are They Important? In marketing terms a niche is kind of like a sub category or a section of a subject that you want to focus on. It's not the big idea with wide appeal that everybody loves, it's a smaller, more specific section of the idea that fewer people really connect with. For example, in the Reallusion 2D Marketplace, Characters are the biggest sellers. Everybody loves and wants to buy characters. They're great! That's all you need to know, characters sell the best, go make some! There are currently 4413 characters for sale in the marketplace. How will people find

Artbreeder - Using AI created Character and Background Content in your Animations

A selection of User/AI generated images from Artbreeder. If you're looking for an endless supply of 2D character and background images for your animations then Artbreeder , an online Artificial Intelligence (AI) that generates image mash-ups you can tweak as much as you like, could be the ultimate content library. What is Artbreeder? Artbreeder is free to use though there are various paid plans, that give you additional features, such as higher resolution download images or more settings to play with. All images created on the site are Public Domain (CC0 License) and can be used in commercial projects. Using Artbreeder's online app you can generate head shot portraits, full body characters, landscapes, and other scenes simply by choosing two or more existing images to mash together then, using a series of sliders, to select which traits from each image you wish to lean toward in the final image. Photo Comparison - Top is my original uploaded photo. Bottom is Artbreeder's ap