Revisiting Replica Studios - The Most Natural A.I. Text To Speech Based on Real Human Voices

Replica A.I. Voices are great for
in game character voices.

I first trialed Replica Studios A.I. Voice Actor Text To Speech (TTS) voices shortly after they launched about a year ago and was suitably impressed by how natural they sounded. Some voices better than others but most sounding very close to human with the right inflections at the appropriate moments. Since then they have been working on improvements and adding more voices to their voice cast library.

I have used Replica Studios on projects in the past, most notably on my 2021 Reallusion, Lip Sync Contest entry featuring my character, Bat Storm.

Mia and the Tourist.

This time I've decided to use Replica to cast the other two voices in my upcoming Mia and the Tourist animated short that I've been blogging the process of creating through demonstrating the apps I'm using to make it.

So far I've covered scriptwriting, storyboarding, background creation, and character rigging.

When I say the other two voices I'm referring to Mia and the Car Rental Manager (see my post on scriptwriting if you want more details of this animated short). The Tourist is loosely based on myself so I typically use my normal speaking voice for this character.

Generating Replica Voices

Replica itself is a desktop app that connects to their server. Once installed and logged in you can begin creating voices right away. If you're using iClone, Unreal Engine or Unity there are integrations available that let you use Replica directly from those apps.

Initially you start with 30 minutes of free credit and you can buy more from the app as needed. You can also earn credit by referring friends to the app with your unique link (as I've done with my link at the beginning of this article).

There are two ways to proceed in the app, the Sandbox, where you can generate individuals lines of dialogue and download as needed, or Projects, where you can enter an entire script worth of dialogue and download all at once. Unlike the Sandbox you can save multiple projects here too.

The Sandbox

Replica Studio's Sandbox interface.
Change attributes like Speech Rate, Pitch,
and the Style.

The Sandbox is about as simple as it gets. Select a voice, type your dialogue, click Play to preview. Select any part of the text to make further adjustments like adding silence, volume, or changing the speech rate, pitch, or style (when options are available).

Saving a take will count towards your time credit so it's best to save only those takes you intend to download.

The sandbox will let you save takes with different character voices and dialogue so it is possible to generate all the voices you might need for a short project here.

When you have all your takes just download each individually with their download button.

Projects

Replica Projects Section helps keep your
script organised and easy to enter.

The Project section is a new addition to the app since I first reviewed it. At the individual line level it works the same as the sandbox but above that it allows you to pick your character voices and assign them to characters in your script. You can then enter your script broken up into its scenes.

The interface for entering dialogue is more of a list so you can work quickly, see an overview of key settings and make changes to text and character voices without having to get down to individual line level (you only need to go that deep if you want to play around with volume, speech rate, and pitch on individual words and phrases).

Just aware that if you preview the same line of dialogue multiple times it does start using up your credit. So keep an eye on this if your credit is running low.

Other useful features include the ability to move lines of dialogue around, and you can preview an entire scene with the play button at the bottom of the scene window.

Once you have all your dialogue entered you can download each line individually (which is more useful if you want to change a line of dialogue later) or download an entire scene at a time as a zip file in WAV, MP3, OGG, and FLAC formats. Each audio file is named after the line number in the scene.

Replica's Voice Library

New voices are being added to Replica all the time. You can browse the library and check out each voice with the pre-recorded samples just by going to the voice library section of the app. However it's actually easier to browse from the sandbox where, when you go to select a voice, you can filter the available voices with search options like Gender, Age, and Accent etc.

Like any Text To Speech system there will always be some voices better than others. Personally I think Replica still has some way to go before their voices will pass as human in every situation but many of them are good enough to use in situations that don't involve too many emotional extremes.

Adding the Voices to my Storyboard Animatic

As mentioned earlier I used Replica to voice two of my characters. The third is being voiced by me.

I recorded my lines using Audacity. I won't be covering that as part of this series but if you do want to get some insight into how I batch export my dialogue from Audacity into individual audio files for each line of dialogue check out my article, A Better Way to Export Clips From a Single Audio Dialogue Track in Audacity

For the purpose of letting you hear my generated Replica voices I imported my audio into Storyboarder and exported an animatic of my rough sketches (below). I wouldn't usually do this. Normally I'd make an animatic using my characters and sets in Cartoon Animator but now, having seen this, I'm glad I did.

As you can see (and hear) my Replica voices aren't as successful as I'd like. Mia's voice is kind of okay but has some issues with volume and pitch. The Rental Car Manager is all over the place. You can easily tell his voice is text to speech. I may have to try something else for him.

My own voice is fine, I guess, given the Tourist only has three very short lines.

Overall I think Replica's voices are coming along. While there is a good selection of voices not all of them work well as character voices. I really struggled to find a voice for Mia that I liked. The one you hear in the animatic is as close as I could get to what I imagined but it still sounds too youthful.

The next step for my animation is to start putting it altogether in Cartoon Animator.

Comments

Inochi2D - Free Open Source 2D VTuber Avatar Rigging and Puppeteering Software (Part 1)

Inochi2D Creator - Free Open Source VTuber Software. If you've been looking for a way to live perform as a 2D cartoon avatar on camera, whether it be for a live stream or for pre-recorded content like educational videos, then VTuber software is a low cost (or even no cost) option worth looking into. In my previous post, How to Become a VTuber - 2D and 3D Software for Creating and Controlling Your Avatar , I took a brief look at the relatively new but completely free and open source Inochi2D which I thought showed great potential for my own needs of creating a live performance character rig for my own TET Avatar that I use for all my promotional materials. While it is possible to live perform my character using Cartoon Animator itself, Reallusion's MotionLive2D capture system isn't great - with lip sync in particular. More importantly though, I can't exactly teach people how to use Cartoon Animator if I'm using Cartoon Animator to control my Avatar. What is Inochi2D

Dollars Mocap: Full Body Webcam Motion Capture (Including Hands and Fingers) For iClone and Cartoon Animator

Even though I should be further away from the camera Dollars Mocap MONO still does a good job of tracking my arms, hands and fingers. Ever since I wrote my series on becoming a VTuber , discovering it was possible to do full body motion capture, including hands and fingers, with just software and a webcam, I've been on the look out for any motion capture software that can bring that functionality to Cartoon Animator. Dollars Mocap is a low cost motion capture application with a free trial that I learned about through the YouTube Channel Digital Puppets and their test video . It can record full body, upper body, arms and hands, and facial mocap from a live video source or pre-recorded video. Investigating further, I discovered not only does Dollars Mocap have a free iClone7, iClone8 character profile file download (look for it at the bottom of the main program download page), so you can use the saved motions with iClone8, they've also got a demo video for how to convert your

Prome AI Sketch Render Tool - Your Tradigital Clean Up and Colorist Artist for Character and Background Design

Random character head, Biro sketches drawn by TET (left). Render by PromeAI (right) using Prome's Sketch Render tool set to 'Comon:Cartoon, Render Mode: Outline'. W hile I don't do New Year Resolutions, one of my plans for the year ahead is to do more of my own art. Specifically character design drawn in an actual, physical sketchbook. To that end, I have been spending the last half hour of most days drawing a page or two of random biro sketches in my sketchbook and posting the pages to my Instagram account (this link will take you to one of my posts). These sketches are mostly practicing my skills because I don't really draw regularly anymore. Here is a tip, if you do this kind of sketching, and push yourself to keep doing it, you will see many drawings that could be taken further, even if you don't have anything they're suited for just at the moment. Which is where my second favorite AI Image Tool (after Leonardo.ai ) PromeAI comes into play. PromeAI

Moho 14 Released - Still the Best 2D Animation Software for Indy Animators on a Budget

Moho 14 Released. Regular readers know I am a Reallusion, Cartoon Animator advocate through and through. Hands down I would recommend Cartoon Animator 5 first over Lost Marble's Moho 14 to anyone who is just starting in 2D animation, is a team of one, or just needs to animate as quickly as possible. However, feature for feature, Moho is, arguably, the best 2D animation software for the rest of us who can't justify a Toon Boom Harmony , or Adobe Creative Cloud subscription (and even with their applications Moho is very competitive on features). You can get started with Moho Debut for just USD$59.99 which is a cut down version of Moho Pro but it still has the most essential features needed for 2D animation. While Moho Pro is a whopping USD$399.99 (Cartoon Animator, which only has one version, is just USD$149.00) upgrades to new version numbers come down to a quarter of the price at USD$99.00. Even though Reallusion just released features like Motion Pilot Puppet Animation and

Wonder Unit Storyboarder - Free Storyboarding Software for People Who Can (or Can't) Draw

Wonder Unit Storyboarder. As an independent and solo animator I'm always tempted to try and skip storyboarding my animated shorts because they're usually only single scene sketch comedy type jokes. As a result I have many unfinished projects that kind of petered out due to having no clear finishing line. Storyboarding your productions, no matter how small, gives you a step by step guide of every shot that needs to be completed (no planning shots as you animate). It also allows you to create an animatic that gives you a rough preview of the finished production. In short, you shouldn't skip storyboards as they, generally, increase the chance of the project being completed. Disclaimer - I'm Not a Fan of Storyboarder Upfront, Wonder Unit's Storyboarder is not my preferred storyboarding software. However it's completely free, has a number of very compelling featu

Can't Draw Characters? Create Highly Detailed Characters from Simple Drawings and Prompts Free with Realtime Canvas by Leonardo.AI

Leonardo.ai's Realtime Canvas. Create highly detailed images from simple drawings. I f you've had an idea for a character but don't have the artistic skill to design it yourself, or the budget to hire someone to do the design work for you, then Leonardo.ai's Realtime Canvas may be your new creative partner. Sure you could use Leonardo.ai's regular text prompt to image generator but that can be very hit and miss, and may take many generations before you finally craft a complex prompt that's getting something close to what you had in mind. Realtime Canvas, on the other hand, lets you craft a simple text prompt and draw a rough image, both of which you can keep refining until you get a final, real time, updated image that looks close to (and probably better than) what you had in mind. Using Realtime Canvas Once you've signed up for a free account with Leonardo.ai (which will give you 150 free credits, renewed daily), click on Realtime Canvas, from the side

Start Your 2D Animation Side Hustle - Sell Your Cartoon Animator Characters, Props, Scenes, and Motion Files in the Reallusion 2D/3D Marketplace

Have you thought about starting a side hustle selling your original Cartoon Animator assets in the Reallusion 2D/3D Marketplace ? In this article, the first in a series on selling in the marketplace, I'll give you an overview of what's involved, why you should give it some thought, and whether you can earn enough to quit your day job (or at least have a worthwhile side hustle). If you're an artist with any kind of drawing skills, and you're creating your own original characters, props, scenes, and even motion files for your Cartoon Animator projects, then setting up your own store in the Reallusion Marketplace should be a no brainer. You're making content already, it doesn't cost you anything to set up, and Reallusion only takes a 30% commission from each item sold. (If you think that's a lot, I'll address that further down). Don't be put off if you think your art skills aren't up to professional standards. There are plenty of artists with naïve

Animation and Video Life

Search This Blog