Skip to main content

Revisiting Replica Studios - The Most Natural A.I. Text To Speech Based on Real Human Voices

Replica AI Voices Hero Website Image.
Replica A.I. Voices are great for
in game character voices.

I first trialed Replica Studios A.I. Voice Actor Text To Speech (TTS) voices shortly after they launched about a year ago and was suitably impressed by how natural they sounded. Some voices better than others but most sounding very close to human with the right inflections at the appropriate moments. Since then they have been working on improvements and adding more voices to their voice cast library.

I have used Replica Studios on projects in the past, most notably on my 2021 Reallusion, Lip Sync Contest entry featuring my character, Bat Storm. 

Mia and the Tourist and their R2-D2 replica droid.
Mia and the Tourist.

This time I've decided to use Replica to cast the other two voices in my upcoming Mia and the Tourist animated short that I've been blogging the process of creating through demonstrating the apps I'm using to make it.

So far I've covered scriptwriting, storyboarding, background creation, and character rigging.

When I say the other two voices I'm referring to Mia and the Car Rental Manager (see my post on scriptwriting if you want more details of this animated short). The Tourist is loosely based on myself so I typically use my normal speaking voice for this character.

Generating Replica Voices

Replica itself is a desktop app that connects to their server. Once installed and logged in you can begin creating voices right away. If you're using  iClone, Unreal Engine or Unity there are integrations available that let you use Replica directly from those apps.

Initially you start with 30 minutes of free credit and you can buy more from the app as needed. You can also earn credit by referring friends to the app with your unique link (as I've done with my link at the beginning of this article).

There are two ways to proceed in the app, the Sandbox, where you can generate individuals lines of dialogue and download as needed, or Projects, where you can enter an entire script worth of dialogue and download all at once. Unlike the Sandbox you can save multiple projects here too.

The Sandbox

Replica Studio's Sandbox interface. Change attributes like Speech Rate, Pitch, and the Style.
Replica Studio's Sandbox interface.
Change attributes like Speech Rate, Pitch,
and the Style.

The Sandbox is about as simple as it gets. Select a voice, type your dialogue, click Play to preview. Select any part of the text to make further adjustments like adding silence, volume, or changing the speech rate, pitch, or style (when options are available).

Saving a take will count towards your time credit so it's best to save only those takes you intend to download.

The sandbox will let you save takes with different character voices and dialogue so it is possible to generate all the voices you might need for a short project here.

When you have all your takes just download each individually with their download button.

Projects

Replica Projects Section helps keep your script organised and easy to enter.
Replica Projects Section helps keep your
script organised and easy to enter.

The Project section is a new addition to the app since I first reviewed it. At the individual line level it works the same as the sandbox but above that it allows you to pick your character voices and assign them to characters in your script. You can then enter your script broken up into its scenes.

The interface for entering dialogue is more of a list so you can work quickly, see an overview of key settings and make changes to text and character voices without having to get down to individual line level (you only need to go that deep if you want to play around with volume, speech rate, and pitch on individual words and phrases).

Just aware that if you preview the same line of dialogue multiple times it does start using up your credit. So keep an eye on this if your credit is running low.

Other useful features include the ability to move lines of dialogue around, and you can preview an entire scene with the play button at the bottom of the scene window.

Once you have all your dialogue entered you can download each line individually (which is more useful if you want to change a line of dialogue later) or download an entire scene at a time as a zip file in WAV, MP3, OGG, and FLAC formats. Each audio file is named after the line number in the scene.

Replica's Voice Library

New voices are being added to Replica all the time. You can browse the library and check out each voice with the pre-recorded samples just by going to the voice library section of the app. However it's actually easier to browse from the sandbox where, when you go to select a voice, you can filter the available voices with search options like Gender, Age, and Accent etc.

Like any Text To Speech system there will always be some voices better than others. Personally I think Replica still has some way to go before their voices will pass as human in every situation but many of them are good enough to use in situations that don't involve too many emotional extremes.

Adding the Voices to my Storyboard Animatic

As mentioned earlier I used Replica to voice two of my characters. The third is being voiced by me.

I recorded my lines using Audacity. I won't be covering that as part of this series but if you do want to get some insight into how I batch export my dialogue from Audacity into individual audio files for each line of dialogue check out my article, A Better Way to Export Clips From a Single Audio Dialogue Track in Audacity

For the purpose of letting you hear my generated Replica voices I imported my audio into Storyboarder and exported an animatic of my rough sketches (below). I wouldn't usually do this. Normally I'd make an animatic using my characters and sets in Cartoon Animator but now, having seen this, I'm glad I did.

As you can see (and hear) my Replica voices aren't as successful as I'd like. Mia's voice is kind of okay but has some issues with volume and pitch. The Rental Car Manager is all over the place. You can easily tell his voice is text to speech. I may have to try something else for him.

My own voice is fine, I guess, given the Tourist only has three very short lines.

Overall I think Replica's voices are coming along. While there is a good selection of voices not all of them work well as character voices. I really struggled to find a voice for Mia that I liked. The one you hear in the animatic is as close as I could get to what I imagined but it still sounds too youthful.

The next step for my animation is to start putting it altogether in Cartoon Animator.

Comments

Popular posts from this blog

Pricing Animation Services - USD$10,000 Per Minute Might Not Be Enough For an Industry That's Collapsing... Wait, What?

T his post was inspired by an Amateur Animators Facebook Group , public conversation between two professional animators working at entirely different places in their careers but both with at least a decade plus of experience.  Indy Animator Versus Studio Animator The conversation  centered around an original post by an amateur animator asking if USD$115 payment for a four to five minute, frame by frame animation, was a good deal? Anyone who has done even one second of frame by frame animation should at least have some concept of how long it would take to make four to five minutes and know that $115 is definitely not a good deal for most people. However, professional animator one, who I would describe as an independent animator, running a successful content and training business, while not suggesting the deal was good, made the point that there are many factors that go into pricing and that ultimately it is for the individual to set their prices based on what they're happy to work

Inochi2D - Free Open Source 2D VTuber Avatar Rigging and Puppeteering Software (Part 1)

Inochi2D Creator - Free Open Source VTuber Software. If you've been looking for a way to live perform as a 2D cartoon avatar on camera, whether it be for a live stream or for pre-recorded content like educational videos, then VTuber software is a low cost (or even no cost) option worth looking into. In my previous post, How to Become a VTuber - 2D and 3D Software for Creating and Controlling Your Avatar , I took a brief look at the relatively new but completely free and open source Inochi2D  which I thought showed great potential for my own needs of creating a live performance character rig for my own TET Avatar that I use for all my promotional materials. While it is possible to live perform my character using Cartoon Animator itself, Reallusion's MotionLive2D capture system isn't great - with lip sync in particular. More importantly though, I can't exactly teach people how to use Cartoon Animator if I'm using Cartoon Animator to control my Avatar. What is Inochi2D

Wonder Unit Storyboarder - Free Storyboarding Software for People Who Can (or Can't) Draw

Wonder Unit Storyboarder.  As an independent and solo animator I'm always tempted to try and skip storyboarding my animated shorts because they're usually only single scene sketch comedy type jokes. As a result I have many unfinished projects that kind of petered out due to having no clear finishing line. Storyboarding your productions, no matter how small, gives you a step by step guide of every shot that needs to be completed (no planning shots as you animate). It also allows you to create an animatic that gives you a rough preview of the finished production. In short, you shouldn't skip storyboards as they, generally, increase the chance of the project being completed. Disclaimer - I'm Not a Fan of Storyboarder Upfront, Wonder Unit's Storyboarder  is not my preferred storyboarding software. However it's completely free, has a number of very compelling featu

Dollars Mocap: Full Body Webcam Motion Capture (Including Hands and Fingers) For iClone and Cartoon Animator

Even though I should be further away from the camera Dollars Mocap MONO still does a good job of  tracking my arms, hands and fingers. Ever since I wrote my series on becoming a VTuber , discovering it was possible to do full body motion capture, including hands and fingers, with just software and a webcam, I've been on the look out for any motion capture software that can bring that functionality to Cartoon Animator. Dollars Mocap is a low cost motion capture application with a free trial that I learned about through the YouTube Channel Digital Puppets  and their test video . It can record full body, upper body, arms and hands, and facial mocap from a live video source or pre-recorded video. Investigating further, I discovered not only does Dollars Mocap have a free iClone7, iClone8 character profile file download (look for it at the bottom of the main program download page), so you can use the saved motions with iClone8, they've also got a demo video for how to convert your

Six Lip Sync Animation Tips for Cartoon Animator (That Really Apply to Any Auto Lip Syncing Application)

A nimating a character's mouth talking in any form of animation is one of the more time consuming jobs of character animation. Particularly if your project is dialogue heavy or your character has the urge to sing. Fortunately auto lip syncing in many animation software studios is becoming more common and is helping to speed up the lip syncing process. Reallusion's Cartoon Animator is no exception with a fairly good auto lip sync feature. I made a tutorial video demonstrating how to auto and manual lip sync in Cartoon Animator here . However auto lip sync is not perfect (yet) so you'll definitely find yourself manually lip syncing a character at some point. Or maybe you just prefer to manually lip sync yourself because you like that level of control. Whatever the reason here are Six tips to help improve your lip sync and to stop you getting bogged down into really small details that literally do not matter for the amount of screen time they'll get. 1. Auto Lip Sync is NO

The Ultimate Independent Animator's App and Resource List 2019-2023 - Animation and Video Life

Image created with Cartoon Animator 4. Being an independent animator is not like a studio animation job. There's so much more to do that is indirectly related to the actual task of animating. Over the years I've sought out many apps, tools, and services that can help me achieve that one single task, expressing myself through animation. Below is my Ultimate Independent Animator's Resource List for 2019-2022. It started out as a list of free or low cost apps that could help you in every stage of producing either 2D or 3D animation, and then just kind of grew from there. You may not have been looking for a Time Management App as much as you needed something to get you started in 3D animation but when those commissioned projects start coming in you'll have a head start on maximizing your time. All the apps and services on this list had to meet two main criteria: They had to be useful and relevant to an Indy Animator/artist. The base app/service had to be US$

Moho 14 Released - Still the Best 2D Animation Software for Indy Animators on a Budget

Moho 14 Released. Regular readers know I am a Reallusion, Cartoon Animator advocate through and through. Hands down I would recommend Cartoon Animator 5 first over Lost Marble's Moho 14 to anyone who is just starting in 2D animation, is a team of one, or just needs to animate as quickly as possible. However, feature for feature, Moho is, arguably, the best 2D animation software for the rest of us who can't justify a Toon Boom Harmony , or Adobe Creative Cloud subscription (and even with their applications Moho is very competitive on features). You can get started with Moho Debut for just USD$59.99 which is a cut down version of Moho Pro but it still has the most essential features needed for 2D animation. While Moho Pro is a whopping USD$399.99 (Cartoon Animator, which only has one version, is just USD$149.00) upgrades to new version numbers come down to a quarter of the price at USD$99.00. Even though Reallusion just released features like Motion Pilot Puppet Animation and