Skip to main content

Revisiting Replica Studios - The Most Natural A.I. Text To Speech Based on Real Human Voices

Replica AI Voices Hero Website Image.
Replica A.I. Voices are great for
in game character voices.

I first trialed Replica Studios A.I. Voice Actor Text To Speech (TTS) voices shortly after they launched about a year ago and was suitably impressed by how natural they sounded. Some voices better than others but most sounding very close to human with the right inflections at the appropriate moments. Since then they have been working on improvements and adding more voices to their voice cast library.

I have used Replica Studios on projects in the past, most notably on my 2021 Reallusion, Lip Sync Contest entry featuring my character, Bat Storm. 

Mia and the Tourist and their R2-D2 replica droid.
Mia and the Tourist.

This time I've decided to use Replica to cast the other two voices in my upcoming Mia and the Tourist animated short that I've been blogging the process of creating through demonstrating the apps I'm using to make it.

So far I've covered scriptwriting, storyboarding, background creation, and character rigging.

When I say the other two voices I'm referring to Mia and the Car Rental Manager (see my post on scriptwriting if you want more details of this animated short). The Tourist is loosely based on myself so I typically use my normal speaking voice for this character.

Generating Replica Voices

Replica itself is a desktop app that connects to their server. Once installed and logged in you can begin creating voices right away. If you're using  iClone, Unreal Engine or Unity there are integrations available that let you use Replica directly from those apps.

Initially you start with 30 minutes of free credit and you can buy more from the app as needed. You can also earn credit by referring friends to the app with your unique link (as I've done with my link at the beginning of this article).

There are two ways to proceed in the app, the Sandbox, where you can generate individuals lines of dialogue and download as needed, or Projects, where you can enter an entire script worth of dialogue and download all at once. Unlike the Sandbox you can save multiple projects here too.

The Sandbox

Replica Studio's Sandbox interface. Change attributes like Speech Rate, Pitch, and the Style.
Replica Studio's Sandbox interface.
Change attributes like Speech Rate, Pitch,
and the Style.

The Sandbox is about as simple as it gets. Select a voice, type your dialogue, click Play to preview. Select any part of the text to make further adjustments like adding silence, volume, or changing the speech rate, pitch, or style (when options are available).

Saving a take will count towards your time credit so it's best to save only those takes you intend to download.

The sandbox will let you save takes with different character voices and dialogue so it is possible to generate all the voices you might need for a short project here.

When you have all your takes just download each individually with their download button.

Projects

Replica Projects Section helps keep your script organised and easy to enter.
Replica Projects Section helps keep your
script organised and easy to enter.

The Project section is a new addition to the app since I first reviewed it. At the individual line level it works the same as the sandbox but above that it allows you to pick your character voices and assign them to characters in your script. You can then enter your script broken up into its scenes.

The interface for entering dialogue is more of a list so you can work quickly, see an overview of key settings and make changes to text and character voices without having to get down to individual line level (you only need to go that deep if you want to play around with volume, speech rate, and pitch on individual words and phrases).

Just aware that if you preview the same line of dialogue multiple times it does start using up your credit. So keep an eye on this if your credit is running low.

Other useful features include the ability to move lines of dialogue around, and you can preview an entire scene with the play button at the bottom of the scene window.

Once you have all your dialogue entered you can download each line individually (which is more useful if you want to change a line of dialogue later) or download an entire scene at a time as a zip file in WAV, MP3, OGG, and FLAC formats. Each audio file is named after the line number in the scene.

Replica's Voice Library

New voices are being added to Replica all the time. You can browse the library and check out each voice with the pre-recorded samples just by going to the voice library section of the app. However it's actually easier to browse from the sandbox where, when you go to select a voice, you can filter the available voices with search options like Gender, Age, and Accent etc.

Like any Text To Speech system there will always be some voices better than others. Personally I think Replica still has some way to go before their voices will pass as human in every situation but many of them are good enough to use in situations that don't involve too many emotional extremes.

Adding the Voices to my Storyboard Animatic

As mentioned earlier I used Replica to voice two of my characters. The third is being voiced by me.

I recorded my lines using Audacity. I won't be covering that as part of this series but if you do want to get some insight into how I batch export my dialogue from Audacity into individual audio files for each line of dialogue check out my article, A Better Way to Export Clips From a Single Audio Dialogue Track in Audacity

For the purpose of letting you hear my generated Replica voices I imported my audio into Storyboarder and exported an animatic of my rough sketches (below). I wouldn't usually do this. Normally I'd make an animatic using my characters and sets in Cartoon Animator but now, having seen this, I'm glad I did.

As you can see (and hear) my Replica voices aren't as successful as I'd like. Mia's voice is kind of okay but has some issues with volume and pitch. The Rental Car Manager is all over the place. You can easily tell his voice is text to speech. I may have to try something else for him.

My own voice is fine, I guess, given the Tourist only has three very short lines.

Overall I think Replica's voices are coming along. While there is a good selection of voices not all of them work well as character voices. I really struggled to find a voice for Mia that I liked. The one you hear in the animatic is as close as I could get to what I imagined but it still sounds too youthful.

The next step for my animation is to start putting it altogether in Cartoon Animator.

Popular posts from this blog

Should You Buy or Upgrade to MOHO 13? *Spoiler* Yes. Yes You Should!

MOHO 13's New Character Set. Smith Micro released MOHO 13 , their all in one, 2D animation studio, this week. The question is should you buy or upgrade to the latest version? Obviously I've already spoiled this in the title, so the actual question is why do I think you should buy or upgrade? To be clear, I'm only talking about MOHO 13 Pro. If you're considering MOHO 13 Debut be aware that you're missing out on some of the new features, and a lot of existing features that are only available in the Pro version. Debut is fine if the budget doesn't stretch to Pro, but, if you never want to be disappointed about not having a feature, it's Pro or nothing! The other thing I need to be transparent about is I'm not, by any stretch, a frequent MOHO user/animator. However I took the time to learn MOHO 12 Pro fairly extensively, blogging about my process and sorting out 104 free MOHO training videos into a logical viewing order in the process. I think I ha

AE Juice - Animation Presets, Motion Graphics, Templates, Transitions for After Effects, Premiere Pro, and Other Video Applications

Level up you video edits and animations with AE Juice's motion graphics and templates. Some days you just don't have the time to create flashy motion graphics for your latest video or animation. For some of us it's more a question of our own artistic abilities being a little less than the awesome we'd like them to be. Whatever reason a resource like AE Juice's animation presets, motion graphics, templates, and transitions packs for After Effects , Premiere Pro , and other video applications can really make your work stand out very quickly. AE Juice gives you access to an instant library of free, premade content elements and sound effects, which you can add to with additional purchases of various themed packs from their store. There are three ways to manage their content, all of which can be used in commercial projects . The AE Juice Standalone Package Manager makes it easy to browse previews of all your pack contents and to download and find just the elements yo

Review: CrazyTalk Animator 3 vs Moho Studio Pro 12

Reallusion's CrazyTalk Animator 3 or Smith Micro's Moho Studio Pro 12. Which of these 2D animation applications is right for you? Regular readers of this blog will know I'm a strong supporter, and fairly proficient user of CrazyTalk Animator since version 1. It's a great piece of software for producing 2D animations from purchased content quickly and, with version 3, is easier than ever to create animations from your own art. Lesser known is that I first purchased Moho Studio Pro 12 (then known as Anime Studio Pro 9) back in October of 2012 and have been upgrading it to the latest version ever since because I believed in it as an application for creating great 2D animation to TV quality standard. As such, it's a much more complex application than CTA3 that I only got around to learning properly late last year. I'm still in the process of blogging my progress . Despite this I feel I've learned enough of Moho to compare it to CTA3 to help you determ

Can You Learn Reallusion's Cartoon Animator 5 for Free Using Their 137 Official YouTube Video Tutorials Sorted Into a Logical Learning Order?

Or you could just buy The Lazy Animator Beginner's Guide to Cartoon Animator . While Reallusion's Cartoon Animator is one of the easiest 2D animation studios to get up and running with quickly, learning it from all of the official, free, video tutorials can be more overwhelming than helpful. With more than 137 videos totaling more than 28 and a half hours of tutorials, spread across three generations of the software (Cartoon Animator 3 through 5) it's hard to know if what you're learning is a current or legacy feature that you either need to know or can be skipped. Many of the official tutorials only teach specific features of the software and don't relate at all to previous or later tutorials. As a result there are many features either not mentioned or are hard to find. To make your learning easier, on this page, I've collected together all of the essential, official, free video tutorials and sorted them into a learning order that makes sense. Simply start at

Make Disney/Pixar Style Characters with Reallusion's Character Creator and Toon Figure Bases

The Extraordinary Tourist Classic Coat outfit created using Reallusion's Toon Designer for CC3. I've talked before how I've wanted to get into 3D Disney/Pixar style character animation since I first saw the animated cutscenes for the very first Tomb Raider game back in 1996. It's why I initially bought Reallusion's iClone 3D studio app as soon as I could afford a computer that would run it. But then Reallusion released their 3D Character Creator (CC) for iClone and I wanted to create my characters with that (and I did try with Bat Storm ). But the focus of CC was realism, even with ToKoMotion's stylised body morphs . Now with Reallusion's Cartoon Designer bundle for CC3 which features two packs, Toon Figures , and Toon Hair , designing Disney/Pixar style 3D characters just got a whole lot quicker. The two packs are the bare essentials for creating Toon style characters. Five body morphs (2 male, 2 female, and one adolescent body morph that w

TimeBolt: Fast Video Editing for Anyone Creating Online Courses, Podcasts, or Vlogs.

I resisted making tutorial videos for a long time because I don't like editing. Specifically I don't like editing me teaching as I step students through a process during a screen record. I have a tendency to insert long pauses not just in the middle of sentences but between multiple words in the middle of sentences as my pace matches what I'm doing onscreen. This makes for very long and very slow paced video tutorials. To counteract this I have to edit out all the pauses. This can take hours, or even days on particularly long tutorials. For example, when I created my main course, The Lazy Animator Beginner's Guide to Cartoon Animator , I literally injured the thumb on my right hand, operating my mouse, as I spent weeks taking out all my pauses (seriously, I had to wear a thumb brace for a few weeks to fix the pain). Recently I came across TimeBolt , a very affordable, fast editing application with the featured purpose of removing all the pauses from your video (and even

Creating a G3-360 Head From a Single Photo in Reallusion's Cartoon Animator

Source Photo from Generated Photos . Ever since Reallusion introduced the G3-360 Character Head into Cartoon Animator 4 I've wanted to see if their 360 Head Creator tool could be used to create an animated head using a photo. Part of the reason I've never given this a shot, until now, is that I just assumed it would be difficult, and require a lot of photo editing to blend out the sprite edges. It turns out, creating a photographic G3-360 head is not that much more difficult than creating a cartoon head, and can be done using a single photographic image using my own G3-360 head rigging system . While this article isn't intended to be a full tutorial, I'll run through the basic steps of how I achieved my photographic G3-360 head, shown in the comparison below, of a Cartoon Animator Morph-based head on the left, and my G3-360 head on the right. Pros and Cons Cartoon Animator's morph-based head system is ideal for animating photographic faces. It uses a semi 3D wire me