Skip to main content

Revisiting Replica Studios - The Most Natural A.I. Text To Speech Based on Real Human Voices

Replica AI Voices Hero Website Image.
Replica A.I. Voices are great for
in game character voices.

I first trialed Replica Studios A.I. Voice Actor Text To Speech (TTS) voices shortly after they launched about a year ago and was suitably impressed by how natural they sounded. Some voices better than others but most sounding very close to human with the right inflections at the appropriate moments. Since then they have been working on improvements and adding more voices to their voice cast library.

I have used Replica Studios on projects in the past, most notably on my 2021 Reallusion, Lip Sync Contest entry featuring my character, Bat Storm. 

Mia and the Tourist and their R2-D2 replica droid.
Mia and the Tourist.

This time I've decided to use Replica to cast the other two voices in my upcoming Mia and the Tourist animated short that I've been blogging the process of creating through demonstrating the apps I'm using to make it.

So far I've covered scriptwriting, storyboarding, background creation, and character rigging.

When I say the other two voices I'm referring to Mia and the Car Rental Manager (see my post on scriptwriting if you want more details of this animated short). The Tourist is loosely based on myself so I typically use my normal speaking voice for this character.

Generating Replica Voices

Replica itself is a desktop app that connects to their server. Once installed and logged in you can begin creating voices right away. If you're using  iClone, Unreal Engine or Unity there are integrations available that let you use Replica directly from those apps.

Initially you start with 30 minutes of free credit and you can buy more from the app as needed. You can also earn credit by referring friends to the app with your unique link (as I've done with my link at the beginning of this article).

There are two ways to proceed in the app, the Sandbox, where you can generate individuals lines of dialogue and download as needed, or Projects, where you can enter an entire script worth of dialogue and download all at once. Unlike the Sandbox you can save multiple projects here too.

The Sandbox

Replica Studio's Sandbox interface. Change attributes like Speech Rate, Pitch, and the Style.
Replica Studio's Sandbox interface.
Change attributes like Speech Rate, Pitch,
and the Style.

The Sandbox is about as simple as it gets. Select a voice, type your dialogue, click Play to preview. Select any part of the text to make further adjustments like adding silence, volume, or changing the speech rate, pitch, or style (when options are available).

Saving a take will count towards your time credit so it's best to save only those takes you intend to download.

The sandbox will let you save takes with different character voices and dialogue so it is possible to generate all the voices you might need for a short project here.

When you have all your takes just download each individually with their download button.

Projects

Replica Projects Section helps keep your script organised and easy to enter.
Replica Projects Section helps keep your
script organised and easy to enter.

The Project section is a new addition to the app since I first reviewed it. At the individual line level it works the same as the sandbox but above that it allows you to pick your character voices and assign them to characters in your script. You can then enter your script broken up into its scenes.

The interface for entering dialogue is more of a list so you can work quickly, see an overview of key settings and make changes to text and character voices without having to get down to individual line level (you only need to go that deep if you want to play around with volume, speech rate, and pitch on individual words and phrases).

Just aware that if you preview the same line of dialogue multiple times it does start using up your credit. So keep an eye on this if your credit is running low.

Other useful features include the ability to move lines of dialogue around, and you can preview an entire scene with the play button at the bottom of the scene window.

Once you have all your dialogue entered you can download each line individually (which is more useful if you want to change a line of dialogue later) or download an entire scene at a time as a zip file in WAV, MP3, OGG, and FLAC formats. Each audio file is named after the line number in the scene.

Replica's Voice Library

New voices are being added to Replica all the time. You can browse the library and check out each voice with the pre-recorded samples just by going to the voice library section of the app. However it's actually easier to browse from the sandbox where, when you go to select a voice, you can filter the available voices with search options like Gender, Age, and Accent etc.

Like any Text To Speech system there will always be some voices better than others. Personally I think Replica still has some way to go before their voices will pass as human in every situation but many of them are good enough to use in situations that don't involve too many emotional extremes.

Adding the Voices to my Storyboard Animatic

As mentioned earlier I used Replica to voice two of my characters. The third is being voiced by me.

I recorded my lines using Audacity. I won't be covering that as part of this series but if you do want to get some insight into how I batch export my dialogue from Audacity into individual audio files for each line of dialogue check out my article, A Better Way to Export Clips From a Single Audio Dialogue Track in Audacity

For the purpose of letting you hear my generated Replica voices I imported my audio into Storyboarder and exported an animatic of my rough sketches (below). I wouldn't usually do this. Normally I'd make an animatic using my characters and sets in Cartoon Animator but now, having seen this, I'm glad I did.

As you can see (and hear) my Replica voices aren't as successful as I'd like. Mia's voice is kind of okay but has some issues with volume and pitch. The Rental Car Manager is all over the place. You can easily tell his voice is text to speech. I may have to try something else for him.

My own voice is fine, I guess, given the Tourist only has three very short lines.

Overall I think Replica's voices are coming along. While there is a good selection of voices not all of them work well as character voices. I really struggled to find a voice for Mia that I liked. The one you hear in the animatic is as close as I could get to what I imagined but it still sounds too youthful.

The next step for my animation is to start putting it altogether in Cartoon Animator.

Popular posts from this blog

AE Juice - Animation Presets, Motion Graphics, Templates, Transitions for After Effects, Premiere Pro, and Other Video Applications

Level up you video edits and animations with AE Juice's motion graphics and templates. Some days you just don't have the time to create flashy motion graphics for your latest video or animation. For some of us it's more a question of our own artistic abilities being a little less than the awesome we'd like them to be. Whatever reason a resource like AE Juice's animation presets, motion graphics, templates, and transitions packs for After Effects , Premiere Pro , and other video applications can really make your work stand out very quickly. AE Juice gives you access to an instant library of free, premade content elements and sound effects, which you can add to with additional purchases of various themed packs from their store. There are three ways to manage their content, all of which can be used in commercial projects . The AE Juice Standalone Package Manager makes it easy to browse previews of all your pack contents and to download and find just the elements yo

Can You Learn Reallusion's Cartoon Animator 5 for Free Using Their 137 Official YouTube Video Tutorials Sorted Into a Logical Learning Order?

Or you could just buy The Lazy Animator Beginner's Guide to Cartoon Animator . While Reallusion's Cartoon Animator is one of the easiest 2D animation studios to get up and running with quickly, learning it from all of the official, free, video tutorials can be more overwhelming than helpful. With more than 137 videos totaling more than 28 and a half hours of tutorials, spread across three generations of the software (Cartoon Animator 3 through 5) it's hard to know if what you're learning is a current or legacy feature that you either need to know or can be skipped. Many of the official tutorials only teach specific features of the software and don't relate at all to previous or later tutorials. As a result there are many features either not mentioned or are hard to find. To make your learning easier, on this page, I've collected together all of the essential, official, free video tutorials and sorted them into a learning order that makes sense. Simply start at

TimeBolt: Fast Video Editing for Anyone Creating Online Courses, Podcasts, or Vlogs.

I resisted making tutorial videos for a long time because I don't like editing. Specifically I don't like editing me teaching as I step students through a process during a screen record. I have a tendency to insert long pauses not just in the middle of sentences but between multiple words in the middle of sentences as my pace matches what I'm doing onscreen. This makes for very long and very slow paced video tutorials. To counteract this I have to edit out all the pauses. This can take hours, or even days on particularly long tutorials. For example, when I created my main course, The Lazy Animator Beginner's Guide to Cartoon Animator , I literally injured the thumb on my right hand, operating my mouse, as I spent weeks taking out all my pauses (seriously, I had to wear a thumb brace for a few weeks to fix the pain). Recently I came across TimeBolt , a very affordable, fast editing application with the featured purpose of removing all the pauses from your video (and even

Review: Headshot Plugin for Reallusion's Character Creator 3

Headshot for CC3. Quite possibly the best 3D Avatar I've made of myself in any 3D application. Creating a realistic 3D human avatar is a whole lot easier with Reallusion's new Headshot Plugin for Character Creator 3. The plugin is an AI powered extension that can generate 3D digital humans from one photo. Which sounds like an amazing proposition but, in practice, if you're trying to achieve a specific likeness to an actual person, Headshot will give you an excellent base to work from. Headshot has two modes, Auto and Pro. Auto Mode Auto is well worth a try if you have an ideal photo of a front facing person that is properly lit and posed to Headshot's optimum requirements. It's also the only mode that will take a crack at generating a hair model. I grabbed an image of Harrison Ford, dragged it into Headshot without changing any of the default settings (other than specifying 'male' and selecting an 'old male' setting) and this is what I

Murf.ai - Text to Speech, AI enabled, Real People, Online Voice Generator and Video Studio Editor

Most of us are not talented enough to perform verbal gymnastics with our voices, easily imitating people of different ages, genders, races etc. Which is real problem for solo animators with little to no budget. How do you voice multiple characters in your animations? Developments in human sounding, Artificial Intelligence (AI), Text to Speech (TTS) voices is making them an increasingly viable and affordable alternative to hiring real actors (or convincing family or friends to help out). Murf.AI is an extremely impressive online, AI voice generation service that sounds like real people. Not just a TTS service, their online studio offers a range of tools that include editing and adding AI voices to entire video projects, converting your own recorded audio to an AI voice, and uploading scripts to be converted into AI voices, and more. A selection of the 120+ available voices. Within Murf's studio you get up to 120+ AI voices in 20+ languages, as well as a library of royalty free musi

Make Disney/Pixar Style Characters with Reallusion's Character Creator and Toon Figure Bases

The Extraordinary Tourist Classic Coat outfit created using Reallusion's Toon Designer for CC3. I've talked before how I've wanted to get into 3D Disney/Pixar style character animation since I first saw the animated cutscenes for the very first Tomb Raider game back in 1996. It's why I initially bought Reallusion's iClone 3D studio app as soon as I could afford a computer that would run it. But then Reallusion released their 3D Character Creator (CC) for iClone and I wanted to create my characters with that (and I did try with Bat Storm ). But the focus of CC was realism, even with ToKoMotion's stylised body morphs . Now with Reallusion's Cartoon Designer bundle for CC3 which features two packs, Toon Figures , and Toon Hair , designing Disney/Pixar style 3D characters just got a whole lot quicker. The two packs are the bare essentials for creating Toon style characters. Five body morphs (2 male, 2 female, and one adolescent body morph that w

KIT Scenarist - Free, Open Source, Screenwriting Software that Helps Research Your Ideas Too

KIT Scenarist Script Writing Software's Mascot, Alexander Cat. While you can write a script in any word processing app, if you're writing stories (screenplays) that feature characters and dialogue, a dedicated script writing app can save a lot of time formatting, letting you focus more on the actual story. Script writing apps are also very useful if you plan to send your screenplays out to production companies, or if you're collaborating with actors and other production people, who are used to scripts being in a particular standard format.  [Note: In case you're wondering there are reasons scripts follow a standard format and are always written in Courier (typewriter) font, including but not limited to; being easy to read by actors, plenty of space for notes, and the general rule that one page of a script (in this format) equals approximately one minute of screen time.] KIT Scenarist , in my opinion, is one of the best script writing apps out there for ease of use, simp