Skip to main content

Revisiting Replica Studios - The Most Natural A.I. Text To Speech Based on Real Human Voices

Replica AI Voices Hero Website Image.
Replica A.I. Voices are great for
in game character voices.

I first trialed Replica Studios A.I. Voice Actor Text To Speech (TTS) voices shortly after they launched about a year ago and was suitably impressed by how natural they sounded. Some voices better than others but most sounding very close to human with the right inflections at the appropriate moments. Since then they have been working on improvements and adding more voices to their voice cast library.

I have used Replica Studios on projects in the past, most notably on my 2021 Reallusion, Lip Sync Contest entry featuring my character, Bat Storm. 

Mia and the Tourist and their R2-D2 replica droid.
Mia and the Tourist.

This time I've decided to use Replica to cast the other two voices in my upcoming Mia and the Tourist animated short that I've been blogging the process of creating through demonstrating the apps I'm using to make it.

So far I've covered scriptwriting, storyboarding, background creation, and character rigging.

When I say the other two voices I'm referring to Mia and the Car Rental Manager (see my post on scriptwriting if you want more details of this animated short). The Tourist is loosely based on myself so I typically use my normal speaking voice for this character.

Generating Replica Voices

Replica itself is a desktop app that connects to their server. Once installed and logged in you can begin creating voices right away. If you're using  iClone, Unreal Engine or Unity there are integrations available that let you use Replica directly from those apps.

Initially you start with 30 minutes of free credit and you can buy more from the app as needed. You can also earn credit by referring friends to the app with your unique link (as I've done with my link at the beginning of this article).

There are two ways to proceed in the app, the Sandbox, where you can generate individuals lines of dialogue and download as needed, or Projects, where you can enter an entire script worth of dialogue and download all at once. Unlike the Sandbox you can save multiple projects here too.

The Sandbox

Replica Studio's Sandbox interface. Change attributes like Speech Rate, Pitch, and the Style.
Replica Studio's Sandbox interface.
Change attributes like Speech Rate, Pitch,
and the Style.

The Sandbox is about as simple as it gets. Select a voice, type your dialogue, click Play to preview. Select any part of the text to make further adjustments like adding silence, volume, or changing the speech rate, pitch, or style (when options are available).

Saving a take will count towards your time credit so it's best to save only those takes you intend to download.

The sandbox will let you save takes with different character voices and dialogue so it is possible to generate all the voices you might need for a short project here.

When you have all your takes just download each individually with their download button.

Projects

Replica Projects Section helps keep your script organised and easy to enter.
Replica Projects Section helps keep your
script organised and easy to enter.

The Project section is a new addition to the app since I first reviewed it. At the individual line level it works the same as the sandbox but above that it allows you to pick your character voices and assign them to characters in your script. You can then enter your script broken up into its scenes.

The interface for entering dialogue is more of a list so you can work quickly, see an overview of key settings and make changes to text and character voices without having to get down to individual line level (you only need to go that deep if you want to play around with volume, speech rate, and pitch on individual words and phrases).

Just aware that if you preview the same line of dialogue multiple times it does start using up your credit. So keep an eye on this if your credit is running low.

Other useful features include the ability to move lines of dialogue around, and you can preview an entire scene with the play button at the bottom of the scene window.

Once you have all your dialogue entered you can download each line individually (which is more useful if you want to change a line of dialogue later) or download an entire scene at a time as a zip file in WAV, MP3, OGG, and FLAC formats. Each audio file is named after the line number in the scene.

Replica's Voice Library

New voices are being added to Replica all the time. You can browse the library and check out each voice with the pre-recorded samples just by going to the voice library section of the app. However it's actually easier to browse from the sandbox where, when you go to select a voice, you can filter the available voices with search options like Gender, Age, and Accent etc.

Like any Text To Speech system there will always be some voices better than others. Personally I think Replica still has some way to go before their voices will pass as human in every situation but many of them are good enough to use in situations that don't involve too many emotional extremes.

Adding the Voices to my Storyboard Animatic

As mentioned earlier I used Replica to voice two of my characters. The third is being voiced by me.

I recorded my lines using Audacity. I won't be covering that as part of this series but if you do want to get some insight into how I batch export my dialogue from Audacity into individual audio files for each line of dialogue check out my article, A Better Way to Export Clips From a Single Audio Dialogue Track in Audacity

For the purpose of letting you hear my generated Replica voices I imported my audio into Storyboarder and exported an animatic of my rough sketches (below). I wouldn't usually do this. Normally I'd make an animatic using my characters and sets in Cartoon Animator but now, having seen this, I'm glad I did.

As you can see (and hear) my Replica voices aren't as successful as I'd like. Mia's voice is kind of okay but has some issues with volume and pitch. The Rental Car Manager is all over the place. You can easily tell his voice is text to speech. I may have to try something else for him.

My own voice is fine, I guess, given the Tourist only has three very short lines.

Overall I think Replica's voices are coming along. While there is a good selection of voices not all of them work well as character voices. I really struggled to find a voice for Mia that I liked. The one you hear in the animatic is as close as I could get to what I imagined but it still sounds too youthful.

The next step for my animation is to start putting it altogether in Cartoon Animator.

Comments

Popular posts from this blog

Featured Animator: Christian Haynes - 'Zack In Time' An Original, Independent, Animated Series on the Rise

Christian Haynes - Zack In Time.  If you've ever wanted to create an animated TV series staring your own original characters and stories then Los Angeles based writer, director, and animator, Christian Haynes is taking those next steps of putting together a team, developing a pitch/trailer for their series, Zack In Time. Featuring professional studio quality animation, they hope the show will get picked up by an animation studio for an official series. The path he and his team are taking is one you could easily follow as they deal with real life commitments, and building a following on Instagram and Tik-Tok showcasing their work behind the scenes. TET: Tell me a little about yourself. Who you are, and why you started animating? My name is Christian Haynes and I've loved animation ever since I was a kid. I would constantly be drawing cartoon characters from TV shows and movies and making my own little homemade comic strips.  As I got older, I became a lot more interested in st

Wonder Unit Storyboarder - Free Storyboarding Software for People Who Can (or Can't) Draw

Wonder Unit Storyboarder.  As an independent and solo animator I'm always tempted to try and skip storyboarding my animated shorts because they're usually only single scene sketch comedy type jokes. As a result I have many unfinished projects that kind of petered out due to having no clear finishing line. Storyboarding your productions, no matter how small, gives you a step by step guide of every shot that needs to be completed (no planning shots as you animate). It also allows you to create an animatic that gives you a rough preview of the finished production. In short, you shouldn't skip storyboards as they, generally, increase the chance of the project being completed. Disclaimer - I'm Not a Fan of Storyboarder Upfront, Wonder Unit's Storyboarder  is not my preferred storyboarding software. However it's completely free, has a number of very compelling featu

Shotcut - Free Open Source Video Editor for Windows, Mac, and Linux

Shotcut Open Source Video Editor. I've been on the hunt for a while now for the best, free, open source, video editing application out there. In Shotcut , by Meltytech , which has versions for Windows, Mac, and Linux, I think I may have found a real front runner. This won't be a feature filled review, rather it will be my first impressions after having used Shotcut on a few of my YouTube videos so far. One of my key criteria for a video editor is the ability to import any video format directly into the project. This may seem like an odd focus initially but having convert video to something your video editor can use is annoyingly time consuming, and it creates a new generation of footage, potentially with a loss in quality if you don't really know much about video format specs (that's this guy right here!). Shotcut will happily work with my OBS recordings (.FLV), and .MOV, .MP4 files that I get from two different cameras. Not only that but Shotcut doesn't hold me up

How to Use Plask and Reallusion's 3DXchange to Create Full Body 3D Motion Capture Animation for Cartoon Animator

Last month I reviewed Plask , a free, browser based app that allows you to create 3D motion capture files for animation from a webcam or prerecorded video footage. At the time my hope was that Plask could be used to create 3D motion capture files for Cartoon Animator 4. Unfortunately my knowledge of how to export 3D animation motion files between applications is fairly basic and I wasn't able to figure it out. However, thanks to 3D animation enthusiast and game developer, Freedom (of the YouTube Channel Freedom Arts ), who published a tutorial on how to use Plask with iClone7 characters (in my review I wasn't able to work that out either) there is now a workflow from Plask, via Reallusion's  3DXchange , to Cartoon Animator that is relatively easy to follow and works well. Note: If you want to try this out, 3DXchange is available as a free 30 day trial download if you don't have it. Creating Your 3D Motion Capture File in Plask I'm not going to do a detailed run

Create 2D Animated Characters with 3D Character Creator Tools and Artistic Filters

3D CC3 TET Character, based on my Oppa Doll Avatar, used as the base for a 2D CA4 character. One of my favorite things to do is to create characters with any type of Avatar/Character creator app. In fact the first test I usually try with these tools is, can I make an avatar of me (or at least my The Extraordinary Tourist persona). Previously I've used 2D character creators like Oppa Doll as a source of artwork for some characters I've made for Reallusion's Cartoon Animator 4 animation studio but 2D character creators are limited to... well 2D. 3D Character Creators While you may think 3D character creators have no place in creating characters for 2D animation there are quite a number of advantages including: No drawing skills required. Render characters in any style. Use an art filter or even hand trace into line art. Need to animate the character in a specific pose or angle? Render out an image of the character in the required angle and animate it in 2D. Quicker to crea

KIT Scenarist - Free, Open Source, Screenwriting Software that Helps Research Your Ideas Too

KIT Scenarist Script Writing Software's Mascot, Alexander Cat. While you can write a script in any word processing app, if you're writing stories (screenplays) that feature characters and dialogue, a dedicated script writing app can save a lot of time formatting, letting you focus more on the actual story. Script writing apps are also very useful if you plan to send your screenplays out to production companies, or if you're collaborating with actors and other production people, who are used to scripts being in a particular standard format.  [Note: In case you're wondering there are reasons scripts follow a standard format and are always written in Courier (typewriter) font, including but not limited to; being easy to read by actors, plenty of space for notes, and the general rule that one page of a script (in this format) equals approximately one minute of screen time.] KIT Scenarist , in my opinion, is one of the best script writing apps out there for ease of use, simp

Artbreeder - Using AI created Character and Background Content in your Animations

A selection of User/AI generated images from Artbreeder. If you're looking for an endless supply of 2D character and background images for your animations then Artbreeder , an online Artificial Intelligence (AI) that generates image mash-ups you can tweak as much as you like, could be the ultimate content library. What is Artbreeder? Artbreeder is free to use though there are various paid plans, that give you additional features, such as higher resolution download images or more settings to play with. All images created on the site are Public Domain (CC0 License) and can be used in commercial projects. Using Artbreeder's online app you can generate head shot portraits, full body characters, landscapes, and other scenes simply by choosing two or more existing images to mash together then, using a series of sliders, to select which traits from each image you wish to lean toward in the final image. Photo Comparison - Top is my original uploaded photo. Bottom is Artbreeder's ap