Skip to main content

LTX Studio (Beta): AI-Powered Visual Storytelling, From Script to Screen in One App.

LTX Studio Resident Dragon Storyboard Panel.
LTX Studio can generate consistent characters across
storyboard panels - even if one character is a dragon!

W
hile text to image, and text to video (and image to video) AI tend to be getting a lot of the press, the real exciting aspect of generative AI implementation is how it can be used to speed up creator workflow.

Being able to realize your creative vision in a shorter length of time can lead to more ambitious projects. Particularly if you're a team of one, with a very limited budget, but you one day dream of creating your own epic animated feature film.

LTX Studio (beta), a new 'all-in-one' AI film making tool, is not going to let you realize that dream from a single text prompt but, by bringing a bunch of generative AI technologies together, the developers have created a one platform workflow that can help anyone rapidly visualize and deliver a story from initial idea to finished film in days rather than weeks (depending upon how ambitious the project is).

Even better, you can try their platform for free, with the main difference between free and paid accounts being the amount of computing seconds per month.

Using LTX Studio

This is the main scene that opens my story. TET, Grrr Dog, Froyd, and Red in the basement.
This is the main scene that opens my Resident Dragon story.
TET, Grrr Dog, Froyd the cat, and Red in the basement.
The characters are all my Cartoon Animator Rigs.
The Script

You can opt to start with a completely blank project, or input an idea you have for a story and LTX's AI will help you turn it into a script. I went the third option of entering an existing script from episode two of my Resident Dragon Series.

The episodes are short and, since one character is a dragon, I wanted to see if LTX could handle a character that wasn't human, and not living in its natural environment.

Casting

LTX auto generates a cast of character images based upon your script. If you don't like these characters you can edit their text description and regenerate them. 

For example, in a lot of styles Red the dragon was coming up as a blue dragon so I changed the description to make red more prominent as a color.

Each character is also assigned an AI generated voice that you can change if you feel it doesn't quite suit.

You can also add additional characters and select the video style for your project from the cast page too. While there are many different styles there aren't too many for 2D animation. I went with Comic Book Style and used one of my Resident Dragon gag cartoons (shown above) to further influence the art style.

LTX Studio - Cast Page Feature the characters it generated for my script.
LTX Studio's Cast Page. Initially the AI only generated the first four characters. I added
Froyd and Grrr Dog. While you can't use your own art to influence character generation
it is possible to do face swaps by uploading your own images. I wasn't game to try it
because I was relatively happy with these generations.

The Breakdown

I was quite impressed by the next step, the Breakdown. Here the AI generates a synopsis of your script as well as a summary description of each scene. You can edit these descriptions (and even add new scenes) but I was more than happy with what the AI produced. 

LTX Studio's Synopsis for Episode 2 of Resident Dragon
This is the completely unedited AI synopsis of my script which very accurately sums
up the key events of the episode. Each of my three scenes had a similar scene summary.

Once you're happy with the script breakdown, hit the 'start' button and the AI will generate a complete storyboard. While it's not always one hundred percent consistent with keeping the character likenesses the same across storyboard panels it is pretty close. Each frame is editable and can be regenerated if anything just isn't right.

It is possible to upload your own images if you wish, or use the Canvas edit feature, to draw your scenes for the AI to use as a guide for its generations. In both cases you're completely bypassing the style and characters established on the Casting Page - making that step almost redundant.

LTX Studio - Canvas Editor.
I could not get the AI to put my characters in this scene so I tried the Canvas Editor which resulted
in a better composition but not in a style matching my characters or the rest of the storyboard.

Storyboard

While the AI creates a storyboard, it does so based on the scene breakdown, and does not include any of the dialogue. It's up to you to go through each panel, adding the dialogue (and extra panels for it) as you go.

There's also options to add sounds and music as you go as well, but I didn't get to this before my computing time ran out.

I had a lot of trouble generating my storyboard panels to be anything close to what I imagined. It was particularly difficult if I wanted two or more characters in a scene. 

Many times my scene image generations featured a single character that was an amalgamation of the characters mentioned in the scene description. Quite a few times I got Grrr Dog wearing a suit and tie. It happened enough that I dropped Grrr Dog and Froyd the cat from scene they were in even though they're kind of an important to the conversation.

There's a very cool feature that lets you move the camera in a scene (up to a point) if you want to change the composition but I couldn't get my initial scenes to look right to even want to explore angles.

Easier But Frustrating and Disappointing

The storyboard stage is the engine of LTX Studio. I imagine if your project includes standard human characters in a photographic style you'll have more success than I did, with a very specific, animated project.

I don't know why the AI completely ignored all the dialogue in my script to build the initial storyboard? Dialogue is pretty important. Especially when each panel has a place for dialogue, and the ability to choose who is speaking it (as well as previewing their voice).

I had to go through and manually add more scenes with all my dialogue. Also problematic is that only one character can talk per panel. If you have two people in frame (or in my case, one person and a dragon), generating the video for that panel didn't always result in the correct actor speaking the line.

Which I found very frustrating because I wasted a number of generations hoping to get the right actor to speak the lines with no luck. Ultimately I ran out of 'Computing Seconds' for the month and was unable to fully generate my whole storyboard to video. Eight out of 23 panels remained as still shots.

You would think the AI would use the cast images to correctly identify the right character speaking in a scene. It also has the name of the character in the scene to reference as well.

While character consistency across scenes isn't perfect, especially in the comic style, the real downfall is scene consistency. My script only has three locations but in every panel the characters are clearly not in the same room. It may be the same type of room but not the exact same room across panels.

Three panels from the storyboard all taking place in the same room.
Three consecutive panels from my storyboard, all taking place in the same break room
with the two characters standing next to each other... but it's clearly not the same room
in any shot! Background consistency is a real problem.

It's was also next to impossible to reproduce the same, character position, shot and camera angle across panels - something that is essential in conversational style scenes.

There's many more issues I could raise, including the final generated videos of each scene which you have little to no control over even if you describe in the scene prompt what action is happening.

Overall

I really wanted LTX Studio to at least be good and leave me feeling it has potential. I do think it would be great for putting together ideas on the fly, just to see where they lead, because, with  generative AI, the output ultimately ends up being a compromise of what you imagined and what the AI could understand and prioritize from the prompts.

My storyboard and video was unusable for anything. Even to pitch an idea using the automatically generated pitch deck you can generate (if you haven't run out of compute time I guess? Since I wasn't able to do this). 


I'm sure with a little more time I could use the included generative AI tools like remove and infill to kind of kit-bash my scenes but it still wouldn't be close to my vision.

As it stands, once you use up your computing time for the month, you're locked out from actually doing any work. You can view your projects but you can't do anything that requires the AI.

I feel AI is not really there yet for a tool like this. At least not if you have a very specific vision. On the plus side, this is as bad as it's ever going to be. As AI generation improves so will this I'm sure.

What Really Didn't Like...

Commercial use of LTX Studio is only available on the US$175.00 per month business plan. While the developers say they're targeting professionals, it shuts out smaller creators who could really benefit from the accelerated workflow.

It's a deal breaker for me. I'm not paying for anything that doesn't give me commercial rights usage at the lowest tier. I use AI to augment my ideas. At that price point it feels like my ideas are being sold back to me after I've given them to this tool for free. 

By all means charge a premium for the extra computing time and premium tools but don't hold creative's ideas hostage in the process.



o---o--- ---o--- o---

Did you find this article useful?
Subscribe to my newsletter and get the
latest articles delivered to your inbox.

Comments

Popular posts from this blog

Inochi2D - Free Open Source 2D VTuber Avatar Rigging and Puppeteering Software (Part 1)

Inochi2D Creator - Free Open Source VTuber Software. If you've been looking for a way to live perform as a 2D cartoon avatar on camera, whether it be for a live stream or for pre-recorded content like educational videos, then VTuber software is a low cost (or even no cost) option worth looking into. In my previous post, How to Become a VTuber - 2D and 3D Software for Creating and Controlling Your Avatar , I took a brief look at the relatively new but completely free and open source Inochi2D  which I thought showed great potential for my own needs of creating a live performance character rig for my own TET Avatar that I use for all my promotional materials. While it is possible to live perform my character using Cartoon Animator itself, Reallusion's MotionLive2D capture system isn't great - with lip sync in particular. More importantly though, I can't exactly teach people how to use Cartoon Animator if I'm using Cartoon Animator to control my Avatar. What is Inochi2D

Wonder Unit Storyboarder - Free Storyboarding Software for People Who Can (or Can't) Draw

Wonder Unit Storyboarder.  As an independent and solo animator I'm always tempted to try and skip storyboarding my animated shorts because they're usually only single scene sketch comedy type jokes. As a result I have many unfinished projects that kind of petered out due to having no clear finishing line. Storyboarding your productions, no matter how small, gives you a step by step guide of every shot that needs to be completed (no planning shots as you animate). It also allows you to create an animatic that gives you a rough preview of the finished production. In short, you shouldn't skip storyboards as they, generally, increase the chance of the project being completed. Disclaimer - I'm Not a Fan of Storyboarder Upfront, Wonder Unit's Storyboarder  is not my preferred storyboarding software. However it's completely free, has a number of very compelling featu

Glif (Alpha) - Make Your Own Tiny AI Powered Niche Image Generator Apps

I  first heard about Glif through a YouTube video that mentioned you could get access to Flux Pro (the latest 'game changer AI' generative image model) through the site for free. While I had a vague notion from the video of what Glif was, I wasn't expecting it to be so easy to get started with, and so good with my very first results. Glif is an easy to use, low-code platform for creating tiny AI-powered generators called Glifs. While that may not sound inspiring, what Glifs allow you to do is create a tiny app that niches down to a specific type of AI generation that the user modifies with their own inputs. The best way to really understand is with an example.  My First Glif I've recently been using VivaGo's AI platform  (free and unlimited at the time of writing) to consistently generate full body characters in a front facing T-Pose that I can rig as front facing characters in Cartoon Animator. Unfortunately it can be a bit hit or miss maintaining the T-Pose part

Krita AI Diffusion - Generative Image AI For Krita is Seriously Useful, Powerful and Free (If You Can Install it Locally)

Generative AI sequence of a woman in a business suit. From sketch to refined image using Krita AI Diffusion - by TET G enerative image AI, where you describe an image with a text prompt to an Artificial Intelligence model and it produces a new image based on your prompt, is gaining a strong hold as a tool for many artists. Krita AI Diffusion brings generative AI image tools right into your favourite free and opensource, graphics editor, Krita. Not only that, if you have a computer with decent specs (and at least 10GB of hard drive space), Krita AI Diffusion is completely free. What If I Don't Have a Powerful Computer? If you're in my situation, with a computer that was around before anyone in the mainstream had even heard of generative AI, you can still access Krita AI Diffusion for free, using a cloud based AI server, Interstice  and 300 tokens, to get you started. Once your initial tokens run out, purchase 5000 more for 10€ (approx US$11.00). Tokens never expire. I would

The Ultimate Independent Animator's App and Resource List 2019-2023 - Animation and Video Life

Image created with Cartoon Animator 4. Being an independent animator is not like a studio animation job. There's so much more to do that is indirectly related to the actual task of animating. Over the years I've sought out many apps, tools, and services that can help me achieve that one single task, expressing myself through animation. Below is my Ultimate Independent Animator's Resource List for 2019-2022. It started out as a list of free or low cost apps that could help you in every stage of producing either 2D or 3D animation, and then just kind of grew from there. You may not have been looking for a Time Management App as much as you needed something to get you started in 3D animation but when those commissioned projects start coming in you'll have a head start on maximizing your time. All the apps and services on this list had to meet two main criteria: They had to be useful and relevant to an Indy Animator/artist. The base app/service had to be US$

Dollars Mocap: Full Body Webcam Motion Capture (Including Hands and Fingers) For iClone and Cartoon Animator

Even though I should be further away from the camera Dollars Mocap MONO still does a good job of  tracking my arms, hands and fingers. Ever since I wrote my series on becoming a VTuber , discovering it was possible to do full body motion capture, including hands and fingers, with just software and a webcam, I've been on the look out for any motion capture software that can bring that functionality to Cartoon Animator. Dollars Mocap is a low cost motion capture application with a free trial that I learned about through the YouTube Channel Digital Puppets  and their test video . It can record full body, upper body, arms and hands, and facial mocap from a live video source or pre-recorded video. Investigating further, I discovered not only does Dollars Mocap have a free iClone7, iClone8 character profile file download (look for it at the bottom of the main program download page), so you can use the saved motions with iClone8, they've also got a demo video for how to convert your

Review: Headshot Plugin for Reallusion's Character Creator 3

Headshot for CC3. Quite possibly the best 3D Avatar I've made of myself in any 3D application. Creating a realistic 3D human avatar is a whole lot easier with Reallusion's new Headshot Plugin for Character Creator 3. The plugin is an AI powered extension that can generate 3D digital humans from one photo. Which sounds like an amazing proposition but, in practice, if you're trying to achieve a specific likeness to an actual person, Headshot will give you an excellent base to work from. Headshot has two modes, Auto and Pro. Auto Mode Auto is well worth a try if you have an ideal photo of a front facing person that is properly lit and posed to Headshot's optimum requirements. It's also the only mode that will take a crack at generating a hair model. I grabbed an image of Harrison Ford, dragged it into Headshot without changing any of the default settings (other than specifying 'male' and selecting an 'old male' setting) and this is what I