-
Notifications
You must be signed in to change notification settings - Fork 304
A Future of Visual Storytelling
In the future, anyone with a story idea will be able to develop and visualize their story without technical limitation. They'll be able to direct a visual story by themselves. They'll be able to collaborate real time, building virtual scenes with people next to them or in different countries. They'll be able to iterate freely and easily to make their stories better.
In the future, anyone with a set of storyboards will be able to pick up a camera and start shooting. They won't need to find or build real sets. They won't need to be in the same country as the performers. As they shoot each shot, the scene will be roughly edited together, in real time. The technology will become invisible. They will only need to worry about the story and the performance.
In the future, any person with a story idea will be able to produce their stories into full reality. They won't be restricted by where they came from or who they are now, but rather, empowered because of where they came from and who they are now. They won't worry about money or possessing technology. They will have a place to create stories, find others to work with, and find audiences that connect with their work. They will be free to play and explore creative story ideas ultimately creating visual stories with their most valuable asset: their unique point of view.
These are a collection of my dumb ideas of a very near future of visual storytelling concepts, processes, and technologies that I'm thinking about. I'm not trying to push anything on anyone. I want everyone to have their own idea of the future so we can have conversations and an exchange of ideas. I'm writing this mostly to organize my thoughts, but also to get feedback from you!
"As far as the future, most of it is already here, it's just not very evenly distributed." – Abraham Lincoln.
Steal the future from everywhere you see it. I'm personally live-action focused, and I'm incredibly surprised live-action production hasn't stolen simple genius ideas from animation and theatrical productions. They don't even dare to steal processes from videogame development, car manufacturing, university research, or tinkerers on youtube.
All the concepts talked about are in place by some people already or in different contexts. As far as technology, most of it exists already, it might just be inaccessable. I want to try to make the concepts and technology accessable to most people if possible.
Stories humanize people. We need to find the humanity in other people's viewpoints. People's stories help people understand each other.
Visual stories can create an immediate emotional impact on a viewer in a way that no other artform can.
Visual Stories are ideas conveyed through sequential images or even a single image. In almost every case, the story involves the imagery of people or personified objects that immediately have an emotional effect on the viewer and do not require translation. Each sequential image progresses the idea.
Examples of visual stories are: animated or live action: movies, tv shows, commercials, webisodes, tiktoks, graphic novels, comics, slide shows, or even a single image. The examples vary in length, format, consumption device, subject matter, production method - but at the highest level, they are images shown in order to convey ideas.
Are sounds/music visual stories? Absolutely. In many cases just as emotionally moving as a the visuals.
The story is the most important part of visual storytelling. Does that statement sound as basic to you as it feels for me to write it? I hope so, because it should be plain as day.
Making stories is hard. It has been the same level hard for the past 10,000 years. It will be the same level hard 10,000 years into the future. Technology may help with minor busywork annoyances, but the ability to contrive a story from nothing, infuse it with love and soul, solve it's infinite potential issues, and economically have an emotional impact on the viewer, all while seeming true and not contrived is a human miracle.
Certainly, you've seen a big budget movie where it was technically astounding, but the story just sucked. What's the point of burning tons of production cash if the story sucks? Why not wait until you have fully developed the story?
Probably because making stories is hard, and people are lazy.
Therefore, the development of the story should be the greatest interest of visual storytellers all the way through production, and even after it.
If we consume visual stories as images, why would a writer use words to describe images to synthesize a story? Why not develop a story visually? Instead of writing with words, write with images.
It's easy to understand why we write with words. We use collections of words as symbols to create ideas. We can speak and listen to words to communicate ideas. We can write words down and read them. We can easily mass distribute written words. Words are cheap and effective. But they are also lossy symbols. They can never exactly convey the true meaning and feeling of the writer. They require the writer to translate the idea to words, and the reader to translate the words to the idea. Those are two translations where the original idea can be confused.
In animation, specifically at a company like Pixar, writers do, in fact, write with images. "Story Artists" draw sequences of images to convey stories. Storyboards. This is an incredibly effective way of conveying a story because the hand drawn images are very close to the desired end product. Story artists can visably explain ideas understood immediately, which would be very difficult to write in words. Image sequences also demonstrate timing, pacing and tone, which is often near impossible to do in words.
The downside of drawing images to synthesize stories is that you need to be able to draw. However, I have been assured by many story artists that the important component of "Story Artist" is "Story". A great stoyteller who has little drawing ability can put together stick figure drawings and is able to move a viewer emotionally. Conversely, a technically great illustrator with no clear idea of story often just makes hollow pretty pictures.
Therefore, I believe great visual storytellers should be able to visually write without having the ability to draw. This is why we built Storyboarder. Initially, Storyboarder was built as a conventional storyboarding tool centered around illustration. Yet, computers have a great ability to render 3D scenes of environments, objects, and characters. So we built a tool that allows people to easily create scenes, pose characters and set up cameras so they can easily create images to tell stories.
Furthermore, you can put on a VR headset to create and manipulate the whole scene just as you would in a real set. It essentially allows you to write and direct a story on the fly.
Editing is very easy by going back into a scene and making changes. There isn't a sense of loss of throwing away a drawing – there is a sense of productivity and creativity by incrementally making things better.
https://www.youtube.com/watch?v=31CdxnXAxiQ
"Film stories are still really rooted in theater. You're frequently stuck doing things that you could do much more poetically. The story has to be written for it and I'm not quite sure who would do it because writers don't write visual things." – Stanley Kubrick
With modern creative tools, it's now possible for writers to write visual things.
So at this point, there is no reason why a visual storyteller should not be writing visually using a tool like Storyboarder or any other tool that works for them.
All of the most creative things I've ever done have been by accident. I was trying something I didn't need to do because: why not? I was playing. With curiousity and wonder you take little low cost risks to see what will happen. You learn new things. Sometimes those things are fantastic. Sometimes you are the first person to discover something and often the first to discover it from your particular point of view.
Keep the costs low and the creative risks high.
When creating or doing something takes a lot of effort or has a big expense, we don't take small risks. We make plans. We make plans to make plans. We often spend more time trying to mitigate the potential disasters of the plan than actually executing. This limits happy accidents. This limits risk taking. This limits play.
It's wierd and sad how people don't play as they get older. Maybe it's because our bodies work slower. Maybe it's because we think we've already discovered everything. Maybe it's because we don't even remember the joy of discovery.
Is it possible to live in the world of your story? Is it possible to act the scene out?
When Walt Disney was developing the story of Snow White, he would grab a random employee from the halls and bring them to the theater. He would act out the full story of Snow White from his head. Every time was a little different. He would try different ideas and often improvise new ideas on the spot. He did this hundreds of times, coming up with tons of new ideas.
Why not play the story out hundreds of times? Improvise new ideas because: why not?
When the cost and ability to try new ideas is low, and the technology and tooling blend into the background, you can reach a flow state. You are hyperfocused on creation. You can try new things and you get immediate feedback. New ideas give you new ideas. The instant you have to wait for something, this is broken.
When creation feels like a game, the stakes are low. The barrier is low. The risks are easy to take. The art is better.
Creativity is reactive. No one synthesizes a completely original idea without prompt. The conversations between people are people reacting to each others ideas. A question is asked for clarification. A person responds with a new thought they'd never thought of before. Another person brings a fact or prior art into the conversation sparking new ideas.
Conversation and collaboration between people is an excellent tool for synthesizing new ideas.
Yet, recent story development is thought of as a solitary act. The sole writer creates a unique, idiosyncratic story with their own take on dramatic conflict.
A strong, unique point of view is absolutely necessary in a story. But why not benefit from multiple minds of similar style, interest and taste, each bringing their own strengths to the development of the story? With a strong central story director, the story can maintain the point of view, while incorporating the very best input from each writer.
Additionally, collaborators don't need to be in the same place. In fact, it seems absurd to find creative, unique, talented people (which is near impossible), expecting they will all live in the same city. We have tools to coexist in virtual worlds together, walking around, playing, and creating together. You can do this within Storyboarder.
Music and thoughtful sound design have a huge impact on how people feel when they watch a visual story. Music can set the mood and tone through the usage of well known cliches. In fact, in the absence of a meaningful story, music is the only hope to keep that visual turd from sinking into the dark abyss of irrelevance.
Some of the most beautiful stories are told through music. Some of the greatest memories are tied to iconic original themes.
So why not develop a story with music as a fundamental writing tool in the first place?
Iteration is easily the most important concept around the future of visual storytelling. I buried it down here because most people aren't even reading this far. But you are. Hey. Isn't this nice? Let's just take a moment to appreciate each other. Thanks for reading this. Anyways...
No creation is great right out of the gate. It is made better over time. You listen to feedback. You change things. You try new things or gain a new perspective. You change things. Every iteration makes the work better.
Iteration is how stories have historically developed. Myths would be created by someone and told to a group of people. People would retell the stories, keeping the best aspects of the story and losing the worst parts. Every time it was retold, it would be slightly different, each time, the story would be iterated on. The bible is a great example of a collection of prototypical myths iterating over several thousand years. All religious mythology and folklore is the result of this.
In videogame development, no game is just made and released to the public. When a game designer has a novel idea for a game, they are feeling pretty good that they are on to something. They make a prototype. They make something people can play and receive feedback on. This isn't done two or three times. This is done thousands of times. When the game is ready for release, perhaps a kernel of the vision is still intact, but the resulting game is very different from the way they originally intended. Even after release, changes are being made to improve the game.
In feature-length narrative screenwriting, there is some sort of arbitrary "A draft and two revisions" rule of thumb. It seems to be centered around worker protectionism more than the work itself. But who really gives a shit about how screenwriting is typically done? Mythology has been iterating on iconic stories for 10,000 years. Screenwriting has only existed for roughly a hundred years and has produced only a handful of rare single hit successes.
Yet in animation, particularly at Pixar, stories are heavily iterated on. Animation is much more expensive per minute to produce than live-action. Therefore, if you are going to spend a ton on production, you should invest in the story you are trying to produce. Stories at Pixar start with the kernel of an idea. The story is workshopped. Scenes are written by story artists that produce visual sequences to pitch story ideas to the team and the director. Because these hand drawn sequences can be edited into full versions of a movie, Pixar can show that version to test audiences and their trusted panel of storytellers for feedback. The feedback is taken and incorporated into revisions of sequences, or even wholly new ideas. This makes the stories better. Pixar stories are developed over the course of two years. Their story and commercial hit rate is nearly 100%.
Even though Pixar literally invented computer graphics, their most valuable innovation has been how they iterate on story. I don't know why every storyteller or entertainment studio hasn't stolen their process. I think it is because Pixar originated as a group of creative software engineers. The value of technological iteration is burned into their culture. Whereas most other creative executives still cling to the golden touch fallacy.
But iteration is hard particularly in visual storytelling because the costs are so high. Do you have 10 story artists at your disposal to visualize your story concepts? I don't. Furthermore, changes to stories have ripple effects. If you change a fundamental value of a character in the story, you have to make sure that change is reflected in the rest of the story. This problem increases exponentially based on the length of the story. Even worse, artists are resistent to losing their work. Imagine if you just drew a beautiful sequence of 300 images. But, a great change was made to a character that will improve the story. This will nullify your sequence and it must be redone.
This is why we built Storyboarder. We wanted scenes and shots to be easily changed so that contrary to feeling loss while iterating story, you would feel a sense of improvement. A creative tool like Storyboarder will reduce the change costs so that iteration is pain free.
Also, because the storyboard sequences generated with Storyboarder are very similar to the desired output (a visual story), full sequences of the entire visual story can be presented to test audiences for feedback. This can be done hundreds or thousands of times.
Pre-production is the stage where the story has been developed enough that it makes sense to move forward and prepare for production. However, this doesn't mean the story is done. Ideally, through the process working with the cast, workshopping the story, further testing, the story will further develop prior to production. Through the step of proto-production, the story will be iterated on at least one more time.
There are a lot of great actors few people have seen yet. If your story is great, you do not need a well-known actor. If the cost of your production is low, you do not need a well-known actor to sell your story to viewers.
Culturally, you want your actors on your team. You want them fully invested in the story. You want them to make your story ring true. You want them to be able to iterate on the story with you. You want them to make the story better.
This will take more work than usually required of a typical narrative story. It's important that they are willing to do this.
A proto-production is a low cost prototype of a visual story. In software or game development terms, it would be like a beta test.
Shoot the movie before shooting the movie.
The objective is to shoot the entire visual story without built up sets, crew, tight time constraints, or high costs. Think: a large warehouse. A couple moveable walls, platforms, tables and chairs. The actors, the writers, and you – with a camera. You'll shoot the whole story there. For months if you need to.
The result is a fully edited visual story that can be viewed by anyone. If the visual story doesn't move your viewers emotionally, it's not because there aren't built sets or fancy visual effects. It's because your story sucks and or your performances are bad.
Wouldn't it suck to find out your story really just isn't working? No one is feeling it. Even you. That's what releasing the first prototype of a game is like. But it's not a bad thing. It's incredibly eye opening and a huge opportunity to make things better.
You've already visually written your story and iterated on it many times. You shouldn't even be shooting a proto-production if you weren't confident in your animatic. So it should be very clear what's not working with this proto-production and what needs to change.
I was talking to a well-known movie actor about the idea of a proto-production. He told me that he actually prefers acting in theater. However, he won't sign on to a production unless he has the time to devote two full months to workshopping the play. When he signed on, he liked the story of the script. He wouldn't have if he didn't. But he describes the first day of workshop as having a feeling that this really isn't going to work. Yet, through everyone's feedback, iterating with the director, the script changes, it becomes really good. And even still, on opening night, there will be a line in the play, that no one in the production ever thought twice about, that gets a huge laugh. And they will change the script to lean into that laugh a bit more. It's an incredibly rewarding experience.
That same actor will be in a movie where he isn't even allowed to read the whole script, but what he is reading seems like trash. He'll ask producers and the directors about it and they will assure him that everything is going to be great. He shows up that day for the shoot. Does his lines. Waits a year. He was right. The story was trash.
On a technical basis, we want to make tools that make much of the technical aspects of production invisible. We want to focus on the story and the performances, but in order to keep costs down, we won't have a crew.
The entire visual story was storyboarded shot for shot in Storyboarder. Each board has tons of metadata: the characters in the shot, what they are doing, where the camera is, what the lens is, etc. So there is a shot list for the whole story.
We built a system called "Shot Core" that imports all of your shots and metadata for the story. The shots can be reordered into a schedule. When you pick up the camera, it shows you the storyboard of the shot you should be shooting. When you record a take, Shot Core logs it and shuttles it automatically off the camera. There's a small device attached to the camera that lets you rate a take or advance to the next shot. As you are shooting a scene, Shot Core automatically assembles a rough edit of the scene in real time. The whole system is designed to keep you on track, organized, and not worrying about anything except the story.
If you want to shoot anything different, you can make impromptu shots. You can even rework the whole scene with no problem.
Shooting to boards
Tyler Perry makes a series of movies starring his character, Madea. The movies are a series of situational comedy sketches targeted to Black audiences. Tyler Perry will randomly show up to local night theaters in Atlanta and try new sketch ideas on stage. If the sketch doesn't do well with the Black audience, he might come back the next night after revising the sketch. And if it kills, that sketch will probably be in the next Madea movie. Tyler Perry is a genius. He iterates on his stories and tests them in near real time.
Show the protoproduction to people get feedback
Prepare base assets and lighting and real props for production
Build virtual sets
Lighting
Reduce costs
You've storyboarded the whole movie, you've shot it at least once already. Production should be the easiest aspect of the entire process.
Technology should melt into the background.
everyone here has been along for the whole ride.
there should be no mercinaries or hired guns. everyone should know the story and want the same thing, to make the best possible representation of the story.
because technology will take care of many parts, there is no need to have a huge crew.
If something is not working storywise, anyone should be able to call a stop to address it. If someone has an idea to make it better, it should be explored.
TPS
I hear a lot that anything that can go wrong, will go wrong. I dont feel that way when I'm using my phone. I don't feel that way when I'm using my computer. I don't feel that way when I'm playing a game. Yet, when people are talking about capturing performances on a technical basis, this seems to be the story. I think it is because the tools that are used are poorly designed and built and relying too much on human error.
Technology should be built that removes technical jobs from people so they can focus on story and performances.
real time compositing
global illumination using disparate led lights
needs to be flexible so that you can change things later
mask people out using infrared
dynamic sets
lights that are remotely controlled
cameras that are remotely controlled
collaborative
repeat movements
autonomous camera movement
dont leave a scene until it's edited
Rotoscoping
Relighting
Face replacement
Performance replacement
Dynamic retiming
Audio style matching
Film is what I wake up with in my mouth before I brush my teeth in the morning. No one makes film anymore except like 10 dudes that aren't dead yet. When I grew up around art fucks in Chicago who went to Columbia College Film School, I would have conversations like, me: "Yo! Did you see that Speed 3 movie this weekend? Bitches are running out of ideas." Him: "Yes. I did indeed see that film. I enjoyed it on a pedestrian level and do wait with anticipation for the next mode of locomotion for their future filmmmm. I went to Columbia." I never seen his punchable face again, so I just assume he jumped off the Sears Tower when Kodak filed for bankruptcy.
©2020 Wonder Unit, Inc. https://wonderunit.com
Storyboarder Help
What is Storyboarder?
Previously Answered Questions
Layers in Storyboarder
Shot Generator
How to use Shot Generator
VR Help
Creating a scene in VR
Creating custom 3D Models for Shot Generator
Creating custom Emotions for Characters in Shot Generator
Development Plan
v1.18 Plan
About Wonder Unit
About Wonder Unit