I describe creating AI-powered films for bedtime stories using ChatGPT and Midjourney. It’s about turning text stories into engaging short films, complete with visuals and music, all automated and versatile for any tale.

Hey there, fellow tech-enthusiasts! Recently, I’ve been totally swept away by the magic of Generative Adversarial Networks (GANs). From the mind-blowing prose generated by ChatGPT to the lifelike images from Midjourney, it’s been an exciting time for the geek in me.

Results Results

So, riding this wave of enthusiasm, a fun idea struck me. Imagine this: Automated short films that narrate stories. Think about those nights when the little ones need a bedtime story, and you’ve run out of tales. Enter my creation - instant, AI-powered bedtime tales.

The timeless tales from the Grimm Brothers became my initial muse. Here’s the magic recipe:

  1. ChatGPT gets a request to weave a story, and it’s crafted segment-wise for ease of adaptation.
  2. These story snippets also come with a specified mood or style.
  3. Armed with the story content and style, I use them as prompts for Midjourney. And voila! We get an engaging cover image and corresponding visuals for each story segment.
  4. Then, the amazing FFmpeg tool strings these images together with some mood-setting background tunes, giving birth to a neat little film.

But wait, there’s more! I’ve also cooked up a backend system that gives real-time updates on Discord. Whether ChatGPT is crafting a tale, or Midjourney is cooking up images, or even if it’s just downloading stuff, every move is broadcasted to the operator via Discord API. And, should a hiccup occur and the system crashes, no sweat! There’s always a backup ready to pick up right where it left off.

Discord Screenshot Discord Screenshot

Oh, and before I forget, while Grimm tales are classic, the system’s versatility doesn’t stop there. Thanks to a customizable parameter file, you can set the tone and topic to create whatever story you fancy.

So, in essence, it’s endless nights of magical AI-curated stories. Exciting, right?

Some Story Snippet Some Story Snippet

Demo

Github

Steps

  1. Using ChatGPT to create story content and story image style
  2. Get the ChatGPT results as Midjourney prompt to generate images
  3. FFmpeg convert images into videos

Features

  • check_point.py will save the process step, if process failed system will restart and check_point.py will restore last step and continue
  • discordbot.py will report program status

Setup

  • config_template.yaml Setup your configs
    • OpenAI API
    • Discord Token
    • Midjourney Bot Token
    • Midjourney Bot Session ID
    • Midjourney Bot Channel ID
  • story_settings_template.yaml Setup story series you want to tell and topics

Usage

  • Rename config_template.yaml to config.yaml
  • Rename story_settings_template.yaml to story_settings.yaml
  • Install pipenv install
  • Run pipenv run python main.py