http://doodlemate.com

It doesn't use generative AI, instead it auto-rigs the drawings in just a few seconds.

If you do give it a try, please let me know how it goes!

But it comes with costs. GenAI video is expensive to generate, and most tools constrain your animation to a handful of seconds, not long enough to tell a real story. You can generate multiple clips and stitch them together, but then you'll run up again another limit of GenAI- subject consistency (especially with non-realistic subjects, like doodles).

It's also difficult to finely control genAI outputs, which I argue limits the creative expressivity of the human. And if you generate numerous clips to try to get things perfect, it can get expensive.

Our approach is limited by the motion/visual/audio assets we have access to. But, when we release DoodleMateStudio users will be able to upload their own visuals, record their own audio, capture their own motions, and specify their own high-level story scenes. This should be enough to let people tell expressive and personal stories. And if we get things right, it will also be a lot more fun than refining a prompt.

Please let me know how it goes and if there's any functionality she/you would want.

We use non-generative AI models to quickly auto-rig the character when it's uploaded. In a traditional computer graphics animation pipeline this would be done by hand and would be a slow process; we use these models to speed that step up. The resulting animations don't use any AI at all(generative or otherwise).

I made an app that takes a photo of a paper drawing and, in a handful of seconds, creates a fully rigged character that can be used in an animation or little story. It doesn’t use any image-to-video generative AI models. Instead, I built it using the years of insights I’ve picked up studying children’s drawings and character animation.

Today we’re releasing a community beta. I respect this community and would value any feedback you offer. It’s easy to try- you don’t need to create an account to check it out. We’ve got several free stories to drop your character into, and a Mother’s Day eCard.

I’m also working on a tool, DoodleMate Studio, to easily allow people to author their own stories instead of using premade templates. But what form that takes is going to be highly dependent on the type of feedback we get from the community with this beta.

How this came to be:

I’ve worked in this space for a while. Here’s an old HN post related to a popular tech demo I did ( https://news.ycombinator.com/item?id=30469321) and another one from when I open sourced the data and code ( https://news.ycombinator.com/item?id=35561203). I also wrote a SIGGRAPH paper about the methodology (https://dl.acm.org/doi/full/10.1145/3592788).

I’d moved on to other things, but had always felt like there was such potential in this space. Last year I decided I was over big tech and, with a lot of encouragement from my family, finally decided to pursue this seriously. Since then, my wife and I have been building this together. We’re bootstrapping at the moment, trying to give ourselves time and space to make sure DoodleMate turns into something wonderful and wholesome.

Thanks, Jesse

Comments URL: https://news.ycombinator.com/item?id=48038654

Points: 15

# Comments: 12

And while I (maybe more than anyone else) am highly aware of all the ways this tool doesn't create high-quality animation, I'm okay with it. The point here isn't to create animations that rival those of professional animators; it's to lower the barrier-to-entry to the point where almost anyone can animate their own character.

And if a tiny number of users become more interested in animation through this tool, learn how to do it the proper way, and create animations your father would approve of... well I think that would be a phenomenal outcome.

And if I hid the code away so only I could use it, there's a good chance no one would ever create any of those experiences. Creating a successful app requires mobile UI people, devops, designers, marketing/SEO, monetization, etc. I don't want to do ANY of that... in the land of app creation, I'm more of a Tom Bombadil-type.

This is also my first project out of graduate school. I feel like I'll probably have other ideas down the road, and maybe I'll try to monetize those ones with what I've learned from this adventure.

I’m guessing that the nose key point is located below the shoulders in your child’s drawing?

That’s a known issue caused by how we ‘apply’ poses onto the characters. It’s fixed in the GitHub code but not in the browser version. Try moving the nose keypoint to the top of the characters head and that should resolve it

The rendering code utilizes an MVC framework. If you need the animation output in a .obj format or something, it's as simple as creating a new View that supports it. Feel free to request this in a Github issue if you want me to prioritize it.

One work that seems particularly pertinent to your use case is:

Gesture3D: Posing 3D Characters via Gesture Drawings http://www.cs.ubc.ca/labs/imager/tr/2016/Gesture3D/