Alibaba and BUPT Introduce FantasyPortrait: A Leap in Digital Human Animation

In a significant advancement for digital human technology, Alibaba has partnered with Beijing University of Posts and Telecommunications (BUPT) to launch FantasyPortrait, a project that pushes the boundaries of facial expression transfer and multi-character control in animations.

Revolutionizing Emotional Expression in Digital Humans

The project leverages an expression-augmented diffusion transformer (DiT) to achieve highly realistic emotional expressions across both single-person and multi-person scenarios. Unlike traditional methods, FantasyPortrait can precisely replicate subtle smiles or intense anger while maintaining high fidelity.

A standout feature is its ability to manage independent facial expressions for multiple characters simultaneously—eliminating the common issue of expression interference seen in older technologies. This breakthrough opens new possibilities for film production, virtual reality, and gaming industries where nuanced character interactions are crucial.

Multimodal Flexibility: Beyond Human Characters

FantasyPortrait isn’t limited to human figures. The technology also supports animal animations, broadening its creative applications. Additionally, it offers audio-driven functionality, synchronizing digital humans' expressions and movements with audio inputs—ideal for virtual anchors or interactive content.

Commitment to Open Source and Industry Collaboration

Alibaba has announced plans to open-source FantasyPortrait’s code and models, lowering barriers for developers worldwide. This move aligns with the company’s broader strategy of fostering AI innovation through accessible tools.

The collaboration between Alibaba and BUPT highlights the power of industry-academia partnerships. BUPT’s expertise in AI research combined with Alibaba’s engineering prowess has been instrumental in developing this cutting-edge solution.

Key Applications Across Industries

• Film & Gaming: Streamlines animation production for multi-character scenes.
• Virtual Reality: Enhances immersive experiences with lifelike avatars.
• Content Creation: Empowers creators with audio-driven animation tools.

The project is poised to set a new benchmark in digital human technology, offering both quality improvements and creative flexibility. Developers can explore its potential once the code is released on GitHub.

Key Points:

• Uses expression-augmented DiT for precise emotional control.
• Enables multi-character expression independence without interference.
• Supports humans, animals, and audio-driven animations.
• Future open-source release to empower global developers.