Liyang Chen (陈礼扬)

I am currently a fourth-year Ph.D student at Tsinghua University, supervised by Prof. Zhiyong Wu.

My research focuses on human-centric video synthesis, 2D/3D talking face generation and speech processing.

Email  /  Scholar  /  WeChat

Publications

A hight-quality and generalized framework for audio-driven talking face generation with efficient diffusion-based network tunning.

A human-specific multi-view diffusion model designed to generate high-quality novel view images from a single reference image.

A novel adaptive speech-driven facial animation approach, which learns the personalized talking style from a reference video of about 10 seconds and generates vivid facial expressions and head poses.

An unsupervised variational style transfer model (VAST) to vivify the neutral photo-realistic avatars. It is able to flexibly capture the expressive facial style from arbitrary video prompts and transfer it onto a personalized image renderer in a zero-shot manner.

A novel robust and efficient Speech-to-Animation (S2A) approach for synchronized facial animation generation in human-computer interaction. Experiments demonstrate the effectiveness of the proposed approach on both objective and subjective evaluation with 17x inference speedup compared with the state-of-the-art approach

we conduct systematic analyses on the motion jittering problem based on the pipeline that uses 3D face representations to bridge the input audio and output video, and improve the motion stability with a series of effective designs.

Audio-driven talking face and face swapping are typically viewed as separate tasks. We propose an end-to-end model that combines these two tasks.

A novel dance generation method designed to generate expressive dances, concurrently taking genre matching, beat alignment and dance dynamics into account.

Previous works on expressive speech synthesis mainly focus on current sentence. The context in adjacent sentences is neglected, resulting in inflexible speaking style for the same text, which lacks speech variations.

We explore an automatic gesture generation system based on multimodal representation learning

A style modeling method for expressive speech synthesis, to capture and predict styles at different levels from a wider range of context rather than a sentence.

SpeakerGAN allows the adversarial networks for distinguishing real/fake speech samples and predicting class labels simultaneously.

Work Experiences

This page is borrowd from Jon Barron.