YunjinPark/awesome_talking_face_generation

836

41 commits

updated Nov 19, 2025

See the code

README

Awesome talking face generation

papers & codes

🔍 Survey / Review Papers

Year	Title	Focus/Tag	Paper
2025	Advancing Talking Head Generation: A Comprehensive Survey of Multi-Modal Methodologies, Datasets, Evaluation Metrics, and Loss Functions	methods, datasets, metrics	paper
2024	A Comprehensive Taxonomy and Analysis of Talking Head Synthesis: Techniques for Portrait Generation, Driving Mechanisms, and Editing	generation/driving/editing, datasets/metrics, applications	paper

2025

Title		Paper	Code	Datasets	Keywords
EmotiveTalk: Expressive Talking Head Generation through Audio Information Decoupling and Emotional Video Diffusion	CVPR(25)	paper	project	HDTF, MEAD[	emotion
InsTaG: Learning Personalized 3D Talking Head from Few-Second Video	CVPR(25)	paper	code	-	Few-shot Learning
Perceptually Accurate 3D Talking Head Generation: NewDefinitions, Speech-Mesh Representation, and Evaluation Metrics	CVPR(25)	paper	code	MEAD-3D, VOCASET	3D
Monocular and Generalizable Gaussian Talking Head Animation	CVPR(25)	paper	project	HDTF, NeRSemble	3D
DualTalk: Dual-Speaker Interaction for 3D Talking Head Conversations	CVPR(25)	paper	code	-	3D, dual

2023

Title		Paper	Code	Datasets	Keywords
CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior	CVPR(23)	paper	code	BIWI, VOCA
DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animation	CVPR(23)	paper		HDTF	Diffusion
AVFace: Towards Detailed Audio-Visual 4D Face Reconstruction	CVPR(23)	paper		Multiface	3D
Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expert	CVPR(23)	paper	code	LRS2
LipFormer: High-fidelity and Generalizable Talking Face Generation with A Pre-learned Facial Codebook	CVPR(23)	paper		LRS2, FFHQ
Parametric Implicit Face Representation for Audio-Driven Facial Reenactment	CVPR(23)	paper		HDTF
Identity-Preserving Talking Face Generation with Landmark and Appearance Priors	CVPR(23)	paper	code	LRS2, LRS3
High-fidelity Generalized Emotional Talking Face Generation with Multi-modal Emotion Space Learning	CVPR(23)	paper		MEAD	emotion
Emotional Talking Head Generation based on Memory-Sharing and Attention-Augmented Networks	InterSpeech(23)	paper		MEAD	emotion
EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation	ICCV(23)	paper	code(not yet)		emotion
Emotionally Enhanced Talking Face Generation		paper	code	CREMA-D	emotion
DINet: Deformation Inpainting Network for Realistic Face Visually Dubbing on High Resolution Video	AAAI(23)	paper	code
CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior		paper	code		3D
GENEFACE: GENERALIZED AND HIGH-FIDELITY AUDIO-DRIVEN 3D TALKING FACE SYNTHESIS	ICLR (23)	paper	code		NeRF
OPT: ONE-SHOT POSE-CONTROLLABLE TALKING HEAD GENERATION		paper
LipNeRF: What is the right feature space to lip-sync a NeRF?		paper			NeRF
Audio-Visual Face Reenactment	WACV (23)	paper	code
Towards Generating Ultra-High Resolution Talking-Face Videos With Lip Synchronization	WACV (23)	paper
StyleTalk: One-shot Talking Head Generation with Controllable Speaking Styles	AAAI(23)	paper	code
DiffTalk: Crafting Diffusion Models for Generalized Talking Head Synthesis		paper	proj		Diffusion
Diffused Heads: Diffusion Models Beat GANs on Talking-Face Generation		paper	proj		Diffusion
Speech Driven Video Editing via an Audio-Conditioned Diffusion Model		paper	code		Diffusion
TalkCLIP: Talking Head Generation with Text-Guided Expressive Speaking Styles		paper		Text-Annotated MEAD	Text

2022

title		paper	code	dataset	keywords
Talking Head Generation with Probabilistic Audio-to-Visual Diffusion Priors		paper	proj
SPACE: Speech-driven Portrait Animation with Controllable Expression	ICCV(23)	paper			Pose, Emotion
SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation	CVPR(23)	paper	code
Compressing Video Calls using Synthetic Talking Heads	BMVC (22)	paper			application
EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model	SIGGRAPH (22)	paper			emotion
Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesis	ECCV(22)	paper	code
Expressive Talking Head Generation with Granular Audio-Visual Control	CVPR(22)	paper
Talking Face Generation With Multilingual TTS	CVPR(22)	paper	code		-
Deep Learning for Visual Speech Analysis: A Survey		paper			survey
StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN		paper	code		stylegan
Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation	ECCV(22)	paper	code(coming soon)		NeRF
Cross-Modal Mutual Learning for Audio-Visual Speech Recognition and Manipulation		paper
SyncTalkFace: Talking Face Generation with Precise Lip-syncing via Audio-Lip Memory	AAAI(22)	paper(temp)		LRW, LRS2, BBC News
DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Rendering		paper			NeRF
Face-Dubbing++: Lip-Synchronous, Voice Preserving Translation of Videos		paper
Dynamic Neural Textures: Generating Talking-Face Videos with Continuously Controllable Expressions		paper
DialogueNeRF: Towards Realistic Avatar Face-to-face Conversation Video Generation		paper
Talking Head Generation Driven by Speech-Related Facial Action Units and Audio- Based on Multimodal Representation Fusion		paper
StyleTalker: One-shot Style-based Audio-driven Talking Head Video Generation		paper		-
AUTOLV: AUTOMATIC LECTURE VIDEO GENERATOR		paper
Synthesizing Photorealistic Virtual Humans Through Cross-modal Disentanglement		paper

2021

title		paper	code	dataset
Depth-Aware Generative Adversarial Network for Talking Head Video Generation		paper	code
		paper	code
Parallel and High-Fidelity Text-to-Lip Generation		paper
[Survey]Deep Person Generation: A Survey from the Perspective of Face, Pose and Cloth Synthesis	-	paper
FaceFormer: Speech-Driven 3D Facial Animation with Transformers	CVPR(22)	paper	code
Voice2Mesh: Cross-Modal 3D Face Model Generation from Voices		paper	code
FACIAL: Synthesizing Dynamic Talking Face with Implicit Attribute Learning	ICCV	paper	code
Imitating Arbitrary Talking Style for Realistic Audio-Driven Talking Face Synthesis		paper	code
Audio-Driven Emotional Video Portraits	CVPR	paper	code	MEAD, LRW
LipSync3D: Data-Efficient Learning of Personalized 3D Talking Faces from Video using Pose and Lighting Normalization	CVPR	paper
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation	CVPR	paper	code	VoxCeleb2, LRW
Flow-guided One-shot Talking Face Generation with a High-resolution Audio-visual Dataset	CVPR	paper	code	HDTF
MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement	ICCV	paper	code(coming soon)
AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis	ICCV	paper	code
Write-a-speaker: Text-based Emotional and Rhythmic Talking-head Generation	AAAI	paper	code(coming soon)	Mocap dataset
Visual Speech Enhancement Without A Real Visual Stream		paper
Text2Video: Text-driven Talking-head Video Synthesis with Phonetic Dictionary		paper	code
Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion	IJCAI	paper	code	VoxCeleb, GRID, LRW
3D-TalkEmo: Learning to Synthesize 3D Emotional Talking Head		paper
AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Person		paper		VoxCeleb2, Obama

2020

title		paper	code	dataset
[Survey]What comprises a good talking-head video generation?: A survey and benchmark		paper	code
One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing	CVPR(21)	paper	code
Speech Driven Talking Face Generation from a Single Image and an Emotion Condition		paper	code	CREMA-D
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild	ACMMM	paper	code	LRS2
Talking-head Generation with Rhythmic Head Motion	ECCV	paper	code	Crema, Grid, Voxceleb, Lrs3
MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation	ECCV	paper	code	VoxCeleb2, AffectNet
Neural voice puppetry:Audio-driven facial reenactment	ECCV	paper
Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatars	ECCV	paper	code
HeadGAN:Video-and-Audio-Driven Talking Head Synthesis		paper		VoxCeleb2
MakeItTalk: Speaker-Aware Talking Head Animation		paper	code, code	VoxCeleb2, VCTK
Audio-driven Talking Face Video Generation with Learning-based Personalized Head Pose	-	paper	code	ImageNet, FaceWarehouse, LRW
Photorealistic Lip Sync with Adversarial Temporal Convolutional Networks		paper
SPEECH-DRIVEN FACIAL ANIMATION USING POLYNOMIAL FUSION OF FEATURES		paper		LRW
Animating Face using Disentangled Audio Representations	WACV	paper
Everybody’s Talkin’: Let Me Talk as You Want		paper
Multimodal Inputs Driven Talking Face Generation With Spatial-Temporal Dependency		paper
Speech Driven Talking Face Generation from a Single Image and an Emotion Condition		paper

2019

title		paper	code	dataset
Hierarchical Cross-Modal Talking Face Generation with Dynamic Pixel-Wise Loss	CVPR	paper	code	VGG Face, LRW

datasets

MEAD link
HDTF link
CREMA-D link
VoxCeleb link
LRS2 link
LRW link
GRID link
SAVEE link
BIWI(3D) link
VOCA link
Multiface(3D) link

metrics

PSNR (peak signal-to-noise ratio)
SSIM (structural similarity index measure)
LMD (landmark distance error)
LRA (lip-reading accuracy) -
FID (Fréchet inception distance)
LSE-D (Lip Sync Error - Distance)
LSE-C (Lip Sync Error - Confidence)
LPIPS (Learned Perceptual Image Patch Similarity) -
NIQE (Natural Image Quality Evaluator) -

Similar repos

harlanhong/awesome-talking-head-generation

1,933

JosephPai/Awesome-Talking-Face

📖 A curated list of resources dedicated to talking face.

1,535

Kedreamix/Awesome-Talking-Head-Synthesis

💬 An extensive collection of exceptional resources dedicated to the captivating world of talking…

1,556

Jason-cs18/awesome-avatar

📖 A curated list of resources dedicated to avatar.

TimoBolkart/FLAME-Universe

Summary of publicly available ressources such as code, datasets, and scientific papers for the…

707

flyingby/Awesome-Deepfake-Generation-and-Detection

[CSUR 2026] A Survey on Deepfake Generation and Detection

652

shivangi-aneja/FaceTalk

[CVPR 2024] FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models

239

Winn1y/Awesome-Human-Motion-Video-Generation

【Accepted by TPAMI】Human Motion Video Generation: A Survey (https://ieeexplore.ieee.org/document/11…

345