Skip to content

A comprehensive list of recources (papers, repositories etc.) about face restoration methods.

Notifications You must be signed in to change notification settings

TaoWangzj/Awesome-Face-Restoration

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

35 Commits
 
 
 
 

Repository files navigation

Deep Face Restoartion: Denoise, Super-Resolution, Deblur and Artifact Removal

A comprehensive list of resources for Deep Face Restoartion


Suggest new item
Report Bug

This repository provides a summary of deep learning-based face restoration algorithms. Our classification is based on the review paper "A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal".

Blind face restoration Face super-resolution Face deblurring Face denoising Face artifact removal
Blind Face Restoration Face Super-Resolution Face Deblurring Face Denoising Face Artifact Removal

Survey paper

T. Wang, k. Zhang, X. Chen, W. Luo, J. Deng, T. Lu, X. Cao, W. Liu, H. Li, and S. Zafeiriou “A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal,” arXiv preprint arXiv:2211.02831, 2022. [pdf]

@article{wang2022survey
  title={A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal},
  author={Wang, Tao, and Zhang, kaihao, and Chen, Xuanxi and Luo, Wenhan and Deng, Jiankang and Lu, Tong and Cao, Xiaochun and Liu, Wei and Li, Hongdong and Zafeiriou, Stefanos},
  journal={arXiv preprint arXiv:2211.02831},
  year={2022}
}

See our paper for more details. If you have any suggestions, feel free to contact me (e-mail: taowangzj@gmail.com). Thanks.

Table of contents

Surveys

year Pub Title Link
2022 Arxiv A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal [paper]

Deep Blind Face Restoration

Year
Pub
Title
Links
Arch
2018 ECCV Learning Warped Guidance for Blind Face Restoration [paper][code] CNN
2018 CVPR FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors [paper][code] CNN
2019 IJCV Identity-preserving Face Recovery from Stylized Portraits [paper][code] GAN
2020 ECCV Blind Face Restoration via Deep Multi-scale Component Dictionaries [paper][code] CNN
2020 CVPR Enhanced Blind Face Restoration with Multi-Exemplar Images
and Adaptive Spatial Feature Fusion
[paper][code] CNN
2020 MM HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment [paper][code] GAN
2020 CVPR Image Processing Using Multi-Code GAN Prior [paper][code] GAN
2021 TPAMI Face Restoration via Plug-and-Play 3D Facial Priors [paper][code] CNN
2021 CVPR Progressive Semantic-Aware Style Transformation for Blind Face Restoration [paper][code] GAN
2021 CVPR Towards Real-World Blind Face Restoration with Generative Facial Prior [paper][code] GAN
2021 CVPR GAN Prior Embedded Network for Blind Face Restoration in the Wild [paper][code] GAN
2022 Arxiv Multi-prior learning via neural architecture search for blind face restoration [paper][code] CNN
2022 CVPR Blind Face Restoration via Integrating Face Shape and Generative Priors [paper][code] GAN
2022 CVPR RestoreFormer: High-Quality Blind Face Restoration
from Undegraded Key-Value Pairs
[paper][code] ViT
2022 NeurIPS Towards Robust Blind Face Restoration with Codebook Lookup Transformer [paper][code] ViT
2022 AAAI Panini-Net: GAN Prior Based Degradation-Aware Feature Interpolation for Face Restoration [paper][code] GAN
2022 Arxiv FaceFormer: Scale-aware Blind Face Restoration with Transformers [paper][code] ViT
2022 Arxiv Blind Face Restoration: Benchmark Datasets and a Baseline Model [paper][code] ViT
2022 ECCV VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder [paper][code] CNN
2022 TPAMI Learning Dual Memory Dictionaries for Blind Face Restoration [paper][code] CNN
2022 Arxiv Difface: Blind Face Restoration with Diffused Error Contraction [paper][code] Diffusion
2023 CVPR DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration [paper][code] Diffusion
2023 CVPR TFRGAN: Leveraging Text Information for Blind Face Restoration with Extreme Degradation [paper][code] GAN
2023 TCSVT DEAR-GAN: Degradation-Aware Face Restoration With GAN Prior [paper][code] GAN
2023 FSP Degradation Learning and Skip-Transformer for Blind Face Restoration [paper][code] GAN
2023 WACV AT-DDPM: Restoring Faces degraded by Atmospheric Turbulence using Denoising Diffusion Probabilistic Models [paper][code] Diffusion
2023 ACMMM DiffBFR: Bootstrapping Diffusion Model for Blind Face Restoration [paper][code] Diffusion
2024 TCSVT Towards Real-World Blind Face Restoration with Generative Diffusion Prior [paper][code] Diffusion
2024 WACV Diffuse and Restore: A Region-Adaptive Diffusion Model for Identity-Preserving Blind Face Restoration [paper][code] Diffusion
2024 AAAI Blind Face Restoration under Extreme Conditions: Leveraging 3D-2D Prior Fusion for Superior Structural and Texture Recovery [paper][code] Diffusion

Deep Face Super-Resolution

Year
Pub
Title
Links
Arch
2015 AAAI Learning Face Hallucination in the Wild [paper][code] CNN
2016 ECCV Deep Cascaded Bi-Network for Face Hallucination [paper][code] CNN
2016 ECCV Ultra-Resolving Face Images by Discriminative Generative Networks [paper][code] GAN
2017 CVPR Attention-Aware Face Hallucination via Deep Reinforcement Learning [paper][code] CNN
2017 CVPR Hallucinating Very Low-Resolution Unaligned and Noisy Face Images by
Transformative Discriminative Autoencoders
[paper][code] CNN
2017 ICCV Learning to Super-Resolve Blurry Face and Text Images [paper][code] GAN
2017 AAAI Face Hallucination with Tiny Unaligned Images
by Transformative Discriminative Neural Networks
[paper][code] GAN
2018 ECCV Face Super-resolution Guided by Facial Component Heatmaps [paper][code] CNN
2018 CVPR Super-FAN: Integrated facial landmark localization and
super-resolution of real-world low resolution faces in arbitrary poses with gans
[paper][code] GAN
2018 ECCV To learn image super-resolution,
use a GAN to learn how to do image degradation first
[paper][code] GAN
2019 CVPRW Exemplar Guided Face Image Super-Resolution without Facial Landmarks [paper][code] CNN
2019 BMVC Progressive Face Super-Resolution via Attention to Facial Landmark [paper][code] CNN
2020 WACV Component Attention Guided Face Super-Resolution Network: CAGFace [paper][code] CNN
2020 TNNLS Dual-Path Deep Fusion Network for Face Image Hallucination [paper][code] CNN
2019 NEUCOM On potentials of regularized Wasserstein generative adversarial networks for
realistic hallucination of tiny faces
[paper][code] GAN
2020 CVPR PULSE: Self-Supervised Photo Upsampling via
Latent Space Exploration of Generative Models
[paper][code] GAN
2021 TBBIS E-ComSupResNet: Enhanced Face Super-Resolution Through Compact Network [paper][code] CNN
2021 MM Face Hallucination via Split-Attention in Split-Attention Network [paper][code] CNN
2021 ICIP Progressive Face Super-Resolution with Non-Parametric Facial Prior Enhancement [paper][code] GAN
2022 CVPR GCFSR: a Generative and Controllable Face Super Resolution Method
Without Facial and GAN Priors
[paper][code] GAN
2022 TPAMI EDFace-Celeb-1 M: Benchmarking Face Hallucination with a Million-scale Dataset [paper][code] CNN
2023 TIP Semi-Cycled Generative Adversarial Networks for Real-World Face Super-Resolution [paper][code] GAN
2023 AAAI GAN Prior based Null-Space Learning for Consistent Super-Resolution [paper][code] GAN
2023 CVPR Spatial-Frequency Mutual Learning for Face Super-Resolution [paper][code] CNN
2023 TMM Sctanet: A Spatial Attention-guided Cnn-transformer Aggregation Network for Deep Face Image Super-Resolution [paper][code] ViT
2023 TMM An Efficient Latent Style Guided Transformer-CNN Framework for Face Super-Resolution [paper][code] ViT
2023 TMM Exploiting Multi-scale Parallel Self-attention and Local Variation via Dual-branch Transformer-CNN Structure for Face Super-resolution [paper][code] ViT
2023 PR A Composite Network Model for Face Super-Resolution with Multi-Order Head Attention Facial Priors [paper][code] ViT
2023 TIP Semi-Cycled Generative Adversarial Networks for Real-World Face Super-Resolution [paper][code] GAN
2023 AS A Multi-Scale Deep Back-Projection Backbone for Face Super-Resolution with Diffusion Models [paper][code] Diffusion
2023 TIM Deep HyFeat Based Attention in Attention Model for Face Super-Resolution [paper][code] CNN
2023 PRL Attentive ExFeat based Deep Generative Adversarial Network for Noise Robust Face Super-resolution [paper][code] GAN
2023 NN Self-attention Learning Network for Face Super-resolution [paper][code] CNN

Deep Face Deblurring

Year
Pub
Title
Links
Arch
2017 Arxiv DeepDeblur: Fast one-step blurry face images restoration [paper][code] CNN
2018 CVPR Deep Semantic Face Deblurring [paper][code] CNN
2020 IJCV Exploiting Semantics for Face Image Deblurring [paper][code] CNN
2020 TIP Deblurring Face Images using Uncertainty Guided Multi-Stream Semantic Networks [paper][code] CNN
2020 MM HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment [paper][code] GAN
2020 AAAI Learning to deblur face images via sketch synthesis [paper][code] CNN
2022 TOG Face Deblurring using Dual Camera Fusion on Mobile Phones [paper][code] CNN
2022 WACV Deep Feature Prior Guided Face Deblurring [paper][code] CNN
2022 Arxiv Multi-prior learning via neural architecture search for blind face restoration [paper][code] CNN
2022 Arxiv Blind Face Restoration: Benchmark Datasets and a Baseline Model [paper][code] ViT
2022 SIGGRAPH Face Deblurring using Dual Camera Fusion on Mobile Phones [paper][code] CNN

Deep Face Denoising

Year
Pub
Title
Links
Arch
2020 MM HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment [paper][code] GAN
2022 Arxiv Multi-prior learning via neural architecture search for blind face restoration [paper][code] CNN
2022 Arxiv Blind Face Restoration: Benchmark Datasets and a Baseline Model [paper][code] ViT

Deep Face Artifact Removal

Year
Pub
Title
Links
Arch
2020 MM HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment [paper][code] GAN
2022 Arxiv Blind Face Restoration: Benchmark Datasets and a Baseline Model [paper][code] ViT
2022 Arxiv Multi-prior learning via neural architecture search for blind face restoration [paper][code] CNN

Other Related Works

  • CG-GAN: Class-Attribute Guided Generative Adversarial Network for Old Photo Restoration, Liu et al., ACMM 2021. Link
  • Old Photo Restoration via Deep Latent Space Translation, Wan et al., TPAMI 2022. Link
  • Demeshnet: Blind Face Inpainting for Deep Meshface Verification, Zhang et al. TIP 2017. Link
  • Face Inpainting based on High-level Facial Attributes, Jampour et al., CVIU 2017. Link
  • ABPN: Adaptive Blend Pyramid Network for Real-Time Local Retouching of Ultra High-Resolution Photo, Lei et al., CVPR 2022. Link
  • Autoretouch: Automatic Professional Face Retouching, Shafaei et al., WACV 2021. Link
  • Network Architecture Search for Face Enhancement, Yasarla et al., Arxiv 2021. Link
  • Towards Automatic Face-to-face Translation, KR, R et al., ACMMM 2019. Link
  • A Sketchtransformer Network for Face Photo-sketch Synthesis, Zhu et al., IJCAI 2021. Link

Image Quality Assessment

Method Type Code/Ref
PSNR (Peak Signal-to-Noise Ratio) Full-Reference Code
SSIM (Structural Similarity Index Measurement) Full-Reference Code
MS-SSIM (Multi-scale Structural Similarity Index Measurement) Full-Reference Code
LPIPS (Learned Perceptual Image Patch Similarity) Full-Reference Code
NIQE (Naturalness Image Quality Evaluator) Non-Reference Code
FID (Fréchet Inception Distance) Non-Reference Code
PI (Perceptual Index) Non-Reference Code
MOS (Mean Opinion Score) Subject-Metric Ref
iPrecision Task Driven-Metric Ref
iRecall Task Driven-Metric Ref
LLE (Landmark Localization Error) Task Driven-Metric Code
Deg (Identity Distance) Task Driven-Metric Code
AFLD (Average Face Landmark Distance) Task Driven-Metric Code
AFICS (Average Face ID Cosine Similarity) Task Driven-Metric Code

Benchmark Datasets

Recommended Datasets

Dataset Usage Quantity Type
EDFace-Celeb training&testing >1M paired dataset
PFHQ training 60,000 paired dataset
FFHQ training 70,000 non-paired dataset
CelebChild-Test testing 180 non-paired real-world dataset
WebPhoto-Test testing 407 non-paired real-world dataset
LFW-Test testing 1,711 non-paired real-world dataset

All Datasets

Dataset Paper Year
BioID Robust face detection using the hausdorff distance 2001
LFW Labeled faces in the wild: A database forstudying face recognition in unconstrainedenvironments 2008
Pubfig Attribute and similar classifiers for face verification 2009
Multi-PIE Multi-PIE 2010
AFLW Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization 2011
Helen Interactive Facial Feature Localization 2012
300W 300 Faces in-the-Wild Challenge: The first facial landmark localization Challenge 2013
CASIA-WebFace
(Password: y3wj)
Learning Face Representation from Scratch 2014
CelebA Deep Learning Face Attributes in the Wild 2015
IMDB-WIKI DEX: Deep EXpectation of apparent age from a single image 2015
LSUN LSUN: Construction of a Large-Scale Image Dataset using Deep Learning with Humans in the Loop 2015
VGGFace Deep Face Recognition 2015
300W-LP Face Alignment Across Large Poses: A 3D Solution 2016
VoxCeleb1 VoxCeleb: a large-scale speaker identification dataset 2017
LS3D-W How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks) 2017
LS3D-W balanced How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks) 2017
Menpo The Menpo Facial Landmark Localisation Challenge: A step towards the solution 2017
VGGFace2 VGGFace2: A dataset for recognising faces across pose and age 2018
VoxCeleb2 Voxceleb2: Deep speaker recognition 2018
FFHQ A Style-Based Generator Architecture for Generative Adversarial Networks 2019
CelebChild-Test Towards Real-World Blind Face Restoration with Generative Facial Prior 2021
WebPhoto-Test Towards Real-World Blind Face Restoration with Generative Facial Prior 2021
CelebA-Test Towards Real-World Blind Face Restoration with Generative Facial Prior 2021
LFW-Test Towards Real-World Blind Face Restoration with Generative Facial Prior 2021
VFHQ VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolution 2022
EDFace-Celeb EDFace-Celeb-1M: Benchmarking Face Hallucination with a Million-scale Dataset 2022
EDFace-Celeb-1M (BFR128) Blind Face Restoration: Benchmark Datasets and a Baseline Model 2022
EDFace-Celeb-150K (BFR512) Blind Face Restoration: Benchmark Datasets and a Baseline Model 2022
statistics

visitors