Skip to content
/ Splice Public

Official Pytorch Implementation for "Splicing ViT Features for Semantic Appearance Transfer" presenting "Splice" (CVPR 2022 Oral)

Notifications You must be signed in to change notification settings

omerbt/Splice

Repository files navigation

Splicing ViT Features for Semantic Appearance Transfer (CVPR 2022 - Oral)

arXiv Pytorch Open In Colab teaser

Splice is a method for semantic appearance transfer, as described in Splicing ViT Features for Semantic Appearance Transfer (link to paper).

Given two input images—a source structure image and a target appearance image–our method generates a new image in which the structure of the source image is preserved, while the visual appearance of the target image is transferred in a semantically aware manner. That is, objects in the structure image are “painted” with the visual appearance of semantically related objects in the appearance image. Our method leverages a self-supervised, pre-trained ViT model as an external semantic prior. This allows us to train our generator only on a single input image pair, without any additional information (e.g., segmentation/correspondences), and without adversarial training. Thus, our framework can work across a variety of objects and scenes, and can generate high quality results in high resolution (e.g., HD).

Getting Started

Installation

git clone https://github.com/omerbt/Splice.git
pip install -r requirements.txt

Run examples Open In Colab

Run the following command to start training

python train.py --dataroot datasets/splicing/cows

Intermediate results will be saved to <dataroot>/out/output.png during optimization. The frequency of saving intermediate results is indicated in the save_epoch_freq flag of the configuration.

Sample Results

plot

Citation

@inproceedings{tumanyan2022splicing,
  title={Splicing ViT Features for Semantic Appearance Transfer},
  author={Tumanyan, Narek and Bar-Tal, Omer and Bagon, Shai and Dekel, Tali},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={10748--10757},
  year={2022}
}

About

Official Pytorch Implementation for "Splicing ViT Features for Semantic Appearance Transfer" presenting "Splice" (CVPR 2022 Oral)

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •