How about a Colab demo?

[roseyyy2022ai]

Hi! I think your project is truly amazing and demonstrates great promise! It appears even more impressive than most other single image deepfake solutions like Thin-SplineMM which seem to produce quite a few strange artifacts whenever there's the slightest head turn or unusual movement in the drive video.

I'd love to explore this myself but my computer is quite slow and likely doesn't precisely meet the demands to run this model and your Hugging Face space seems overloaded quite a bit of the time and/or it takes an awfully long time to generate the videos, sometimes even just returning an error after a half hour wait.

Therefore I'd like to suggest creating a Jupiter(Colab) notebook with similar functionality so anyone could utilize this model themselves without needing to wait so long and/or face all the bugs on HF or wear out their home potato PCs😅

Does that suggestion sound good to you?

[harlanhong]

Yes. I would like to start a Colab demo for this project. But I have a lot to do currently and my time is limited. Would you like to develop it with me together?

[roseyyy2022ai]

Yup!!

[roseyyy2022ai]

Yup!!

In fact I am already working on it, but I'm pretty much stuck at the moment... Had to install a lot of stuff manually and I keep getting this error...
Logs:
/content/CVPR2022-DaGAN /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:135: UserWarning: Using 'weights' as positional parameter(s) is deprecated since 0.13 and may be removed in the future. Please use keyword parameter(s) instead. warnings.warn( /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum orNonefor 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passingweights=None. warnings.warn(msg) Traceback (most recent call last): File "/content/CVPR2022-DaGAN/demo.py", line 165, in <module> loaded_dict_enc = torch.load('depth/models/depth_face_model_Voxceleb2_10w/encoder.pth') File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 809, in load return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args) File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1172, in _load result = unpickler.load() File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1142, in persistent_load typed_storage = load_tensor(dtype, nbytes, key, _maybe_decode_ascii(location)) File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1116, in load_tensor wrap_storage=restore_location(storage, location), File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 217, in default_restore_location result = fn(storage, location) File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 182, in _cuda_deserialize device = validate_cuda_device(location) File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 166, in validate_cuda_device raise RuntimeError('Attempting to deserialize object on a CUDA ' RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.

Is there any way to resolve this maybe?

Colab link

[harlanhong]

Ok, It seems that we should load the weights into the CPU instead of the GPU in Line 165 of demo.py. You could modify the code of demo.py

[roseyyy2022ai]

Excuse me what do I change exactly?
I'm not really that much of a coder 😅

[roseyyy2022ai]

Excuse me what do I change exactly? I'm not really that much of a coder 😅

Okayyy I fixed it but now I'm getting this:
/content/CVPR2022-DaGAN /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:135: UserWarning: Using 'weights' as positional parameter(s) is deprecated since 0.13 and may be removed in the future. Please use keyword parameter(s) instead. warnings.warn( /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum orNonefor 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passingweights=None. warnings.warn(msg) /content/CVPR2022-DaGAN/demo.py:176: DeprecationWarning: Starting with ImageIO v3 the behavior of this function will switch to that of iio.v3.imread. To keep the current behavior (and make this warning disappear) use import imageio.v2 as imageioor callimageio.v2.imreaddirectly. source_image = imageio.imread(opt.source_image) Traceback (most recent call last): File "/content/CVPR2022-DaGAN/demo.py", line 189, in <module> generator, kp_detector = load_checkpoints(config_path=opt.config, checkpoint_path=opt.checkpoint, cpu=opt.cpu) File "/content/CVPR2022-DaGAN/demo.py", line 27, in load_checkpoints config = yaml.load(f) TypeError: load() missing 1 required positional argument: 'Loader'

[roseyyy2022ai]

Aaaand another update: I sorta got it running on Colab with Gradio UI but now I just keep getting this error:
Traceback (most recent call last): File "/content/DaGAN/demo_dagan.py", line 123, in <module> config = yaml.load(f) TypeError: load() missing 1 required positional argument: 'Loader' Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 437, in run_predict output = await app.get_blocks().process_api( File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1349, in process_api data = self.postprocess_data(fn_index, result["prediction"], state) File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1283, in postprocess_data prediction_value = block.postprocess(prediction_value) File "/usr/local/lib/python3.10/dist-packages/gradio/components.py", line 2190, in postprocess processed_files = (self._format_video(y), None) File "/usr/local/lib/python3.10/dist-packages/gradio/components.py", line 2249, in _format_video video = self.make_temp_copy_if_needed(video) File "/usr/local/lib/python3.10/dist-packages/gradio/components.py", line 259, in make_temp_copy_if_needed temp_dir = self.hash_file(file_path) File "/usr/local/lib/python3.10/dist-packages/gradio/components.py", line 223, in hash_file with open(file_path, "rb") as f: FileNotFoundError: [Errno 2] No such file or directory: 'temp/rst.mp4'
New colab test link

[Alexankharin]

Made a fork to fix some broken dependencies for gradio app on huggingface. You can run ot in colab https://colab.research.google.com/drive/1wT8G_aIDcvfp1ztaq0JvXBvloWhaOF-b#scrollTo=JNxXAwkaJzRc if interested