Data Preprocessing Error

[hasanshahid5678]

I am trying to preprocess the video https://drive.google.com/file/d/1e7VejTUh9fEg-DHEdestvHZJSx1FklBU/view?usp=sharing , but it is giving the following errors.

video:811654kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
frame=11835 fps=2369 q=1.0 Lsize=N/A time=00:07:53.36 bitrate=N/A speed=94.7x    
todo videos number: 1
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1714576703.928276  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
| Extracting Segmaps && Saving...
generating segment images in multi-processes...:   0%|                                                                                                             | 0/11835 [00:00<?, ?it/s]I0000 00:00:1714576704.382687  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.383241  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.384124  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.386262  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.387167  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.387454  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.388938  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.388938  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.390370  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.393299  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.394696  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.397276  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.403220  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.405642  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.434605  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
I0000 00:00:1714576704.454487  289780 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
generating segment images in multi-processes...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 11835/11835 [06:24<00:00, 30.76it/s]
| Extracted Segmaps Done.
| Extracting background...
combining backgrounds...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 515/515 [04:24<00:00,  1.95it/s]
<class 'ValueError'> Found array with 0 sample(s) (shape=(0, 2)) while a minimum of 1 is required by NearestNeighbors.
Traceback (most recent call last):
  File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/utils/process_video/extract_segment_imgs.py", line 361, in extract_segment_job
    bg_img = extract_background(img_lst, segmap_mask_lst, method=background_method, device=device, mix_bg=mix_bg)
  File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/utils/process_video/extract_segment_imgs.py", line 140, in extract_background
    distances, indices = nbrs.kneighbors(bg_xys) # 对non-bg img，用KNN找最近的bg pixel
  File "/opt/conda/envs/geneface/lib/python3.9/site-packages/sklearn/neighbors/_base.py", line 826, in kneighbors
    X = self._validate_data(X, accept_sparse="csr", reset=False, order="C")
  File "/opt/conda/envs/geneface/lib/python3.9/site-packages/sklearn/base.py", line 633, in _validate_data
    out = check_array(X, input_name="X", **check_params)
  File "/opt/conda/envs/geneface/lib/python3.9/site-packages/sklearn/utils/validation.py", line 1072, in check_array
    raise ValueError(
ValueError: Found array with 0 sample(s) (shape=(0, 2)) while a minimum of 1 is required by NearestNeighbors.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/utils/process_video/extract_segment_imgs.py", line 491, in <module>
    extract_job(*fn_args[0])
  File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/utils/process_video/extract_segment_imgs.py", line 384, in extract_segment_job
    traceback.print_exc(e)
  File "/opt/conda/envs/geneface/lib/python3.9/traceback.py", line 163, in print_exc
    print_exception(*sys.exc_info(), limit=limit, file=file, chain=chain)
  File "/opt/conda/envs/geneface/lib/python3.9/traceback.py", line 103, in print_exception
    for line in TracebackException(
  File "/opt/conda/envs/geneface/lib/python3.9/traceback.py", line 517, in __init__
    self.stack = StackSummary.extract(
  File "/opt/conda/envs/geneface/lib/python3.9/traceback.py", line 340, in extract
    if limit >= 0:
TypeError: '>=' not supported between instances of 'ValueError' and 'int'
todo videos number: 1
Root 0: extracing MP-based landmark2d:   0%|                                                                                                                           | 0/1 [00:00<?, ?it/s]WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1714577393.724594  292523 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
W0000 00:00:1714577393.725103  292523 face_landmarker_graph.cc:174] Sets FaceBlendshapesGraph acceleration to xnnpack by default.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
I0000 00:00:1714577393.774896  292523 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
W0000 00:00:1714577393.775419  292523 face_landmarker_graph.cc:174] Sets FaceBlendshapesGraph acceleration to xnnpack by default.
Warning: failed detect ldm in idx=0, use previous frame results.
Traceback (most recent call last):
  File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/utils/process_video/extract_lm2d.py", line 75, in extract_landmark_job
    img_lm478, vid_lm478 = face_landmarker.extract_lm478_from_video_name(video_name)
  File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/utils/mp_feature_extractors/face_landmarker.py", line 80, in extract_lm478_from_video_name
    img_lm478, vid_lm478 = self.extract_lm478_from_frames(frames, fps, anti_smooth_factor)
  File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/utils/mp_feature_extractors/face_landmarker.py", line 102, in extract_lm478_from_frames
    img_face_landmarks = np.array([[l.x, l.y, l.z] for l in img_ldm_i])
UnboundLocalError: local variable 'img_ldm_i' referenced before assignment
finished 1 / 1 = 1.0000, failed 1 / 1 = 1.0000
Root 0: extracing MP-based landmark2d: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:09<00:00,  9.64s/it]
args Namespace(vid_dir='data/raw/videos/bipolar_2.mp4', ds_name='nerf', seed=0, process_id=0, total_process=1, id_mode='global', keypoint_mode='mediapipe', large_yaw_threshold=9999999.9, debug=True, reset=True, load_names=False)
['data/raw/videos/bipolar_2.mp4']
process 0: fitting 3dmm ...:   0%|                                                                                                                                     | 0/1 [00:00<?, ?it/s]lms_2d file not found, try to extract it from video... data/processed/videos/bipolar_2/lms_2d.npy
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1714577403.960238  292647 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
W0000 00:00:1714577403.960656  292647 face_landmarker_graph.cc:174] Sets FaceBlendshapesGraph acceleration to xnnpack by default.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
I0000 00:00:1714577404.004503  292647 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
W0000 00:00:1714577404.004936  292647 face_landmarker_graph.cc:174] Sets FaceBlendshapesGraph acceleration to xnnpack by default.
Warning: failed detect ldm in idx=0, use previous frame results.
local variable 'img_ldm_i' referenced before assignment
finished 1 / 1 = 1.0000, failed 1 / 1 = 1.0000
process 0: fitting 3dmm ...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:06<00:00,  6.76s/it]
all failed image names: ['data/raw/videos/bipolar_2.mp4']
All finished!
Traceback (most recent call last):
  File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/runs/binarizer_nerf.py", line 338, in <module>
    binarizer.parse(video_id)
  File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/runs/binarizer_nerf.py", line 317, in parse
    ret = load_processed_data(processed_dir)
  File "/home/europa/work/video_deep_fake/GeneFacePlusPlus/data_gen/runs/binarizer_nerf.py", line 200, in load_processed_data
    assert os.path.exists(background_img_name)
AssertionError

[858hxa]

I met the similar issue, have you solved it?

[vincentrsr]

I met the similar issue, have you solved it?

[vincentrsr]

I met the similar issue, have you solved it?

[vincentrsr]

I met the similar issue, have you solved it?

[MohitPanpaliya]

I met the similar issue, have you solved it?
The error is because bg.jpg is not present in the directory GeneFacePlusPlus/data/processed/videos/${VIDEO_ID}/bg.jpg
once again try the complete data preprocessing step
it worked for me

[2hiTee]

I met the similar issue, have you solved it?
The error is because bg.jpg is not present in the directory GeneFacePlusPlus/data/processed/videos/${VIDEO_ID}/bg.jpg
once again try the complete data preprocessing step
it worked for me

thanks for your reply, so how to extract bg.jpg