High memory usage when upscaling a lot of images

[ToniLipponen]

Checklist

• I have tried ALL of the steps mentioned in the troubleshooting guide.
• I have searched for this in the issues tab.

Describe the Bug

When upscaling many images, the memory usage keeps climbing higher and higher until the OS runs out of memory.

To Reproduce

Batch upscale a lot of images and watch the memory usage.

Upscayl Version (or commit hash)

2.11.0

Platform

Linux

OS Version

Arch Linux

GPU Name

Nvidia RTX 2070

Expected Behavior

I think memory should be freed at some point during batch upscaling.

Screenshots

No response

Logs

-

[NayamAmarshe]

How high is it? You didn't mention.

[jasonsansone]

Depends on size of source image, whether jpg or png, etc. I found I have to work on batches of less than 150 images to avoid OOM kills. This is on LXC containers with 4-6GB memory. Regardless of the amount of memory, the usage just steadily climbs until OOM unless you cap the batch size. Memory is never freed until the entire process completes.

[ToniLipponen]

How high is it? You didn't mention.

If I'm upscaling a lot of images, say a thousand, the memory usage can go as high as 10Gb or higher.

[ToniLipponen]

Depends on size of source image, whether jpg or png, etc. I found I have to work on batches of less than 150 images to avoid OOM kills. This is on LXC containers with 4-6GB memory. Regardless of the amount of memory, the usage just steadily climbs until OOM unless you cap the batch size. Memory is never freed until the entire process completes.

So is this by design? I would have thought once an image in the batch is upscaled, the memory should be freed.

[jasonsansone]

I don’t know. I am not a developer. I was sharing my experiences, similar to yours. I agree that this seems to be a memory leak.

[TGS963]

Thanks for reporting this, we'll investigate this

[EZaplata]

Same here on version 2.11 using RX 7900 xt on Endevour OS with Mesa drivers.

I have hit 28gb of RAM usage after processing 4900 images (1920x1080 PNG, no up scaling just re processing using realesr-animveviodev3-x4).

With such a high number another thing that can happen is the progress count goes out of sync as it will freeze up. Minor thing but I though I will bring that up as well.

[vakarian92]

I'm having the same issue, trying to convert more than 1000 images. I'm on Windows with version 2.11.0. I see usage reaching almost 30GB and then crashing (I've got 32). Interesting enough, it happens with both GUI and CLI version upscayl-ncnn.