MDX Net - Segment size and Overlap? What do they mean?

[deadniell]

How do you even configure that?

[dogtopus]

Segment size controls how large each chunk of audio data gets loaded onto the GPU. The bigger it is, the less time you spend on swapping data between RAM and VRAM, thus the faster the processing. Also the bigger the segment size, the less it needs to rely on overlapping, thus might give better results. Set it to as large as your VRAM allows.

Overlap presumably controls how much the audio data chunks are overlapped. I just leave it as default and probably won't change it unless there are artifacts in the result that I suspect to be caused by insufficient context given to the model (like gaps in the audio).

[dgoryeo]

Does 256 segment size mean 256 bytes? Or mega bytes?

[jordigoyanes]

@dogtopus I have the same question as @dgoryeo

[dogtopus]

Does 256 segment size mean 256 bytes? Or mega bytes?

It's probably some internal unit, so neither.

[jarredou]

segment size value is the number of STFT frames by audio chunk.

Here are some overlap/segment size benchmarks made by Bas Curtiz from Audio Separation Discord community:

[dgoryeo]

Then as a reference/guideline this can be a method to estimate VRAM useage roughly:

For a 1-second mono audio signal sampled at 44.1 kHz, with a frame size of 1024 samples and 50% overlap, and using a 1024-point FFT:

The audio has 44,100 samples.
With 50% overlap, each frame overlaps the previous one by 512 samples, so the hop size (step between consecutive frames) is 512 samples.
The number of frames is approximately (44100 - 1024) / 512 + 1 ≈ 86 frames.
Memory usage is roughly 86 frames * 1024 FFT points * 16 bytes ≈ 1.39 MB per channel.