Resample RGB images in C++ #29453
Resample RGB images in C++ #29453
Conversation
Agg already has RGB resampling with output to RGBA builtin, so we just need to correctly wire up the corresponding templates. With this RGB resampling mode, we save the extra copy from RGB to RGBA in NumPy land that was required for the previous always-RGBA resampling.
In the case of RGBA, the RGB and A channels are resampled separately, but they are created as a view on the original to pass to the C++ code. The C++ code then copies it to a contiguous buffer, but Agg's RGB resampler supports manually stepping the RGB input by a custom stride. As this step is a template parameter, we can't handle any arbitraray array, but can special case steps of 3 or 4 units, which should cover the common cases of RGB or RGBA-viewed-as-RGB input.
|
Using
Using
The first commit here removes entry 3 by applying Agg templates for RGB, but we still end up allocating a similar size one in The second commit removes the need for that latter copy as well by applying more Agg templates for step size of 3 or 4. Strangely though, memray now attributes more allocations to
I think There are a few other copies that I can find by inspection:
Copies 1, 2, and 4 are 8 bytes, so 800M each, and copy 3 should be 3*byte = 300M. Many of these are replacing the previous version, or otherwise discarded, so I guess they don't count for I believe we can remove copies 2-4 with a port of I haven't committed that last thing, as I didn't finish alpha processing and wasn't sure if we wanted to move it to C++, but I think the numbers indicate this is a good idea. |
|
On a side note, |
|
I think(?) #29776 would essentially remove the need for this PR (as it removes the use of _rgb_to_rgba). |
PR summary
Agg already has RGB resampling with output to RGBA builtin, so we just need to correctly wire up the corresponding templates. With this RGB resampling mode, we save the extra copy from RGB to RGBA in NumPy land that was required for the previous always-RGBA resampling.
With the example from #29434, this saves 800MB of peak memory usage,
which actually works out to two copies of 10000x10000x4 bytes; I'm not sure where the second copy comes from.I was thinking it was 4-byte RGBA, but it's actually 8-byte float64 input, so 1 copy removed as expected.While this does pass tests, I think there may be some room for further improvement, as the input to the C++ code isI have implemented this to save another copy.A[..., :3], which should be a discontiguous view, but then the pybind11 code forces it to be contiguous, which would make another copy. There are some offset and step settings in Agg that I think we can use to avoid this copy as well, but it might require extra template expansions.PR checklist