POV-Ray

The Persistence of Vision Raytracer (POV-Ray).

This is the legacy Bug Tracking System for the POV-Ray project. Bugs listed here are being migrated to our github issue tracker. Please refer to that for new reports or updates to existing ones on this system.

Tasklist

FS#170 - Reduce memory footprint of output image buffer

Attached to Project: POV-Ray
Opened by Christoph Lipka (clipka) - Sunday, 21 November 2010, 09:49 GMT
Last edited by Christoph Lipka (clipka) - Thursday, 09 December 2010, 13:55 GMT
Task Type Unimp. Feature/TODO
Category Source code → Refactoring/Cleanup
Status Closed
Assigned To Christoph Lipka (clipka)
Operating System All
Severity Low
Priority Normal
Reported Version 3.70 beta 39
Due in Version 3.70 release
Due Date 2010-12-31
Percent Complete 100%
Votes 0
Private No

Details

Currently, output image is buffered using POV-Ray’s RGBFT color model and floating point values, leading to a memory consumption of 20 bit per pixel. This is an unproportionally large memory footprint, given that the only further processing performed on the buffered data is conversion of the data to the desired output file format, which will typically use only 3 bytes per pixel (at most 12 bit per pixel).

The situation can be improved by choosing the output image buffer container based on the desired output file format and parameters. To this end, the following code changes should be made:

  • Bundle image file format handler code into classes (typically one per file format)
  • Include a method to determine the best data container type for the image buffer
  • Instantiate the desired image file format handler object prior to rendering, querying the handler object for an image buffer

In addition to picking a suitable image data container from the already existing palette, careful design might also allow to use custom containers to directly pass the data to a library (in case the library provides its own buffering anyway), or write it directly to the output file (e.g. when writing image data to stdout). To make this compatible with multithreaded rendering, the following changes would have to be made:

  • Add code to detect when a row of SMP blocks has been finished, and call a certain method on the image handler object
  • Design the custom containers in such a way that they can buffer any number of unfinished SMP block rows as needed
This task depends upon

Closed by  Christoph Lipka (clipka)
Thursday, 09 December 2010, 13:55 GMT
Reason for closing:  Fixed
Additional comments about closing:  Reported as fixed
Comment by Thorsten Fröhlich (thorsten) - Sunday, 21 November 2010, 10:33 GMT

The storage as float is actually intentional, and it was quite clear when it was designed that it consumes a fair bit of memory. The goal of the output image buffer implementation as float is to allow post-processing (in a future version). There are two good ways to mitigate the memory consumption. The first one is to rather buffer it to disk than memory. The second is to simply wait another two to three years and with the increased spread of 64 bit operating systems, they will take care of it for us. The solution is not to go back to the old style handling of the issue and closely couple image output and render output.

Comment by Chris Cason (chrisc) - Sunday, 21 November 2010, 16:55 GMT

Prior to release we need to ensure that this is buffered on disk by default, except in cases where the image is quite small. It's worth doing a test for image size since in a fast animation (several frames per second) the additional I/O might matter. Probably there ought to be a platform-specific knob that allows setting the threshold.

Comment by Grimbert Jérôme (Le_Forgeron) - Tuesday, 30 November 2010, 15:25 GMT

Aren't we already buffering that information twice ? once in memory, as a collection of pixels, and once in the povstate file, per rendered block ?
Or am I completely lost in the code and observations of the current directory while rendering ?
(I just know there is a big povstate file, growing... its structure currently avoid my knowledge, it seems to grow as blocks are rendered, so maybe instead of a memory array of pixels, we could have a filemapped array of pixels using povstate ? or its own file ( I hate duplication, when avoidable) ?)

Comment by Grimbert Jérôme (Le_Forgeron) - Thursday, 09 December 2010, 12:46 GMT

Checked with beta 40 (take 2): rendering to huge size does not take all the memory anymore.
(tested with 38400 (H) x 51200 (W) )

Loading...