POV-Ray

The Persistence of Vision Raytracer (POV-Ray).

This is the legacy Bug Tracking System for the POV-Ray project. Bugs listed here are being migrated to our github issue tracker. Please refer to that for new reports or updates to existing ones on this system.

Tasklist

FS#172 - Re-implement progressive image output

Attached to Project: POV-Ray
Opened by Christoph Lipka (clipka) - Sunday, 21 November 2010, 10:51 GMT
Last edited by William F Pokorny (wfpokorny) - Thursday, 03 November 2016, 16:27 GMT
Task Type Unimp. Feature/TODO
Category Frontend → Image format
Status Tracked on GitHub
Assigned To No-one
Operating System All
Severity Low
Priority Normal
Reported Version 3.70 beta 39
Due in Version Future release
Due Date Undecided
Percent Complete 0%
Votes 1
Private No

Details

With previous versions of POV-Ray, it was possible to turn off display output, but still assess the output during render by viewing the output file as it was progressively generated. This allowed e.g. to run a long render on a remote machine as a background process, and check the output from time to time via FTP or similar.

This task depends upon

Comment by Christoph Lipka (clipka) - Sunday, 21 November 2010, 10:51 GMT

See  FS#29  for some earlier discussion.

Comment by Grimbert Jérôme (Le_Forgeron) - Sunday, 21 November 2010, 11:24 GMT

I'm not supporting that request (in term of priority, at least, it could be never done), but it might be complex to perform progressive output.
Nevertheless, an additional program on the side (yes, a binary) able to use the state-file to display it (or rather an "instant" copy of it or whatever, if FTP is used, as it would still be extended by the renderer) might be a solution.

Comment by Chris Cason (chrisc) - Sunday, 21 November 2010, 17:19 GMT

If we implement the on-disk image file cache as discussed in  FS#170 , this might be a solution, provided that we have a documented standard format in which we store the data. It would however be up to the user to work out which bits have been rendered (unless we build a blockmap into the file).

One alternative is to define a 'native' file format which is POV-specific and contains sequences of raw RGBFT data blocks written to disk in the sequence that they were received from the rendering engine. Each block would have a header indicating x, y, width and height. Unlike other formats we'd write this data as it arrives rather than waiting until the end of a render. (We already have something kind of like this in the .povstate file, but that consists of POVMS messages and would be quite difficult for any third-party code to interpret).

While the suggested native file format would not be usable by any standard image utilities, this still is attractive in some ways as it provides solutions to a number of requests, including continuous output to stdout and enabling post-processing of our full raw rendered data should anyone care to do so, as well as to this bug report (provided a custom utility is used to view or grab the output). It could also be used for #FS170 provided we provided a reader class that pulled in the requisite blocks from the output file when the image writing code asks for it (this could probably be wrapped in transparently).

This may sound like a lot of work for a non-typical requirement (in respect of this particular bug), however I am strongly in favor of doing something because the amount of memory POV currently uses for large renders is really a problem. While I'm not averse to using memory when it's needed, chewing up for example one or two gigs of RAM to cache data that's not required during a render bothers me.

Comment by Grimbert Jérôme (Le_Forgeron) - Wednesday, 15 December 2010, 15:15 GMT

Looks like the auto-on-disk image file cache has been done with beta 40.
All that remains to do ("lol"), is either documenting the format and hope someone will make a generic tools to display the raw file, or write ourself such tool.
Graphic programming can be such a pain for portability, I wonder if we could benefit from some toolkit ? (and which one ?)
(I have a small preference for C++, otherwise some java might be the way to go to provide a ready to run binary)
Would the LGPL be allowed for such toolkit ? (I have some experience with Fox toolkit, which is LGPL)

Comment by Chris Cason (chrisc) - Wednesday, 15 December 2010, 15:32 GMT

assuming you are alluding to FS #170, the file written is only a temporary file (it's deleted after the final file is written), contains no meta-data indicating the actual dimensions, and also is in whatever native floating-point format the processor uses. so it's not really useful as-is.

however see my response to FS #180; if Jon is willing to make the effort to write the block-based code suggested, we might be able to extend that to do the above (though it would still need to handle the blocks in the order of render rather than by block number).

Comment by Grimbert Jérôme (Le_Forgeron) - Thursday, 30 December 2010, 10:37 GMT

yes, fs#170 was alluded.
Idea of the day: the metadata (in fact the x & y dimensions are all we need, right ?) could be written at the very end of the file, as 2 x 8 bytes, when creating the file.
Each 8 bytes would be a 64 bit unsigned long in whatever order (Intel vs Motorola) suits the coder of that pov-patch. (and to be documented)

e.g. : access from external program would be with fseek(.., -16, SEEK_END) + fread(..., 16,1, ...) to get back the picture's dimensions.
With the dimensions & the size of file (-16), it's easy to know the size of pixel data in bytes. writing code able handle the native floating point is part of the challenge, but with the size of each pixel, it is really possible.
A simpler program would easily be able to read the same architecture.

So, nothing is impossible, but feed-back from original poster would be welcome. (at the current state of code, it would be a post-release update, IMHO)

Comment by Simon (infoised) - Sunday, 26 February 2012, 18:49 GMT

As a frequent user I'll describe my perspective (if this feature is still relevant). A temporary file in a strange format doesn't help at all. In the old version, the file (like partially written png) could be monitored by any standalone image viewer (possibly auto-refreshing) and the image would persist when finished. When povray is called in a loop (common in scientific visualization), the preview windows would just popup all over the place during work. The progressive display allowed to view both complete images and partially rendered with the same existing software of personal choice. Let's not reinvent the wheel.

If I get this correctly, progressive output is problematic with multithreading and depends on the output format? Isn't a good solution to just leave the unrendered parts black (or transparent) and provide optional periodic update of the output file in its proper format?

Comment by Thorsten Fröhlich (thorsten) - Sunday, 26 February 2012, 18:59 GMT

If I get this correctly, progressive output is problematic with multithreading and depends on the output format? Isn't a good solution to just leave the unrendered parts black (or transparent) and provide optional periodic update of the output file in its proper format?

No, it is problematic because you will have to write the *whole* image over and over. The point of a temporary format is not to "reinvent the wheel" as you put it, but to be able to properly resume a render, which you cannot do with image output formats, period.

Comment by William F Pokorny (wfpokorny) - Thursday, 03 November 2016, 16:27 GMT
  • Field changed: Status (New → Tracked on GitHub)

Now tracked on github as issue #144.

Loading...