Oleg Bliznuk, the author of Exocortex's Implosia FX, posted a tip for avoiding repeat loops a few months back using Generate Sample Set.
It's a great tip and one which can generate good time savings over repeat loops. I wanted to test just how much of a saving the tip could provide by calculating a cumulative sum array i.e. given an array of integers, produce another array which gave you the cumulative sum of all elements in the array at any given point.
The conventional way to do that would be to use a simple repeat loop, but as is always the case with ICE, repeat loops are not necessarily the best way to achieve your desired results as ICE's multithreading isn't optimised in that scenario.
Oleg's brilliantly lateral idea was to generate a sample for each array member and create an array on that sample the same size as the element's index. You can then populate that array with all the members of your original array up to that point and perform tasks on that segment of the original array.
I decided to set up a test scene to compare the performance of a conventional repeat loop and Oleg's method. In that test scene, Oleg's method was twice as fast as a repeat loop which is a significant gain.
But, there is a downside - you pay in RAM usage, sometimes to the point where you might end up paging memory. In my sample scene, the repeat loop method used 200Mb RAM for an array of size 50k whereas in the Generate Sample Set scene RAM usage shot up to nearly 10 Gigabytes! The stepping in terms of RAM usage relative to array size (on my machine) went: 5,000: 373Mb; 10,000: 849Mb; 20,000: 2.6Gig; 30,000: 5.5Gig and so on. It looks like there's a step in RAM usage between 10,000 and 20,000 although I'm guessing this might be machine dependent.
It's clear that if you're going to use this method in a compound you need to be careful about the maximum array size you're going to allow and possibly switch over to a conventional repeat loop over a certain threshold.
If you'd like to play with the scene or do your own timings to verify these results it can be downloaded here along with the Cumulative Array Sum compound (2013 SP1).
Subscribe to:
Post Comments (Atom)
I haven't looked at your scene, but I think a slightly simpler approach to doing this calculation is this:
ReplyDeleteAdd as many particles to your point cloud as the input array has entries.
Then, for each particle use "Select Sub Array in Array" from 0 to (Point ID + 1). Hook the result of that into "Array Sum" and the result of that into "Build Array from Set". There's your cumulative array.
However, RAM does shoot up similarly to what you described.
Hi Anonymous - isn't that almost exactly what the original method does with the additional overhead of having to create a vector array for the Add Points node...you're simply substituting Generate Sample Set with Add Points? Internally, Select Sub Array in Array is using the same mechanism as the original compound. See here:
ReplyDeletehttp://www.exch.demon.co.uk/xsi/files/gen_sample_test2.rar