OpenCL
OpenCL is the biggest change to QC and Snow Leopard in general. It is most immediately being exploited by graphics related applications, no surprise as the chief developers have been GPU manufacturers. Other types of hardware, such as DSL chips can also be levered by OpenCL, so the possibilities only begin historically with GPUs and CPUs. It won't stop there. QC wise the OpenCL programming patch takes CL kernels and allows them to be rendered via Mesh Renderers.This is seriously cutting edge technology; when it works it can work blazingly fast. When one blunders into OpenCL memory leaks, it can create total snarl ups and other less than desirable traits.
Kernels
Apart from the Sample Code that Apple have made available for the purposes of exploring some of the various applications of parallel processing that OpenCL provides, nVidia provides an SDK that provides a CLI and to some extent GUI to its chip specific OpenCL, namely CUDA, more about which elsewhere. The examples employed here have been drawn from the in built templates installed with QC 4 and the Sample Code made available on Apple's student developer site. Kernels are the core routines that OpenCL runs. Apple's Sample Code OpenCL projects provide a good starting point for getting to grips with the potential of this framework .
Examples
Please note, pretty much everything here is based upon code samples from the Developer list or else is using an edited version of a default SL only patch. What I'm trying to say is play carefully. These items have not been soak tested and proved. All I know is that I can run them on a fairly low end Intel Mac, with a by now pretty much entry level OpenCL capable GPU. I also know that they have been running when crashes occurred and have themselves caused crashes, especially in regards of audio interactivity.
It seems to me that iTunes and Quartz Composer are no longer such good bed fellows, although developing audio interactive compositions can be faciltated by running SoundFlowerBed, which I've begun to depend upon when working in SL QC 4.
Snow
Snow, one of Apple's OpenCL code samples, was pretty much my first ever introduction to OpenCL in QC 4, not withstanding the fact that the Template Chooser defaults to the Mesh Filter visualizer by default. Somehow or another I managed to transplant the key Particle System into Jelly, along with some .dae for the Core action Mesh Renders to turn that into a purely OpenCL driven visualizer. I also managed to get a meaningful visualizer out of Snow itself, SnowViz. The Particle System is made up both Core Image routines and CL Kernels. This does pass the 10.6.2 update test - it continues to work in iTunes.
Please note , this, no doubt, is not what was originally intended to be the use of the example Particle System and well, there is a possibility of freeze or even crash.
The file should be installed to ~/Library/Compositions. Requires iTunes visualizer on 10.6.x .
Aurora
A mystery item amongst other code samples that demonstrated various aspects of OpenCL. I'm still researching just how this can be used, but have managed to prune what is a pretty complex set of nested iterations of Meshes into a visualizer that runs well on my iMac. Funny thing is it has left and right arrow keys set to change the filter employed, which kind of clashes with the iTunes defaults.
There is absolutely no doubt that the future of computing on Apple hardware is going to be inextricably linked in with chipset leap-frogging for quite a while. Even a fairly up to date iMac is lagging behind what OpenCL currently best works upon, namely faster multi CPUs and GPUs. The net result is likely as not going to be a slightly degraded performance upon the OpenCL front in particular for low end Mac users, although that doesn't by any means have to impact upon the general user experience as such. Although this does pass the 10.6.2 update test, there is much about this composition that I am still finding out about, much of the composition only realy works audio interactively outside of iTunes.
Aurora3 should be installed to ~/Library/Compositions. Requires iTunes visualizer on 10.6.x .
The following movie shows fairly well how Aurora looks by default. It is protocolised out of the box to be a music visualizer. However, for a number of reasons, primarily down to the way the Fog environment and the nested Environmental Sphere and Mesh Render Macro are setup, Aurora's performance external to iTunes is far livelier than when presented as it is ostensibly meant to be presented, within iTunes itself. Puzzling indeed. This , more than any other visualizer I've ever worked upon has prompted me to alter my standard working practice, namely to stick with the iTunes visualization running in ITunes whilst changing the file in QC 4. Using SoundFlower is a real boon as it removes that necessity and lessens the likelihood of what OpenCL related routines seem to be especially vulnerable to when rendered in some way or another audio-interactively, namely freeze ups.
Freeze ups are nothing especially new, but they don't signal that all is comfortably or tightly engineered on some point or another. OpenCL routines almost seem to demand that any visualization depending upon them have a health warning sticker.
Update, I have now managed to reduce the Aurora example down to a music visualizer that works well upon 10.6.x, that despite the fact that 10.6.2 update seems to have accentuated a pre-existing problem with iteration routines. Whatever, this is published up with all the usual caveats concerning cutting edge technology and so forth. The accompanying zip file contains all the files required and should be installed to ~/Library/Compositions.
I was reading somewhere recently that our brain's neurons fire synchronistically with music, so I tired to bring that kind of cybernetic feel to the visualizer resulting.
To achieve this, the OpenCL routines used in the original code sample were simply ditched. This meant losing the mesh ribbon effect.
It is no doubt down to my still learning about OpenCL, but I kept getting lots of Console messages about one thing or another not being rendered due to OpenCL not being fired up. I also got a lot of unclosed or unfulfilled OpenCL error messages too. I do find some OpenCL routines work very nicely indeed, but it doesn't seem to take much to cause the System to freeze up. The diciness of the interaction between iTunes and Quartz Composer compositions and OpenCL is especially worrying. The last thing one wants to do is to create some visulaizer that is mostly remembered for crashing a user's computer.
I shall leave you, dear reader, to venture over to Apple's Sample Code library and download that code yourself and experiment a little. I shan't be presenting a download link to what I've done actually with Aurora without my pruning it down [ a whole lot ], as applies to Aurora3, link above.
Suffice to say that Aurora has great potential and seems to be one of those items that is sent out to see who can get the best out of what is hidden under the fog and the environmental haze. Either that , or else it's one of those items that works brilliantly upon a particular configuration , but doesn't float on other graphics cards or hardware configurations. Once one casts a little light within Aurora, it really does start to shine. Huge.
It also needs a lot of careful handling, as does most OpenCL graphics work in Quartz Composer. A tiny little bit too much input can cause long term processing freeze ups. I wonder just how much that is related to some state of unrefinement in regards of garbage collection or memory management within OpenCL, but haven't, as yet, enquired deeply enough of ongoing processes to comment beyond that speculation.
2D Smoke
This uses one of the new patches , 2D Fluid Simulation, which is a patch meriting an editing study in itself, [ use the Actions menu on the Patch Library panel ]. It is the patch that has been most often in editing state at the time of a system or QC crash. This usually turns out to be related to live rendering when altering input amounts. It also happens more frequently when running QC 4 in 32 bit mode. This code example reminds me a great deal of work previously exemplified by other means in QC 3 / Leopard. That said, this is pretty nifty, although, I can't help but wonder just how best one should structure compositions when drawing upon OpenCL. Funnily enough, since the 10.6.2 update this is one composition that seems to work better than before.
The original code sample from Apple is absolutely chock a block full of OpenCL kernels and certainly makes for a good way of studying up on a working example of OpenCL visualization.
The study example, 2D Smoke can easily be modified to take on board different graphical inputs.
All compositions created in QC 4 using only Apple's default patches. All movies rendered offline using Quartz Crystal from Kineme and output with QuickTimePro.