Friday, June 18, 2010

F#, WPF and XAML, oh my!

Anyone why tries to use WPF with F# will know, or at very least, soon find out, that F# doesn't get along with WPF very well. Specifically, binding F# code to a XAML control is a nasty ordeal.

F# doesn't currently have any support for codeDOM, which is needed to create the magic glue between the XAML and the code. Thus, we must create this glue ourselves.

Tearing into a C# WPF project revealed the secrets of the glue - the XAML is compiled into a BAML resource in the final executable, which is then bound to the control class through Application.LoadComponent(). At first glance, it would seem that this would be easily reproducible on F# - simply include the XAML as a resource, get its Uri, and pass that into the LoadComponent. This, however, will not work. Why? LoadComponent requires that the XAML be compiled into BAML, which doesn't happen just from putting the XAML into a resource. In fact, any attempt to compile the XAML during build will likely fail, since doing so involves codeDOM, which F# doesn't support.

So what do we do? We could use a XamlReader to load the GUI from a loose XAML file, but then we loose the capability to use code-behind, which means we have to bind all the controls the hard way. Or do we?...

As it turns out, it *is* possible to use code-behind with loose XAML. It just takes some clever workarounds as well as very explicit namespace definitions in the XAML. This is the subject of this post.

The first thing to do is modify out XAML to be able to locate the code-behind in our executable. In order to do this, we have to explicitly list both the namespace and the assembly, as well as bind directly to the specific derived class. This basically requires that we change this:

<Window x:Class="Flam4GUI.MainWindow"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:my="clr-namespace:Flam4GUI"
....


into this:

<app:MainWindow
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:app="clr-namespace:Flam4GUI;assembly=Flam4OCL"
....


This brings us to our second problem: The standard convention is to call LoadComponent() from the constructor of the control class. However, this doesn't work with XamlReader.Load() since it creates a new control class and binds that to the XAML. Thus, if one were to call XamlReader.Load() inside the constructor, it would callback to new ControlClass(), which would call XamlReader.Load(), which would call new ControlClass(), and so forth until the stack overflowed. Thus, we need to create a new static method in the control class, which would call XamlReader.Load(), which would call new ControlClass(), which would return a neatly bound ControlClass object. Here's the code for this:


type MainWindow() as s=
inherit Window()


let mutable _contentLoaded = false
do s.InitializeComponent()

member s.InitializeComponent()=
if _contentLoaded = false then
_contentLoaded <- true
()

static member Create() =
let fs = new FileStream(@".\Resources\MainWindow.xaml",FileMode.Open)
System.Windows.Markup.XamlReader.Load(fs) :?> (MainWindow)


We can now open the window like so:


let window = MainWindow.Create()
(new Application()).Run(window)


The best part is that any methods in MainWindow can be bound to in the XAML, just as you would expect. Thus, we have reached our goal, using WPF + XAML with code-behind, with nothing but F#.

Friday, May 7, 2010

Parallel Compilation with OpenCL II

Continuing from the last post, we will now go over the communication code in Flam4OCL.

Starting with the master process, the CompilerService class is instantiated once for each compiler slave process. The primary constructor, new CompilerService(deviceName) starts a new complier slave process based on the device name string passed in. Once the main program exits, it closes CompilerService with Dispose() in which CompilerService sends the quit message "END" to the associated process, waits for that process to read the message, then closes the pipe.

Points of interest are the pipeName, which is dynamically generated with Guid.NewGuid() to ensure that nothing else on the system collides with it, and the readMessage function, which exhibits the pecularity of Async.RunSynchronously. While at first glance, running an asynchronous read in a synchronous fashion might make no sense at all, it serves a vital purpose: During a normal pipe.Read, the CPU spins until it's read the number of bytes requested. This means that if one were to use pipe.Read() to listen for the next message, the CPU will spend it's time eagerly awaiting an update from the pipe rather than doing something more important, such as running the compile service so that the update can be sent in the first place! By contrast, pipe.AsyncRead yields the CPU until there is enough data in the pipe to fulfill the read request. Hence, it can listen for a message without hogging system resources. Since the master program can't do much until it receives it's compiled binary, the RunSynchronously simply makes it (or rather, whichever thread in the program that is running the class) yield the CPU and do nothing until the compilation is finished.

The code for the slave process works very similarly to the master program code, the one big difference being that it uses NamedPipeClientStream instead of NamedPipeServerStream, since it didn't create the pipe, but is merely using the pipe provided by the master program. Since the compile process could easily be idling for a long time between compile requests, it's even more important that it's readMessage is asynchronous, so that it actually idles rather than running full throttle in neutral.

Once the compilation is complete, there is a "peculiarity" involved with getting it to actually be run. After creating the program with clCreateProgramWithBinary(), you simply call clCreateKernel(), it will fail! To fix this, you have to call clBuildProgram() after clCreateProgramWithBinary(). This seems an odd thing, since the whole point of a binary is that it's already built! It seems, however, that there is some other work involved, likely something to do with building up a symbol table for kernel entry points.

Thursday, May 6, 2010

Parallel Compilation with OpenCL

If you've ever developed with OpenCL, you may have noticed something quite annoying - the runtime compiler is not thread-safe! In an age of computers with ever more cores and GPUs with two orders of magnitude more cores than that, this is really an embarrassment, especially since the entire point of OpenCL is to take advantage of this parallelism!

However, fear not, for there is a work around to this madness. While the runtime compiler may not be thread safe, it is at least process safe, at least for Nvidia's implementation. It may also be process safe for ATI's implementation as well, but I don't have a card available to test this on.

Since two OpenCL compilers can in fact run in parallel if they're created by different programs, all that needs to be done to exploit this is to simply run a compiler program for each CPU and Boom! Problem solved! Right?

Well, not so fast. As with anything else concerning programming, there are details...

First off, Process.Start() is known to be one of the most expensive calls in the Windows API, taking up to a second to execute! This is because there's all manner of overhead for starting a new process, from security concerns to disk access to read in the binary. Thus, the naive method of starting a new process each time we need to compile something is a bad idea.

Instead, we need persistent compile processes, which continue running throughout the lifespan of the master program. Ideally, the master program would start up the processes, then send them bits of source code as it needed things to be compiled. The slave compile processes would then build the source and send the binaries back to the master program, which would execute them. After some time of doing this, the master program would finish, at which point it would need to not only close itself, but close the slave processes as well. Since the standard kill process command doesn't give the process time to clean up, the master will have to send the slave processes a kill message, so that they can close themselves.

The key to all this is to find an API through which the processes can send messages back and forth asynchronously and without mangling binary data (this rules out stdin/stdout. In addition, it would be nice if the API also worked with MONO.NET, so that the program can be run on other OSes.

One such API is System.IO.Pipes. Here is the messaging system that flam4OCL uses:

Here's the messaging class used by the master process:
module CompilerDriver

open System
open System.IO
open System.IO.Pipes
open System.Text
open System.Diagnostics

type CompilerService (pipe:NamedPipeServerStream)=
let pipe = pipe
let readMessage() =
async {
let! length = (pipe:>Stream).AsyncRead(sizeof)
let buf = Array.create(BitConverter.ToInt32 (length,0)) 0uy
let! msg = (pipe:>Stream).AsyncRead(buf)
return buf
} |> Async.RunSynchronously
let writeMessage (msg:byte[])=
let lengthBuf = BitConverter.GetBytes (msg.Length)
pipe.Write (Array.append lengthBuf msg,0,msg.Length+sizeof)

new(deviceName) =
//ensure a unique pipe name
let pipeName = Guid.NewGuid().ToString()
let pipe = new NamedPipeServerStream(pipeName,PipeDirection.InOut,1,PipeTransmissionMode.Message)
let compilerProcess = new Process()
let startInfo = new ProcessStartInfo()
startInfo.FileName <- @".\Flam4CompileService.exe"
startInfo.Arguments <- pipeName + " " + "\"" + deviceName + "\""
startInfo.UseShellExecute <- false
compilerProcess.StartInfo <- startInfo
compilerProcess.Start() |> ignore
pipe.WaitForConnection()
new CompilerService(pipe)

interface System.IDisposable with
member s.Dispose() =
writeMessage <| Encoding.UTF8.GetBytes("END")
pipe.WaitForPipeDrain()
pipe.Close()
pipe.Dispose()

member s.Compile (kernel:string) =
let buf = Encoding.UTF8.GetBytes(kernel)
writeMessage buf
pipe.WaitForPipeDrain()
readMessage()

And the slave compiler process:
module Program

open System
open System.IO
open System.IO.Pipes
open System.Text
open Cloo

let standardOCLCompilerOptions = @"-cl-single-precision-constant -cl-denorms-are-zero -cl-mad-enable -cl-no-signed-zeros -cl-fast-relaxed-math"

[<STAThread>]
[<EntryPoint>]
let main args =
//Connect to the pipe
use pipe = new NamedPipeClientStream(args.[0])
pipe.Connect()
pipe.ReadMode <- PipeTransmissionMode.Message
try
let readMessage (pipe:NamedPipeClientStream)=
async {
let! length = (pipe:>Stream).AsyncRead(sizeof)
let buf = Array.create(BitConverter.ToInt32 (length,0)) 0uy
let! msg = (pipe:>Stream).AsyncRead(buf)
return buf
} |> Async.RunSynchronously
let writeMessage (pipe:NamedPipeClientStream) (msg:byte[])=
let lengthBuf = BitConverter.GetBytes (msg.Length)
pipe.Write (Array.append lengthBuf msg,0,msg.Length+sizeof)

let msgToStr buffer =
buffer |> Encoding.UTF8.GetString

//set up the OpenCL compiler for the selected device
let clPlatform = ComputePlatform.Platforms.Item(0)
let clContextProps = new ComputeContextPropertyList(clPlatform)
let clDevice = (clPlatform.Devices |>
(fun devs -> [for device in devs -> device]) |>
List.filter (fun device -> device.Name=args.[1])).[0]
let devList = new System.Collections.Generic.List()
devList.Add clDevice
let dev = new System.Collections.ObjectModel.ReadOnlyCollection(devList)
let clContext = new ComputeContext(dev,clContextProps,null,System.IntPtr 0)

//Listen to pipe and compile recieved kernels until the message "END" is recieved
let rec readLoop() =
let msg = readMessage pipe |> msgToStr
if (msg.Length>0)&&(msg<>"END") then
let clProgram = new ComputeProgram(clContext,msg)
try
clProgram.Build(dev,standardOCLCompilerOptions,null,System.IntPtr 0)
with
| :? Cloo.ComputeException ->
printfn "%s" (clProgram.GetBuildLog(clDevice))
| _ ->
reraise()
writeMessage pipe (clProgram.Binaries.Item(0))
pipe.WaitForPipeDrain()
if (msg<>"END") then readLoop() else () //Tailcall to simulate while loop

readLoop()
finally
pipe.Close()
0


Since Wordpress keeps messing with the code format and eating the HTML attributes I insert to preserve the code in a readable, not flowing off the page format, I think it's best that I stop this post here and continue with the description in the next post.

*EDIT* Seems that Wordpress now strips every attribute from HTML, including style="overflow-x:scroll". This makes it impossible to post code without it flowing off the page. I don't have time for this nonsense, so I've moved the blog over to Blogger instead.

Thursday, April 29, 2010

Thursday, April 15, 2010

*.flam4 Overview

A simple *flam4 file looks like the following:

The base node is <flam4>.  This identifies that the file is in fact in the *.flam4 format, but more importantly, allows the file to (eventually) contain multiple flames, since XML only allows for a single base node.  Ultimately, it will contain some other data.  In particular, there will be a version attribute introduced once the format reaches a point where I can consider maintaining backwards compatibility, allowing the format to evolve over time.

Inside the <flam4> node is the <flame> node.  Currently, you can only have a single <flame> in the file, but in the future it will be more representative of a single frame, such as a keyframe in an animation.  The flame node contains all the necessary data to render a single frame.  It contains the following:

The <viewport> node defines the view area.  Unlike flam3, <viewport> is completely resolution independent, meaning that you don't have to edit the file to render a preview frame, for instance.  Eventually, the viewport will define a minimal coverage area for the fractal, so flam4 will grow the viewport to accommodate the entire specified area, plus some extra at the top and bottom or at the left and right to force a square aspect ratio for whatever resolution the image is to be.  The minimal coverage area is defined simply by the <coefs> needed to map the canonical viewport to the desired image region.  Hopefully, this will simplify some tricky operations in flam3, such as dividing the image into strips for rendering very high resolution images.

The <gamut> node contains information specifying how the raw accumulation buffer is tone mapped to produce the final image.  Right now, this only includes brightness, gamma, and vibrancy, defined very similarly to flam3.  Eventually, this will be expanded to include things like density estimation and advanced (custom) post processing algorithms.

The <code> node contains all necessary code for custom variations (and eventually other things) used by the flame.  Note that you can use an <include> node to import custom code from another file, which allows for a library of standard variations to be included with flam4.

The <palette> node contains the palette.  It has a format attribute that can be set to rgb_unorm, where the standard display range is mapped from 0.0-1.0, rgb_255norm, which maps the standard display range to 0-255, rgb8_hex, in which it is encoded as a hexadecimal string, three bytes or six digits to a color, and the more exotic rgb32_unorm_hex and rgb32_255norm_hex, where a hexadecimal string representing the palette expressed in 32 bit floating point is used.  In all cases (except rgb_hex, which is limited to 0-255 by definition), out of gamut values are perfectly legal, which includes strange things like negative numbers.  Thus, the palette is defined in true HDR.

The last two types of nodes form the core of any flame fractal, the <xform_node> and the <xform>.  These each come in two flavors, the literal definition, and the reference.  This allows us to reuse xforms, and more importantly, to link xform nodes together.

The <xform> nodes are the simplest.  Each <xform> contains a color_index and a color_speed attribute, which control the xform's usage of the palette.  They also contain a <coefs> node, as well as one or more <var> nodes.  Each <xform_node> is allowed to have a single <xform> node, which describes the xform attached to that node in the chaos game.  Literal <xform>s can also be defined outside of any <xform_node>, where they can be only used by an appropriate reference.

<xform_node>s are used to describe the behavior of the chaos game in creating the fractal.  Each literal <xform_node> can (optionally at some point, but required right now) contain a single <xform>, and must contain at least one other <xform_node>, with an attached weight, opacity, and state, from which the <xform_node> for the next iteration will be selected.  These children <xform_node>s can either be empty references to literal <xform_node>s described elsewhere, or can be full literal nodes themselves.

Of the three attributes of any <xform_node> child, which we call a node link, weight and opacity are obvious in usage.  Weight determines the chance that node will be selected from among its siblings when iterating out of a given node, while opacity describes the opacity of the linked node.  State is more complex.

As a point it iterated through the flowchart of xform nodes, it maintains a stack, containing it's "history" at key locations.  Using this stack, an xform link can either save a state, or else revert to a previous point.  In the simplest case, this is used for final xforms, where a point is saved after going through a normal xform, then passed through the final xform, after which it is ploted.  As the control flow goes through the links out of the final xform, the state dictates that it restore the last point on the stack, which produces the behavier of a final xform, which is applied at each iteration to the plotted point, but not fed back into the chaos game.

The actual values for state are hold, push, and pop.  Hold tells the node link to do nothing to the state stack.  Push tells it to push the current point onto the stack and continue, while pop tells it to revert to the topmost point on the stack.  You can also do more advanced operations like pop 2, which tells it to use the point beneath the topmost point on the stack, sorta like hitting undo twice.  You can also do things like pop -1, which is analogous to redo, and push -1, which basically just moves the current index down the stack without replacing the current point.

Applying this, one always enters a final xform with standard opacity and state push, and leaves it with opacity 0 and state pop.  One can also render multiple independent fractals by defining each fractal with its own node network, and then randomly going back and forth with a pop operation, pop 1 one direction and pop -1 the other, which basically forces the active point to be drawn from the current fractal's pool.

Sunday, April 11, 2010

Welcome to the Flam4 Development Blog!

Welcome to the Flam4 Development Blog!  Here we will discuss the trials and tribulations of the ongoing development of the flam4 flame fractal renderer.  We shall also cover the various aspects of usage, in particular the *.flam4 format and how to make the most of it's features.  Come join us as we explore the frontier of flame fractal rendering!

Understanding the *.flam4 format: Part 1: Introduction

When I first envisioned creating a new *.flam4 format to describe the parameters of a particular flame or set of flames, I was pondering how to go about adding support for custom variations in the flame code, so that the user could enter their own fractal equations instead of relying on an increasingly long (and thus hard to maintain) list of predefined variations.  Apophysis had already taken this path, but the problem here was that adding a new variation required writing a plugin, which is basically a .dll containing the function call wrapping the little piece of code that the user wanted to try out.  The downsides of this approach are obvious.  First, the amount of work involved in writing and compiling a plugin was absurd given the scope of a variation, which may be a simple as taking a point (x,y) and returning (sin(x),sin(y)).  A second, more nefarious problem was that in order for any one else to render a flame that uses a given nonstandard variation, the user must first track down and obtain the plugin for that variation, which, owing to the nature of the internet, may indeed be lost, perhaps forever.

Another goal was to replace the chaos tag with something more sensable, but hopefully more powerful.  As it proved, doing this allowed boiling down several special cases in flam3 to use a single general case in flam4, namely post transformations, final xforms, symmetry kind, and pre/post variations, as well as expanding above and beyond.

Finally, I wanted to provide a cleaner format than the flam3 format, which, to put it bluntly, is a textbook example of how not to use XML.

Other topics also came up, even if they haven't (yet!) all been implemented into the *.flam4 format.  These include more control over animations, post processing procedures, and true 3D support.

With this in mind, the next series of posts will detail the anatomy of the *.flam4 format, and how you can (ab)use it to generate images not otherwise possible.