Nermerle, Nitra, Parsing and Functional Programming

 Monday, March 30, 2015
Functional Programming
I started looking into Functional Programming about ten years ago and when F# 1.0 came out, I decided to start paying attention again. I initially explored it as part of a project I had embarked upon to parse a DSL and provide a suitable development environment that a non-developer could use to customize my application. It was a lofty goal and ultimately was scrapped in favor of a much stripped down model that was driven via web forms and a small subset of the features, but the DSL was still available for more complex needs.
Parsing HLSL
Recently that need arose again. I've been messing around with WPF Shader Effects and 2D sprites. No, I'm not developing a game (sorry, no real interest here). The biggest frustration for me has been the lack of debugging information available. My usual process for exploring new languages is a combination of reading/research and poking/reverse engineering. I was the kid that took apart every piece of broken/outdated electronics growing up. After picking up "The Basics", I usually dive right in and write some (terrible) code to see what it does. I'll also read/step through open source code that I think I conceptually understand to validate my understanding.
Hlsl makes my learning process more difficult. I can't step through the code; I can't watch. The way it encodes colors into float4 values is sufficiently different from what you'd expect that predicting what the value would be requires you to know about things like Premultiplied Alpha, mapping textures to float values between 0 and 1. Your college calculus class will come in handy even if you did poorly as most things related to programming/math require you to have a good understanding of how it works even if you don't remember the details. Wolfram|Alpha will will help you if you've forgotten the specifics.
Built-in IDE tools provide barely any information when the HLSL files are rendered by WPF. I'm not sure how valuable they are outside of that, but they're useless here. The larger tools available from NVidia/Intel are overkill and mostly don't cover the areas I was hoping (the NVidia product simply didn't support most of its features on my particular GPU ... Notebooks).
Building a narrow-purpose IDE
I started out exploring WPF ShaderEffects using the wonderful Shazzam. Unfortunately, at the time of this writing, downloading the tool from his site isn't possible but a slightly outdated version (in code only) can be acquired from CodePlex. I tweaked it a bit for my purposes and created a shader effect that I mostly liked. Getting to the fine details, though, was difficult and I knew I'd need more insight into what was going on.
Enter Nemerle, F# and Nitra
The first thing I wanted was better parsing of the code before compile. I went digging around and found a syntax highlighting definition for HLSL. The language has some unique elements, such as Vectors whos parts are accessed via array notation, or .xyzw .agbr notation. It's quite elegant, but my eyes weren't naturally parsing it coming from languages that didn't do that.
The rabbit hole began there. The editor used by Shazzam was a way old version of that used by SharpDevelop. It's since been updated and the core text editor can be grabbed independently from NuGet as AvalonEdit. It works quite differently from the version offered by Shazzam, and syntax highlighting is much more powerful. So I created a Syntax Highlighting Definition for AvalonEdit that would color the .argb items as a variant of gray, red, green and blue and used shades of gray with underlining for the wxyz (I'll get the definition out on GitHub when I have a chance and link to it here).
The Shazzam tool was also missing Intellisense and some of the code signals that Visual Studio provides, so I went looking for a solution there knowing that my old friend F# was probably going to find a way back into my life for this one.
Surprisingly, it was Nemerle and the [Nitra Project from JetBrains]|(http://blog.jetbrains.com/blog/2013/11/12/an-introduction-to-nitra/) that got my attention this time. Tom Jones' Getting Started with JetBrains Nitra specifically covered parsing HLSL! Thanks for that!
It lacked the ability to understand shader2D and had no preprocessor support, but after adding shader2D, I now had a library that easily integrated with AvalonEdit to provide syntax highlighting failures and guidance for repair... All from a concise syntax file.
I wanted something a little more. Inspired a while ago by Bret Victor's Ted Talk, I decided to see if I could create an IDE that would give useful feedback about how the program works while it is being developed. HLSL is a limited purpose language, and WPF ShaderEffects limitations limit it even further. It's the perfect target for an IDE that gives intelligent feedback about the program being written while it is being written. By providing sample inputs and the ability to provide custom inputs for the function, a person can see what the outcome of the code would be in various scenarios,... as close to live as possible.
At this point, I'm building a new HLSL parser specifically targetting the limitations imposed by WPF to ensure that code syntax and rules are followed carefully.
I'm then going to start evaluating ...
Enter the God Awful Visitor Pattern
I'm an OOP guy most of the time. The pattern fits well much of the time and most .Net programmers are comfortable with it. However, The Visitor Pattern hurts my brain.
C# lacks Multiple Dispatch without the DLR. I'm not interested in passing type safety all the way to runtime. The Visitor Pattern is the way around that problem. The pattern is relatively straight forward, but figuring out its intent by looking at the code is painful. F# and Nemerle have pattern matching capabilities and allow multiple dispatch. The resulting code is readable with a small reference to the syntax (Nemerle is easier to read coming from C# than F#, IMHO) and the amount of resulting code is substantially less.
My Plan for Next Time
Unfortunately, this is not my day job. It's Saturday/Sunday Early Morning/Late Evening work. I have no code samples at the moment because I just moved my F# code to Nemerle (and it's so much easier to understand). I hope to spend some time with the parser next weekend when I will have a little more time to poke at it. If I get it to the point I want it, I'll write it up and thow it out on GitHub.
Until then ... Don't panic!
Nemerle documentation can be found here.
Nitra syntax, not terribly well documented, but somewhat easy to follow after going through the Nemerle documentation.

No comments :

Post a Comment

Subscribe to: Post Comments ( Atom )