Tag: .net framework

Dew Drop – February 12, 2013 (#1,497)

Top Links

 

Web Development

 

XAML

 

Miscellaneous .NET

 

Design / Methodology / Testing

 

Other Mobile Platforms

 

Podcasts / Screencasts / Videos

 

Community / Events

 

Database

 

SharePoint

 

PowerShell

 

Miscellaneous

 

More Link Collections

 

The Geek Shelf

 

Q&A: Meet Kevin Hazzard & Jason Bock – Authors of Metaprogramming in .NET

Manning Publications has just released a new title, Metaprogramming in .NET by Kevin Hazzard and Jason Bock. The PDF eBook is currently available in its final form (ePub and Kindle formats due on January 15th) and the print book will be available on December 28th. I recently had the opportunity to ask them a few question about the release of the book.

About the Authors

Kevin Hazzard is a Microsoft MVP, consultant, teacher, and developer community leader in the mid-Atlantic USA. Jason Bock is an author, Microsoft MVP, and the leader of the Twin Cities Code Camp.

Alvin: What motivated you to write this book?

Jason: I’ve been fortunate with the books I’ve written in the past that they’ve been based on topics that I was passionate about. However, it takes a lot of effort to write a book, and that’s why I decided to stop doing them for a while after I co-authored “Applied .NET Attributes” in 2003. I said to myself that I wouldn’t write another book unless I loved its theme, and that’s exactly what happened with this book. Metaprogramming sounds like a fancy term, but to me it comes down to using techniques such that you write less code to do more. I wanted to take some of the fear away from Reflection and other dynamic coding techniques that some people have in the .NET world where both C# and VB are primarily used as static languages. Being able to write a book that could do that was very appealing to me.

Kevin: The opportunity to work with Jason was a strong motivator for Kevin. Also, we looked around the blogosphere and the book-o-sphere (is that a real word) and saw that there just wasn’t much available regarding metaprogramming. Some books do exist but none that we could find that addressed anything beyond specific tools and techniques. Our book is also carved up into tools and techniques at the chapter level but we tried hard to make the writing address more fundamental questions about metaprogramming. We wanted to convey that metaprogramming isn’t just a set of tools. It’s a mindset: a way of thinking about software development from the start of design all the way through to the end. We didn’t aim to convey that metaprogramming should be used for all your development though. Software creation should be approached like the development of a car, for example. Some parts are rigid by design, some parts are flexible and some parts can be classified as supple. Combining these in the right way makes the automobile safe, durable and comfortable. Software can be safe, durable and comfortable, too. We simply looked at the way we were using metaprogramming in our own designs and realized that there should be a book that helped others to understand it, too.

Alvin: Who do you see at your target audience? Is the book intended for expert developers only?

Kevin: I remember the first phone call we had with the folks at Manning about this book. Jason and I wrestled with this very question during that call. This isn’t a book with mass appeal like those from Jon Skeet or Jesse Liberty. But it’s also not strictly for experts. Metaprogramming is a way of thinking about software development and you don’t have to be an expert developer to think that way. It’s true that many of the best software developers you’ll ever meet know how to metaprogram and, more importantly, they know when to do it. But this book isn’t necessarily aimed at them either. The content is aimed at the middle: those who have good development skills and those who understand that adaptable, flexible code is usually better than rigid, brittle code. After all, we call it SOFT ware. This book is for those developers who want to develop software that remains pliable after it ships and you don’t have to be an expert to appreciate the value of that.

Alvin: How did you divide up the topics in the book? Do you each have any particular facets of metaprogramming that are strengths or of particular interest?

Jason: That was pretty easy. Kevin and I had strengths in certain areas of the material, so we assigned the chapters based on that level of understanding. It ended up being close to a 50/50 split. Kevin did the introduction, CodeDOM, T4 and DLR chapters and I handled the rest.

Kevin: Our skills and experience were somewhat different. Jason has a lot of experience deep in the .NET CLI that helped him write and Kevin brought real-world experience with technologies like the DLR and T4 to his writing. I think we learned a lot from each other as we wrote the book which was great. A lot of co-authored technology books have a disjoint feel to them because the individual authors deliver their chapters individually to the publisher. But we worked hard to make sure that ours didn’t feel that way. We had weekly conference calls with each other and with our development editors. We edited each others chapters and code. It was a great experience that brought our individual strengths to bear in a collaborative way.

Alvin: Which topics in the book do you use most in your day-to-day development work?

Kevin: Nearly every .NET developer uses the Reflection API every day, whether they realize it or not. That’s why we opened the book with an examination of it. Reflective, metadata-driven programming techniques add tremendous value to the software development process for most developers, including us. This is why WinRT is so important to the future of Microsoft’s tools. Moving the metadata model down into the operating system core means that we can share conventions and abstract expressions between programs after the moment of compilation. That’s a fundamentally "meta" way of thinking and working. I suppose that the other common practices that we use the most are Expression Trees and IL re-writing. Expressions are used throughout LINQ, of course, but they are used rather statically in that context, i.e. not as a metaprogramming tool. However, Expression Trees are the underpinning of a lot of dynamic capability that I add to my code. I used generic Expression<T> and Lazy<T> all over the place in my code, in places where I need flexibility and performance. As we mentioned throughout the book, higher-order functions and lazy evaluation are the hallmarks of functional programming. As it turns out, metaprogramming and functional programming intersect greatly through classes like Expression<T> and Lazy<T>. So I find myself using them all the time both for functional reasons and to make my code more pliable and adaptable. Most of the IL-rewriting that I do is through IOC/DI frameworks that allow for AOP interceptors and such. But I occasionally find myself doing that by hand, so to speak.

Alvin: We heard a lot about the Dynamic Language Runtime when it was released several years ago. Where does it stand today?

Kevin: As you learn about Microsoft’s long-term plans for their languages and tools, it makes sense to limit the investments in the DLR’s way of doing things. What Jim Hugunin and the others on the team did to create the DLR was brilliant for .NET and the DLR will remain a viable platform for dynamic language development for a while longer. However, in a WinRT world, dynamic languages shouldn’t necessarily be implemented through a dynamic core like the DLR. With WinRT in place, Microsoft is likely to choose the path for new implementations that languages like Boo and Nemerle have taken. In fact, they already have. Look at WinJS, Microsoft’s JavaScript implementation for Windows 8. Because WinJS runs atop WinRT, it can co-mingle with F#, C#, C++, and Visual Basic in a single application, for the most part. This gives you the flexibility that the DLR designers were after without the DLR’s ceremony. In the end, language interop is always about sharing metadata and conventions for things like data types. That’s what the DLR is about and so is WinRT. In recent days, I’ve begun porting a Python language implementation over to WinRT as an exercise and it’s going smoothly so far. I know that others are working on porting IronPython over to WinRT, too. In 2013, I predict that you’ll see one or two more major dynamic languages appear on WinRT, in addition to WinJS. Whether Microsoft will be sponsoring any of that development, who knows? Anyone thinking of implementing any language on WinRT should study the DLR first. There are some truly remarkable inventions in there.

Alvin: Did you have access to Microsoft’s language and compiler teams while writing the book?

Kevin: I think we had better access than the average developer and even the average MVP. They knew that we were working on the book and pulled us aside at the MVP Summit to talk about our work. We also exchanged a bunch of e-mails and had some phone calls with various team members. Of course, we stayed true to our Non-Disclosure Agreements and only wrote about the material that was public or was going to become public. The various relevant teams at Microsoft working on WinRT were under a very strict gag order of sorts so we didn’t learn much about it until the rest of the world did. In hindsight, I wish we had shifted more of the book’s content to WinRT and Roslyn based on what we learned but it was a bit late for that. Furthermore, the metaprogramming capabilities in the new development stack are very different from .NET so we would have needed to rethink the book to address all of the changes in full. In the end, we ended up with a very .NET-centric book with an eye toward the future.

Alvin: What does the future hold for metaprogramming in .NET? Do you see some new tools and techniques coming down the line?

Jason: I see Project Roslyn having a big impact in the .NET world, even if a fair amount of .NET developers never use those APIs directly. Having a common set of APIs to analyze, change and compile code will open more doors for .NET developers to do some amazing things. There’s a lot of cool work already done in the .NET world to facilitate metaprogramming, but I can see Roslyn being a core part of that work in a lot of areas.

Kevin: .NET has had rich metadata and a delegate class since day one. Metaprogramming is baked into the soul of .NET, you might say. With the advent of Expression Trees, metaprogramming took a big leap forward. The next frontier is access to the Abstract Syntax Tree (AST) of code. In this respect, we see hints of what will be available in the future through the Roslyn project. The capability to inspect ASTs will come first, allowing some truly revolutionary development tools to be created. The logical step after that is opening access to the ASTs at runtime. When that happens and after we’ve addressed the obvious security concerns, metaprogramming will become much more popular. Reflection against metadata to drive program behavior is fairly common in .NET development today. When ASTs, which are really just algorithmic metadata, become available, we expect developers to embrace this way of working as a natural extension of what they already know. You see this mindset emerging today in jQuery, for example, albeit at a higher level. It’s common in jQuery to test for the existence of a function or even attach one on the fly to accomplish a task. This is a hallmark of dynamic languages but will soon become a common capability in traditionally statically-typed languages. After all, functions are just building blocks that are made up of smaller building blocks. The assembly of functions in the right order and at the right time is really the only thing that separates traditionally imperative languages from declarative ones. The lines between those types of languages are blurring and it will be metaprogramming frameworks like Roslyn and its derivatives that drive it.

 

 

The Building Blocks of a F# Markdown Parser

by Tomas Petricek, author of F# Deep Dives

Editor’s Note: The F# Deep Dives MEAP will be 50% on December 18, 2012. Use code dotd1218au at checkout.

Markdown is a simple text-based markup language that can be used to produce clean HTML and is used by sites such as StackOverflow or GitHub. You can build your very own an efficient parser that can be extended with custom features and that allows you to process the document after parsing. In this article, based on chapter 11 of F# Deep Dives, author Tomas Petricek describes the key elements of such a project, in particular, the representation of a Markdown document.

I’ve been writing a blog for a number of years now. Since the beginning, I wanted the website to use clean and simple HTML code. Initially, I just wrote articles in HTML by hand, but then I became a big fan of Markdown, a simple text-based markup language that can be used to produce clean HTML and is used by sites such as StackOverflow or GitHub. However, none of the existing Markdown implementations supported what I wanted: I needed an efficient parser that can be extended with custom features and that allows me to process the document after parsing. That’s why I decided to write my own parser in F#.

In this article, I’ll describe the key elements of the project. In particular, I’ll discuss the key subject from a functional perspective: the representation of a Markdown document.

You might not need to implement your own text-formatting engine, but you may often face a similar task. Text processing is not only useful when working with external files (test scripts, behavior specifications, or configuration files) but also when processing user inputs in an application (such as commands or calculations).

Introducing the Markdown format

The Markdown format is a markup language that has been designed to be as readable as possible in the plain text form. It is inspired by formatting marks, such as *emphasis*, that are often used in text files, emails, or README documents. It specifies the syntax precisely and thus it is possible to translate Markdown documents to HTML.

Formatting text with Markdown

The formatting of Markdown documents is based on whitespace and common punctuation marks. The document consists of block elements (such as paragraphs, headings, and lists). A block element can contain emphasized text, links, and other formatting. The following sample demonstrates some of the syntax:

Visual F#  
=========
F# is a **programming language** that supports _functional_, as 
well as _object-oriented_ and _imperative_ programming styles. 
Hello world can be written as follows: 

    printfn "Hello world!" 

For more information, see the [F# home page] (http://fsharp.net) or 
read [Real-World Functional Programming](http://manning.com/petricek) 
published by [Manning](http://manning.com).

The document above consists of four block elements. It starts with a heading. The separator "=" is used for first level headings. We can also create second level headings using "-" as a separator. An alternative style uses a certain number of "#" characters at the beginning of the line, so, for example, ## Example is a second-level heading.

The second block is a paragraph, followed by a code sample and one more paragraph. The text in paragraphs is formatted using ** (strong) and _ (emphasis). Both asterisk and underscore can be used for strong and emphasized text—one character means emphasis and two characters means strong text. We can also create hyperlinks, which is demonstrated by the last line.

From a programming language perspective, formats such as Markdown can be viewed as domain specific languages, which is explained in the following sidebar.

Meta: external domain specific languages

The term domain specific languages (DSLs) refers to programming languages that are designed to solve problems from a particular domain or field. DSLs are useful when you need to solve a large number of problems of the same class. In that case, the time spent on developing the DSL will be balanced out by the time that you save when using the DSL to solve particular problems.

DSL can be categorized into two groups. Internal DSLs are embedded in another language (like F# or C#). Functions from the List module with the pipelining operator (|>) can be viewed as a DSL. They solve a specific problem—list processing—and solve it very well without other dependencies.

External DSLs are languages that are not constructed on top of other languages. They may be used as embedded strings (for example, regular expressions or SQL) or as standalone files (including Markdown, configuration files, Makefile, or for example, behavior specifications using language such as Cucumber).

Now that I’ve introduced the Markdown format and domain specific languages in general, let’s look at a number of benefits that we can expect from a Markdown parser written in F#.

Why another Markdown parser?

Markdown is a well-established format and there is a number of existing tools that convert it to HTML. Most of these are written using regular expressions and there are some written for almost any platform, including .NET. So, why do we need yet another processor? Here are a few reasons:

  • Creating a custom syntax extension for Markdown is quite difficult when using an implementation based on regular expressions. It is hard to find where the syntax is being processed, and changing a regular expression can lead to various unexpected interactions.
  • Most of the tools transform Markdown directly to HTML. This makes it hard to add a custom-processing step, for example, to process all code samples in the document before generating HTML.
  • A related problem is that HTML is the only supported output. What if we wanted to turn Markdown documents into another document format, such as Word or LaTeX?
  • Finally, performing a single regular expression replacement may be quite efficient, but, if the processor performs a huge number of them, the code can get quite CPU consuming. A custom implementation may give us better performance.

Let’s now look how we can achieve these goals using F#. The key element of the solution is an elegant functional representation of the document structure.

Representing Markdown documents

When solving problems in functional languages, the first question we need to answer often is: "What data structures do we need to represent the data we work with?" In case of Markdown processor, the data structure represents a document. As discussed earlier, a document consists of blocks of different kinds. Some of the blocks (like paragraphs) may contain additional inline formatting and hyperlinks.

fdeep101

Figure 1 Here you can see how different MarkdownBlock elements and different MarkdownSpan elements are used to format the sample document. All other unmarked text is represented as Literal.

Listing 1 Representation of Markdown document

type MarkdownDocument = list

and MarkdownBlock =  
  | Heading of int * MarkdownSpans
  | Paragraph of MarkdownSpans
  | CodeBlock of list

and MarkdownSpans = list

and MarkdownSpan =  
  | Literal of string
  | InlineCode of string
  | Strong of MarkdownSpans
  | Emphasis of MarkdownSpans
  | HyperLink of MarkdownSpans * string

The types that model Markdown documents are shown and explained in listing 1. I defined the types as a mutually recursive using the and keyword for two reasons. Firstly, the MakdownSpans and MarkdownSpan types are mutually recursive and they both reference each other. Secondly, I wanted to start with a type that represents the entire document rather than starting from the span to make the explanation easier to follow.

Summary

Broadly speaking, this article was about external domain specific languages. An external DSL is a small programming language or document format that has its own syntax and represents some script, document, or command. External DSLs can be used to configure an application, to provide scripting capabilities, customization, and various other tasks.

The domain specific language that we focused on was the Markdown document format. When working with external DSLs, we first write an F# representation of the language and then implement processing of the DSL.

The functional representation that I described in this article is the cornerstone of a new Markdown processor. Other components are all built around this representation. Chapter 3 of F# Deep Dives looks at three additional aspects: writing a parser that turns text into MarkdownDocument, writing an HTML generator that turns MarkdownDocument into a HTML file, and implementing the pre-processing of a document that generates the references section with all of the document links. All of these tasks are built on top of a simple representation using powerful F# features like pattern matching and active patterns.

1. The project can be found at https://github.com/tpetricek/FSharp.Formatting

2. For more information about Markdown, see http://daringfireball.net/projects/markdown

Here are some other Manning titles you might be interested in:

fdeep102

The Real-World Functional Programming

Tomas Petricek with Jon Skeet

fdeep103

HTML5 for .NET Developers

Jim Jackson II and Ian Gilman

fdeep104

IronPython in Action

Michael J. Foord and Christian Muirhead

 

del.icio.us Tags: ,,,
Mastodon
github.com/alvinashcraft