Code Generation, Served in a Lightweight Process

So, why another code generation tool? If you search for code generators on SourceForge you'll find 897 tools. Surely, out of all these projects there's one that would meet my needs.

Unfortunately, once you sort out all the Java tools, tools solving specific and limited code generation problems (e.g. tools that only generate data access layers), and projects that never got beyond the planning stage, there are very few general purpose code generators for the .NET platform. "Okay," some will say, "but there are tools, right? How many do you need?"

Only one. But here's the final sticking point, I want a tool that fits well into a lightweight iterative development methodology. For me this means:

• Readable templates. I expect to refactor the templates frequently during the development process. Readability supports maintainability.


• Fits well into a continuous build environment. Code generation should be just another task in the build process. Modify the templates, start the build, and away you go.


• No specific tool dependencies. Let me develop my templates in my favorite IDE. I'll be more efficient if I'm not switching back and forth between tools.

Here are the development steps:

1. Test-first develop some classes that seem to fit the requirements.


2. If these classes seem amenable to code generation (more on what I mean by this in another blog-ette), then use the NGraver tool to transform this sample code into starter templates.


3. Test-first develop the templates so that they produce the desired code. Presumably, the more well-factored the code is to begin with, the simpler the transformation code will be.


4. Refactor the templates as needed.

Ward Cunningham, a leading light in the XP movement, summarized these steps very nicely in a recent email:

1. Be concrete


2. Reflect


3. Wield meta-power


4. Consolidate

This could be the NGraver mantra.

Generated by Machines...Maintained by Humans

Machine generated code is often the worst slop you'll ever see. It's unreadable, unmaintainable, and un-extensible. The code often sports comments like, "Tool generated code. Changes may cause incorrect behavior and will be lost if the code is regenerated." But the very first thing we ever want to do with this code is change it.

What if we assumed that generated code would be changed by developers? What if we decided that not only will the developers tweak the code, but that they should be encouraged to do so? That tweaking generated code is part of the natural and healthy lifecycle of an application with generated parts. Now, all of sudden, the code must be readable, maintainable, and extensible.

But what happens when a template, or the data that drives it, changes? Don't you want to regenerate? And won't that overwrite your changes? Maybe. If your generated code is well-factored, then you'll have two reasonable choices given this situation: regenerate and put your changes back into the newly generated code, or simply continue maintaining the code by hand. Pick the one with the least pain.

I'd argue, however, that this kind of choice typically only raises its ugly head in the early stages of the project. For the majority of the application's lifecycle, maintenance by hand will be the obvious choice. The reason is that, over time, requirements will arise that weren't, could not have been, and perhaps should not have been anticipated by the template developers. These are things that fall into the category of "special cases". They don't apply to every instance of code generated by a template, so they don't belong in the template.

Some might argue that "special cases" should never happen. If they do, then you haven't reached the proper level of abstraction. I'm not a believer. I think that every new level of abstraction requires an order of magnitude increase in design savvy. Such levels of design sophistication are difficult to achieve and difficult to grock for those who must maintain it. Well-factored, more concrete designs--machine generated in the first couple of iterations, then maintained by hand--provide a more feasible, maintainable, and time-efficient solution over the lifecycle of the application.

CodeDom-Based Templates: An Undocumented Feature

The SourceFile object provides overloaded Add methods that take a CodeCompileUnit. The methods are:

public void Add(CodeCompileUnit codeUnit)

public void Add(CodeCompileUnit codeUnit, CodeGeneratorOptions options)

public void Add(CodeCompileUnit codeUnit, CodeDomProvider provider)

public void Add(CodeCompileUnit codeUnit, CodeDomProvider provider, CodeGeneratorOptions options)

If the Add(CodeCompileUnit codeUnit) or Add(CodeCompileUnit codeUnit, CodeGeneratorOptions options) methods are used, the SourceFile object will generate source text from the CodeCompileUnit using the CodeDomProvider appropriate for the SourceFile’s CodeType. If the SourceFile can’t map the CodeType to a CodeDomProvider it will throw an error.

NGraver currently only supports the CSharpCodeProvider and the VBCodeProvider. To accommodate other CodeDomProviders, we provide the overloaded Add methods that accept any CodeDomProvider instance.

This feature has been left undocumented because we are not convinced of the utility of templates built using the CodeDom. We are, however, open to users proving us wrong.

"Designer" Templates

On those days when I allow fantasy free reign, I imagine that the best software designers of our age will contribute templates to the NGraver project. NGraver's repository of templates would be a research library for those desiring to study great design. More than that, you could take the design and apply it to a domain familiar to you, seeing how it addresses problems you live with day-to-day.

We're seeding the pot by building two sets of templates that might have come out of the collective mind of Martin Fowler, et al (see The Grand Vision). But wouldn't it be great if we could coax others to contribute their visions? Ahhh...it would be a good thing, a very good thing.

Ladies and Gentlemen, designers of our age, come with your vision and the NGraver team will codify it to your liking.

Template Dev Guide: Sealed Classes and Non-Virtual Members

One of the ways to handle special cases is by extending a class (see Martin Fowler's SpecialCase pattern). This can't be done, however, if the class is sealed. So consider carefully the use of sealed classes in the code generated by templates, it may inhibit the maintainability of the resultant code.

On a similar note, don't be shy about marking properties and methods "virtual". Virtual members give the maintainers more options. They can either override the behavior, or simply hide it with the "new" modifier. Lets look at some code:

  public class BaseClass
  {
    virtual public string VirtualMethod1() {return "base";}

    virtual public string VirtualMethod2() {return "base";}
  }

  public class DerivedClass : BaseClass
  {
    new public string VirtualMethod1() {return "derived";}

    override public string VirtualMethod2() {return "derived";}
  }

Calling the methods on the derived class thusly...

  DerivedClass derivedClass = new DerivedClass();

  Console.WriteLine(derivedClass.VirtualMethod1());
  Console.WriteLine(derivedClass.VirtualMethod2());
  Console.WriteLine(((BaseClass)derivedClass).VirtualMethod1());
  Console.WriteLine(((BaseClass)derivedClass).VirtualMethod2());

...will produce:

  derived
  derived
  base
  derived

Notice that an inherited virtual member can be overridden or hidden (using the "new" keyword). If it is hidden, then the base class' behavior is still available if the object is cast to the base type.

Non-virtual members, however, can only be hidden. While hiding is the right choice in some circumstances, it does leave open the possibility that users of the class will inadvertently call the base member.

Making your properties and methods virtual gives the maintainers the choice: override or hide.

NGraver Generates Design Pattern Implementations (But We're Not The First)

Here's an interesting article from IBM Systems Journal, Automatic code generation from design patterns. This 1996 article describes a tool developed to generate code based on design patterns.

Here's the motivation for the work:

A design pattern only describes a solution to a particular design problem; it is not itself code. Some developers have found it difficult to make the leap from the pattern description to a particular implementation, even though the pattern includes code fragments in the Sample Code section. Others have no trouble translating the pattern into code, but they still find it a chore, especially when they have to do it repeatedly. A design change might require substantial reimplementation, because different design choices in the pattern can lead to vastly different code.

I couldn't have said it better myself.