Delphi Programming

and software in general.

Saturday, August 2, 2008

Anonymous Methods - When to use them?

As many others, I am still trying to wrap my head around this. All the hubbub about anon.methods, but lack of direct descriptions of actual use, lead us to take a "Emperor's New Clothes" kind of view on them.

I believe there are some areas where anon.methods will have a significant impact. Not dramatic or revolutionary - but significant.

That said, we do really need something solid which demonstrate the advantages of anon.methods beyond the trivial one-liner demo methods (which easily can be done in the "old ways"). Until we see some, here is some speculation and conjecture from my side...

1. Lambda expressions and Generics
I assume that for generic classes, the compiler generates something to the effect of anon.methods behind the scene. I also assume that anon.methods are one of the building blocks in type inference.

2. Isolation Glue
When you write classes that interact in the traditional OOP way, you often have to derive new classes from the base classes and implement a set of features (read: method override) where you apply your knowledge of the two new classes to create new interaction rules. Very often, such override methods are just a handful of lines. Yet you have to add yet another two classes to achieve it. In concept, this is a similar problem domain to generics - but here we could be talking about trying to make two specific classes (such as a visual and a non-visual) work together without having to reimplement all the plumbing in great explicitness.

Anon.methods can simplify this by allowing one of the classes to implement all the glue using anon.methods instead of having to implement overridden virtual methods in both classes.

3. Threads
Threads today are a fairly elaborate construction project. Anon.methods may enable us to create something similar to fibers. Where the old style single kernel fibers had to deal with manual scheduling on one CPU, today's multi-core aware fiber management code can delegate the "packaged" anon.methods and data (or fibers if you like) to multiple kernels for actual concurrency.

In what way will this differ from threads? Threads are elaborate to design, requiring yet another class descendant to implement the Execute method. You have to manually feed them their data/scope, and manually retrieve any generated content, and manually implement a "tie it all together" sync.point (ie where you are waiting for all threads to complete so that you can continue). They are expensive to set up, kick off and there is a lot of housekeeping involved.

From the top of my head, here are some forms of processing can be compartmentalized using anon.methods without the overhead of multiple thread class implementations: Sorting (key generation/comparison), matrix math / SSD type processing like compression/decompression, and other types of algoritmic code. In theory, any sequential processing that don't have backwards or forwards dependencies in the dataset, would be trivial to parallelize.

The result should be less obstacles to writing code that actually can utilize all your hardware.

• Is it possible to do this with the existing TThread? Yes it is.
• Can anon.methods reduce the complexity of doing it? Most likely, yes.

• Is it is still possible to do horrific mistakes? Yes, but they are the same old mistakes as for regular threads (race, starve, deadlock, data you can't/shouldn't touch, etc), and since you are now defining your thread/fiber in the scope of it's deployer - it may be that the compiler can make more intelligent judgement about the validity of your thread/fiber.

Will anon.methods lead to spaghetti?
I don't think so. Generally speaking, they may make some things clearer as more of the logic can be packed into one class, instead of being spread over multiple related classes. All "generic" code (pardon the pun) can be kept simple and ..ehm.. generic, and you don't have to build a huge inheritance tree where methods are virtualized up the wazzoo.

I think a possible factor in explaining why it is hard to come up with examples that are short, detailed and easily understood, is that we will benefit the most from anonymous methods in code that is anything but trivial.

P.S. In Bart Roozendaal's blog about anon.methods, Thomas Mueller mentions that the good old Turbo Pascal TCollection ForEach and FirstThat will be possible again. That's not a bad example either. I actually missed those a lot when I couldn't use them anymore.

Edit: Make sure to read Jarle Stabell's article on Lambda functions!

Edit 2:
• An older post from Barry Kelly on how how closures presents a challenge in Delphi for Win32. I guess we know by now that they landed on refcounting.
• The "Pascal gets Closures before Java" Reddit thread.

Edit 3:
Wayne Niddery on Understanding Anonymous Methods
Jolyon Smith on Anonymous Methods - When should they be used?
Andreano Lanusse - Tiburon - Anonymous Methods

Edit 4: • Joel Spolsky - Can your programming language do this?

7 comments:

  1. I think I am going to repeat the essence of the comment I wrote on Bart's post - as that touched on something I hadn't really put into writing before.

    Anonymous methods contain context aware code. The neat thing is that you can deliver this contextually aware code into other code that by definition is context agnostic, and have that agnostic code do processing that really is bound to your current context.

    Again - Less worrying about abstract, virtual and overridden methods.

    ReplyDelete
  2. Hei Lars! :-)

    You may find something relevant in this article I wrote about lambda expressions (back when Delphi was still a Borland product).

    http://dn.codegear.com/article/33336

    ReplyDelete
  3. Thanks, Jarle! That was an excellent article! On second thought, I think I might have to upgrade the rating of Delphi's addition of anonymous methods from "significant impact" to "dramatic impact".

    ReplyDelete
  4. We need to make a distinction between when you COULD use them, and when you SHOULD and when they are BETTER than the alternative(s).

    My idea of a compelling example is one that meets all of these criteria. So far they only meet the first.


    With the exception of example 1, your own examples are cases where anonymous methods are one way to do a particular thing, but I don't see that anonymous methods are the unique enabler. They are just another way of doing things that - in Delphi - are already possible in other ways.

    (Aside: Using/relying on TThread when discussing threading in Delphi is a mistake. TThread is not a religious artefact. It is flawed in many ways, but it's perfectly possible, and really quite easy, to devise lighter and more flexible encapsulations of threads in Delphi without using TThread at all)

    The question then is, are anonymous methods a superior approach to those already at our disposal?

    It may certainly be the quickest way, shaving a few seconds or even (GASP!) minutes off of the production of some piece of code, but at the cost of immediate clarity in that code, that I suspect will be returned in spades when people try to read back that code to understand it on numerous occasions, over the years of life of code that follow it's creation.

    There is a myth that less code is inherently easier to read. It may be quicker to read, but it isn't necessarily easier.

    Again, a crucial distinction is missed between "reading" and "comprehending".


    Your first example is - I suspect - the only one that is uniquely achievable with anonymous methods, but this benefit is in producing further language features. If this is to be a benefit more generally, then the enabled features themselves have to be of benefit.

    In other words, this concrete example needs FURTHER examples in order to validate. i.e. concrete examples of Lambda expression and type inference themselves having benefits.


    Spaghetti code?

    I agree, most likely not. But scrambled code - absolutely.

    Or put another way perhaps - "Spaghetti Debugging".

    Imagine stepping into a procedure call and then finding yourself not in a procedure, but in a chunk of code embedded within the parameter list of some otherwise unrelated code.


    But I am going to stop there as I intend writing further on this subject in my own blog at http://www.deltics.co.nz/blog.

    I've got to keep some content for myself.

    :)

    ReplyDelete
  5. @Jolyon: Surely there are more shades to improvement than inferior, equal or superior?
    Personally, I'm willing to settle for "slightly better", as long as it doesn't come at a price not worth paying. I am not seeing a huge price tag on anon.methods yet.

    Anon.methods don't reduce clarity anymore than overridden virtual methods calling inherited methods or the cursed/blessed OnSomeEvent methods that not only have been plugged in outside our code so we have to scrutinize all the components on the form to spot them all, but also may be virtual and overridden. When we look at those alone, we actually have no clue who will execute them and when.

    From the humorous side, at least you can follow the thread in spaghetti debugging. OnEvent methods are more like Gnocchi debugging - Isolated and encapsulated :P

    Also, when debugging - the anon.method usually will be the routine we WANT to debug, not the generic container that call it. It is the generic code that is the unrelated code. The anonymous method will be tightly linked to what we are trying achieve, and the generic code is reduced to pure plumbing.

    I am starting to dislike the term "anonymous" method - it is misleading. If anything, the method is intimately and directly related to what we want to make happen, and - it can be found at the location where we are trying to make it happen. We need a new term, something like Intimate Methods or Embedded Methods.

    I agree with you on the reading vs comprehending bit - but do I find myself trying to simplify my code all the time. Reduce the clutter by breaking code into manageable sized chunks with meaningful names that add context to the code.

    I also agree that the thread example is not the best of examples, but it still is valid. With the CPU architects starting to mumble about dozens, maybe even hundreds of cores - we need to have a long hard think about how we are going to make good use of them.

    On Lambda and type inference: Lisp was first, but PHP and Ruby are also living examples of the power of these mechanisms. You can write a simple symbolic derivation evaluator in Lisp in less than 100 lines (although personally, I would need a lot of help to do so - haven't touched Scheme in 20 years).

    Having the benefits of dynamic languages in a compiled language like Delphi seems like a good thing, me thinks.

    ReplyDelete
  6. The name "anonymous method" to my mind is perfect. It describes the nature of the thing perfectly.

    It's a method and it has no name. No identity, outside of the context in which it is used.

    Which - I suspect - is why the examples offered are all trivial in nature. Because as soon as you depart the bounds of a trivial - and tightly bound - context, identity and name become crucial aspects of your code.

    Not crucial at the time you write it, perhaps, but absolutely indisposable when you return to that code weeks, months or even years later and have to figure out what it was doing - and what you intended it to do - so far removed from the time at which you (or someone else) originally wrote it.

    But that cost - or risk of cost - is, by it's nature, something that will be incurred in the future, so of course you can't see that cost. Yet.

    Just like you can't see the nine stitches that you will save by investing in that one extra stitch up front.

    (An English language adage - "A stitch in time saves nine". It most likely translates into any number of equivalent sentiments in other languages, but I realised that that paragraph above might not make sense if English is not your first language)

    :)


    Your final sentence is interesting, as it goes back around again to The Emperors New Clothes (I was startled when I read that comment of yours as I used the exact same simile in a post being prepared for my own blog! I thought maybe you'd hacked my site! LOL).

    It's not enough (for me) for something to "seem like a good thing". You'll need to show me HOW it is a good thing.

    Lisp, PHP and Ruby maybe good examples of the power of certain mechanisms. But the limited arena (limited in terms of type of application, not numerical user count) in which those languages have found success could itself be seen as a demonstration of the limited applicability of those mechanisms - powerful or not - to the field in which Delphi, specifically, is used.

    ReplyDelete
  7. From my viewpoint, anonymous methods are more stringently bound to context than let's say a virtual method. The anon.methods can only be called by the piece of code that you are calling at the location where you defined the anon.method.

    The biggest "cost" or challenge is probably to learn to identify where it will be appropriate to break away from our current mindset, which is currently oriented around virtual method overrides and event handlers.

    On the topic of examples: It is hard to make one, without having access to the tool and be able to experiment a little.

    I might be tempted to refer to C#'s LINQ. With anonymous methods, LINQ should become possible in Delphi as well.

    On metaphors: I guess "A stitch in time saves nine" is similar to the Norwegian "Bedre føre vár enn etter snar" or "Better to take care upfront, than acting fast afterwards". I am not sure I would have immediately caught the rewrite of it though, so thanks for clarifying.

    I didn't hack your site, but I might have hacked your brain while reading the non-tech posts :) Something in there lead me to believe that you don't lend too much confidence to the Next New Things until you see it in action. With the current lack of "action", the metaphor just plunked down into my brain.

    I am an optimist by nature so I am giving anon.methods the benefit of doubt. I don't really buy that the definition of a pessimist is ... an optimist with long experience. :)

    Looking forward to read your post!

    ReplyDelete