Delphi Programming

and software in general.

Friday, September 2, 2011

Forms and Data Entry Validation - Part 1

This is not an article about LiveBinding. I was once hoping it was going to be, but instead it has become an alternative to LiveBinding. If anything, it is about compile-time binding and quality assuring the data input from of your users.

Forms, forms, forms...

How many forms have you created?  Chance is - quite a few - and what do they have in common?   If people type rubbish, your data becomes rubbish.  So - what do you do?  You validate the input to prevent rubbish getting into the system.  You do... don't you? Sure you do!

When do you validate it?  When someone clicks Submit or OK?  Right - then you have to go through the input, field by field, and first ensure that what the user typed in actually is understandable in the current context - such as no funny characters in an integer - and sometimes you have to check  the values against each other for logical states. If someone said they took three melons, their combined weight should at least be bigger than zero, and blue shirts don't go well with pink pants, and what else not.

If the user typed in rubbish - you have to inform him or her so that it can be corrected.

Been there, done that

There is a certain amount of logic in this scene that we keep recreating scaffolding for.  Stuffing things into listboxes, formatting and filling in the values, validation of numbers and dates, converting enumerated types into strings (and back again). If you want the dialog to be slick - you might even want to validate as you go, which means eventhandlers for focus changes, keys pressed, UI items clicked, dropped down and selected, also adding to all the scaffolding code.

Some time ago, I had to create yet another dialog.  Lines and lines of housekeeping code that surround the real validation logic.  And naturally I don't have to be clearvoyant to foresee numerous more such dialogs, as it is a major part of writing applications that deal with configuration, input and control.

So - I thought to myself - can I spend a little time now, and save a lot of time later?  Dangerous, innit, thinking like that...  suddenly you could find yourself writing a framework, and we all know what happens to frameworks, right?  They turn to endless amounts of code written with good intentions of handling the unexpected, covering functionality you won't ever need, and at some point collapse on themselves to become a black hole of sketchily documented (since noone updated the docs as new features got added) , and hastily changed (since you always are in a hurry for that extra functionality) code.  And when someone else misread your framework intentions and applied it like a hammer to a screw - it just doesn't end well.

Narrowing down the scope

Hence - Sticking with the KISS principle, I have decided to try to make it independent of other libraries, and limit what I implement to basic functionality while attempting to allow for future expansion.

I am going to create a TInput<T> that wraps a GUI control.  To put it simply - a TInput that points to a specific TEdit, and takes care of stuffing values from the variable and into the GUI control, and vice versa.  The job of that TInput<T> is the present the value correctly, and to ensure that what ever is written into that TEdit, can be converted into an integer.

I will also create a TInputList that is a collection of TInput<T>s, that will have the job of going through the list to fill the controls, to validate the contents, and finally - if all input is syntactically correct - semantically validate the input for logical correctness.

Some of the code that I will present here, is probably centric to the type of data that I work on.  For me, an input form will  typically wrap an object with a set of properties that reflect a row or set of related rows in a database.  Why am I not using data aware controls?  Mostly because the applications we create actually can't write to the database themselves, except through calling stored procedures that perform more magic before, during, or after the data has been written.  For that reason, the TInputList will be a TInputList<T>, and the TInputList<T> will have a property Current:T that I can populate, and each TInput<T> will know that it is member of a TInputList<T>, so that it can kick of the necessary actions for stuff to get validated.

[kom-pli-kei-tid]

By now you have probably thought to yourself: TEdit?  What about the other controls?

Because there are a number of input types, and a number of controls, and these make a number of combinations. TEdit/Double, TEdit/Integer, TEdit/String, and TEdit/Enum is already a list, and I haven't even mentioned TComboBox yet,- so it is obvious that TInputList<T> has to be polymorphic.

This brings us to the first part of complicated - creating a set of generic and polymorphic classes.  Generics in Delphi XE still don't to well with forward declarations, and to create polymorphic parent/children lists, it really helps to be able to forward declare.

After some consideration, I have chosen to use an abstract class without generics as my inner base class.  TAbstractInput will know nothing about the data type we want to work with, nor will it know anything about the control type.  All TAbstractInput will do, is define the virtual abstract methods that will be our type agnostic operators or verbs and queries, if you like.  Hence, our TInputList will use TAbstractInput as its element type.

/// <summary> TAbstractInput defines the bare minimum base class for our list of inputs <summary>
  TAbstractInput = class abstract
  private
  protected
    function GetEdited: Boolean; virtual; abstract;
    procedure SetEdited(const Value: Boolean); virtual; abstract;
    function GetEnabled: Boolean; virtual; abstract;
    procedure SetEnabled(const Value: Boolean); virtual; abstract;
    function ControlValueIsValid:Boolean; virtual; abstract;
    function VariableValueIsValid:Boolean; virtual; abstract;
    procedure FillControl; virtual; abstract;
    procedure FillVariable; virtual; abstract;
    procedure SetDisabledState; virtual; abstract;
    procedure SetErrorState; virtual; abstract;
    procedure SetNormalState; virtual; abstract;
    procedure SaveNormalState; virtual; abstract;
    procedure Setup; virtual; abstract;
  public
    procedure Clear; virtual; abstract;
    procedure Update; virtual; abstract;
    function Validate: Boolean; virtual; abstract;
    property Edited: Boolean read GetEdited write SetEdited;
    property Enabled: Boolean read GetEnabled write SetEnabled;
  end;

From the outside of the list, we need TInput<T> that expose the correct type that we want to access, so that will be our outer base class type - which knows how to set and get the value, and hence the class that we use to reference an input field.

/// <summary> TInput<T> defines the input wrapper as we want it to be
  /// visible from the outside of our list of controls</summary>
  TInput<T> = class abstract(TAbstractInput)
  private
    FOnCanGetValue: TGetValue<Boolean>;
    procedure SetOnCanGetValue(const Value: TGetValue<Boolean>);
  protected
    function GetValue:T; virtual; abstract;
    procedure SetValue(const Value:T); virtual; abstract;
    function CanGetValue:Boolean; virtual; abstract;
  public
    property Value:T read GetValue write SetValue;
    property OnCanGetValue: TGetValue<Boolean> read FOnCanGetValue write SetOnCanGetValue;
  end;
Please note that this is a simplified view of TInput<T> class.

Inside TInputList, I will subclass TInput<T> again, and add knowledge of the controls.  In fact, I will create several subclasses that handle type conversions for each data type and control type, but instead of having the user instantiate all these different class types - I will add factory methods to the TInputList instead.

Here are some excerpts from the declaration of TInputList and the basic control wrapper.
/// <summary> TInputList is a wrapper for all our input controls. </summary>
  TInputList<IT:class, constructor> = class(TList<TAbstractInput>)
  ... 
  public
    type
      /// <summary> This is our core input control wrapper on which we base wrappers for specific controls </summary>
      TInputControl<TCtrl:class; SVT, CVT> = class(TInput<SVT>)
      private
        FController: TInputList<IT>;
        FControl: TCtrl;
        FValue: SVT;
        ...  
      end;
  end;

Properties and Binding

This is the second part of complicated. Will I be using the XE2 LiveBinding? No. IMO, LiveBinding uses the least desirable method to bind a property for setting and getting. I lamented this in my previous article, Finding yourself in a property bind. In my opinion, LiveBinding is a good idea that is implemented in the wrong way, and in it's current form will be vulnerable to property and variable name changes during refactoring. In addition, it appears that LiveBinding is not quite mature yet. Then there is the fact that XE and older, doesn't have LiveBinding.

After some experimentation, I came to the conclusion that even if it appears to be more elegant to use visitors or observers and RTTI binding, I will get more flexibility, readability, and maintainability by using anonymous methods.

Anonymous methods allow me to do manipulation of the value before it is set/get, and allow the setter/getter events to have side effects. It also ensures that all references are validated compile-time. It will not guarantee protection from referencing the wrong properties and variables, but they will at least be of the right type, and actually exist.

Since my primary development platform is Windows, I am a VCL developer - and when I started this little project, I had only VCL in mind. However, as the code matured, I found that I might want to be able to use this for FireMonkey as well. That still remains to be seen as FireMonkey still smell of Baboon.

Still, the core logic is platform agnostic, and the VCL bits are separated into a unit of their own.

Here is an excerpt from the VCL implementation with complete declarations.
TInputListVCL<IT:class, constructor> = class(TInputList<IT>)
  public
    type
      TInputControlVCL<TCtrl:TWinControl; SVT, CVT> = class(TInputList<IT>.TInputControl<TCtrl, SVT, CVT>)
      protected
        procedure ControlEnable(const aState:Boolean); override;
        function ControlEnabled:Boolean; override;
        procedure ControlSetFocus(const aFocused:Boolean); override;
      end;

      /// <summary> Basic wrapper for a TEdit </summary>
      TEditTemplate<SVT> = class abstract(TInputControlVCL<TEdit, SVT, String>)
      private
        FNormalColor: TColor;
      protected
        procedure SetControlValue(const Control:TEdit; const v:String); override;
        function GetControlValue(const Control:TEdit): String; override;
        function ControlValueAsString:String; override;
        procedure SetErrorState; override;
        procedure SetNormalState; override;
        procedure SaveNormalState; override;
        procedure SetDisabledState; override;
      public
        procedure Clear; override;
        procedure Setup; override;
      end;

      /// <summary> TEdit wrapper for editing a string </summary>
      TEditString = class(TEditTemplate<String>)
      protected
        function ConvertControlToVariable(const cv: String; var v:String; var ErrMsg:String):Boolean; override;
        function ConvertVariableToControl(const v:String; var cv:String):Boolean; override;
      end;

      /// <summary> TEdit wrapper for editing a float </summary>
      TEditDouble = class(TEditTemplate<Double>)
      private
        FDecimals: Integer;
      protected
        procedure SetDecimals(const Value: Integer); override;
        function GetDecimals:Integer; override;
        function ConvertControlToVariable(const cv: String; var v:Double; var ErrMsg:String):Boolean; override;
        function ConvertVariableToControl(const v:Double; var cv:String):Boolean; override;
      end;

      ...

  end;

Putting it to use

This will be covered in part 2. Until then, don't forget to try out RAD Studio XE2 and join the RAD Studio World Tour presentations!

10 comments:

  1. I have to agree with you. Actually LiveBinding does not deserve the binding in its name because it's implementation is just a shame compared to equal solutions in other languages.

    I don't know what to think of your implementation but from just looking at the code I expect a huge bunch of binding code in the end. Also I think you are overusing generics here (not to mention the non telling names for your different type parameters). Then again you implement specific classes for like integer, string and so on where you could actually benefit from the generics if you would use TValue (don't tell me it's slow, we are talking about GUI) Actually the way you are going could even be implemented in Delphi 7 or earlier without using generics.

    While I agree that the string approach is not desirable it cannot be don't any different right now if you want to write your software in a declarative way. Personally I don't want to write actual "classic" code to bind some Lastname property to a usercontrol. I just want to define it somewhere, be it naming the control Lastname, Person_Lastname or in any different declarative way. Because that in the end leads to reusable code and that can be shared for multiple projects ("Hey, look, I can reuse my EditUser Dialog") or for different UI elements ("Wow, Bob just created this new awesome twinkling FireMonkey User Dialog with rotating edits...")

    ReplyDelete
  2. I haven't really spent much time with TValue, so that is probably something that I should look at closer.

    I probably also should expand
    IT - InputType
    SVT - SourceValueType
    CVT - ControlValueType

    There still is a few "secrets" to this that I haven't revealed, which hopefully will explain the heavy use of parameters.

    The goal is to have something that empower you to create smart dialogs with a relatively limited amount of code.

    ReplyDelete
  3. Personally I believe that validation should not be part of the input dialog itself. There are some exceptions, as always, most notably avoiding annoying the user with late validation instead of reacting immediately to user input.

    But in general late validation is better and, in many cases, unavoidable. Getting the data from the user is just one slice of the cake, what happens when you start saving, reading, exporting and importing the data? In all those cases you need to validate the data before acting upon it. And code tied to a dialog won't do you any good. And surely you don't want to code something as boring a validation twice, you don't even really want to code it at all.

    Many times, at least in my experience, dialog validation just doesn't cut it because the dialog doesn't know the whole story, just bits and parts. And sometimes you need more data to validate. Sure, you can send that necessary data to the dialog but that violates the principle of 'be a secretive as possible', i.e. don't spread you data around.

    I usually create a TValidation object with at least one method, Validate. The code in TValidation generally turns out really awful but I think that can't be helped because it reflects real-life complexity and that tends to be messy.

    One very nice, an unexpected, boon stemming from TValidation is that with time the code will come to act as documentation for whatever it is you're validating. It will have all the answers to questions like "If A and B are both greater than 10, C must be less - why was that again?". In some cases, my code has become the authority, especially concerning the more exotic aspects.

    So, to briefly recap, validation should, in almost all cases, be done in a centralized place and not in the dialog.

    ReplyDelete
  4. @Dag - In principle, I agree - in practice, centralized validation can be unnecessary for operations only performed in a specific dialog.

    What if you can do both?

    ReplyDelete
  5. In a way you're right of course, late validation can be very annoying and is not perceived as user friendly. Sometimes you can almost hear the user mutter "Why didn't you tell me so in the first place you stupid program!".

    But the risk of dialog validation is that it will start out as a few minor restrictions but before you know it has grown and grown and grown ...

    And no matter how small, when you load/import data the same checks must be performed which means double code.


    And then there is the worst case of dialog validation: the graying out of a control without telling the user why. I can't really think of a single case where graying out is acceptable unless perhaps a hint is used to spell out why the control is disabled. But much better is, I think, to keep the control enabled and tell the user why he/she can't use it if and when it's clicked.


    It's nice to note that although coding for user input and validation is pretty boring, thinking about how it best can be done is not. I often find that striking a balance between being too restrictive (but simple) and too flexible (and complex) is really hard ...

    ReplyDelete
  6. @Stefan - I have been thinking a little about using TValue instead of explicit types, but I wonder: how would I control individual formatting? Number of decimals for a float, as well as different date and time formats?

    ReplyDelete
  7. @Dag - Yes, validation of data can be a very complex exercise.

    I separating the validations into two different categories.

    Basic, i.e. ensuring that a number is a number, a date is a date, etc. and Context - where we have to make decisions about the data in their validity in a contextual relation to other data.

    The Basic validation is something we shouldn't have to re-implement umphteen times.

    The Context validation will be specific to each dialog implementation.

    As for dialog vs load/import validation - and repeated code - this is something that depend a lot on the application and use requirements. I have tried to make the model possible to adapt to different ways of doing the validation.

    ReplyDelete
  8. @Lars: With TValue you can't and I actually think that should not be the purpose of this. I can't say yet because I don't know where this is going, but so far I smell violation of SoC.

    ReplyDelete
  9. @Stefan: Well, I have two concerns. Conversion/validation - which TValue would deal with nicely, and presentation - which TValue won't support. I am not in control of how many decimals my floats would be displayed with. For an input dialog that deals with volumes and weights, as well as other quantitative values - I need to be able to specify a number of decimals.

    I think I will forgo the TValue version for now - and stick with the oldschool approach.

    ReplyDelete
  10. I've had some new ideas - so I am gonna do a little more work before I put it out there.

    ReplyDelete