The Potatoes

(skip to the meat)

As some may know, I’ve been developing my own PHP Model-View-Controller framework off and on for the last year. It started out as a purely educational exercise, and developed into sort of a side-job of its own. In the beginning I knew nothing about the MVC pattern, or even large-scale programming in general. Bootstrapping? Dispatcher? What are those?

As time went by, I learned more and more each day. A large portion of what I’ve learned has come from studying the CakePHP documentation. Despite never having used CakePHP to build a web app, I’d say I’m more familiar with the inner workings of the framework than 90% of the developers that use it on a regular basis.

The entirety of my framework has been rewritten at least 10 times since its first incarnation, and while I focus on perfecting certain features, others start to collect dust or even cease to exist. I go back to implement/rewrite those features and the cycle continues, almost endlessly it seems.

Certain aspects have been pondered for months on end. Things like hierarchical models, input validation, model-to-model relationships, and datasource abstraction just have SO MANY different implementations in SO MANY existing frameworks, trying to decide what approach to take can easily become overwhelming. I could easily write an essay on each of these; there is such a wealth of information on each subject– but I won’t make you sit through that. This article focuses on the topic of validating user input.

The Meat

When it comes to website and application development, the user is a mysterious entity. It is the reason we do all of this in the first place; after all, what is a website with no users? Yet somehow, users also manage to become a programmer’s number one enemy. They do this by being so gosh-darn unpredictable; or dare I say, predictably malevolent.

Even if we scrutinize every line of code we write, security holes form in sometimes the most obscure of places. A great deal of these holes form around points of user input– forms, query strings, you name it. If the user can change it, they can break it. That user registration form may look mighty fancy, but it’s only a matter of time until Little Bobby Tables comes along and starts dividing Winnipeg by the square root of turtles.

Due to the lack of necessary processing power required for geographic-reptilian arithmetic, we must employ what many consider to be the must mundane and tedious aspect of web development, and application development in general: input validation.

I’m going to assume you have a somewhat rudimentary knowledge of what input validation involves (think: lots of IF statements, throw in a few regular expressions). You should also be familiar with the MVC and ActiveRecord (row as model) design patterns.

Let me start off with an example case. Let’s say you have a website that tracks local events, and you want to allow your users to post details about events they’re organizing. We can already see a few models emerging: Event and User. I’d like to expand a little bit on the Event model, let’s give it a few properties:

• Event.start – the start time of the event, in the form of a UNIX timestamp for simplicity
• Event.end – the end time of the event, again a UNIX timestamp

Anyone with experience developing input-enabled applications can immediately identify some restrictions we should put on the start and end properties. These restrictions come in two flavours. I like to call these flavours technical and logical validation rules.

Technical Input Validation

Technical validation is any form of data validation that aims to ensure the application doesn’t break, and I mean break in the most critical sense. If you put in an IF statement to ensure a user-inputted start time is a number before you try to calculate the duration, you are ensuring you don’t get a fatal error due to arithmetic on non-numerical values.

In our Event example, there are two obvious technical validation rules we should define:

• Event.start must be a number
• Event.end must be a number

Technical validation rules must always be satisfied in order for your application to work without risking a crash. They are always the same, regardless of context. Each rule is connected to a single field.

Obviously, these rules alone aren’t going to ensure our website runs as expected. It wouldn’t make much sense to allow users to post an event that started 3 years ago, yet with only these rules in place, they are able to do that. This brings us to logical input validation.

Logical Input Validation

Logical validation is any form of input validation that ensures our application works the way we want it to.

Let’s add a few obvious logical validation rules to our Event model:

1. We shouldn’t allow users to supply a start time that is in the past
2. We shouldn’t allow users to supply an end time that comes before, or is less than, the start time

Ahh, now this is where we begin to encounter problems. We want to allow the user to edit an Event when it is in progress. If an Event is being edited while it’s in progress, the start time they submit will inevitably be in the past, and this contradicts rule 1. This is where logical validation really begins to differ from technical validation.

Logical validation rules must be satisfied in order for your application to work as intended, but failing to satisfy them will not result in a crash. They may change, depending on the context. Logic rules are generally not connected to individual fields (and so error messages are “general”) although allowing them to be connected to fields may allow for less confusing error messages. A logical validation rule must also require successful technical validation of all involved fields before evaluating the logic.

Current Implementation

Most web frameworks I’ve examined have technical validation down-pat. CakePHP uses a simple array of regular expressions, callbacks, and error messages within the model definition itself to automate that portion of the validation process. Also, this is undoubtedly where model validation rules should reside– in the model.

One area in which Cake and all other frameworks I’ve seen drop the ball, however, is handling of logical validation. Or should I say, the lack thereof. The obvious solution is to house this logic in the controller. This works, and it might be easy, but it is not the best way to go about it.

Let’s say our Events application becomes wildly successful. Our users demand a mobile version so they can post new event information on the go with ease. We decide to develop an entire mobile version of the site– in other words, a second application. This second application will obviously share data and models with its desktop counterpart, but will have new stripped-down and streamlined versions of the controllers and views, optimized for viewing and usage by unorthodox devices. We needn’t include that auto-updating Twitter gadget when our target audience doesn’t even support JavaScript!

And so here we come to the grand problem. Our logical validation is being handled by our controller, and we have two different controllers facilitating the same tasks– adding and editing Events. The inevitable code duplication ensues, and the DRY principle is out the window. How do we solve this?

The Grand Solution

Our ultimate goal is to provide a seamless solution to this problem in the form of a new design pattern. This pattern will allow us to perform all validation, technical and logical, from the model, while allowing for situational variance as determined by the controller. To accomplish this, one starts to think of a Situation as being an entity in and of itself, and that’s exactly the theory we’re going to apply.

Let’s construct a simple CakePHP-like validation rule set for our Event model:

$validate = array(
	'start' => 'numeric',
	'end' => 'numeric'
);

These rules take care of our technical validation. A call to Model.validates() will return true if the start and end time of the event are indeed numbers. However, if we want our validator to do more, we’re going to have to give it some information about the current situation– or Context. We do this by adding a parameter to our model’s validate() method, and in a default variable or initiation method of our Event model, we define our context-specific rules.

Author’s note: From here on out, I will be speaking in terms of a generic MVC framework. The following examples will not work in CakePHP, as it has not been written this way.

Let’s build a “situational hierarchy,” so to speak, of our logical validation process. We have 2 main contexts: creating an Event, and updating an Event. When updating an Event, there are two possible sub-contexts: updating an Event that is currently in progress, or updating a future Event. Since we don’t need to apply any special rules to validating a future event, we can ignore that context. This gives us a hierarchy that looks like the following:

• creation – start must be in the future
• updating – start must be in the future
  • inProgress – start doesn’t have to be in the future

And so when we go to validate our Event objects (before saving, or whatever we wish to do with them afterwards), we determine the context in which we are modifying them. I like to use dotPaths– paths.separated.by.dots– to communicate this information, and that’s exactly what I’ve done in my example script.

In a real-world situation, we would check to see if the Event is in-progress right after we pull it from the database. This would, of course, happen in our controller. Then, after applying the user-submitted changes, we would call Event.validate(‘updating.inProgress’) if it was indeed in progress, or Event.validate(‘updating’) if it wasn’t.

And now, for an example!

What I’m Talking About

The rest is more easily explained by my Two-Tiered Input Validation example. It’s an extremely simplified pseudo-implementation of the this solution. Look mostly at the controllerAction() method and the Event class– they simulate what the rule definitions would look like from the framework user’s perspective in a real application.

The Model class is just a simulation of a typical MVC overloadable model base, and the Validator/Context classes show sort of what an implementation would look like from the framework developer’s perspective.

Update: I have prepared a few diagrams to better-illustrate the validation model and validation processes. They also show an example validation similar to the above, but do a better job at illustrating the process.

• Frawst Two-Tiered Validation Model
• Two-Tiered Validation Process
• Two-Tiered Validation Process – With a Contextual Override

Conclusion

While it would be ambitious of me to expect this pattern to be implemented in any framework but my own, I do feel it solves a common issue with frameworks and poor validation systems.  Anything to improve or simplify this grueling process is a win in my books. If you have any questions, don’t hesitate to ask.

Oh, and no, I don’t plan to release my framework to the public any time soon. I do feel it’ll be ready for that one day, but it’s still very much incomplete and releasing it would be a bit of a waste of time. I am, however, using it to power a site I’m working on for a client right now, so when that gets released I will let you know.

And now, I need to sleep.

No tags Hide

A few days ago, the thought occurred to me that PHP scripts might be able to communicate with a client at run-time if they used a client wrapper (in this case, an HTML page). I’m not sure what this would be useful for, I just think the idea is kind of neat. It’s sort of the opposite to the approach most websites use today– handle logical operations in JavaScript and request data from the server bit by bit.

Here is the fruit of my labour: ActivePHP.

The “technology” is kind of slow at the moment– but that’s to be expected, considering the extra communication that needs to happen between the client and server.

If you have any idea what this could be useful for, please let me know. If you want to use it yourself, the source code should be posted up soon (although you can view it rather easily using the links and a little URI-modding).

No tags Hide

Well, after a fair bit of drama with my previous landlord, that came on quite unexpectedly, my living situation has once again changed. Glad I didn’t get that business license!

So, my landlord had recently decided to try and sell her house. I helped her fix up some things around the place, and hooked her up with my friend who helped take the remains of her front yard tree to the dump (since he had a truck). A few days before showings were going to start, she informed me I couldn’t be home during them– it was 4 days in a row, Thursday through Sunday. But she assured me I could still sleep at the house, so I didn’t cause a fuss and I said “yeah sure, I can be out of the house during the day”.

Well, come Tuesday night around 11pm, I get a text message from her informing me she had made a deal with her realtor saying no one would be in the house at all during the showing period, AND Wednesday (as in, the next day!). This simply wasn’t going to work, as I needed a place to sleep and I had already paid rent through those days. I told her of this, and it was almost as if she snapped… I’m not sure if she was just stressed out or what, but she started telling me how I “didn’t have the right to be in her home” since we hadn’t signed a written agreement for the rental.

Long story slight shortened: authorities got involved, and of course, they said I had every right to be there as per the Manitoba landlord-tenant laws, which do not require a written agreement. Despite this, I decided to gather my things and move out as fast as I could, which I did Thursday morning, to avoid further drama and the hell she would’ve most likely tried to put me through until my lease ran out.

I’m happy to announce that I’ve already found another, much better place to stay– walking distance from school, the room is twice as big, and my roommate is a student, too.

Lesson learned: always get things in writing! And thank you to all of my family and friends that helped me through this ordeal.

No tags Hide

I had originally intended to turn Four Staples into a development company, but lately a few factors have caused me to stray from this original idea.

First of all, as a college student, I don’t exactly have the most reliable of schedules. Some days I can fit in maybe 1-2 hours of development, whereas others I can put in a full 8. My point being, I don’t think my schedule would allow me to put in the raw hours necessary, at least in the beginning, to bring a company up off the ground. Also a lack of capitol and business experience would put me at a disadvantage.

My second major reason, though it sort of ties in with the lack of capitol I just mentioned, is the fact that over the past few years I have been moving around a lot– within Winnipeg, and of course my trips to San Francisco. I can’t recall the exact number, but the price of a home-based business license in Winnipeg is in the neighbourhood of $225, and must be reapplied for every time you move.

Once I can dedicate more time to development, I would like to get some office space and eventually expand Four Staples beyond just myself, but for now it’s simply going to be the site showcasing my freelance work. Being a freelancer will also give me greater freedom to take on projects when I have the time to dedicate them, while being able to take on a lighter load when school or other ventures start to require a bit more of my time.

Don’t let this post deter you though– if I do take on your project, I will make sure it is completed and goes beyond your expectations, all in a timely manner (so long as you are willing to work towards that same goal, should I require information or assistance on your part).

As part of my site maintenance over the next couple weeks, I will be rewording certain areas to clarify this new, freelance-based direction.

My rates will stay the same, and the quality of my work will not be affected by any of these changes– I’m still the same guy, after all!

No tags Hide

Good day!

So I’ve decided to start a development blog as part of the website. There are few reasons for this.

First, I’ve always felt a dynamic, constantly changing site is much more appealing than a few static pages (which is what Four Staples currently is, with the exception of the Announcements). It keeps visitors coming back, and hopefully will make it easier to stay connected with clients.

Secondly, the comment system will allow a friendlier and more accurate form of customer feedback than say, a Testimonials section. This doesn’t mean you must be a customer to comment; by all means, please say whatever you want to say or ask whatever you want to ask! I’m always open to questions and feedback.

At the moment, I’m using Wordpress to update the blog. This might change, it might not, but the design will almost certainly change to something that fits in better with the main portion of the site. You can expect this to happen over the next few weeks.

I think I will cut this post off here– I’m at school right now and I have a class coming up. I will make another post this afternoon or this evening regarding the aim of Four Staples and what I’ve been up to for the last couple months.

Thanks for reading!

No tags Hide