This post is, simply put, about the next step in web design.

If you have been paying attention (you have been paying attention, right?), there have been countless opinions, views, pundits, open letters, pontifications, rebuttals, whining, coddling, and fear.

Now that we got all of that out of our system, what are we – as developers who do stuff daily- going to do about it?

Driving home from work yesterday, I started thinking about a new approach to web design. An approach that will allow free form web presentation, semantic HTML (read: bot crawl-able and indexable), and separation of concerns. None of this is revolutionary – but I have yet to see it spelled out this way anywhere else.

This Morning’s Broken Client Side Model

Below is a picture that describes the way it is today. This is the ‘before’ (Flash aside because its a “black box” in an HTML document, and since some argue its the most important and successful technology on the web lets table that for a moment).

The HTML DOM as it exists today is the view in the prototypical MVC pattern. HTML5 and CSSphiles will say that it should semantically describe the data. So really the document describes the model, and the view. Javascript is obviously the controller in this case – but almost invariably it also holds state. What we are left with is this Venn Diagram where the model-view-controller is bastardized. CSS helps separate out presentation, but lets face the fact that it doesn’t (why would tags have ‘hasLayout’ properties if it did?). Don’t let CSS distract you here, I’m simply pointing it out. Moving on.

Separating concerns allows technology to move

So what can we do differently today?

This Afternoon’s Client Side MVC Model

Since HTML describes the data, lets leave it as the model. Javascript/ECMAScript works well enough and browsers support it, so that’s still the controller.

Ah, the ever-so debatable SVG/Canvas technology. This is new stuff, we’ve all seen demos. We’ve all debated. Which one is better? It doesn’t matter. There are browser differences in javascript and DOM implementations, and yet there are many many many javascript libraries and frameworks to address that. So while IE will have SVG support, all the other browsers will support Canvas. We are a resourceful bunch – there will be a layer that takes care of that for us transparently (this is already happening with smaller libraries).

So with cross-browser free form taken care of, whats the approach? Make the HTML DOM invisible, use Javascript to $(‘lookup’) the data you need, and feed it to the free form technology to make it pretty like Flash.

The clincher here is since the DOM still exists (and crawlers don’t  care about javascript or CSS), it can still be semantically described and indexed. Documents can still be linked, data can still  be pulled out by external services, and “Flashies” still get their pretty free form design. CSS can work for small visual requirements where HTML for presentation makes some sense (controls/etc), but most of the work *could* be done free form.

Browsers obviously are a huge play here. People who say browsers aren’t innovating haven’t been paying attention. They will get faster, which is really the main issue here. Mike Erlanger suggests that HTML5 is really the turning point. This modern approach can be done today, but ultimately suggests that what is sent over the wire is data (existing as a DOM in today’s parlance), and the presentation will be done by the programmer, on the client (this is my thoughts on HN regarding this issue).

Full model-view-controller, free form web design, semantic web, can be done today (and be done better tomorrow). Is anyone already doing this?

Note: This article is more technical than the rest of my posts so I’ve added addendum to kind-of explain some of the things I’m talking about. They will be paragraphs in italic like this.

So I’ve done all sorts of programming in all sorts of languages.  Ti-83, Visual Basic, C++, Java, straight C, php, python, LUA, ruby, javascript, etc etc. Command line tools, web serivices, high performance servers, low level libraries, games, web sites, web applications, mobile applications, calculator apps, the list goes on.

But I must say, I love web development. Why?

Because web development is a mix of all of these things. Each type of programming solves a different problem, is done in a different environment, has different rules, and requires its own mindset to decompose and solve the problem at hand.

Take a look at a typical Rails client/server stack (courtesy of joshcarter.com):

Starting from the front of the client stack – you have web design. This includes graphics, layout, styling, and user interface design. I use Photoshop, hand coded HTML with a W3C validator, css, and javascript. Right there I need to have a discriminating visual eye, and competencies in fomatting, styles, and scripting. And that’s just to get something to show up in the browser. This doesn’t even include the idiosyncrasies between browsers, supported standards, typefacing, or compatibility (damn you Internet Explorer 6).

For the uninitiated, this means that I can write HTML that works perfectly in Firefox, a popular open source browser. However, if my friend uses IE (which comes with Windows), or Safari (which comes along with a Mac), you could potentially end up with a pages that look similar, but different enough to cause layout problems – I’m sure you’ve seen weird pages before. This is why you’re “geek” friends tell you to drop IE, because catering to the way IE displays its web pages is a difficult challenge.

How do you create that page? Well thats the stack on the left. Again, starting from the front – I’m probably are using a web application framework like Ruby on Rails, Django, CakePHP, etc. If you aren’t using a framework (or have written your own), theres a whole class of other issues right there. Web servers, CGI, request dispatching, MVC design, databases and SQL. In most cases (unless you are using Node.js) you’ll have to learn yet another language to use or create a web app framework.

Basically, you need a way to pull information from a database, get it in a form that is easy to work with, and then use it to *generate* the HTML for the web page. So not only does the resulting web page have to look right in all the different web browsers, but the developer has to write a *program* that creates this HTML – on the fly – correctly – every time. This requires another language on top of the visual element languages.

So the thats the front-end of the server (even though the database could be considered the back end). Then there is the ‘OS’ layer, which I would consider number crunching tasks. Web development and application providing increasingly requires heavy workloads, computation, or number crunching. In a lot of cases this can be done in the same server-side scripting language used in the framework, but not always. Ruby and python make it easy to create and bind to C for the heavy lifting. This brings in more low level programming languages, environments, and *gasp* – memory management.

The languages used to generate the HTML are usually slow because that too has to be converted into a language that the computer understands, *while* it is running. Meaning the “code” is just text in a file, and a computer reads it and does the corresponding “ones and zeros.” This conversion process takes time and the languages are usually limited. To do number-heavy or data intensive stuff, one must use *yet another language* (usually simpler in form but more low level and sophisticated) that is “ones and zeros” from the get-go. There is less and less of this but most large web applications require it.

On top of all this you are worrying about response time, ease of use, ability to service and maintain the application, CPU loads, search engine optimization,  and having fun somewhere along the way.

Does the web site load slower than Myspace? Where the hell is the delete button? This error has been happening for months! Why can’t I find the site anymore on Google? This website sucks.

Now don’t get me wrong – I love all programming. Low level performance code, games, animation and graphics, command line and server tools. Web development brings in all of this. The skills required to develop for the web – by yourself – literally encompasses a lifetime of learning.

The web moves faster than any other technology out there – and it requires the widest range of skills. If you are a developer and value learning as a life long endeavor, then web development is a challenge awaiting.

For some reason, I remember a very specific exchange with my mother when I was 11. I had just begun middle school, and having my own locker, a binder, and text books made me feel like a grown up. I had seen how the older 8^th graders had boyfriends and girlfriends, holding hands, kissing goodbye in the hallway, etc.

Like any youthful sprite, I wanted to grow up too quickly. The previously aforementioned exchange with my mother ended with “why can’t I have a girlfriend? Its not fair!” Her response?

“Because.”

Not the best answer for an 11 year old. But what I didn’t know at the time (amongst many, many other things) was…

Interpersonal relationships are equally rewarding as they are challenging.

But please, do not misunderstand this statement (not that I automatically assume you have). I don’t mean to say that love is amazing but at the same time difficult, although this is true. I have realized, through my own experiences, that the amazing part comes from the challenge.

The converse of this being that relationships that are not challenging can only be temporarily rewarding.

Here is a well known prototype for relationships:

“We met, and everything was wonderful. I felt like I was on cloud nine with him/her. I couldn’t possibly be away from them for more than a day. Yet somewhere along the line, the spark was gone and what was left in its place was everything that had grown to annoy me. So I dumped ‘em.”

Why does this happen? The initial excitement of having a new person in your life wears off within a couple of weeks. Sometime after, you seem to get bored and your eye wanders while simultaneously picking on the negative things about your partner.

I would suffice to say at this point there is no challenge in the relationship. What creates challenge? Well, its no longer in getting another date. It may not be making them laugh, feel comfortable, or having them spend the night anymore. Humans are inherently goal seekers. So whats left?

Respect, trust, and emotional and mental intimacy.

Now thats a loaded list. There are plenty of books written about this stuff. Many counselors make a living on it. One blog post couldn’t possibly cover it all. I’m not going to attempt to, not just because it takes a while but also because I don’t know everything. (Anything?)

What I do know, however, is that figuring yourself out, figuring your partner out, and helping each other grow as people (and as a pair) is challenging as hell and probably takes a lifetime. So if you’re bored with a relationship you’re probably not doing these things.

Helping each other with life’s challenges (as well as each other’s) is huge. If you and your partner can not work as one coordinated unit you’ll probably end up leading two uncoordinated lives. Just in the same house. Sure, you have your time together, and you talk about your days at work, but thats not synergy.

I have a dorky metaphor for coordinated efforts. My girlfriend Rebecca and I have recently started playing Rock Band. Now, we are seemingly a good band because we do well on all the songs. But she plays the drums and I play the guitar. We are really doing our own thing.

But what if one song was really really hard on drums?  Rebecca keeps failing at one particular spot in the song (for the uninitiated, this happens when you miss enough notes over a period of time). If I wasn’t paying attention to her needs, I could have used my star power to rack up more points (seemingly good for both of us) earlier. In Rock Band, I could use this star power to save Rebecca from her terrible drum playing, bringing her back on stage and preventing us both from failing. But I was too busy doing my own thing, so when she failed I couldn’t bring her back and we lose. Recognizing this weakness after failing a couple of times would tell me to save that star power in the case that she fails, so I can bring her back.

Now thats team work!

But thats not really that challenging, not after the first 2 times failing. It takes minimal change on my part. I could figure it out pretty quickly and move onto the next song.

Now lets say the next song is even harder on drums – the bass pedal is so difficult that she misses more than half, and she fails quickly (OK I fail too, alot. I’m not perfect). I’ve learned from our last endeavor that I need to save up star power – and so I do.

Except in Rock Band, you can only save your band member twice. So if they fail three times before the song is done, you can’t bring them back and the entire band will eventually fail the song.

What do we do?

Well we could reduce the difficulty, but in life you can’t change the circumstances like that so for this metaphor we’ll ignore that one. We can switch, but we aren’t as proficient with the other instruments as we are our own. We are stuck.

Wait a second, what if I play the bass drum?

I know I play guitar but I can hit a bass drum on beat for a while, allowing Rebecca to concentrate on the other notes. Sure, I’ll probably miss more of my own notes because I can’t “rub my belly and pat my head,” but it may prevent her from failing out.

So we try it – and this time I get close to failing. But we both make it through alive, and we finally complete the song after failing so many times before! We’ve worked in a coordinated way whose sum was greater then our individual abilities.

How does challenging each other come into play here?

Well, what if I when I suggested that I play the bass drum Rebecca got defensive of her abilities? What if she snapped at me in her own frustration?

“Fine, you come over here and play the drums then. Lets see how far you get!”

I could then get upset with her, starting a fight of our own when we were originally working together on a common goal.

But she didn’t. Here is a challenge of ego. She trusts that I respect her ability to play the game, but realize that I may need to help her out to get through the song. She’s challenging herself by admitting her limitations. I’m also challenging myself because I’m now playing one and a half instruments. Which is not easy might I add.

But guess what happened afterwards? We talked about how crazy that song was, and walked away from the fake stage setup feeling awesome that we could accomplish it in such a way. The game became more fun. The challenge has brought the reward. We have gone back to play many times after that – together.

So while Rock Band isn’t necessarily a serious example of the true hardships encountered in serious relationships, I like the analogy. I could have finished that song the first time through if I was playing by myself. But I wouldn’t have learned anything.

When my mom told me I couldn’t have a girlfriend when I was 11 and answered my prototypical “why?” with the parental staple “because,” she was telling me that if I didn’t understand the non-answer, I couldn’t understand the real one. The typical “you don’t know why because you don’t know why.”

Very wise, those parents. Now theres a concept.

If there is anything I can state as a fact about my post-graduate life, it is that more has changed in the last year and a half (I graduated in 2008) then the 21 years before it, which includes graduating high school, going to an out of state college, and learning to appreciate a good beer.

In 2009 alone, I started my first business (and second), lost my best friend, had back surgery, lived in my first apartment by myself, and eventually moved in “full time” will my girlfriend. I got off Windows and pretty much only own Apple hardware, I’ve switched completely to non-fiction reading, and I don’t have $80 weekend bar tabs (anymore).

Today is New Years Eve. We are on the brink of something new. For some reason this year feels like it will be even bigger than the last, even though I’m not entirely sure how I could accomplish that.

Small changes are hard to make. Because it is so close to what was before the change, it is easy to slide back into the previous groove, habit, schedule, or mind-set. People pick New Years to make big changes. Drop the new year resolutions, I say. I’m setting goals for my life. I’m completely changing my mentality.

What mentality we choose to live by is so important because it is literally the framework in which we perceive our reality – the mental objects and abstractions we use to idenitify our surroundings. It prioritizes our goals (do we want to slam beer and ride motorcycles or do we like chess and a glass of wine?), dictates who our friends are by reflecting our commonalities, and within its cracks defines our values.

I love my life. But I want more from it. I’ve learned that if you’re locked up and your “mouse cursor” isn’t moving, the easiest thing to do is reboot and start over.

Happy New Years everyone, see you in 2010.