April
17th, 2008
Why everyone should write a framework and never use it
I will scream if one more person tells me that I’ve wasted my time by writing a web framework because there are already so many available. The truth of the matter is that they’re right. The part that they’re missing out on is that writing a framework is one of the best exercises you can do as a web developer. So regardless if you plan on using the framework you write is irrelevant though I suggest you do because ironing out the fine details will make you a better programmer.
Why a framework?
The best way to understand anything is to take it apart and reassemble it. In this case we won’t be taking something apart but we’ll be assembling it. First off we’ll need to redefine what a framework is. From Wikipedia a framework is “a defined support structure in which another software project can be organized and developed“. For learning purposes we’ll define a framework to meet certain requirements and criteria.
Defining our framework
It’s important to touch on technologies you’re not familiar with so I will include general requirements.
- Pretty urls
- Model, View, Controller methodology (or your methodology of choice)
- Support for web services and AJAX
- 404 error handling
You’re on your own
This post isn’t a how-to on building a framework so I’m not going into the details. But the above requirements should force you to learn more about specific technologies (Apache, mod_rewrite, HTTP error codes, AJAX and web services). Only once you get under the hood can you begin to create really compelling software.
Other ideas
If you want to focus on a specific technology then it’s just as useful to do more specific frameworks. Writing a JavaScript framework will no doubt get you more familiar with JavaScript that you would otherwise. Writing a PHP or Python framework will do the same.
Like I said, it’s not so important if you use your own framework. I do on this site and numerous other sites. But there are lots of great frameworks out there which have had more time spent on them than you could ever do yourself.
If you have other ideas on improving your skill sets through real life exercises then let me know in the comments.
April 17th, 2008 at 12:51 pm
I completely agree. Writing a web framework is a good exercise. I built a half working JavaScript web framework and that experience has made it much easier for me to read the code in other frameworks.
I hear a lot of “don’t reinvent the wheel” talk and sometimes I think it is defense for laziness.
I think another important exercise is writing a compiler. It doesn’t have to be for a big language, optimized or even bug free. The many parts of a compiler appear individually in many projects.
April 17th, 2008 at 12:55 pm
@Peter, Writing a compiler is a great idea. Something I haven’t even considered.
I got my hands dirty by looking at the prototype.js source and understanding how it worked. That’s been more beneficial to my understanding of JavaScript than most of the other “articles” and “tutorials” I’ve read.
In regard to “don’t reinvent the wheel” … my response is generally that if the wheel was never reinvented then we’d still be driving on tires made of rock.
April 17th, 2008 at 12:56 pm
I definitely agree. even though you may never use your framework, it helps to understand what’s going on the framework you use in your daily programming.
April 17th, 2008 at 1:06 pm
By “JavaScript web framework” I meant server-side with Rhino :-)
I wrote a client-side JavaScript library that I use and that was a great exercise too. I don’t quite know why they are always called “frameworks”, however. They just seem like libraries to me.
April 17th, 2008 at 1:47 pm
II usually dismiss the complainers as the followers, the people who can’t think outside the box so they don’t want the box to change.
I’m always amused by the people who get furious at people who innovate with new software. They almost always respond with: “Why should we use your framework when we can use the one that came out 3 months ago”. Never once do they speak of a framework that has been around for years and years. You would expect them to say Struts, but they almost always speak of something else. For example Spring, Wicket, JPA, …… on and on. They are all relatively new frameworks! Not one of which would exist if everyone thought like they do.
Furthermore, there would not be anywhere near the number of software products that exist if we were all happy with one of them. It’s not a one size fits all world. A lot of people are happy with Struts, many with JSF, many with Wicket, and many of us don’t like any of them. And that’s only if you’re a Java guy. There’s a ton of other platforms as well. The same thing that’s’ true in frameworks is true in everything. How many different languages are there? How many databases? On and on.
April 17th, 2008 at 3:16 pm
I think you’ve really wasted your time writing a web framework. There are already so many available.
April 17th, 2008 at 3:34 pm
A few very different concepts here. Writing a framework as an exercise is perfectly valid. Writing one with a view to implementing a project on it is insane. It makes about as much sense as writing your own OS.
April 17th, 2008 at 3:43 pm
@SF, I agree. Rails is a great example. While I’m undecided about introspection vs. reflection the fact that the Rails framework was created and clones in many languages is necessary for the evolution of software.
April 17th, 2008 at 3:51 pm
And yet there is quite a few project aimed at writing an OS out there at present. Experimenting with other approaches to doing things is never a waste of time if that is your stated aim.
what starts out as an experiment or personal project can grow much much bigger. I’m not suggesting it will, or that its even likely but it is possible.
April 17th, 2008 at 3:52 pm
@ben, Writing your own framework for implementation isn’t always insane (maybe for 98% of the cases). If you have very specific requirements and enough experience with frameworks it’s completely within reason to be able to write a more efficient framework than ones available.
I do agree with you in the sense that 98% of sites don’t need to worry about that. But still…:)
April 17th, 2008 at 3:53 pm
@Konrad, That’s precisely my point. You don’t have to claim that your work is only for practice…start working on it and see where it takes you. So many libraries and frameworks we’ve come to rely on are a direct result of such projects.
April 17th, 2008 at 4:12 pm
Yep, I completely agree. The other things which I’d recommend doing (and then not using):
- A transitive dependency management build framework (like Maven or Ivy)
- An object relational database mapper
- An aspect/filter based security framework
All these things have been done before, but having a go actually writing one usually will give you a much better understanding of why certain things were done a certain way.
April 17th, 2008 at 4:40 pm
I agree that writing a framework is an excellent exercise; however, I also feel using it is equally important. Putting the pieces together to see how they work is useful, but using it (at least on trivial tasks or hobby projects) lets one see the warts in the design, and requires one to refactor/change the approach taken as the only one to blame for the design is you, the creator.
April 17th, 2008 at 4:43 pm
@Peter, The best part about using it is that you’ll be forced to tie up the lose ends which tends to be some of the more difficult pieces of any project. Like they say…the last 10% will take you 90% of the time :).
April 17th, 2008 at 4:54 pm
Here is a secret… When everyone thinks that something is impossible, those that prevail do so with less competition. I agree everyone thinks it is too hard to make apis, frameworks, products etc. This is great! Not only does it mean they are just more willing to use or buy your framework but they have automatically taken themselves out of the running. I agree you have to build many on many different platforms and fail numerous times to understand the details and trenches. But fail 9 times succeed once. If you want a good example of this Richard Adams wrote Watership Down, one of the biggest book sellers of all time. He failed epically with 13 publishers but ultimately scored one, the 14th try. Of course everyone told him he could not make a cartoon for adults and children. He didn’t listen and was ahead of his time. But what does this tell you. Yes it can be bad when people tell you that you can’t. Because they are building up a wall for keeping your competitors out. Or it could be an epic fail. It depends…
April 17th, 2008 at 5:15 pm
Other things I’d recommend doing:
- A filesytem
- A UNIX kernel
- A CMS
- A RDBMS
- A Window server
- A graphics layer
- A programming language
- A scripting subsystem
- I’m probably leaving something out
All of these things have been done before, etc., etc.
April 17th, 2008 at 5:57 pm
There was a time when my University wouldn’t teach you C until you wrote a compiler yourself. Every time you learned about a feature or concept in C, you had to add it to your compiler. Worked pretty well from what I hear.
April 17th, 2008 at 6:56 pm
This is exactly why I work on my own framework, wrapping your mind around a difficult problem sometimes requires actually solving it for your self, maybe what you come up with is not with the best or only solution but at least it gives you more than a superficial understanding of a concept.
April 17th, 2008 at 7:10 pm
Noah, that sure sounds like a great comp sci program! Which university? I’m guessing they stopped doing that in the mid 80’s or around 1990? If they did that into the mid 1990’s or beyond that is proud.
April 17th, 2008 at 7:16 pm
I think it needs to be said that there’s a difference sometimes between work and play coding. Often it’s a better idea to pick a powerful, stable and popular framework to help speed development and give a common baseline for a team to work with.
April 17th, 2008 at 9:18 pm
@Brent, while there is a difference between work and play it’s also important to note that it’s the fine details which provide the greatest learning opportunity. I think another great way to be forced into ironing out the fine details is to release something for general use.
April 17th, 2008 at 9:28 pm
@jaisen: It’s true that there will always be those projects that have custom requirements and require a custom system. But there are also projects that can fit into a certain template and sometimes that template is your in-house framework or a pre-existing one if it’s stable, better and cost-effective.
I totally agree with you that personally as programmers we should write our own frameworks to understand how a framework works in the first place. I was just saying that this isn’t necessarily the best for a team or a business.
April 17th, 2008 at 9:38 pm
@Brent, Indeed…here at Yahoo I know that both of those approaches have been taken. The group I’m working with is using a home grown framework and the Answers team for example used Symphony. The nice part about a company like Yahoo is that engineers have already evaluated (and continue to evaluate) which frameworks (internal or external) meet the requirements for performance, maintenance, etc.
April 17th, 2008 at 11:59 pm
What a great article, I’ve got experience with a open source framework, but I’m currently going through the process of writing a framework with my employer, I’m finding the process enlightening as attempting this task is a great insight of the general aspects of my preferred open source framework.
When I was at college (Not university) we had to write an Assembler simulator which included all the major registers, processing units and flags. Then attach an applications onto it with bit masking interface, some examples where a washing machine and traffic lights, then had to write the ASM to control the specific application. This was a great experience.
April 18th, 2008 at 3:51 am
Well, I have spent my pause after the exams to build classes like OtkRequest, OtkResponse, OtkController and so far.. Now that I am extensively using Zend Framework I think every piece of the puzzle is going to his place…
That doesn’t mean that every real programmer should write a Unix kernel, but in every field of computer science it’s better to dirty your hands to learn “how stuff works”. That’s the reason why Assembly is teached nowadays.
April 18th, 2008 at 8:12 am
I feel it is important to point out that if you do write a cusom framework for a project, you can make sure it is streamlined and small to include only the necessary portions. If you use another framework, your website could end up being slower than it should be. Not practical for all projects, but for some it is very beneficial.
April 18th, 2008 at 9:14 am
Tom, Highly modular frameworks (e.g. Catalyst from what I understand) allow you to pick and choose which parts you want to use. This reduces the need to write your own framework for the efficiency argument you make.
April 18th, 2008 at 9:37 am
@Tom,Peter … I’ve heard the same about the Zend Framework. Which brings up another great excercise. Building plugins/modules for existing applications will go a long way to help you get comfortable starting on an existing project with engineers. Which in my opinion is much harder than doing it yourself from scratch.
May 18th, 2008 at 11:16 am
I totally agree. I think you have to reinvent the wheel if you want to understand how the wheel works. I don’t want to imagine what would I be right now if I never had data structures classes where I had to (re)write them. I see many people going straight to Struts without even a good grasp on the Java basics. For heaven’s sake!! When it comes to learning, reinventing the wheel is a must.
June 23rd, 2008 at 12:22 pm
I want to learn PHP. Any suggestions?
May 1st, 2009 at 6:13 pm
Thats looking good and it’s the reality that explore the framework would be the best part of the day ! To find out what the lacks belong to a programmer that would be the first step to go with writting a framework and about the design pattern i think must study Gang of Four :)
May 17th, 2009 at 4:16 pm
I wrote a lightweight MVC framework long time ago and built a pretty large project on it mainly because at the time none of the PHP frameworks under development had the functionality or ease of use that I needed. I like some of the ZF modules and I know you can use them w/o the framweork. ZF still isn’t easy to use. And BTW back in the old days I wrote my own compiler too.