|
Archive: July 24, 2003
Programming is hard
Programming is hard. Everyone knows this; there are hundreds of books written each year about how hard it is and what to do about it. Universities teach classes in managing programming projects, because the general difficulty of scheduling software is an acknowledged business problem. Can nobody do anything to make it easier?
Well, people have tried. They try all the time, and a lot of pretty innovative improvements to programming technique have come along. Structured programming, object-oriented programming, functional programming, data-driven programming, even extreme programming have been devised. New languages have been invented and new system architectures have been derived. New development environments have been created, including spiffy on-the-fly compilers and interactive debuggers. All of these things help, no question. But the thing that helps most is writing good specifications.
Ask any programmer about writing specs, and they'll cringe. Generally coders don't like to code in English. So, if spec writing is the one thing that helps, why do programmers hate writing specs?
Writing specs is hard
Yeah, writing specs is hard. Why? What makes specs hard? What makes anything hard? And why do we resist doing hard things?
Okay, here we go, let's launch into a bit of philosophy. First, things are easy when you know what to do. Even if the thing you have to do itself is not simple, the overall task is still "easy", because it is fully understood. Things are hard when you don't know what to do. The act of figuring out what to do is usually more difficult and complicated than the "what to do" itself. The reason spec writing is hard is that it requires figuring out how something is going to be built. Essentially you must force yourself to be creative; you must derive a solution given a set of constraints.
So that's why writing specs is hard, now, why do we resist hard things?
I believe happiness comes from liking oneself. Things which make you feel good about yourself are "fun", and things which make you feel bad about yourself are not. Anything you don't know how to do is necessarily going to make you feel a little less good about yourself. After all, if you were that good, you'd know what to do! There is uncertainty; what if you can't figure out what to do? Or what if you figure it out, but your solution proves sub-optimal? The fear of failure hangs over any "hard" task. It is actually must easier simply not to try than to try and fail, so we resist. (This is related to the whole idea of Trying, see my essay on this...)
Writing specs is good
Okay, writing specs is hard, and we resist it, but why is it good? Can't we simply avoid doing this thing which is hard :)
Unfortunately writing specs is good for the same reason that it is hard. It requires figuring out how something is going to be built. This figuring out is crucial. It avoids a lot of blind alleys and wasted time, and this is why programming is made more efficient by spec writing. Most of the time if you know exactly how to solve a programming task, writing the spec for that task is not hard. It is only when you don't know what to do - when there is creativity involved - that spec writing is hard, and this is precisely when you should write specs.
In the old days (yeah, I'm dating myself here), programmers used to write code on paper first. There was such a think as "desk checking", which essentially meant carefully reading your written-on-paper code to try to find errors. Then you would laboriously keypunch the program onto cards, and run the deck. Keypunching took a long time and fixing errors was difficult, and running a deck took a long time and fixing errors was even more difficult, so it really paid off to spend some time planning up front, and carefully desk checking code.
Now with fullscreen editors and interactive IDEs and incremental compilers and just-in-time debuggers, there is no need to do any planning. You launch the IDE, you start typing some code, and poof - you run it. If you have a problem, you figure it out. The careful up-front planning is lost, as is the efficiency of thinking through a problem and its solution before diving in to implementation. Writing specs is the best way to get that efficiency back.
Writing good specs is good
So writing specs is good, but certainly some specs are better than others. What makes a good spec good?
First, you need the right level of detail. There must be an overall description of the problem to be solved, from the user's point of view. These are often called functional requirements, and are sometimes packaged as a separate document. They are often written by product marketing folks rather than programmers. (They may even be written by customers!) Regardless, this information is crucial for a programmer. They must understand the desired behavior of the program, its reason for being, and the other functional considerations. This is the only way the programmer can develop a good solution.
Beyond the functional requirements, there must be information about the design constraints. This includes compatibility with other software, tools to be used, target platform, support considerations, performance goals, etc. The emphasis here should be on describing the constraints and goals of the solution without specifying the solution itself. As a spec writer you want to leave the programmer as much scope for creativity in devising a solution as possible - especially if the programmer is yourself :)
Too much detail can be as bad as too little. Overspecification is time-consuming and rarely helpful. Specs can have GUI guidelines, but they should not contain pixel-perfect screen shots or detailed example reports. Specs can give design considerations, but they should not dictate a design.
W=UH and specs
Okay, so spec writing is hard, and spec writing is good. Doesn't this contradict the fundamental equation of life, W=UH?
At first glance it would certainly seem so. After all, things which are hard, are wrong, right? Well, no. The problem is that writing specs cannot be considered in a vacuum. We have to consider the whole task, which is to program a solution to some problem. So we have to ask, which is harder:
-
Programming without a spec
-
Writing a spec, then programming
Overall, approach (1) is harder than approach (2), and uglier, and hence wrong-er. Although writing specs is hard, programming without a spec is harder, since the spec makes the subsequent programming so much easier that the entire task considered together is easier as well.
Well maybe you accept that, but you're still troubled by the fact that writing specs is hard, and W=UH. How can spec writing be made easier? Here's some ideas...
-
Get as much functional requirement information as possible. As noted above, much of the difficulty in spec writing stems from uncertainty. So more information is good.
-
Leverage existing documents and code as much as possible. Why reinvent the wheel? Nothing is easier than cutting-and-pasting, both for documents like specs and for programs. Often crafting a design in such a way as to allow document and code reuse is well worth it.
-
Leverage other people as much as possible. Is there a product marketing person or salesperson who knows a lot about the problem to be solved? Ask for their help. Is there a customer who can help? Very often customers and prospects will gladly document their requirements, simplifying the spec writing process and making it more likely the problem you're solving is one the customers care about.
If you can actually make spec writing seem easy, then you've truly got it made :)
|
RSS for financial transactions?
Today Dave Winer linked Deane Barker, who asked about a realtime RSS feed for credit card purchases. Actually Tim Bray made similar mention a few weeks ago, asking for an RSS feed of his stock portfolio. Jon Udell indicated [in email] that he'd like to receive higher-order messages from his bank in RSS - for example, periodic email alerts about an account being low, or about newly presented bills.
Introducing OFX
It turns out most large financial institutions support an XML-based standard called OFX. (Open Financial Exchange.) This standard was created in 1997 by Intuit, Microsoft, and CheckFree, as a way to interface personal financial managers like Quicken and Money to financial institutions like banks and bill payment services like CheckFree. Most home banking vendors like Digital Insight support OFX as well, both as clients (they use OFX to get account information from banks) and as servers (they export account information to end-users as OFX).
In addition to account and transaction history, OFX also includes messages for bill payments and bill presentment. Bill presentment has turned out to be a fizzle so far - there are too few billers which support it, and for consumers it is really only useful if the majority of their bills are presented electronically, so that they can get all their bills in one place.
I was with Intuit's Web Finance group for a couple of years, which included Online Bill Payment. We worked hard to get billers to signup for bill presentment, which at that time we thought would be the Next Big Thing. We had a deal with AOL, a deal with Excite, etc. (This was 1999.) Quicken and Money have the ability to present bills right in-band, and OFX has the messages. The big problem was getting billers to agree to present bills electronically. You'd think billers would be anxious to save printing and mailing costs, but the big billers all value their relationship with their customers, which is currently mostly conducted through their statement. (These are phone companies, utilities, cable companies, etc.) Billers just didn't want to turn over that relationship to an online transaction system, even after we put a ton of branding in for them (including HTML links to their websites). Today there are a number of bills you can get through "online bill presentment", but mostly when you want to pay a bill online, you have to do it through the biller's website, because of the customer relationship issue.
An interesting solution to the "all your bills in one place" was an idea several companies had at once; scan paper bills. A company called Paytrust still offers a service which does this; overall it was a great idea but proved too expensive for most consumers. The idea is that you tell all your billers to send your bills to an address as the scanning company, which opens your mail, scans all the paper, and then makes it available online through a website. This makes it truly possible to get all your bills online, whether paper or electronic.
There are two main things offered by OFX which are slight complications over RSS. First, there is security; OFX is always exchanged over SSL connections, with validation of server certificates by clients. Also a client must establish identity through a logon sequence (typically userid/password, validated by server). Second OFX servers support client-side state. An OFX client provides the high-water mark of previously seen transactions (all transactions are sequence-numbered by the server), and the server then provides ONLY transactions newer than the high-water mark. This enables small updates to be easily downloaded from large datasets (e.g. all the history on your credit card would make for a large message).
OFX vs. RSS
Both of these OFX features – security and client-side state – would be useful additions to RSS. If added, they would bring RSS up to the level of OFX, but would not necessarily add anything to the OFX standard already in place. There are already a bunch of defined XML tags for modeling financial transactions (these could be incorporated into RSS as a large “financial” namespace). Additionally there is already software - like Quicken and Money - which “understands” OFX feeds, and incorporates the data seamlessly. Such software wouldn't necessarily know what to do with RSS versions of the same data.
The idea of client-side state in OFX was really just born of expediency; the OFX designers realized financial institutions would be sending a ton of redundant information if all account history were sent every time. (And the target client base was mostly dial-up.) There had to be a unique identifier for each transaction anyway (today we would call it a GUID :) and it was simple to have the server sequence-number each item. Then a client would just keep their high-water mark, and send it on the inquiry request. The server then could provide only items "newer" than then high-water mark. If the client was confused or got reloaded or something, they'd just send zero and get everything again, but only the first time.
I actually think this form of client-side state would have value. Using aggregators like Radio or SharpReader which poll every hour, typically when a blog has an update it is only one or two items. But currently the aggregator pulls down the whole RSS feed, which could have a month's worth of items in it, and then filters the duplicates. If the aggregator kept a high-water mark (GUID of last-seen item) for each feed, it could send this on the request (as an optional parameter on a GET) and servers which wanted could use this to optimize their bandwidth by sending only items newer than the high-water mark. It wouldn't save me much on my blog, but really active blogs like Scripting News would probably save quite a bit of bandwidth over time. This would be a strictly optional feature for both the client and server - if either doesn't support the feature there is no harm done - but given most RSS feeds are served by a few products (Blogger, Radio, Movable Type) and most RSS feeds are retrieved by a few products (Radio, SharpReader, NewsGator, etc.) the possibility of implementation seems pretty real.
What's next?
OFX is pretty entrenched now, with Intuit, Microsoft, CheckFree, and most large financial institutions already on board. It isn't going anywhere. Meanwhile RSS is also pretty entrenched now but not as a protocol for obtaining financial information.
Jon's point about getting alerts as RSS seems really well taken. Many online services currently present such alerts by sending emails, but putting them in RSS seems very doable. Of course you might have to add client identification but that isn't too hard - you could simply pass credentials in the HTTP Authorization: header.
I predict more and more business-to-consumer applications will use RSS as a communication mechanism, particularly if security and client-side state are added. This could be done fairly easily with a namespace extension...
|
|
Scoble: Linux is not Microsoft's enemy. "OK, dummy, so if Linux isn't the main enemy of Microsoft, what is Microsoft's biggest enemy? Analog thinking." I hate to disagree with Robert, but I do. Analog thinking is the opportunity, not the enemy. Digital solutions are the future in many fields, offering efficiency and functional advantages, and the competitor in providing many of these solutions is Linux.
Robert was commenting on Doc Searles' "Saving the Net" article, which is well worth a read. Doc is a little hung up on the "who owns what" issue - this has been a longtime theme for him - but it has definitely been brought into sharp focus by the SCO lawsuits. (There are actually big companies hesitating to deploy Linux now for fear of being targeted by SCO, which is [I'm sure] exactly what they were hoping for...) Doc also discusses the infamous "Eldred" case. "I believe Hollywood won because they have successfully repositioned copyright as a property issue." I hadn't thought about it, but Doc is exactly right.
In other digital-rights news, Disney announced a distribution deal with Movielink to make recent Disney flicks available online. Looks like the studios are learning from the record companies' experience!
On the other hand, the RIAA has gone crazy; they are now issuing subpoenas for parents and grandparents of suspected file-traders. Ottmar Liebert says he's embarrassed to be a musician, but having enjoyed his work I think he should be very proud; it is the silly crazy weird RIAA which purports to represent the artists that should be embarrassed...
Robert X Cringley thinks the solution is Snapster (Son of Napster). This sounds a lot like what MP3.com tried to do a while back; buy a bunch of albums, keep them online, and let people who already owned the albums listen to them. He for sees a market cap of $33B...
Steven Den Beste in WSJ: We won't back down. "The real reason we're in Iraq--and why we will stay."
 The Onion has a great parody: Deficit-wracked Maryland calls it quits. Of course this could be written about California with no parody at all; our $38B deficit is not even funny. And it looks like Governor Davis is going to be called on it...
David Burbridge discusses the social issues of "growing old". This is actually the main reason why declining birth rates don't imply declining population growth. The U.N. usually misses this in their press releases about worldwide population...
If you want to see how this plays out in the U.S., check out this cool interactive map from USAToday, which shows state-by-state breakdowns of demographic information from the 2000 census.
Are senses a zero-sum game? This post on GNXP suggests that humans lost their sense of smell to make way for more acute vision... Interesting. Clearly senses are not quite zero-sum, but there is a "cost" associated with each sense and anything which was not helpful to survival would get selected out.
 Bored with conventional sports? How about quantum soccer? "the aim is to shape the wave function of a quantum-mechanical “ball” so that the probability of it being inside one of the goals rises above a set threshold." And I thought Quiddich was cool :)
Southeast Airlines is offering free in-flight WiFi! Talk about the wave of the future. They are a small charter airline based in Largo, Florida. As part of the same service they are offering in-flight cell phone usage. This seems like such a no-brainer, I could easily see business travelers going well out of their way and paying significantly more for these services in-flight. [ via Boing Boing ]
Andrew Anker writes about the importance of PR. "... since PR has a difficult to prove ROI, it is becoming an unused tactic in these capital constrained times." A great point. And yet, getting "buzz" around your company and its products is as important as it always was.
And over here we have Darwinian Poetry. I am not making this up.
Adam Curry's QOTD from Sir Winston Churchill: "There are a terrible lot of lies going around the world, and the worst of it is half of them are true." And the other 90% are not!
|
Return to the archive.
|
|
