Developing the Right Software With Test

It occurs to me that testing is mistakenly treated as a distinct task in software development almost every time I've seen software developed. Here are three unpleasant side-effects I see in conventional testing:

1. The best people want to be creative, not test someone else's work
2. Test coverage asymptotically approaches functionality
3. Test pushes come at the end of late cycles

But I think there is something we can do about it.

Test Becomes a Non-Creative Effort

Despite pushes from management, testing often fails to tap into creative reservoirs. Testing involves creative efforts to be sure. I've worked with many creative testers; however, I've rarely seen testers apply their passion for quality, reliability, and measurement to the early planning efforts for original functionality. To participate, testers are asked to put on the hats of other roles, roles typically filled by their co-workers. So the bright, the ambitious, and the talented want little to do with testing.

Test is a Subset of Functionality

Without an integrated test and measurement focus, features are defined and developed in one phase, then handed off to test in another. Testing is planned with a focus on what was developed previously, typically without a precise or up-to-date documentation. This virtually guarantees that some functionality will ship without test coverage, despite high—or often low—code coverage. After all, testing like this can only asymptotically approach the developed functionality.

Test Push Comes Last, Gets Cut

Despite efforts to parallelize testing and development, the test push typically comes last. As a tester I might say, "don't ship it, I haven't tested that enough." Management comes back with, "is it broken?" What can I say, but "I don't know." Or worse, I say: "I want to add another test suite," and they reply: "It's too late to start new testing." And they're right. If we find a new bug so late in the game, we can't take a fix without the risk of new regressions. So they make the tough decision, and cut the test.

So What Can We as Testers Do?

One thing we can try to do—and should do—is to ask require developers to include unit tests with every check-in. Chromatic from ONLamp compares the number of test assertions in the perl vs. the ruby standard libraries. He makes a nice comparison, and his implication is correct: software would be of higher quality if we had more unit tests and more test assertions per unit code. But I see another deeper point.

I like how Marc Clifton puts it in his Code Project article on Unit Test Patterns:

Unit testing must be formalized so that it becomes a real engineering discipline rather than an ad hoc approach...the unit test is supposed to test the code that the programmer writes. If the programmer writes bad code to begin with, how can you expect anything of better quality in the tests?...the unit test should be written first, before the code that is to be tested.

Aha! I think he's on to something. But even this focuses too strongly on raw programming. The push—and the discipline—is not just about unit-testing. It's got to start with product and feature design. As soon as you decide to develop a new feature, get your testers involved. I'm getting at test driven development, or rather Example Driven Design, as Peter Provost puts it in this excellent video on test driven design.

How will you know when you're successful? What has to be true? What assumptions are you making? There are your test cases. There is what you've got to develop to be successful. When put this way, it's obvious that test and dev and design must come together.

I'd like to see are more testers asserting product quality from the very beginning. You testers are, after all, the ones responsible for asserting product quality in the end. Focus more on what it is you're trying to build, rather than the code that makes it happen. As Professor Bill Arms from the Department of Computer Science at Cornell University puts it, "Most failed software development projects fail because they develop the wrong software."

If you can pull it off, think of the peace of mind you'll have when product quality is measured from the very beginning, and nothing is developed that doesn't have corresponding tests. You'll wonder how you ever developed software before. And your testers will proudly stand as shining stars in your organization since you've empowered them to achieve their mandate.

Have an opinion on the role of test in software development? Post a comment, and let me and our readers know.

LOLCODE

Don't know if you know anything about lolcat But now there's lolcode:

HAI
CAN HAS STDIO?
PLZ OPEN FILE "LOLCATS.TXT"?
    AWSUM THX
        VISIBLE FILE
    O NOES
        INVISIBLE "ERROR!"
KTHXBYE

Maybe this is a great opportunity to get involved with an important OSS initiative...

A+ Carbon Footprint

I was just going through some of my bills (all on-line of course!) and I came across a nifty little tool to help compare my energy usage with "similar" homes. It asked my a bunch of questions about type of home, when it was built, appliances that use electricity or gas, square footage, etc. Having collected all that information it plots where Susan and I stand compared to other homes that fit the same profile.

Here are the results:

We get this tool from our utility company, Puget Sound Energy. Despite some recent problems, PSE is pretty good overall. Maybe your carrier provides something similar

If not, you should check out this post on climate change (here's the ArsTechnica article it's based on) or see if Zerofootprint has a calculator for your energy footprint.

The search generation

A couple weeks ago Marissa Mayer unveiled Google's new universal search on the Googleblog. But... universal search? Last time I checked, we were still only indexing the internet (and some books that we put on the internet), not the universe. And lately I've actually been wishing that we did index more of the universe.

I'm sure kids my brother's age must be even worse about this than we are, but lately I've really been feeling like Nick and I are part of a generation of people who've had sufficient exposure to internet technologies (such as search) that they've become part of our daily way of thinking. The simple example is that each time my (parent-aged) co-worker asks me a question (including totally random stuff that I know absolutely nothing about), my first instinct is not to say "I don't know," but rather to go straight to Google and search on a few keywords. It seems like the most natural thing to me; while she's always a little taken aback and says "Funny, I never would have thought to use search," or "How do you even know which keywords to search for?"

Lately I've found this "search instinct" popping up more and more, in ways that seem to indicate that search has become less a tool that I use, and more of a way of thinking or interacting with the world. Like when I'm at a restaurant with a 14-page menu and I know just what I want to eat, but I don't know whether it's something they serve. Instead of trying to find it under the appropriate menu category (would chicken fried steak be under 'Entrées', 'Surf & Turf', or 'Homestyle Dinners'?), I just want to be able to search for [fried steak], and know in an instant whether they have it or not. Why can't I search for [avocado] and find all dishes whose name or description contains avocado? Why can't I search for [label:vegetarian] and get a submenu showing only vegetarian entrées, instead of having to page through the full menu looking for the little vegetable icon next to each dish?

Today we stood in front of the spice shelves at the grocery store looking for ground white pepper for a good 10 minutes. Do they really think they're making it easy for you by organizing the spices alphabetically within brand (McCormick A-Z, Penzey's A-Z, etc.) for the name-brands on the top shelves, and then by spice style (savory vs. sweet vs. leafy) on the bottom shelves? And of course the gourmet and bulk spices have their own section entirely... My fingers were itching to query for [white pepper] and be done with it.

Y'know those handheld devices that're all the rage in HCI experiments these days, where you walk around the museum and it tells you about whatever exhibit you're standing in front of? The completely personalized, interactive guided tour? The grocery stores should work on a similar technology, a little hand-held (they could mount them on shopping carts) where you enter search queries and it guides you to the precise location of the item you're looking for. Or tells you whether they're out of it at the moment. This would've been really nice to have when they re-organized the whole layout of our grocery store last month, so that we had no idea where anything was anymore. Or when we had to get two stockers, a cashier, a sommelier, and a manager involved in order to find out whether they carried minute tapioca pearls.

Now that's what I'd call universal search.

Washington (not the state)

This video just gets funnier every time we watch it:

[Edit: As of Dec. 2010 the video is available here.]

Firefox and the anxiety of growing pains

It sounds like Mozilla has some spare cash:

...a windfall came in the form of a royalty contract with Google, which, like the other search companies, is always competing for better placement on browsers...In the last two years, the deal has brought in more than $100 million

The dilemma is what should an open-source, community driven organization do with such a large amount of money.

Mozilla's solution is to put money into what Kapor calls "community purposes." To that end, the foundation is looking for a new executive director who would focus on worthy projects, although no decisions on what constitutes a worthy project has been made. "We go out and ask," Baker said, "and even the community is not actually clear where large amounts of money should go."

If I can put my two cents in here, I think involving the community in such a discourse is a good idea. This may sound naive, and you may be thinking that this is impossible. Perhaps you're right. But it would be nice to see Mozilla maintaining it's obligation connection to the community. I realize that there will always be someone who is disappointed, and maybe a majority will be disappointed by any decisions which are made, but open discourse is a great way to find a common spirit at least.

I think investing in open projects supporting innovation, education, development of new human resources (within and without the Mozilla organization) is a good start. And it sounds like they're going to try and do exactly that. Way to go Mozilla.

14.8, baby!

These days, every time I sit down to write all I can think about is roller derby. I just got back from one of those practices where I really feel like I'm improving; the kind that keep me craving more (as opposed to the practices where I feel like I'm plateauing and my endurance is nil). I still can't believe I've found a way to exercise that I actually enjoy! My form felt good tonight, my endurance was up; now I just have to figure out how to stop getting foot cramps!! (grrrr)

Tonight we had to do push-ups until I literally couldn't lift myself off the ground anymore (I don't think I'm gonna be able to raise my arms above my head tomorrow). Thumper started singing a call-and-response song while we were doing it and it totally made my night. We also timed each skater doing 1 lap around the rink; my time was 14.8 seconds (thank you, freshly-cleaned bearings). Here's to being able to beat that a month from now.

Rat City Rollergirls tryouts are in 9 days. I'm so curious to see what they're going to be like, and who's going to get picked. I hope all the PFM can be supportive of each other and happy for whichever ones of us make it, rather than catty and jealous. We're planning to go out for drinks afterward to celebrate the new Fresh Meat. Cross your fingers for me...

The new Y2K Problem: Y292,277,026,596

You may remember some distress around supposed Y2K issues several years ago. Many of those systems moved to storing four digits for years. Some other systems that used a slightly more technically clean approach still had a Y2038 problem. This has since been fixed for the most part in a number of ways. For instance:

Date/Time Functions in PHP 5.1 and up [use] a (signed) 64-bit value [introducing] a new wraparound date in about 290 billion years, on Sunday, December 4, 292,277,026,596. However, this problem is not widely regarded as a pressing issue.

I'm a natural-born leader... in consumption of worthless media

Every once in awhile, I can't resist reading some particularly awful piece of cotton-candy "news" that I stumble across on the internet ("10 Fashion Mistakes Men Make at the Office"? Irresistible. FYI, Nick doesn't make any of them). Today I just had to sneak a glance at "How your birth order affects your relationships". Without giving my opinion on this advice, here's what it says about me:

It’s no coincidence that most U.S. Presidents were first-born [children]: This is the sign of natural leaders. You’re a take-charge person, so not the type to drive friends and romantic partners crazy asking questions like, "I dunno where we should eat—where do you want to go?" Instead, you’ll make sure you have reservations—and land a prime table, too. And anyone lucky enough to pair up with you won’t spend weeknights wondering whether he or she has Saturday night plans, because "oldest kids are planners," says Dr. Leman. You’re also old-fashioned—in a good way. You always come through on anniversaries and Valentine’s Day.
Your love challenge: Being more spontaneous. First-borns aren’t the seize-the-day sort (you’re not one to text your sweetie to suggest meeting at this fun café you just walked past). Likewise, "you hate surprises," Dr. Leman warns. Pity the fool who springs meeting the parents on you when you thought it was just the two of you going out!
Best match: Youngest child. "It’s a case of opposites attracting," says Dr. Leman. "You help the last-born be more organized, and the last-born helps you lighten up."

I can't figure out whether the appropriate response to that is "Good thing I got me a last-born!" or "I can't believe I reprint this stuff."

Geek Heroes

You gotta see this video of true geek heroes.

Moving from Open Source to Open Data

Ars Technica has this story about Insight, a new open community data analysis project started by Business Objects. Insight seeks to create a community focussed on specific challenges which bring expertise together with openly available data sets to create analysis of those data sets for many different purposes.

There are collaborative projects for code, but no equivalent for data. We're hoping to combine data sources, analysis, visualization, and more.

This reminds me a little bit of (one of) the premise behind Vernor Vinge's Rainbows End.

We already collect and share massive amounts of data (see the U.S. Department of Labor Bureau of Labor Statistics for an excellent example). Today the analysis of this data occurs in isolated projects for isolated goals. Vinge, and now Business Objects, describes a world where the analysis also becomes a massively distributed, but shared, process.

Here's a little quote to live by from Franz Aman, an exec at Business Objects:

We are a commercial business, but we also believe in helping nonprofits and charities where we can. We believe in doing well by doing good.

American politics in 2016

"The story boils down to this: the Democrats can't win, and the Republicans can't govern."

No kidding, eh? This tidy little distillation of American politics is part of an intriguing article I came across recently from The Atlantic Monthly's July 2005 issue. It's written in the form of a letter to the first third-party president from his campaign manager, discussing the political context of his election and how the past several decades have affected the current political scene in America. The interesting part? It's dated 2016, so it discusses some "history" which hasn't happened yet. Given that, along with the fact that it was published almost two years ago, it's uncanny how spot-on some of the analyses and predictions are. To whet your appetite:

The past fifty years have shown that the Democrats can't win the presidency except when everything goes their way. Only three Democrats have reached the White House since Lyndon Johnson decided to leave. In 1976 they ran a pious-sounding candidate against the political ghost of the disgraced Richard Nixon—and against his corporeal successor, Gerald Ford, the only unelected incumbent in American history. In 1992 they ran their most talented campaigner since FDR, and even Bill Clinton would have lost if Ross Perot had not stayed in the race and siphoned away votes from the Republicans. And in 2008 they were unexpectedly saved by the death of Fidel Castro... As for the Republicans, fifty years have shown they can't govern without breaking the bank. Starting with Richard Nixon, every Republican president has left the dollar lower, the federal budget deficit higher, the American trade position weaker, and the U.S. manufacturing work force smaller than when he took office.

When we read histories of the late 1920s, we practically want to scream, Stop! Don't buy all that stock on credit! Get out of the market before it's too late!... In retrospect, the ugly end is so obvious and inevitable. Why didn't people see it at the time? The same clearly applies to what happened in 2009... Once the run on the dollar started, everything seemed to happen at once. Two days after the Venezuelan oil shock the dollar was down by 25 percent against the yen and the yuan. Two weeks later it was down by 50 percent.

So go curl up with some tea and read it! It's worth the length.

La délation communautaire

I got a most excellent email last week:

La France découvre le Web 2.0 et les efforts communautaires:

http://delation-gouv.fr
"Aidez-nous à remettre la France sur les rails en signalant tout comportement suspect grâce à ce formulaire."

My favourites:

• X est bronzé toute l'année
• X possède un didgeridoo
• X est "bizarre"
• X joue au minigolf avec des balles roses
• X ne se drogue pas