Testing update...

We have 3,400 tests running.

Our tests now take 5,000 seconds to run against a database on a PC.

Our tests take 138 seconds to run when we stub out the database. 36x faster. We couldn't keep running this many tests without the stubbed out database.

If you need the java code to stub out your database, I'm going to post it on sourceforge. It also contains code for automatic test cleanup.

- ted

Hockey refactoring...

Zach has coined the term "Hockey refactoring" on our team. It is refactoring the way you play hockey. Loose and fast, often dangerous. It is the form of refactoring most often practiced by developers. It uses the refactoring tools lightly, relies on cut and paste, and is often just a thin veneer over rewrite and redesign.

We are introducing our senior developers to more disciplined refactoring. Showing them how to combine steps in refactoring to achieve the majority of the work. Teaching the power of "replace from head". The actual steps are not long to do, figuring them out takes longer - but still way less time and way safer than "hockey refactoring".

Disciplined refactorings are often 10 to 30 simple refactoring steps put together in a chain. "Refactoring to patterns" hints at them (showing you how to put small refactoriung together to achieve a purpose).

The customers move

Our customers have agreed on putting real data into the system. Our customers have agreed to sit "in" the pit with us.

I think this means the project will see much better progress. First we will be able to understand how well the product is performing, and talk about how far along we are in terms of the customers business.

Second our customers will be less isolated from development and possibly share a common domain model with us.

Finally, I hope our developers will get a better grasp of the domain from the customer.

All good signs.

Interesting perspective on development

From Steve McConnell:

http://www.stevemcconnell.com/CargoCultSe.pdf

He's talking about how mimicking a process or level of commitment does not guarantee success.

"Rather than debating process vs. commitment, we should be
looking for ways to raise the average level of developer and manager
competence. That will improve our chances of success regardless of which
development style we choose."

Mimickry is part of educating managers and employees. Mimicking the behaviour of successful people and attempting to apply it yourself is a great way to succeed. The key is you can't blindly mimick, you must adapt.

Toplink Wiki

http://toplink.waldura.com/Main

Looks interesting...

Site statistics

I've been using freestats.com for a while now, it gives me surprising information (Like people have been reading this), but tends to be light on any other details. It tells me about Visits not Visitors. (and I care about the browser they're using because...?)

Some of my friends (and previous startup cohorts) have a new stats tool that tries to tell you about visitors to your site. This simple understanding of the purpose of the information makes a good product. I found the sort of information I was looking for right away, clearly presented. Good example of understanding the problem rather than understanding a technology . You can find their beta at http://www.activemeter.com. Good job.

Testers as Editors

Jonathan has a nice blog on testers as editors http://www.kohl.ca/blog/archives/000072.html.

It is a good metaphor that describes the roles. The only thing I want to add to it is that, unlike an "editor" for a magazine, it is more like having a collegue edit your paper before you submit it. It is a highly collaborative equal footing endeavour. If the product goes out and isn't reliable it is everyones fault or everyones success.

Creating a new API

So we're changing the way we do something in our system, it is functionality that has existed for a long time, but is getting a new API. There have been two approaches taken, when building this. In one we have refactored the old calls to look like the new calls (sometimes difficult). In another we have used TDD to write the new calls.

I way prefer the refactoring method. The TDD driven way appears quicker - at first. It assumes that you have captured all the tests. Well, what you are losing is the code history. The code has been shaped into however a poor form usually to meet a wide range of requirements, that may be expressed in many different tests. By replacing those tests with new ones - you lose the experience.

Refactoring may be harder, but it keeps the experiences. You are forced to touch and acknowledge the past experiences of the code. Removing one becomes very explicit (a design change) rather than an oversite. Best of all - the original tests continue to work.

Brian Maricks comment

I had a previous post that critiqued one of Brians papers (extremely briefly).

Brian Responded to the post recently:

"On an Agile project, I lean much more heavily toward automated tests. For my thinking along those lines, see this: http://www.testing.com/writings/2004-roadmap.htmland this: http://www.testing.com/cgi-bin/blog/2004/05/26#directions-toc"

Very good articles I have read before. I think the point that I was trying to make (no doubt badly) is that on agile projects the automated tests change the way you develop. How do they change the way you develop? As a developer they give you confidence about change (yours and others).

Up until about 4 years ago when I was doing test-after development I lacked confidence about changing stuff. It became slower, more planned and less experimental.

Confidence is what has changed my development practices. Confidence means that code goes from rigid to putty. That has been a fundamental and significant change in the way developers work.

Jonathans Virtual Bugs

http://www.kohl.ca/blog/archives/000066.html
"As a tester, what do I do with the bugs we discovered but couldn't get the unit tests finished for"

"On small teams, I will keep a running tab in my notes of these bugs...This doesn't scale well though."

In general, we view bugs as part of the progression of the story. It can take several passes to get a story and all its interactions correct. The specification may be wrong, the implementation may be wrong, even the testing may be wrong. It takes time to hammer a story into shape.

Bugs are simply cards that clarify the story or tests. Most importantly, it is no ones fault.

We differentiate our bugs from our stories by using green bug cards instead of yellow story cards - they are easily written on, modified, attached to stories or thrown out. They can contain hints for testers (about virtual bugs), or feedback to developers. We find paper much more versatile, organized and personal than computers at tracking this stuff.

XP doesn't help the customer

I’ve been thinking a lot about Tom Poppendiecks statement that “XP provides little information/help for the customer” to misquote him. This is so clearly bang on and explains some of the smells I’ve seen on other projects, where we’ve made it through but I felt it could have been better for the customer.

Interestingly, we had Gerard Meszaros (from Clearstream) run training courses for our customers (SPOCS) at the beginning of this project to help them identify *how* to write a story, how to write tests and how best to interact with the team. But really, there are no rules for mindfulness other than it should provide “business value” and “do the most important things first”. I’ve seen our customers struggle with those concepts and it is even harder on a project this large.

So I’m revisiting Eric Evans’ Domain Driven Design, Toms paper, agile modeling, and Managing the Unexpected with customer “mindfulness” on my mind!

Work Cells

We're specializing our teams - with each team taking a stream.

We're now thinking of assigning customers to teams, and a QA person to a team. This might give us one of Mary Poppendicks workcells.

Some hoped for benefits include:
1) Better communication (effortless) between development, QA and Customers
2) More Specializiation for development - so less learning asbout new areas, more learning about one.

Some concerns:
1) A loss of the overall vision among individual team members
2) More difficulty integrating between teams
3) Some of the more senior members will not be able to share their skills across teams

Some potential solutions:
1) More senior members will rotate at a moderate rate between the teams (every couple of weeks? once a month?)
2) Our technical architect will rotate fairly quickly through the teams (A couple of days at a time?)
3) Our other teammates will rotate more slowly (Every couple of months?)

Something to try...

Roles of Design on an Agile Project

A note from Tom on the course he teaches about helping the business people specify requirements in an Agile world:

"The key concept is that all the classic analysis techniques used for requirements, analysis, and design work are still valid and valuable but they should be viewed as techniques for shaping the conversation among all the people designing the business process/product, not as formal, complete, version controlled documents. Manage the work with story cards (see Mike Cohn’s User Stories book). Manage the conversation with agile modeling techniques (see www.agilemodeling.com) and record and version control Customer tests."

Looking at the agile modelling site it has matured alot from a few years ago.

It seems to me we have a lot of people (Eric Evans, Tom Poppendieck, Scott Ambler) saying similar things. Customers and developers must share a model of the system - even though that model evolves.

Mindfulness

Tom Poppendieck emailed me, he is reading Managing the Unexpected by Weick & Sutcliffe. It is a book that focuses Mindfulness. Guess I'll order it.

In our project, the developers are Mindful as they look for "Bad smells" in the code. When things start to feel bad, we look for ways to improve what is going on. Our retrospectives provide opportunities for the team to improve as a whole.

How is the business side of our project Mindful? Currently this seems to be our weakness. But we have seen some positive signs after the arrival of Mary and Tom.

1) They test each month - this confirms development is on track with what the business is saying.
2) With some prompting, we have got all the companies starting to give us real properties.
3) We now have an eight foot long (high visibility) Mindfulness reminder. It is a blown up spreadsheet that one of our users uses to track tests results of before they put a productin accounting upgrade into production. We are using it to track our progress.
4) We are working hard to get our customers to look at an early deployment. It looks like one of them might agree. If we get one...

Progress...

After Mary came by, one of our SPOCS (customers) said "Why didn't you tell us you wanted to go into production?".

A different person, in a different environment can sometimes get the message across.

Clean up iterations...

Jane mentioned that her teams use "cleanup" iterations occasionally. In order to get things working for in production, the team must spend time making it work. Sometimes, this is problems introduced by the development team as you go into production, sometimes it is problems caused by incomplete customer specification.

In the end, it doesn't matter, you need to get things working.

Good luck Jane!

I saw Jane Robarts today (an ex collegue). She's been working for Thoughtworks for a couple of years and is now off to work in India for a year. She's been running large agile projects with them and is off to help them with their India effort while exploring the world.

I'm sure she'll have fun!

What is a zero defect team?

I don't get why you would want a zero defect team. This would suggest that the team is over testing. Personally, I would like one defect per member. This would mean they could have written one more test. They were only off by one.

By the way this is a principle from the newspaper business, who always want one paper left in the box at the end of the day. It guarantees not too much and not too little - eliminate waste.

Refactor Mercilessly, or not

XP tells us to refactor mercilessly. In some ways this is totally true, in others it leads to difficulties for real live teams.

For clarity, refactoring should be done mercilessly. If you can't understand the code immediately, it is a no brainer. It is of no use to anyone using it.

For encapsulation, refactoring should be done mercilessly. This will save your bacon, when you defer the refactorings below.

For duplication, you should refactor almost all the time. Duplicate code creates multiple points of potential failure and hence harder debugging. However sometimes, duplication is hard to spot. Code does the same function, but doesn't look the same. Other people haven't noticed the similarity. Sometimes it is hard to remove the duplication, because the responsibilities are slightly wrong and the duplicate code is dispersed.

For redesign, you need to refactor when the code can converge. Sometimes you just don't know enough about the behaviour of the code to redesign by refactoring. In this case you must wait until there is enough "convergence" in the code. I call convergence the point where the code starts screaming at you what the objects are and how to put them together.

Jonathan Kohl

Met Jonathan Kohl during the Mary Poppendick tutorial and then again at the Brightspot soiree for Mary. He's attacking a difficult area of XP - how to test in an agile environment. It's great to see his efforts, and we've certainly used some of his techniques on our project.

Thanks Mary & Tom

Appreciated the time you spent with our project, the course you gave, and the banter in the car as we drove around Calgary. I appreciate, that despite being tired you shared your thoughts about the project with us, helping to provide clarity to the problem we were struggling with.

I look forward to our next meeting...

- ted

There's a corpse under the floor and it is starting to stink

Mary & Tom Poppendick visited our project today. I left with a sense that they thought our project was reasonably well run by lean standards, that we had everything in place for the technical team to succeed. They felt that the fixed price contract might spell doom on our project - that it is driving a bunch of undesirable behaviour in our customers that is mostly irreversable.

I would agree, if we keep interacting the way we're interacting.

Why do I have hope though? The SPOCS have change A LOT. They arrived expecting to have high level requirements meetings once a week. Instead they have learned to write smallish, mostly incremental stories. They figure out detailed requirements. They painstakingly test the product month after month, with the stone age tools of a development environment.

Our developers have changed A LOT. They have adopted test first development. They have adopted simple design (that was hard!). They code for today and let tommorrow worry about tommorrow. etc. etc. All the practices of XP.

Each group *is* concerned about getting it done.
Each group *can* smell the corpse.

How we get it done is in dispute.

If we let the contract guide us we're screwed.
If we let the corpse lie between the floor boards we're screwed.

We need to deal with the corpse, acknowledge its existance and figure out what to do with it. Then, maybe, we can get on with being successful as a team.

Stay tuned for more exciting episodes of "As the corpse rots". I'm sure it'll be on my mind.