Discovery phase timing

"Ted,How soon before production do you think a team should go into a 'discovery'
phase?" - Jamie

As soon as possible, preferably at the start of the project. You want to make sure that your product is "production ready" from the day you start.

The later you wait to start the larger your backlog of bugs and missing features.

However, my reality has not always allowed discovery to commence with the project. Generally, customers do not want to explore a partially built system, they don't mind testing pieces of it, but have trouble seeing if all the pieces will hang/operate together. Often customers are shared between your project and another, their time constraints do not permit them to explore the application. Quality assurance people are really good at uncovering bugs, but it requires a real user to understand how the application hangs together in their environment.

There are many mitigation strategies I have used in the past:

1) Get it into production. If you have a truly green field application (no predecessor) you might be able to get it into production really early. The bare minimum of function can be useful to the end user. (Teamwave was good at this)

More often than not, there is an existing application, that does at least some of the same work that the new one will. In this case I try to duplicate the old application and put it into production once it has the same functionality as the old app. The second release is then the new enhancements.

Arguments I haved faced include: You will not be able to add new functionality as it will be just as brittle as the old app. It is too expensive for us to put it into production so we must add all the new features first. There are not enough features to be useful to a customer.

2) Get real data into the system early. As a substitute (or when getting into production is a long way away) this can work really well. Users get to see their data in the system and see what works and what doesn't. The data is meaningful, and having the users work with real life examples brings out all the complexity of real data. (Inform Alberta did this)

On a previous project we started migration a few months into the project so that data was readily available to the users to test with. They had lots of complaints, but also lots of learnings about how things worked. We got really good at migration, and did it multiple times over the remainder of the project as new functionality was added.

Arguments I have faced include: The data is too complex to migrate. We will oversimplify the problem, because the real data doesn't reveal all cases.

3) Eat your own dogfood. If you are really lucky, you can use the product as you produce it. Basically, you use your own product as you develop it. (Teamwave and OTI did this). OTI was actually a master of this, we built the Smalltalk language in Smalltalk, using our latest builds. You bet the code became production quality quickly. (I managed to screw up the AIX Smalltalk VM causing it to hang every couple of hours, no one was impressed).

Arguments I have faced include: We can't use our own product.

4) Simulated data. Make up data that you believe demonstrates that the system can handle all of your actual data. The idea is that the business changes rapidly, and meeting only the needs of current production data is not enough, you need to show that the system can handle more general cases.

This is what we are trying on our current project. It's later that I would like, but the business people are ready.