I was browsing Implementing Lean Software Development: From Concept to Cash by Mary Poppendieck & Tom Poppendieck, and found some great information about reducing waste. We have just started examining how we can improve our test processes as a way of introducing Test Driven Development. We have found two or three practices that remove most of our pain. Re-reading the book showed me that we have re-created some of the best practices that the Poppendiecks wrote about. Perhaps surprisingly, the practices are not about testing, but about development.
We have found that most elements of testing simply cannot wait until the end of the project or iteration. They need to be integrate much earlier. For instance, one legacy architecture is tightly coupled, not modular and with an amazing mountain of crippling ‘testing debt’, i.e. we have not built unit, functional or regression tests for the code. This means that more often that not, testing is the bottleneck around which we should plan, not development. In turn, this means that building a release plan around development estimates does not make sense: the critical path for our software lifecycle comes from testing. Therefore, we need our test manager involved in release planning.
Similarly, the earlier we can generate good tests of the software the faster we highlight and can fix bugs. The longer we leave these to fester throughout the development life-cycle the more damage they cause. Our ideal is making the user acceptance testing a verification rather than flushing out new bugs. We approximate RTY, the manufacturing measure of first time right by measuring how many features work without bugs in UAT. The goal is to drive this to 100%.
“According to Shigeo Shingo, there are two kinds of inspection: inspection after defects occur and inspection to prevent defects [See Shigeo Shingo, Study of ‘Toyota’ Production System, Productivity Press, 1981, Chapter 2.3]. If you really want quality, you don’t inspect after the fact, you control conditions so as not to allow defects in the first place. If this is not possible, then you inspect the product after each small step, so that defects are caught immediately after they occur. When a defect is found, you stop-the-line, find its cause, and fix it immediately.”
“The job of tests, and the people that develop and runs tests, is to prevent defects, not to find them. A quality assurance organization should champion processes that build quality into the code from the start rather than test quality in later. This is not to say that verification is unnecessary. Final verification is a good idea. It’s just that finding defects should be the exception, not the rule, during verification. If verification routinely triggers test-and-fix cycles, then the development process is defective.”