A brief introduction to modern testing approaches
James Bach, Cem Kaner, and Michael Bolton are well-regarded thought leaders in modern approaches to software testing who have collectively evangelized Context-Driven Testing (CDT) and Exploratory Testing (ET). In this post I will briefly describe these approaches and mindsets that underpin their school of thought and have formed the current cornerstones of modern software testing.
Context-Driven Testing was co-developed by Bach, Kaner, Brian Marick, and Bret Pettichord in 2001. CDT rejects the notion of generalised “best practices” that apply to all projects, and instead accepts that different practices work best under different circumstances. The third principle of the seven defined in CDT states that people are the most important part of any project’s context. Less of a focus on processes and tools, with more emphasis on people and their collaboration empowers testers with the freedom to make choices about how best to do their job without following a restrictive plan. Bach is highly critical of bloated testing documentation, including creating large test plans (that ultimately aren’t followed) and creating hundreds or thousands of test cases (that don’t find bugs). This doesn’t mean that planning isn’t valued or required, instead he advocates minimal investment in planning documentation up front, with an understanding that plans need to be flexible to change as a clearer picture of the project and its purpose becomes clear during development.
Exploratory Testing was first coined by Cem Kaner in 1983 and is defined as a style of software testing that emphasises the personal freedom and responsibility of an individual tester to simultaneously learn about how the software works while testing. The knowledge gained while testing can then be used to better design and execute their testing approach on the project incrementally. The advantages of this approach are that it requires little preparation and can find critical bugs quickly, which is the antithesis of scripted testing. Scripted testing requires a significant investment of preparation time upfront to write the test scripts (aka test cases) and may obscure bugs from a tester’s observation.
At their core CDT and ET were developed to extract the most value from testing time, by providing as much benefit possible to project team members, enabling them to perform their job more productively. This positions the discipline of testing as a service to project stakeholders, which is echoed by Michael Bolton in his blog article titled Testers: Get Out of the Quality Assurance Business. In the article Bolton implores testers to recognise that they don’t solely “own” the quality of a project, and that it is instead owned by the whole team. He continues by listing key points about how testers should provide a service to programmers and project managers on a project. I’ve paraphrased the most important ones below:
- A tester’s principal goal is to help the team look good. A tester’s job is not to shame or blame
- Be a service to the project, not an obstacle. Testers provide information, they don’t enforce process
- Testers don’t “own” quality anymore than anyone else on the team does
- Testers are often the bearer of bad news. Recognise this and deliver it with compassion and humility
- Testers should be skeptical about their own conclusions and welcoming of counterpoints
- Testing should be an activity focused on exploring, discovering, investigating, and learning about the product
- Provide information to the team which allows them to make informed decisions, and let them make the decisions
- Be aware that they’re making business decisions, not just technical ones
- Don’t report only bugs. Report issues that slow testing
James Bach, Cem Kaner, Michael Bolton, and other eminent testers in the industry are quick to warn that the approaches and philosophies mentioned above are subject to abuse and are often used as an excuse for bad testing by lazy and/or unskilled testers. It’s important to realise that these approaches place more personal responsibility on testers that employ them. The efficacy of these approaches is subject to the capabilities of testers who are constantly striving to improve their skill-set and expand their knowledge.
I hope this post has served as a good starting point for anyone interested in using the latest testing techniques, and suggest following the links I’ve provided throughout the post if you’re interested in knowing about them in more detail. In future posts we will look at more specific methodologies from James Bach and Michael Bolton, which are employed within CDT and ET, such as Rapid Software Testing and Session-Based Testing. Leave a comment below to let me know what you think of Bach’s, Kaner’s, and Bolton’s approaches.