Testing begins before implementation starts. It begins with better understanding and challenging the idea of a software change or a new feature. It begins with information gathering and developing a deeper knowledge of the consequences it can have on the existing system. It begins with building a real common understanding and working together by all people involved.
It is only then that meaningful test ideas come to mind.
Information Gathering Through Questioning
When I start in a new project or study a new feature, I begin by answering a list of questions. These questions help me to dig deeper, realise what I don't know, and identify the assumptions I have made, as well as the hard facts I do know.
6 Onboarding Questions
- How does the project make money?
- Who are the competitors in the market?
- What are the most important features?
- What are the most used features?
- What are the current challenges?
- How can testing help?
6 Feature Questions
- Who can give me a business and technical point of view?
- What is the problem we are trying to solve?
- What are the important use cases and scenarios?
- What is the user’s and business' impact if the feature does not work?
- Are 2nd- or 3rd-party services involved?
- What test data and test setup do I need and is it already in place?
Each of these questions comes with an intention. These intentions help me to gather knowledge about business, operational and technical aspects.
I then use these questions to connect the dots:
- Business rules and requirements
- System architecture and dependencies
- Responsibilities
- Risks
Using the questions as a starting point, I can gain an initial overall picture and then make knowledge- and risk-based decisions on what to test.
Kick-Start Your Thinking
As well as questionnaires, I use cheat sheets, mind maps, tables and process visualisations to organise my thoughts. Having a starting point makes it much easier to generate test ideas.
All of this helps me to understand the behaviour of real users, the behaviour of the system and the most important risks that we want to reduce through testing and other activities.
Most Good Ideas Come From Reality
Test ideas can be easy to come up with. However, some of these ideas are very theoretical and do not address real behaviours or risks. This means that you have invested time, money and effort into something that offers little to no value. Therefore, it is important to understand the reality.
Data is one of the things that helps me to understand this reality. I check our metrics, user reviews, and talk with product owners and customer support to understand how our customers use our product or service. I look into databases to gain an understanding of the actual data generated by our system. I use our bug reporting tool to check which money-losing bugs we have had in the past.
All this gives me an understanding of what actually matters.
Nightmare-Driven Testing
At its core, my approach to test idea generation is driven by risk. One technique I often use is called "Nightmare Headlines". I imagine the worst possible headline about our product or service in combination with a new feature or change.
- Users not charged for purchase.
- Colleagues from the finance department have to work overtime to fix broken data.
- Sensitive data about our customers leaked.
It’s a simple exercise, but it changes the conversation. It forces us to think about:
- What could go wrong?
- How bad would it be?
- How likely is it?
It's fascinating to see the range of ideas that people come up with when they do this exercise as a team.
Sharing is Caring
Although I often start by drafting ideas myself, I rarely keep them to myself. I share them in discussions.
- Developers help me to understand technical constraints.
- Product owners clarify business impact.
- Stakeholders challenge assumptions.
Sometimes, ideas grow stronger. Sometimes they are removed. Both outcomes are useful. The goal is not to create more test ideas. The goal is to create the right ones.
Experience Matters
While all this helps me to do a good job as a tester, experience still matters. I need to experience what it means to plan, develop, ship and operate software within an organisation. Then I can learn from our mistakes and recognise patterns. These patterns help me to identify error-prone or complex areas. They also help me to catch mistakes from the past.
Real World Examples
For one project, we tested a ticket-selling system offering many different ticket types and combinations of tickets. I created a matrix of these combinations and began to explore them. It was then that we discovered:
- Application crashes.
- Incorrect pricing.
In another case, something important almost went unnoticed. There was a pricing issue when users selected two or more tickets of the same type. Having already verified that the backend worked correctly, I assumed that it would continue to do so when integrated with the frontend.
It didn't. This assumption almost caused a real issue in production. Thankfully, a developer found it just hours before release. This serves as a good reminder that even logical assumptions can be wrong.
My Current View on AI in Testing
I rarely use AI to generate test ideas at the moment. However, I use my own test ideas, knowledge, and list of questions to provide AI agents with more context when they analyse or generate code.
I am also using AI to get a better understanding of existing code bases:
- What the data flow from an endpoint to the database looks like.
- Where specific decisions are made during processing.
- What other systems might be involved?
AI provides me with a starting point. I then verify its statements and delve deeper. For me, combining AI with my methodical approach helps me learn and understand faster, especially in complex environments.
So what actually makes a good tester?
If I had to summarise it, I would say: A good tester:
- Seeks understanding before acting.
- Thinks in risks, not just scenarios.
- Uses structure to support creativity.
- Learns from real users and real data.
- Collaborates with others.
- Continuously builds experience.
Because in the end: The best test ideas are not invented. They are uncovered.
About A Tester's Toolkit
This way of thinking is what I bring into my workshop "A Tester's Toolkit". The workshop covers how to kickstart and structure your thinking, how to manage your knowledge, and how to use all of this to generate meaningful test ideas.
Here's a brief introduction to the workshop.
