Factors to consider while providing QA estimate for a story
One of the important ceremonies done during IPM — Iteration Planning Meeting — is estimation. Where we estimate the effort required to finish a story based on the complexity.
If you have arrived here, I assume that you would know whats and why’s of story estimates. Still let me just summarise what estimates are and what are they for.
What is a story estimate?
It is simply a measure of effort taken by the given team to deliver a story. Story estimates are relative in nature. What that means is that the teams would have a baseline story and estimate the stories relative to the baseline.
Who estimates a story?
The team — backend developers, QAs, UI/UX developers, BAs, PMs — all are part of an IPM and can provide estimates for a story. But in most cases it is Devs and the QAs who will throw estimates for a story.
Why is it needed?
It is needed to plan how much a team can accomplish or deliver in a sprint or an iteration and communicate it to the stakeholders too for a larger planning.
Why is QA estimate important?
Estimate on a story should include the effort involved in development and testing. Because any extra effort/time consumed in testing will lead to delay in story completion and would affect the larger planning of the sprint and the project.
Now coming to the question, ‘What should I consider while providing a QA estimate?’
Factors to be considered for QA estimation
First important thing is to go prepared to an estimation meeting. Go through the stories to be estimated and the tech tasks and understand the impact.
I have listed some some factors that I consider before providing a QA estimate:
- Quantitative impact: If quite a few components are being touched, regression areas could be more increasing the testing effort.
Consider if multiple component changes could be covered with a few end to end tests or does it require testing of multiple user journeys.
2. Qualitative impact: A story might only need a few lines of code change but the impact on the system is more.
Ex: Removal of a column in DB could introduce a bug in any part of the application where it was used, if not deleted safely. Or changing of an API contract might need testing of multiple components where the API is used, if there are no contract tests in place.
3. Failure points: If there could be multiple error scenarios in the change that is being introduced as part of a story.
Ex: Introduction simple text box which is user facing with free text might end up having a lot of error scenarios based on the user input if not handled properly.
4. NFRs/CFRs : Cross Functional Requirements for a story could lead to additional testing effort as well. Every story has to be assessed in terms of CFRs, but think if the change being introduced calls for a specific CFR.
Ex: Change introduced in authorisation flow of an application might require a detailed security testing.
5. Automation:Once we have the test cases in hand,
- Check how many regression cases are already automated, this will reduce the testing effort
- If there are some cases that should be automated as part of the story,, this might add to the effort.
- If there are any existing test cases that needs to be updated, this would add to the QA effort too.
6. Experience/Skillset: I don’t know how to define this. But this is simply taking into account if the QAs in the team are familiar testing the change that is being introduced as part of the story being estimated.
Ex: Testing a DB migration. It can be possible that no one in the QA team have tested DB migration earlier and it could add to the QA effort.
7. Unknowns/Assumptions:If there are some unclear areas in the story it is best to add a buffer in the estimation. Especially when we think that upon clarification the story might bloat up in size(qualitatively or quantitatively). Or a team can even choose not to estimate a story if the requirements are not clear and needs client input. It is always important to call out unknowns or if the requirement is not clear when a story is being estimated.
8. Test environments and test data preparation :
Usually we have a test environment to test the stories during development. But it is always important to think ahead, ‘How are we going to test this story?’
Any extra effort involved in setting up an environment to test a story or getting the test data ready could delay the story. So it is a good idea to call those out.
While the points I have mentioned in this blog is based on my personal experience, there could be other factors that I may not have encountered or which are based on a specific project set up.
Finally, in order to ensure quality we can not deliver any story without testing. And hence raising the complexity and hence the effort involved in testing is very important in planning a sprint better in an agile team.
