Many
Scrum teams feel there's something not quite right about
testing and their use of a task board. In this blog post I'll go in-depth on the issue of having testing-related tasks or not on the task board. I have found that this is a typical question especially in beginner Scrum teams.
Example Scrum team scenario
To keep this blog post relatively short I will focus on the following scenario:
- Automated testing code coverage is medium to low so there is an extra need for manual testing.
- The product is relatively complicated with lots of intricate scenarios/settings to test.
- The product is a SaaS with thousands of users.
- You do not have dedicated testers but you have plenty of access to people outside the team that can help to carry out testing.
- Testers are not experienced in testing. They know the product to be tested but are not super users. They prefer not to use developer oriented testing tools because they are not highly technical persons.
- Testers outside the team come and go and they are quite busy so they need a simple way to carry out the tests without having to log in to a complicated testing suite. Because of the turnover you do not want to provide a lot of training for testers.
- You do not have a test manager, QA lead etc.
- Developers are also inexperienced testers and have limited knowledge about QA.
- Developers are writing the test scripts because the testers are not qualified to write high quality test cases.
This scenario may describe a team with a medium
Agile maturity level so other more fundamental actions could also be needed such as QA coaching, adding QA related metrics etc. but we will focus on the question of testing tasks on the board.
What is testing anyway?
You should never rollout a feature/User Story/Product Backlog Item (PBI) without testing. Someone has to test it. This might take the form of unit testing, acceptance testing, security/penetration testing, exploratory testing, regression testing, performance testing, load testing, code review, integration testing, web tests e.g. Selenium.
In addition to local testing during development. Maybe you want to add UX user testing and maybe you even need to do some regulatory testing.
The purpose of the task board
Before we discuss if testing tasks should be on the board we should first do a recap of what a task board is for. The task board, as the name implies, is for tasks. The purpose of the task board is to keep track of tasks and create visibility to the team and other stakeholders. This helps the team make sure the right things are completed at the right time [4]. The three pillars of the
Empirical process which Scrum and Agile is based upon is Visibility, Inspection and Adaption. Tasks on the task board enables visibility and transparency.
Now, some teams try to cram more information onto the board by adding lanes for server environments or lanes such as "Ready for testing", "In testing", "Ready for staging" etc.
The starter task board with the 3 lanes: "To do", "In progress/WIP/Doing" and "Done" is still the recommended basic setup [1],[2],[3],[4],[5],[6],[9]. Remember that the task board is primarily for tracking tasks.
A task can be in progress, done etc. To say a task is in QA/staging or in testing does not make sense. A user story/feature can be in testing but a task is a piece of work to be done. You don't test a testing task. This inconsistency lies at the root of disagreements on how to visualize testing on the board.
Why not have explicit testing tasks on the board
Typical arguments for not having testing tasks for each story:
- It's extra work to add test-related tasks to each user story. Each user story will often have the same duplicated testing tasks. Seems like unnecessary extra work to add them for each story.
- Testing is an integral part of development so we don't need tasks for it. It is part of the programming work.
- More tasks means there will be more overhead needed to pass all the tasks through the lanes of the task board.
- The board gets messy when we add tasks that are to be partially carried out by others not in the core team. We like to have full control of what all the tasks are and not have things that relate to other persons in there.
- Testing is something that is done after we have deployed to the QA servers. Seems out of place and not logical to have them on the board. Testing is sort of another phase of the process and it seems illogical to have it alongside programming tasks.
Some of these arguments include faulty assumptions.
- "The board gets messy when we add tasks that are to be carried out by others not in the core team". This claim assumes that the team don't need to do anything related to testing. Someone has to write the test cases, make sure the test environment is testable, someone has to coordinate testing, make sure testing gets done, provide support for testers etc. There are also tests that only developers can do like performance testing.
- "Testing is sort of another phase". According to Agile practices this is just wrong [7],[8],[10],[11]. This thinking leads some teams to add a testing lane to their board which again strengthens a view that testing is a phase in the sprint and so test-related tasks are not needed on the board.
 |
| Example task board with lane for testing |
This can also have the effect that epics are not broken down into small enough user stories and tasks. Since it does not always make sense to have a task in testing you may end up having tasks that are actually user stories. When a task on the board is actually a user story it makes sense to have the "task" In testing.
Testing is not a phase, but a way of life
-
Elisabeth Hendrickson
Ideally testing should be a collaborative effort between developers and testers going on in parallel [8],[10],[13],[14]. In our scenario this is difficult since there are no skilled testers so developers have to step up and write test cases, provide testing support and coordinate testing [11].
In Scrum and Agile development we strive to complete one by one user story. Testing is of course part of that. One by one story is completed to reduce risk and to be able to deliver value increments.
Alternatives to testing tasks on the task board
Based on our scenario I have identified a few alternatives to having testing tasks.
| Alternative | Implication for test script writing | Implication on carrying out the tests |
|---|
| Developers write test specifications at the start of the sprint based on the requirements specifications, dev tasks, DOD and Acceptance Criteria. | You don't yet have a working feature so you will not be able to try it out and come up with all the test cases at the start. There will be changes underway so the test script will be outdated. To avoid rewriting the test cases or have to come back to it at a later time the developer will delay writing the test cases. Eventually the developer may forget to do it. | Poor test coverage and outdated test cases that cannot be carried out.
Rush at the end of the sprint to get the test script together. |
| Developers update the test specification continuously as they go along. | Will be forgotten because the developer is in a coding mode and the team does not yet have a high quality focus. | The developer may forget to do testing tasks such as performance tests, integration tests, code reviews, security/penetration tests etc.
Rush at the end of the sprint to get the test script together. |
| Developers write the tests when all dev tasks of a story are done. | Who is responsible? Several developers have worked on the user story. It is not clear who should write the test cases so it doesn't get done. | Rush at the end of the sprint to get the test script together. |
| Developers write test cases for each dev task. | Many tasks are not testable [12]. Often there are no relevant manual test cases to write and there are different types of testing needed of different tasks and so the developer would constantly need to think about testing. Not all developers care that much about quality.
Also the test script needs to be organized into a readable non-overlapping set of test cases. You may not end up with a complete and easy to use test script just from piecing together tests from the individual dev tasks.
There is no reminder or test task to check off as completed so the developer may forget about it or defer writing the test cases. | Missing test cases. Tests that are hard to understand. Overlapping tests.
Rush at the end of the sprint to get the test script together.
If you have a lane for testing this does not mean that tests will automatically be carried out. Because some tasks are testable and others not you don't get into a systematic process of writing test cases and eventually things are forgotten or delayed. Also there might not be a clear mapping between the dev task and its test cases so others will not know if the test cases are written or not. |
| Developers write the tests at day x into the sprint. | The developer will always want to complete as much features as possible to look good or because he don't like writing test specs.
Quality is jeopardized because the developer rush to complete the test specification in order to be able to complete the testing before the sprint ends.
This is not Agile. | Testing is postponed, it gets chunked up towards the end of the sprint. You may not be able to complete any stories at all because testing was halted for some reason. The feedback cycle lags behind so critical bugs found during testing could not be fixed in time before the sprint ended or the team have to work overtime, again. |
| Developers test by trying to break each other's code and find bugs. | In this case there is less need to write a test script. Although this may sound fun to developers this alternative ultimately depend on the QA maturity of the developers and the process. If developers are poor in testing they will find fewer bugs. | Without specific tasks with a timeslot developers may do a sloppy testing job especially if there are no guidelines from managers on expected quality levels or if there are no other incentives for finding bugs. |
Some readers might have noticed how central the test script has become in our scenario. Maybe you would like to suggest dropping test scripts all together and just end the discussion right here. Developers should be end to end developers and have the skills needed to test everything along the way as until the story is done [14]. In our scenario with a complicated product this is simply not an option because a test script is absolutely needed for QA to make sure edge cases are both identified and tested. Without a test script you get into a testing procrastination situation. You don't know where to even start testing and what has been tested. Systematic testing is needed in our scenario.
Typical problems in immature Scrum teams when not having testing tasks for each story
Developers usually don't like to do or prepare testing other than technical unit testing and performance tests so you may end up with poor tests and test scripts being ready too late. Developers love to code and develop stuff and often find test-related work boring.
Typically the team will end up postponing testing as a bulk job to be done towards the end of the sprint [
7]. As you don't have explicit tasks for writing test specs and the task of writing them is usually seen as boring work for developers they will tend to focus on development and postpone writing test scripts as long as they can. This means that they will usually start to work on another story before the first one has gone through testing. You are now in a situation where developers try to have all stories ready to be tested some time before the sprint ends so testers can take over and begin their work. This is what we would call mini-waterfall (not Agile).
Another problem is responsibility. Who is responsible for writing the manual test scripts, the acceptance tests etc? Who is responsible for coordinating or doing the actual testing. If there's no assigned person you can bet it will be deferred or not done at all when the QA maturity level is low amongst developers.
Another side-effect of not having testing tasks on the board is that some person will voluntarily or not take the role of writing the test specifications because that person know it is the only way it will be done. This person might be good at it and he might even like to do it. The problem here is that it creates a dependency. What happens if this person goes away on vacation or is busy doing other stuff for a period of time? The testing grinds to a halt or you have a severe drop in test coverage.
Why have testing tasks as part of the task board
 |
| Example of Scrum board with testing tasks |
Better estimates
Time will go into writing test specs, so estimates for it should be registered somewhere. Time will be spent setting up the test environment, doing initial integration tests, regression tests etc. Time will go into fixing the bugs found during testing.
By using test-related tasks you can during Sprint planning have a more conscious discussion around what kind of testing is needed, how much time will be needed and how much extra time should be added for fixing bugs found during testing). With explicit testing tasks you will be able to plan better and come up with better estimates.
Increased Quality
Sometimes tests can only be done by developers. E.g. performance testing or testing that require specific tools e.g. testing infrastructure related stuff. By defining these tests as explicit you don't forget to do them and you have accountability. Everyone sees who is responsible and if it has been done or not.
If you add test-related tasks during sprint planning you will have a slot to think about testing. You will be able to kick-off some thoughts about what needs to be tested, how thoroughly etc.
If there's no defined task for writing the test script you might be pressured to hurry the job of writing the test cases because the burn-down chart will look bad if you spend too much time writing it. When there are no testing-related tasks on the board there are no estimates to burn. The developer might feel he look less productive than other developers who are working on defined dev tasks so the developer may want to just get the test cases done as quick as possible. Not thought through test cases = poor testing.
If you have testing-related tasks on the board you will be reminded if there are some other types of testing missing.
Who is responsible for updating the test scripts when a bug is found and there was no test to cover it? Without a person in charge of overseeing the testing of a user story this may very well be forgotten.
Reporting and status visibility
If you don't have a task for testing the user story you can't display the status of the testing work on the board. There is no way to see if the testing has started and how many hours remain etc. Testing-related work as any other task can have impediments. If you have test-related work on the board it's easier to have discussions about status and impediments.
Efficiency
In our scenario someone in the team has to coordinate or connect developers with testers. Testers will have questions for how to carry out some test task. There are usually questions such as; is it a bug or is it a known issue? A developer has to be available to answer questions such as why a test task is not testable. The correct conditions, for example the right data might not be present to be able to carry out the test. Without a explicit task for the responsibility of coordinating and supporting the testers the tests might very well stop without anyone knowing. You might have unclear communication lines and get extra noise in the team because people don't know who is responsible for overseeing the testing.
Avoid duplicated work. If the developers test each other work it is still useful to have testing tasks so you know which developer is testing who's work.
Continuous improvement
Reducing waste is central to the Agile movement. If everything is visible you get a better overview of everything going on and how things depend on other things. With testing tasks you can more easily improve the entire process.
Without tasks and responsibilities you don't have accountability. If the test script is bad and there are bugs because of poor tests we know who wrote the test spec and we can improve the process by talking to the responsible persons.
References
