Sunday 7 February 2010

Scrum taken to the extreme, doing design of features in the same sprint as they are to be developed

On one of the projects I participated on last year I had the opportunity to work on a large project (more than 15 participants, several locations and several interfacing systems). It was decided to do design of products/features in the same sprint/iteration as it was to be developed. This is not the ideal situation as several others have experienced before me [1],[2],[3],[4],[5] and as described by best practices [6],[7],[8],[9],[10].

The background for this decision was that the clients company profile rebranding project was not finished so the designers did not have what was needed to do the complete design job beforehand. Therefore the design team could not really start their job before the developers. Because of this it was decided to have sort of sub iterations within each sprint. For each sprint a set of products were planned and then designers started designing products as on an assembly line. As each product was designed it was delivered to the developers at once to be implemented.

About the project and team composition
Both developers and designers were quite flexible and competent. The development team had some experience with design and the design team had technical knowledge and could even do frontend programming. The project aim was to develop a new Intranet on a new platform. Re-branding was a key goal. It must be mentioned that the developers outnumbered the designers considerably. There was a fixed deadline and the total project length was about 8 months.

This is what happened
The project started out in good spirit. Everyone was confident and happy but after a while things started to go less smooth. As with all projects with a certain level of complexity there is not just one single thing slowing down production. I'll try to concentrate on the issue of in-sprint design and development (design and development of a product in one sprint).

Planning
Since design was not complete at the start of each sprint, developers could not estimate the job since they did not know what exactly they were to develop. This had the affect that the project lead did not have the slightest idea how much would be completed after each sprint.
Doing design in the same sprint as it is to be developed also made the progress flow extremely unpredictable. The time required for design is often hard to estimate and designers often meet unexpected problems in their design as we developers do in implementation. This means, without any lag between design and development, the risk of encountering blocking impediments are doubled.

Specification and Handover
As designers were to handover several smaller designs it was easy for them to to make readjustments to the design specs as these mini, continuous handovers became less formal. This resulted in frustration on the developers who had already started developing the design.

Because there were no big formal handover and the designers were already overworked it was easy for designers to get in to the trap of handing over items that did not follow the required documentation guidelines. Items got sent by e-mail and people forgot to upload it on the correct place and adding the agreed descriptions etc. This again led to difficulties in tracking changes and finding the specs.

Since the designers worked more or less in parallel with development there was always some spec lacking. Developers constantly needed to ask the designers for specs and after some time all the designer was doing was feeding developers with design. This situation where the designer was pressured also had the effect that designers did not always have time to do good overall design making the entire design harmonize. 

In other cases the designer were overrun by developers and developers started dictating the design because the designer was overloaded with work.

Another side affect was that designers had less time to prepare and get a strong acceptance on the next design. On a few occasions design did not go through a quality assurance loop with the customer resulting in bad features that had to be redesigned later. As each user story was a minor handover stakeholders could not really be involved in the acceptance of the design as it would then slow down the project considerable if several parties were to be included for each of these small sub deliveries. Instead the designers controlled the process and sought out who to contact to get the necessary feedback. This had the effect that some persons did not get the chance to see the design and give feedback before it was delivered to the developers. In some cases this proved disastrous as some important stakeholders had important information that did reached neither the design nor the development team. Some participants felt overlooked as they did not get the chance to participate in the design work or not even being able to comment on the design proposals.

Communication and noise
Often specs were not very detailed and in many cases completely absent. Less documentation is part of the agile manifesto but since we had developers at several locations this became a problem. Less documentation means more face to face communication and communication with instant messaging, phone, e-mail, but in large projects, participants tend to have a lot of meetings and thus not being available for communication about specs. Developers often found themselves sitting and waiting for people to come back from meetings when having questions about some design that may have had an incomplete spec. In other cases they just started on some other product which in turn led to a chaotic work environment when the developer ended up with working on 4 user stories at the same time. Not being able to complete a module/product adds noise in the project as the project needs to bring this product up in each scrum meeting and you have to tell once again why the product is not complete. Several products also seemed half finished since there were no complete specs and relevant persons were not available at the same time to clear up design issues.

The incomplete or lacking specs had the effect that everyone on the project needed to communicate a lot more. Time was wasted as messages had to be repeated. As much of the work surrounding specs or the specs themselves was replaced by face to face communication people outside the core group had to ask for information, this resulted in multiple and overlapping communications.

The IA/GD/UX team will always have some questions for the developers but since both design and development is done continuously and in parallel there will be a continuous flow of inquiries for the developers. Instead of having planned design workshops were designers could get feedback from developers and others, we were doing development and design in a continuous and overlapping fashion. This is not optimal for a typical developer. Developers will always be prompt and ready to delve in to planning, problem solving and answering questions in the planning phase of the sprint. But, once developers get down into the nitty gritty programming, many developers will have their head wrapped around difficult tech problems and are not in a mindset to constantly respond to small design issues put forth by designers. Many programmers are also competitive. Non-programming related distractions in the middle of the day are not always what the developer is most keen on. This had the effect that some design problems were not well thought through because the solution was based on feedback from a developer who had other more important things to think about.

Experienced developers want to know what features should be developed next so they can chunk together related work including fixing bugs on related stuff. But as they now live in this day by day design hand over they did not know what features would come the next day and ended up scratching their head trying to figure out this himself or trying to find this out from the IA or designer. The designer works on other things so there is noise. Not being able to chunk related work led to less efficiency.

Workflow, predictability and work conditions
In the early sprints developers tended to have little work in the start of the sprints as they were waiting for design. In the end of the sprint developers were often unmotivated from taking on new design since it was something completely new that they might not have seen before. Completely new design are more intimidating than a design that has been slowly setting in the developers head during the sprint and where the developer has gotten the time to figure out how to best solve the task.
As designers were fully entangled in pushing design they had little time to look over the end result and developers had to go ask designers to test if the implementation of the design was ok.

Developers did not know when they would get a design and what it would look like. This led to unpredictability and unsystematic work making the flow less streamlined and more ad-hoc.
It all became very unstructured. Some people, often the creative ones thrive in chaotic environments but for many developers, often systematic engineers, it became very messy.
For new developers, especially the juniors, coming on to the project late it must have been a nightmare not having the knowledge that had been communicated only by mouth and feeling that they add even more noise in the project. For some of the seniors on the project it did result in a lot of extra guidance and support on these new participants. Incomplete specs were even more problematic for new project participants because they did not have the big picture. So again more noise.

Technical forward thinking, architecting and code quality
Since we did not know what design would come later we could not estimate development time sufficiently, therefore the tech lead did not know if we had time available to do maintenance  refactoring and other things that would benefit the project in the long run. All he could do was either concentrate on getting things up and running or taking the chance on doing refactoring and risking not being able to complete later features.

In general there was a lot of unsystematic work and unpredictability making it harder for the tech lead to plan in advance and see where to do refactoring etc. Unit testing was out as everything was changing day by day. Setting up unit tests takes time and is a waste of time if the requirements change and the code is likely to be totally rewritten the next week.

As specs seemed incomplete or being suddenly changed developers often became demotivated knowing that what they developed would probably have to be rewritten. This resulted in lower productivity, poor code quality and features that were only partially completed.

Since designers were overworked they where not always available to do these kind of in front of the screen 2 minute designs that often are needed for some small thing that was overlooked in the design phase. In this case the developer just design the thing him self.

Testing
Some developers felt stressed out as some design was delivered just few days before the end of the sprint. Most developers want to deliver as much as possible so this results in a situation where the developer wants to produce new features but gets overworked and forgets the importance of testing or preparing for testing of what has already been implemented.

Because of poor specs, test scripts were poor and this resulted in test results that were not realistic and testers did not know what had been completely developed. Some products were set as ready for test before they should have.

Because specs was hard to find specs were in some cases even skipped. Some features were even developed without a developer looking at the specification. In some cases features were developed without a spec as there where no spec available. In many cases this was not a problem as the feature worked out quite well. The designer could then go over afterwards and suggest improvements.

Because design was immediately delivered to the developers when ready it did not get to mature and therefore some design was a bit premature and ended up as features that eventually had to be replaced. If the designers had a week or two to digest and circulate the design it is more likely that the design would be better and that it would lead to a better solution.

Coupling design specs to prototypes to issue/task tracking and to test scripts became difficult. For project participants not sitting next to the designer it was hard to know what design was finished.

Incomplete specs led to vague definitions of done criteria and test scripts were hard to write.

Summary of problems with doing design in the same sprint as it is to be developed
  • Difficulties planning and estimating work
  • More unpredictability and increased risk
  • Poor documentation and traceablity and findability of documents
  • Often change in specifications
  • Design not always harmonizing
  • Redesign and reimplementation because first version was not good enough
  • Relevant persons are not included in the design phase and felt overlooked
  • Noise, duplicate messages and repeated tasks
  • More waiting for clarifications
  • More difficult for new and remote participants
  • Design and development cycle partly broke down
  • Unit testing got more difficult and code quality got worse
  • Lower productivity
  • More half finished features
  • Difficulties in writing and organizing tests
  • Poor tests and test results
  • More distractions and chaos
  • Less happy team members
Discussion
Again, it is difficult to exactly know what factors led to the problems listed above. There were several other decisions that are likely to have contributed to this situation. Having analysed the situation and thinking back on other projects I do believe the list of problems are representative to the issue of doing design in the same iteration as it is to be developed.

Being a developer on this project I am certainly not without some bias, so I'm inclined to say something positive about the process. But hey, as I'm sitting here writing this blog post I can't really come up with any benefits of setting up the process this way. Doing design in iteration i+0 might work perfectly well for small co-located teams but for a project with scaling factors [11] in a large corporation having strict requirements on compliance, traceability etc with a large distributed team this just don't work very well. Agile is all about good communications [12], but our situation got to the point of communications overload and the development process almost broke down. 

What can you learn from this?
Do not do design in the same sprint as it is to be developed. The design team should always be at least one sprint ahead of the developers.

The optimal solution would be for the designers to have a complete initial design ready before the bulk of developers start development. During this design phase selected developers, stakeholders and customer representatives etc help form the initial design. During development designers are tightly integrated into the group but are mainly working on evolving the design based on user feedback and on unexpected technical limitations etc. They would also focus on specifying details or things that have been overlooked.

How to avoid this situation
Plan and make sure managers, project lead, steering committee, product owner, decisions makers or other relevant persons are aware that designers should start ahead of developers.

What to do if you end up in this bad situation
  1. Have the developers implementing core features without any UI design while the designers are working on the initial design. Get the solution up and running. After all this is what Agile is all about; making sure you have something working and then iterating to improve it.
  2. For projects with scaling factors [11], formal design specifications are more important than on smaller co-located projects. A person can be assigned as responsible for documentation making sure specs are in order. If designers don't have time to do deliver good specifications then someone else should be in charge of writing up and formatting the specs.
  3. Make sure the project is staffed up with a enough designers. This will obviously depend on the type of project and aesthetic requirements. If there are 8 developers you might need 4 designers, such as an Information architect, an art director, an user experience designer and a graphical designer.
  4. Using teams of designers and developers that have previously worked together on projects with the same technologies. This will minimize the friction between designers and developers and reduce uncertainties.
  5. Designer(s) must be tightly integrated and co-located with the project.

References