The process we follow is a blend of ideas from various different methodologies that we’ve used throughout the years. It is primarily based on Extreme programming (XP), Lean product development (LPD), and Kanban. We’ve cherry-picked the concepts that work for our types of projects and fused them into a single, coherent and evolving process.

The synopsis is that we work in short, time-boxed development cycles referred to as iterations. In the beginning of every new iteration we meet to reflect on the current state of the project. The client lets the team know what items should be prioritized, and the team formulates a plan. Work is carried out throughout the course of the iteration. The goal is to end each iteration with fully tested and useful updates that can be shown to the client and released to the end-users.

The process is centered around:

• Eliminating waste and reducing time to reach a usable product. The project plan can be re-adjusted after each iteration or as new needs are discovered.
• Increase predictability and manage expectations. New features are shipped frequently and deployed through a continuous delivery cycle.
• Focusing on impact and obtaining rapid feedback. The client plays an important role in deciding the order in which new features are added.
• Improve quality of the product. Breaking the work down into mini-releases allows us to continuously test and review the product.
• Increase trust and ensure clear and open communication. Everyone knows the current state of the project and what is being worked on at any given moment.

Team and roles

The team is a self-organized, cross-functional group of people that share a common goal. Communication and collaboration happens in real-time and is open across all roles.

Product owner

Party: Client

The Product owner conveys the product vision and mission, and provides access to domain expertise. They represent the client in all things and must have authority to say what goes (and does not go) into the final product. The Product owner is considered part of the team and is expected to be actively engaged throughout the entire project. Responsibilities include:

• Conveying an up-to-date version of the project goals.
• Prioritizing strict requirements making it clear to the team what is most important for the project from the business’s perspective.
• Working with the organization and the Technical product owner to discover what is important from the end-user’s perspective.
• Providing information about the domain, business, and stakeholders.
• Act as a bridge between the project and the stakeholders.

Technical product owner

Party: Helicon

The Technical product owner organizes the team and project as a whole. They have a good understanding of what the outcome should look like and understands the project requirements. It is a dedicated role that is primarily responsible for:

• Evaluating and expanding on project requirements to make sure that relevant items are actionable.
• Helping the Product owner prioritize requirements in the backlog.
• Scheduling and coordinating meetings.
• Keeping a birds eye view on project progress and resources.
• Protecting progress by making sure that there are no distractions or bottlenecks.

Technical lead

Party: Helicon

The Technical lead ensures smooth operations from a technical perspective. It can be a dedicated role or a person that actively participates in the development. Responsibilities include:

• A holistic view of the system architecture and overall quality.
• Working together with the team to decide who does what and with whom.
• Motivating the technical team members and maintain a good, energized working environment.
• Maintaining a healthy ratio between quality and velocity.
• Keeping up-to-date with, and research into, new technologies and techniques.
• Final review before code goes into production.

Technical team members

Party: Helicon (and sometimes Client or third party)

The team members that carry out the technical implementation of the project. A team always consists of developers, but can also include other roles such as dev operations engineers, data engineers, data scientists, UI and UX designers, security experts, testers, specialists, and system architects. Responsibilities include:

• Day-to-day implementation work.
• Raising any concerns that may arise.
• Come up with solutions to problems or opportunities identified by the Product owner.
• Contributing with ideas on how to do things.
• Giving feedback on how to adapt the backlog and roadmap.
• Maintaining technical documentation and tests.
• Monitoring, automation, and infrastructure.
• Works together to solve the tasks at hand.

Stakeholders

Anyone that is directly or indirectly affected by, or otherwise related to, the project (e.g. client’s customers, users of the product, other development teams, IT, legal and compliance, marketing, managers, domain experts). Relevant stakeholders should be identified and their roles established at an early stage. Some of these stakeholders can, but are not necessarily considered to, be a part of the team.

Process and fundamental concepts

Initial research phase

We do enough research and discovery to understand the primary end-user and business needs. We look for insights that help us identify what to focus on, uncover unspoken needs and opportunities, and obtain a fundamental understanding for the domain. Our goal is to understand the vision and to gather enough information to get started, but we leave room for changing requirements.

This phase should result in:

• An initial idea of what the main problems worth solving are.
• Identify the smallest possible releasable outputs that provide the highest value, and an understanding of the work required to achieve them.
• A short document containing:
  • A brief description of the project.
  • A list of intended users.
  • Any non-functional requirements known at the time.
  • A list of who fills what role (Product owner, Technical product owner, Technical lead, and important stakeholders), including relevant contact information.
• A plan for the first 1 or 2 iterations of work.
• A rudimentary system architecture, simplified wireframes, and a plan (very rough time frame, tools to use, and skills required).

Roadmap

The roadmap is a high-level plan that gives a long-term perspective on different broad requirements and strategic goals. It describes the vision, direction, and how a project will evolve over time.

A good roadmap:

• Explains how short-term efforts align with long-term business goals.
• Is the primary tool used when presenting strategic plans to management. Most executives do not deal in stories, requirements, or features. They want clear strategic explanations and reasons to move forward.
• Helps the team understand how their work contributes to the bigger picture and enables them to make autonomous decisions.
• Provides updates on the status of the work that is easy to understand for stakeholders without a technical background.
• Is coordinated with all stakeholders (e.g. end-user needs, supports sales conversations, corresponds with the overall company vision).
• Is continuously updated throughout the project.

Useful tools: Miro, Trello, Jira

Backlog

The backlog is a prioritized list of items that are considered to have a high impact on the project, but on which development has not yet started. The most important and well-documented items are found at the top of the backlog. Unplanned features, rough ideas, and concepts are found closer to the bottom.

There are many ways to store items in the backlog. For more refined items (e.g. things that are not just rough ideas) we use stories, tasks, and bug reports.

Now, we often describe the backlog as a single comprehensive list. This is rarely the case. It is often divided into multiple different lists with different purposes. What lists we use depend on the specific needs of the project. A couple of common examples:

• Next: High priority items can be moved from the main backlog into a list called Next. Separating these tasks from the main backlog makes it easier to identify items that have gone through review and are in fact ready to be worked on.
• Blocked: Items that are blocked are usually kept in a separate list to distinguish them from other items. These are high-priority items that, for whatever reason, could not be completed during an iteration. This list should be checked frequently and any item that has become unblocked is put in play again.
• Requests: Feature requests from end-users and other stakeholders.
• Ideas: New product ideas that need to be ironed out.
• Bugs: Bugs or defects often have their own prioritization.
• Icebox: De-prioritized items, ideas where the value isn’t clear, or items that may never materialize, are kept in a separate list. We keep the items here to prevent them from obscuring more important items in the main backlog. This list should be pruned often, otherwise it just becomes a graveyard.
• Rejected: Features that have been discussed but that we’re fairly certain aren’t going to be pursued. Having a place for these items can reduce the psychological resistance often associated with removing ideas entirely. It is also a good place to track reasoning behind not pursuing something if it comes up again.
• [Initiative]: It is often useful practice to separate larger initiatives into their own lists during planning. These initiatives often correspond to items on the roadmap (e.g. “MVP release”).

The backlog is one of the most important tools we use in a project. It is the primary means by which the Product owner prioritizes the work to be done. Keeping the items in the backlog arranged like this allows the team members to pull work, as there is capacity, into iterations.

In order for this to work the backlog must be continuously managed and kept up-to-date at all times:

• Backlog prioritization: Items are prioritized by urgency of the end-user, need for feedback, business needs, difficulty of implementation, synergies etc. And this must happen as soon as new information becomes available.
• Backlog pruning: The backlog must be kept lean in order to make prioritization possible, and to not lose sight of important requirements. During pruning we remove duplicates or items that have been solved by other means. It is also a good idea to remove items that have been inactive for too long.

Related activities: Splitting stories, Estimation, Backlog management
Useful tools: Jira, Trello (notes on how we use it here)

Estimation

An important element to consider when prioritizing the backlog is often the effort required to complete something. A Product owner might consider feature A more important than feature B. But what if A takes N amount of effort, and B, C, and D, together, only require an effort of N/2? That information could affect the order in which things are built.

It is also important to have a hunch about the effort required to complete the different items when planning a new iteration. Is it realistic to expect the team to be able to complete items A, B, and C during the next iteration?

Yet a maxim of software development is that it is very, very difficult to accurately estimate development time. Not only is it hard to get right, it can also be very time consuming. There are many uncertainties and variables to take into account, and a lot can happen during development:

• Software development is not just about writing code. It requires research, communication, debugging, testing, and time to think. Cerebral activity, in particular, is almost impossible to predict.
• Code isn’t written in a vacuum. There are often external dependencies that can be very hard to control.
• Requirements change or are not specific enoughi. Details that might seem insignificant to the Product owner can have a big impact on the effort required.
• Interruptions and time spent context switching cannot, with any certainty, be forecasted.
• There’s always a bias during estimation. While one developer wants to look good and offers an optimistic prediction, another might subconsciously plan to over-deliver and is overly pessimistic.
• Unanticipated challenges and uncertainties. Most development work requires the developers to do something they’ve never done before. More seasoned developers might have done something similar before, but there are always new elements and surprises.
• Who is going to implement something might not be known at the time. Experience and other attributes differ between developers and affects the time it takes to complete something.

How do we proceed then, if estimates are in fact important? We start by being very honest and specific about how accurate we need the estimates to be.

Rough estimates

During backlog management, sprint planning, and most day-to-day tasks we can often proceed with a rough understanding of the effort required to complete something. We do not need to spend large amounts of time on creating more accurate predictions.

For these types of estimates we employ either one of two techniques:

• T-shirt sizing is a measure of relative size using the terms: XS, S, M, L, and XL (XXS and XXL can be used for fringe-cases). This practice helps us focus on the size of the task at hand, rather than the time it would take to complete it. The team talks about the item briefly and assigns a size. Consensus is reached by means of discussion, or by open/closed voting.
• Story points is a very similar concept, but instead of T-shirt sizes we rely on a numerical scale. Having an infinite number of points to choose from can be confusing and lead to too much time being spent on finding just the right point. Using the Fibonacci sequence is a good way to limit the scale (i.e. 1, 2, 4, 8, 16, 32).

What each size represents is not be explicitly quantified. It is a subjective estimate of an item’s size, complexity, and risk. A good way to get started is to select an item, estimate it, and use that items as a baseline (e.g. “Story A is a M, therefor B is probably a L.” or “Task X has 2 story points, Y is roughly double the amount of work and is given 4 story points.”).

More accurate estimates

There are times when more accurate estimates are required. When the client has to give another department a general timeline, or when two more more features need to be thoroughly compared, for example.

First we make it very clear that an estimate is, in every way, an estimate. It is a very educated guess, not a deadline or a promise.

It is important that the developers who are going to do the work are involved in the estimation process. Previous experience and individual knowledge is going to have a big impact on the predictions, and the best way to take that into account is by letting the developers themselves participate.

We break the problem down into several small actionable tasks. We analyze each task thoroughly, define the exact scope, and try to identify any unspoken requirements. These tasks are then estimated individually. We try to be as realistic as possible and are wary of best-case scenarios.

We deal with uncertainty by doing research, exploring different solutions with simple prototypes (often referred to as spikes) or wireframes, and by applying a multiplier based on the level of uncertainty. This multiplier commonly ranges from 1.1 when the level of uncertainty is low, to 4.0 when it is high.

We monitor everything closely when the work starts, and the estimates are continuously refined as progress is made.

The estimation activity should be a first class item in the backlog and prioritized like any other item (and preferably estimated – we seldom want to estimate something if the estimation activity is bigger than actually building the feature we’re estimating).

Related activities: T-shirt sizing, Spikes, Planning poker
Useful tools: Firepoker.io

Iteration

An iteration is a time-boxed development cycle that spans either one or two weeks. It starts with a planning phase, continues into a development phase, and ends with a review and a retrospective.

The length of an iteration is determined by the team and the decision is based on the project needs. One-week iterations lead to faster feedback loops, but leave a smaller window for development relative to the iteration length. These conditions are reversed for two-week iterations. Regardless of which, iterations are never considered deadlines. Their purpose is to create a rhythm in the project that allows the whole team to pause, reflect, learn, and adapt.

We rely on lists to track the status of individual items during the iteration. Every item starts in the list Current iteration. We move it to In progress when it is time to start working on it, it goes into Ready for review if it needs a review, and is finally moved to Done when the work has been completed. Items that are blocked are moved to Blocked.

Related activities: Code has to be scrutinized before it goes into production, Iteration planning, Daily meeting, Iteration review, Iteration retrospective

Design

Refactoring is a recurrent part of day-to-day programming. It is a process where we simplify code in order to make the addition of new changes a smoother experience.

Serious flaws in a design, however, cannot be solved by merely refactoring. Design flaws require a redesign. Redesigns are costly and should be prevented by spending ample time designing the solution well in the first place.

Time must be allocated as needed to consider the design of the solutions being implemented. By the time we start coding we should have a good idea of what we’re doing and how it will fit into the existing codebase.

This doesn’t mean that we have to go back to doing waterfall-style-projects, what it does mean is that design has to be seriously considered throughout the project.

Continuous integration and delivery

Work completed during an iteration is automatically integrated into the current environment dedicated to the purpose (often development or staging) as soon as it is ready. The entire test suite (including linting) is automatically run on each pushed commit, and any failures result in a rejected push.

All artifacts are built automatically and an internal release plan decides when the updates are delivered to the end-users (i.e. put into production). The most common strategies we use:

• Changes are made available to the end-users as soon as they’re done (Facebook refers to this strategy as “Move fast and break things”). It is a good strategy when you need access to end-user feedback quickly, but the organization needs to be able to support it.
• Each iteration ends with a new release. A variation of the above, but releases are a bit more controlled (e.g. a specific amount of time each iteration can be dedicated to pre-release activities).
• Regular releases are scheduled and focused on in sequence. We create a specific branch and work on that for the duration of the release. Work can continue on the next release in a new branch in parallel with testing of the branch currently being released. Bugs are fixed on the previously released branch, deployed, and merged into the current release branch.

Documentation

We generally place more value on delivering working software than writing comprehensive documentation. With that said, there are certain cases where documentation is essential.

Any documentation we do produce should have a clear purpose and be kept simple, short, and to the point (just barely good enough to serve its purpose). It is important to realize that documentation, much like software, is not just written – it needs to be maintained.

Project README

Every project should contain a README that provides the following:

• What the project is/does (optionally also the motivation behind it).
• How to install and run the project.
• Usage instructions (how to use the project, run tests etc).
• Instructions on how to access the API reference.
• Anything in particular that should be considered when deploying the project.
• Any known shortcomings or flaws.

Project log

Keep a centralized document that contains important project-related information:

• System architecture.
• Technologies used.
• Operating procedures and backup routines.
• Universally relevant non-functional requirements.
• A list of intended users and their personas.
• Primary roles, stakeholders, and contacts.
• Write down important decisions and motivations behind them, agreements with stakeholders, and reasons for doing things certain ways.

Don’t neglect the last item. Those are things that will be forgotten or missed as people come and go. Understanding why you opted for library X instead of Y, or why a service was designed in a particular way, can save a lot of time later down the line.

Meeting logs

Write a quick summary of every (meaningful) meeting and keep it somewhere accessible. Information from meetings that is not recorded has a tendency to be forgotten or be misquoted later. It is also a good way to keep people that were not able to attend informed.

You can use the following template:

API documentation

API documentation should always and without exception be automated. Never ever should anyone be writing API documentation manually. Doing so is extremely time consuming and manually written API documentation is impossible to maintain.

Meetings

We’re a bit careful with meetings. Not because we do not love them, but because they take time. Time that could otherwise have been spent writing code. Having that said, good communication is a pillar of any healthy project, and a well-planned meeting where everyone understands their role and why they’re there is a blessing.

Iteration planning

• Who: (Product owner,) Technical product owner, Technical lead, and Team members
• When: At the start of a new iteration

The team reflects on the current state of the project and if anything needs to be adjusted based on findings from the last completed iteration.

Items are pulled from the backlog into the new iteration. Effort is roughly estimated in order to assess how work much can be comfortably completed during the period. Judgement calls can be made if synergies between some items exist or if there’s something blocking work from being carried out.

Larger items are broken down into more manageable logical chunks of work. We want to prevent large stories or tasks that span entire – or even multiple – iterations (e.g. “Create Kubernetes setup”) and end up with an iteration where all items are of a similar size.

Sufficient time should be added to properly test and review the work being carried out during the iteration. Don’t be afraid of being too conservative when planning the iteration. It is easy to add more items to the sprint if there’s nothing left to do.

The Technical lead works with the team to determine who does what, which items are suitable candidates for pair programming, and which items should be reviewed and by whom.

It is important that everyone gets their say when planning an iteration. The people in the trenches often have very valuable insights about, and an intrinsic understanding of, what they’ve been working on and can identify problems or potential benefits that might not be immediately obvious.

Daily meeting

• Who: Technical product owner, Technical lead, and Team members
• When: Every day

A short (normally 15-20 minutes) meeting where the team coordinates itself and the current iteration. These meetings are held every morning to ensure a tight communication loop throughout the iteration. It is also a forum to share quick insights related to the current iteration or the project as a whole.

The team members take turns to describe what they did the previous working day, what they are going to work on today, and if there’s anything blocking or impeding progress. The developers are also encouraged to describe how they plan to tackle problems so that their plan of attack can be briefly discussed with the team.

Detailed discussions are deferred to separate meetings. Team members with relevant experience and knowledge join these meetings, and the rest of the team can start their day without delay.

The daily meetings are not about micromanaging the team members. It is a tool used to encourage communication and to get a realistic picture of the current state of things.

Iteration review

• Who: Product owner, Technical product owner, Technical lead, Team members, any relevant stakeholders
• When: At the end of each iteration

A review of what has been accomplished during the iteration. The team walks the client through the outcomes and feedback is collected.

It is not a presentation, but rather a collaborative work session. We discuss any potential changes to the backlog or roadmap and talk about what should come next.

Iteration retrospective

• Who: Technical lead and Team members
• When: After the iteration demo

Meeting dedicated to discussing what went well during the last iteration and what could be improved upon in order to increase efficiency, happiness, communication, and quality.

The meeting is focused on improving the next iteration, not on placing blame on anyone.

Backlog management

• Who: Product owner, Technical product owner, (and, as needed: Technical lead, Team members)
• When: As needed

While the backlog should be reviewed continuously, it is often good to schedule regular meetings to ensure that all items are still relevant and correctly prioritized. The goal with the process is to:

• Ensure there is a sufficient backlog of work to cover at-least two iterations.
• Proactively adjust the backlog in line with new insights so that it correctly reflects the current goals.
• Identify any items in the iteration, or high up in the backlog, that need additonal steps in order to proceeed (dedicated time to solution design, prototypes etc). Use the Technical team here.
• Correct assumptions in light of new discoveries.
• Make sure everything at the top of the backlog is converted into stories, tasks, and bug reports.
• Split stories and tasks into smaller, more manageable, counterparts.

Code quality review

• Who: Technical lead and Team members
• When: Once every 2 months or as needed

Quality is continuously evaluated, but it is still healthy to have a meeting dedicated to the current state of the quality of the project.

Topic meetings

• Who: The person seeking information and whomever has the relevant experience or knowledge.
• When: As needed

Meeting with a well-established agenda called to discuss a specific topic. All relevant stakeholders are invited.