Product and Project Success Criteria: How Do We Know If We're Succeeding?

How do we know if we're succeeding?

This is a question that many people struggle to definitively answer for their projects. Success can mean different things to different people depending on their perspective. Simon Sinek tells us in his Start With Why Ted Talk and book that we should be asking what is our cause, our purpose and our belief about why this product should exist. 

This is what inspires teams to build it and customers to buy it. I think this advice rings true for success criteria as well. 

Many projects have attempted to assess success based on the project management iron triangle (i.e. cost, scope, schedule), but you can lose sight of the true overall goal of the project. You can't ignore the iron triangle, but when determining the success of a project, as Angelo Baratta points out, these are more constraints rather than measures of success, and identifying the value of a project is what matters.

The goal of this article is to help teams and stakeholders have meaningful conversations about what success means to each of them and align on a set of shared criteria that can be tracked, reviewed and modified as the project evolves.

When thinking about success, it can be helpful to view it through the lens of the product (what are we hoping to achieve by building this?) as well as the project (how do we build trust and learning as we go?). By aligning on these two aspects early on, it allows teams to make better decisions, track progress against goals and develop a shared language for discussing success.

Product success criteria

Product success criteria should support the product vision and clarify how the product will benefit the organization. Jesse Lynn Stoner says that "a vision is a clearly-articulated, results-oriented picture of a future you intend to create. It is a dream with direction." 

Product success criteria give concrete examples of how the vision will be measured. These are aspirational goals for what the business believes will happen if we build "feature X" or "product Y" and help add clarity to the product vision. 

Impact mapping developed by Gojko Adzic is a great way to discover product success criteria. This method involves getting stakeholders, businesspeople and technical folks together and working "backwards" from the goal to the features/stories, uncovering ideas on how to solve a problem. For the purposes of success criteria, identifying the goal, actor and impact is usually enough (see Figure 1). However, if you are feeling inspired by the process, continuing on might help with identifying and clarifying the features needed to achieve validate the goal.

In the example above, you might derive a success criterium of:

• Grow mobile advertising by X percent by engaging super-fans with mobile devices to come back more frequently.

As with almost all product success criteria, this is a hypothesis that will have to be validated, and it's entirely possible that the hypothesis could be wrong. That's okay — everyone learned something, and you can move on to the next idea.

A strength of this method is that it takes into the account the "why" and the "who" before you start trying to figure out the "what" and the "how." This helps everyone stay focused on the real problem trying to be solved, rather than getting hung up in the details.

Maybe impact mapping isn't right for your situation. Another method for discovering success criteria is the 1-2-4-All method outlined in Liberating Structures. This method's strength is that it leverages the creativity of the individual, along with the synergy gained by pair and then group collaboration. When facilitating this method, you'll need to pose a question to the team, such as:

• At project end, I sit down and open up the application — what can I do that I can't do today?
• What change in user behavior am I trying to achieve?

Following the question, you'll have everyone ideate on the question — first alone, then in pairs, then in groups of four and then with the whole group. You can easily modify this model based on the group size. The important part is to make space for people to be creative on their own and then start collaborating.

It's also entirely possible that you have a product owner or stakeholder with a clear idea of product success. If so, you may just need to make sure everyone has a shared understanding. 

Other examples of product success criteria might include:

• In Q1, decrease abandoned carts by 25 percent by making the checkout process easier
• In Q2, make information more easily accessible to analysts so they can deliver their results more quickly.

Project success criteria

Project success criteria should be focused on enablers to learning and delivery. These may include things like DevOps practices, such as 1) improving your delivery pipeline or ensuring your team is creating a 12-factor app, 2) non-functional requirements like building automated performance or load testing into your pipeline, 3) modernizing a piece of the tech stack for your app or 4) something as simple as building a high trust partnership with the client and stakeholders. Project success criteria can help the team achieve the following:

• focus on the right things early on;
• drive conversations about trade-offs between feature and enabler work;
• drive clarity of understanding;
• help to identify early dependencies;
• identify areas for improvement; and
• allow the team to monitor progress.

Examples might look like:

• In Q1, put an analytics framework in place to measure our hypotheses.
• Within 4 weeks post-kickoff, start delivering to production or a production-like environment once per week.
• Within the first 2 months, increase number of production deployments per month from 1 to 4.
• Engage beta users by June to start getting feedback.
• Throughout the engagement, build a collaborative partnership between the team and the stakeholders.

Questions that may help define project success criteria include:

• What improvements are we hoping to make in non-functional requirements?
• What is the expected deployment frequency?
• What kind of relationship are we hoping to build during this project?

Visualizing success criteria

Both product and project success criteria can either be qualitative or quantitative and normally number between 2-5 for both project and product success. Visualizing these on a whiteboard can help the team ensure there is a balance of each type and at least some are easily measurable.

Revisiting success criteria

If we are defining success criteria at the beginning of a project, it's likely we didn't get it completely correct since this is when we know the least about the project and the product. The second principle in the Agile Manifesto is about welcoming change, so revisiting what success means on a regular cadence keeps everyone in alignment as the project evolves. 

Longer projects of a year or more may only need to revisit success criteria quarterly, while shorter projects may benefit from reviewing monthly, with a bias toward more frequency early on in the project. This could be as simple as taking five minutes at a stakeholder demo, including them in your project status report or displaying them in your team area to ensure everyone is still in alignment on success criteria. 

Tracking progress around the success criteria and checking items off along the way is also a great way to recognize good work and keep the team motivated. If you are finding the initial success criteria no longer make sense or think you need new ones, you can always revisit the discussion and realign.

Ready to get started? Learn more about our development expertise or reach out directly to start the discussion.