Setting a Project Up for Success

After working as QA Lead for two large-scale projects with multiple teams, there are several essential items that have contributed to the successful launch of these products. My primary suggestion for anyone starting a new project:

Picture your end product and identify what needs to be in place for this to become the reality. 

Work backward to a starting point that will allow for the successful delivery of the envisioned reality.

A ski racing friend of mine tells a story of standing at the top of a ski slope with the legendary Billy Kidd who asked, "When you are ready to start your race, what do you think about?" My friend replied, "I see where I want to hit the 1st gate and then the 2nd gate, etc." Billy replied, "No wonder I always beat you. I identify the optimal spot to cross the finish line. I then work backward, starting where I need to be coming off the last gate to hit that spot and working my way up the course to exactly where I need to be starting to have the fastest run."

This got me thinking about the WWT teams I've worked on and what made our deliveries successful. My friend's story resonated with how we approached our projects and set them up for success.

Project starting point

Most projects are defined by the features of an application, API, or some other desired end-user product. What needs to be in place for an end user to have that product?  Working backward from that vision, what enables that envisioned product to be a reality: a downloadable package, accessible via the internet, installable on a device?

This method can work for an IoT app, phone app or any kind of product. For demonstration purposes, let's use a backend service, accessible via an API, as the product we are delivering. We need a server environment that can host the service with a URL that is publicly accessible. We may need to ensure security policies exist if the API is exposing any kind of sensitive data. There are possibly other "end game" things to consider as well: 

• integration with other API services
• support of existing functionality
• desire for continuous, zero-downtime deployments
• data management and migration

There may be more, all Iterating back through the "what do we need for that to be a reality?" 

This leads to the following initial items for successful delivery.

Deployment environment

This is one of the first things that needs to exist. Someplace to deliver code. Surprisingly, clients often find it challenging to realize this needs to be done early in the project. These days, with the ubiquitous availability of cloud services, deployable environments can readily be created.

There should be at least three environments available to enable seamless, quality continuous deployments: 

• Development
• Pre-production staging
• Production environment

The development environment is used to define and refine the delivery of our product. The pre-production staging environment is used to practice delivery to make sure it is correct. Production hosts the delivery of our product. I prefer to have at least one more environment: QA that slips in between development and pre-production staging which allows for controlled verification and experimentation. This supports integration, experimentation for performance tuning, migration strategies, compatibility with existing systems, etc.

Each of these environments will be progressively more production-like so that pre-production staging looks as similar to production as possible. If the deployment model is to have a blue-green release, pre-production staging and production are identical with the exception of the new code/features being developed and delivered.

Setting up delivery pipelines

This may seem like a "duh" item, but there are many ways to do this. You want to ensure that the pipeline is set up to support the final product. Quality continuous deployments require testing of the code, gating of code promotion to different environments, environment variable management, data and schema migrations, as well as a way of building the code once and promoting often. There is also a need for visibility of what is deployed where, and a method to enable rapid corrections. Another thing to consider is code branching strategies. The build pipeline should support the branching strategy agreed upon, while also allowing for alternatives like "hot fixes". 

Each of these elements can have myriad methods of delivery. Some successful patterns in pipelines for high-quality, zero-downtime in production deployment include: 

• Build and test on every code check-in
• Automatic deployment of the package(s) to the development environment on main-branch check-in
• Gating deployments for subsequent environments through manual and/or automated regression tests passing
• Methodology for seeing and knowing what is deployed in which environments. (This is extremely important for micro-service back-ends since it is essential to understand the big picture of what is being delivered.)

If the project is a larger, multi-team project, the pipelines and patterns must be consistent across the teams. Using a pipeline template to standardize each team, code repository, and microservice makes this a fairly easy task.

Automation with the first lines of code

Automation is a buzzword these days. It is essential to support high-quality deliveries. Manual processes are far too prone to mistakes, potentially leading to catastrophic consequences.  If you are going to need something for delivery (unit tests, regression tests, data and schema migrations, environment variables, etc.), automate it with the first drop of code that needs the service and iterates through the automation the same way you iterate through code development. Automate these from the very beginning:

• Environment setup (variables, security policies, network routes, etc.)
• Build pipelines
• Test execution (unit, integration, regression)
• Data and schema migrations

Automation can be perceived as expensive and a velocity inhibitor. Admittedly, automation makes the first few Jira cards and the first lines of code slower to deliver but the investment is well worth it. When the system is more complex and full-featured, automation enables increased velocity of working software delivery. It is almost impossibly expensive to adequately automate once the system has matured and the system's complexity has been established. That doesn't mean you should not start automating today! Start automation with the features being delivered today and backfill as you need to for existing systems.

Automation starts with the first Jira card defining work for the project. 

Digital evolution in action

By asking the questions to identify the product reality and what is needed to make that reality, we are approaching product delivery from a "bottom-up" view. Defining the infrastructure (automated software build pipelines and deployment environments) as the first thing to do has proven business benefits for our customers:

• Quicker time to market
• Observability of business metrics
• Increased confidence in releases
• Better customer experience
• Increased innovation

Getting deployed to production quickly gives everyone fast feedback on the product vision and the experience of that product vision being a reality. This in turn reinforces what is possible in their environment.