Ensuring Software Quality in the Cloud
In this article
In some respects, building software has never been easier than it is today. Despite a proliferation of platforms, frameworks and programming languages from which we must choose, we enjoy outstanding and constantly improving tooling. Integration among disparate platforms is a focal point of growth for industry leaders. And the ubiquity of software in our everyday lives also ensures a vast global knowledge base that assists software developers and teams to tackle many challenges.
However, no matter how easy it may become to "quickly throw together" a software application or service, your organization's ultimate success depends upon the overall quality of your output. This idea of quality, with respect to software, encompasses many different facets of delivery to your customers.
Yes, we like to strive for minimal bugs in our codebase, but quality is much more than that. You want users—whether external consumers or internal personnel—to feel delight using your software. This requires, among other things:
- an intuitive, non-confusing and consistent user experience;
- excellent performance, with limited degradation under load;
- resistance to failure with clear error messaging and reporting; and
- strong resiliency, with quick recovery in the event that failure does occur.
Low-quality software inevitably leads to reduced usage, which can prevent your organization from attaining your business goals. It is for this reason that WWT encourages dedicated quality advocacy and well-considered, automated test plans for the majority of software projects.
Enter the cloud
The growth in maturity of multiple public cloud platforms over the last several years has changed the technology landscape in many ways. With increasing numbers of organizations looking at a cloud-centric future, focus has understandably shifted to how to they can strategically enable and migrate their cloud transformations. Through all of this, it is important not to lose sight of a continued focus on quality and delivery, which still must form a part of that successful cloud strategy.
Benefits of testing in the cloud
While a cloud migration—like anything new—adds certain complexities and considerations, it also provides some unique opportunities to improve and streamline your testing and quality control.
An unknown, but very wise, person once said:
The idea of test and staging environments for pre-release deployments is certainly not new. But when you are building your software in and for the cloud, procuring and utilizing test environments is easier than it has ever been.
The bottom line: Don't rely on testing your systems in production. Please.
With the rise of various infrastructure-as-code tools, as well as the use of containerization, one can easily spin up new environments, run deployments and execute suites of tests in them, wipe them clean, and destroy or recreate them at will. With proper planning and experience, this can be done in a "push-button" manner easily, cheaply and ideally automated.
Good test data is also important. Leverage test data scripts to duplicate real-world, scrub the personally identifiable information and deploy. Test data is always hard to come by, so automate the process of scrubbing production data. Testing with anything other than production-like data is not particularly helpful, and can even be a detriment to your efforts.
Being in the cloud means you don't have to replicate an offline version of the environment locally. While you should not test in production, the further you get from a production environment, the more likely you are to encounter problems. Or rather, you are solving problems that may not exist, or are different, in the "real" environment.
Provisioning multiple environments means your team will not be stepping on each other's toes. It is hard to work when a live system is constantly changing, making it difficult to pin down exactly what caused a defect. Separate environments also mean logging is way more useful to debug and test.
General quality guidelines
Having established that quality continues to be an important factor in your cloud strategy, what kinds of items should be included within your testing plan? To begin, there are general practices that you should be following already, regardless of whether you are deploying software to remote machines, on-premises data centers or the cloud. The migration of an application to the cloud is a great time to revisit these.
Functionality
Functional testing is often the first step of a successful testing plan, with or without the cloud. Does the application do what it is supposed to do and respond the way that it should? This is called "happy-path testing," where you assume everything goes right. It's a fairly narrow path.
The unhappy path
It's also important to ensure that your software does not do what it is not supposed to do. This requires a far broader range of tests, encompassing the vast set of ways things can be done by users that are unexpected, malicious, mistyped, timed-out, interrupted, disconnected, etc. This is typically the bulk of your testing effort.
The unhappy path also includes a range of related tests: security, accessibility, usability, localization and internationalization.
Security
Good security testing, inevitably, will save you. It will help keep your data safe, it will keep your customers safe and it requires care and attention to detail. Unfortunately, there are countless examples of major organizations being on the wrong end of a data breach. No solution is infallible, but taking security seriously can go a long way to protect you.
Creating your own custom security system is not recommended. Many companies have tried-and-tested solutions you can integrate into your solutions that greatly reduce your risks of a security incident. And it's worth noting that using a dedicated third-party security solution can help shift liability away from you if the worst happens.
Accessibility
It is always important to leverage the talents of a user experience and accessibility expert in your testing and quality assurance planning. It goes without saying that you should avoid excluding users because they are not physically or mentally flawless. Beyond the practical concerns, there are often express regulatory requirements as well.
Usability
Your user experience experts can also assist with validating the overall usability of customer-facing applications. Is the software frictionless? Does it provide value to and delight the user?
Localization and internationalization
Are your available users potentially around the globe? If so, pay attention to internationalization. Dates and times are hard. Is your system using server time? Client time? Zulu time? Military time? What about daylight savings, where the southern hemisphere goes in the opposite direction when the clocks change? Also consider that local jargon, slang and idioms that are normal to one culture may be difficult for another to figure out.
Cloud testing considerations
When software, infrastructure and processes move to the cloud, there are additional points added to the architecture that require careful and purposeful testing.
Automated backup
All of the public cloud providers make it easy to provision replication and backup for data and services. Whether leveraging basic availability zones within a single region or enabling multiple nodes of geo-redundant backup, it's important to understand and test the way these backups are working. Automated processes are great and should be used, but be sure to regularly revisit and smoke-test the configuration of these backups.
Communication between services
At some point, if not already, your architecture is likely to move toward becoming more "cloud native." It's hard to ignore the value of utilizing some form of orchestrated deployment of containerized microservices, not to mention serverless compute. But remember this:
This means you should strive to thoroughly test the entry and exit points for each individual service, and not only the public-facing client endpoints.
Security (again)
Yes, we are going to talk more about security testing. If you are hosting your data in the cloud, do you know where it lives? How do you get to it? The cloud provides many safeguards against a breach, but you still need to think about security measures.
Your data is valuable to people besides you and your customers. Secure everything. Test your passwords and timeouts. Ensure both client-side and server-side validation. Have plans in place to periodically audit for compliance with GDPR, HIPAA, PCI and PII regulations, to the extent they impact you.
When it comes to credentials, rare is the developer who has not (at least once) inadvertently committed access tokens to a shared code repository. All cloud providers offer secure storage options—e.g., Azure Key Vault, AWS Key Management Service—for production "secrets" that live separately from your codebase. Take advantage of these, and never use the same secrets or credentials for production that you use for development and test.
Don't become complacent
One of the biggest "gotchas" we have seen with clients migrating to the cloud is a relaxation of previously rigorous standards around performance testing. The cloud promises "infinite scalability." Experiencing peak load? Scale out or scale up. Exponential growth of your data? Increase the size of your storage accounts.
The major cloud providers have more hardware than you can hope to use. But that does NOT mean you can, or should, disregard scalability or performance in your testing strategy.
Load and performance testing
Yes, if you run into performance issues at scale, you can just "throw more machines" at the problem. But should you? That costs money—sometimes a lot of money—and you want to be efficient with your resources. Consider using an application monitoring tool like AppDynamics to identify and trace performance issues when they occur.
Keep history on your performance tests to maintain an ongoing health metric. If your system is not meeting targets or sees degraded performance, determine why it is slowing down. Don't just assume you need more machines.
Resiliency testing
Is your system truly fault-tolerant? Even when offering extremely high availability, cloud providers can have outages or be attacked by malicious parties. If part of your system is inaccessible, how do you handle that? How do you recover when that part comes back online?
Always think about maximum load and data flow through an application, as you want the flow to be within the tolerances you expect. Test for what happens outside those tolerances. Always fail gracefully, without confusing the user or making them think it is their fault.
Monitor your costs
While not strictly a traditional measure of "quality," your cloud operating expenses deserve attention. Even if your application is meeting performance goals, you should use a tool like CloudCheckr and place metrics around your usage of cloud services.
The cloud is often cost-efficient, but it is certainly not free. Compute time costs money. Storage costs money. Data egress costs money. Always be looking for ways to improve the efficiency of your applications.
Constructing and executing a plan
It is clear that your cloud enablement strategy should incorporate deliberate quality and testing plans, just as with any other software development and delivery project. Wherever you are along your cloud migration journey, WWT is here to help.
The consultants in our Multicloud Practice can help ensure you take the appropriate steps to follow a successful path. Dedicated quality advocates from Application Services can guide you in best testing practices, or can even embed with your delivery teams. Working side-by-side as your trusted partner, you can be confident that your cloud-based solutions will help you meet your business objectives both today and tomorrow.
To have a conversation with us, contact your WWT account manager directly or reach out to us here.