Test Automation for a Major Pharmaceutical Company to Maintain Federal Compliance

Digital signature compliance challenges 

A pharmaceutical company needed to ensure compliance with FDA Part 11 - a set of regulations surrounding the validity and security of digital signatures so that digital records are analogous to their paper counterparts.

World Wide Technology, a technology solution provider, was presented with a unique challenge: to develop automated tests that create and validate data while seamlessly switching between two commercial off-the-shelf (COTS) products, Adobe Sign and Adobe Acrobat Reader.

The pharmaceutical company uses a branded version of Adobe Sign for its electronic document digital signature management. The company encountered an incident where an updated version of Adobe Sign contained a bug that impacted the Part 11 compliance of digital signatures. 

The company approached World Wide Technology due to its functional testing expertise and experience using its Tricentis test automation tool. The goal was to provide automated tests to give the pharmaceutical company advanced notice of potential Part 11 non-compliance. 

Obstacles along the way

Writing tests against commercial software without access to the codebase creates a challenge in writing resilient automated tests. Lack of code access typically increases test development time due to tasks such as reading test tool documentation, community posts, and experimentation.

Additionally, it is difficult to gain insights into how the software could be broken – which would inform the creation of more resilient automated tests. During this engagement, World Wide Technology encountered issues such as: 

• Dynamically changing element IDs made it difficult to write stable tests as they will change value between subsequent visits to the site.
• Tricentis Tosca's bitmap image comparison engine had varying degrees of accuracy when interacting with graphical elements in Adobe Acrobat Reader.

A test automation tool solution 

With over twenty years of test automation expertise in multiple projects across many industries, World Wide Technology was well positioned to help the pharmaceutical company achieve its goals. WWT  experts created an automated suite of tests for adherence to Part 11 compliance that alert the client if compliance is broken. The tests were written using the pharmaceutical company's existing test automation tools: Tricentis Tosca (for script development) and Tricentis qTest (for distributed test execution management and scheduling).

Each test was responsible for creating and signing a PDF using the pharmaceutical company's branded instance of Adobe Sign. The key difference between the tests was how the signed PDF was obtained (download directly after signing, download from the Adobe Sign dashboard, or download via email attachment). 

The first series of validations was performed against the PDF's security metadata after opening it in Adobe Acrobat Reader. This confirmed that the document had not been tampered with. Finally, the test clicked a link within the PDF, opening a web page to validate the document's specific transaction ID. This last step is a key piece of evidence that the document was legitimately from Adobe Sign.

The lack of access to the application code was addressed by adopting a test design pattern to select technical properties that were less likely to change. Where possible, World Wide Technology used the same properties across all automated test steps to ensure consistency and to create a collective understanding that makes collaboration and onboarding easier.

The tests were then linked to Tricentis qTest for test management so that they could be scheduled to run regularly on one of the dedicated remote test execution machines. This continuous testing eliminated the need for manual testing for Part 11 non-compliance. 

Outcomes that reduce non-compliance risks 

The test suite provides the pharmaceutical company with the awareness of any departure from FDA Part 11 compliance – ensuring adherence to regulatory requirements. Reducing the risk of non-compliance helps protect the company's brand and reputation in the industry. 

Additionally, the Tricentis test automation tools help drive efficiency and reduce costs by removing tedious manual workflows. The client can now rely on a set of powerful cross-platform automated tests that free up their personnel to focus on business-critical work.  

What's next 

With the success of this engagement, the next step is to expand and develop new tests to cover other functional areas of the products that the pharmaceutical company uses (such as Jira, Confluence, and Microsoft Azure) and ensure the workflows they follow are validated by test automation.