WWT's New AI Blueprint for Webpage Quality Assurance

Improving automation with human expertise

Maintaining the quality of web applications is crucial in the ever-evolving software development landscape. Quality assurance (QA) has long been a cornerstone of delivering reliable and user-friendly software. With the advent of artificial intelligence (AI), the possibilities for enhancing QA processes have expanded tremendously. WWT has developed an AI-based quality assurance automation testing blueprint designed to analyze and automate web page testing.  

We are currently using the blueprint with RFP Assistant, an AI-powered tool developed internally by WWT that automates and streamlines the human-led process of responding to requests for proposals (RFPs).

The AI-driven QA process

By leveraging large language models (LLMs) and Browser Use — a tool created to allow LLMs to operate in the browser — the QA blueprint can be used by a quality assurance tester to evaluate and create automated tests that run using Playwright. The process begins with a description of the change or feature that needs to be tested. This description is fed into an LLM, which breaks the problem down into specific test cases. These test cases are then handed over to Browser Use, which meticulously goes through the web page to identify the steps required to execute each test. The final step involves feeding these steps into another agent that generates a Playwright test, a robust and repeatable test script.

AI-powered automation tests

The AI component of this tool excels at writing solid, basic automation tests. These tests are essential for verifying core functionalities and ensuring the application behaves as expected under normal conditions. AI's ability to quickly generate these tests means that development teams can focus on more complex aspects of their projects without compromising on initial quality checks.

The blueprint can even generate tests from incomplete information, such as a Jira card, by making intelligent assumptions about the necessary test cases. However, advanced testing scenarios, such as negative test cases and complex user flows, still often require human intervention. These tests necessitate a deeper understanding of the application's nuances and user behavior, which AI alone cannot fully capture. Therefore, human expertise is indispensable in creating and refining these advanced test cases.

Human-in-the-loop and continuous improvement

The complicated nature of deriving, executing and converting these tests means that errors or bad assumptions can propagate into the final output, necessitating the ability to step in and fix outputs every step of the way. We have designed the quality assurance automation testing blueprint with this in mind. At every step, there's the ability for the human-in-the-loop to have visibility and transparency into what the AI is doing. This approach allows humans to modify, edit and correct the generated tests as needed, resulting in more accurate and reliable test scripts.

Even the finished Playwright tests might still require minor modifications. These adjustments often relate to validating that assertions pass for the right reason or fine-tuning specific test scenarios to match real-world conditions. By allowing for human oversight and intervention, the tool empowers QA teams to maintain high standards of quality and adaptability. However, these changes can be made in minutes rather than the hours previously required to write such tests.

We believe that the AI-based QA tool is a powerful asset for the QA force. We focus on continually improving this process to maximize its efficiency and effectiveness. By combining the strengths of AI and human expertise, we are creating a dynamic and resilient QA environment that adapts to the ever-changing demands of software development.

Expanding possibilities

The integration of AI in QA is a testament to the many possibilities that technology can offer. As we continue to refine and enhance the QA testing automation blueprint, we are excited about the future of QA automation. This tool is a step forward in approaching quality assurance, making it more efficient, accurate and adaptable. With minor tweaks, the tool can be used to write automation tests in a variety of frameworks, not just Playwright. BDD styles, like Cucumber, can be included. This could even use already-defined behavior-driven development (BDD) steps to help integrate these tests into an existing ecosystem.

The AI-based QA testing automation blueprint, powered by Browser Use, represents a significant advance in QA automation. It exemplifies the synergy between AI capabilities and human expertise, helping to thoroughly test web applications and maintain high quality. 

As we move forward, WWT remains dedicated to harnessing AI's full potential while valuing the contributions of human professionals. Together, we are shaping the future of QA, one test at a time.