Considerations to Develop Safe and Secure GenAI Solutions

The goal of WWT's software and automation engineering AI research team is to enable our workforce to leverage generative AI (GenAI) to produce more value for our clients.

We started with a curiosity-driven approach. We don't know what we don't know, so let's figure it out. When people talk about the benefits of GenAI, they often talk about how it can help people work faster (e.g., developers coding faster or designers building presentations faster). 

As our AI research team experimented with different models and applications, we learned that GenAI could do more than just accelerate production processes. It can also:

• Open previously unconsidered workstreams
• Foster creativity among all roles within our workforce
• Build innovations on top of existing GenAI applications

Again, our goal is to enable our workforce to leverage GenAI to produce more value for our clients. We emphasize creating more business value because we want to go beyond the low-hanging fruit of efficiency and quality to focus on creating novel, impactful and even transformative GenAI solutions.

Our framework: Key considerations to develop safe and secure generative AI solutions

As we worked toward our goal, we encountered various challenges and risks associated with generative AI development. We needed a common language and a systematic process to ensure we were building safe, secure and ethical solutions. This led to the creation of an AI development framework that includes the following considerations:

• Security: Ensures security by understanding intellectual property risks and training engineers to handle creative flaws. Considers ownership of data, including who can see the prompts used and what happens if the technology is prompted with unintended or unanticipated inputs.
• Ethics: Follows the tenets of "Responsible AI" by developing systems in an ethical manner to maintain the trust of partners, clients, employees and other stakeholders who may be troubled by the negative implications of AI.
• Accountability: Holds teams accountable by implementing auditable solutions to track prompts and data usage. This can help prepare for unknown legal or regulatory grounds ahead.
• Innovation: Conducts rigorous experimentation by fostering a culture of safe exploration that adheres to our shared AI development framework. Continuously innovates to gain a competitive advantage.

The importance of human intervention

It is critical to understand the potential risks associated with GenAI before introducing it to your organization as a development tool. You should strive to train your teams to safely use generative AI without putting your organization or clients at risk. 

Protecting intellectual property

Without the necessary security settings and policies, an organization's use of AI can unintentionally expose proprietary and confidential information, including employee and client data. 

Consider this scenario: A software engineer is tasked with fixing a bug in a set of code and decides to use a GenAI tool to help. The tool finds the bug but exposes a client name and a proprietary table called "discount rates" in the process — all unbeknownst to the software engineer.

In another scenario, we can imagine a sales individual or product owner who wants to pitch a new feature to a client using GenAI to help build the presentation. While the AI assistant provides valuable support in creating an impressive deck in a short amount of time, the presentation unfortunately includes the client's name and protected details about their intellectual property. 

If a public instance of generative AI was used in the scenarios above, the private information that was inadvertently shared would be available to others using the same tool in future instances.

Verifying all output

Creative flaws and hallucinations are two common risks associated with GenAI. Creative flaws can take the form of perfectly valid code that is objectively bad. For example, a piece of code generated by GenAI may contain an SQL injection bug that gives an attacker unauthorized access to data or worse.

Hallucinations, on the other hand, can happen when the AI model is not trained on enough data or when the training data is not representative of the real world. 

For example, an attorney used GenAI to write a legal brief, only to discover after submitting it to the judge that every case referenced in the brief was completely made up. Generative AI did a great job of creating a reasonable-sounding argument, but it was also completely false. This incident was widely reported in the media, and you can imagine how embarrassing and damaging it would be for a company to end up in a similar situation.

Whatever the purpose of the output, it is incumbent upon the user to ensure that the information generated by an AI is accurate, unbiased and presented in the most effective manner. 

Maintaining value

Merely using the output generated by AI without incorporating new information will result in diminishing returns. The AI will continue to restructure the same information without adding any additional value. 

Other risks

There are other risks associated with using GenAI, including important ethical implications. It is critical to have transparency around how our models are trained, what data they are trained on, and what biases might be present in the training data. For example, if a model is trained on historical data from the internet, it may contain biases that could expose an organization to legal risks not yet fully understood. Copyright and intellectual property rights are another area where the risk landscape remains in fluctuation.

Conclusion

Generative AI is a game-changing technology that can revolutionize the field of software and automation engineering. The use of GenAI is expected to grow rapidly in the coming years as new applications of this technology are explored. As usage increases, GenAI's value to an organization should also continue to grow. However, it is important to note that this technology cannot replace human knowledge and expertise. Human intervention and oversight are essential to ensuring that the results produced by generative AI models are accurate, appropriate and transparent.