In his recent best-selling book, “Digital Impact: The Human Element of AI-Driven Transformation,” Boomi CEO Steve Lucas captured both the promise and pitfalls of AI agents when he wrote: “With great power comes great responsibility.”
He was quoting Uncle Ben’s famous advice to Peter Parker from a Spider-Man movie. And who doesn’t like a good superhero film reference! It’s a bit of pop-culture wisdom that carries added weight when we consider agents. They’re incredibly powerful and hold the potential to help us achieve great things in our businesses.
But they also require responsibility.
That brings us to perhaps the most essential element I want to discuss in our series about demystifying agents. So far, we defined what agents are, explored their different levels of autonomy, and examined the world of tools. But there’s more to building, embedding, and enabling agents. In our final post, let’s look at how agents need to be managed and governed with care.
The strength of large language models (LLMs) and agentic systems lies in their ability to make connections at scale that humans typically cannot. They make sense and provide structure to vast amounts of unstructured data at near-instantaneous speeds. That’s raw, unprecedented power.
So, yes, there’s responsibility that comes in controlling that. If we deploy agentic systems that use generative AI and let them run amok, either through hallucinations or unbounded execution, it could create chaos by leading to unintended actions. After all, the absence of governance is anarchy.
On the other hand, proper governance fosters trust and confidence in agentic systems, which in turn yields the long-term stability necessary to transform today’s hype into tomorrow’s ROI.
The Governance Checklist
Here are some key technical areas to consider when managing agents.
- Access and Control: Define who can build, deploy, modify, and oversee agents. Determine the types of permissions or approvals that should be allowed and ensure that you understand all of your agent-enabled existing workflows. That will minimize the likelihood of “shadow IT” that involves agents operating throughout the enterprise without the knowledge of your technical teams.
- Guardrails and Policy Enforcement: Set boundaries on what AI agentic systems can and cannot do. Because agentic solutions operate at a speed and scale that boggles the human mind, there needs to be firm guardrails that keep agents within their designated scope of activity without acting as brakes. There should also be triggers that raise red flags whenever an agent violates a boundary. For instance, this can involve content filtering and topic restrictions to prevent chatbot agents from engaging in subjects that are not safe for work or could tarnish reputations. The enforcement guardrails and policies are crucial to establishing trust in the system.
- Auditing and Accountability: Extensive logging of how agents retrieve information, communicate, and access data will create an easily followable audit trail for compliance and forensic analysis. You need the ability to identify the root cause of problems to resolve them whenever an agent is not performing as expected.
- Data Governance and Sovereignty: Controlling what agentic systems can access ensures data quality and lineage, while preventing sensitive data, such as personally identifiable information, from being leaked. You need to keep agents honest with the data they access and handle. Additionally, ensure they work with high-quality, timely data to avoid the “garbage in, garbage out” issue.
- Life Cycle Management: Agents are next-generation software applications. But they should be treated and controlled just like traditional software. If an agent isn’t behaving as it should, there should be a way to roll it back to an earlier version. And when the moment arrives for a more advanced agent to take over, it’s retirement time for the existing one – only without the gold watch.
- Cost Management: You want predictable budgetary outcomes by tracking token costs and expenses, and managing “infinite loops” or “fruitless cascades” by implementing time bounds on queries to minimize unexpected bills. Tokens, by the way, are finite units of cost for AI, such as the amount of words, pixels, or sound bites used. Loops and cascades are when agents “get stuck” on something. Setting a cost baseline and alerts when agents exceed established cost limits will prevent unexpected overruns.
- Active Monitoring: Policies to manage agents are not “set it and forget it.” A longitudinal approach to governance is required. Boundaries must be continually maintained, evaluated, and updated as needed.
But there is one other component to agent governance that we haven’t discussed. It’s the most important: people.
Humans in the Loop
Humans are the primary safeguard in keeping agents on the straight and narrow. They must be involved in every step of the agent lifecycle process, from their building to deployment to ongoing management. People need to decide how much they should trust agents within a given process to operate autonomously.
They need to decide when (and when not) it’s appropriate to use agents in a process, continually check their work, and create alerts that sound when something goes wrong. The bottom line is humans will feel compelled to audit everything they do to ensure accuracy, safety, and to keep agents “honest.” Humans OUGHT to be in the loop until we trust the system enough to cede autonomy to agents.
The world of mainstream generative AI burst onto the scene just over three years ago with OpenAI’s release of ChatGPT. The introduction of agentic systems that can operate independently is even more recent. It makes perfect sense for us to maintain a degree of skepticism about something that we don’t fully understand.
There’s tremendous value in having structure around agents. But something else that has value is the old idea of “trust, but verify.” Oversight builds confidence in the system by protecting our businesses, helping us learn from mistakes to ensure we don’t repeat them. Establishing a level of trust in agents provides companies with greater freedom to deploy them in more expanded ways.
AI represents an incredible leap forward for both businesses and society as a whole. Large language models can help us make sense of a messy world. Agents can be invaluable tools that enable us to achieve more than we ever thought possible.
Only when we exercise governance with responsibility can we unlock the full power of agents.