AI Agents and the Principal-Agent Problem

Imagine you are away for the summer. You hire a gardener and give the guidance to clean up the yard while you’re away. When you come home, the rose bush you like is aggressively pruned and the wildflowers you’d been cultivating is now a new patch of grass. The gardener was not acting out of malice. He was implementing what he saw as the best version of “clean up.” The problem was that it was dramatically different from what you had envisioned.

In this hypothetical scenario, the gardener had cleared your prized flora. But imagine the scenario where now he has access to your credit card, calendar, and email, and the outcome might be a bit less innocuous. That’s a world closer to what we are now seeing with AI agents.

While AI agents themselves are new, many of the problems they present are not. As long as people delegate some tasks to others, they will have to contend with what social scientists call the principal-agent problem. We constantly delegate tasks to others. Delegation is nearly indispensable because of time and resource efficiencies. We are not our own dentists because there’s expertise involved in studying the problems associated with oral health, not to mention the equipment and resources needed for more invasive treatments. Sometimes people delegate not because they lack the expertise but because they do not value the time it takes to carry out the task. Some people delegate gardening to a landscaper, others enjoy the activity of planning, cultivating, and harvesting a garden.

But delegation can come with a cost. The principal gives up control to someone who may know more but whose incentives are not always aligned. Who are we to question their recommendation since we lack their know-how? But they have incentives to recommend more treatment than the patient would choose with the same information, because the implication is more revenue for their services.

These principal-agent problems arise in a number of settings. Real estate agents value quick sales, which may mean recommending a lower house sticker price even though the seller wants to maximize profits even if it might mean taking a bit longer to make the sale. A manager tells an employee to conserve costs so the employee shifts to a supplier who takes longer to deliver or who delivers shoddy quality.

The problem, as these examples suggest, can be competing incentives, but it can also be that it is impossible to articulate one’s preferences ex ante. Asking the gardener to clean up the yard embeds countless unstated preferences. It does not explicitly communicate that my now deceased dad bequeathed me those rose hedges and that seeing them removed now removes a piece of him. We delegate tasks to the agent but without specifying the full set of underlying preferences. Agents must fill in countless gaps that human principals and agents learn over many interactions and time.

For AI agents, the problems are analogous. We are delegating tasks and yet often not specifying, nor is it necessarily possible, every circumstance or eventuality. One difference is that technology moves more quickly and with considerably more scale. Someone who gives an agent access to credit cards, calendars, and email also raises the stakes of those incomplete instructions compared to the gardener not appreciating the wildflower cultivation quite the same way I did.

The important question is not necessarily whether AI agents are good or bad, or whether people should or should not use AI agents. It’s not unlike the gardening analogy. Some people may derive value from doing their own gardening when they can. Some people will continue to derive value in making their own dinner or flight reservations. Others consider these tasks worth delegating. In the AI case, the decision to delegate is not a simple yes/no, but how to articulate the reasonable range of individual preferences that we take for granted but could become relevant in these decisions. Limiting discretion is important when the consequences are significant (e.g., cost ceilings), or requiring confirmation for non-refundable plane tickets. Then monitoring the results and refining as you observe where the agent has not learned or you did not clearly communicate preferences.

Over a period of centuries, an array of contracts, rules, and licensing has emerged to help align the relationship between principals and agents. A contractor, for example, provides an estimate for work, you give them authority to carry out the work, but they do not have unfettered authority to carry out repairs that are out of scope of the agreed-upon work. In AI, the agent could search airfares or construct an itinerary, but would be required to obtain confirmation before spending over a specified threshold or cancelling a nonrefundable ticket.

Not everyone is ready for the AI agent era, but those who think they are still need to heed the “caveat emptor” warning. They need to be clear about what they are asking agents to do, put limits on authority, and pay attention to the results. “Clean up the yard” might seem straight-forward, but my garden would say otherwise.

Dr. Sarah Kreps, Director Tech Policy Institute