Â鶹´«Ã½

An AI agent that was right in March can be worse in September and still look like it is working. The operations manager has been correcting its recommendation every Monday since the spring. The same suggestion comes back, gets fixed by hand, and nobody mentions it. The run logs look normal. The bill looks normal. No alarm goes off, because nothing in the system is broken. The agent is just worse at the job than it was.

Catching that does not take a monitoring platform. It takes a few things a contractor can check in fifteen minutes a week. The first is override rate: how often people accept the agent’s recommendation, and how often they quietly do the job themselves. The agents in Â鶹´«Ã½ AI propose and wait for a planner to confirm. Team Builder proposes a team and Assignment Finder proposes a role for someone coming free. That confirmation step produces the override number every week, at firms that already run it. Almost nobody reads it as a signal.

Why a staffing call is harder to check than a submittal

Construction supervises output by measuring things it can point at. A wall gets inspected. A quantity gets checked against the takeoff. A submittal gets reviewed. When the work is wrong, someone finds out fast: route a submittal to the wrong trade package and the person who opens it sees the mistake and sends it back. Document agents fit that habit.

A staffing recommendation is different. It picks one of four qualified names for a March start, and a slightly worse choice looks like a slightly different choice. There is no defect log for it. Nobody raises it. The planner changes the name by hand and gets on with the week. The system does not know it made a mistake, so the quiet fix does not improve anything.

Where drift enters the workforce planning week

Agents fail in two places, and only one of them has an alarm.

The first failure happens before launch, when nobody wrote down what a good output looks like. Those agents were never going to work, and firms find out within a few weeks. Most of what is written about why AI initiatives stall before they ship is about this failure.

The second failure happens to an agent that passed every check. Here is how it goes. The agent proposes a staffing plan for the weekly look-ahead, built from current project data and from its record of past runs. The planner confirms the recommendation or overrides it before the week is staffed. Reviewing each recommendation before you act on it is a control on one output at one moment. It stops a wrong answer from becoming a staffing decision. It says nothing about the trend. A firm can review every recommendation for six months, catch every bad one, and never notice that the share of bad ones has doubled.

An override also leaves no trace that the agent can read. The planner fixes the name, the week moves on, and the agent’s record shows no sign that anyone disagreed.

Drift is the word for what happens next. Nothing breaks. The agent still runs on schedule, still returns a recommendation, still produces output that looks the same as it did in the spring. It is getting worse at the judgment. Â鶹´«Ã½’s names the other way in. Stale or inconsistent records get amplified when an agent acts on them, so an agent recommending a team from outdated availability data is working from a false picture, and nothing in its logs says so.

The weekly look-ahead is where drift shows first: more corrections, and people reading the recommendation less carefully. It is also the only place it shows.

Four practices for monitoring an AI agent after go-live

Start with override rate

Your review step already creates the data you need. Every time someone confirms or overrides a recommendation, that is a data point. Over time the data points form a pattern that tells you how well the agent is doing. Three months of confirmations is enough to see the direction.

Firms with no instrumentation at all do not get an alert when quality drops. They get a complaint, a billing oddity, or people quietly ignoring what the tool says. The people closest to the work lose confidence before any system reports a problem. A rising override rate is that loss of confidence, written down.

Read cost per run as an early tell

An agent checks its own work against its instructions and tries again when the work fails. So a struggling agent costs more than a healthy one doing the same job. Cost per run moves before the output looks obviously wrong.

One expensive month proves nothing on its own. Runs of the same agent on the same task can , and the study that measured it also found that spending more did not buy a better answer. There are innocent explanations too: more people are using it, the jobs got harder, somebody widened the data it reads. A cost line tells you when to go and look. It does not tell you what you will find.

Record a baseline at go-live and reread it weekly

Three things in the first month give you a baseline, and together they take an afternoon. Note what one run costs. Note how often people accept the recommendations while the agent is new and everyone is paying attention. Save eight or ten real staffing cases with the answers the agent gave, in a folder somebody will be able to find.

Then reread it. A baseline in a folder nobody opens is a baseline nobody has. Fifteen minutes on a Monday against those three numbers is the whole practice.

There is no correct override rate and no correct cost per run. The only thing that matters is the change from what this agent used to do, on this work, with this team. A firm that skips the baseline ends up arguing about impressions. Somebody thinks the recommendations got worse. Somebody else thinks the jobs got more complicated. No record settles it. Reconstructing six months of staffing decisions to find the week the quality slipped is not an afternoon’s work, and by then nobody can say what the agent used to do.

Start with one agent and one reviewer

One agent with a named reviewer beats five agents nobody owns. The reviewer’s job is the weekly read, and it gets easier the fewer agents it covers.

Hugh Seaton works with construction firms on AI adoption. He described the job on a : “[S]omeone has to keep an eye on how the agents are performing, and that’s a dashboard. It’s not technical, but nevertheless it has become more of a thing than it was before … almost like HR for agents.”

That job belongs with operations, not IT. Only someone who knows a good staffing call from a passable one can see the decline, and that judgment sits with operations and workforce directors, not with whoever administers the software. The work is closer to supervising a new hire than running a system. Check the early work closely, notice the month the quality slips, and have the conversation.

What to watch weekly and what to leave alone

The left column takes about fifteen minutes a week. The right column is real work for somebody with an engineering function, and a general contractor does not need it to catch an agent that has been getting worse since March.

Put a name against it this month

If an agent has been running since the spring and nobody has written down its override rate, start this month. Note the rate, note the cost per run, save ten cases, and give the weekly read to one person in operations. If none of that exists, the honest position is that nobody knows whether the agent is better or worse than it was, and nobody will until the numbers do.

The confirmation step is the instrument. Â鶹´«Ã½’s agents propose and a planner confirms before anything is staffed, so the override number already exists every week. Reading it is the whole practice.