Tensor AnalyticsTM
AI in operationsCourse moduleIntermediate

Module 7: AI agents and human-in-loop

Kislay S·1 Jul 2026·12 min

Self-assessment answers from Module 6

One. Copy-and-edit fails because the scenario is a snapshot of the baseline at a point in time. When the baseline updates (new actuals, new forecast, changed overrides), the scenario is stale. Rebuilding the scenario against the new baseline takes 20 minutes each time, and after a few weeks the maintenance cost exceeds the insight value, so teams stop doing scenarios.

Two. A sparse overlay is a small set of override instructions ("for SKU X in week Y, use value Z") stored on top of the current baseline, rather than a full copy of the baseline. It solves the maintenance problem because when the baseline changes, the overrides are still valid. They replace the baseline value regardless of what the new baseline value is. The scenario automatically reflects the new baseline everywhere you did not override.

Three. The three consequences are: scenarios become cheap to create and maintain (you can maintain 20 simultaneously, each is a tiny overlay, the baseline updates automatically flow through), scenarios become comparable (all scenarios sit on the same baseline so comparison is apples to apples), and scenarios become promotable (promotion means merging overrides into the baseline, not manually copying numbers back).

Four. The four metrics are supply gap impact (how many SKUs have gaps and total gap magnitude), inventory impact (projected inventory position at end of horizon), revenue impact (projected revenue compared to baseline), and working capital impact (projected inventory value).

Five. The three steps are review (planning team reviews overrides and impact), approval (scenario goes through S&OP gates), and merge (overrides become new baseline values). The merge must be atomic (all or nothing) because a partial merge leaves the plan in an inconsistent state that is hard to diagnose and fix.

If you got all five, continue.

What this module covers

This module covers AI agents in the planning cycle. AI can assist with forecasting, exception triage, supply netting, scenario generation, and board pack creation. But the key design principle is human-in-loop: the agent proposes, the planner disposes. This module covers what that means, why it matters, and how to evaluate whether a tool's human-in-loop claim is genuine.

What AI agents can do in planning

AI agents in a planning context can perform several categories of action. Each category has a different level of risk and a different appropriate level of human oversight.

Forecast generation. The agent runs the statistical or ML models, produces the forecast, and presents it for review. This is the lowest-risk action because the forecast is the baseline, not the published plan. Human review focuses on exceptions and overrides, not on the full forecast.

Exception triage. The agent scans the forecast for anomalies (variance exceeding 2 standard deviations from historical pattern), flags supply gaps (demand-supply gaps exceeding 10% of demand), and ranks exceptions by severity. This is where AI adds the most value, because a planner reviewing 50 ranked exceptions is far more productive than a planner reviewing 500 unranked items.

Supply netting. The agent runs the netting computation, produces planned orders, and surfaces gaps. This is a deterministic computation (demand minus supply), so AI's role is execution and gap detection, not judgment.

Scenario generation. The agent can propose scenarios based on patterns it detects. For example, "Supplier X's lead time has increased 20% in the last 3 months. Model a scenario where it increases another 15%." The agent proposes the scenario. The planner decides whether to run it.

Board pack generation. The agent assembles the cycle performance report, forecast accuracy summary, supply position, and exception summary into a document. This is where AI saves the most time, because report generation is time-consuming and formulaic.

The principle: agent proposes, planner disposes

The design principle for AI in planning is that the agent proposes and the planner disposes. The agent can read any data, run any computation, and propose any action. But the agent cannot execute a write action (publish a plan, place an order, lock a cycle, override a forecast) without human approval.

This principle exists for three reasons.

First, accountability. When a plan goes wrong, someone must be able to explain what happened and why. If the AI executed the action, the explanation is "the AI did it," which is not an acceptable answer to a board or a regulator. If a human approved the action, the explanation includes the human's reasoning, which is auditable.

Second, judgment. The AI is good at pattern recognition and computation. It is bad at context. The AI does not know that Customer X is in financial difficulty and their orders may drop. It does not know that Supplier Y had a quality issue last quarter and their lead times are unreliable. The human planner knows these things, and the override or rejection of an AI proposal is where this context enters the plan.

Third, regulatory compliance. The EU AI Act, the DPDP Act, and CCPA/CPRA all increasingly require human oversight of automated decisions. The "agent proposes, planner disposes" architecture satisfies these requirements by design, because every write action has a human approval event in the audit trail.

What genuine human-in-loop looks like

A genuine human-in-loop system has four properties. A rubber-stamp system fails at least one.

The AI proposes, never executes. Every write action is proposed by the AI and requires human approval before it lands. No write action is taken without an approval event. This is the foundational property.

The proposal includes a rationale. Every proposal comes with an explanation of what inputs were used, what the AI computed, and why it recommends this action. The rationale does not need to be a full model interpretability report. It needs to be enough for the human to form an independent judgment.

The human can reject with a reason. The rejection is not a bare "no." It is a "no, because [reason]." The reason is captured in the audit trail. Over time, the pattern of rejections and reasons becomes a signal for improving the AI's proposal logic.

The human can modify. Between approve and reject, there is modify. The human changes the proposed action, the changed action is what gets executed, and the modification is captured with the original proposal, the modification, and the rationale.

The rubber-stamp test

The test for whether human-in-loop is genuine is the audit trail. Look at the last 100 AI-influenced decisions in the system. Check three things.

Time between proposal and human action. If the median is under 10 seconds, the human is not reviewing. No one reads a forecast, considers alternatives, and decides to approve in 10 seconds. They are clicking a button.

Modification rate. If the human modifies the AI proposal in under 1% of cases, either the AI is perfect (it is not) or the human is not exercising judgment. A genuine review process produces modifications in 5-15% of cases, because the AI is roughly right but not exactly right, and the human applies corrections.

Rejection rate. If the rejection rate is under 0.1%, either the AI is perfect or the human is rubber-stamping. A genuine process rejects 1-5% of proposals, because some proposals are wrong and the human catches them.

If any of these tests fail, the human-in-loop is theatre. The audit trail shows a human was present, but the outcome was determined by the AI.

The audit trail that proves it

A genuine human-in-loop system produces an audit trail that tells a story. For any AI-influenced decision, the trail shows: the AI proposed X at time T1, with rationale R. The human (actor A) reviewed at time T2, modified to X-prime with reason M. The system executed X-prime at time T3.

This trail satisfies three needs. It satisfies accountability (you can trace any decision to a human actor). It satisfies regulatory compliance (the EU AI Act and DPDP Act require this level of traceability). And it satisfies operational improvement (the pattern of modifications and rejections tells you where the AI needs improvement).

Without this trail, you cannot answer the question "why did we plan 1,247 units for SKU-4471?" With the trail, the answer is specific: "The AI proposed 1,247 based on a demand spike in the last 3 weeks. The planner reviewed, confirmed the spike was real, and approved."

What to do next

This module covered AI agents and human-in-loop. Module 8, the final module, covers audit, compliance, and the board pack: how to close the planning cycle with a defensible record of what was decided, why, and by whom.

Before moving to Module 8, do this exercise. If you use any AI-assisted planning tool today, pull the audit trail for the last 50 AI-influenced decisions. Check the three tests: median time to action, modification rate, rejection rate. If you do not use an AI-assisted tool yet, write down the three properties of genuine human-in-loop (AI proposes never executes, proposal includes rationale, human can reject with reason and modify) and use them as evaluation criteria when you next evaluate a tool.

Self-assessment

One. What are the five categories of action AI agents can perform in a planning context?

Two. What is the principle "agent proposes, planner disposes," and why does it exist?

Three. What are the four properties of genuine human-in-loop?

Four. What are the three tests for whether human-in-loop is genuine, and what thresholds indicate rubber-stamping?

Five. What does a genuine audit trail entry for an AI-influenced decision contain?

Answers are in Module 8's introduction.

Learning trackFoundationsAI agentsHuman-in-loopAutomationCourse
Written by Kislay S