AI Agent Go-Live Checklist: 15 Questions Before Production

In 30 seconds

What it is Fifteen yes-or-no questions; any honest no means the agent waits.

You use it when You are about to approve, or ask approval for, an AI agent going into production.

Next in this section AI Agent Governance Operating Model: Who Signs Off, and When →

Fifteen plain questions in five groups. If any honest answer is no, the agent waits or runs at a lower autonomy level.

40-second version of this explainer, no sound.

Part 10 of 26 in SAP Business AI, Explained.

In my experience, most agent problems in production trace back to a question that went unasked before go-live. Here are 15 to ask, grouped so the right person answers each, tested on a real-world style onboarding agent.

Before any AI agent touches live data, its owner should answer 15 plain questions in writing, and any honest “no” means it waits.

What the checklist covers

Five groups, three questions each: purpose and owner, data and access, actions and approvals, quality and testing, monitoring and exit.
SAP tooling helps with evidence: Joule Studio runs evaluations before production, and AI Agent Hub records each deployed agent. The questions are still yours.
A failed question doesn’t have to kill the agent. Often you just launch at a lower autonomy level.

Why the list is short

Back to the hospital. Before a new specialist sees patients, the medical board checks qualifications, permitted procedures, supervision and how to suspend them, however capable the doctor seems.

An artificial intelligence (AI) agent is the new doctor, and this checklist is the board’s list. It’s short on purpose, because long checklists don’t get used.

Who fills in the card, and who signs it

The agent owner fills in the card. Each answer is a sentence or two plus a link to evidence, such as a test report or register entry.
IT (information technology) reviews data and access, risk and compliance review actions and approvals, and the business owner reviews purpose and test results.
Every “no” gets fixed, or autonomy drops until it becomes a “yes”, or the agent stops.
The business owner signs go-live, and the card goes on the agent’s register entry.
When autonomy, tools or data change, you run it again.

Purpose and owner

What business outcome is the agent for, and how will we measure it?
Who is the named business owner, and who is the named agent owner?
Which autonomy level does it start at, from L0 Inform to L5 Fully autonomous, and who approved that level?

Data and access

What data does it read, and does any of it include personal or sensitive data?
Whose identity does it act under, and are its permissions the minimum it needs?
Which tools, interfaces and servers can it call, and is that list as short as possible?

Actions and approvals

Which of its actions create or change records in a live system?
Which actions need a human approval, and at what thresholds?
Can a human override or reverse its output, and stop it entirely, and has that been tested?

Quality and testing

Does the test set include edge cases, bad inputs and attempts to mislead it?
What is the acceptance bar, and who signed the test results?
Was it tested away from production, so tests had no live side effects?

Monitoring and exit

Is it registered, with its owner, in the agent register, and verified?
Who watches it, what is measured, and what automatically pauses it?
How do we roll it back or retire it, and who may do that without asking?

This draws on public guidance. The OWASP (Open Worldwide Application Security Project) entry on excessive agency names three root causes: too many functions, too many permissions and too much autonomy. Questions 5 to 8 go straight at those. The NIST (National Institute of Standards and Technology) risk framework expects testing before deployment and during operation, covered by questions 10 to 14.

Three honest noes on the onboarding agent

Agent 2, the new-joiner onboarding agent, is custom-built in Joule Studio, event-triggered and planned at L3 Act with approval. Once a hire is confirmed in SAP SuccessFactors, it creates onboarding tasks, drafts a welcome message, books induction slots and raises laptop and system-access requests. NextSAP Utilities, the fictional Dubai utility used across this site’s SAP content, hires about 45 people a month across the UAE and India, many of them field technicians joining depots across the electricity, water, gas and district cooling divisions, so treat the numbers here as illustrative. Mariam, the human resources (HR) Director, is the business owner and Priya the IT platform owner.

Two weeks before go-live, the HR operations lead, as agent owner, fills in the card:

#	Short question	Answer	Status
1	Outcome measured?	Day-one readiness: laptop and access ready for 95% of joiners, from 70% today	Yes
2	Owners named?	Mariam (business), HR operations lead (agent)	Yes
3	Autonomy approved?	L3, approved by Hana’s governance board	Yes
4	Data read?	Name, start date, role, location, manager. No salary or passport data	Yes
5	Identity and least privilege?	Runs under a shared technical user with wide HR read access	No
6	Tool list minimal?	Four tools, plus a fifth, “update employee record”, that is unused	No
7	Record-changing actions?	Creates tasks and access requests, never edits the employee record	Yes
8	Approvals and thresholds?	Every access request needs the joiner’s manager, and finance-system access also needs Priya’s team	Yes
9	Override and stop tested?	Manager can reject. Stop switch tested in test system	Yes
10	Edge cases tested?	50 past joiner records with personal details masked, including rehires, cancelled offers and a misleading note in a job description	Yes
11	Acceptance bar signed?	48 of 50 correct, the two misses routed to a human; signed by Mariam	Yes
12	Tested away from production?	Sandboxed evaluations, then a test tenant	Yes
13	Registered and verified?	Registered on deployment, verification pending	Pending
14	Monitoring and pause?	Weekly review, no automatic pause defined	No
15	Rollback and retire?	Switch off the trigger. Manual onboarding checklist still exists	Yes

That’s three honest “no” answers. Priya swaps the shared user for a dedicated agent identity that can only read the five fields in question 4, and the unused tool comes out. The board adds a pause rule: if managers reject more than 10% of access requests in a week, the agent drops to L2 Propose. Go-live slips eleven days, which the board accepts as a fair price.

SAP’s toolchain fits this. Joule Studio deployment registers the agent in AI Agent Hub, and SAP Cloud ALM (Application Lifecycle Management) is described as validating that agents are tested before go-live and tracking their actions afterwards.

How hard to push at each autonomy level

The questions don’t weigh the same at every level. Roughly how hard I’d push:

Group	L1 Answer	L2 Propose	L3 Act with approval	L4 Act then report
Purpose and owner	Required	Required	Required	Required
Data and access	Required	Required	Required, with IT sign-off	Required, with IT sign-off
Actions and approvals	Light: it should not act	Check proposals stay proposals	Full review	Full review plus risk sign-off
Quality and testing	Answer accuracy	Proposal accuracy	Action accuracy and edge cases	Action accuracy, edge cases and sampling plan
Monitoring and exit	Monthly	Weekly at first	Weekly plus pause rule	Daily plus pause rule and audit sampling

What a checklist can’t catch

It forces named owners and written answers, catches gaps like broad permissions or a missing stop rule, and leaves evidence auditors and regulators can follow. It also gives you a clean way to say “not yet” without saying “never”. But it has limits:

It doesn’t prove the agent is correct. Testing does that.
It doesn’t replace security design or legal review.
It goes stale, so re-run it when anything material changes.

Explain it to your CFO

Before a digital worker touches live systems, its manager answers fifteen plain questions in writing. A weak answer means it waits or starts with less freedom.
If something goes wrong later, we can show what we checked and who signed.

Settle these before the first card

Who signs go-live? The business owner, with the agent owner presenting the card and the governance lead consulted.
Is “not applicable” allowed? Yes, but only with a written reason the reviewer accepts.
When is the card re-run? On any change of autonomy level, tools, data scope or underlying model, and at least once a year. Any honest no on the card means the agent waits.

Go deeper

Technical detail for architects and developers

Evaluation evidence (questions 10 to 12). SAP’s Joule Studio post describes a native evaluation framework that runs before production and assesses two dimensions: technical correctness and business value. Offline evaluations for custom agents are sandboxed from production, and Joule Studio generates test artefacts, including evaluations, alongside the solution code. Keep your domain edge cases in that suite, not a side spreadsheet.

Deployment and register (question 13). Deployment uses a Git-based pipeline from development to production and registers the agent in Joule Work and in SAP AI Agent Hub. An SAP Community post on agent management describes discovered agents starting unverified until a verification record marks them verified or rejected, and recommends going live only after verification. Check the current status of these features in the Joule capability watch.

Tools and permissions (questions 5 and 6). SAP says its prebuilt agents use MCP (Model Context Protocol) and A2A (Agent2Agent) to reach tools and other agents. OWASP’s mitigations for excessive agency include minimising extensions, applying least privilege, acting in the user’s context, requiring human approval for high-impact actions and checking authorisation in the downstream system rather than in the model.

Override and stop (question 9). For high-risk systems under the European Union (EU) AI Act, Article 14 expects people overseeing the system to be able to disregard, override or reverse its output and to halt it safely, for example with a stop button. Even where the Act doesn’t apply, I’d use it as the standard for any agent at L3 or above.

Monitoring (question 14). SAP describes SAP Cloud ALM as tracking agent actions once live, so operations can catch deviations early. Define the pause rule as a measurable threshold, then test that crossing it really pauses the agent. Most teams skip that last test.

Words you’ll hear

Go-live: the moment an agent starts working on live business data.
Agent owner: the named person answerable day to day for one agent’s behaviour.
Least privilege: giving an agent only the permissions it needs, and nothing more.
Evaluation (eval): a repeatable test that scores an agent’s answers or actions against expected results.
Excessive agency: OWASP’s name for an agent that can do more than it should.
Pause rule: a measured threshold that automatically drops an agent to a lower autonomy level.
SAP Cloud ALM (Application Lifecycle Management): SAP’s tool for running, testing and monitoring cloud solutions.

Keep going

Independent explainer, not affiliated with or endorsed by SAP SE. SAP, Joule and other SAP product names are trademarks of SAP SE.

Frequently asked questions

Does a prebuilt SAP agent need the same checklist as a custom one?

Yes, though some answers come from SAP's documentation rather than your own build. You still own the business purpose, the data it sees in your landscape, the approvals, the monitoring and the exit plan.

Who should run the go-live checklist?

The agent owner prepares the answers and evidence. The business owner signs go-live. Risk and compliance review the data and actions groups, and internal audit can sample the evidence afterwards.

Can an agent go live with one or two open items?

I recommend not. If an item is genuinely not applicable, write down why. If the honest answer is no, fix it or reduce the agent's autonomy until the answer becomes yes.

Is passing SAP's built-in evaluations enough testing?

It is a strong start. SAP describes Joule Studio evaluations for technical and business quality before production. Add your own edge cases, your own acceptance bar and a named person who signs the results.