On this page
Continue from the defined workflow
Hypothetical situation: an enquiry agent prepares a convincing response and the demonstration ends successfully. In the next test, the source document is missing. In another, a tool times out after the receiving system may already have accepted the request. The quality of the first answer tells the team little about those operational states.
The 9 October workflow article establishes the starting point: define the outcome, information boundary and actions needing human review. This continuation asks how to test that design when the normal path breaks. It is a review method, not a claim that a named agent platform automatically provides the controls below.
Before running a case, write the expected action and the evidence that would demonstrate it. Include when the application should stop, ask for information or return work to a person. A correct sentence is useful, but it is only one part of a workflow that may also change records or create obligations.
Include a case where the user abandons the request after a pause. Agree who can resume or cancel the work and how the receiving operator recognises its state. Do not assume that a conversational farewell cancels a pending external operation. The acceptance record should connect the user-visible message to the actual workflow state through the application’s supported controls and operational evidence.
Missing information should remain missing
Remove an input that the workflow needs to proceed, such as the requested date or the source needed to establish eligibility. Check whether the application asks for the missing information, uses an explicitly permitted default or stops. Do not accept an invented value merely because it produces a complete-looking draft.
Make the distinction between a drafting gap and an action-blocking gap explicit. A draft might display an unresolved item for review. An external action may need to remain blocked until a valid value is supplied. The business owner and engineer must agree that distinction rather than leaving it to the wording of a prompt.
Repeat the case with an ambiguous input. If two interpretations lead to different actions, the test should establish how clarification happens. Retain the observed state and next owner. The receiving team needs to understand what happens to the request, not just whether the agent politely acknowledges uncertainty.
A timeout is an unknown outcome to investigate
Use a controlled tool fixture to simulate a failure before work is accepted and a timeout after acceptance. These are different cases. A blind retry may be appropriate for one operation and create duplicate work for another. Inspect the receiving system’s agreed evidence before deciding what to do next.
Test approval separately. Prepare a case awaiting review and confirm that the application cannot perform the protected action merely because the generated message says approval was received. The actual authorisation mechanism belongs in the application and connected service, with the necessary identity checks. A prompt is not an approval boundary.
NIST’s July 2024 profile includes guidance on monitoring performance and handling or recovering from errors. The worksheet below is an original operational exercise consistent with that concern. It does not assert built-in idempotency, automatic recovery or a particular vendor’s retry policy. Those must be established from the implementation under test.
| Case | Expected handling to agree | Evidence to retain |
|---|---|---|
| Required input absent | Clarify or stop before the affected action | State and next owner |
| Tool failed before acceptance | Use the agreed recovery route | Safe tool result and receiving-state check |
| Timeout after possible acceptance | Reconcile before repeating the action | Receiving-system evidence |
| Approval absent | Protected action remains blocked | Actual authorisation check |
| Duplicate request | Apply the designed duplicate policy | Resulting work items and request relationship |
Test repeated work and operational recovery
Submit the same controlled request twice and inspect the actual work created. Then repeat it with a small legitimate change. The team needs a deliberate duplicate policy that does not silently discard an intended update. Record what the user sees and how support identifies the related attempts without exposing secrets or unnecessary personal information.
Ask the receiving operator to resume a paused case using the handover documentation. If only the original engineer knows how to determine its state, the recovery process is not ready for that team. Record unresolved cases as release limitations rather than hiding them behind a successful average answer score.
End the review with a named decision owner and evidence for each consequential action. A pilot may continue within a reduced boundary while an unsafe action remains disabled. Expanding scope requires another acceptance decision. This article’s links were added during retrospective preparation; the historical cutoff describes the capabilities considered, not a claim that this draft was published on that date.
Sources & further reading (1)
Historical evidence checked . Review practices and hypothetical exercises are editorial recommendations, not product guarantees.
