Review the answer from yesterday's conversation
A freshness review should begin with information the product has already served. Old conversations expose assumptions that a clean search demo misses.
Read articleCobnex blogs / Australia
Cobnex blog
AI implementation, software architecture, cloud operations and Australian technology policy.
512 articles
Page 24 of 29
Review the controls with the detection prompt removed
A prompt-injection detector is an additional signal. The application's access and execution rules should still hold when that signal is absent or wrong.
Read articleReview the tool schema against the business rule
A typed tool can still expose too much authority. Compare every accepted field and side effect with the operation the user is actually permitted to request.
Read articleCompare the approval screen with the submitted payload
Review the values a person saw against the values the executor used. This catches hidden fields, stale state and post-approval changes that a polished screen can conceal.
Read articleReview what the transcript cannot prove
A conversation can describe a completed action without proving its effect. Review the records and state transitions used to recover when messages and mutations diverge.
Read articleAsk reviewers to inspect refusals and escalations
A candidate can improve correctness by answering less. Review what it declines, what it passes to people and whether those decisions fit the product's intended scope.
Read articleRequire a reproducible release comparison
A reviewer should be able to identify the candidate, rerun its important cases and understand every material difference from the baseline.
Read articleReview one accepted invoice all the way to posting
Trace a record that bypassed manual review from its source page to the downstream payload. This reveals whether acceptance means more than a high extraction score.
Read articleReview the architecture with one real request trace
Use a controlled request to compare the documented data path with the deployed one. Then exercise an error and a fallback to expose routes the main diagram may omit.
Read articleReview the cost of one unsuccessful task
Failed tasks expose missing accounting, uncontrolled retries and weak stopping states. Trace one from its first request through its final resolution.
Read articleReview the next step after the assistant says no
A refusal should have a reason and a useful consequence. Inspect whether the product clarifies, escalates or denies access according to the actual gap.
Read articleReview the exception to the module contract
The unusual path often reveals whether a boundary is real. Inspect administrative operations, imports and reports that reach beyond the public API.
Read articleAsk where the consumer records a duplicate
The outbox makes publication recoverable, which also makes repeated delivery possible. Review duplicate handling at the consumer's actual side-effect boundary.
Read articleReview the same key with a different amount
A duplicate check must distinguish a matching retry from conflicting reuse. Test a valid request whose material value changes while its key stays the same.
Read articleAsk whether the field's meaning changed
A schema diff can miss a breaking change when the type and name stay the same. Review units, defaults and interpretation with an older caller.
Read articleAsk what cannot actually be undone
Review each completed effect against the real correction the business can perform. A method named compensate does not prove the workflow can restore its prior state.
Read articleReview the administrative escape hatch
Privileged tools often bypass the normal tenant context. Inspect their authority, target selection and audit trail as carefully as the customer-facing API.
Read articleMissing and cleared fields are different operations
Optional data needs an explicit interpretation. An omitted field, a null value and an empty string should not accidentally trigger the same update.
Read article