Does the reranker earn its latency?
A second ranking stage should improve the passages people use. Measure its contribution separately from the cost of retrieving more candidates.
Read articleCobnex blogs / Australia
Cobnex blog
AI implementation, software architecture, cloud operations and Australian technology policy.
512 articles
Page 21 of 29
How much neighbouring context is enough?
Adding nearby text can repair an incomplete passage, but it can also add duplication and conflicting material. Measure the information gained rather than choosing the largest context window available.
Read articleThe review cost of claim-level evidence
Citation checking adds retrieval, evaluation and human review work. Spend that effort where an unsupported claim would change a decision.
Read articleRefresh urgent policies more often than reference material
One refresh schedule rarely suits an entire knowledge base. Spend processing capacity according to how often information changes and what a stale answer would cost.
Read articleBudget adversarial runs before they become expensive loops
Injection testing needs repeatable attempts and clear stopping rules. Bound model calls, tool work and test data so failures remain affordable to investigate.
Read articleCap the whole run, not only the model request
A small per-call token limit does not prevent a long sequence of calls, tool retries and queued work. Budget the complete task and define how it stops.
Read articlePlace checkpoints where the consequence changes
Asking for approval after every small step creates queues and weak attention. Put review around meaningful commitments, with enough detail to understand the combined effect.
Read articlePrice durable recovery against repeated work
Checkpoints add storage and coordination, but restarting a long task can repeat model calls, document processing and operator effort. Compare the whole recovery path.
Read articleSpend expert review on the cases that separate candidates
Human judgement is most valuable where correctness is consequential or automation is uncertain. Use deterministic checks and targeted sampling to make that time count.
Read articleCompare cost after retries and validation failures
A cheaper response is not necessarily a cheaper completed task. Include rejected outputs, repeated calls and human recovery in the comparison.
Read articleCalculate automation savings after exception handling
Extraction charges are only part of document-processing cost. Include review time, false acceptance, duplicate work and the effort needed to maintain supplier formats.
Read articleSelf-hosting transfers an operating workload to your team
Running a model yourself can change control and cost, but it also creates responsibility for capacity, upgrades and recovery. Compare those obligations with the actual task workload.
Read articleInclude the person who resolves the AI exception
Automation cost includes the work handed back to staff. Measure whether an exception arrives with enough evidence to resolve it efficiently.
Read articlePrice the work that leaves the automated path
Abstention can prevent an incorrect answer, but it may create clarification and review work. Compare that effort with the consequence of answering incorrectly.
Read articleCompare one deployable application with several operated services
Service boundaries add independent control and operational work. Compare the cost of the problem being solved with the cost of running the split.
Read articleBudget for backlog recovery as well as ordinary traffic
A publisher sized only for normal arrivals may never catch up after an outage. Estimate spare throughput and protect the database while draining delayed work.
Read articleChoose retention around the actual retry window
Idempotency records consume storage, but deleting them too soon weakens replay protection. Base retention on client behaviour and downstream guarantees.
Read articleInclude the cost of supporting older clients
Compatibility work extends beyond one adapter. Tests, documentation, data mappings and support all continue while an older contract remains active.
Read article