Auckland, New Zealand Applied AI  /  Evaluation  /  Governance

Sai Kiran Kurmilla  ·  Applied AI, evaluation and governance

Confidence is not accuracy.

I build AI systems that have to prove they are right, and I publish the evidence either way.

I design, build and operate KiwiStart, a production AI service that answers immigration, tax and settlement questions for people arriving in New Zealand. It has been live since May 2026 and I own it end to end: retrieval, model routing, guardrails, evaluation, cost control and incident response.

The domain sets the engineering standard. When someone is deciding whether they can lawfully accept work next week, a plausible answer is a liability. So the system is built to abstain rather than improvise, to carry verifiable provenance on every claim, and to make its own degradation visible. That discipline is the work, and it transfers to any organisation putting AI in front of people who depend on it.

BSc Computer Science  ·  Microsoft Certified: AI Transformation Leader (AB‑731), September 2026  ·  AWS Certified AI Practitioner, October 2026

In production

KiwiStart

A retrieval-grounded assistant for migrants and international students in New Zealand, covering visas, IRD and tax, tenancy, banking, work rights and KiwiSaver.

Operating profile

Single-operator system running on managed infrastructure with daily cost ceilings. Answer quality, model routing and failure modes are instrumented, so degradation surfaces as a counter rather than as a complaint.

May 2026Live since
~3,000Questions answered
93.3%Judged retrieval recall
0False eligibility claims
Retrieval

Hybrid dense and lexical search over a curated New Zealand corpus, fused by reciprocal rank so a query never returns an empty context window. Measured at 93.3% judged recall against a thirty-question gold set.

Abstention

A blocking output gate on eligibility determinations and a premise check that rejects false assumptions before an answer is constructed on them. Unknown facts are elicited, not inferred.

Provenance

Citations are resolved server-side and never pass through the model, which removes fabricated URLs as a failure class rather than reducing their frequency.

Safety

Personal information is redacted ahead of inference. Prompt-injection and scope controls are enforced in application code, so compliance does not depend on the model choosing to comply.

Reliability

Multi-provider routing with tiered fallback and quality-signal escalation, semantic response caching, sliding-window rate limits and per-day spend breakers on anonymous and authenticated traffic.

Observability

Per-day counters on classifier fallbacks, model tier, cache hit path and stream stalls. When the classifier timed out during the August evaluation run, the fallback was recorded rather than absorbed.

Evaluation

Measured, and openly unmeasured

Each figure comes from an evaluation harness written for this system and run against it. Rows carry the method used, because a number produced by keyword overlap and a number produced by a model judge are not comparable and should not be shown as though they were. Current as at 16 August 2026.

Metric
Result
Status
Retrieval recall, judged Whether the retrieved top‑5 actually answers the question, scored by a separate model as judge rather than by keyword overlap. Thirty-question gold set, run 16 August 2026.
93.3% 28/30
Measured
Empty retrievals Queries returning no usable context at all. Hybrid dense and lexical retrieval with reciprocal rank fusion, against a dense-only baseline on the same set.
0/30 dense 1/30
Measured
Eligibility verdict accuracy Whether the premise check reaches the correct verdict across twenty visa-eligibility cases, including dual nationality and false-premise inputs.
20/20 100%
Measured
False eligibility claims Asserting that someone qualifies for a visa without the facts to support it. Treated as a defect class rather than an acceptable error rate.
0 all runs
Measured
Nationality extraction Recovered from unstructured user text across fourteen cases. Where extraction fails the system elicits rather than infers, which is why the gap does not become a wrong verdict. Deflections: 0.
11/14 78.6%
Measured
Citation coverage, current build Whether every answer carries a resolvable link to an official source. Requires driving the live endpoint, which the service’s own abuse controls currently refuse.
not established
Open

On the open row. A stored result set labelled “after” turned out to have been captured thirty-two minutes before the change it was meant to measure. The figure was accurate and described a system that no longer existed, so it was retired rather than published. The re-run then failed for a better reason: the harness drives the production endpoint on the anonymous path, and the free-usage cap refused all twenty-five requests. A measurement blocked by a control that is working as designed is still a measurement I do not have. The row stays open until the harness runs under an authenticated path.

Alignment

Claude’s Constitution as an acceptance test

Anthropic published Claude’s Constitution on 22 January 2026 under a Creative Commons public domain licence. It states an explicit priority order. Read as a specification rather than a statement of values, it produces testable requirements.

Stated priority
Implemented control
1Safety and human oversight
Blocking output gate on eligibility determinations. Where the supporting facts cannot be established, no determination is returned. 0 false eligibility claims across every recorded run.
2Ethical conduct
Personal information redacted before inference. Injection and scope controls prevent the assistant being argued out of its operating boundary.
3Adherence to guidelines
Scope enforced in application code rather than requested in a prompt, so compliance does not depend on model behaviour.
4Helpfulness
Ranked last by design. Where helpfulness and safety conflict, the system elicits the missing fact instead of returning a confident answer it cannot support.
Built

Claims triage

A decision workflow for retailer deduction claims — the shortage, damage and late‑delivery notices an FMCG supplier receives from grocery chains and adjudicates by hand, against a deadline, one email at a time. Built on n8n, with a language model doing the reading and application code doing the deciding. The dataset is synthetic throughout; no employer data was used at any point.

Design position

The model extracts. The code decides. No arithmetic, no threshold and no routing call is left to the language model. The validation step recomputes the claim value from the invoice record and compares it against what was claimed, because a model that is confident about a dollar figure is still only confident.

7Deterministic gates
3Routing outcomes
6Defects found by running it
0Emails sent without a person
Extraction

Classification into four claim types plus an explicit not‑a‑claim category, then extraction into a fixed schema. The schema is defined by attribute description rather than generated from a JSON example, because example‑generated schemas mark every field mandatory, and a mandatory invoice number forces a model to invent one instead of reporting its absence.

Verifiability

A claim that could not be checked is not a claim that failed a check. The verifiability gate runs ahead of any adverse decision, so missing data routes to a person rather than to a dispute letter. This was not the original design; it is the fix for the first defect found.

Arithmetic

Claimed value is recomputed as quantity multiplied by the contracted unit price and compared within a two percent tolerance. Quantity is checked against the delivery shortfall on record, so a claim for more units than were ever short is caught by subtraction rather than by judgement.

Escalation

Claims above the auto‑clear threshold reach a person by email carrying the evidence rather than the request: amount claimed against amount independently calculated, the variance, units claimed against known shortfall, and which of the seven checks failed. A malformed approval response resolves to declined, never to approved.

Restraint

Where the delivery record contradicts the claim, the workflow composes the dispute letter and writes it to drafts. It does not send. The machine assembles the evidence and the wording; a person decides whether a customer hears it.

Audit

Twenty‑five columns per claim: every extracted field, each named check and its result, the route taken, the reason it was taken, and where a person was involved, their decision and the timestamp of it. A decision that cannot be reconstructed six months later has not really been made.

Six defects were found during the build, every one by executing the workflow rather than reviewing it. The most serious treated an unverifiable claim as an invalid one, and would have drafted disputes against customers from data the system never had. A second produced a confident, well‑formed, entirely empty result when a node silently discarded its input — the failure mode that most resembles success. A thirty‑case test set was designed across the four routing outcomes and the failure modes that matter: missing invoice numbers, amounts written in prose, claims exceeding the known shortfall, duplicate references, zero and negative values, remittance advice and autoreplies. Five representative cases were executed end to end — one for each routing outcome, plus a non‑claim and a claim with the amount written in prose rather than as a figure — and all five produced the expected result. The remaining twenty‑five are specified and unrun.

Governance

Operating under voluntary standards

Neither New Zealand nor Australia has enacted an AI act. Existing law still applies in full, and the voluntary standards are the benchmark regulators measure against. These are the instruments I build to.

Aotearoa New Zealand

National AI Strategy

Published July 2026. A deliberately light-touch, agile posture: no dedicated AI act, existing frameworks amended as needed, aligned to the OECD AI Principles.

Aotearoa New Zealand

Privacy Act 2020

The thirteen Information Privacy Principles already govern AI systems. The Privacy Commissioner expects leadership approval, privacy impact assessment, transparency, human review and data minimisation.

Aotearoa New Zealand

Algorithm Charter

Binding on public agencies rather than on private services. Applied here voluntarily, including embedding a Te Ao Māori perspective in how decisions are explained.

Australia

Voluntary AI Safety Standard

Ten guardrails, September 2024, voluntary at present. An Office of AI was announced on 15 July 2026 with legislation expected in early 2027.

International

ISO/IEC 42001 and NIST AI RMF

The management-system and risk frameworks most often named in enterprise requirements. Used as the mapping layer between this system’s controls and an organisation’s assurance model.

Position

Assessment

Voluntary is not optional. A service producing visa-adjacent guidance for people with material exposure should meet the standard ahead of any obligation to.

Writing

In preparation

Three pieces, published on completion.

Claude’s Constitution as an acceptance test

Treating a published values document as a specification and running it against a production system, including the requirements it failed.

Drafting

Making an AI answer verifiable

Retrieval, server-resolved citations, and the evaluation artefact that described a system which no longer existed.

Awaiting re-run

Responsible AI without an AI act

The New Zealand strategy, the Privacy Act, the Algorithm Charter and the Australian guardrails, applied to one production service.

In research