Turning regulatory PDFs into agent-ready packs

TL;DR

• Models are bad at 60-page government PDFs: tables flatten, enumerations get paraphrased and nothing is cited. Do the reading once, deterministically, and hand the agent data instead of documents.
• The pack layers a short context file and an API contract JSON for most questions, JSONL lookups for exact codes, and page-anchored markdown plus CSV tables whose file names are their own citations.
• A README written in the imperative sets the rules: cite everything, take code values from lookups and never from prose, and when documents disagree report both and stop.
• Every source carries a version and a SHA-256 in a manifest, so a spec refresh becomes a diff on derived files instead of a reread.
• Code review of the agent-written harness still caught shallow redaction, credentials implying consent and a fixture writer that could delete its inputs. The pack fixes domain logic, not operational safety.

The fastest way to get a confident wrong answer from a coding agent is to hand it a government PDF. It will read the first forty pages carefully, skim the rest, paraphrase an enumeration it found in a footnote, and cite nothing. The answer will look right. Whether it is right depends on which of three overlapping documents it happened to read, and nobody can tell you which one that was.

I recently built a test harness for the US Consumer Product Safety Commission’s eFiling Product Registry API, the system importers use to file certificates of compliance electronically. Everything about it is public: the API specification, the CSV upload guide, the code workbooks, the HTS flagging list (HTS is the Harmonized Tariff Schedule, the codes every US import is classified under). What was interesting was not the API, which is a fairly ordinary set of JSON endpoints. It was the work required before an agent could build against it reliably. That work generalizes to any regulatory or policy corpus, so this post walks through it.

The source material

The developer resources come as six documents and a spreadsheet:

• the API specification (version 4.0, January 2026);
• the user guide for CSV upload (version 3.0, December 2025);
• the Product Registry user guide;
• guidance on citations, testing and disclaimers;
• the citation and testing-exception code workbook (December 2025);
• the CBP (US Customs and Border Protection) implementation guide for the PGA message set, the customs data format for partner government agencies, for filers going through customs software rather than the API;
• the CPSC guidance and HTS list for eFiling, dated January 2026, which maps tariff codes to product categories likely to need a certificate.

Versions, counts and field values in this post are as of February 2026, when I retrieved the documents. The spec is a living one, so check the current versions before building against it.

Converted to text, the page content is about 38,000 words with 76 distinct tables. The tables are where the hard facts live, and they are exactly what PDF extraction mangles: they split across pages, lose header rows on continuation, and merge cells in ways that turn an enumeration into a paragraph.

The documents also disagree with each other in small ways, which is normal for a living spec. The API’s primaryProductIdType accepts Model # and Serial #; the CSV guide spells the same values Model Number and Serial Number. The CSV allows a lab type of NOL (no lab, used with testing exclusion codes); the API’s lab type enum is ITL or LAB. The spec uses an x-api-secret header globally but labels it Secret in one endpoint section. manufactureDate is MM/CCYY while the other dates are MM/DD/CCYY. A human integrator catches these by reading carefully. A model reading a 60-page PDF averages them.

“Just upload the PDF” fails for three reasons. Tables get flattened. Enumerations get paraphrased, so you get “Model Number” in an API payload. And there is no provenance: when the answer is wrong you cannot tell which document, which version or which page it came from.

The pack

The fix is to do the reading once, deterministically, and give the agent data instead of documents. Here is the layout I settled on:

cpsc_efiling_agent_pack/
├── README_FOR_AGENTS.md            rules of engagement + jq recipes; read first
├── cpsc_efiling_agent_context.md   ~1,800-word summary of flows, auth, gotchas
├── cpsc_efiling_api_contract.json  11 endpoints, headers, enums, date formats,
│                                   required fields, negative-test catalog
├── cpsc_efiling_lookup_codes.jsonl 183 rows: 130 citation codes, 53 testing exclusions
├── cpsc_efiling_hts_flags.jsonl    23 product categories, 583 ten-digit HTS codes
├── cpsc_efiling_corpus.jsonl       421 chunks, each with doc_id / page / kind / source_url
├── page_markdown/                  one file per source document, "## Page N" anchors
├── tables/                         every table as CSV: <doc_id>_p<page>_t<n>.csv
└── source_manifest.json            URL, version, date and checksum per source

The layering is the point. An agent should be able to answer most questions from the compact context and the contract, which together fit comfortably in a prompt. Exact code values come from the lookup files, which are small enough to filter with jq (a command-line JSON filter) and never need to be loaded whole. The corpus and the page markdown are for the long tail, and they are addressed by doc_id and page, never by “search the PDFs”.

Table file names carry their own citation. api_spec_v4_p021_t01.csv is the first table on page 21 of the API spec, and an agent that quotes it has told you exactly where to check.

Here is a worked example. I asked an agent with only the PDFs whether a product record submitted through the API could use lab type NOL. It said yes, citing “the eFiling guide”, and generated a payload that the validator would reject. With the pack, the same model ran a one-line filter on the contract, found labType_api: ["ITL", "LAB"] next to labType_csv: ["ITL", "LAB", "NOL"], and answered that NOL is a CSV-only value, citing bulk_csv_upload_v3_p007_t01 for the CSV side. Same model, same question. The difference was simply visible now.

Rules of engagement, written for the model

The README is not documentation for people. It is a short contract with the agent, and I write it in the imperative:

# Rules for agents using this pack

1. Read cpsc_efiling_agent_context.md first. Use the API contract for endpoints,
   enums, date formats and validation rules.
2. Every factual claim cites doc_id, page, and table file if one applies.
   No citation, no claim.
3. Code values (citation codes, testing exclusion codes, enums) come from the
   JSONL lookups or the contract. Never from prose, never from memory.
4. API and CSV enums differ. State which interface you are talking about.
5. HTS flags are guidance about which products may need a certificate.
   They are never a compliance determination. Say so when you use them.
6. Never generate, request or store real certificate data. Fixtures are
   synthetic and deterministic.
7. If the documents disagree, report both values with citations and stop.
   Do not pick one.

Rule 7 is the one I would keep if I could keep only one. A model’s instinct is to resolve ambiguity, and in regulatory work resolving it silently is the failure. I would much rather get “the spec’s general section says x-api-secret and the import status section says Secret; verify in the developer portal” than a confident guess.

Provenance and checksums

Every source has a manifest entry recording the URL including the object version ID that the publisher’s storage exposes, the stated version and date, and a SHA-256 of the downloaded file. Every derived file records which sources it was built from.

{
  "id": "hts_flagging_list_jan_2026",
  "title": "CPSC Guidance and HTS List for Electronic Filing of Certificates of Compliance",
  "type": "pdf",
  "url": "https://www.cpsc.gov/s3fs-public/CPSC-Guidance-and-HTS-List-for-Filing-of-Electronic-Certificates-6B-Cleared.pdf?VersionId=...",
  "version": "January 2026",
  "retrieved_at": "2026-02-09",
  "sha256": "…",
  "derived": ["cpsc_efiling_hts_flags.jsonl", "page_markdown/hts_flagging_list_jan_2026.md"]
}

This buys two things. When the agency publishes a new version, a scheduled job re-downloads, compares hashes, and opens a diff on exactly the derived files that changed, so I review a 12-line change to an enum instead of rereading a PDF. And when someone asks “where did this rule come from?” six months later, the answer is a version, a page and a hash, not “the model said so”. This is the same discipline I apply to certified data models; I just had not applied it to documents before.

From pack to test harness

With the pack in place, the harness was mostly mechanical. Typed request and response models (Pydantic v2) are generated from the contract. Each validator carries a reference to the page or table it implements, so a failing test points at the source. A FastAPI mock server implements the eleven endpoints with the documented error shapes. A fixture generator, seeded for determinism, produces 50 valid and 30 invalid synthetic product records. Each invalid one is tagged with the rule it violates: bad date format, unknown citation code, productUpdate=Y without a version to update, doCertify=true without a certifier role, and so on through fifteen categories taken from the contract’s negative-test catalog.

Live calls are where a test harness can do real damage, so they get three layers of protection. They only run against a CPSC Software Development Account, never a business account. They are skipped unless credentials and an explicit allow flag are present. And doCertify defaults to false, with certification flows requiring a second explicit setting and a verified certifier role.

import os
import re

REQUIRED_LIVE_ENV = ("CPSC_EFILING_BASE_URL", "CPSC_EFILING_JWT", "CPSC_EFILING_SECRET")
SENSITIVE_KEY = re.compile(r"jwt|secret|token|authorization|certifier_?id|collection_?id|account_?id", re.I)


def live_smoke_allowed(env=os.environ) -> bool:
    """Credentials alone are not consent. The allow flag must be set explicitly."""
    allow = env.get("CPSC_EFILING_ALLOW_LIVE_SMOKE", "").strip().lower() in {"1", "true", "yes"}
    return allow and all(env.get(name) for name in REQUIRED_LIVE_ENV)


def redact(obj, secret_values: frozenset[str] = frozenset()):
    """Redact sensitive keys and known secret values at any depth before anything is written."""
    if isinstance(obj, dict):
        return {
            k: "[REDACTED]" if SENSITIVE_KEY.search(str(k)) else redact(v, secret_values)
            for k, v in obj.items()
        }
    if isinstance(obj, (list, tuple)):
        return [redact(v, secret_values) for v in obj]
    if isinstance(obj, str):
        for s in secret_values:
            if s:
                obj = obj.replace(s, "[REDACTED]")
        return obj
    return obj

Every report goes through redact with the session’s actual JWT and secret passed as secret_values, so a token echoed inside an error message is caught even when it is not under a sensitive key.

What went wrong

A coding agent wrote most of the harness code, and code review of its first version caught three problems, and all three are the kind I now look for by default.

Redaction was too shallow. The first version redacted top-level keys only. Live-style responses nest certifier and collection IDs two or three levels down, inside lists. The test that caught it used a realistic nested response, not a flat dictionary. Write the redaction test against the ugliest payload you can find.

Credentials implied consent. Live smoke tests originally ran whenever the environment variables were present. On a developer laptop with credentials in a shell profile, pytest would have hit the real service. The allow flag exists because of this.

The fixture writer could delete the source pack. Generated fixtures were written by clearing the target directory first, and the default target was the fixture root, which also held the normalized lookups. One run with the wrong argument would have wiped the inputs. Generated output now goes to its own subdirectory and the writer refuses to clear anything that contains a manifest.

None of these were exotic. They were the ordinary mistakes of code written quickly, which is what agents produce. The pack made the domain logic reliable. It did nothing for the operational safety, which still needed a reviewer who had been burned before.

The recipe, generalized

Strip away the product-safety specifics and the recipe is short. Extract once, deterministically, and keep the page anchors. Turn every table into a file whose name is its citation. Normalize every enumeration and code list into a lookup that is filtered, not read. Record provenance with versions and hashes, and diff on refresh. Write a README for the model that tells it where to look, what to cite, and when to stop.

I have since used the same shape for accounting standards excerpts and an internal policy wiki, and the result is the same each time. The model’s reading comprehension was never the bottleneck. The bottleneck was that we kept handing it documents designed for a person with a highlighter and a free afternoon. If you also want to know how often the agent gets things right, the scoring approach in evals for analytics agents carries over: the golden questions just cite pages instead of tables.