Agent guardrails

Verify before you finish

This page describes an optional agent pattern inspired by harness-style evaluation: do not tell the user (or your own planner) that a download or Bay Run finished until local checks pass. Hugging Bay already verifies bytes inside download_file; these helpers are for agents that need an explicit, testable finish gate and for swarms that retry in parallel.

When to use it

• Downloads. After bytes land on disk, recompute SHA-256 and compare the full digest to the download plan. For .gguf files, also check the first four bytes are the GGUF magic (GGUF) so truncated downloads and LFS pointers fail before you claim success. A matching hash proves the file arrived unchanged, not that it runs on your hardware.
• Runs. After Bay Run returns, require a non-empty chat reply or structured result (classification labels, rerank scores, and similar) before marking the step done. When the API returns a signed receipt, verify the Ed25519 proof with published keys and confirm the write response includes a receipt id before you treat persistence as done.
• Budget errors. On HTTP 429/503 or rate_limited-style envelopes, read Retry-After / retry_after_s and back off. Do not mark a step finished on a throttled call.
• Swarms. Parallel attempts can recover from flaky networks, but each attempt should still pass the same completion guards. Extra parallelism mostly buys wall-clock time, not correctness.

Parallel attempts: cost and diverse configs

Hugging Bay does not publish a fixed call count (such as nine) that guarantees recovery. SDK helpers expose recommendSwarmAttempts / recommend_swarm_attempts with conservative defaults (parallel cap 3, attempts per task cap 5, effective k capped at 3) so agents can log an explicit totalCallsCeiling and cost note instead of unbounded fan-out.

If retries keep failing the same digest, GGUF-header, output, or receipt checks, change one justified lever (quantisation, runner, temperature, prompt)—not blind identical resampling. We have no measured gain from duplicate configs. Proof Lab measurements on real hardware are a better place to spend effort when you need run confidence; see how we measure in Proof Lab for what those numbers do and do not establish.

SDK and CLI

TypeScript (@hugging-bay/sdk): verifyFileBeforeFinish (optional requireGgufHeader), verifyRunBeforeFinish, verifyReceiptBeforeFinish, verifyReceiptWriteBeforeFinish, extractBudgetCompletionHint, recommendSwarmAttempts. Python (hugging-bay): the same names in hugging_bay.guardrails (repository source; published PyPI/npm packages are unchanged until a separate release).

Shell check without importing the SDK:

node packages/agent-guardrails-kit/cli/verify-file.mjs --sha256 <64-hex> --path ./model.gguf --require-gguf-header

Enable the MCP template with HUGGING_BAY_AGENT_GUARDRAILS=1, then call prompts/get for verify-before-finish.

Minimal example

/**
 * Verify-before-finish sample: mock download + explicit digest check + run output gate.
 * No network; safe to run offline.
 */
import { createHash } from "node:crypto";
import { mkdtemp, writeFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import {
  recommendSwarmAttempts,
  verifyFileBeforeFinish,
  verifyRunBeforeFinish,
} from "../../packages/sdk-js/dist/index.js";

const bytes = Buffer.from("fixture weights\n");
const expectedSha256 = createHash("sha256").update(bytes).digest("hex");

const dir = await mkdtemp(join(tmpdir(), "hb-guardrails-"));
const path = join(dir, "weights.bin");
await writeFile(path, bytes);

const file = await verifyFileBeforeFinish({ path, expectedSha256, expectedSizeBytes: bytes.length });
const run = verifyRunBeforeFinish({
  run: { choices: [{ message: { content: "hello from a verified run" } }] },
});
const swarm = recommendSwarmAttempts(4);

console.log(JSON.stringify({ file, run, swarm }, null, 2));
await rm(dir, { recursive: true, force: true });

"""Verify-before-finish sample (offline, no network)."""

from __future__ import annotations

import hashlib
import json
import tempfile
from pathlib import Path

from hugging_bay.guardrails import (
    recommend_swarm_attempts,
    verify_file_before_finish,
    verify_run_before_finish,
)

payload = b"fixture weights\n"
expected = hashlib.sha256(payload).hexdigest()

with tempfile.TemporaryDirectory(prefix="hb-guardrails-") as directory:
    path = Path(directory) / "weights.bin"
    path.write_bytes(payload)
    file_result = verify_file_before_finish(path, expected, expected_size_bytes=len(payload))

run_result = verify_run_before_finish(
    {"choices": [{"message": {"content": "hello from a verified run"}}]},
)
swarm = recommend_swarm_attempts(4)
print(json.dumps({"file": file_result, "run": run_result, "swarm": swarm}, indent=2))

Full sample directory: examples/agent-guardrails-verify/ in the repository.