This is an additional catalog of internet-recommended engineering approaches, technical jargon, and named tools that are worth knowing because they compress a real operating model into a few words. It is deliberately separate from Cory Boehne's signature vocabulary: Volume I captures Cory's language; this volume expands the shared technical language available to the team.
The inclusion test is strict. A term belongs here only if it reliably changes at least one of these things:
- what artifact gets produced;
- how alternatives are searched or selected;
- how evidence and uncertainty are handled;
- how a system behaves under failure or load;
- how a change is deployed, verified, or reversed;
- how authority and trust are bounded.
The fastest useful shortlist#
If a team learns only twenty additions, start here:
- Grammar-constrained decoding — make invalid output impossible at token-generation time.
- Program-aided language models (PAL) — let the model formulate; let a deterministic runtime calculate.
- Hybrid sparse–dense retrieval + reciprocal rank fusion + cross-encoder reranking — retrieve for recall, then rerank for precision.
- Late-interaction retrieval — retain token-level semantic matching without full cross-encoder cost.
- Selective prediction with calibrated abstention — optimize the accuracy/coverage tradeoff instead of forcing an answer.
- MAP-Elites / quality-diversity search — preserve the best solution in every useful behavioral niche.
- NSGA-II / Pareto-front search — keep non-dominated tradeoffs instead of collapsing everything into one score.
- Bayesian optimization — use a surrogate and acquisition function when each experiment is expensive.
- Hyperband / successive halving — kill weak experiments early and reallocate the budget.
- Durable execution with deterministic replay — resume long workflows from event history instead of restarting.
- Parallel change / expand–migrate–contract — evolve an interface without coordinated downtime.
- Traffic teeing / shadow deployment — replay real traffic against a candidate while discarding its effects.
- Fencing tokens — make stale lock holders unable to corrupt a newer owner.
- Backpressure with explicit load shedding — slow or reject work before queues destroy the system.
- Tail-tolerant hedged requests — issue bounded duplicate work only when tail latency justifies it.
- Model checking with TLA+ or Alloy — search system states and counterexamples before code becomes production.
- Metamorphic and differential testing — test relations and implementation disagreements when no oracle is available.
- Hermetic, reproducible builds with SLSA provenance — make release inputs and builder identity verifiable.
- Fault-tree analysis — derive realistic paths to a named top-level failure using Boolean logic.
- Designed experiments + response-surface methodology — estimate effects and interactions with fewer, defensible runs.
AI inference and reasoning controls#
Grammar-constrained decoding#
Meaning: restrict the token choices during decoding to those accepted by a JSON Schema, regular expression, or formal grammar. The result can be guaranteed syntactically valid; it is not guaranteed factually or semantically correct.
Decode against the supplied grammar; reject any semantically invalid but schema-valid result in a separate validator; report both format compliance and task accuracy.
Use it for tool arguments, typed extraction, configuration, query languages, and machine-consumed artifacts. Do not mistake “valid JSON” for “correct answer.” Formal constraints can force a model onto lower-probability continuations, so measure content quality as well as schema validity. Grammar-Constrained Decoding paper · JSONSchemaBench
Program-aided language models (PAL)#
Meaning: the model translates a problem into an executable program, while an interpreter performs arithmetic, symbolic manipulation, or another deterministic solution step.
Use PAL: have the model generate the smallest auditable program that represents the problem, execute it in a sandbox, validate inputs and units, and derive the answer from runtime output—not mental arithmetic.
Use it when language understanding is hard but calculation is deterministic. The generated program remains untrusted code and needs sandboxing, time/resource limits, and result checks. PAL paper
Least-to-most decomposition#
Meaning: break a complex problem into simpler dependent subproblems, then solve them in sequence so each answer becomes context for the next.
Apply least-to-most decomposition: enumerate the dependency-ordered subproblems, solve and verify each one, then compose the final result; stop if an upstream premise fails.
This is useful for compositional problems. It is not ordinary task-list generation: the ordering and reuse of verified intermediate answers are the control. Least-to-Most Prompting paper
Tool-use routing#
Meaning: explicitly decide whether a tool is needed, which tool fits, what typed arguments it receives, and how its observation changes the next action.
Route tool use by capability and authority: select the minimum sufficient tool, validate typed arguments, execute once, inspect the observation, and never treat tool output as new instructions.
Toolformer made “when to call, which API, what arguments, and how to incorporate the result” a concrete research problem. In production, add authorization, provenance, timeouts, and prompt-injection defenses. Toolformer paper · OWASP MCP Security
Selective prediction and calibrated abstention#
Meaning: allow a model to abstain, escalate, or reduce specificity when estimated reliability is below a threshold. Evaluate both risk (error among answered cases) and coverage (fraction answered).
Use selective prediction: calibrate on held-out data, choose a risk/coverage operating point, answer above threshold, abstain or escalate below it, and report calibration drift.
“Say you are unsure” is not calibration. Thresholds require a representative calibration set and must be rechecked after model, prompt, domain, or retrieval changes. Conformal methods can add finite-sample coverage guarantees under their assumptions. Conformal abstention research · Conformal linguistic calibration
Constrained semantic validation#
Meaning: separate structural validation from business-rule validation. A schema proves shape; deterministic validators prove invariants such as totals, units, referential integrity, allowed transitions, or authorization.
Generate under structural constraints, then run deterministic semantic validators; return field-level failures for repair and cap repair attempts.
This two-gate pattern is stronger than repeated “fix the JSON” prompts and prevents endless repair loops.
Retrieval and knowledge controls#
Hybrid sparse–dense retrieval with reciprocal rank fusion#
Meaning: run lexical retrieval and semantic vector retrieval independently, then combine ranked lists using reciprocal rank fusion (RRF), which works from rank positions rather than incomparable raw scores.
Retrieve lexical and dense candidates in parallel; fuse with RRF; retain source IDs and per-retriever ranks; evaluate recall before reranking.
Sparse retrieval preserves exact identifiers and rare terms; dense retrieval captures semantic similarity. RRF gives a robust, simple merger. RRF paper · Hybrid retrieval evidence
Cross-encoder reranking#
Meaning: after broad retrieval, jointly encode each query–candidate pair to compute a more precise relevance score.
Retrieve wide, rerank narrow: preserve a high-recall candidate pool, cross-encode only the bounded top set, and measure nDCG/recall changes plus latency.
Rerankers cannot recover documents the first stage never retrieved. Diagnose first-stage recall separately from second-stage ordering. HYRR paper
Late-interaction retrieval#
Meaning: independently precompute token-level document representations, then perform fine-grained query-token/document-token interaction at search time. ColBERT's MaxSim is the canonical example.
Use late interaction when single-vector embeddings lose exact token relationships but full cross-encoding is too expensive; benchmark index size, recall, and p95 latency together.
It occupies a useful middle ground between single-vector search and expensive pairwise reranking. ColBERT paper
Maximal Marginal Relevance (MMR)#
Meaning: select results by balancing relevance to the query against redundancy with already-selected results.
Apply MMR with an explicit relevance/diversity coefficient; inspect whether unique evidence survives; do not use diversity to admit irrelevant material.
Use it when top-k results repeat the same evidence and crowd out complementary facts. MMR paper · Diversity-focused RAG evidence
Query rewriting and decomposition#
Meaning: turn context-dependent or compound questions into standalone retrieval queries and subqueries before searching.
Rewrite the request into a standalone canonical query plus non-overlapping subqueries; retain the original wording; retrieve both; compare whether rewriting changed intent.
This improves multi-turn retrieval but can silently erase constraints. Always retain and evaluate against the original request.
Retrieval budget allocation#
Meaning: allocate fixed token and latency budgets among query variants, retrievers, reranking, context compression, and generation instead of treating context length as free.
Declare the retrieval budget; reserve evidence slots by subquestion; deduplicate by claim, not string; stop adding context when marginal evidence gain falls below cost.
This is the operational answer to “retrieve more.” More context can introduce noise, redundancy, and lost-in-the-middle failures.
Search, optimization, and evolutionary controls#
MAP-Elites and quality-diversity search#
Meaning: define behavioral dimensions, divide them into niches, and retain the highest-quality solution found in each niche. The output is a repertoire, not one global champion.
Run MAP-Elites: define measurable behavior descriptors and fitness; mutate/crossover occupants; keep the elite per niche; preserve the archive and lineage; test whether niches are genuinely useful.
This is a precise upgrade to genetic “winner retention” when diversity itself has value. It protects useful specialists from being eliminated by one scalar objective. MAP-Elites / quality-diversity research · Scaling MAP-Elites
NSGA-II and Pareto-front search#
Meaning: optimize multiple conflicting objectives while retaining non-dominated candidates—solutions for which no objective can improve without worsening another.
Use NSGA-II with explicit objectives and constraints; publish the Pareto front; choose a final operating point through a documented human or policy preference, not a hidden weighted sum.
Use it for accuracy/cost/latency, novelty/reliability, or automation/human-effort tradeoffs. A Pareto front exposes choices; it does not choose for you. NSGA-II paper
Bayesian optimization#
Meaning: fit a probabilistic surrogate to an expensive objective, then use an acquisition function—such as expected improvement—to choose the next evaluation by balancing exploration and exploitation.
Use Bayesian optimization when evaluations are expensive: declare the search space and constraints, fit the surrogate, select by a named acquisition function, run the experiment, update, and stop at a budget or improvement threshold.
It is generally a poor fit for huge, highly discrete spaces unless specialized methods are used. Bayesian optimization tutorial
Hyperband and successive halving#
Meaning: start many configurations with small resource allocations, discard weak performers, and progressively allocate more budget to survivors.
Run successive halving with predeclared rungs, comparable early metrics, and a protected minimum budget; retain checkpoint and elimination evidence.
Use it when early performance predicts later performance. It can kill slow starters unfairly, so verify the fidelity of the early signal. Hyperband paper
Multi-armed bandits#
Meaning: allocate traffic or trials adaptively among alternatives while balancing exploration against exploitation. Common policies include UCB and Thompson sampling.
Use a contextual bandit only when rewards arrive online: define reward delay, regret objective, exploration floor, safety constraints, and off-policy evaluation before activation.
Bandits optimize during learning; A/B tests estimate fixed treatment effects. Do not substitute one for the other without understanding the inference tradeoff.
Design of experiments and response-surface methodology#
Meaning: choose factor combinations systematically so main effects and interactions can be estimated efficiently, then model the response surface to find improved settings.
Run a DOE: define factors, levels, responses, nuisance blocks, randomization, and interactions; validate the fitted model and residuals; confirm the predicted optimum with a new run.
Changing one factor at a time cannot efficiently reveal interactions. Designed experiments produce defensible conclusions with fewer runs. NIST DOE definition · NIST DOE analysis
Architecture and migration controls#
Durable execution with deterministic replay#
Meaning: persist workflow history so code can replay deterministically after crashes and resume outstanding activities without manually reconstructing state.
Model this as a durable workflow: separate deterministic orchestration from side-effecting activities; assign idempotency keys and timeouts; version replay-breaking changes; test crash recovery at every boundary.
This is ideal for hours-to-months processes, approvals, retries, and external callbacks. “Put it on a queue” is not durable workflow state. Temporal durable-execution guide
Parallel change / expand–migrate–contract#
Meaning: add the new interface or field compatibly, migrate consumers and data, then remove the old path only after evidence shows it is unused.
Use expand–migrate–contract: expand compatibly, instrument old/new usage, migrate every consumer, prove zero old-path traffic, then contract in a separate release.
It prevents lockstep deployment and is especially useful for schemas and APIs. Parallel Change · Pact expand/contract guidance
Traffic teeing / shadow deployment#
Meaning: duplicate production requests to a candidate system while the control system remains authoritative and candidate responses are discarded or compared.
Shadow real traffic with side effects disabled or isolated; compare response semantics and latency; redact sensitive fields; cap volume; never let the shadow become an accidental second writer.
Shadow traffic improves state coverage over synthetic load but is dangerous around writes, billing, shared caches, and privacy. Google SRE traffic-teeing guidance
Event sourcing and CQRS#
Meaning: event sourcing stores immutable domain events as the authoritative history; CQRS separates write commands from read models. They can be used independently.
If using event sourcing, define event versioning, invariant enforcement, idempotency, replay cost, snapshots, correction events, and read-model rebuild proof before implementation.
Do not adopt it merely to obtain an audit log. It introduces temporal modeling and migration obligations.
Anti-corruption layer#
Meaning: translate between a legacy/external model and the internal domain model so foreign concepts do not spread through the codebase.
Place an anti-corruption layer at the boundary; map identity, units, lifecycle, errors, and terminology explicitly; keep the external schema out of core domain types.
This is a domain-integrity control, not just an API wrapper.
Control plane / data plane separation#
Meaning: separate policy, desired state, and orchestration from the high-volume path that performs actual work.
Separate control and data planes; define propagation delay, stale-policy behavior, last-known-good state, authorization, and what the data plane does when control is unavailable.
This term should trigger failure-mode design around stale control, not only a diagram with two boxes.
Distributed-systems correctness controls#
Fencing tokens#
Meaning: issue monotonically increasing tokens with leases or locks; the protected resource rejects operations carrying an older token, preventing a paused former owner from writing after a new owner takes over.
Use a lease plus fencing token: every ownership grant increments the token; every write carries it; the resource rejects tokens lower than the latest accepted value.
A distributed lock without fencing may still permit stale-owner corruption after pauses or network delays.
Backpressure and load shedding#
Meaning: backpressure propagates inability to accept work upstream; load shedding deliberately rejects lower-priority work before overload causes universal failure.
Define queue ceilings, admission policy, priority classes, retry-after behavior, and shedding metrics; reject early enough that accepted work still meets its SLO.
An unbounded queue converts overload into memory pressure and catastrophic tail latency. Google SRE managing load · AWS throttling guidance
Tail-tolerant hedged requests#
Meaning: when a request exceeds a delay threshold, send a bounded duplicate to another replica and use the first valid response, canceling the remainder.
Hedge only idempotent reads above a measured percentile threshold; cap duplicate load; diversify failure domains; cancel losers; compare p99 gain against added capacity.
Blindly duplicating every request can amplify overload. Tail tolerance is a conditional, measured strategy. The Tail at Scale
Consistent hashing with bounded loads#
Meaning: assign keys to changing nodes while minimizing remapping, with explicit capacity bounds to prevent badly imbalanced nodes.
Use consistent hashing with virtual nodes or bounded-load guarantees; test skew, churn, hot keys, node loss, and movement cost—not only uniform random keys.
Plain consistent hashing controls remapping but does not automatically guarantee acceptable balance. Google bounded-load hashing research
CRDT#
Meaning: a conflict-free replicated data type has mathematically defined merge behavior that lets independently updated replicas converge without central coordination.
Choose a CRDT whose algebra matches the domain; state the merge law and causal assumptions; test concurrent operations and tombstone/metadata growth.
“Last write wins” is not a universal conflict strategy. CRDTs preserve specific semantics and often carry storage or product tradeoffs. Verified CRDT research
Exactly-once processing scope#
Meaning: exactly-once claims apply only inside a defined transactional boundary. External side effects still require idempotency, deduplication, or transactional coordination.
State the exact-once boundary; atomically bind input offsets, state changes, and output records where supported; make every external effect idempotent and replay-test it.
Kafka's documentation is explicit that guarantees depend on producer, consumer isolation, transactions, and where effects are written. Kafka delivery semantics
Verification and security controls#
Model checking#
Meaning: specify states, transitions, invariants, and temporal properties, then exhaustively or symbolically search for counterexamples within a bounded model.
Model the protocol in TLA+ or Alloy; encode safety and liveness properties; explore retries, reordering, duplication, delay, and crash recovery; retain minimal counterexamples as design tests.
Model checking validates the model, not the implementation. Connect findings to code through tests, invariants, and telemetry.
Metamorphic testing#
Meaning: when the correct output is unknown, define transformations whose outputs must have a known relationship—for example permutation invariance, scale consistency, or round-trip preservation.
Write metamorphic relations, generate source cases, transform them, and assert the required relation between outputs; treat every violated relation as a minimal reproducible defect.
This is powerful for ML, search, simulation, compilers, and numerical systems where exact test oracles are scarce.
Differential testing#
Meaning: run the same inputs through independent implementations, versions, models, or execution modes and investigate disagreements.
Run blinded differential tests across independent implementations; normalize only irrelevant variation; cluster disagreements; adjudicate with a stronger oracle or human review.
Agreement is evidence, not proof—implementations can share the same bug or source.
Hermetic and reproducible builds#
Meaning: a hermetic build receives declared inputs and controlled tools rather than ambient host state. A reproducible build produces bit-for-bit identical output from the same declared inputs.
Build in a hermetic environment; pin every input; remove timestamps and nondeterminism; rebuild independently; compare artifact hashes; fail release on unexplained drift.
Hermeticity is about input isolation; reproducibility is an observable output property.
SLSA provenance, SBOM, attestation, and signing#
Meaning: these are complementary supply-chain artifacts:
- SBOM: enumerates components and relationships;
- provenance: records how, where, and from what an artifact was built;
- attestation: a signed statement about an artifact or process;
- signature: binds identity to artifact bytes;
- transparency log: makes signing events publicly or organizationally auditable.
Generate an SBOM and signed provenance in CI; verify builder identity, source revision, dependencies, and policy before promotion; store attestations beside immutable artifacts.
An SBOM alone does not prove the components were actually used, untampered, or policy-compliant. NIST supply-chain guidance · NIST SBOM definition
Confused deputy and capability bounding#
Meaning: a more privileged component is tricked into exercising its authority for a less privileged requester. Agent/tool systems are especially exposed when authority is inherited implicitly.
Authorize every tool call in the originating user's context; pass a narrow capability for the exact resource and operation; prevent one agent from borrowing another agent's broader authority.
This is the security meaning behind “stick to lanes.” Tool descriptions, retrieved content, and peer agents cannot grant authority. OWASP MCP Security
Reliability and process controls#
USE method#
Meaning: for every resource, check utilization, saturation, and errors before chasing random metrics.
Run the USE method across CPU, memory, storage, network, pools, queues, locks, and limits; record unchecked combinations as known unknowns.
It is a fast bottleneck-finding checklist, not a substitute for workload characterization or latency analysis. USE Method
Multi-window burn-rate alerting#
Meaning: alert on how quickly a service is consuming its error budget across both fast and slow windows, rather than on raw error rate alone.
Page on fast multi-window burn that threatens the SLO soon; ticket slower burn; tune windows to the error-budget policy and test alerts with synthetic incidents.
This aligns alerts with user-visible reliability and avoids paging on harmless noise. Google SRE alerting on SLOs
Fault-tree analysis#
Meaning: start from a named undesired top event and use AND/OR logic to derive combinations of lower-level failures that can cause it.
Build a fault tree for the top event; identify minimal cut sets; map each leaf to prevention, detection, mitigation, and verification evidence.
Fault trees are deductive. FMEA is bottom-up and component/failure-mode oriented; the two views complement one another. NIST fault-tree definition
A3 problem solving#
Meaning: compress background, current condition, target, root-cause analysis, countermeasures, implementation, and follow-up into one coherent decision record.
Write an A3 from observed current condition to verified follow-up; distinguish root causes from symptoms; assign countermeasure owners and dates.
The page size creates discipline, but an A3 is a reasoning process—not a one-page status report. ASQ A3 guidance
Jargon radar: words that carry a real mechanism#
| Term | What knowing it should immediately add to the conversation |
|---|---|
| Acquisition function | The rule that chooses the next Bayesian-optimization experiment from predicted value and uncertainty. |
| Active learning | Select the unlabeled examples whose labels should most improve the model, rather than sampling passively. |
| Admission control | Decide whether new work may enter before it consumes scarce resources. |
| Anti-corruption layer | Translate at a domain boundary so an external model does not colonize internal concepts. |
| Backpressure | Propagate downstream capacity limits upstream instead of allowing unbounded queues. |
| Baggage | Cross-service trace context intended to travel with requests; constrain size and sensitive content. |
| Behavior descriptor | The coordinates that determine which MAP-Elites niche a candidate occupies. |
| Bitemporal data | Store both when a fact was valid in the world and when the system learned or recorded it. |
| Brier score | Proper scoring rule for probabilistic predictions; lower means probabilities better match outcomes. |
| Calibration curve | Compare predicted confidence bins with observed accuracy; confidence is useful only if calibrated. |
| Cardinality budget | Bound the number of unique metric or trace attribute values so telemetry remains affordable and queryable. |
| Causal consistency | Preserve happens-before relationships while allowing concurrent operations to appear in different orders. |
| Change-data capture (CDC) | Stream committed database changes from a log rather than repeatedly polling whole tables. |
| Characterization test | Capture existing behavior before changing code, including ugly behavior that might be relied upon. |
| Clock skew | Difference between node clocks; design protocols that do not mistake wall-clock agreement for ordering. |
| Compensating action | A semantic undo for a completed distributed step when atomic rollback is impossible. |
| Constraint programming / CP-SAT | Declare variables, domains, and constraints; let a solver search scheduling and assignment spaces. |
| Content-addressed artifact | Identify bytes by cryptographic digest so identity changes whenever content changes. |
| Control plane | Distributes policy and desired state; define stale-control and outage behavior for the data plane. |
| Coordinated omission | A load-test measurement error that stops issuing work during stalls and therefore hides the worst latency. |
| CRDT | A replicated data type with a merge rule proven to converge under its assumptions. |
| Critical path | The dependency chain that determines minimum completion time; speedups elsewhere do not shorten the result. |
| Dead-letter queue | Quarantine work that exceeded retry policy, with reason, payload reference, replay controls, and ownership. |
| Deterministic replay | Re-execute orchestration from event history and obtain the same decisions. |
| Differential privacy | Bound how much one individual's data can affect released output using a declared privacy budget. |
| Drift detector | Test whether input, label, concept, calibration, or performance distributions changed materially. |
| Evaluation contamination | Benchmark examples or answers leaked into training, prompts, tools, or tuning data. |
| Event time / watermark | Process streaming records by occurrence time and declare how late data may arrive before windows close. |
| Fencing token | Monotonic ownership number that lets a resource reject stale lock holders. |
| Golden master / approval test | Compare complex output to a reviewed baseline; require intentional approval for every change. |
| Hedged request | Delayed, bounded duplicate sent to reduce tail latency; cancel losers and measure amplification. |
| Hermetic build | Build whose inputs and tools are declared rather than inherited from ambient host state. |
| Idempotency key | Stable operation identity that lets retries produce one external effect. |
| Invariant | A property that must remain true across every allowed state transition. |
| Lamport timestamp | Logical ordering that respects causality without pretending to capture physical time. |
| Late interaction | Match query and document at token level after independently precomputing representations. |
| Lease | Time-bounded ownership requiring renewal; pair with fencing when stale actors can still write. |
| Linearizability | Operations appear atomic and ordered consistently with real-time observation. |
| Load shedding | Deliberately reject work so accepted traffic continues meeting reliability targets. |
| Log-structured merge tree (LSM) | Write-optimized storage using memtables, sorted runs, and compaction—with read/write amplification tradeoffs. |
| Maximal Marginal Relevance | Select for relevance and novelty relative to already-selected evidence. |
| Minimal cut set | Smallest combination of fault-tree leaf events sufficient to cause the top event. |
| Model card | Document intended use, evaluation, limitations, and risk—not a marketing readme. |
| Mutation score | Fraction of non-equivalent injected defects killed by tests. |
| Non-dominated solution | Candidate no other candidate improves on every objective; member of the Pareto front. |
| Off-policy evaluation | Estimate a new decision policy from logged data produced by another policy, with propensity assumptions. |
| Optimistic concurrency control | Detect conflicts at commit/version check instead of locking work in advance. |
| Pareto front | The exposed tradeoff surface among conflicting objectives. |
| Poison message | Input that repeatedly fails processing; isolate it before it blocks or churns the entire queue. |
| Policy-as-code | Version authorization or compliance rules as executable policy with tests and decision logs. |
| Probabilistic programming | Express a generative model and infer posterior distributions rather than writing one-off estimators. |
| Provenance | Evidence of an artifact's source inputs, builder, process, and identity. |
| Quorum | Minimum replicas required for an operation; state read/write quorum and failure assumptions. |
| Read repair | Reconcile stale replicas discovered during reads; account for latency and conflict semantics. |
| Reciprocal rank fusion | Merge ranked lists using reciprocal rank rather than incomparable raw scores. |
| Red/black or blue/green deployment | Maintain old and new environments so traffic switching and rollback are routing operations. |
| Rejection option | Classifier/model may abstain when confidence is below an operating threshold. |
| Rendezvous hashing | Assign each key to the highest-scoring node, minimizing redistribution when membership changes. |
| Replay attack | Reuse a previously valid message or approval; prevent with nonce, expiry, sequence, and binding. |
| Response-surface methodology | Fit and navigate a model of responses over interacting factors to find improved settings. |
| Schema evolution | Change serialized data compatibly with explicit reader/writer and migration rules. |
| Selective risk | Error rate among cases a selective model chooses to answer. |
| Semantic cache | Reuse results for sufficiently equivalent requests; define similarity, freshness, tenancy, and invalidation. |
| Shadow traffic | Send copied real traffic to a non-authoritative candidate and discard or compare its response. |
| Span link | Connect traces with causal relationships that are not a strict parent/child tree, such as batch processing. |
| Stable sort / deterministic tie-breaker | Ensure equivalent scores produce repeatable ordering and reproducible outputs. |
| State-machine replication | Replicas apply a totally ordered deterministic command log to remain identical. |
| Surrogate model | Cheap approximation of an expensive objective used to decide where to experiment next. |
| Tail latency | High-percentile response time; averages conceal the requests users remember. |
| Taint tracking | Propagate untrusted-data labels through transformations and block them at sensitive sinks. |
| Tombstone | Durable deletion marker used in replicated/log-structured systems; plan compaction and resurrection prevention. |
| Traffic shaping | Control request rate, burst, priority, or route before work reaches constrained resources. |
| Vector clock | Track causal histories sufficiently to detect concurrency, at metadata cost. |
| Write amplification | Physical bytes written per logical byte; a key compaction and storage-endurance cost. |
Tools worth recognizing#
Knowing a tool name is not the same as knowing the mechanism. These tools are included because each is a common concrete implementation of a valuable control.
| Tool or standard | Mechanism it represents | What to ask when someone proposes it |
|---|---|---|
| TLA+ / TLC | Temporal specification and exhaustive state exploration | Which safety/liveness properties and failure interleavings are modeled? |
| Alloy | Relational modeling with bounded counterexample search | What scope was searched, and what does the bound exclude? |
| Temporal | Durable workflow execution and deterministic replay | Which code is deterministic workflow logic versus retryable activity? |
| OR-Tools CP-SAT | Constraint programming for scheduling and assignment | What are the variables, hard constraints, objective, and optimality gap? |
| Hypothesis / QuickCheck | Property-based and stateful model-based testing | Which invariants and generators matter, and do failures shrink well? |
| Stryker / PIT | Mutation testing | Which surviving mutants are missing assertions versus equivalent mutants? |
| Pact | Consumer-driven contract testing | Are contracts generated from real consumer behavior and verified before deploy? |
| OpenTelemetry | Vendor-neutral traces, metrics, logs, resources, and semantic conventions | Are names low-cardinality and context propagated end to end? |
| OpenFeature | Vendor-neutral feature-flag evaluation API | What is the targeting context, safe default, owner, telemetry, and removal date? |
| OPA / Rego | Policy-as-code decision engine | What inputs are trusted, what decision is returned, and which tests prove deny-by-default? |
| SLSA | Supply-chain provenance maturity and build integrity | Which provenance level is actually met and verified at admission? |
| in-toto | Attested software-supply-chain steps and layout verification | Which actors may perform each step and which materials/products are bound? |
| Sigstore / cosign / Rekor | Keyless signing, verification, and transparency logging | Which identity signed the digest and is log inclusion/policy verified? |
| Syft / CycloneDX / SPDX | SBOM generation and interchange | Is the SBOM complete, tied to the artifact digest, and retained per release? |
| Grype / Trivy | Artifact and dependency vulnerability scanning | What database freshness, reachability context, exception owner, and remediation SLA apply? |
| ColBERT | Token-level late-interaction retrieval | Does its recall/latency/index-size tradeoff beat the simpler baseline? |
| FAISS | High-performance approximate vector search | Which index type, recall target, memory budget, and rebuild strategy are chosen? |
| BM25 | Lexical relevance ranking | How are analyzers, fields, boosts, and exact identifiers handled? |
| XGrammar / Outlines / Guidance | Grammar- or schema-constrained decoding | Does the backend guarantee compliance for the schemas actually used? |
| MLflow | Experiment tracking, model registry, and lifecycle metadata | Are data, code, prompt, model, parameters, and metrics version-bound? |
| OpenLineage | Standardized job/run/dataset lineage events | Can an output be traced to exact upstream datasets and transformations? |
| k6 / Locust | Programmable load generation | Does the workload represent arrival rate, state, data distribution, and coordinated omission correctly? |
| Jepsen | Black-box distributed-systems correctness testing under faults | Which consistency model and nemeses are being tested? |
| Chaos Mesh / Litmus | Controlled infrastructure fault injection | What steady state, blast radius, abort condition, and recovery evidence are defined? |
| Prometheus / Alertmanager | Time-series monitoring and alert routing | Are alerts tied to symptoms/SLO burn rather than noisy causes? |
| eBPF / bpftrace | Low-overhead kernel and application observability | Which probe answers the hypothesis, and what overhead/cardinality limit applies? |
| FlameGraph | Aggregated stack-profile visualization | Is the profile on-CPU, off-CPU, allocation, lock, or another specific dimension? |
Compact commands that should change behavior#
Generate the output under a formal grammar, then run deterministic semantic validators; repair only field-level failures and cap retries.
Retrieve with BM25 and dense embeddings, fuse with RRF, rerank with a cross-encoder, apply MMR for evidence diversity, and evaluate first-stage recall separately from final-answer faithfulness.
Use selective prediction at a calibrated risk/coverage threshold; abstain or escalate below it, and report calibration drift by cohort.
Run MAP-Elites over explicit behavioral descriptors; retain the elite and lineage in every occupied niche; periodically reevaluate the archive on a fixed holdout.
Search the accuracy/cost/latency Pareto front with NSGA-II; do not collapse objectives until the decision maker selects an operating preference.
Use Bayesian optimization with a named acquisition function for expensive experiments; stop at the budget or marginal-improvement threshold and confirm the predicted optimum.
Model this as a durable workflow with deterministic replay; isolate side effects as idempotent activities and crash-test every boundary.
Migrate by expand–migrate–contract; instrument old-path use; remove compatibility only after every consumer proves migration.
Shadow production traffic with all candidate side effects disabled; compare semantic output and p95/p99 latency; bound traffic and delete copied sensitive data on schedule.
Apply backpressure and admission control before saturation; shed low-priority work while accepted traffic can still meet its SLO.
Model the protocol in TLA+; check invariants and liveness across duplication, reordering, partitions, crashes, retries, and stale ownership; turn counterexamples into implementation tests.
Build hermetically, reproduce the artifact independently, emit SBOM and signed SLSA provenance, and admit only the verified digest.
Run a fault-tree analysis from the named top event, compute minimal cut sets, and bind every leaf to prevention, detection, mitigation, and a verification test.
Run a designed experiment with randomization, blocking, interaction terms, and a capable measurement system; fit the response surface and confirm the optimum in a fresh run.
Terms that still need a qualifier#
| Incomplete phrase | Operational qualifier to demand |
|---|---|
| “Use tools” | Selection policy, typed contract, authority, timeout, observation handling, and injection boundary. |
| “Structured output” | Grammar/schema guarantee plus semantic validators and repair ceiling. |
| “Hybrid search” | Named retrievers, fusion rule, candidate depths, reranker, and evaluation metrics. |
| “Confidence” | Definition, calibration data, scoring rule, threshold, coverage, and drift check. |
| “Genetic optimization” | Genome, mutation/crossover, fitness, constraints, diversity mechanism, lineage, and holdout. |
| “Exactly once” | Exact transactional boundary and handling of every external side effect. |
| “Distributed lock” | Lease duration, renewal, fencing, clock assumptions, and stale-owner behavior. |
| “Event driven” | Event ownership, schema evolution, ordering, delivery semantics, idempotency, replay, and DLQ. |
| “Zero downtime” | Compatibility sequence, traffic switch, state migration, rollback, and proof window. |
| “Formally verified” | Model/specification, property, proof/checker, bounds, assumptions, and link to implementation. |
| “Reproducible” | Independent rebuild, pinned inputs, normalized nondeterminism, and digest equality. |
| “Observability” | User-centered questions, signals, semantic conventions, correlation, cardinality, retention, and owner. |
| “Chaos testing” | Steady state, hypothesis, one named fault, blast radius, abort condition, recovery, and retained evidence. |
| “Data lineage” | Dataset, field, job/run, code version, temporal validity, and queryable upstream/downstream edges. |
| “Optimize” | Objective, constraints, baseline, search method, budget, holdout, and regression guardrail. |
Research boundary#
This is a selected operating lexicon, not a claim that every named method fits every system. Sources were limited primarily to original research papers, standards, official project documentation, and authoritative engineering handbooks. Selection favored mechanisms that are:
- precise enough to change implementation behavior;
- testable or inspectable;
- useful across more than one vendor or framework;
- commonly misunderstood when reduced to a buzzword;
- complementary to, rather than duplicative of, the first field guide.
The fastest way to misuse technical vocabulary is to invoke a name without its assumptions. Treat every term as a compressed contract: expand it into inputs, authority, algorithm, artifact, metrics, stop conditions, failure behavior, and evidence before trusting the result.
Put the language to work
Specific words should create specific behavior.
Use the compact command patterns as written, then add your artifact and acceptance condition. The phrase is the control surface; verification is the finish line.
Back to the library