4. Text Generation, Structured Outputs, and Reliable Extraction

Build generation, classification, summarization, and schema-constrained outputs with robust parsing and validation.

Text Generation and Controlled Output

Text generation is useful for drafting, rewriting, explanation, and synthesis. Specify audience, purpose, constraints, tone, and maximum length when these matter. Generation parameters such as temperature are provider/model-specific and should be tuned empirically; lower randomness does not guarantee truth.

Use model output as a draft or candidate when the task affects money, eligibility, compliance, or user access. For exact computations, call deterministic code. For claims about supplied documents, require grounding and inspectable sources.

Reusable generation function

async function draftReleaseNotes(generate, changes) {
  return generate({
    instructions: [
      "Write release notes for software users.",
      "Group changes into Added, Improved, and Fixed.",
      "Do not invent features absent from the input."
    ].join("\n"),
    input: JSON.stringify(changes)
  });
}

💡 The wrapper keeps the task contract reusable. The downstream application should still review or validate the result, particularly if release notes must exactly match shipped changes.

Key Takeaways

Define audience and constraints.
Do not use model generation for exact arithmetic.
Ground high-impact claims in evidence and review.

Knowledge Check

What is the safest role for a model when generating high-impact business text?

Structured Outputs and JSON Schema

Structured output features can constrain a model response to a declared schema. This is more reliable than asking “return JSON” in plain text, but schema conformance does not prove that values are true or authorized. Keep schemas focused, make required versus optional fields explicit, use enums for closed sets, and validate cross-field rules in application code.

Use a runtime schema library such as Zod for validation at the application boundary. If the provider SDK supports schema-based structured outputs, use the current documented integration and handle refusals, incomplete output, timeouts, and provider errors as separate outcomes.

Validate model output with Zod

import { z } from "zod";

const TicketSchema = z.object({
  category: z.enum(["billing", "technical", "account", "other"]),
  priority: z.enum(["low", "normal", "high"]),
  summary: z.string().min(1).max(300)
});

const parsed = TicketSchema.safeParse(modelOutput);
if (!parsed.success) {
  throw new Error("Model output failed validation");
}
const ticket = parsed.data;

💡 The schema checks types, allowed labels, and summary length. Production code should return a controlled error or retry safely instead of crashing the whole request on invalid output.

Key Takeaways

Prefer structured output for machine-consumed results.
Validate domain constraints after structural parsing.
Handle refusal and incomplete responses explicitly.

Knowledge Check

What does schema-constrained output guarantee most directly?

Classification, Extraction, and Summarization

These common patterns need different evaluation criteria. Classification should be measured with per-class precision and recall, not only overall accuracy when classes are imbalanced. Extraction should track field-level correctness, missing-field behavior, and normalization errors. Summarization should preserve key facts and avoid adding unsupported claims.

For bulk jobs, use bounded concurrency, retry only transient failures, and store job status so the batch can resume. Keep raw inputs and outputs only when retention policy permits; redact sensitive fields and establish deletion behavior.

Batch classify tickets with bounded concurrency

async function processBatch(items, classify, concurrency = 5) {
  const results = new Array(items.length);
  let next = 0;

  async function worker() {
    while (true) {
      const index = next++;
      if (index >= items.length) return;
      results[index] = await classify(items[index]);
    }
  }

  await Promise.all(
    Array.from({ length: Math.min(concurrency, items.length) }, worker)
  );
  return results;
}

💡 This small worker pool caps simultaneous calls and preserves input order in the result array. Production code should also implement per-item retries, cancellation, rate-limit handling, and durable job checkpoints.

Key Takeaways

Evaluate each task type with suitable metrics.
Use bounded concurrency for bulk requests.
Design retention and redaction before logging raw AI data.

Knowledge Check

Which metric set is especially useful for an imbalanced classification task?

Text Generation and Controlled Output

Structured Outputs and JSON Schema

Classification, Extraction, and Summarization

STAY UPDATED WITH BYTES!