Skip to content

AI extraction engine: scope-line / assembly / equipment / vendor-quote from PDFs (PIN-420 assembly extraction)

StageBuilding
StatusPartial
Design statusApproved
Spec statusIn review
EstimateM-L
ConfidenceMedium
LinearPIN-420 ↗ PIN-228 ↗
Linear statusDone · +1 planned
CycleC14
DesignDesign ↗
Linear epicEstimating & Proposals
ModuleEstimating ↗

Route: /bids/:bidId?tab=estimate (extraction runs + review modals)

Scope

The AI extraction engine pulls structured estimate data out of bid PDFs — scope lines (CSI codes + descriptions), assemblies (multi-component groups), equipment schedules, and vendor-quote amounts — into the Estimate Builder. It runs asynchronously (RPC + SSE progress), packs documents under a hard request-byte budget, and routes every extracted item through a review surface (Accept / Edit / Reject per item, or Accept-all / Reject-assembly). Nothing enters the estimate without estimator confirmation; extraction runs are logged separately for training. PIN-420 is the shipped assembly-extraction slice; PIN-228 is the broader production integration across document types. (Source: PIN-420, PIN-228.)

Acceptance criteria

Happy path:

  • "Extract Assembly" and "Extract Equipment" actions in the estimate toolbar; "Upload bid package PDFs to extract equipment, scope, and vendor information" is the entry copy in the design.
  • Extraction runs async; the UI streams progress over SSE while the document is processed (30–120 s).
  • Per-item review: Accept, Edit, or Reject each extracted item; Accept-All / Reject-Assembly in one action.
  • Confirmed items become estimate line items; re-confirming does not create duplicates (repo skips Id==0, iterates saved models).
  • Equipment-schedule extraction (specs + drawings) feeds the Equipment tab with CSI codes; scope-line extraction suggests CSI + description + labor basis.
  • Vendor-quote extraction yields vendor + amounts and maps to scope lines for bulk approval.
  • A manual page picker is offered when auto-detect confidence is low; multi-page fallback retries later pages.

Negative / error:

  • On a provider error, the terminal handler stores/publishes only ai.FriendlyAIError(err) (never raw provider text); the raw error is logged.
  • Oversized requests are classified as ai.ErrRequestTooLarge (HTTP 413) and the batch is trimmed to a fitting subset; skipped docs are named in the analyzing SSE message and the stored summary.
  • A document that can't be parsed degrades gracefully and does not block the estimate.

Validation rules:

  • Total base64-encoded request bytes ≤ ai.MaxRequestEncodedBytes (~24 MB); PDFs count-capped (≤10, priority-sorted) then trimmed to budget.
  • No synthetic data — no extracted value auto-enters the estimate; the estimator confirms first.
  • Extraction/comparison runs are stored separately from the live estimate for training.

Walkthrough scenario(s)

Happy path.

  1. Estimator uploads a bid package and clicks Extract Equipment.
  2. runEquipmentExtraction sends one PDF per request under the byte budget; the UI streams "analyzing…" progress via SSE.
  3. The equipment review modal lists extracted units (name, capacity, model, manufacturer, connections); estimator Accepts most, edits one, rejects a duplicate.
  4. Accepted units land in the Equipment tab; CSI codes resolve via the classification goroutine (cache → Claude).

Exception.

  1. A large drawing set exceeds the request budget.
  2. selectScopePdfCandidates count-caps and selectPdfDocsWithinBudget trims to a fitting subset; the SSE message names the skipped drawings.
  3. Extraction completes on the subset; the estimator re-runs on the remainder.

Edge cases

  • Assembly BOMs from CAD — per-component Accept/Override/Reject (PIN-420 modal), mirroring the equipment modal semantics.
  • Handwritten / image annotations — low-confidence → desk review, never a silent fallback.
  • Scope-line reuse — GetClassifiedLineItemByDescription(desc, category) reuses a prior classification (category-scoped) before calling Claude; csi_confidence = 0 means "ran, no match" (retry-able), distinct from null (never ran).
  • Two entry points — lines added via GhostAddRow go through BidService.AddLineItem; batch edits through EstimateService.BatchUpdate. The CSI goroutine lives in both.

Flow

sequenceDiagram
    participant Est as Estimator
    participant API as Estimate service
    participant AI as Claude (vision)
    participant SSE as Extraction events (SSE)
    participant Rev as Review modal
    Est->>API: Extract Equipment / Assembly
    API->>API: select + budget-trim PDFs (MaxRequestEncodedBytes)
    API->>AI: async extraction request(s)
    API-->>SSE: analyzing… (names skipped docs)
    AI-->>API: extracted items
    API-->>Rev: items with confidence
    Est->>Rev: Accept / Edit / Reject (no auto-enter)
    Rev->>API: confirmed → line items (+ CSI goroutine)

Backend / API notes

  • Service: runScopeExtraction / runEquipmentExtraction in apps/api/service/estimate.go; selectScopePdfCandidates, selectPdfDocsWithinBudget; terminal handlers finishExtraction / finishEquipmentExtraction publish FriendlyAIError only.
  • Transport: async RPC + SSE (EventSource) for non-blocking progress; separate SSE prefixes for estimate vs assembly extraction (distinct handler registration).
  • CSI: classification goroutine after AddLineItem and in BatchUpdate; estResolveCostCode, GetClassifiedLineItemByDescription, UpdateLineItemCostCode (writes csi_code text + FK + confidence).
  • Client: ai.SendMessageWithContext classifies 413 as ErrRequestTooLarge (errors.Is-detectable; raw body preserved via %w).

AI integration

  • Prompt intent: extract equipment/scope/assembly/vendor data from bid PDFs and drawings; suggest CSI codes and labor basis.
  • Model / budget: Claude vision (Sonnet tier) for the 30–120 s extraction; hybrid vector-parse + verify is the later production target. Byte budget ≈ 24 MB base64 per request.
  • Accuracy target: ~70–80% first pass → ~85–90% after ~20–30 corrections; corrections feed future accuracy.

Conor's comments

Core AI extraction from drawings & quotes.

Open questions for Conor

  • No open scope decision — this is a shipped/hardening capability. Confirm the priority order for extending extraction (assembly Phase 2/3, symbol/legend library) relative to the C14 estimate work.