will
8e6ee31596
ci / test (push) Successful in 6m18s
History-free snapshot of the private development repository at cbe45c5 (2026-09-09). Code, docs and public catalogue only; no plant or job knowledge layers, no client or job names (tools/scrub.py clean). The intro deck is left out of this snapshot because its slide images could not be checked for job drawings.
1548 lines
72 KiB
Python
1548 lines
72 KiB
Python
#!/usr/bin/env python3
|
|
"""aiextract_run - the DSV4F-vision half of Design Extract, run out of process.
|
|
|
|
python tools/aiextract_run.py --job <job.json>
|
|
|
|
Design Extract now chains two extraction passes over one design summary
|
|
(owner, Discord, 2026-09-01, approving the proposal: "可以，不过别用opus，用
|
|
dsv4f带vision吧" -- yes, but not opus, use DSV4F with vision):
|
|
|
|
1. the mechanical reader (`vibe.designspec.build_spec`) -- regex and
|
|
geometry only, never guesses, leaves everything it cannot place as
|
|
`null` or in `unresolved`. It never even attempts the strand matrix:
|
|
its own docstring says a wrong grid is worse than an honest blank.
|
|
2. THIS SCRIPT -- reads whatever the mechanical pass left behind, plus
|
|
the strand matrix (drawn geometry, not text, which is why it needs
|
|
vision), by putting a rendered PNG of the summary view in front of a
|
|
vision-capable model and asking it to fill in the gaps.
|
|
|
|
It exists as a separate CPython process, run detached from Revit, for the
|
|
same reason `vibe.aiextract`'s old opus-CLI path did: the extension runs on
|
|
IronPython 2.7 inside Revit's UI thread, a model call can take a couple of
|
|
minutes, and this script touches no Revit API at all -- it reads a job file,
|
|
an evidence JSON file and (optionally) some PNGs, talks to a local HTTP
|
|
server, and writes a spec file. `vibe.aiextract.launch()` is the IronPython
|
|
side of this boundary: it gathers the paths, writes the job file, and starts
|
|
this script in a visible detached window so its reasoning stays readable.
|
|
|
|
THE RULE THAT MAKES THIS SAFE TO RUN UNSUPERVISED: NEVER OVERWRITE, NEVER
|
|
HALF-MERGE
|
|
--------------------------------------------------------------------------
|
|
The mechanical reader outranks this script on anything it actually matched.
|
|
`apply_ai_fill()` below is the enforcement, and it does not trust the model
|
|
to have followed the prompt's own copy of this rule -- it re-checks every
|
|
field against the spec on disk before accepting a proposed value:
|
|
|
|
* a scalar field is filled only if it is currently `None` (or an empty
|
|
string);
|
|
* a "structured" list -- `strand_groups`, `strand_layout.columns_in`,
|
|
`strand_layout.rows` -- is a single coherent value, and is filled only
|
|
if it is currently empty; a spec with even one entry already there is
|
|
left exactly as it was;
|
|
* `notes` and `unresolved` are collections of independent lines, so new,
|
|
distinct entries are appended rather than gating on emptiness -- but
|
|
nothing already in either list is ever changed or dropped;
|
|
* `confidence` entries are accepted only for paths this pass actually
|
|
just filled, never for a path the model merely commented on.
|
|
|
|
A run that fails for any reason -- dsh unreachable, the session times out,
|
|
the reply cannot be parsed as the agreed JSON shape -- leaves the spec's
|
|
FIELD VALUES exactly as the mechanical reader wrote them and appends exactly
|
|
one line to `unresolved` saying the AI pass did not run and why. There is no
|
|
partial-merge outcome: either every field this pass proposes is validated
|
|
and applied, or none of them are.
|
|
|
|
DSV4F PRICING
|
|
-------------
|
|
DSV4F has an off-peak pricing window the owner otherwise reserves for his own
|
|
manual daytime use of the harness. This script only ever runs from a button
|
|
press inside Revit, which is inherently a daytime, on-peak action -- it is
|
|
never scheduled to run at all, let alone into the off-peak window, so it
|
|
never competes with that reservation.
|
|
|
|
THE DSH WIRE CONTRACT
|
|
----------------------
|
|
dsh serves a local HTTP RPC at `POST http://127.0.0.1:3080/api/<method>`,
|
|
body `{"type": "client-request", "rpcId": <uuid>, "method": "session.create"
|
|
| "session.prompt" | "session.history" | "session.list", "payload": {...}}`,
|
|
with the rpcId echoed back. CONFIRMED against the documented contract
|
|
(owner, 2026-09-01).
|
|
|
|
The RESPONSE envelope's first two levels are CONFIRMED against a REAL dsh
|
|
reply: `session.create` on a live WPA5/sample_job run (owner, 2026-09-01)
|
|
answered `{'type': 'server-response', 'rpcId': ..., 'result': {'ok': True,
|
|
'value': {'sessionId': 'session-5969e01e-...', 'agentPreset': 'standard'}}}`
|
|
-- TWO levels of nesting below `result` (`result.ok`, `result.value.sessionId`),
|
|
not one. The first version of this script read the wrong level
|
|
(`result.sessionId`) and declared `DshError: session.create did not return
|
|
a session id` on a call that had, in fact, fully succeeded server-side --
|
|
the session was live on dsh with nobody in this process able to see it. See
|
|
`_rpc_value()`, which enforces and centralises this exact shape, and
|
|
`cancel_session()` / `run()`'s failure path, added the same day to limit how
|
|
long a session like that stays orphaned once ordinary in-flight failures
|
|
(not this specific parsing bug, which is now fixed) leave one behind.
|
|
|
|
MEASURED LIVE against a real dsh at http://127.0.0.1:3080, 2026-09-03
|
|
(supersedes every earlier guess about `session.prompt` / `session.history`,
|
|
below -- those guesses are what produced the `invalid payload for
|
|
session.prompt` failure on every prior run):
|
|
|
|
1. `session.prompt` payload must be
|
|
`{"sessionId": <id>, "mode": "queue", "content": [{"type": "text",
|
|
"text": <prompt>}]}`. `send_prompt()` sends exactly this now.
|
|
|
|
2. `session.history` payload is `{"sessionId": <id>, "maxMessages": N}` --
|
|
N is an EVENT count, not a message count: the server returns the last N
|
|
raw events, and one turn produces on the order of 20,000 of them, almost
|
|
all `assistant/chunk` streaming deltas. `result.value` is
|
|
`{"events": [{"event": {"type": <str>, "data": {...}}}, ...],
|
|
"hasMore": bool, "projections": {...}}` -- there is NO `messages` key and
|
|
NO `status`/`state` key anywhere in it. Event types seen: `user/message`
|
|
(`data.content` is a list of `{"type": "text", "text": ...}` blocks),
|
|
`assistant/chunk` (streaming deltas -- IGNORED entirely, see
|
|
`_history_messages()`), `assistant/message` (`data.message` is
|
|
`{"role": "assistant", "content": [...]}`, one per completed step of the
|
|
turn -- the LAST one of a turn holds the final answer), `tool/call`,
|
|
`tool/result`, `step/start`, `step/end`, `turn/start`, plus a few config
|
|
events. `_history_messages()` projects the events shape into
|
|
`{"role", "text"}` pairs; it still tolerates the old guessed
|
|
`{"messages": [...]}` shape too, for the same reason `_rpc_value` stays
|
|
permissive elsewhere -- but the events shape is what a live server
|
|
actually sends and takes precedence when both are present.
|
|
|
|
3. `session.list` payload `{}` -> `result.value = {"items": [{"sessionId":
|
|
<id>, "running": bool, "agentPreset": ..., "cwd": ..., "updatedAt": ...,
|
|
"projections": ..., "blank": ...}, ...]}`. `running` is true while a turn
|
|
is in progress and false once it has finished -- THIS is the done signal
|
|
`wait_for_reply()` polls (`session_running()`), not a status field on
|
|
history, because `session.history` carries no such field at all.
|
|
|
|
Every RPC call is funnelled through `_rpc()` / `_rpc_value()` /
|
|
`_http_post_json()` so any further correction stays in one place.
|
|
|
|
`resolve_model()`'s settings.yaml scan WAS confirmed against the real file
|
|
(owner, 2026-09-01): `~/.dsh/settings.yaml` nests a model's `id:` and its
|
|
`input:` list several keys apart inside provider/model list structure --
|
|
`_iter_mappings()` walks the whole parsed tree rather than assuming any
|
|
particular shape above the two keys. See its docstring.
|
|
|
|
4. THE ACTUAL BUG (found 2026-09-03, reading dsh's own source at
|
|
`%APPDATA%/npm/node_modules/@deepseek-ai/dsh/node_modules/@deepseek-ai/
|
|
dsh-host-apiproxy/lib/types/api/sessions.schema.js`, and confirmed on a
|
|
live throwaway session): `session.create`'s payload zod schema
|
|
(`sessionCreateRequestSchema`) is `{workspaceId?, cwd?, sessionId?,
|
|
agentPreset?}` -- there is NO `model` field at all. The `{"model": ...}`
|
|
key this file used to send there was silently stripped by zod's default
|
|
unknown-key handling, so every session inherited whatever
|
|
`agent-default-model` names in settings.yaml (on this machine:
|
|
`commandcode` / `deepseek/deepseek-v4-flash`, no image input) regardless
|
|
of what was asked for -- exactly the failure the task that added this
|
|
section was opened to chase down. A session's model is set by a
|
|
SEPARATE, later RPC: `session.selectModel`, payload `{"sessionId",
|
|
"provider", "model", "reasoningEffort"?}` -> `result.value = {"selected":
|
|
{"provider", "model", "reasoningEffort"?}}` (`sessionSelectModelRequestSchema`
|
|
/ `sessionSelectModelValueSchema`, same file). `provider` is REQUIRED --
|
|
the model catalog's grouping key (`~/.dsh/settings.yaml`'s `providers:`
|
|
mapping, keyed by provider id -- `commandcode` hosts both
|
|
`deepseek/deepseek-v4-flash` and `deepseek/deepseek-v4-flash-vision-exp`
|
|
on this machine), never inferable from the model id string alone. Live
|
|
proof, 2026-09-03: `session.create` -> `session.selectModel({provider:
|
|
"commandcode", model: "deepseek/deepseek-v4-flash-vision-exp"})` ->
|
|
`session.models` immediately echoed that pair back as `current`, and the
|
|
turn's own `request/context` history event (`{"type": "request/context",
|
|
"data": {"provider": "commandcode", "model":
|
|
"deepseek/deepseek-v4-flash-vision-exp", "contextWindow": 1000000}}`) --
|
|
the event that says what the turn ACTUALLY ran on, not merely what was
|
|
asked for -- named that exact pair. `select_model()` sends the RPC;
|
|
`verify_session_model()` re-reads history afterwards and checks that
|
|
event before this file ever trusts a reply, because a request that dsh
|
|
accepted is not proof the turn ran on it -- a silent fallback to the
|
|
account default looks identical at every earlier layer.
|
|
|
|
IronPython does not run this file -- it is plain CPython 3, same rule as
|
|
`tools/rvt.py` and `tools/vision_read.py`.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import argparse
|
|
import copy
|
|
import io
|
|
import json
|
|
import os
|
|
import re
|
|
import sys
|
|
import time
|
|
import types
|
|
import urllib.error
|
|
import urllib.request
|
|
import uuid
|
|
from pathlib import Path
|
|
|
|
|
|
# --------------------------------------------------------------- vibe import
|
|
#
|
|
# `specformat.py` and `specsave.py` are the Revit-free half of the design
|
|
# spec pipeline (see their own docstrings, and CLAUDE.md's `vibe` layout
|
|
# section) -- pure data in, text out -- but `vibe/__init__.py` imports the
|
|
# Revit API at module scope the instant ANY submodule of `vibe` is imported
|
|
# the ordinary way. `WillTools.extension/tests/conftest.py` solves this for
|
|
# pytest by registering a bare package stub before the first import; the
|
|
# same trick, unchanged, lets this standalone script reach `specformat`
|
|
# without a Revit licence or an IronPython interpreter.
|
|
_REPO_ROOT = Path(__file__).resolve().parent.parent
|
|
_VIBE_DIR = _REPO_ROOT / 'WillTools.extension' / 'lib' / 'vibe'
|
|
if 'vibe' not in sys.modules:
|
|
_stub = types.ModuleType('vibe')
|
|
_stub.__path__ = [str(_VIBE_DIR)]
|
|
_stub.__file__ = str(_VIBE_DIR / '__init__.py')
|
|
sys.modules['vibe'] = _stub
|
|
|
|
from vibe import specformat # noqa: E402 (see the stub above)
|
|
|
|
|
|
# ------------------------------------------------------------------- config
|
|
|
|
DEFAULT_DSH_BASE_URL = 'http://127.0.0.1:3080'
|
|
# Vision model on a long prompt + drawing PNG measured well over 180s to
|
|
# produce even a first assistant message on 2026-09-03 -- the old 180s
|
|
# default cancelled a session that was still working, never one that had
|
|
# failed. Keep in lockstep with the identically-named constant in
|
|
# WillTools.extension/lib/vibe/aiextract.py.
|
|
DEFAULT_TIMEOUT_S = 900
|
|
POLL_INTERVAL_S = 3
|
|
MODEL_ENV = 'WILLTOOLS_DSV4F_MODEL'
|
|
# `session.selectModel` requires an explicit provider id alongside the model
|
|
# id (see the module docstring's point 4) -- there is no fallback constant
|
|
# for this the way `FALLBACK_VISION_MODEL` is one for the model, because a
|
|
# wrong provider guess is exactly the silent-mismatch failure this file was
|
|
# rewritten to stop; `resolve_model()` fails closed (`provider is None`)
|
|
# rather than invent one.
|
|
PROVIDER_ENV = 'WILLTOOLS_DSV4F_PROVIDER'
|
|
# Named because the owner named it ("dsv4f带vision") -- not because this
|
|
# script has ever confirmed it against a real settings.yaml. See
|
|
# `resolve_model()` and the module docstring's owner-must-verify note.
|
|
FALLBACK_VISION_MODEL = 'dsv4f-vision'
|
|
|
|
|
|
# ---------------------------------------------------------------- the prompt
|
|
#
|
|
# Every field name below is copied from `specformat.SPEC_TEMPLATE` (via
|
|
# AI_REPLY_TEMPLATE, further down) rather than retyped, for the same reason
|
|
# rule 0 gives: a renamed field is read as absent by every check downstream,
|
|
# and the check then silently does not run. The OLD prompt in
|
|
# `vibe.aiextract` (the opus-CLI path this replaces) had drifted from that
|
|
# template -- it still asked for `spacings_in` / `positions_in` /
|
|
# `strands_per_row`, which stopped existing the day the spec grew the
|
|
# `columns_in` / `rows` matrix shape (commit `73fdd9a`, "spec 改成人类可读
|
|
# YAML + 矩阵 + 去掉 AI") -- because nothing called that prompt any more to
|
|
# notice. This one is built from the live template so it cannot drift the
|
|
# same way without a test noticing (see `test_aiextract_run.py`).
|
|
|
|
_PROMPT_HEADER = u"""You are the SECOND pass of a two-stage precast concrete design-summary
|
|
extractor, running as DSV4F with vision. A mechanical regex/geometry reader
|
|
already made a first pass over this design and wrote everything it could
|
|
match with certainty into the spec below. Your only job is the leftovers:
|
|
whatever that reader left null, empty, or listed under "unresolved" -- and,
|
|
because you have vision and it deliberately does not, the strand matrix,
|
|
which is drawn geometry, not text, and was never something the mechanical
|
|
reader should have guessed at.
|
|
|
|
THE RULE THAT MATTERS MOST: A VALUE ALREADY IN THE SPEC BELOW IS NOT YOURS
|
|
TO TOUCH. The mechanical reader outranks you on anything it actually
|
|
matched. If a field already carries a non-null value, or a list already has
|
|
entries, leave it out of your reply entirely -- even if you would have read
|
|
it differently. Only propose a value for a field that is null, an empty
|
|
list, or named in "unresolved" below. A host-side merge enforces this
|
|
regardless of what you send back, so there is no advantage to repeating or
|
|
"correcting" an already-filled field: your proposal for it is simply
|
|
discarded, and repeating values only wastes your own context.
|
|
"""
|
|
|
|
_PROMPT_RULES = u"""Rules that matter:
|
|
|
|
1. NEVER invent a value. If you cannot place a piece of text confidently,
|
|
add an entry to "unresolved" of the form {{"text": "<verbatim>", "reason":
|
|
"<why you would not encode it>"}}. A missing value becomes a visible
|
|
warning downstream; a wrong value silently passes and could reach the
|
|
plant.
|
|
|
|
2. The min/max envelope table is written as separate text notes: a label
|
|
such as "MAX. LENGTH" and, beside it at nearly the same y, its value such
|
|
as "49'- 3\\"". Pair them by position. Record both the decimal feet
|
|
(min_ft, max_ft) and the original string (min_text, max_text). Only
|
|
propose these if the corresponding field above is still null -- the
|
|
mechanical reader usually gets this one, so most of the time you will
|
|
have nothing to add here.
|
|
|
|
3. There can be MORE THAN ONE strand group. Shielded or de-tensioned
|
|
strands are called out separately, e.g. "(14) 1/2\\" ... @ 31,600 # I.T."
|
|
together with "(4) ... @ 5,000 # I.T.". Emit one object per group and set
|
|
"role" to what the text says it is (e.g. "main", "shielded"), or null if
|
|
unstated. dia_in is a decimal: 1/2 -> 0.5, 9/16 -> 0.5625. Propose the
|
|
WHOLE "strand_groups" list only if it is currently empty -- it is one
|
|
coherent value, not something to patch entry by entry.
|
|
|
|
4. Units: psi for strengths, pcf for unit weight, pounds for I.T., inches
|
|
for diameters and thickness, decimal FEET for the envelope.
|
|
|
|
4a. THE STRAND MATRIX -- the whole point of your having vision. Fill
|
|
reinforcing.strand_layout the same way a detailer would read it off the
|
|
drawing:
|
|
|
|
columns_in -- strand positions ACROSS the piece, as cumulative
|
|
distance in inches from the datum (below), same numbers
|
|
a detailer would read off the dimension chain.
|
|
rows -- one entry per layer through the piece's thickness:
|
|
{{"from_bottom_in": <number>, "strands": [true/false, ...]}}
|
|
-- "strands" has exactly len(columns_in) entries, true
|
|
where that row actually has a strand at that column,
|
|
false where it does not. A design where every column has
|
|
a strand at every row -- the ordinary case -- has
|
|
"strands" all true; a design that is not a full grid
|
|
says so column by column instead of forcing a human to
|
|
reverse-engineer it from a total count.
|
|
total_in -- the chain total; should equal the piece width. Say so in
|
|
"unresolved" if it plainly does not.
|
|
note -- anything about the chain a human should double check.
|
|
|
|
How to read the chain:
|
|
|
|
a. Dimensions at (nearly) the same y are pieces of ONE chain --
|
|
concatenate them left to right in increasing x, even when the
|
|
evidence hands them to you as several separate dimension elements.
|
|
Quote the concatenated chain, not the separate pieces.
|
|
b. One chain per row. A section cut through two strand rows normally
|
|
carries two chains -- the top row's drawn ABOVE the section, the
|
|
bottom row's BELOW it -- while a full grid (every row the same)
|
|
carries a single chain shared by every row. A real chain either
|
|
closes to the far edge (total == piece width) or stops at the last
|
|
strand (total == that strand's position); it does not fall short of
|
|
both.
|
|
c. A horizontal chain whose total is neither the width nor consistent
|
|
with a strand row is NOT a strand chain -- mild bars, blocking
|
|
diagrams and similar callouts can draw their own dimension chain
|
|
nearby. Ignore it rather than forcing it to be a row.
|
|
d. Row depths come from the VERTICAL chain, cumulative from the bottom
|
|
face -- its segments sum to the piece thickness.
|
|
e. Count check before you answer: the strands you place across every
|
|
row MUST equal the stated quantity in the strand group text, and,
|
|
when "symbols" are present in the evidence, the number of strand
|
|
symbols too. If they do not agree, leave the matrix EMPTY and say in
|
|
"unresolved" which count disagreed with which -- do not force a
|
|
placement just to make the arithmetic close.
|
|
f. "symbols" (when the evidence carries them) give the centre of every
|
|
drawn detail component, in view feet -- the drawn strand marks are
|
|
among them. Cluster their y to find the rows, and their x-gaps to
|
|
match against a chain's segments; drafting views are true scale, so
|
|
a 1.0 ft gap between symbols reads as a `1'-0"` segment.
|
|
g. If "unresolved" already carries an entry for "STRAND PATTERN (matrix)",
|
|
read its "reason" FIRST -- it is the mechanical reader's
|
|
own geometric attempt telling you exactly what did not add up (a
|
|
symbol drawn as the wrong kind, a chain that would not close, ...).
|
|
Your job is to resolve THAT disagreement, not to re-derive the
|
|
matrix from scratch and hope it happens to agree.
|
|
|
|
Only propose strand_layout.columns_in / rows / total_in if they are
|
|
CURRENTLY EMPTY -- never propose reinforcing.strand_layout.datum: that is
|
|
plant knowledge (which rail the chain is measured from), not something
|
|
you can read off one drawing, and it is filled in already. The datum in
|
|
force for this design:
|
|
|
|
{datum}
|
|
|
|
Do not change your reading to make the numbers look nicer from the other
|
|
end; if the chain plainly does not start where the datum says, record
|
|
that in "unresolved" and leave the datum alone.
|
|
|
|
QUOTE THE DIMENSION CHAIN VERBATIM in your prose summary below the JSON
|
|
block -- the exact numbers you read off the drawing, in order -- so a
|
|
person can hold your quote next to the drawing and check it without
|
|
redoing your reading.
|
|
|
|
5. Prose instructions that are not a number -- "USE MC STANDARD
|
|
REINFORCEMENT FOR CUT DOWN PANELS", "STRIPPING & BLOCKING DETAIL < 25'"
|
|
-- belong in "notes" as written. Do not try to encode them. Only add
|
|
entries here for lines nothing above already covers.
|
|
|
|
6. Fill "confidence" with an entry for every field you propose a value for
|
|
(and ONLY for fields you propose a value for), mapping the dotted field
|
|
path to "high", "medium" or "low". Use "high" only when the drawing
|
|
states the value unambiguously.
|
|
|
|
7. Never set anything under "source" -- extractor, reviewed_by and the rest
|
|
are the host's to stamp, not yours. Your reply carries none of it.
|
|
"""
|
|
|
|
_PROMPT_OUTPUT = u"""Reply with exactly one fenced ```json code block matching the shape given
|
|
above (omit or leave null anything you are not proposing -- do not resend a
|
|
value already present in the spec you were given), followed by a short
|
|
prose summary: what you filled and why, the strand matrix's dimension chain
|
|
quoted verbatim so a person can check it against the drawing, and anything
|
|
you are adding to "unresolved" and why. Do not ask questions -- if something
|
|
is ambiguous, leave it out of the JSON and say so in the summary and in
|
|
"unresolved".
|
|
"""
|
|
|
|
|
|
def _json_block(value):
|
|
return json.dumps(value, indent=2, sort_keys=True, ensure_ascii=False)
|
|
|
|
|
|
def build_prompt(evidence, regex_spec, datum_clause, image_paths=None):
|
|
"""The exact text sent to the model. Pure -- no Revit, no dsh, no I/O.
|
|
|
|
Kept as one function so the prompt actually sent can be captured and
|
|
reviewed (see `--print-prompt`) rather than only existing implicitly
|
|
inside `run()`.
|
|
"""
|
|
image_paths = list(image_paths or [])
|
|
parts = [_PROMPT_HEADER]
|
|
|
|
parts.append(
|
|
u"\nEVIDENCE (a deterministic scrape of this design's summary view "
|
|
u"and its sheet -- every text note with its X/Y position, every "
|
|
u"dimension with its segments, and one real assembly of this "
|
|
u"design so you can sanity-check magnitudes; nothing in it has "
|
|
u"been interpreted):\n\n" + _json_block(evidence))
|
|
|
|
parts.append(
|
|
u"\nTHE SPEC AS THE MECHANICAL READER LEFT IT (this is what you "
|
|
u"may not overwrite -- see the rule above):\n\n" +
|
|
_json_block(regex_spec))
|
|
|
|
parts.append(
|
|
u"\nTHE SHAPE YOUR REPLY MUST FIT. Reply with a JSON object using "
|
|
u"EXACTLY these field names -- do not rename, do not add fields, "
|
|
u"do not drop fields:\n\n" + _json_block(AI_REPLY_TEMPLATE))
|
|
|
|
parts.append(u"\n" + (_PROMPT_RULES.format(datum=datum_clause)))
|
|
|
|
if image_paths:
|
|
lines = u"\n".join(u" - {0}".format(p) for p in image_paths)
|
|
parts.append(
|
|
u"\nDRAWING IMAGES -- read each with your read_image tool. "
|
|
u"These are file paths on this machine, not embedded bytes:\n\n"
|
|
+ lines +
|
|
u"\n\nUse these for anything the deterministic evidence above "
|
|
u"cannot give you -- above all the strand matrix, which is a "
|
|
u"drawn grid, not text.\n")
|
|
else:
|
|
parts.append(
|
|
u"\nNo drawing image was available for this pass -- text and "
|
|
u"the evidence above only. Do not attempt the strand matrix "
|
|
u"without a drawing to read it from; leave strand_layout empty "
|
|
u"and say why in \"unresolved\" if it matters.\n")
|
|
|
|
parts.append(u"\n" + _PROMPT_OUTPUT)
|
|
return u"\n".join(parts)
|
|
|
|
|
|
# ------------------------------------------------------------ the AI's shape
|
|
#
|
|
# What the model is allowed to propose values for. Deliberately narrower
|
|
# than SPEC_TEMPLATE: no "source", no "design_number", no "schema" -- those
|
|
# are the host's to stamp (see CONTENT_PATHS below, which this mirrors
|
|
# exactly) and never travel over the wire to the model at all.
|
|
|
|
AI_REPLY_TEMPLATE = {
|
|
'piece_description': None,
|
|
'concrete': {'fc_psi': None, 'fci_psi': None,
|
|
'unit_weight_pcf': None, 'weight_class': None},
|
|
'reinforcing': {
|
|
'mild_grade_ksi': None,
|
|
'wwf': {'mesh': None, 'layers': None, 'per_face': None},
|
|
'strand_groups': [], # [{qty, dia_in, grade_ksi, it_lb,
|
|
# role, note}]
|
|
'strand_layout': {
|
|
'columns_in': [], # cumulative in from datum
|
|
'rows': [], # [{from_bottom_in, strands: [bool,..]}]
|
|
'total_in': None,
|
|
'note': None,
|
|
},
|
|
},
|
|
'envelope': {'length': {'min_ft': None, 'max_ft': None,
|
|
'min_text': None, 'max_text': None},
|
|
'width': {'min_ft': None, 'max_ft': None,
|
|
'min_text': None, 'max_text': None}},
|
|
'thickness_in': None,
|
|
'notes': [],
|
|
'unresolved': [],
|
|
'confidence': {},
|
|
}
|
|
|
|
# The scalar / single-structured-value paths the AI may fill -- one path per
|
|
# leaf in AI_REPLY_TEMPLATE except the two list-of-independent-entries
|
|
# fields, which get their own merge rule (see ATOMIC_LIST_PATHS) and
|
|
# "confidence", which is merged separately because it depends on what else
|
|
# was just filled.
|
|
CONTENT_PATHS = (
|
|
'piece_description',
|
|
'concrete.fc_psi', 'concrete.fci_psi',
|
|
'concrete.unit_weight_pcf', 'concrete.weight_class',
|
|
'reinforcing.mild_grade_ksi',
|
|
'reinforcing.wwf.mesh', 'reinforcing.wwf.layers',
|
|
'reinforcing.wwf.per_face',
|
|
'reinforcing.strand_groups',
|
|
'reinforcing.strand_layout.columns_in',
|
|
'reinforcing.strand_layout.rows',
|
|
'reinforcing.strand_layout.total_in',
|
|
'reinforcing.strand_layout.note',
|
|
'envelope.length.min_ft', 'envelope.length.max_ft',
|
|
'envelope.length.min_text', 'envelope.length.max_text',
|
|
'envelope.width.min_ft', 'envelope.width.max_ft',
|
|
'envelope.width.min_text', 'envelope.width.max_text',
|
|
'thickness_in',
|
|
)
|
|
|
|
# These three are each ONE coherent value (a grid, not independent lines):
|
|
# filled only if currently empty, and replaced whole -- never patched
|
|
# entry-by-entry, and never touched at all once anything is there.
|
|
STRUCTURED_LIST_PATHS = (
|
|
'reinforcing.strand_groups',
|
|
'reinforcing.strand_layout.columns_in',
|
|
'reinforcing.strand_layout.rows',
|
|
)
|
|
|
|
# These two are collections of INDEPENDENT lines: new, distinct entries are
|
|
# appended; nothing already present is ever changed or dropped.
|
|
ATOMIC_LIST_PATHS = ('notes', 'unresolved')
|
|
|
|
|
|
# ------------------------------------------------------------- merge logic
|
|
#
|
|
# The part of this file that is actually tested (see test_aiextract_run.py)
|
|
# and the part that makes an otherwise-unsupervised process safe to run:
|
|
# regardless of what the model sends back, nothing here can overwrite a
|
|
# value the mechanical reader placed, and nothing here can produce a
|
|
# half-merged file -- `apply_ai_fill` either fills a field or it does not,
|
|
# there is no partial write of one field.
|
|
|
|
def _is_empty(value):
|
|
if value is None:
|
|
return True
|
|
if isinstance(value, str) and value.strip() == '':
|
|
return True
|
|
if isinstance(value, (list, dict)) and len(value) == 0:
|
|
return True
|
|
return False
|
|
|
|
|
|
def _set_path(tree, dotted, value):
|
|
parts = dotted.split('.')
|
|
node = tree
|
|
for part in parts[:-1]:
|
|
nxt = node.get(part)
|
|
if not isinstance(nxt, dict):
|
|
nxt = {}
|
|
node[part] = nxt
|
|
node = nxt
|
|
node[parts[-1]] = value
|
|
|
|
|
|
def _entry_key(entry):
|
|
"""A dedupe key for one 'unresolved' / 'notes' entry, tolerant of shape.
|
|
|
|
Mirrors `vibe.specsave.unresolved_entries`'s tolerance: an entry may be
|
|
a bare string (an older writer) or a {'text':, 'reason':} dict.
|
|
"""
|
|
if isinstance(entry, dict):
|
|
text = entry.get('text')
|
|
else:
|
|
text = entry
|
|
return re.sub(r'\s+', ' ', (u'{0}'.format(text or ''))).strip().lower()
|
|
|
|
|
|
def _merge_atomic_entries(current, proposed):
|
|
"""New entries from `proposed` not already present in `current`, in order."""
|
|
seen = set(_entry_key(e) for e in (current or []))
|
|
added = []
|
|
for entry in (proposed or []):
|
|
key = _entry_key(entry)
|
|
if key and key not in seen:
|
|
added.append(entry)
|
|
seen.add(key)
|
|
return added
|
|
|
|
|
|
def apply_ai_fill(spec, ai_payload):
|
|
"""Merge one AI proposal into a regex-written spec.
|
|
|
|
Returns (merged_spec, filled_paths) -- a NEW dict; `spec` is untouched,
|
|
which is what lets a failed or bad reply be handed the original spec
|
|
back unmodified. `filled_paths` is the honest record of what actually
|
|
changed (`source.ai_filled` downstream), and it is used again inside
|
|
this function to gate which "confidence" entries are accepted -- a
|
|
confidence rating for a field the model did not just fill is noise, not
|
|
provenance, and is dropped.
|
|
"""
|
|
merged = copy.deepcopy(spec)
|
|
ai_payload = ai_payload if isinstance(ai_payload, dict) else {}
|
|
filled = []
|
|
|
|
for path in CONTENT_PATHS:
|
|
proposed = specformat.get_path(ai_payload, path)
|
|
if proposed is None:
|
|
continue
|
|
if path in STRUCTURED_LIST_PATHS:
|
|
current = specformat.get_path(merged, path, []) or []
|
|
if current:
|
|
continue # already placed -- untouchable
|
|
if not isinstance(proposed, list) or not proposed:
|
|
continue
|
|
else:
|
|
current = specformat.get_path(merged, path)
|
|
if not _is_empty(current):
|
|
continue # already placed -- untouchable
|
|
_set_path(merged, path, proposed)
|
|
filled.append(path)
|
|
|
|
for path in ATOMIC_LIST_PATHS:
|
|
proposed = specformat.get_path(ai_payload, path, [])
|
|
if not proposed:
|
|
continue
|
|
current = specformat.get_path(merged, path, []) or []
|
|
added = _merge_atomic_entries(current, proposed)
|
|
if added:
|
|
_set_path(merged, path, list(current) + added)
|
|
filled.append(path)
|
|
|
|
proposed_conf = ai_payload.get('confidence')
|
|
if isinstance(proposed_conf, dict):
|
|
current_conf = dict(specformat.get_path(merged, 'confidence', {}) or {})
|
|
changed = False
|
|
for field_path, level in proposed_conf.items():
|
|
if field_path in filled and field_path not in current_conf:
|
|
current_conf[field_path] = level
|
|
changed = True
|
|
if changed:
|
|
merged['confidence'] = current_conf
|
|
|
|
return merged, filled
|
|
|
|
|
|
FAILURE_REASON_PREFIX = 'AI pass (dsv4f-vision) did not run'
|
|
|
|
|
|
def apply_failure_note(spec, reason):
|
|
"""The failure outcome: field values untouched, one line in unresolved.
|
|
|
|
Never raises, never drops an existing note, and never adds a second
|
|
line for the exact same reason on a retry (`_entry_key` dedupes by
|
|
normalised text, same as the ordinary unresolved-entry merge).
|
|
"""
|
|
merged = copy.deepcopy(spec)
|
|
note = {'text': FAILURE_REASON_PREFIX, 'reason': reason}
|
|
current = specformat.get_path(merged, 'unresolved', []) or []
|
|
added = _merge_atomic_entries(current, [note])
|
|
if added:
|
|
merged['unresolved'] = list(current) + added
|
|
return merged
|
|
|
|
|
|
# ----------------------------------------------------------- dsh RPC client
|
|
#
|
|
# See the module docstring's "THE DSH WIRE CONTRACT ASSUMED HERE" section --
|
|
# every assumption about the wire format is contained to this section, and
|
|
# every call goes through `_rpc()` / `_http_post_json()` so it can be fixed
|
|
# in one place. Tests fake `transport` (a plain callable), never the
|
|
# network, which is the HTTP boundary the task asked to fake at.
|
|
|
|
class DshError(Exception):
|
|
"""dsh reached, but the RPC failed or answered something unusable."""
|
|
|
|
|
|
class DshTimeout(DshError):
|
|
"""dsh did not answer (connect/read timeout, or no reply within budget)."""
|
|
|
|
|
|
def _http_post_json(url, body, timeout):
|
|
"""The one real network call in this file. Swappable via `transport=`."""
|
|
data = json.dumps(body).encode('utf-8')
|
|
req = urllib.request.Request(
|
|
url, data=data, headers={'Content-Type': 'application/json'},
|
|
method='POST')
|
|
try:
|
|
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
|
raw = resp.read().decode('utf-8', 'replace')
|
|
except urllib.error.HTTPError as exc:
|
|
raw = exc.read().decode('utf-8', 'replace')
|
|
except urllib.error.URLError as exc:
|
|
if isinstance(getattr(exc, 'reason', None), TimeoutError):
|
|
raise DshTimeout('{0}: {1}'.format(url, exc.reason))
|
|
raise DshError('could not reach dsh at {0}: {1}'.format(url, exc.reason))
|
|
except TimeoutError as exc:
|
|
raise DshTimeout('{0}: {1}'.format(url, exc))
|
|
try:
|
|
return json.loads(raw)
|
|
except ValueError:
|
|
raise DshError('dsh returned non-JSON from {0}: {1!r}'
|
|
.format(url, raw[:200]))
|
|
|
|
|
|
def _rpc_value(res, method):
|
|
"""Unwrap a dsh response envelope down to its `result.value` payload,
|
|
raising a DshError that names EXACTLY which layer was missing or the
|
|
wrong shape -- never a silent `None` a caller then has to re-check.
|
|
|
|
CONFIRMED against a REAL dsh reply (owner, WPA5/sample_job run,
|
|
2026-09-01)::
|
|
|
|
{'type': 'server-response', 'rpcId': '3d7ad...',
|
|
'result': {'ok': True,
|
|
'value': {'sessionId': 'session-5969e01e-...',
|
|
'agentPreset': 'standard'}}}
|
|
|
|
That is `session.create`'s envelope, confirmed to this exact depth --
|
|
it is what `create_session` failed to reach before this fix, because the
|
|
code that used to live here (and in `_history_messages` /
|
|
`_history_status`, which each re-derived their own partial version of
|
|
this same unwrap) stopped one level too shallow, at `result.sessionId`
|
|
rather than `result.value.sessionId`. Centralising it here means the
|
|
next shape drift gets fixed in one place instead of three, and every
|
|
caller now sees the same failure mode instead of three different ones.
|
|
|
|
`session.prompt`, `session.history` and `session.list` share this exact
|
|
two-level `result.ok` / `result.value` envelope -- MEASURED against a
|
|
live dsh, 2026-09-03 (see the module docstring's wire-contract section),
|
|
not merely assumed any more.
|
|
"""
|
|
if not isinstance(res, dict):
|
|
raise DshError('dsh {0} response was not a JSON object: {1!r}'
|
|
.format(method, res))
|
|
result = res.get('result')
|
|
if not isinstance(result, dict):
|
|
raise DshError('dsh {0} response had no usable "result" object '
|
|
'(got {1}): {2!r}'
|
|
.format(method, type(result).__name__, res))
|
|
if not result.get('ok'):
|
|
raise DshError('dsh {0} response result.ok was not truthy ({1!r}): '
|
|
'{2!r}'.format(method, result.get('ok'), res))
|
|
value = result.get('value')
|
|
if not isinstance(value, dict):
|
|
raise DshError('dsh {0} response result.value was not an object '
|
|
'(got {1}): {2!r}'
|
|
.format(method, type(value).__name__, res))
|
|
return value
|
|
|
|
|
|
def _rpc(base_url, method, payload, timeout, transport=None):
|
|
"""One `{"type": "client-request", ...}` round trip -> the method's
|
|
`result.value`, ALREADY UNWRAPPED (see `_rpc_value`). Raises DshError on
|
|
any failure: an rpcId mismatch, a top-level `error`, or a `result.value`
|
|
that is missing or the wrong shape.
|
|
|
|
Checks the echoed `rpcId` the same way `tools/rvt.py` checks its own
|
|
correlation id against the bridge -- cheap insurance against exactly the
|
|
class of bug that cost real time there (REQUEST_CORRELATION in that
|
|
file), and free to add here since dsh's contract already carries an id.
|
|
|
|
Every caller gets the unwrapped payload from this ONE place -- see
|
|
`_rpc_value`'s docstring for why that centralisation is the actual fix,
|
|
not just this function's own change.
|
|
"""
|
|
transport = transport or _http_post_json
|
|
rpc_id = uuid.uuid4().hex
|
|
body = {'type': 'client-request', 'rpcId': rpc_id, 'method': method,
|
|
'payload': payload}
|
|
url = base_url.rstrip('/') + '/api/' + method
|
|
res = transport(url, body, timeout)
|
|
if isinstance(res, dict):
|
|
got = res.get('rpcId')
|
|
if got is not None and got != rpc_id:
|
|
raise DshError('dsh answered a different request on {0} '
|
|
'(asked {1}, answered {2})'
|
|
.format(method, rpc_id, got))
|
|
if res.get('error'):
|
|
raise DshError('dsh {0} failed: {1}'.format(method, res['error']))
|
|
return _rpc_value(res, method)
|
|
|
|
|
|
def create_session(base_url, timeout, transport=None):
|
|
"""Ask dsh for a new session; return its id, or raise DshError.
|
|
|
|
Sends an EMPTY payload -- `session.create`'s real schema
|
|
(`sessionCreateRequestSchema`, see the module docstring's point 4) is
|
|
`{workspaceId?, cwd?, sessionId?, agentPreset?}`. There used to be a
|
|
`model` argument here and a `{"model": model}` payload built from it;
|
|
that key has never been part of the schema, so zod silently stripped it
|
|
and every session came up on whatever `agent-default-model` names
|
|
instead. Setting a session's model is `select_model()`'s job now, as a
|
|
separate RPC after this one returns -- see `run()`.
|
|
|
|
`_rpc` has already confirmed `result.ok` and `result.value` by the time
|
|
this runs -- the ONLY thing left to validate here is that `value` itself
|
|
actually carries a usable `sessionId` (a non-empty string), because that
|
|
is the one part of the shape `_rpc_value` cannot check on session.create's
|
|
behalf: it does not know which key inside `value` matters to this
|
|
particular method.
|
|
"""
|
|
value = _rpc(base_url, 'session.create', {}, timeout, transport)
|
|
session_id = value.get('sessionId')
|
|
if not isinstance(session_id, str) or not session_id.strip():
|
|
raise DshError('session.create response value had no usable '
|
|
'"sessionId" (got {0!r}): {1!r}'
|
|
.format(session_id, value))
|
|
return session_id
|
|
|
|
|
|
def select_model(base_url, session_id, provider, model, timeout,
|
|
transport=None):
|
|
"""`session.selectModel` -- the RPC that actually pins a session's
|
|
model. See the module docstring's point 4 for how this was found:
|
|
`session.create` accepts no `model` field at all, so a freshly created
|
|
session keeps running on the account's `agent-default-model` until this
|
|
is called.
|
|
|
|
Payload `{"sessionId", "provider", "model"}` -> `result.value =
|
|
{"selected": {"provider", "model", "reasoningEffort"?}}`
|
|
(`sessionSelectModelRequestSchema` / `sessionSelectModelValueSchema`,
|
|
`dsh-host-apiproxy`). `provider` is required by that schema -- there is
|
|
no default and no way to omit it, which is why `resolve_model()` fails
|
|
closed rather than calling this with a guessed one.
|
|
|
|
Accepting this RPC is NOT proof the turn actually ran on the requested
|
|
model -- see `verify_session_model()`, which `run()` calls afterwards
|
|
to check the session's own history rather than trusting this echo.
|
|
"""
|
|
payload = {'sessionId': session_id, 'provider': provider, 'model': model}
|
|
return _rpc(base_url, 'session.selectModel', payload, timeout, transport)
|
|
|
|
|
|
def cancel_session(base_url, session_id, timeout, transport=None):
|
|
"""Best-effort `session.cancel` -- swallows every error, never raises.
|
|
|
|
ORPHAN-SESSION RISK: `session.create` succeeding server-side is not the
|
|
same event as this script noticing it succeeded -- anything that fails
|
|
AFTER a valid `session_id` comes back (the prompt call, the history
|
|
poll, an unparsable reply) still leaves that session open on the dsh
|
|
server with nothing left in this process to close it, because the whole
|
|
point of `run()`'s failure path is that it gives up and writes the
|
|
documented "left untouched" outcome rather than retrying. `run()` calls
|
|
this from that failure path whenever it already has a trusted
|
|
`session_id` in hand (never a guessed one -- there is nothing to cancel
|
|
when `create_session` itself never returned).
|
|
|
|
This is deliberately the only cleanup attempted. A `session.create` that
|
|
fails before returning an id (the bug this file was fixed for) is not a
|
|
case this function can help with -- the id was never something the
|
|
caller could see, only something visible with hindsight once
|
|
`_rpc_value`'s error message is read; there is nothing here to guess it
|
|
from safely, and guessing wrong would risk cancelling an unrelated
|
|
session. See the module docstring / owner report for that residual risk.
|
|
|
|
Returns True if the cancel RPC came back without raising, False
|
|
otherwise -- purely informational for the log; neither outcome changes
|
|
`run()`'s documented failure result.
|
|
"""
|
|
try:
|
|
_rpc(base_url, 'session.cancel', {'sessionId': session_id}, timeout,
|
|
transport)
|
|
return True
|
|
except Exception: # noqa: BLE001
|
|
return False
|
|
|
|
|
|
def send_prompt(base_url, session_id, text, timeout, transport=None):
|
|
payload = {'sessionId': session_id, 'mode': 'queue',
|
|
'content': [{'type': 'text', 'text': text}]}
|
|
return _rpc(base_url, 'session.prompt', payload, timeout, transport)
|
|
|
|
|
|
def _text_from_content_blocks(blocks, kinds=('text',)):
|
|
"""Join the `text` of every block in `blocks` whose `type` is in `kinds`.
|
|
|
|
Used for both `user/message`'s `data.content` (kinds defaults to just
|
|
"text") and `assistant/message`'s `data.message.content`, where
|
|
"reasoning" blocks sit beside "text" ones and must be dropped -- the
|
|
reasoning is the model's scratch work, never part of the settled reply.
|
|
"""
|
|
parts = []
|
|
for block in (blocks or []):
|
|
if isinstance(block, dict) and block.get('type') in kinds \
|
|
and isinstance(block.get('text'), str):
|
|
parts.append(block['text'])
|
|
return ''.join(parts)
|
|
|
|
|
|
def _history_messages(value):
|
|
"""`value` is what `_rpc(..., 'session.history', ...)` returns -- already
|
|
unwrapped to `result.value` by `_rpc_value`.
|
|
|
|
MEASURED against a live dsh, 2026-09-03 (see the module docstring): the
|
|
real shape is `{"events": [{"event": {"type": ..., "data": {...}}}, ...],
|
|
"hasMore": bool, "projections": {...}}`, not `{"messages": [...]}`. This
|
|
projects that events list down to the two event types that matter --
|
|
`user/message` and `assistant/message` -- as an ordered list of
|
|
`{"role": "user"|"assistant", "text": str}` dicts. Everything else
|
|
(`assistant/chunk` streaming deltas above all -- there are ~20,000 of
|
|
them in one turn's history) is skipped entirely, not just ignored by a
|
|
later filter, so callers never have to know these event types exist.
|
|
|
|
Still tolerates the OLD guessed `{"messages": [...]}` shape (a bare list
|
|
of `{role, content|text}` dicts) so tests written against that guess
|
|
keep passing -- but the events shape takes precedence when both keys
|
|
are present, since that is what a live server actually sends.
|
|
"""
|
|
if isinstance(value, dict) and isinstance(value.get('events'), list):
|
|
messages = []
|
|
for entry in value['events']:
|
|
event = entry.get('event') if isinstance(entry, dict) else None
|
|
if not isinstance(event, dict):
|
|
continue
|
|
etype = event.get('type')
|
|
data = event.get('data') or {}
|
|
if etype == 'user/message':
|
|
text = _text_from_content_blocks(data.get('content'))
|
|
messages.append({'role': 'user', 'text': text})
|
|
elif etype == 'assistant/message':
|
|
message = data.get('message') or {}
|
|
text = _text_from_content_blocks(message.get('content'),
|
|
kinds=('text',))
|
|
messages.append({'role': 'assistant', 'text': text})
|
|
# every other event type (assistant/chunk, tool/call,
|
|
# tool/result, step/*, turn/start, config, ...) is skipped.
|
|
return messages
|
|
if isinstance(value, dict) and isinstance(value.get('messages'), list):
|
|
return [{'role': _message_role(m), 'text': _message_text(m)}
|
|
for m in value['messages']]
|
|
if isinstance(value, list):
|
|
return [{'role': _message_role(m), 'text': _message_text(m)}
|
|
for m in value]
|
|
return []
|
|
|
|
|
|
def _message_role(message):
|
|
return (message.get('role') or message.get('type') or '').lower()
|
|
|
|
|
|
def _message_text(message):
|
|
if isinstance(message.get('text'), str):
|
|
return message['text']
|
|
content = message.get('content')
|
|
if isinstance(content, str):
|
|
return content
|
|
if isinstance(content, list):
|
|
parts = []
|
|
for chunk in content:
|
|
if isinstance(chunk, dict) and isinstance(chunk.get('text'), str):
|
|
parts.append(chunk['text'])
|
|
elif isinstance(chunk, str):
|
|
parts.append(chunk)
|
|
return ''.join(parts)
|
|
return ''
|
|
|
|
|
|
def session_running(base_url, session_id, timeout, transport=None):
|
|
"""Whether `session_id` currently has a turn in progress, per
|
|
`session.list` -- the done-signal `wait_for_reply()` polls.
|
|
|
|
MEASURED against a live dsh, 2026-09-03: `session.list` payload `{}` ->
|
|
`result.value = {"items": [{"sessionId": ..., "running": bool, ...}]}`.
|
|
Returns True/False from the matching item's `running` field, or None if
|
|
the session is not listed at all, or the shape is not what is expected
|
|
-- callers must treat None as "cannot tell", never as False, the same
|
|
"could not judge vs judged and found nothing" discipline as everywhere
|
|
else in this tool.
|
|
"""
|
|
try:
|
|
value = _rpc(base_url, 'session.list', {}, timeout, transport)
|
|
except DshError:
|
|
return None
|
|
items = value.get('items') if isinstance(value, dict) else None
|
|
if not isinstance(items, list):
|
|
return None
|
|
for item in items:
|
|
if isinstance(item, dict) and item.get('sessionId') == session_id:
|
|
running = item.get('running')
|
|
return running if isinstance(running, bool) else None
|
|
return None
|
|
|
|
|
|
def wait_for_reply(base_url, session_id, baseline_count, timeout_s,
|
|
transport=None, sleep=time.sleep, clock=time.time,
|
|
poll_interval=POLL_INTERVAL_S):
|
|
"""The model's final reply text, or raises DshTimeout.
|
|
|
|
Polls `session.history` (asking for enough events -- 50,000 -- to cover
|
|
a whole turn, per the module docstring's measured event-count-not-
|
|
message-count contract) and projects it with `_history_messages`. The
|
|
candidate reply is the text of the LAST assistant message after
|
|
`baseline_count` -- a turn that calls tools produces several
|
|
`assistant/message` events, one per step, and only the final one holds
|
|
the settled answer.
|
|
|
|
Done when `session_running()` says the turn has finished (`running` is
|
|
False) and a non-empty candidate exists. If `session_running()` cannot
|
|
tell (returns None -- session not listed, or an unexpected shape), falls
|
|
back to the old stability check: the same non-empty text on two
|
|
consecutive polls, which tolerates a reply arriving mid-update without
|
|
declaring victory on a still-streaming partial answer.
|
|
"""
|
|
deadline = clock() + timeout_s
|
|
last_text = None
|
|
while clock() < deadline:
|
|
value = _rpc(base_url, 'session.history',
|
|
{'sessionId': session_id, 'maxMessages': 50000},
|
|
min(30, timeout_s), transport)
|
|
messages = _history_messages(value)
|
|
assistant_all = [m for m in messages if m['role'] == 'assistant']
|
|
assistant_msgs = assistant_all[baseline_count:]
|
|
if assistant_msgs:
|
|
text = assistant_msgs[-1]['text']
|
|
running = session_running(base_url, session_id,
|
|
min(30, timeout_s), transport)
|
|
if running is not None:
|
|
if running is False and text:
|
|
return text
|
|
elif text and text == last_text:
|
|
return text
|
|
last_text = text
|
|
sleep(poll_interval)
|
|
raise DshTimeout('no settled assistant reply on session {0} within {1}s'
|
|
.format(session_id, timeout_s))
|
|
|
|
|
|
# ------------------------------------------------------ model verification
|
|
#
|
|
# THE CHECK THIS TASK WAS OPENED TO ADD. See the module docstring's point 4:
|
|
# neither `session.create`'s response nor `session.selectModel`'s own
|
|
# response is proof a turn actually ran on the requested model -- both only
|
|
# echo what was asked for (or, for `session.create`, ignore it silently).
|
|
# The one event that says what a turn ACTUALLY used is `request/context`,
|
|
# emitted into the session's own history once the turn starts. `run()` calls
|
|
# `verify_session_model()` after `wait_for_reply()` settles and BEFORE it
|
|
# ever parses the reply as data -- a wrong-model reply can look exactly like
|
|
# a right-model one (confident, well-formatted, plausible numbers), so this
|
|
# has to gate on the model, never on whether the reply merely parsed.
|
|
|
|
def _extract_context_events(history_value):
|
|
"""Every `request/context` event's `{"provider", "model"}` pair, oldest
|
|
first. `history_value` is already unwrapped to `result.value` (same
|
|
input shape as `_history_messages`) -- MEASURED live, 2026-09-03: the
|
|
event is `{"type": "request/context", "data": {"provider": ...,
|
|
"model": ..., "contextWindow": ...}}`, one per LLM call the turn makes
|
|
(a compaction or a retry can add more than one -- the caller wants the
|
|
LAST).
|
|
"""
|
|
events = history_value.get('events') if isinstance(history_value, dict) \
|
|
else None
|
|
if not isinstance(events, list):
|
|
return []
|
|
out = []
|
|
for entry in events:
|
|
event = entry.get('event') if isinstance(entry, dict) else None
|
|
if not isinstance(event, dict) or event.get('type') != 'request/context':
|
|
continue
|
|
data = event.get('data') or {}
|
|
provider, model = data.get('provider'), data.get('model')
|
|
if isinstance(provider, str) and isinstance(model, str):
|
|
out.append({'provider': provider, 'model': model})
|
|
return out
|
|
|
|
|
|
def verify_session_model(base_url, session_id, expected_provider,
|
|
expected_model, timeout, transport=None):
|
|
"""Raise `DshError` -- naming BOTH the requested and the actual pair --
|
|
unless `session_id` really ran on `expected_provider` / `expected_model`,
|
|
per its own `request/context` history event.
|
|
|
|
Re-fetches `session.history` rather than trusting anything already in
|
|
hand from `wait_for_reply()`'s polling, so this stays correct regardless
|
|
of how that loop settled. Also raises when NO `request/context` event
|
|
exists yet at all -- a session with no such event has not verifiably run
|
|
on anything, and this refuses to treat "cannot tell" as "must be fine"
|
|
the same way the rest of this codebase refuses to collapse "could not
|
|
judge" into a pass (see CLAUDE.md's `knowledge.same_face()` example).
|
|
|
|
Returns `(actual_provider, actual_model)` on success, for the caller to
|
|
log.
|
|
"""
|
|
value = _rpc(base_url, 'session.history',
|
|
{'sessionId': session_id, 'maxMessages': 50000},
|
|
timeout, transport)
|
|
seen = _extract_context_events(value)
|
|
if not seen:
|
|
raise DshError(
|
|
'could not verify session {0}\'s model -- no "request/context" '
|
|
'event in its history yet (requested {1}/{2}); refusing to '
|
|
'trust an unverified session rather than risk a confident '
|
|
'text-only fill from the wrong model'.format(
|
|
session_id, expected_provider, expected_model))
|
|
actual = seen[-1]
|
|
if actual['provider'] != expected_provider or actual['model'] != expected_model:
|
|
raise DshError(
|
|
'session {0} actually ran on {1}/{2}, not the requested {3}/{4} '
|
|
'-- refusing to proceed: a non-vision model filling this prompt '
|
|
'with a confident-looking text-only reply is exactly the silent '
|
|
'failure this check exists to catch'.format(
|
|
session_id, actual['provider'], actual['model'],
|
|
expected_provider, expected_model))
|
|
return actual['provider'], actual['model']
|
|
|
|
|
|
# ------------------------------------------------------- extracting the JSON
|
|
|
|
_JSON_FENCE_RE = re.compile(r'```(?:json)?\s*(\{.*?\})\s*```', re.DOTALL)
|
|
|
|
|
|
def _first_balanced_object(text, start):
|
|
depth = 0
|
|
for i in range(start, len(text)):
|
|
ch = text[i]
|
|
if ch == '{':
|
|
depth += 1
|
|
elif ch == '}':
|
|
depth -= 1
|
|
if depth == 0:
|
|
return text[start:i + 1]
|
|
raise ValueError('unbalanced JSON object in reply')
|
|
|
|
|
|
def extract_json_payload(reply_text):
|
|
"""The model's structured proposal, pulled out of its prose reply.
|
|
|
|
Looks for a fenced ```json block first (the prompt asks for exactly
|
|
one); falls back to the first balanced {...} in the text. Raises
|
|
ValueError -- never returns {} -- when nothing parses, because a reply
|
|
that cannot be read as data must never be silently treated as an empty,
|
|
harmless one: that is exactly the "half-merged" outcome this whole
|
|
module exists to avoid, so it must surface as a failure, not a no-op
|
|
success.
|
|
"""
|
|
reply_text = reply_text or ''
|
|
match = _JSON_FENCE_RE.search(reply_text)
|
|
if match:
|
|
candidate = match.group(1)
|
|
else:
|
|
start = reply_text.find('{')
|
|
if start == -1:
|
|
raise ValueError('no JSON object found in the reply')
|
|
candidate = _first_balanced_object(reply_text, start)
|
|
payload = json.loads(candidate)
|
|
if not isinstance(payload, dict):
|
|
raise ValueError('the reply\'s JSON was not an object: {0!r}'
|
|
.format(type(payload).__name__))
|
|
return payload
|
|
|
|
|
|
# --------------------------------------------------------------- model name
|
|
|
|
def resolve_model(job):
|
|
"""Which model AND which dsh provider to ask for. Fails closed on
|
|
either rather than guessing wrong.
|
|
|
|
Returns `(model, provider, why)`. `session.selectModel` needs BOTH --
|
|
see the module docstring's point 4: `provider` is a required field of
|
|
its own, never inferable from the model id string, and this refuses to
|
|
invent one the way it will (loudly) invent a fallback MODEL -- a wrong
|
|
provider guess is exactly the silent-session-runs-on-something-else
|
|
failure this file was rewritten to stop, and `run()`'s
|
|
`verify_session_model()` check downstream cannot save a session that
|
|
never legitimately requested the right model to begin with.
|
|
|
|
Model order: an explicit job['model'] (aiextract.py can set this once
|
|
the owner has confirmed the real name); the WILLTOOLS_DSV4F_MODEL env
|
|
var; a real parse of ~/.dsh/settings.yaml for a model id whose `input`
|
|
list names "image"; the FALLBACK_VISION_MODEL constant, which is a guess
|
|
and is flagged as one in the returned `why` string every time it is
|
|
used, so it shows up in the log rather than passing quietly.
|
|
|
|
Provider order: an explicit job['provider']; the WILLTOOLS_DSV4F_PROVIDER
|
|
env var; whichever provider the settings.yaml scan above already
|
|
associated with the chosen model (free, when the model came from that
|
|
same scan); failing that, a SEPARATE settings.yaml lookup
|
|
(`_find_provider_for_model_id`) for whatever provider block actually
|
|
lists the resolved model id -- covers an explicit/env model whose
|
|
provider was never looked up by the vision scan. `provider` comes back
|
|
`None`, never a guess, when none of those found one.
|
|
"""
|
|
explicit_model = job.get('model')
|
|
env_model = os.environ.get(MODEL_ENV)
|
|
scanned_provider = None
|
|
if explicit_model:
|
|
model, model_confirmed = explicit_model, True
|
|
elif env_model:
|
|
model, model_confirmed = env_model, True
|
|
else:
|
|
scanned_model, scanned_provider, scan_why = \
|
|
_scan_settings_for_vision_model()
|
|
if scanned_model:
|
|
model, model_confirmed = scanned_model, True
|
|
else:
|
|
model, model_confirmed = FALLBACK_VISION_MODEL, False
|
|
|
|
explicit_provider = job.get('provider')
|
|
env_provider = os.environ.get(PROVIDER_ENV)
|
|
if explicit_provider:
|
|
provider, provider_why = explicit_provider, None
|
|
elif env_provider:
|
|
provider, provider_why = env_provider, None
|
|
elif scanned_provider:
|
|
provider, provider_why = scanned_provider, None
|
|
else:
|
|
provider, provider_why = _find_provider_for_model_id(model)
|
|
|
|
why_parts = []
|
|
if not model_confirmed:
|
|
why_parts.append(
|
|
'could not confirm the vision model name from {0} or {1} ({2}) '
|
|
'-- using the unconfirmed fallback {3!r}. See '
|
|
'aiextract_run.py\'s module docstring.'.format(
|
|
MODEL_ENV, '~/.dsh/settings.yaml', scan_why,
|
|
FALLBACK_VISION_MODEL))
|
|
if not provider:
|
|
why_parts.append(
|
|
'could not determine which dsh provider serves model {0!r} '
|
|
'({1}) -- session.selectModel requires an explicit provider id '
|
|
'and this refuses to guess one. Set {2} explicitly, or {3} to '
|
|
'name a different model whose provider IS in settings.yaml.'
|
|
.format(model, provider_why, PROVIDER_ENV, MODEL_ENV))
|
|
why = ' '.join(why_parts) if why_parts else None
|
|
return model, provider, why
|
|
|
|
|
|
def _iter_mappings(node):
|
|
"""Every dict anywhere in a parsed YAML tree, at any depth.
|
|
|
|
dsh's real settings.yaml nests a model's `id:` and its `input:` list
|
|
several keys apart inside deep, provider-specific structure (measured on
|
|
the owner's machine: `providers: [...] -> models: [...] -> - id: ... /
|
|
(other keys) / input: [text, image]`). Walking every mapping rather than
|
|
assuming a fixed shape above it is what survives that -- the two keys
|
|
only need to be IN THE SAME MAPPING, never adjacent lines.
|
|
"""
|
|
if isinstance(node, dict):
|
|
yield node
|
|
for value in node.values():
|
|
for found in _iter_mappings(value):
|
|
yield found
|
|
elif isinstance(node, list):
|
|
for item in node:
|
|
for found in _iter_mappings(item):
|
|
yield found
|
|
|
|
|
|
def _iter_model_entries(node, provider_id=None):
|
|
"""Every dict that looks like ONE model catalog entry (carries an `id`),
|
|
paired with the provider id it sits under -- `None` when it sits under
|
|
no recognisable provider block at all.
|
|
|
|
A `providers` key names a provider block in the two shapes this file has
|
|
had reason to handle: a MAPPING keyed by provider id (the real
|
|
`~/.dsh/settings.yaml` on this machine, confirmed 2026-09-03:
|
|
`llm-pi-ai: providers: {commandcode: {models: [...]}, ...}` -- the dict
|
|
key IS the provider id `session.selectModel` wants), or a LIST of
|
|
provider objects that name themselves via their own `id` or `name` field
|
|
(an earlier guessed shape some fixtures here still use). Either way,
|
|
once a provider block is found everything nested under it -- at any
|
|
depth, same `_iter_mappings` tolerance -- is tagged with that provider id
|
|
until a closer `providers` block overrides it.
|
|
"""
|
|
if isinstance(node, dict):
|
|
providers = node.get('providers')
|
|
if isinstance(providers, dict):
|
|
for pid, pconf in providers.items():
|
|
for found in _iter_model_entries(pconf, provider_id=pid):
|
|
yield found
|
|
elif isinstance(providers, list):
|
|
for item in providers:
|
|
if isinstance(item, dict):
|
|
pid = item.get('id') or item.get('name')
|
|
for found in _iter_model_entries(item, provider_id=pid):
|
|
yield found
|
|
for key, value in node.items():
|
|
if key == 'providers':
|
|
continue # already walked, above
|
|
for found in _iter_model_entries(value, provider_id=provider_id):
|
|
yield found
|
|
if isinstance(node.get('id'), str) and node.get('id').strip():
|
|
yield provider_id, node
|
|
elif isinstance(node, list):
|
|
for item in node:
|
|
for found in _iter_model_entries(item, provider_id=provider_id):
|
|
yield found
|
|
|
|
|
|
def _declares_image_input(mapping):
|
|
value = mapping.get('input')
|
|
if not isinstance(value, list):
|
|
return False
|
|
return any(isinstance(v, str) and v.strip().lower() == 'image'
|
|
for v in value)
|
|
|
|
|
|
def _load_settings_yaml(settings_path=None):
|
|
"""Parse ~/.dsh/settings.yaml (or `settings_path`); `(data, None)` or
|
|
`(None, why)` -- shared by both settings.yaml lookups below so a missing
|
|
file, a missing PyYAML, or a YAML syntax error says the same thing in
|
|
both places rather than being spelled out twice.
|
|
"""
|
|
path = settings_path or os.path.join(
|
|
os.path.expanduser('~'), '.dsh', 'settings.yaml')
|
|
if not os.path.isfile(path):
|
|
return None, 'no settings file at {0}'.format(path)
|
|
try:
|
|
import yaml
|
|
except ImportError as exc:
|
|
return None, ('PyYAML is not installed ({0}) -- see requirements.txt'
|
|
.format(exc))
|
|
try:
|
|
with io.open(path, 'r', encoding='utf-8') as fh:
|
|
return yaml.safe_load(fh), None
|
|
except Exception as exc: # noqa: BLE001
|
|
return None, 'could not parse {0}: {1}'.format(path, exc)
|
|
|
|
|
|
def _scan_settings_for_vision_model(settings_path=None):
|
|
"""The (model id, provider id) in ~/.dsh/settings.yaml whose `input`
|
|
list names "image" -- a REAL `yaml.safe_load`, not a line-scan.
|
|
|
|
PyYAML is a host-CPython dependency of this repo already (see
|
|
requirements.txt), so there is no reason to approximate parsing a YAML
|
|
file this script can just parse. `_iter_model_entries` finds every
|
|
mapping in the tree that carries BOTH an `id` and an `input`, tagged
|
|
with whichever provider block it sits under; nothing here assumes which
|
|
key of the surrounding structure holds the model list, only that `id`
|
|
and `input` live in the same mapping, which is the one thing dsh's
|
|
settings.yaml is stable about across its otherwise-nested shape.
|
|
|
|
When more than one entry declares image input, one whose id contains
|
|
"vision" wins without asking -- the owner's own naming convention
|
|
("dsv4f带vision", `deepseek-v4-flash-vision-exp`) makes that safe.
|
|
Otherwise this refuses to guess and lists every candidate id, the same
|
|
discipline `vibe.joblocate` uses when more than one job folder qualifies
|
|
and nothing breaks the tie -- `provider` comes back `None` alongside the
|
|
refusal, same as the model does.
|
|
|
|
Returns `(model_id, provider_id, why)` -- `why` is `None` only when
|
|
`model_id` was found unambiguously; `provider_id` may still be `None`
|
|
even then, if that particular entry sat outside any recognisable
|
|
`providers` block (see `_iter_model_entries`).
|
|
"""
|
|
path = settings_path or os.path.join(
|
|
os.path.expanduser('~'), '.dsh', 'settings.yaml')
|
|
data, load_why = _load_settings_yaml(settings_path)
|
|
if load_why:
|
|
return None, None, load_why
|
|
|
|
candidates = []
|
|
providers_by_id = {}
|
|
seen = set()
|
|
for provider_id, mapping in _iter_model_entries(data):
|
|
model_id = mapping.get('id')
|
|
if not isinstance(model_id, str) or not model_id.strip():
|
|
continue
|
|
if not _declares_image_input(mapping):
|
|
continue
|
|
if model_id not in seen:
|
|
candidates.append(model_id)
|
|
providers_by_id[model_id] = provider_id
|
|
seen.add(model_id)
|
|
|
|
if not candidates:
|
|
return None, None, ('no entry in {0} has an id with an input: list '
|
|
'naming "image"'.format(path))
|
|
if len(candidates) == 1:
|
|
return candidates[0], providers_by_id[candidates[0]], None
|
|
|
|
vision_named = [c for c in candidates if 'vision' in c.lower()]
|
|
if len(vision_named) == 1:
|
|
return vision_named[0], providers_by_id[vision_named[0]], None
|
|
|
|
pool = vision_named or candidates
|
|
return None, None, (
|
|
'{0} image-capable model id(s) in {1} and none is distinguishable '
|
|
'by name ({2}) -- nothing here guesses. Name one explicitly via {3}.'
|
|
.format(len(pool), path, ', '.join(pool), MODEL_ENV))
|
|
|
|
|
|
def _find_provider_for_model_id(model_id, settings_path=None):
|
|
"""Which provider block in ~/.dsh/settings.yaml lists `model_id` --
|
|
for a model that came from an explicit job/env override rather than the
|
|
vision scan above, which is the one case `resolve_model()` has a model
|
|
id in hand but no provider tag that came free with it.
|
|
|
|
Deliberately does not care whether the entry declares image input --
|
|
unlike `_scan_settings_for_vision_model`, the caller already knows which
|
|
model it wants; this only answers "who serves it". Returns
|
|
`(provider_id, None)` on the first match, or `(None, why)` -- never
|
|
guesses between two providers that happen to list the same id, the
|
|
same refuse-the-tie discipline as everywhere else in this file; `why`
|
|
names the id it could not place.
|
|
"""
|
|
path = settings_path or os.path.join(
|
|
os.path.expanduser('~'), '.dsh', 'settings.yaml')
|
|
data, load_why = _load_settings_yaml(settings_path)
|
|
if load_why:
|
|
return None, load_why
|
|
|
|
found = set()
|
|
for provider_id, mapping in _iter_model_entries(data):
|
|
if mapping.get('id') == model_id and provider_id:
|
|
found.add(provider_id)
|
|
|
|
if not found:
|
|
return None, 'no entry for model id {0!r} in {1}'.format(
|
|
model_id, path)
|
|
if len(found) > 1:
|
|
return None, ('model id {0!r} is listed under more than one '
|
|
'provider in {1} ({2}) -- nothing here guesses'
|
|
.format(model_id, path, ', '.join(sorted(found))))
|
|
return next(iter(found)), None
|
|
|
|
|
|
# ------------------------------------------------------------------ file IO
|
|
|
|
def _read_text(path):
|
|
with io.open(path, 'r', encoding='utf-8') as fh:
|
|
return fh.read()
|
|
|
|
|
|
def _read_json(path):
|
|
with io.open(path, 'r', encoding='utf-8') as fh:
|
|
return json.load(fh)
|
|
|
|
|
|
def _write_spec(spec_path, spec):
|
|
text = specformat.spec_text(spec)
|
|
with io.open(spec_path, 'w', encoding='utf-8') as fh:
|
|
fh.write(text)
|
|
|
|
|
|
# --------------------------------------------------------------------- run
|
|
|
|
def run(job, transport=None, sleep=time.sleep, clock=time.time, log=None):
|
|
"""The whole AI pass for one design. Never raises for an ordinary
|
|
failure (unreachable dsh, timeout, unparsable reply) -- those become
|
|
the documented "left untouched, note added" outcome. Only a bad job
|
|
file (unreadable spec/evidence path) escapes, because no dsh behaviour
|
|
could produce that and it means this script was invoked wrong.
|
|
|
|
Returns a small dict describing what happened, for the caller's log.
|
|
"""
|
|
log = log if log is not None else (lambda *_a: None)
|
|
spec_path = job['spec_path']
|
|
original_text = _read_text(spec_path)
|
|
spec = specformat.normalize_spec(specformat.parse_spec_text(original_text))
|
|
|
|
# Read up front, outside the try below -- these are plain `job.get()`
|
|
# calls that cannot themselves fail, and the except block needs them
|
|
# (for the orphan-session cleanup call) regardless of which line inside
|
|
# the try raised, including one that runs before this point used to.
|
|
base_url = job.get('dsh_base_url') or DEFAULT_DSH_BASE_URL
|
|
timeout_s = int(job.get('timeout_s') or DEFAULT_TIMEOUT_S)
|
|
|
|
session_id = None
|
|
try:
|
|
evidence = _read_json(job['evidence_path'])
|
|
model, provider, model_why = resolve_model(job)
|
|
if model_why:
|
|
log(model_why)
|
|
if not provider:
|
|
# Fail BEFORE spending a session on it -- see the module
|
|
# docstring's point 4: `session.selectModel` requires an
|
|
# explicit provider id and this refuses to guess one, so there
|
|
# is nothing a session could correctly be pinned to yet.
|
|
raise DshError(
|
|
'no dsh provider resolved for model {0!r} -- '
|
|
'session.selectModel requires one and this pass will not '
|
|
'guess (see the note above)'.format(model))
|
|
image_paths = job.get('image_paths') or []
|
|
|
|
prompt = build_prompt(evidence=evidence, regex_spec=spec,
|
|
datum_clause=job.get('datum_clause') or
|
|
'(no datum recorded -- treat any strand '
|
|
'reading as unresolved rather than guessing '
|
|
'which rail it is measured from)',
|
|
image_paths=image_paths)
|
|
log('--- prompt ---\n' + prompt + '\n--- end prompt ---')
|
|
|
|
session_id = create_session(base_url, timeout_s, transport)
|
|
select_model(base_url, session_id, provider, model, timeout_s,
|
|
transport)
|
|
log('session: {0} (requested model {1}/{2})'.format(
|
|
session_id, provider, model))
|
|
before = _rpc(base_url, 'session.history',
|
|
{'sessionId': session_id, 'maxMessages': 50000},
|
|
timeout_s, transport)
|
|
baseline = len([m for m in _history_messages(before)
|
|
if m['role'] == 'assistant'])
|
|
send_prompt(base_url, session_id, prompt, timeout_s, transport)
|
|
reply_text = wait_for_reply(base_url, session_id, baseline, timeout_s,
|
|
transport, sleep=sleep, clock=clock)
|
|
log('--- reply ---\n' + reply_text + '\n--- end reply ---')
|
|
actual_provider, actual_model = verify_session_model(
|
|
base_url, session_id, provider, model, timeout_s, transport)
|
|
log('confirmed session {0} ran on {1}/{2}'.format(
|
|
session_id, actual_provider, actual_model))
|
|
payload = extract_json_payload(reply_text)
|
|
except Exception as exc: # noqa: BLE001
|
|
reason = '{0}: {1}'.format(type(exc).__name__, exc)
|
|
log('AI pass failed: ' + reason)
|
|
if session_id:
|
|
# A session was created successfully before something LATER in
|
|
# this same pass failed -- see `cancel_session`'s docstring for
|
|
# why this is the one orphan-cleanup case this script can
|
|
# attempt safely (a trusted id already in hand, not a guessed
|
|
# one). Best-effort: its own failure never masks `reason` above.
|
|
cancelled = cancel_session(base_url, session_id, timeout_s,
|
|
transport)
|
|
log('session {0}: {1} after the failure above'.format(
|
|
session_id, 'cancelled' if cancelled else
|
|
'could not be cancelled -- may be an orphan on the dsh '
|
|
'server'))
|
|
_write_spec(spec_path, apply_failure_note(spec, reason))
|
|
return {'ok': False, 'error': reason}
|
|
|
|
merged, filled = apply_ai_fill(spec, payload)
|
|
merged['source']['extractor'] = 'dsv4f-vision'
|
|
merged['source']['ai_filled'] = filled
|
|
_write_spec(spec_path, merged)
|
|
log('filled: {0}'.format(', '.join(filled) if filled else '(nothing new)'))
|
|
return {'ok': True, 'filled': filled}
|
|
|
|
|
|
# --------------------------------------------------------------------- CLI
|
|
|
|
def main(argv=None):
|
|
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
|
ap.add_argument('--job', required=True,
|
|
help='path to the job JSON written by vibe.aiextract')
|
|
ap.add_argument('--print-prompt', action='store_true',
|
|
help='print the prompt that would be sent, and exit '
|
|
'-- no dsh call, no file written')
|
|
args = ap.parse_args(argv)
|
|
|
|
with io.open(args.job, 'r', encoding='utf-8') as fh:
|
|
job = json.load(fh)
|
|
|
|
log_lines = []
|
|
|
|
def log(msg):
|
|
log_lines.append(msg)
|
|
print(msg)
|
|
|
|
if args.print_prompt:
|
|
spec = specformat.normalize_spec(
|
|
specformat.parse_spec_text(_read_text(job['spec_path'])))
|
|
evidence = _read_json(job['evidence_path'])
|
|
print(build_prompt(evidence=evidence, regex_spec=spec,
|
|
datum_clause=job.get('datum_clause') or '',
|
|
image_paths=job.get('image_paths') or []))
|
|
return 0
|
|
|
|
result = run(job, log=log)
|
|
log_path = job.get('log_path')
|
|
if log_path:
|
|
try:
|
|
with io.open(log_path, 'a', encoding='utf-8') as fh:
|
|
fh.write(u'\n'.join(log_lines) + u'\n')
|
|
except OSError:
|
|
pass
|
|
print(json.dumps(result, indent=2))
|
|
return 0 if result.get('ok') else 1
|
|
|
|
|
|
if __name__ == '__main__':
|
|
raise SystemExit(main())
|