Public demo snapshot of an in-house pyRevit extension for precast concrete shop-ticket QC (37 checks / 46 checklist rows) and a standalone panel handling solver (PCI Chapter 8). - No client, job, plant or product data: knowledge layers ship as structural templates only. - Ticket QC's model-read escalation targets a LAN model server endpoint (configurable), not a cloud provider. - Built from the private development repo via tools/make_dist.py.
This commit is contained in:
Binary file not shown.
@@ -0,0 +1,777 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Is a frozen job deck still what CURRENT knowledge would freeze today?
|
||||
|
||||
python tools/deckcheck.py JOBS_DIR
|
||||
python tools/deckcheck.py JOBS_DIR --json OUT.json
|
||||
python tools/deckcheck.py JOBS_DIR --job-root DIR # skip the auto-guess
|
||||
|
||||
JOBS_DIR is a sweep's `jobs/` folder -- `*.job.json` files, each one panel's
|
||||
complete frozen input deck (`docs/lift-check.md`, `docs/plan-panellift-
|
||||
three-stage.md` SS5). Job files are input decks on purpose: a deck must
|
||||
reproduce its own result forever, so `liftexport.py` freezes every knowledge-
|
||||
derived value into it at export and `panellift` never re-reads the catalogue.
|
||||
That is correct, and it is not what this tool checks.
|
||||
|
||||
WHAT WENT WRONG, TWICE, ON 2026-09-01
|
||||
--------------------------------------
|
||||
1. A capacity in `knowledge/catalogue.yaml` was corrected after 282 decks had
|
||||
already been exported carrying the old number. Five real panels showed
|
||||
FAIL at 113% off the stale value before anyone could say why, and
|
||||
explaining it took a git-history dig across two repositories.
|
||||
2. A frozen `panel.design_number` was read as CURRENT by an agent answering
|
||||
the owner, and the model had moved since the export.
|
||||
|
||||
Freezing was never the defect. The defect is that staleness was
|
||||
UNDETECTABLE -- nothing recorded which knowledge version froze a value, and
|
||||
nothing could tell a live deck from a fossil. This tool detects it, for the
|
||||
knowledge half only (point 4 below); a full-fidelity version, with a sha per
|
||||
layer folded into every future deck, is designed in
|
||||
`docs/plan-panellift-three-stage.md` SS5 and is not built here.
|
||||
|
||||
WHAT IT DOES
|
||||
------------
|
||||
For every `*.job.json` in JOBS_DIR, it re-runs the SAME knowledge lookups
|
||||
`liftexport.py` ran at export time -- `knowledge.lifter_capacity()`,
|
||||
`lifter_basis_fc()`, `lifter_edge_distance()`, `lifter_safety_factor()`, and
|
||||
the stage-level `handling.tension_limit_sqrt_fc` -- against knowledge
|
||||
re-resolved from the files ON DISK RIGHT NOW, and compares the fresh answer
|
||||
to the number the deck already carries. A field is judged only when the deck
|
||||
actually carries it (an older deck missing a field is silently not compared
|
||||
on that field, never treated as a match).
|
||||
|
||||
Four outcomes, never collapsed into each other:
|
||||
|
||||
current the deck's frozen value is what current knowledge gives too
|
||||
stale they disagree -- named: mark, field, frozen value, current
|
||||
value, and (when `lifter_capacity`'s own `why` names one)
|
||||
which layer moved
|
||||
conditional `lifter_capacity`'s with-tension-bar branch depends on
|
||||
`assembly_families` -- every OTHER member's family name on
|
||||
the same assembly, which `liftexport.py` reads off the
|
||||
live model and NEVER WRITES INTO THE DECK (see its own
|
||||
comment on this: the basis/why strings ride along
|
||||
specifically so this information is not lost, and the
|
||||
gate's corroborating evidence is lost anyway). So when the
|
||||
family is gated and the deck cannot settle which side of
|
||||
the gate applies, this reports BOTH values the current
|
||||
catalogue could give and does not guess which one this
|
||||
piece would get. This is not a shortfall in this tool; it
|
||||
is a real gap in what the deck records, named rather than
|
||||
papered over.
|
||||
unjudgeable current knowledge cannot resolve the field at all (no
|
||||
catalogue alias, no matching stage spec, ...) -- named,
|
||||
with why. An unjudgeable deck is NOT a passing deck; the
|
||||
summary counts it apart from `current` for exactly that
|
||||
reason (see CLAUDE.md SS3, "could not judge" and "judged,
|
||||
found nothing" are different answers).
|
||||
|
||||
WHAT IT DOES NOT DO
|
||||
--------------------
|
||||
This checks a deck against KNOWLEDGE, not against the MODEL. A deck can pass
|
||||
every check here and still describe a panel whose geometry has since changed
|
||||
in Revit -- design numbers, lifter positions, strand layouts. Verifying that
|
||||
half needs the live model and is out of scope offline; incident 2 above is
|
||||
that half, and it is not what this tool answers. The caveat prints once, at
|
||||
the top of every run, so it cannot be scrolled past.
|
||||
|
||||
WHY THIS DOES NOT CALL `knowledge.load(doc)`
|
||||
---------------------------------------------
|
||||
`load()` takes a live Revit `doc` and its file layers merge through
|
||||
`knowledge._merge_file`, which reaches for `pyrevit.coreutils.yaml` -- a
|
||||
reader that exists only inside Revit. Off Revit that import fails on every
|
||||
layer, `_merge_file` records each one "unreadable" and returns the tree
|
||||
unmerged, and every lookup would silently run against bare `DEFAULTS`. This
|
||||
tool is the same substitution `tools/knowledge_drift.py` documents making,
|
||||
for the same reason: `build_knowledge()` below re-implements the identical
|
||||
five-layer order and the identical `job.plant` two-pass `load()`'s own
|
||||
docstring insists on, reading each YAML file with PyYAML instead and reusing
|
||||
`knowledge._coerce` / `knowledge._deep_update` / `knowledge.DEFAULTS`
|
||||
directly -- so a coercion rule fixed in `knowledge.py` (the leading-zero
|
||||
label rule, for one) is not a second bug waiting to happen here.
|
||||
|
||||
CONSTRAINTS THIS RESPECTS
|
||||
--------------------------
|
||||
Host CPython, no Revit, standalone. Imports `vibe.knowledge` and
|
||||
`vibe.plantpath` the way `tools/ticket_delta.py` / `tools/snapshot_defer.py`
|
||||
do -- see the stub registration below, and
|
||||
`WillTools.extension/tests/conftest.py`'s docstring for why a plain
|
||||
`import vibe.knowledge` cannot work off Revit.
|
||||
|
||||
Prints no plant or client name of its own. Family names, marks and plant ids
|
||||
read out of a job file are DATA flowing through, exactly as `sweep.py` and
|
||||
`knowledge_drift.py` already print them at runtime; nothing about that is
|
||||
baked into this file's own strings. See `.scrub-terms` and CLAUDE.md SS "No
|
||||
client or job information in the repo" -- design numbers, marks and part
|
||||
codes are not client information.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import io
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import types
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
REPO = os.path.abspath(os.path.join(HERE, os.pardir))
|
||||
LIB = os.path.join(REPO, 'WillTools.extension', 'lib')
|
||||
if LIB not in sys.path:
|
||||
sys.path.insert(0, LIB)
|
||||
|
||||
# Same trick `tools/ticket_delta.py`, `tools/snapshot_defer.py` and
|
||||
# `WillTools.extension/tests/conftest.py` all use, and for the same reason:
|
||||
# `vibe/__init__.py` imports the Revit API at module scope, which is right
|
||||
# inside Revit and fatal on host CPython. Register a bare `vibe` package
|
||||
# pointed at the directory so the Revit-free submodules import normally and
|
||||
# `__init__.py` never runs.
|
||||
if 'vibe' not in sys.modules:
|
||||
_stub = types.ModuleType('vibe')
|
||||
_stub.__path__ = [os.path.join(LIB, 'vibe')]
|
||||
sys.modules['vibe'] = _stub
|
||||
|
||||
from vibe import knowledge # noqa: E402
|
||||
from vibe import plantpath # noqa: E402
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError: # pragma: no cover
|
||||
sys.stderr.write('PyYAML is required: pip install pyyaml\n')
|
||||
raise SystemExit(2)
|
||||
|
||||
|
||||
CURRENT = 'current'
|
||||
STALE = 'stale'
|
||||
CONDITIONAL = 'conditional'
|
||||
UNJUDGEABLE = 'unjudgeable'
|
||||
|
||||
# Worse first. A deck's overall status is the worst of its findings' -- the
|
||||
# same "not-judged-is-not-passed" rule as everywhere else in this codebase:
|
||||
# an unjudged deck is not a clean deck, and a conditional one is not either.
|
||||
_SEVERITY = {STALE: 3, CONDITIONAL: 2, UNJUDGEABLE: 1, CURRENT: 0}
|
||||
|
||||
|
||||
# ------------------------------------------------------------- knowledge
|
||||
|
||||
class KnowledgeLayers(object):
|
||||
"""What `build_knowledge` resolved, and what it found on disk.
|
||||
|
||||
Kept apart from the merged dict itself so a report can name which
|
||||
layer's file is newest, and which layers were absent or unreadable,
|
||||
without re-deriving any path a second time.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.defaults_path = None
|
||||
self.catalogue_path = None
|
||||
self.catalogue_status = None
|
||||
self.plant_id = None
|
||||
self.plant_path = None
|
||||
self.plant_status = None
|
||||
self.job_path = None
|
||||
self.job_status = None
|
||||
self.config_path = None
|
||||
self.config_status = None
|
||||
|
||||
def rows(self):
|
||||
"""(label, path, status) for every layer that names a path."""
|
||||
return [
|
||||
('defaults', self.defaults_path, 'source'),
|
||||
('catalogue', self.catalogue_path, self.catalogue_status),
|
||||
('plant', self.plant_path, self.plant_status),
|
||||
('job', self.job_path, self.job_status),
|
||||
('config', self.config_path, self.config_status),
|
||||
]
|
||||
|
||||
def newest_mtime(self):
|
||||
best = None
|
||||
for _label, path, status in self.rows():
|
||||
if not path or status in ('absent', 'missing', None):
|
||||
continue
|
||||
if not os.path.isfile(path):
|
||||
continue
|
||||
m = os.path.getmtime(path)
|
||||
if best is None or m > best:
|
||||
best = m
|
||||
return best
|
||||
|
||||
|
||||
def _read_yaml(path):
|
||||
with io.open(path, 'r', encoding='utf-8') as fh:
|
||||
return yaml.safe_load(fh)
|
||||
|
||||
|
||||
def _merge_layer(merged, path):
|
||||
"""Merge one YAML layer file, PyYAML instead of pyRevit's reader.
|
||||
|
||||
Mirrors `knowledge._merge_file`'s contract: a file that is not there is
|
||||
ordinary and silent ('absent'); a file that IS there and will not parse
|
||||
is recorded and skipped WHOLE, never partially merged ('unreadable') --
|
||||
the same discipline `_merge_file`'s own docstring explains, because a
|
||||
layer that fails to load must never look like a layer that agrees with
|
||||
the defaults.
|
||||
"""
|
||||
if not path or not os.path.isfile(path):
|
||||
return 'absent'
|
||||
try:
|
||||
data = _read_yaml(path)
|
||||
except Exception:
|
||||
return 'unreadable'
|
||||
if isinstance(data, dict):
|
||||
knowledge._deep_update(merged, knowledge._coerce(data))
|
||||
return 'merged'
|
||||
|
||||
|
||||
def build_knowledge(job_root):
|
||||
"""Re-resolve knowledge the way `knowledge.load(doc)` does, no `doc`.
|
||||
|
||||
Same five layers, same order, same `job.plant` two-pass `load()`'s own
|
||||
docstring insists cannot be collapsed into one pass (the plant file has
|
||||
to be read from the name the job file states, and the job file merges
|
||||
AFTER the plant layer -- so the name is read on its own, first, exactly
|
||||
as `knowledge.plant_id()` does for the live-doc path).
|
||||
|
||||
`job_root` is the folder that holds the job's OWN `knowledge.yaml` --
|
||||
for a sweep folder `<job_root>/sweep/jobs/*.job.json`, that is
|
||||
`<job_root>/knowledge.yaml`, the same relationship
|
||||
`designspec.root_dir()` / `sub_dir(doc, 'sweep')` establish for the live
|
||||
path. See `_guess_job_root` for how the CLI finds it when not told.
|
||||
|
||||
The project config layer (last of the five) is included only in the
|
||||
thin sense the rest of this tool needs: unwrap `knowledge:` if present,
|
||||
else merge the file whole. The `decisions:` friendly-name mapping
|
||||
`knowledge._merge_decisions` does for a live project config is NOT
|
||||
replicated here -- no active-config.txt exists on the reference sweep
|
||||
this was built against, and a config layer written entirely under
|
||||
`decisions:` would be under-read by this tool. Named here rather than
|
||||
silently: if that ever matters, `_merge_config_file`'s mapping needs a
|
||||
host-side twin, not a guess bolted on here.
|
||||
"""
|
||||
layers = KnowledgeLayers()
|
||||
merged = knowledge._deep_copy(knowledge.DEFAULTS)
|
||||
layers.defaults_path = os.path.abspath(knowledge.__file__)
|
||||
|
||||
layers.catalogue_path = knowledge.catalogue_path()
|
||||
layers.catalogue_status = _merge_layer(merged, layers.catalogue_path)
|
||||
|
||||
job_path = os.path.join(job_root, 'knowledge.yaml') if job_root else None
|
||||
layers.job_path = job_path
|
||||
|
||||
plant = None
|
||||
if job_path and os.path.isfile(job_path):
|
||||
try:
|
||||
raw = _read_yaml(job_path) or {}
|
||||
except Exception:
|
||||
raw = {}
|
||||
job_block = raw.get('job') if isinstance(raw, dict) else None
|
||||
if isinstance(job_block, dict) and job_block.get('plant') is not None:
|
||||
plant = str(job_block['plant']).strip() or None
|
||||
layers.plant_id = plant
|
||||
|
||||
if plant is not None and not plantpath.valid_plant_id(plant):
|
||||
layers.plant_status = 'invalid_plant_id'
|
||||
else:
|
||||
plant_path = plantpath.plant_knowledge_path(plant)
|
||||
layers.plant_path = plant_path
|
||||
if plant is not None and not (plant_path and os.path.isfile(plant_path)):
|
||||
# Same discipline as `knowledge.load()`'s `_plant_missing`:
|
||||
# NEVER fall back to the machine-wide file when a plant was
|
||||
# named and its folder is not there.
|
||||
layers.plant_status = 'missing'
|
||||
else:
|
||||
layers.plant_status = _merge_layer(merged, plant_path)
|
||||
|
||||
layers.job_status = _merge_layer(merged, job_path) if job_path else 'absent'
|
||||
|
||||
if job_root:
|
||||
pointer = os.path.join(job_root, 'active-config.txt')
|
||||
if os.path.isfile(pointer):
|
||||
with io.open(pointer, 'r', encoding='utf-8') as fh:
|
||||
config_path = fh.read().strip()
|
||||
layers.config_path = config_path or None
|
||||
if config_path:
|
||||
if os.path.isfile(config_path):
|
||||
try:
|
||||
holder = _read_yaml(config_path) or {}
|
||||
except Exception:
|
||||
layers.config_status = 'unreadable'
|
||||
else:
|
||||
body = holder.get('knowledge')
|
||||
body = body if isinstance(body, dict) else holder
|
||||
knowledge._deep_update(merged, knowledge._coerce(body))
|
||||
layers.config_status = 'merged'
|
||||
else:
|
||||
layers.config_status = 'missing'
|
||||
|
||||
merged['_plant_id'] = plant
|
||||
return merged, layers
|
||||
|
||||
|
||||
def _guess_job_root(jobs_dir):
|
||||
"""`<job_root>/sweep/jobs` -> `<job_root>`, verified against disk.
|
||||
|
||||
`tools/sweep.py`'s `sweep_dir()` writes `jobs/` at
|
||||
`designspec.sub_dir(doc, 'sweep')/jobs`, i.e. two levels under
|
||||
`root_dir(doc)` -- the same folder `knowledge_path(doc)` puts
|
||||
`knowledge.yaml` in. So the parent of JOBS_DIR's parent is tried first;
|
||||
if it holds no `knowledge.yaml`, every ancestor up to five levels is
|
||||
tried in turn (a renamed sweep folder, or `--check`'s custom `--out`,
|
||||
can move the depth). None found returns the two-level guess anyway, so
|
||||
the caller always has a slightly wrong parent to complain about instead
|
||||
of no parent at all.
|
||||
"""
|
||||
jobs_dir = os.path.abspath(jobs_dir)
|
||||
two_up = os.path.dirname(os.path.dirname(jobs_dir))
|
||||
candidate = two_up
|
||||
node = jobs_dir
|
||||
for _ in range(5):
|
||||
node = os.path.dirname(node)
|
||||
if not node or node == os.path.dirname(node):
|
||||
break
|
||||
if os.path.isfile(os.path.join(node, 'knowledge.yaml')):
|
||||
return node
|
||||
return two_up
|
||||
|
||||
|
||||
# ------------------------------------------------------------ comparisons
|
||||
|
||||
def _num_eq(a, b):
|
||||
if a is None or b is None:
|
||||
return a is b
|
||||
try:
|
||||
return math.isclose(float(a), float(b), rel_tol=1e-9, abs_tol=1e-6)
|
||||
except (TypeError, ValueError):
|
||||
return a == b
|
||||
|
||||
|
||||
def _verdict(current_val, frozen_val):
|
||||
"""current/stale/unjudgeable for one plain numeric field.
|
||||
|
||||
Both None: current (frozen agreement on 'nothing known'). current is
|
||||
None, frozen is not: unjudgeable -- current knowledge cannot confirm OR
|
||||
deny the frozen value, which is not the same as it being wrong. frozen
|
||||
is None, current is not: stale -- something is knowable now that was not
|
||||
at export, which is exactly the kind of drift this tool exists to catch.
|
||||
Both known: compared with a float tolerance.
|
||||
"""
|
||||
if current_val is None and frozen_val is None:
|
||||
return CURRENT
|
||||
if current_val is None:
|
||||
return UNJUDGEABLE
|
||||
if frozen_val is None:
|
||||
return STALE
|
||||
return CURRENT if _num_eq(current_val, frozen_val) else STALE
|
||||
|
||||
|
||||
def check_capacity(know, family, kind, frozen_val, tension_bar_families):
|
||||
"""(status, detail) for `capacity_lb` / `shear_capacity_lb`.
|
||||
|
||||
The gate `knowledge.lifter_capacity` applies (tension only; see its own
|
||||
docstring) needs `assembly_families` -- every OTHER member's family
|
||||
name on the same assembly -- which the deck never carries (see this
|
||||
module's own docstring). Calling with `assembly_families=None` always
|
||||
resolves SOMETHING (the conservative without-bar figure, by design --
|
||||
"an omitted list never earns the 2x rating by accident"), and that
|
||||
number can even happen to equal the frozen one. Trusting that match
|
||||
would be exactly the guess point 1 of the spec this was built against
|
||||
forbids: it is not evidence the piece lacks its tension bar, only that
|
||||
nothing here was told either way. So whenever the without-bar branch is
|
||||
the one that resolved, the with-bar branch is also computed (by forcing
|
||||
corroboration with the plant's own `tension_bar_families`) and, if it
|
||||
gives a different number, this reports CONDITIONAL naming both --
|
||||
regardless of which one the frozen value happens to match.
|
||||
"""
|
||||
val, basis, why = knowledge.lifter_capacity(
|
||||
know, family, kind=kind, assembly_families=None)
|
||||
if kind == 'tension' and basis == 'without_bar':
|
||||
alt_val, alt_basis, alt_why = knowledge.lifter_capacity(
|
||||
know, family, kind=kind, assembly_families=tension_bar_families)
|
||||
if alt_basis == 'with_bar' and not _num_eq(alt_val, val):
|
||||
matches = [b for b, v in (('without_bar', val), ('with_bar', alt_val))
|
||||
if _num_eq(v, frozen_val)]
|
||||
return CONDITIONAL, {
|
||||
'frozen': frozen_val,
|
||||
'without_bar': val, 'without_bar_why': why,
|
||||
'with_bar': alt_val, 'with_bar_why': alt_why,
|
||||
'frozen_matches': matches,
|
||||
'reason': (
|
||||
"gated on a welded tension bar; this deck does not "
|
||||
"carry the assembly's other member families, so "
|
||||
"whether the bar was corroborated on THIS piece cannot "
|
||||
"be re-derived offline. Current catalogue gives {0} lb "
|
||||
"without the bar or {1} lb with it corroborated; "
|
||||
"frozen {2} lb matches {3}."
|
||||
).format(val, alt_val, frozen_val,
|
||||
'/'.join(matches) if matches else 'neither'),
|
||||
}
|
||||
status = _verdict(val, frozen_val)
|
||||
return status, {'current': val, 'basis': basis, 'why': why, 'frozen': frozen_val}
|
||||
|
||||
|
||||
def _match_current_stage_spec(know, stage):
|
||||
specs = knowledge.get(know, 'handling.stages', []) or []
|
||||
pci = stage.get('pci_case')
|
||||
name = stage.get('name')
|
||||
for spec in specs:
|
||||
if pci and spec.get('pci_case') == pci:
|
||||
return spec
|
||||
for spec in specs:
|
||||
if name and spec.get('name') == name:
|
||||
return spec
|
||||
return None
|
||||
|
||||
|
||||
def check_stage_tension_limit(know, stage, job_default_limit):
|
||||
frozen = stage.get('tension_limit_sqrt_fc')
|
||||
source = stage.get('tension_limit_source')
|
||||
if source in (None, 'job'):
|
||||
current = job_default_limit
|
||||
why = 'handling.tension_limit_sqrt_fc, current job layer'
|
||||
elif source == 'stage':
|
||||
spec = _match_current_stage_spec(know, stage)
|
||||
if spec is None:
|
||||
return UNJUDGEABLE, {
|
||||
'frozen': frozen,
|
||||
'reason': (
|
||||
'no current handling.stages entry matches pci_case={0!r}'
|
||||
' / name={1!r} to compare the pinned limit against'
|
||||
.format(stage.get('pci_case'), stage.get('name'))),
|
||||
}
|
||||
spec_limit = spec.get('tension_limit_sqrt_fc')
|
||||
current = spec_limit if spec_limit is not None else job_default_limit
|
||||
why = ('handling.stages[pci_case={0!r}].tension_limit_sqrt_fc'
|
||||
.format(stage.get('pci_case')) if spec_limit is not None else
|
||||
'handling.tension_limit_sqrt_fc, current job layer (matched '
|
||||
'stage spec states none of its own)')
|
||||
else:
|
||||
return UNJUDGEABLE, {
|
||||
'frozen': frozen,
|
||||
'reason': 'unrecognised tension_limit_source {0!r} on this deck'
|
||||
.format(source),
|
||||
}
|
||||
return _verdict(current, frozen), {'current': current, 'frozen': frozen,
|
||||
'why': why}
|
||||
|
||||
|
||||
# ------------------------------------------------------------- per deck
|
||||
|
||||
def check_deck(path, know):
|
||||
"""The findings for one job file. Never raises on a well-formed deck."""
|
||||
with io.open(path, 'r', encoding='utf-8') as fh:
|
||||
deck = json.load(fh)
|
||||
|
||||
panel = deck.get('panel') or {}
|
||||
mark = panel.get('mark') or os.path.splitext(os.path.basename(path))[0]
|
||||
tension_bar_families = knowledge.get(know, 'lifters.tension_bar_families',
|
||||
[]) or []
|
||||
findings = []
|
||||
|
||||
def add(field, status, detail, lifter_mark=None, family=None):
|
||||
findings.append({
|
||||
'deck': path, 'mark': mark, 'lifter_mark': lifter_mark,
|
||||
'family': family, 'field': field, 'status': status,
|
||||
'detail': detail,
|
||||
})
|
||||
|
||||
for lifter in panel.get('lifters') or []:
|
||||
family = lifter.get('family')
|
||||
lmark = lifter.get('mark')
|
||||
if not family:
|
||||
add('family', UNJUDGEABLE,
|
||||
{'reason': 'this lifter carries no family name in the deck'},
|
||||
lmark, family)
|
||||
continue
|
||||
|
||||
if 'capacity_lb' in lifter:
|
||||
status, detail = check_capacity(
|
||||
know, family, 'tension', lifter['capacity_lb'],
|
||||
tension_bar_families)
|
||||
add('capacity_lb', status, detail, lmark, family)
|
||||
|
||||
if 'shear_capacity_lb' in lifter:
|
||||
status, detail = check_capacity(
|
||||
know, family, 'shear', lifter['shear_capacity_lb'],
|
||||
tension_bar_families)
|
||||
add('shear_capacity_lb', status, detail, lmark, family)
|
||||
|
||||
if 'basis_fc_psi' in lifter:
|
||||
cur = knowledge.lifter_basis_fc(know, family)
|
||||
add('basis_fc_psi', _verdict(cur, lifter['basis_fc_psi']),
|
||||
{'current': cur, 'frozen': lifter['basis_fc_psi']},
|
||||
lmark, family)
|
||||
|
||||
if 'min_edge_distance_in' in lifter:
|
||||
cur = (knowledge.lifter_edge_distance(know, family)
|
||||
or knowledge.get(know, 'lifters.min_edge_distance_in'))
|
||||
add('min_edge_distance_in',
|
||||
_verdict(cur, lifter['min_edge_distance_in']),
|
||||
{'current': cur, 'frozen': lifter['min_edge_distance_in']},
|
||||
lmark, family)
|
||||
|
||||
if 'tension_fos' in lifter:
|
||||
cur, src, why = knowledge.lifter_safety_factor(
|
||||
know, family, kind='tension')
|
||||
add('tension_fos', _verdict(cur, lifter['tension_fos']),
|
||||
{'current': cur, 'frozen': lifter['tension_fos'],
|
||||
'source': src, 'why': why}, lmark, family)
|
||||
|
||||
if 'shear_fos' in lifter:
|
||||
cur, src, why = knowledge.lifter_safety_factor(
|
||||
know, family, kind='shear')
|
||||
add('shear_fos', _verdict(cur, lifter['shear_fos']),
|
||||
{'current': cur, 'frozen': lifter['shear_fos'],
|
||||
'source': src, 'why': why}, lmark, family)
|
||||
|
||||
job_default_limit = knowledge.get(know, 'handling.tension_limit_sqrt_fc',
|
||||
5.0)
|
||||
for i, stage in enumerate(deck.get('stages') or []):
|
||||
if 'tension_limit_sqrt_fc' not in stage:
|
||||
continue
|
||||
status, detail = check_stage_tension_limit(know, stage,
|
||||
job_default_limit)
|
||||
field = 'stages[{0}:{1}].tension_limit_sqrt_fc'.format(
|
||||
i, stage.get('name') or stage.get('pci_case') or '?')
|
||||
add(field, status, detail)
|
||||
|
||||
return findings
|
||||
|
||||
|
||||
def deck_status(findings):
|
||||
if not findings:
|
||||
return CURRENT
|
||||
return max((f['status'] for f in findings), key=lambda s: _SEVERITY[s])
|
||||
|
||||
|
||||
# ----------------------------------------------------------------- CLI
|
||||
|
||||
def _fmt_num(v):
|
||||
if v is None:
|
||||
return 'null'
|
||||
if isinstance(v, float) and v == int(v):
|
||||
return str(int(v))
|
||||
return str(v)
|
||||
|
||||
|
||||
def _group_key(f):
|
||||
"""Findings that would print an identical line collapse into one.
|
||||
|
||||
A deck with several lifters of the same gated family produces the same
|
||||
CONDITIONAL (or STALE) numbers once per lifter; grouping is what keeps
|
||||
"one line per stale deck" true instead of one line per lifter on it.
|
||||
Two lifters of the same family that land on genuinely DIFFERENT numbers
|
||||
-- a plant override that only matched one of them, say -- still get
|
||||
separate lines, because the key includes the numbers themselves.
|
||||
"""
|
||||
d = f['detail']
|
||||
if f['status'] == CONDITIONAL:
|
||||
tail = (d.get('frozen'), d.get('without_bar'), d.get('with_bar'))
|
||||
elif f['status'] == STALE:
|
||||
tail = (d.get('frozen'), d.get('current'))
|
||||
else:
|
||||
tail = (d.get('reason') or d.get('why'),)
|
||||
return (f['field'], f['status']) + tail
|
||||
|
||||
|
||||
def _deck_lines(mark, findings):
|
||||
"""One line per DISTINCT (field, status, values) on this deck.
|
||||
|
||||
Not one line per lifter -- see `_group_key`. `current` findings never
|
||||
print here; pass `--all` for the raw per-finding dump instead.
|
||||
"""
|
||||
non_current = [f for f in findings if f['status'] != CURRENT]
|
||||
if not non_current:
|
||||
return []
|
||||
groups = {}
|
||||
for f in non_current:
|
||||
groups.setdefault(_group_key(f), []).append(f)
|
||||
lines = []
|
||||
for key in sorted(groups, key=lambda k: (-_SEVERITY[k[1]], k)):
|
||||
fs = groups[key]
|
||||
f0 = fs[0]
|
||||
d = f0['detail']
|
||||
families = sorted(set(x['family'] for x in fs if x['family']))
|
||||
marks = sorted(set(x['lifter_mark'] for x in fs if x['lifter_mark']))
|
||||
note = ''
|
||||
if marks:
|
||||
note = ' ({0} lifter{1}{2})'.format(
|
||||
len(marks), '' if len(marks) == 1 else 's',
|
||||
' ' + '/'.join(families) if families else '')
|
||||
status = f0['status']
|
||||
if status == CONDITIONAL:
|
||||
line = (' CONDITIONAL {0}{1} {2}: frozen={3} without_bar={4} '
|
||||
'with_bar={5} -- {6}'
|
||||
.format(mark, note, f0['field'], _fmt_num(d.get('frozen')),
|
||||
_fmt_num(d.get('without_bar')),
|
||||
_fmt_num(d.get('with_bar')), d.get('reason')))
|
||||
elif status == STALE:
|
||||
why = d.get('why') or d.get('reason') or ''
|
||||
line = (' STALE {0}{1} {2}: frozen={3} -> current={4}{5}'
|
||||
.format(mark, note, f0['field'], _fmt_num(d.get('frozen')),
|
||||
_fmt_num(d.get('current')),
|
||||
(' (' + why + ')') if why else ''))
|
||||
else:
|
||||
line = (' UNJUDGEABLE {0}{1} {2}: {3}'
|
||||
.format(mark, note, f0['field'],
|
||||
d.get('reason') or d.get('why') or
|
||||
'current knowledge could not resolve this'))
|
||||
lines.append(line)
|
||||
return lines
|
||||
|
||||
|
||||
def _finding_line(f):
|
||||
d = f['detail']
|
||||
who = f['mark']
|
||||
if f.get('lifter_mark'):
|
||||
who = '{0} lifter {1} ({2})'.format(f['mark'], f['lifter_mark'],
|
||||
f['family'])
|
||||
if f['status'] == CONDITIONAL:
|
||||
return (' CONDITIONAL {0} {1}: frozen={2} without_bar={3} '
|
||||
'with_bar={4} -- {5}'
|
||||
.format(who, f['field'], _fmt_num(d.get('frozen')),
|
||||
_fmt_num(d.get('without_bar')),
|
||||
_fmt_num(d.get('with_bar')), d.get('reason')))
|
||||
if f['status'] == STALE:
|
||||
why = d.get('why') or d.get('reason') or ''
|
||||
return (' STALE {0} {1}: frozen={2} -> current={3}{4}'
|
||||
.format(who, f['field'], _fmt_num(d.get('frozen')),
|
||||
_fmt_num(d.get('current')),
|
||||
(' (' + why + ')') if why else ''))
|
||||
if f['status'] == UNJUDGEABLE:
|
||||
return (' UNJUDGEABLE {0} {1}: {2}'
|
||||
.format(who, f['field'], d.get('reason') or d.get('why') or
|
||||
'current knowledge could not resolve this'))
|
||||
return ' current {0} {1}'.format(who, f['field'])
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument('jobs_dir', help="a sweep's jobs/ folder of *.job.json")
|
||||
ap.add_argument('--job-root', help='the folder holding this job\'s own '
|
||||
'knowledge.yaml (default: guessed '
|
||||
'from jobs_dir, see _guess_job_root)')
|
||||
ap.add_argument('--json', dest='json_out', help='write machine-readable '
|
||||
'output here too')
|
||||
ap.add_argument('--all', action='store_true',
|
||||
help='print every finding, not just non-current ones')
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
jobs_dir = args.jobs_dir
|
||||
if not os.path.isdir(jobs_dir):
|
||||
sys.stderr.write('deckcheck: not a directory: {0}\n'.format(jobs_dir))
|
||||
return 2
|
||||
|
||||
job_root = args.job_root or _guess_job_root(jobs_dir)
|
||||
try:
|
||||
know, layers = build_knowledge(job_root)
|
||||
except Exception as exc: # pragma: no cover
|
||||
sys.stderr.write('deckcheck: could not build current knowledge: '
|
||||
'{0}: {1}\n'.format(type(exc).__name__, exc))
|
||||
return 2
|
||||
|
||||
names = sorted(n for n in os.listdir(jobs_dir) if n.endswith('.job.json'))
|
||||
if not names:
|
||||
sys.stderr.write('deckcheck: no *.job.json under {0}\n'.format(jobs_dir))
|
||||
return 2
|
||||
|
||||
deck_paths = [os.path.join(jobs_dir, n) for n in names]
|
||||
newest_deck_mtime = max(os.path.getmtime(p) for p in deck_paths)
|
||||
newest_knowledge_mtime = layers.newest_mtime()
|
||||
|
||||
print('=' * 78)
|
||||
print('DECKCHECK -- knowledge staleness only, NOT model staleness.')
|
||||
print('This verifies each deck against KNOWLEDGE, never against the live')
|
||||
print('MODEL: a deck can pass every check here and still describe a')
|
||||
print('panel whose geometry has since changed in Revit.')
|
||||
print('=' * 78)
|
||||
if newest_knowledge_mtime is not None:
|
||||
age_hours = (newest_knowledge_mtime - newest_deck_mtime) / 3600.0
|
||||
if age_hours > 0:
|
||||
print('these decks predate the current knowledge by {0:.1f} hour(s)'
|
||||
.format(age_hours))
|
||||
elif -age_hours > 0:
|
||||
print('current knowledge is {0:.1f} hour(s) OLDER than the newest '
|
||||
'deck -- decks were exported after the last knowledge edit'
|
||||
.format(-age_hours))
|
||||
else:
|
||||
print('newest deck and newest knowledge layer share a timestamp')
|
||||
else:
|
||||
print('no knowledge layer file found to compare a timestamp against')
|
||||
|
||||
print('')
|
||||
print('job root: {0}'.format(job_root))
|
||||
for label, path, status in layers.rows():
|
||||
print(' {0:<10} {1} [{2}]'.format(label, path or '(none)', status))
|
||||
print('')
|
||||
|
||||
all_findings = []
|
||||
deck_statuses = []
|
||||
stale_lines = []
|
||||
for path in deck_paths:
|
||||
try:
|
||||
findings = check_deck(path, know)
|
||||
except Exception as exc:
|
||||
findings = [{
|
||||
'deck': path, 'mark': os.path.basename(path),
|
||||
'lifter_mark': None, 'family': None, 'field': '(deck read)',
|
||||
'status': UNJUDGEABLE,
|
||||
'detail': {'reason': '{0}: {1}'.format(type(exc).__name__, exc)},
|
||||
}]
|
||||
all_findings.extend(findings)
|
||||
status = deck_status(findings)
|
||||
deck_statuses.append(status)
|
||||
mark = findings[0]['mark'] if findings else os.path.basename(path)
|
||||
if args.all:
|
||||
stale_lines.extend(_finding_line(f) for f in findings)
|
||||
elif status != CURRENT:
|
||||
stale_lines.extend(_deck_lines(mark, findings))
|
||||
|
||||
if stale_lines:
|
||||
print('-' * 78)
|
||||
for line in stale_lines:
|
||||
print(line)
|
||||
print('-' * 78)
|
||||
|
||||
n_total = len(deck_statuses)
|
||||
n_current = sum(1 for s in deck_statuses if s == CURRENT)
|
||||
n_stale = sum(1 for s in deck_statuses if s == STALE)
|
||||
n_conditional = sum(1 for s in deck_statuses if s == CONDITIONAL)
|
||||
n_unjudgeable = sum(1 for s in deck_statuses if s == UNJUDGEABLE)
|
||||
|
||||
unjudgeable_reasons = sorted(set(
|
||||
f['detail'].get('reason') or f['detail'].get('why') or '(no reason given)'
|
||||
for f in all_findings if f['status'] == UNJUDGEABLE
|
||||
))
|
||||
|
||||
print('')
|
||||
print('{0} deck(s) checked, {1} current, {2} stale, {3} conditional, '
|
||||
'{4} unjudgeable.'
|
||||
.format(n_total, n_current, n_stale, n_conditional, n_unjudgeable))
|
||||
if n_unjudgeable:
|
||||
print('unjudgeable because:')
|
||||
for r in unjudgeable_reasons:
|
||||
print(' - {0}'.format(r))
|
||||
|
||||
if args.json_out:
|
||||
payload = {
|
||||
'job_root': job_root,
|
||||
'layers': [{'layer': label, 'path': path, 'status': status}
|
||||
for label, path, status in layers.rows()],
|
||||
'newest_deck_mtime': newest_deck_mtime,
|
||||
'newest_knowledge_mtime': newest_knowledge_mtime,
|
||||
'decks_checked': n_total, 'current': n_current, 'stale': n_stale,
|
||||
'conditional': n_conditional, 'unjudgeable': n_unjudgeable,
|
||||
'findings': all_findings,
|
||||
}
|
||||
with io.open(args.json_out, 'w', encoding='utf-8') as fh:
|
||||
json.dump(payload, fh, indent=1, default=str)
|
||||
print('\njson written to {0}'.format(args.json_out))
|
||||
|
||||
return 0 if n_current == n_total else 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,283 @@
|
||||
# Knowledge keys deliberately exempted from tools/knowledge_drift.py.
|
||||
#
|
||||
# FORMAT -- one per line:
|
||||
#
|
||||
# <category> <dotted.path> -- why this one is fine
|
||||
#
|
||||
# categories: unread undeclared defaults surface-only test-only
|
||||
#
|
||||
# THE REASON IS MANDATORY. A line with no ' -- why' is refused and the run
|
||||
# exits 2, because an unexplained suppression is how a checker quietly stops
|
||||
# checking. The point of an exception is that the next person can read why it
|
||||
# was made and decide whether it still holds; a bare path cannot be argued
|
||||
# with, only inherited.
|
||||
#
|
||||
# WHY A FILE AND NOT A YAML COMMENT. Two of the five categories are about keys
|
||||
# that exist only in Python (a hidden default) or only in a gitignored plant
|
||||
# or job file that a colleague's clone does not have. A comment convention
|
||||
# could not express those, so half the exceptions would have nowhere to live
|
||||
# and would end up as a second, undocumented mechanism. One file, in the repo,
|
||||
# beside the tool that reads it, greppable, and reviewed like code.
|
||||
#
|
||||
# HOW IT STAYS HONEST. Every run prints how many findings this file suppressed,
|
||||
# so a shrinking report is never mistaken for a shrinking problem. Every run
|
||||
# also lists entries here that no longer match anything, so a key that was
|
||||
# fixed or deleted does not leave a permanent blindfold behind.
|
||||
#
|
||||
# DO NOT put a real drift here to make the report green. An exception says
|
||||
# "this key is not meant to be read"; it does not say "we know and have not
|
||||
# got to it yet". For the second, leave the finding and fix the code.
|
||||
#
|
||||
# Example of the shape (commented out -- this is not an active exemption):
|
||||
#
|
||||
# unread nomenclature.view_design_token_re -- derived from design_number_re
|
||||
# when left None, so nothing reads the key itself by design.
|
||||
|
||||
# ---- active exemptions below this line ----
|
||||
|
||||
# ===========================================================================
|
||||
# GROUP 1 -- the key IS read, by a call shape this detector cannot resolve.
|
||||
# ===========================================================================
|
||||
# Every line below was verified by hand on 2026-08-30 by opening the reader
|
||||
# and reading it. None of them is a deferral: the code acts on the key today.
|
||||
# Two shapes recur, and the tool's own blind-spot report already names both:
|
||||
# a knowledge getter passed in as a FUNCTION ARGUMENT (so the call reads
|
||||
# get(...) / getter(...), not K.get(...)), and a subtree fetched whole and
|
||||
# then walked by dict access. Where the fix belongs in the detector rather
|
||||
# than in the code the line says so; the day it lands, the run will call
|
||||
# these lines stale and they should be deleted.
|
||||
|
||||
# takeoffcore.settings(know, get) loops its own module-owned DEFAULTS table
|
||||
# and calls get(know, 'takeoff.' + key) once per entry. Both halves are
|
||||
# invisible: `get` is an argument -- injected so the module needs no Revit and
|
||||
# stays testable off a model -- and the path is concatenated. The takeoff acts
|
||||
# on all seven; they are what classifies a family as bar / part / shape /
|
||||
# not-ordered. See takeoffcore.DEFAULTS and takeoffcore.classify().
|
||||
unread takeoff.bar_prefixes -- read by takeoffcore.settings() through an injected getter and a concatenated path; classifies bar families in takeoffcore.classify()
|
||||
unread takeoff.part_prefixes -- read by takeoffcore.settings() through an injected getter and a concatenated path; classifies part families in takeoffcore.classify()
|
||||
unread takeoff.shape_markers -- read by takeoffcore.settings() through an injected getter and a concatenated path; marks a bend-shape row in takeoffcore.classify()
|
||||
unread takeoff.not_ordered -- read by takeoffcore.settings() through an injected getter and a concatenated path; drops families the plant does not order
|
||||
unread takeoff.family_class -- read by takeoffcore.settings() through an injected getter and a concatenated path; a family named here beats every pattern
|
||||
unread takeoff.nested_in -- read by takeoffcore.settings() through an injected getter and a concatenated path; rolls a nested family up into its parent
|
||||
unread takeoff.order_attached_bars_separately -- read by takeoffcore.settings() through an injected getter and a concatenated path; decides whether an attached bar is ordered on its own line
|
||||
|
||||
# jobreconscan._roles() resolves nine roles in one loop over a literal tuple
|
||||
# and calls K.param_names(know, role) with the loop variable. The tool sees
|
||||
# the call and reports it as a blind spot -- 'non-literal key into
|
||||
# param_names' -- but cannot bind the roles. Job Recon reads all four of
|
||||
# these off the model on every run.
|
||||
unread parameters.member.ticket_created -- resolved in jobreconscan._roles()'s role loop and read off every member by Job Recon; the loop variable hides it from the scan
|
||||
unread parameters.member.ticket_detailed -- resolved in jobreconscan._roles()'s role loop and read off every member by Job Recon; the loop variable hides it from the scan
|
||||
unread parameters.sheet.total_required -- resolved in jobreconscan._roles()'s role loop; Job Recon compares it against the pieces the model carries
|
||||
unread parameters.sheet.sheet_count -- resolved in jobreconscan._roles()'s role loop; Job Recon compares it against the sheets actually placed
|
||||
|
||||
# ticketchecks._sheet_param_value(t, know, 'sheet.weight') -> ticketqc.rstr()
|
||||
# -> K.param_names(). The literal role is right there, but behind two
|
||||
# wrappers, and only _nomen / param_names / _s are unwrapped by the detector.
|
||||
# It is deliberately NOT in knowledge.PARAMETER_ROLES: the printed-weight
|
||||
# check stands down with its own sentence when the plant maps no name for it,
|
||||
# so Self Check must not nag a plant that never prints a weight.
|
||||
unread parameters.sheet.weight -- read by ticketchecks through _sheet_param_value() -> ticketqc.rstr() -> param_names; kept out of PARAMETER_ROLES on purpose, the weight check stands down and says so itself
|
||||
|
||||
# ticketpage.muted_checks(know, getter=None) reads it through an injected
|
||||
# getter, the same shape as takeoffcore. Ticket QC calls it on every run and
|
||||
# filters checklist rows with the result, so this is behaviour, not display.
|
||||
surface-only checks.muted -- read by ticketpage.muted_checks() through an injected getter; Ticket QC mutes checklist rows on it every run
|
||||
|
||||
# liftexport fetches handling.dunnage WHOLE at behaviour tier and
|
||||
# dunnage_supports() walks the dict by hand for mode and fraction -- together
|
||||
# they decide where the transportation supports go. The detector only skips a
|
||||
# surface-read path when a behaviour reader sits BELOW it; a covering
|
||||
# behaviour read of an ANCESTOR is not considered, which is the only reason
|
||||
# these two fire. Delete both lines when that is fixed in the tool.
|
||||
surface-only handling.dunnage.mode -- liftexport reads the whole handling.dunnage subtree at behaviour tier and dunnage_supports() walks it; the tool does not credit a covering behaviour read of an ancestor
|
||||
surface-only handling.dunnage.fraction -- liftexport reads the whole handling.dunnage subtree at behaviour tier and dunnage_supports() walks it; the tool does not credit a covering behaviour read of an ancestor
|
||||
|
||||
# liftexport._fast_box picks the key by run mode and passes `not batch` as the
|
||||
# fallback: for handling.fast_lifter_box that expression IS True, and for
|
||||
# handling.fast_lifter_box_batch it IS False -- exactly what configdump
|
||||
# passes. The two call sites agree on the value and differ only in source
|
||||
# text, which is all this category can compare.
|
||||
defaults handling.fast_lifter_box -- liftexport._fast_box passes `not batch`, which for this key evaluates to True, the same fallback configdump prints; the two differ in source text only
|
||||
defaults handling.fast_lifter_box_batch -- liftexport._fast_box passes `not batch`, which for this key evaluates to False, the same fallback configdump prints; the two differ in source text only
|
||||
|
||||
# The two call sites ask different questions and both answers are written
|
||||
# down. designspec.design_key MUST always return a key, so it falls back to a
|
||||
# built-in generic pattern. ticketchecks._view_design_token_re would have to
|
||||
# strip the leading anchor off that pattern and hunt for it UNANCHORED inside
|
||||
# a view name; guessing there is how a view called 'Elevation Top 2' becomes
|
||||
# a design. It returns None instead and checklist 1.4 reads NOT RUN, which
|
||||
# its docstring states outright. Not a disagreement about what unset means.
|
||||
defaults nomenclature.design_number_re -- deliberate: designspec.design_key falls back to a built-in pattern because it must always return a key, while ticketchecks._view_design_token_re refuses to guess an unanchored view token and lets checklist 1.4 read NOT RUN
|
||||
|
||||
# ===========================================================================
|
||||
# GROUP 2 -- not knowledge keys at all: a design spec read as a layer
|
||||
# ===========================================================================
|
||||
# knowledge_drift._job_files() walks the job folder recursively and takes
|
||||
# every .yaml in it. A design spec -- designspec.spec_path writes them to
|
||||
# <job>/specs/<DESIGN>.yaml, first line `schema: 1` -- is a scraped record of
|
||||
# one design, not a knowledge layer, and merging one plants its whole shape
|
||||
# as declared paths: schema, source, confidence, unresolved, and the measured
|
||||
# values under them. Nothing reads them through knowledge.get because nothing
|
||||
# should; designspec loads a spec by path.
|
||||
#
|
||||
# THE FIX IS IN THE TOOL and it is one line: _job_files() must not descend
|
||||
# into specs/. These thirteen lines live only until it does, and the run will
|
||||
# name them stale the moment it lands. While they are here they are also a
|
||||
# small hazard worth knowing about: `source` and `schema` are generic enough
|
||||
# that a future real knowledge key of either name would be suppressed by them.
|
||||
unread schema -- design-spec field, not a knowledge key; _job_files() descends into <job>/specs/ and reads a spec document as a layer
|
||||
unread source -- design-spec field (which model, sheet and view it was scraped from), not a knowledge key; _job_files() reads <job>/specs/ as a layer
|
||||
unread confidence -- design-spec field, not a knowledge key; _job_files() reads <job>/specs/ as a layer
|
||||
unread unresolved -- design-spec field (what the scrape could not settle), not a knowledge key; _job_files() reads <job>/specs/ as a layer
|
||||
unread design_number -- design-spec field, not a knowledge key; _job_files() reads <job>/specs/ as a layer
|
||||
unread piece_description -- design-spec field, not a knowledge key; _job_files() reads <job>/specs/ as a layer
|
||||
unread thickness_in -- design-spec field, not a knowledge key; _job_files() reads <job>/specs/ as a layer
|
||||
unread envelope -- design-spec field, not a knowledge key; _job_files() reads <job>/specs/ as a layer
|
||||
unread reinforcing -- design-spec field, not a knowledge key; _job_files() reads <job>/specs/ as a layer
|
||||
unread concrete.fc_psi -- design-spec field, not a knowledge key; _job_files() reads <job>/specs/ as a layer (the knowledge-layer concrete branch carries min_fci_ratio, which is read)
|
||||
unread concrete.fci_psi -- design-spec field, not a knowledge key; _job_files() reads <job>/specs/ as a layer
|
||||
unread concrete.unit_weight_pcf -- design-spec field, not a knowledge key; _job_files() reads <job>/specs/ as a layer
|
||||
unread concrete.weight_class -- design-spec field, not a knowledge key; _job_files() reads <job>/specs/ as a layer
|
||||
|
||||
# ---- declared inert on purpose ----
|
||||
# A key that is deliberately not wired, ruled on rather than overlooked. Not
|
||||
# the same thing as muting a CHECKLIST ROW (that is `checks.muted`, a job
|
||||
# fact, deep-merged, and set from the Project Config window): this changes no
|
||||
# run on any job, is the same everywhere, and its audience is whoever reads
|
||||
# this report next.
|
||||
unread tolerances.psi -- MUTED, NOT DELETED. Owner's ruling on triage D1, 2026-08-30: neither wire nor delete. Nothing compares a psi against it (3.1 and 3.2 compare the ticket and the design summary exactly), so it decides nothing on any job today; it is kept declared and settable so that the day a tolerance is wanted the number and its name are already there. The engineer-facing half is configform's `inert` row, which shows it with this ruling instead of as a number box that decides nothing.
|
||||
|
||||
# ---- the master checklist moved into the public layer, 2026-08-31 --------
|
||||
unread checklist -- the master checklist is read by ticketpage.load_master, which opens knowledge/catalogue.yaml DIRECTLY at import and parses it; it never goes through knowledge.get on the merged tree, so this detector cannot see the reader. It is the most-read data in the tool -- CHECKLIST_ITEMS is derived from it -- and two tests in test_ticketchecks_host.py pair every entry against the check functions.
|
||||
|
||||
# ---- connector knowledge, 2026-08-31 ------------------------------------
|
||||
unread connectors.what_it_is -- the definition of a connector, in prose, for a person reading the catalogue. Deliberately not a predicate: what a connector IS belongs in the file that publishes the products, and what RECOGNISES one is connectors.marks, which is read.
|
||||
unread connectors.insulation_note -- the model's own description of these pins as NON-COMPOSITE, quoted and dated. Recorded because it states a design assumption with engineering weight; deliberately read by nothing, and it must not become a check without the owner's word.
|
||||
unread connectors.recognition_verified_on -- provenance for connectors.recognition_verified, which IS read. A date beside a flag is for the person deciding whether the flag still holds.
|
||||
unread connectors.recognition_verified_by -- as above: what was measured, so the flag can be argued with rather than inherited.
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# GROUP 3 -- LANDED FROM THE PDF LIBRARY, READER PENDING. 2026-08-31.
|
||||
# ===========================================================================
|
||||
# Stage 3c of knowledge/library/RUNBOOK.md, executed against library row D1
|
||||
# (the plant standards manual, 929 pages) whose 25-fact audit had been
|
||||
# EXTRACTED AND NEVER LANDED. The owner's ruling that commissioned this pass
|
||||
# is quoted in that runbook and in knowledge/library/LEDGER.md.
|
||||
#
|
||||
# EVERY LINE BELOW IS A FACT IN A KNOWLEDGE LAYER THAT NO CODE READS YET, and
|
||||
# that is the intended state, not a deferral of a fix. Knowledge leads and
|
||||
# readers follow: the alternative is a drift gate that PUNISHES LANDING, and a
|
||||
# gate that punishes the right behaviour is a gate people route around --
|
||||
# which is exactly how roughly 150 A-class records ended up in notes instead
|
||||
# of in YAML. With these lines the drift list becomes the to-do list: every
|
||||
# unread key is a reader somebody owes, named, dated, with its cite attached.
|
||||
#
|
||||
# ---- AND THIS FILE'S OWN HEADER SAYS THE OPPOSITE, IN AS MANY WORDS -------
|
||||
# "An exception says 'this key is not meant to be read'; it does not say 'we
|
||||
# know and have not got to it yet'. For the second, leave the finding and fix
|
||||
# the code." THE COMMISSION AND THAT SENTENCE DISAGREE. The honest
|
||||
# resolution is either a new category (`reader-pending`, dated) or an
|
||||
# amendment to the header -- NOT a quiet re-use of `unread` that makes the
|
||||
# header a lie. Per RUNBOOK.md stage 3c, until the owner rules: `unread` is
|
||||
# used, every reason says "reader pending", and THE CONFLICT IS ON HIS LIST
|
||||
# (QNA/open-questions-2026-08-30.md). The header was NOT edited here -- that
|
||||
# is not this pass's call to make.
|
||||
#
|
||||
# Cites are PDF page / printed section, the convention the extraction notes
|
||||
# use. Both halves always: the PDF page is how the next agent finds it, the
|
||||
# printed section is how the owner does, and they do not match. Full
|
||||
# argument, page cites and confidence marks for every one of these:
|
||||
# QNA/knowledge-audit-plant-2026-08-30.md, and knowledge/library/LEDGER.md D1.
|
||||
#
|
||||
# Three of these are also PARKED CONTRADICTIONS -- a live value the manual
|
||||
# disagrees with, landed BESIDE it rather than over it, awaiting a ruling.
|
||||
# They are marked "RULING PENDING" and carry their open-questions number.
|
||||
|
||||
unread beds -- landed 2026-08-31 from the plant standards manual (library D1) p584-600 / 6110-6126, reader pending: the per-bed strand-hole grid, panel width and abutment capacity. 4.6 BED-WIDTH/BED-OFFSET reads the rails.max_bed_width_in scalar today and the manual contradicts it (open-questions A6.1); 4.6 STRAND-GRID does not exist yet
|
||||
unread rails.narrow_panel -- landed 2026-08-31 from the plant standards manual p819 / 10315.1 and p804 / 10301.1 notes 27-29, reader pending: a narrow panel is HELD TO THE FIXED RAIL, so 4.6 BED-OFFSET's permitted band is an outer bound and not an expectation, and 4.4's asymmetric pattern is expected rather than a finding
|
||||
unread strand.standard_patterns -- landed 2026-08-31 from the plant standards manual p174-178 / 3201-3205 and p873 / 12204, reader pending: the plant's three standard strand chains, so 4.4 STRAND-POS gets a second opinion beside the design summary
|
||||
unread strand.edge_strand_in -- landed 2026-08-31 from the plant standards manual p876 / 12210, reader pending: EDGE STRAND @ 3 5/8 in, the first and last segment of every standard chain
|
||||
unread strand.both_wythes_same_stations -- landed 2026-08-31 from the plant standards manual p174-178 / 3201-3205, reader pending: STRAND E.L. means both wythes share x-positions, so total strand is 2x the per-wythe count
|
||||
unread strand.debond_by_blockout -- landed 2026-08-31 from the plant standards manual p817 / 10306.1, reader pending: 4.7 has the opening geometry and strand.debond_below_continuous_ft, but the L-vs-W ratio rows have no counterpart in the tool at all
|
||||
unread strand.deflection -- landed 2026-08-31 from the plant standards manual p804 / 10301.1 note 20C and p817 / 10306.1, reader pending: deflection is the alternative to debonding and is ranked LAST, max 2 in, min cover 1 1/2 in
|
||||
unread strand.detensioning -- landed 2026-08-31 from the plant standards manual p628 / 6320, reader pending: nothing in the tool reads a cutting sequence. Landed because knowledge leads readers, audit 3.25
|
||||
unread reinforce.panel_standard -- landed 2026-08-31 from the plant standards manual p183-191 / 3280-3288, reader pending: ReinforceSuggest works only from donor pieces today, and this is its first non-donor source. The middle band is PER DESIGN and must never be proposed
|
||||
unread reinforce.design_sheet_selector -- landed 2026-08-31 from the plant standards manual p183-191 / 3280-3288, reader pending: which of the nine standard reinforcing sheets applies, from panel geometry alone
|
||||
unread reinforce.weight_plf -- landed 2026-08-31 from the plant standards manual p183-191 / 3280-3288, reader pending: a per-linear-foot sanity band for Takeoff and for bar-schedule plausibility
|
||||
unread reinforce.master_ticket -- landed 2026-08-31 from the plant standards manual p716 / 8430.1, reader pending: it defines what reinforce.master_token marks. The durations and the submission list are process with no check behind them
|
||||
unread lifters.stripping_rows_by_height_ft -- landed 2026-08-31 from the plant standards manual p183 / 3280, reader pending: panellift.layout decides where inserts go and does not know the plant's own row count by panel height
|
||||
unread lifters.stripping_rows_gap_at_ft -- landed 2026-08-31 from the plant standards manual p183 / 3280, reader pending: the sheet's own bands leave H = 39 ft exactly unassigned; a reader must pick a side and say which
|
||||
unread lifters.stripping_rows_gap_because -- landed 2026-08-31 from the plant standards manual p183 / 3280, reader pending: the sentence a reader must print when it resolves the 39 ft boundary itself
|
||||
unread lifters.default_layout -- landed 2026-08-31 from the plant standards manual p713 / 8320.2, reader pending: fifth points is a DEFAULT and the sheet says to verify the insert type with the engineer
|
||||
unread lifters.insert_type_verified_with_engineer -- landed 2026-08-31 from the plant standards manual p713 / 8320.2, reader pending: the half of that sentence that stops the default being read as governing
|
||||
unread lifters.common_elevation_panel_to_panel -- landed 2026-08-31 from the plant standards manual p804 / 10301.1 note 14, reader pending: lifting inserts at a common elevation panel to panel
|
||||
unread lifters.dimension_centre_to_centre_and_both_edges -- landed 2026-08-31 from the plant standards manual p812 / 10303.1, reader pending: a dimensioning rule for lift inserts, checkable once something reads printed insert dimensions
|
||||
unread lifters.edge_lifter_centre_matches_cg -- landed 2026-08-31 from the plant standards manual p812 / 10303.1, reader pending: 6.2 LIFT-EDGE has the geometry but not this condition
|
||||
unread lifters.side_lifters -- landed 2026-08-31 from the plant standards manual p813 / 10303.2 and p804 / 10301.1 note 10, reader pending: an ADD'L FINISH = YES title block with no side lifters is a new check needing only two title-block reads
|
||||
unread lifters.top_pick_count -- landed 2026-08-31 from the plant standards manual p183 / 3280, reader pending: 4 top-pick inserts. The PART is deliberately not landed -- the sheet names M42/M43 and the model uses P76 (audit Q5)
|
||||
unread part_numbers.prefix_class -- landed 2026-08-31 from the plant standards manual p406 / 5001, reader pending: the material-type prefix, which gives families.* a fallback that cannot go stale the way a list does
|
||||
unread part_numbers.number_bands -- landed 2026-08-31 from the plant standards manual p406 / 5001 and p713 / 8320.2, reader pending: the number range tells a standard part from an erection item from a repair item with no list at all
|
||||
unread part_numbers.finish_suffix -- landed 2026-08-31 from the plant standards manual p406 / 5001, reader pending: g/p/z/s/e finish suffixes on a material designation
|
||||
unread part_numbers.job_specific_prefixes -- landed 2026-08-31 from the plant standards manual p406 / 5001, reader pending: the printed rule behind the SP-series move to the job layer on 2026-08-30, which this file acted on without holding the rule
|
||||
unread part_numbers.rebar_mark -- landed 2026-08-31 from the plant standards manual p536 / 5302, reader pending: 5.6 BARMARK-UNIQUE could catch a mark whose first digit disagrees with the modelled bar size with no geometry at all
|
||||
unread panel_types.make_ups -- landed 2026-08-31 from the plant standards manual p174-178 / 3201-3205 and p879 / 12221, reader pending: 2.1 SIZE-* reads panel_type.thickness and has no list of the five thicknesses this plant builds
|
||||
unread girders -- landed 2026-08-31 from the plant standards manual p805 / 10301.2 note 3 and p879 / 12221, reader pending: girder depth, the insulation-plus-3in size rule and the opening-proximity rules. The COUNT is deliberately null -- three sources give three answers (audit Q3)
|
||||
unread nomenclature.control_numbers -- landed 2026-08-31 from the plant standards manual p693-695 / 8102.3-8102.5, reader pending: JobRecon does the 163-vs-61 reconciliation and does not hold the arithmetic rule that makes it a defect
|
||||
unread nomenclature.marks -- landed 2026-08-31 from the plant standards manual p693-695 / 8102.3-8102.5, reader pending: a gap in the mark sequence is EXPECTED, and the renumber trigger reframes 2.1 SIZE-* and 2.2 WGT-* as a piece that should have been renumbered
|
||||
unread nomenclature.ticket_set -- landed 2026-08-31 from the plant standards manual p827 / 10340.2 and p807 / 10301.4, reader pending: 1.3 TICKET-SET gathers sheets today but does not know that sheets after the first carry no product data, nor that third-party sheets are legitimately appended
|
||||
unread nomenclature.sheet_families_manual_stated -- landed 2026-08-31 from the plant standards manual p819 / 10315.1, p822 / 10330.1, p821 / 10320, p716 / 8430.1, p851 / 10950, reader pending AND RULING PENDING: nomenclature.detail_sheet_prefix is live and unchanged; this is the manual's fuller set, parked beside it as open-questions A6.2
|
||||
unread drawings.rebar_placement_manual_stated -- landed 2026-08-31 from the plant standards manual p804 / 10301.1 notes 25, 26, 40, 47 and p805 / 10301.2 note 15C, reader pending AND RULING PENDING: drawings.rebar_must_appear_on is a residual and unchanged; this is the manual's per-bar-class enumeration, parked beside it as open-questions A6.5
|
||||
unread drawings.dimension_exceptions -- landed 2026-08-31 from the plant standards manual p804 / 10301.1 notes 6, 33, 46, 48, p816 / 10305, p726 / 8520.1, reader pending: every entry is a place where a MISSING dimension is correct by standard, and a completeness check that does not know them produces a steady drip of false findings
|
||||
unread drawings.bom_rebar_has_no_description -- landed 2026-08-31 from the plant standards manual p713 / 8320.2, reader pending: an empty BOM description on a rebar row is CORRECT, and without this 5.1 BOM-MEMBERS would fire on every rebar row of every ticket
|
||||
unread drawings.not_printed_on_ticket -- landed 2026-08-31 from the plant standards manual p701 / 8203, reader pending: the manual's own statement of what is deliberately absent from a ticket -- the panel ID and the behind-panel sections
|
||||
unread drawings.alternate_datum -- landed 2026-08-31 from the plant standards manual p811 / 10302.2 and p814 / 10304, reader pending: on an angled-top panel embeds are dimensioned to a WORK POINT, a legitimate alternative datum that will confuse a naive datum check
|
||||
unread drawings.dimensions -- landed 2026-08-31 from the plant standards manual p804 / 10301.1 note 5, p810 / 10302.1, p814 / 10304, reader pending: centre-of-embed is this plant's universal datum and is a precondition for any check comparing a printed dimension to a modelled position
|
||||
unread drawings.orientation_on_sheet -- landed 2026-08-31 from the plant standards manual p804 / 10301.1 notes 2, 3, 12, 13 and p819 / 10315.1, reader pending: drawings.section_a and section_b were measured off crop boxes on three pieces; this is the convention that binding should agree with
|
||||
unread drawings.attached_bar_rule -- landed 2026-08-31 from the plant standards manual p413-418 / 5100.1-5100.6 and p701 / 8203, reader pending: the rule behind drawings.annotation_covered_by_parent, which is a two-entry list today and goes stale in silence the way drawings.reinforcing_rule's predecessor did
|
||||
unread drawings.cad_layers -- landed 2026-08-31 from the plant standards manual p701 / 8203, reader pending: the layer a thing is drawn on IS its classification, and REBAR-HEAVY / REBAR-MED / REBAR-LIGHT is a reinforcing size split
|
||||
unread handling.weight_band_lb -- landed 2026-08-31 from the plant standards manual p800 / 10230.1, reader pending AND RULING PENDING: knowledge.DEFAULTS' weight_tolerance 0.02 is unchanged and 2.2 WGT-* still runs on it; this is the manual's -0/+100 lb band, parked beside it as open-questions A6.4
|
||||
unread geometry -- landed 2026-08-31 from the plant standards manual p168-170 / 3040-3042, p814 / 10304, p712 / 8320.1, p278 / 4294.0, p784 / 10150, reader pending: the recessed-plate rule ties a geometric property to a required X-sheet callout, which is checkable against DETAIL-CALLOUT
|
||||
unread finishes -- landed 2026-08-31 from the plant standards manual p165-166 / 3010, 3011, reader pending: a closed vocabulary with two checkable obligations -- a raked panel needs a direction on the ticket, and an end finish is checkable against the product type
|
||||
unread concrete.wall_panel_default -- landed 2026-08-31 from the plant standards manual p183-191 / 3280-3288, reader pending: f'ci 3500 gives 3.1/3.2/3.3 a plausibility anchor. f'c is null on purpose -- printed 5000, hand-revised 6000 in red, and neither state is settled (audit Q2)
|
||||
unread concrete.release_range_psi -- landed 2026-08-31 from the plant standards manual p627 / 6200, reader pending: the release-strength range this plant offers
|
||||
unread concrete.release_standard_psi -- landed 2026-08-31 from the plant standards manual p627 / 6200, reader pending: 3500 psi is the standard release, the rest is specialty
|
||||
unread concrete.day28_range_psi -- landed 2026-08-31 from the plant standards manual p627 / 6200, reader pending: 5000-7000 psi at 28 days available or more
|
||||
unread concrete.mix_role -- landed 2026-08-31 from the plant standards manual p800 / 10230.1, reader pending: FACE = BOTTOM wythe and BASE = TOP, because the panel is cast face-down. A reader that takes FACE MIX for the exterior mix is right by accident; one that takes it for the top wythe is wrong
|
||||
unread concrete.single_wythe_uses -- landed 2026-08-31 from the plant standards manual p800 / 10230.1, reader pending: a single-layer piece uses the BASE mix number, not the face one
|
||||
unread concrete.named_mixes -- landed 2026-08-31 from the plant standards manual p164 / 3003, reader pending: the plant's named mixes quoted directly. The positional digit decoding an extraction inferred is filed C there and deliberately not landed
|
||||
unread material_grades -- landed 2026-08-31 from the plant standards manual p863 / 11301.1 and p412 / 5015, reader pending: a plant CHOICE among published ASTM options, confidence B because it is read off a filled example form. HSS is null -- A513 struck to A500-C in red, neither state settled
|
||||
unread openings -- landed 2026-08-31 from the plant standards manual p760 / 9301 and p887 / 12261, reader pending: a void below 100 sq in or 8 in is one the plant does not cast, which filters the population 6.4 DUN-OPENING, 4.7 and liftexport._openings all count
|
||||
unread revisions -- landed 2026-08-31 from the plant standards manual p723 / 8511, reader pending: a ticket at revision 3 whose two predecessors are missing is NORMAL, because the sequence advances project-wide. The old-cloud layer name is null -- two spellings in one manual (audit Q10)
|
||||
unread connections -- landed 2026-08-31 from the plant standards manual p216-220 / 4201-4205, p231 / 4214, p243 / 4232, p375-403 / 4900-4974, reader pending: base-connection count by seismic category and butt-joint count by panel height are countable findings for 5.1 BOM-MEMBERS. The alignment count is null -- its table is red-marked (audit Q2)
|
||||
|
||||
# FOUR FACTS THIS PASS LANDED HAVE NO LINE HERE, AND THE REASON IS A BLIND
|
||||
# SPOT RATHER THAN A READER: notes.handling_views_on_sheet,
|
||||
# notes.handling_view_order, face_designations.carried_by_hatch and
|
||||
# face_designations.insulation_hatch all sit under a subtree that some caller
|
||||
# fetches WHOLE. The detector credits a covering read of an ancestor, so it
|
||||
# never reports them and an allowlist line for them is refused as stale. They
|
||||
# are as unread as everything below -- nothing acts on any of the four -- and
|
||||
# they are recorded in knowledge/library/LEDGER.md D1 instead. If the covering
|
||||
# read is ever narrowed, four findings appear at once and they are these.
|
||||
|
||||
|
||||
# ---- two owner rulings landed the same day, 2026-08-31 -------------------
|
||||
# Not from the library; from the owner directly, landed in the same pass
|
||||
# because it holds the plant file. See knowledge/library/LEDGER.md D1's row
|
||||
# for the pass and QNA/will-owes-2026-08-31.md for the open consequence.
|
||||
unread lifters.min_edge_distance_in_by_panel_type -- landed 2026-08-31 from the owner directly ("insulated panel minimum edge distance is 2ft, plant knowledge"), reader pending: knowledge.families_for() resolves a *_by_panel_type map for a LIST role only and there is no scalar resolver, so 6.2 still runs on the published-first order in ticketchecks._edge_limit(). The composition a reader must implement is LARGER-OF with the provenance named -- the plant's 24 in exceeds every published part minimum, so published-first would silently discard it
|
||||
unread panel_types.detect_verified_on -- provenance for panel_types.detect_verified, which IS read and was armed 2026-08-31. A date beside a flag is for the person deciding whether the flag still holds; same shape as connectors.recognition_verified_on above
|
||||
unread panel_types.detect_verified_by -- as above: what the arming rested on (three signals at 100% separation over 282 panels, a 1,951-mark connector census, and the owner's ruling), and the one thing it did NOT rest on -- a live press -- so the flag can be argued with rather than inherited
|
||||
|
||||
|
||||
# ---- the live-model guard, 2026-09-01 ------------------------------------
|
||||
# Both ARE read, on every single bridge request, by
|
||||
# `vibe/bridgeguard.config_from()`. The detector cannot see it: that function
|
||||
# is deliberately given `knowledge.get` as an ARGUMENT rather than importing
|
||||
# it, so the module stays free of anything that reads a file and can be unit
|
||||
# tested off-Revit (tests/test_bridgeguard.py). The literal paths are there,
|
||||
# one line apart -- they just arrive through a parameter named `getter`.
|
||||
unread bridge.refuse_titles -- read every request via bridgeguard.config_from(know, knowledge.get); the path literal is passed to an injected getter, not to knowledge.get by name, which is the wrapper shape this detector says it does not follow
|
||||
unread bridge.allow_roots -- as above, same call, same line pair
|
||||
@@ -0,0 +1,378 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Strict-parse every knowledge layer on this machine. Exit 1 if one will not.
|
||||
|
||||
python tools/knowledgelint.py # every standard layer here
|
||||
python tools/knowledgelint.py a.yaml b.yaml # just these
|
||||
python tools/knowledgelint.py knowledge/ # everything under a folder
|
||||
python tools/knowledgelint.py --list # what it would check
|
||||
python tools/knowledgelint.py --json # machine readable
|
||||
|
||||
**AFTER EDITING ANY KNOWLEDGE LAYER, RUN THIS.** A landing agent that writes
|
||||
a fact into `knowledge/plant/<plant>/knowledge.yaml`, a job file, the
|
||||
catalogue or a project config is not done until this passes. It takes under a
|
||||
second and it is the only thing between a mistyped file and a whole layer
|
||||
disappearing inside Revit.
|
||||
|
||||
WHY IT IS A GATE, AND WHY `scrub.py --private` IS NOT
|
||||
-----------------------------------------------------
|
||||
`--private` is an inventory: what it prints is raw material, and a check that
|
||||
fails on every run gets wrapped in `|| true`. This one is the opposite. An
|
||||
unparseable knowledge layer is ALWAYS wrong -- there is no state of the world
|
||||
in which a file the tool cannot read is acceptable -- so a failure here blocks,
|
||||
and exit 1 means exactly that.
|
||||
|
||||
WHAT IT IS FOR (measured 2026-09-01)
|
||||
------------------------------------
|
||||
pyRevit's reader is YamlDotNet, and it is STRICT: a mapping key stated twice
|
||||
raises `Duplicate key` and the read returns nothing. `knowledge._merge_file`
|
||||
wrapped that read in a bare `except Exception: return merged`, so one nested
|
||||
duplicate in one plant file silently discarded the ENTIRE 255KB plant layer.
|
||||
Every parameter role mapped to None, the sweep enumerated 0 assemblies, and
|
||||
every report rendered normally -- `sources()` and `config_problems()` both
|
||||
said healthy. A layer that fails to load looks exactly like a layer that
|
||||
agrees with the defaults.
|
||||
|
||||
`_merge_file` now records `_<layer>_unreadable` and `config_problems()` refuses
|
||||
on it, so Revit will say so. But that is the late warning, one button press
|
||||
before a wrong report; this is the early one, on the machine where the edit was
|
||||
made, before Revit is even open.
|
||||
|
||||
Host CPython 3.x, PyYAML. It reads with a unique-key SafeLoader -- the same
|
||||
strictness YamlDotNet has and the same one `tests/pyrevit_yaml_fake.py`
|
||||
doubles -- and NOT with `vibe.yamlkit`, which is a hand-rolled subset that
|
||||
takes the last of two duplicate keys and says nothing.
|
||||
|
||||
A NOTE ON WHAT IT PRINTS
|
||||
------------------------
|
||||
It names files, including the gitignored plant and job layers, because a lint
|
||||
tool that will not tell you which file failed is not a lint tool. Those paths
|
||||
carry job and client names. The output is for your terminal: do not paste it
|
||||
into a tracked file, a commit message or a doc.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError: # pragma: no cover
|
||||
sys.stderr.write('PyYAML is required: pip install pyyaml\n')
|
||||
raise SystemExit(2)
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
CATALOGUE = os.path.join(ROOT, 'knowledge', 'catalogue.yaml')
|
||||
CATALOGUE_TEMPLATE = os.path.join(ROOT, 'knowledge', 'catalogue.yaml.template')
|
||||
|
||||
# Mirrors `vibe.plantpath` / `vibe.designspec`, which cannot be imported here:
|
||||
# `vibe/__init__.py` pulls in the Revit API.
|
||||
PLANT_ENV = 'WILLTOOLS_KNOWLEDGE'
|
||||
PLANT_FOLDER = 'WillTools'
|
||||
JOBS_ENV = 'WILLTOOLS_JOBS'
|
||||
LAYER_FILE = 'knowledge.yaml'
|
||||
CONFIG_POINTER = 'active-config.txt'
|
||||
|
||||
SKIP_DIRS = {'__pycache__', '.git', 'dist', '_archive', '_scratch',
|
||||
'.pytest_cache', 'node_modules', 'library'}
|
||||
YAML_SUFFIXES = ('.yaml', '.yml')
|
||||
|
||||
|
||||
class DuplicateKeyError(yaml.YAMLError):
|
||||
"""A mapping key stated twice -- what YamlDotNet refuses and PyYAML allows.
|
||||
|
||||
Named as its own class so a caller can tell the failure this tool exists
|
||||
for from an ordinary syntax error, and worded to lead with the same two
|
||||
words pyRevit's reader uses (`Duplicate key`) so the string a person sees
|
||||
here is the string they will see in Self Check.
|
||||
"""
|
||||
|
||||
|
||||
def _unique_key_loader():
|
||||
"""SafeLoader that raises on a duplicate mapping key at ANY depth.
|
||||
|
||||
The standard recipe: `construct_mapping` runs once per mapping node, so
|
||||
nesting comes free, and `key_node.start_mark` points at the SECOND
|
||||
statement of the key -- the line to delete.
|
||||
"""
|
||||
class UniqueKeyLoader(yaml.SafeLoader):
|
||||
pass
|
||||
|
||||
def construct_mapping(loader, node, deep=False):
|
||||
seen = set()
|
||||
for key_node, _value in node.value:
|
||||
key = loader.construct_object(key_node, deep=deep)
|
||||
try:
|
||||
duplicate = key in seen
|
||||
except TypeError: # unhashable key: not this question
|
||||
continue
|
||||
if duplicate:
|
||||
mark = key_node.start_mark
|
||||
raise DuplicateKeyError(
|
||||
'Duplicate key: {0!r}, stated again at line {1}, column '
|
||||
'{2}. A mapping may state a key only once; pyRevit\'s '
|
||||
'reader refuses the WHOLE FILE over it, so the layer is '
|
||||
'not merged at all.'.format(
|
||||
key, mark.line + 1, mark.column + 1))
|
||||
seen.add(key)
|
||||
return yaml.SafeLoader.construct_mapping(loader, node, deep=deep)
|
||||
|
||||
UniqueKeyLoader.add_constructor(
|
||||
yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG, construct_mapping)
|
||||
return UniqueKeyLoader
|
||||
|
||||
|
||||
def check_file(path):
|
||||
"""(ok, message) for one file. `ok` False means the tool cannot read it.
|
||||
|
||||
The message carries line and column wherever the parser gives them --
|
||||
a duplicate key in a 4,000-line plant file is otherwise a search.
|
||||
"""
|
||||
try:
|
||||
handle = io.open(path, 'r', encoding='utf-8')
|
||||
except IOError as ex:
|
||||
return False, 'cannot open: {0}'.format(ex)
|
||||
try:
|
||||
text = handle.read()
|
||||
except (IOError, UnicodeDecodeError) as ex:
|
||||
return False, 'cannot read: {0}'.format(ex)
|
||||
finally:
|
||||
handle.close()
|
||||
|
||||
try:
|
||||
yaml.load(text, Loader=_unique_key_loader())
|
||||
except DuplicateKeyError as ex:
|
||||
return False, str(ex)
|
||||
except yaml.YAMLError as ex:
|
||||
return False, _yaml_message(ex)
|
||||
except Exception as ex: # pragma: no cover
|
||||
return False, '{0}: {1}'.format(type(ex).__name__, ex)
|
||||
return True, 'ok'
|
||||
|
||||
|
||||
def _yaml_message(ex):
|
||||
"""A PyYAML error as one line, with the line/column it knows about."""
|
||||
problem = (getattr(ex, 'problem', None) or str(ex)).strip()
|
||||
mark = getattr(ex, 'problem_mark', None)
|
||||
if mark is None:
|
||||
return problem
|
||||
context = (getattr(ex, 'context', None) or '').strip()
|
||||
where = 'line {0}, column {1}'.format(mark.line + 1, mark.column + 1)
|
||||
if context:
|
||||
return '{0} ({1}) at {2}'.format(problem, context, where)
|
||||
return '{0} at {1}'.format(problem, where)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ targets
|
||||
|
||||
def plant_root():
|
||||
"""The folder the plant layers live in. Mirrors `plantpath.plant_root`.
|
||||
|
||||
Set to a folder, that is the root; set to a file, its directory is (the
|
||||
machine-wide file lives IN the root); unset, %APPDATA%/WillTools.
|
||||
"""
|
||||
override = os.environ.get(PLANT_ENV)
|
||||
if not override:
|
||||
base = os.environ.get('APPDATA') or os.path.expanduser('~')
|
||||
return os.path.join(base, PLANT_FOLDER)
|
||||
if os.path.isdir(override):
|
||||
return override
|
||||
return os.path.dirname(override) or override
|
||||
|
||||
|
||||
def default_targets():
|
||||
"""Every standard layer location that EXISTS on this machine.
|
||||
|
||||
The four layers `knowledge.load()` merges, resolved the way it resolves
|
||||
them minus the document it does not have:
|
||||
|
||||
* `knowledge/catalogue.yaml`, in this repo;
|
||||
* the plant root -- `<root>/knowledge.yaml` and every
|
||||
`<root>/<plant>/knowledge.yaml`;
|
||||
* every `<jobs root>/<job>/knowledge.yaml`, plus the project config each
|
||||
job's `active-config.txt` points at;
|
||||
* and, whatever the environment says, everything under this checkout's
|
||||
own `knowledge/plant/` and `knowledge/job/`, because that is where
|
||||
these files are edited here and an agent whose environment is not set
|
||||
up must still be checking the files it just wrote.
|
||||
|
||||
Deduplicated by real path and returned in that order. A `.bak-*` snapshot
|
||||
is skipped: nothing reads one, and yesterday's file failing would make
|
||||
this a check people learn to ignore.
|
||||
"""
|
||||
out = []
|
||||
_add(out, CATALOGUE)
|
||||
|
||||
root = plant_root()
|
||||
_add(out, os.path.join(root, LAYER_FILE))
|
||||
for name in _entries(root):
|
||||
_add(out, os.path.join(root, name, LAYER_FILE))
|
||||
|
||||
jobs = os.environ.get(JOBS_ENV)
|
||||
if jobs:
|
||||
for name in _entries(jobs):
|
||||
_add(out, os.path.join(jobs, name, LAYER_FILE))
|
||||
_add(out, _config_pointed_at(os.path.join(jobs, name)))
|
||||
|
||||
for folder in ('plant', 'job'):
|
||||
for path in walk_yaml(os.path.join(ROOT, 'knowledge', folder)):
|
||||
_add(out, path)
|
||||
return out
|
||||
|
||||
|
||||
def _add(out, path):
|
||||
"""Append an existing file, once, by real path."""
|
||||
if not path or not os.path.isfile(path):
|
||||
return out
|
||||
real = os.path.realpath(path)
|
||||
if real not in [os.path.realpath(p) for p in out]:
|
||||
out.append(path)
|
||||
return out
|
||||
|
||||
|
||||
def _entries(folder):
|
||||
try:
|
||||
return sorted(os.listdir(folder))
|
||||
except OSError:
|
||||
return []
|
||||
|
||||
|
||||
def _config_pointed_at(job_dir):
|
||||
"""The project config a job's one-line pointer names, if it is there.
|
||||
|
||||
A pointer to a file that is GONE is `knowledge._config_missing`, which the
|
||||
buttons already refuse on, and it is not a parse failure -- the file may
|
||||
simply live on a share this machine is not on today. So it is not this
|
||||
tool's verdict: it returns None and the run stays about parseability.
|
||||
"""
|
||||
pointer = os.path.join(job_dir, CONFIG_POINTER)
|
||||
if not os.path.isfile(pointer):
|
||||
return None
|
||||
try:
|
||||
handle = io.open(pointer, 'r', encoding='utf-8')
|
||||
except IOError:
|
||||
return None
|
||||
try:
|
||||
target = handle.read().strip()
|
||||
finally:
|
||||
handle.close()
|
||||
return target or None
|
||||
|
||||
|
||||
def walk_yaml(folder):
|
||||
"""Every YAML file under a folder, sorted, without the `.bak-` snapshots."""
|
||||
out = []
|
||||
if not os.path.isdir(folder):
|
||||
return out
|
||||
for dirpath, dirnames, filenames in os.walk(folder):
|
||||
dirnames[:] = sorted(d for d in dirnames if d not in SKIP_DIRS)
|
||||
for name in sorted(filenames):
|
||||
if name.endswith(YAML_SUFFIXES) and '.bak-' not in name:
|
||||
out.append(os.path.join(dirpath, name))
|
||||
return out
|
||||
|
||||
|
||||
def resolve(paths):
|
||||
"""Command-line arguments as a list of files. A folder means everything in it."""
|
||||
out = []
|
||||
for path in paths:
|
||||
if os.path.isdir(path):
|
||||
for found in walk_yaml(path):
|
||||
_add(out, found)
|
||||
elif os.path.isfile(path):
|
||||
_add(out, path)
|
||||
else:
|
||||
out.append(path) # kept, and reported as unreadable
|
||||
return out
|
||||
|
||||
|
||||
# --------------------------------------------------------------------- main
|
||||
|
||||
def run(paths):
|
||||
"""[(path, ok, message)] for everything asked for, in order."""
|
||||
return [(path,) + check_file(path) for path in paths]
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Strict-parse the knowledge layers. Exit 1 if one fails.')
|
||||
parser.add_argument('paths', nargs='*',
|
||||
help='files or folders; default: every standard '
|
||||
'layer location that exists on this machine')
|
||||
parser.add_argument('--list', action='store_true',
|
||||
help='print what would be checked and stop')
|
||||
parser.add_argument('--json', action='store_true', dest='as_json',
|
||||
help='machine-readable results')
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
targets = resolve(args.paths) if args.paths else default_targets()
|
||||
|
||||
if args.list:
|
||||
for path in targets:
|
||||
print(path)
|
||||
return 0
|
||||
|
||||
if not targets:
|
||||
# Not a pass, ordinarily: nothing was checked, and saying "ok" to
|
||||
# that is the exact collapse this repository keeps paying for on the
|
||||
# owner's own machines, where knowledge/catalogue.yaml is always
|
||||
# tracked and this branch should never be reachable.
|
||||
#
|
||||
# ONE EXCEPTION, and it is checked by absence of the file this
|
||||
# checkout SHOULD have plus presence of the file that stands in for
|
||||
# it on a stripped install (see tools/make_dist.py): a colleague
|
||||
# trial distribution genuinely ships no knowledge layer at all, on
|
||||
# the owner's own instruction, and "nothing to check" is the honest,
|
||||
# expected answer there -- SKIP-shaped, not FAIL-shaped, same
|
||||
# discipline tools/setup_machine.py --check applies to the same
|
||||
# state. This is the only branch where exit 0 with nothing checked
|
||||
# is not a lie.
|
||||
if (not args.paths and os.path.isfile(CATALOGUE_TEMPLATE)
|
||||
and not os.path.isfile(CATALOGUE)):
|
||||
print('no knowledge layers configured -- checks that need them '
|
||||
'stand down. This looks like a knowledge-less trial '
|
||||
'install ({0} is present in place of the real catalogue): '
|
||||
'expected here, not a failure. Ask the owner for '
|
||||
'knowledge/catalogue.yaml plus a plant/job layer to check '
|
||||
'them for real.'.format(CATALOGUE_TEMPLATE))
|
||||
return 0
|
||||
print('no knowledge layer found to check -- neither {0} nor {1} is '
|
||||
'set and this checkout has no plant or job files. Name the '
|
||||
'files to check, or set the variables.'.format(PLANT_ENV,
|
||||
JOBS_ENV))
|
||||
return 1
|
||||
|
||||
results = run(targets)
|
||||
failed = [r for r in results if not r[1]]
|
||||
|
||||
if args.as_json:
|
||||
print(json.dumps(
|
||||
{'checked': len(results), 'failed': len(failed),
|
||||
'results': [{'path': p, 'ok': ok, 'message': msg}
|
||||
for p, ok, msg in results]},
|
||||
indent=2))
|
||||
return 1 if failed else 0
|
||||
|
||||
for path, ok, message in results:
|
||||
print('{0} {1}'.format('ok ' if ok else 'FAILED', path))
|
||||
if not ok:
|
||||
print(' {0}'.format(message))
|
||||
|
||||
print('')
|
||||
if failed:
|
||||
print('{0} of {1} layer file(s) WILL NOT LOAD.'.format(len(failed),
|
||||
len(results)))
|
||||
print('pyRevit refuses the whole file, so the whole layer is dropped '
|
||||
'and every value in it silently falls back -- which looks '
|
||||
'exactly like a layer that agrees with the defaults. Fix the '
|
||||
'file; nothing will guess at what it said.')
|
||||
return 1
|
||||
print('{0} layer file(s) parse.'.format(len(results)))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,283 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Ribbon icons, drawn rather than drawn-in-a-paint-program.
|
||||
|
||||
python tools/make_icons.py
|
||||
|
||||
Needs Pillow. Writes icon.png into the pushbutton folders it knows about and
|
||||
leaves everything else alone.
|
||||
|
||||
The house style, taken from the icons that already existed:
|
||||
|
||||
96 x 96 RGBA, a rounded square of ONE flat colour filling the canvas with
|
||||
a 2px margin and a 20px corner radius, a white glyph on top, nothing else.
|
||||
No gradients, no outlines, no text.
|
||||
|
||||
Colour is the identifier. Every button on the tab gets its own and they are
|
||||
chosen to survive being 32px next to each other, which is the only size
|
||||
anybody sees:
|
||||
|
||||
ExtractDesign blue 45, 108, 178
|
||||
TicketQC green 43, 138, 90
|
||||
SelfCheck slate 86, 98, 112
|
||||
PanelLiftCheck purple 109, 76, 163
|
||||
ProjectConfig amber 198, 141, 26
|
||||
SnapshotClose indigo 46, 52, 130
|
||||
Config moss 55, 118, 45 (the pulldown, not a button)
|
||||
DesignHub plum 124, 58, 136 (the pulldown, not a button)
|
||||
PanelAnalysis seagreen 35, 139, 120 (a DesignHub row -- WIP)
|
||||
|
||||
PanelAnalysis sits at hue 169, the midpoint of the widest arc still open once
|
||||
every hue above (14, 40, 75, 112, 149, 189, 211/212/214, 235, 262, 291, 321,
|
||||
353) is plotted on the ring -- between TicketQC's green at 149 and Gravity's
|
||||
teal at 189, 20 degrees from each. The ring is full enough now that 20 is the
|
||||
best separation left anywhere on it.
|
||||
|
||||
Purple and amber went to those two because they used to sit side by side on
|
||||
one QC panel and were being pressed by mistake -- opposite hue and opposite
|
||||
value is as far apart as the palette goes. The 2026-08-30 rebuild put them on
|
||||
different panels, which fixes the confusion a second way; the colours stay as
|
||||
they are because the owner has learned them.
|
||||
|
||||
THE TWO PULLDOWNS. A `.pulldown` is a component in its own right and it draws
|
||||
its own `icon.png` (`genericcomps` resolves one for every component, and
|
||||
`uimaker._produce_ui_pulldown` hands it to `create_pulldown_button`). Without
|
||||
one it renders BLANK -- `_create_button_group` catches the icon error and only
|
||||
debug-logs it, so nothing tells you. They are drawn here for that reason, in
|
||||
the same canvas, palette and supersample as the seven above.
|
||||
|
||||
Their hues were not picked by eye. The tab already spends 14, 40, 75, 149,
|
||||
189, 211/212/214, 235, 262, 321 and 353; the two widest unused arcs in that
|
||||
ring are 75->149 and 262->321, so these two sit at their midpoints -- 112 and
|
||||
291 -- which is as far from every existing button as the ring still allows.
|
||||
Both are taken darker than their nearest neighbour (moss L .32 against
|
||||
TicketQC's .35, plum L .38 against PanelLift's .47) so the separation is a
|
||||
value difference before it is a hue difference, the same argument as
|
||||
`archive_icon`.
|
||||
|
||||
Their glyphs are containers rather than tools on purpose: neither pulldown
|
||||
does anything, it opens. Config is sliders and NOT a gear -- the gear is
|
||||
ProjectConfig, which is a row inside it, and a gear opening onto a gear reads
|
||||
as a bug. DesignHub is a hub, which promises no particular tool, which is
|
||||
right for a button whose two rows are both unbuilt.
|
||||
|
||||
Everything is drawn at 4x and resampled. Pillow has no anti-aliased
|
||||
primitives, and a 96px icon with hard edges looks broken next to the others.
|
||||
"""
|
||||
from __future__ import print_function
|
||||
|
||||
import math
|
||||
import os
|
||||
|
||||
from PIL import Image, ImageDraw
|
||||
|
||||
S = 4 # supersample factor
|
||||
N = 96 # final size
|
||||
MARGIN = 2
|
||||
RADIUS = 20
|
||||
|
||||
PURPLE = (109, 76, 163, 255)
|
||||
AMBER = (198, 141, 26, 255)
|
||||
INDIGO = (46, 52, 130, 255)
|
||||
MOSS = (55, 118, 45, 255)
|
||||
PLUM = (124, 58, 136, 255)
|
||||
SEAGREEN = (35, 139, 120, 255)
|
||||
WHITE = (255, 255, 255, 255)
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
TAB = os.path.join(HERE, os.pardir, 'WillTools.extension', 'WillTools.tab')
|
||||
|
||||
|
||||
def canvas(colour):
|
||||
im = Image.new('RGBA', (N * S, N * S), (0, 0, 0, 0))
|
||||
d = ImageDraw.Draw(im)
|
||||
d.rounded_rectangle([MARGIN * S, MARGIN * S,
|
||||
(N - MARGIN) * S, (N - MARGIN) * S],
|
||||
radius=RADIUS * S, fill=colour)
|
||||
return im, d
|
||||
|
||||
|
||||
def lift_icon():
|
||||
"""A panel hanging from two slings.
|
||||
|
||||
Sized for 32px, which is where it actually gets looked at. Three earlier
|
||||
versions lost there: inserts drawn on the panel turned into a pair of
|
||||
eyes and the whole thing read as a face -- and at 32px they were grey
|
||||
mush anyway, complexity paid for and not delivered. A deep panel read as
|
||||
a box. What survives is a thin slab under a triangle of rigging, which is
|
||||
the one silhouette nothing else on this ribbon has.
|
||||
"""
|
||||
im, d = canvas(PURPLE)
|
||||
k = S
|
||||
# shackle: an open ring, so it reads as rigging and not as a bubble
|
||||
d.ellipse([38 * k, 8 * k, 58 * k, 28 * k], outline=WHITE, width=6 * k)
|
||||
# slings, landing on the top corners of the panel
|
||||
for x in (24, 72):
|
||||
d.line([48 * k, 26 * k, x * k, 56 * k], fill=WHITE, width=6 * k)
|
||||
# the panel: thin, because a panel is
|
||||
d.rounded_rectangle([16 * k, 56 * k, 80 * k, 76 * k],
|
||||
radius=4 * k, fill=WHITE)
|
||||
return im
|
||||
|
||||
|
||||
def gear_icon():
|
||||
"""A gear: settings, and shaped like nothing else on the tab."""
|
||||
im, d = canvas(AMBER)
|
||||
k = S
|
||||
cx = cy = 48.0 * k
|
||||
teeth = 8
|
||||
r_out, r_root, r_hole = 35.0 * k, 26.5 * k, 12.0 * k
|
||||
|
||||
pts = []
|
||||
pitch = 2 * math.pi / teeth
|
||||
for i in range(teeth):
|
||||
base = i * pitch
|
||||
# flanks at 9% and 41% of the pitch, root for the rest
|
||||
for frac, r in ((0.00, r_root), (0.09, r_out),
|
||||
(0.41, r_out), (0.50, r_root)):
|
||||
a = base + frac * pitch
|
||||
pts.append((cx + r * math.cos(a), cy + r * math.sin(a)))
|
||||
d.polygon(pts, fill=WHITE)
|
||||
d.ellipse([cx - r_hole, cy - r_hole, cx + r_hole, cy + r_hole], fill=AMBER)
|
||||
return im
|
||||
|
||||
|
||||
def archive_icon():
|
||||
"""An arrow going down into an open tray: put it away, then shut the door.
|
||||
|
||||
Hue was the constraint. The tab already spends 14 (terracotta), 40
|
||||
(amber), 75 (olive), 149 (green), 189 (teal), 211/214 (blue and two
|
||||
slates), 262 (purple), 321 (magenta) and 353 (red), which leaves the
|
||||
indigo band around 238 genuinely unused. It is nearest to ExtractDesign's
|
||||
211 blue, so this one is taken much darker and much less green -- navy
|
||||
against medium blue, at 32px, is a value difference before it is a hue
|
||||
difference.
|
||||
|
||||
The glyph carries more of the load than the colour here on purpose: no
|
||||
other button on the tab has an arrow, and none has a container. The
|
||||
silhouette is legible before the colour is.
|
||||
"""
|
||||
im, d = canvas(INDIGO)
|
||||
k = S
|
||||
# the arrow: shaft, then a head wide enough to survive being 32px
|
||||
d.rectangle([40 * k, 12 * k, 56 * k, 38 * k], fill=WHITE)
|
||||
d.polygon([(28 * k, 36 * k), (68 * k, 36 * k), (48 * k, 60 * k)],
|
||||
fill=WHITE)
|
||||
# the tray: open at the top, so the arrow reads as going INTO it
|
||||
d.rectangle([16 * k, 70 * k, 80 * k, 80 * k], fill=WHITE)
|
||||
d.rectangle([16 * k, 52 * k, 26 * k, 80 * k], fill=WHITE)
|
||||
d.rectangle([70 * k, 52 * k, 80 * k, 80 * k], fill=WHITE)
|
||||
return im
|
||||
|
||||
|
||||
def sliders_icon():
|
||||
"""Three sliders at three settings: this button holds the set-up commands.
|
||||
|
||||
NOT a gear. The gear is `gear_icon`, which is ProjectConfig -- the first
|
||||
row inside this very pulldown -- and a gear that opens onto a gear reads
|
||||
as a misclick rather than a menu.
|
||||
|
||||
Three horizontal bands is the one silhouette nothing else on this tab has;
|
||||
every other glyph here is centred and roughly square. The knobs sit at
|
||||
three different positions on purpose: evenly spaced they collapse into a
|
||||
hamburger menu at 32px, which would promise a menu of anything at all
|
||||
rather than a menu of settings.
|
||||
"""
|
||||
im, d = canvas(MOSS)
|
||||
k = S
|
||||
# track, then knob, top to bottom. x is where that knob sits on its track.
|
||||
for y, x in ((28, 62), (48, 34), (68, 54)):
|
||||
d.rounded_rectangle([18 * k, (y - 3) * k, 78 * k, (y + 3) * k],
|
||||
radius=3 * k, fill=WHITE)
|
||||
# the knob is drawn in the field colour with a white ring, so it stays
|
||||
# readable where it overlaps its own track
|
||||
d.ellipse([(x - 10) * k, (y - 10) * k, (x + 10) * k, (y + 10) * k],
|
||||
fill=MOSS, outline=WHITE, width=5 * k)
|
||||
return im
|
||||
|
||||
|
||||
def hub_icon():
|
||||
"""A hub: one centre, three spokes, three nodes.
|
||||
|
||||
A container, not a tool. Both rows behind this arrow are buttons-and-docs
|
||||
only, so a glyph that promised a beam, a load or a moment would be
|
||||
claiming more than the panel delivers -- what it has to say is "there is
|
||||
more than one thing under here", and a hub says exactly that.
|
||||
|
||||
Three nodes rather than two: two reads as a dumbbell, and the count is not
|
||||
meant to be a count of the rows (there are two today and the picture must
|
||||
not go stale when a third arrives).
|
||||
"""
|
||||
im, d = canvas(PLUM)
|
||||
k = S
|
||||
cx = cy = 48.0 * k
|
||||
r_hub, r_node, reach = 13.0 * k, 9.0 * k, 30.0 * k
|
||||
|
||||
# spokes first, so the nodes sit on top of their own ends
|
||||
for a in (-math.pi / 2, math.pi / 6, 5 * math.pi / 6):
|
||||
x, y = cx + reach * math.cos(a), cy + reach * math.sin(a)
|
||||
d.line([cx, cy, x, y], fill=WHITE, width=6 * k)
|
||||
d.ellipse([x - r_node, y - r_node, x + r_node, y + r_node], fill=WHITE)
|
||||
d.ellipse([cx - r_hub, cy - r_hub, cx + r_hub, cy + r_hub], fill=WHITE)
|
||||
return im
|
||||
|
||||
|
||||
def form_icon():
|
||||
"""A page of fields, with a small filled tag on one of them.
|
||||
|
||||
This button pre-fills an Eriksson-shaped input form from the model
|
||||
rather than checking anything, which is a different promise than every
|
||||
other glyph on this tab makes -- none of them is a page. Three plain
|
||||
field lines read as a form; the short solid tag beside the last one --
|
||||
smaller than the field it sits beside, so it reads as attached to it
|
||||
rather than as a fourth line -- is the source chip this button's whole
|
||||
idea rests on.
|
||||
"""
|
||||
im, d = canvas(SEAGREEN)
|
||||
k = S
|
||||
d.rounded_rectangle([20 * k, 12 * k, 76 * k, 84 * k],
|
||||
radius=6 * k, fill=WHITE)
|
||||
for y in (30, 48, 66):
|
||||
d.rounded_rectangle([30 * k, (y - 3) * k, 58 * k, (y + 3) * k],
|
||||
radius=3 * k, fill=SEAGREEN)
|
||||
# the chip: a small solid tag beside the bottom field, in the page's own
|
||||
# field colour, standing for the source badge every pre-filled value on
|
||||
# the real page carries.
|
||||
d.rounded_rectangle([62 * k, 63 * k, 74 * k, 69 * k], radius=3 * k,
|
||||
fill=SEAGREEN)
|
||||
return im
|
||||
|
||||
|
||||
# Paths are relative to TAB and they are panel structure, which moves: the
|
||||
# 2026-08-30 rebuild put two of the first three inside a `*.stack`, and the
|
||||
# 2026-08-30 one moved all of them again and added the two pulldowns. `main()`
|
||||
# prints "skipped, no such button" rather than failing, so a stale line here
|
||||
# is silent -- check the output, not the exit.
|
||||
#
|
||||
# The last two are `.pulldown` folders, not buttons. That is not a mistake:
|
||||
# see the module docstring. Everything else on the tab keeps the icon it
|
||||
# already has and is deliberately not regenerated from here.
|
||||
ICONS = (
|
||||
('Tools.panel/PanelLiftCheck.pushbutton', lift_icon),
|
||||
('Setup.panel/Config.pulldown/ProjectConfig.pushbutton', gear_icon),
|
||||
('Tools.panel/SnapshotClose.pushbutton', archive_icon),
|
||||
('Setup.panel/Config.pulldown', sliders_icon),
|
||||
('Design.panel/DesignHub.pulldown', hub_icon),
|
||||
('Design.panel/DesignHub.pulldown/PanelAnalysis.pushbutton', form_icon),
|
||||
)
|
||||
|
||||
|
||||
def main():
|
||||
for where, draw in ICONS:
|
||||
folder = os.path.join(TAB, *where.split('/'))
|
||||
if not os.path.isdir(folder):
|
||||
# Still not an error: the ribbon is allowed to move without this
|
||||
# script being run. But nobody reads an exit code, so say it.
|
||||
print('skipped, no such component: %s' % where)
|
||||
continue
|
||||
path = os.path.join(folder, 'icon.png')
|
||||
draw().resize((N, N), Image.LANCZOS).save(path, 'PNG', optimize=True)
|
||||
print('%s (%d bytes)' % (where, os.path.getsize(path)))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,910 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""The Panel Analysis deck schema -- `panelanalysis/1`, and how it maps.
|
||||
|
||||
WHY A SECOND SCHEMA. `panellift/1` (see `panellift.cli`'s docstring) is a
|
||||
HANDLING job: a panel, a rig, and PCI Table 8.3.1 stages. Panel Analysis is a
|
||||
different report -- Eriksson Wall's structural analysis of a wall panel -- and
|
||||
it is organised the way Eriksson organises its own input, in named blocks, so
|
||||
that later stages can fill each block in without re-shaping the file:
|
||||
|
||||
project_information mark, design number, materials (f'c / f'ci), losses.
|
||||
settings analysis options: stress limits, resistance factors,
|
||||
development multipliers. Seeded from knowledge in a
|
||||
later stage; a Stage 0 fixture hand-authors them.
|
||||
concrete_extents the panel as a section: length / width / thickness,
|
||||
finish, product type, and the openings. Section
|
||||
properties come off this at L/10, Eriksson's own
|
||||
convention.
|
||||
structural_model how the piece is held. `supports` is a list of
|
||||
{x_ft, type} -- the dunnage / bearing lines the
|
||||
Handling beam spans. `restraints` and `bearing` are
|
||||
seeded from clash detection in a later stage (Stage
|
||||
1); here they carry an explicit null + reason.
|
||||
reinforcement `prestress`: the strands, exactly the shape
|
||||
`panellift.cli.read_prestress` already reads.
|
||||
loading `self_weight` is the only real load now (a factor and
|
||||
an on/off); `wind` and `combinations` are the
|
||||
in-place engine (Stage 3) and carry null + reason.
|
||||
handling the Handling result block -- self-weight beam on the
|
||||
`structural_model.supports`. Computed and rendered by
|
||||
`panelanalysis.py`; in a deck it is an empty result
|
||||
placeholder the tool fills.
|
||||
in_place the In-Place result block -- panel on pin supports
|
||||
under lateral + gravity. NOT built (Stage 3): an
|
||||
explicit null + reason, never fabricated zeros.
|
||||
source provenance. For a Stage 0 fixture it is synthetic and
|
||||
says so.
|
||||
|
||||
THE FILE CONTRACT, unchanged from `panellift`: a field that was gathered but
|
||||
not read is an EXPLICIT null carrying its reason, never a silent omission and
|
||||
never a fabricated value. `unread(reason)` is the one shape those take --
|
||||
`{"value": null, "reason": "..."}` -- so a reader (and a test) can tell "nobody
|
||||
has wired this yet" apart from "this is genuinely empty". See
|
||||
`willtools-error-patterns` / the whole of CLAUDE.md sec.3 on why a silent zero
|
||||
is the cardinal sin here.
|
||||
|
||||
HOW IT REACHES THE ENGINE. Nothing in this file re-implements any mechanics.
|
||||
`to_panel()` translates the blocks into the panel dict `panellift.cli.
|
||||
read_panel` already validates, so every refusal that function makes (an empty
|
||||
strand list, a strand with no pull, a bad number) is made here too, unchanged.
|
||||
`handling_stage()` builds a `panellift.model.Stage` whose `supports` are the
|
||||
`structural_model.supports` x-positions -- which is exactly the dunnage path
|
||||
`panellift.statics.analyse` already solves as a continuous beam on rigid
|
||||
supports. The support `type` (pinned / roller / fixed) is carried for the page
|
||||
to show; the Handling solve treats every support as a rigid line, which is what
|
||||
that path is.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
REPO = os.path.abspath(os.path.join(HERE, os.pardir))
|
||||
PANELLIFT_SRC = os.path.join(REPO, 'panellift', 'src')
|
||||
if PANELLIFT_SRC not in sys.path:
|
||||
sys.path.insert(0, PANELLIFT_SRC)
|
||||
|
||||
from panellift import cli as pl_cli # noqa: E402
|
||||
from panellift import inplace # noqa: E402
|
||||
from panellift import windload # noqa: E402
|
||||
from panellift.model import Reinforcement, Rigging, Stage # noqa: E402
|
||||
|
||||
# The wind PRECEDENCE (direct > computed > blank) is the form's pure logic,
|
||||
# host-tested in `vibe.panelanalysisform`; reuse it here rather than restating
|
||||
# it. The `vibe` stub is already registered by `deckcheck` (imported first in
|
||||
# `panelanalysis.py`) and by the test conftest, so this bare import resolves the
|
||||
# Revit-free module the same way the tests do.
|
||||
from vibe import panelanalysisform as _paf # noqa: E402
|
||||
|
||||
|
||||
SCHEMA = 'panelanalysis/1'
|
||||
|
||||
# The blocks, in the order the page prints them. Named here so the tool and a
|
||||
# test can agree on the set without either one hardcoding a list of strings.
|
||||
BLOCKS = (
|
||||
'project_information',
|
||||
'settings',
|
||||
'concrete_extents',
|
||||
'structural_model',
|
||||
'reinforcement',
|
||||
'loading',
|
||||
'handling',
|
||||
'in_place',
|
||||
'source',
|
||||
)
|
||||
|
||||
# The default handling load factor. 1.0 is the bare self weight -- the Handling
|
||||
# model is "the panel's own weight on its dunnage / bearing lines", and a stage
|
||||
# that wants PCI Table 8.3.1's stripping/transport factor states it in
|
||||
# `loading.self_weight.load_factor` rather than having one guessed here.
|
||||
DEFAULT_SELF_WEIGHT_FACTOR = 1.0
|
||||
|
||||
|
||||
class DeckError(ValueError):
|
||||
"""The panelanalysis deck does not describe a panel that can be analysed.
|
||||
|
||||
A subclass of ValueError, so a caller that already catches the ValueError
|
||||
`panellift` raises (the <2-supports refusal) catches this the same way.
|
||||
"""
|
||||
|
||||
|
||||
def unread(reason, value=None):
|
||||
"""The one shape a gathered-but-unread field takes: null + why.
|
||||
|
||||
`{"value": None, "reason": "..."}`. A reader tells this apart from a field
|
||||
nobody thought of (absent) and from a real empty value; a test asserts the
|
||||
reason is present rather than that the key merely exists. `value` is kept
|
||||
so a partially-known field can carry what little is known beside the reason,
|
||||
but for Stage 0 it is None everywhere.
|
||||
"""
|
||||
return {'value': value, 'reason': str(reason)}
|
||||
|
||||
|
||||
# ----------------------------------------------------------------- reading
|
||||
|
||||
def _block(deck, name):
|
||||
block = deck.get(name)
|
||||
if block is None:
|
||||
return {}
|
||||
if not isinstance(block, dict):
|
||||
raise DeckError('block {0!r} must be a mapping, got {1!r}'
|
||||
.format(name, block))
|
||||
return block
|
||||
|
||||
|
||||
def check_schema(deck):
|
||||
"""Refuse a deck that is not this schema, by name. Absent is allowed.
|
||||
|
||||
A file with no `schema` key is taken as this one (the same leniency
|
||||
`panellift` gives its own legacy files); a file naming a DIFFERENT schema
|
||||
is refused rather than misread -- a `panellift/1` job has no
|
||||
`structural_model` block and reading it here would silently analyse a piece
|
||||
on no supports.
|
||||
"""
|
||||
schema = deck.get('schema')
|
||||
if schema and schema != SCHEMA:
|
||||
raise DeckError('deck says schema {0!r}; this reads {1!r}'
|
||||
.format(schema, SCHEMA))
|
||||
|
||||
|
||||
def to_panel(deck):
|
||||
"""A `panellift.model.Panel` from the deck's blocks.
|
||||
|
||||
Built by translating the blocks into the panel dict `read_panel` already
|
||||
validates, so every refusal that function makes is made here unchanged --
|
||||
this file adds no second opinion about what a valid panel is.
|
||||
"""
|
||||
pi = _block(deck, 'project_information')
|
||||
ce = _block(deck, 'concrete_extents')
|
||||
rf = _block(deck, 'reinforcement')
|
||||
panel_dict = {
|
||||
'mark': pi.get('mark'),
|
||||
'design_number': pi.get('design_number'),
|
||||
'length_ft': ce.get('length_ft'),
|
||||
'width_ft': ce.get('width_ft'),
|
||||
'thickness_in': ce.get('thickness_in'),
|
||||
'unit_weight_pcf': pi.get('unit_weight_pcf', 150.0),
|
||||
'fc_psi': pi.get('fc_psi'),
|
||||
'fci_psi': pi.get('fci_psi'),
|
||||
'weight_lb': ce.get('weight_lb'),
|
||||
'extra_cast_ft3': ce.get('extra_cast_ft3') or 0.0,
|
||||
'finish': ce.get('finish') or 'smooth',
|
||||
'product_type': ce.get('product_type') or 'flat',
|
||||
'blockouts': ce.get('blockouts') or [],
|
||||
'recesses': ce.get('recesses') or [],
|
||||
# The Handling model here is a beam on dunnage supports, not a pick, so
|
||||
# the panel carries no lifters -- the supports live on the stage.
|
||||
'lifters': [],
|
||||
'prestress': rf.get('prestress'),
|
||||
'source': deck.get('source') or {},
|
||||
}
|
||||
try:
|
||||
panel = pl_cli.read_panel(panel_dict)
|
||||
except pl_cli.JobError as exc:
|
||||
# Re-raise as our own type so a caller catches one exception, but keep
|
||||
# the message -- it names the exact field `read_panel` refused.
|
||||
raise DeckError(str(exc))
|
||||
# Mild (non-prestressed) reinforcement, so the flexure check has a mild
|
||||
# capacity path for a rebar wall panel. `read_panel` does not read it --
|
||||
# `panellift.cli` is strand-only by construction and a test pins that -- so
|
||||
# the deck's mild layers are attached here, onto `Panel.reinforcement`
|
||||
# (which the model already carries and `capacity.mild_flexure_at` reads).
|
||||
# The strand path is untouched; a pure-strand or bare deck gets [].
|
||||
panel.reinforcement = mild_reinforcement(deck)
|
||||
return panel
|
||||
|
||||
|
||||
def mild_reinforcement(deck):
|
||||
"""[`model.Reinforcement`] -- the deck's mild bar / mesh layers, or [].
|
||||
|
||||
Read off `reinforcement.mild`, a list of layers each
|
||||
`{area_in2, z_in, grade_ksi, count?, kind?, label?}`. `area_in2` is the
|
||||
area of ONE bar and `count` how many are in the layer (total = area*count,
|
||||
the shape `model.Reinforcement` already uses); `z_in` is the layer's depth
|
||||
up from the form face, the same axis the strands use, so `capacity` can
|
||||
compare it against a neutral axis directly. `grade_ksi` is the yield f_y.
|
||||
|
||||
A missing block is a piece with no mild steel and returns [] -- NOT an
|
||||
error, the ordinary case for a prestressed piece. A layer missing a
|
||||
required number is refused loudly, the same discipline `read_panel` uses,
|
||||
rather than silently dropped (a dropped tension layer overstates nothing
|
||||
but understates capacity, which is a FAIL the section does not have).
|
||||
"""
|
||||
rf = _block(deck, 'reinforcement')
|
||||
items = rf.get('mild')
|
||||
if items is None:
|
||||
return []
|
||||
if not isinstance(items, list):
|
||||
raise DeckError('reinforcement.mild must be a list of layers, got '
|
||||
'{0!r}'.format(items))
|
||||
out = []
|
||||
for i, r in enumerate(items):
|
||||
at = 'reinforcement.mild[{0}]'.format(i)
|
||||
if not isinstance(r, dict):
|
||||
raise DeckError('{0} must be a mapping, got {1!r}'.format(at, r))
|
||||
for key in ('area_in2', 'z_in', 'grade_ksi'):
|
||||
if r.get(key) is None:
|
||||
raise DeckError('{0} has no {1!r}; a mild layer needs area_in2 '
|
||||
'(per bar), z_in (depth up from the form face) '
|
||||
'and grade_ksi (f_y)'.format(at, key))
|
||||
try:
|
||||
out.append(Reinforcement(
|
||||
kind=str(r.get('kind') or 'bar'),
|
||||
label=str(r.get('label') or 'M{0}'.format(i + 1)),
|
||||
area_in2=float(r['area_in2']),
|
||||
z_in=float(r['z_in']),
|
||||
grade_ksi=float(r['grade_ksi']),
|
||||
count=int(r.get('count', 1))))
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise DeckError('{0}: a mild-layer field is not a number ({1})'
|
||||
.format(at, exc))
|
||||
return out
|
||||
|
||||
|
||||
def supports(deck):
|
||||
"""[(x_ft, type)], in the order written. The Handling beam's supports.
|
||||
|
||||
`type` is carried for the page; the solve treats every one as a rigid
|
||||
line. A missing/short list is NOT refused here -- the refusal belongs to
|
||||
the solve, which raises `panellift`'s own "a supported stage needs at least
|
||||
two supports" so the page can surface that exact sentence as NOT JUDGED
|
||||
rather than this file inventing a parallel message.
|
||||
"""
|
||||
sm = _block(deck, 'structural_model')
|
||||
out = []
|
||||
for i, s in enumerate(sm.get('supports') or []):
|
||||
if not isinstance(s, dict):
|
||||
raise DeckError('structural_model.supports[{0}] must be a mapping '
|
||||
'with an "x_ft", got {1!r}'.format(i, s))
|
||||
if s.get('x_ft') is None:
|
||||
raise DeckError('structural_model.supports[{0}] has no "x_ft"'
|
||||
.format(i))
|
||||
try:
|
||||
x = float(s['x_ft'])
|
||||
except (TypeError, ValueError):
|
||||
raise DeckError('structural_model.supports[{0}].x_ft is not a '
|
||||
'number ({1!r})'.format(i, s['x_ft']))
|
||||
out.append((x, str(s.get('type') or 'support')))
|
||||
return out
|
||||
|
||||
|
||||
def self_weight_factor(deck):
|
||||
"""The Handling load factor from `loading.self_weight`, or the default."""
|
||||
sw = (_block(deck, 'loading').get('self_weight')) or {}
|
||||
if not isinstance(sw, dict):
|
||||
return DEFAULT_SELF_WEIGHT_FACTOR
|
||||
factor = sw.get('load_factor')
|
||||
if factor is None:
|
||||
return DEFAULT_SELF_WEIGHT_FACTOR
|
||||
try:
|
||||
return float(factor)
|
||||
except (TypeError, ValueError):
|
||||
raise DeckError('loading.self_weight.load_factor is not a number '
|
||||
'({0!r})'.format(factor))
|
||||
|
||||
|
||||
def handling_fc_psi(panel):
|
||||
"""The concrete strength the Handling model is checked at.
|
||||
|
||||
f'ci -- the release strength -- when the deck states one, because handling
|
||||
(stripping, yard) happens on young concrete and that is the conservative
|
||||
of the two. f'c otherwise. Returned separately from the panel so the caller
|
||||
can label the page with which one it used, and so the capacity table can
|
||||
also run the 28-day strength beside it.
|
||||
"""
|
||||
return panel.fci_psi or panel.fc_psi or 0.0
|
||||
|
||||
|
||||
# The stress limits are a JOB decision resolved through the knowledge layers
|
||||
# (see `Stage.tension_limit_sqrt_fc` / `.compression_limit_ratio`), and the
|
||||
# deck carries whatever they resolved to in its `settings` block. Until now
|
||||
# nothing read them; the Handling solve built a Stage with the panellift
|
||||
# DEFAULTS (5.0, 0.6) even when the deck plainly stated otherwise. These pull
|
||||
# them off `settings`, falling back to the panellift default when the deck is
|
||||
# silent -- so a deck with no settings behaves exactly as before.
|
||||
DEFAULT_TENSION_LIMIT_SQRT_FC = 5.0
|
||||
DEFAULT_COMPRESSION_LIMIT_RATIO = 0.6
|
||||
|
||||
|
||||
def _settings_number(deck, key, default):
|
||||
st = _block(deck, 'settings').get(key)
|
||||
if st is None:
|
||||
return default
|
||||
try:
|
||||
return float(st)
|
||||
except (TypeError, ValueError):
|
||||
raise DeckError('settings.{0} is not a number ({1!r})'.format(key, st))
|
||||
|
||||
|
||||
def tension_limit_sqrt_fc(deck):
|
||||
return _settings_number(deck, 'tension_limit_sqrt_fc',
|
||||
DEFAULT_TENSION_LIMIT_SQRT_FC)
|
||||
|
||||
|
||||
def compression_limit_ratio(deck):
|
||||
return _settings_number(deck, 'compression_limit_ratio',
|
||||
DEFAULT_COMPRESSION_LIMIT_RATIO)
|
||||
|
||||
|
||||
def development_length_multiplier(deck):
|
||||
"""Eriksson's 'Development Multipliers' setting -- 1.0 unless stated.
|
||||
|
||||
It lengthens the transfer/development region the phi ramp reads across, so
|
||||
a deck that carries 1.3 holds phi below 0.90 further from the piece ends.
|
||||
"""
|
||||
return _settings_number(deck, 'development_length_multiplier', 1.0)
|
||||
|
||||
|
||||
def _stage(deck, panel, name, pci_case, multiplier, fc_psi):
|
||||
"""A supported Stage carrying THIS deck's stress limits, not the defaults.
|
||||
|
||||
`tension_limit_source` / `compression_limit_source` are 'job' when the deck
|
||||
stated a limit and 'default' when it did not, exactly the provenance
|
||||
`Stage` documents -- so the page can say who set the allowable it printed.
|
||||
"""
|
||||
st = _block(deck, 'settings')
|
||||
xs = [x for x, _type in supports(deck)]
|
||||
return Stage(
|
||||
name=name, multiplier=multiplier, fc_psi=fc_psi, pci_case=pci_case,
|
||||
supports=xs,
|
||||
tension_limit_sqrt_fc=tension_limit_sqrt_fc(deck),
|
||||
tension_limit_source=('job' if st.get('tension_limit_sqrt_fc')
|
||||
is not None else 'default'),
|
||||
compression_limit_ratio=compression_limit_ratio(deck),
|
||||
compression_limit_source=('job' if st.get('compression_limit_ratio')
|
||||
is not None else 'default'))
|
||||
|
||||
|
||||
def handling_stage(deck, panel):
|
||||
"""A `panellift.model.Stage` for the self-weight Handling solve.
|
||||
|
||||
`supports` set means `panellift.statics.analyse` runs the dunnage path: a
|
||||
continuous beam on rigid supports, self weight only. The x-positions come
|
||||
straight off `structural_model.supports`; nothing else about the stage
|
||||
matters to the solve except `fc_psi`, which sets the elastic modulus for
|
||||
the (indeterminate, 3+-support) load distribution and the strength the
|
||||
capacity columns are measured at.
|
||||
|
||||
The multiplier is `loading.self_weight.load_factor` (the deck's own
|
||||
stripping factor), the strength is f'ci, and the stress limits are the
|
||||
deck's settings -- see `_stage`. Kept as the single-stage entry point the
|
||||
page's Handling section and the parity harness already call; the two-stage
|
||||
default is `handling_stages`.
|
||||
"""
|
||||
return _stage(deck, panel, name='handling (self weight)',
|
||||
pci_case='stripping', multiplier=self_weight_factor(deck),
|
||||
fc_psi=handling_fc_psi(panel))
|
||||
|
||||
|
||||
# PCI/Eriksson default handling stages (docs/eriksson-wall-algorithms.md sec.7,
|
||||
# QC p.133 `gamma_strip=1.4, gamma_transport=1.5, gamma_erection=1.2`):
|
||||
# stripping at the release strength f'ci with the stripping dynamic factor,
|
||||
# transportation AND erection at the 28-day strength f'c with their own
|
||||
# factors. Overridable per deck from the `handling` block -- a `handling.stages`
|
||||
# list of {name, pci_case, multiplier, strength} entries, or
|
||||
# `handling.stripping_multiplier` / `.transport_multiplier` / `.erection_
|
||||
# multiplier` to nudge just the factors. Defaults per the reference tool.
|
||||
DEFAULT_STRIPPING_MULTIPLIER = 1.4
|
||||
DEFAULT_TRANSPORT_MULTIPLIER = 1.5
|
||||
DEFAULT_ERECTION_MULTIPLIER = 1.2
|
||||
|
||||
|
||||
def _strength_for(panel, which):
|
||||
"""'release' -> f'ci (falling back to f'c); '28day'/'service' -> f'c."""
|
||||
if which in ('release', 'stripping', 'fci'):
|
||||
return panel.fci_psi or panel.fc_psi or 0.0
|
||||
return panel.fc_psi or panel.fci_psi or 0.0
|
||||
|
||||
|
||||
def handling_stages(deck, panel):
|
||||
"""The handling stages to run, each a `Stage` carrying the deck's limits.
|
||||
|
||||
Default is the two Eriksson runs -- Stripping x1.4 at f'ci, Transportation
|
||||
x1.5 at f'c -- and the `loading.self_weight.load_factor`, if the deck
|
||||
states one, overrides the stripping multiplier so a deck hand-tuned for one
|
||||
factor still drives it. A `handling.stages` list replaces the defaults
|
||||
outright; `handling.stripping_multiplier` / `handling.transport_multiplier`
|
||||
nudge just those numbers.
|
||||
|
||||
A stage whose strength comes out 0 (a deck stating neither f'c nor f'ci) is
|
||||
dropped rather than run at no strength -- the same refuse-loudly discipline
|
||||
the solve uses, one layer earlier.
|
||||
"""
|
||||
handling = _block(deck, 'handling')
|
||||
out = []
|
||||
|
||||
override = handling.get('stages')
|
||||
if override is not None:
|
||||
if not isinstance(override, list):
|
||||
raise DeckError('handling.stages must be a list, got {0!r}'
|
||||
.format(override))
|
||||
for i, s in enumerate(override):
|
||||
if not isinstance(s, dict):
|
||||
raise DeckError('handling.stages[{0}] must be a mapping, got '
|
||||
'{1!r}'.format(i, s))
|
||||
fc = _strength_for(panel, s.get('strength') or 'release')
|
||||
if not fc:
|
||||
continue
|
||||
out.append(_stage(
|
||||
deck, panel, name=s.get('name') or 'stage {0}'.format(i + 1),
|
||||
pci_case=s.get('pci_case') or 'stripping',
|
||||
multiplier=float(s.get('multiplier', DEFAULT_STRIPPING_MULTIPLIER)),
|
||||
fc_psi=fc))
|
||||
return out
|
||||
|
||||
# A deck-stated self-weight factor is the stripping factor the page has
|
||||
# always driven; keep honouring it so nothing regresses.
|
||||
sw = self_weight_factor(deck)
|
||||
strip_mult = handling.get('stripping_multiplier')
|
||||
strip_mult = (float(strip_mult) if strip_mult is not None
|
||||
else (sw if sw != DEFAULT_SELF_WEIGHT_FACTOR
|
||||
else DEFAULT_STRIPPING_MULTIPLIER))
|
||||
trans_mult = handling.get('transport_multiplier')
|
||||
trans_mult = (float(trans_mult) if trans_mult is not None
|
||||
else DEFAULT_TRANSPORT_MULTIPLIER)
|
||||
erect_mult = handling.get('erection_multiplier')
|
||||
erect_mult = (float(erect_mult) if erect_mult is not None
|
||||
else DEFAULT_ERECTION_MULTIPLIER)
|
||||
|
||||
fci = _strength_for(panel, 'release')
|
||||
if fci:
|
||||
out.append(_stage(deck, panel, name='Stripping', pci_case='stripping',
|
||||
multiplier=strip_mult, fc_psi=fci))
|
||||
fc = _strength_for(panel, '28day')
|
||||
if fc:
|
||||
out.append(_stage(deck, panel, name='Transportation',
|
||||
pci_case='transportation', multiplier=trans_mult,
|
||||
fc_psi=fc))
|
||||
out.append(_stage(deck, panel, name='Erection',
|
||||
pci_case='erection', multiplier=erect_mult,
|
||||
fc_psi=fc))
|
||||
return out
|
||||
|
||||
|
||||
def handling_rigging():
|
||||
"""The rigging object the dunnage path needs but does not read.
|
||||
|
||||
`statics.analyse` takes a `Rigging`; on the supports path it is only
|
||||
stored on the returned `Analysis` (no cable groups, no sling angle). Static
|
||||
mode names what a trailer/dunnage actually is.
|
||||
"""
|
||||
return Rigging(mode='static')
|
||||
|
||||
|
||||
# ------------------------------------------------------------- in-place (stage3)
|
||||
#
|
||||
# The In-Place model inputs -- lateral restraints, wind pressures, applied axial
|
||||
# loads and the load combinations -- read off the deck's structural_model and
|
||||
# loading blocks. Every one is OPTIONAL and STAGE-3 tagged: a deck that has not
|
||||
# had them wired carries the `unread(...)` shape (value null + reason) these
|
||||
# readers already understand, and the page keeps its explicit placeholder rather
|
||||
# than fabricating an in-place result. The handling path is untouched.
|
||||
|
||||
|
||||
def _is_unread(block):
|
||||
"""True for the `{value: None, reason: ...}` gathered-but-unread shape."""
|
||||
return (isinstance(block, dict) and 'reason' in block
|
||||
and block.get('value') is None and 'x_ft' not in block
|
||||
and 'zones' not in block)
|
||||
|
||||
|
||||
def restraints(deck):
|
||||
"""([x_ft], reason) -- the lateral restraints the in-place panel spans.
|
||||
|
||||
A list of {x_ft, type} under `structural_model.restraints`, OR the
|
||||
`unread(...)` placeholder (-> [], reason). The in-place solve refuses fewer
|
||||
than two the same way the handling solve does; this reader does not
|
||||
second-guess it, it only parses.
|
||||
"""
|
||||
sm = _block(deck, 'structural_model')
|
||||
blk = sm.get('restraints')
|
||||
if blk is None:
|
||||
return [], 'no structural_model.restraints on this deck'
|
||||
if _is_unread(blk):
|
||||
return [], blk.get('reason') or 'restraints not wired yet'
|
||||
items = blk.get('value') if isinstance(blk, dict) and 'value' in blk else blk
|
||||
if not isinstance(items, list):
|
||||
raise DeckError('structural_model.restraints must be a list of '
|
||||
'{{x_ft, type}} or an unread placeholder, got {0!r}'
|
||||
.format(blk))
|
||||
out = []
|
||||
for i, r in enumerate(items):
|
||||
if not isinstance(r, dict) or r.get('x_ft') is None:
|
||||
raise DeckError('structural_model.restraints[{0}] needs an "x_ft"'
|
||||
.format(i))
|
||||
try:
|
||||
out.append(float(r['x_ft']))
|
||||
except (TypeError, ValueError):
|
||||
raise DeckError('structural_model.restraints[{0}].x_ft is not a '
|
||||
'number ({1!r})'.format(i, r['x_ft']))
|
||||
return out, None
|
||||
|
||||
|
||||
def compute_asce_pressures(deck):
|
||||
"""(windload.WindResult, reason). Runs `windload.compute` off loading.wind.asce.
|
||||
|
||||
(None, reason) when there is nothing to compute from -- no `asce` block, or
|
||||
one missing the numeric inputs `windload.compute` cannot run without (Vult,
|
||||
mean roof height, effective wind area). A WindResult whose `.not_judged` is
|
||||
set is returned AS the result (h>60, un-mined ASCE version, bad selectors):
|
||||
the reason travels on it, never fabricated into a pressure. This is the
|
||||
CPython deck-solve step that owns the ASCE arithmetic -- `windload` is
|
||||
CPython and cannot be reached from the IronPython form.
|
||||
"""
|
||||
blk = _block(deck, 'loading').get('wind')
|
||||
if not isinstance(blk, dict):
|
||||
return None, 'no loading.wind block'
|
||||
a = blk.get('asce')
|
||||
if not isinstance(a, dict):
|
||||
return None, (blk.get('asce_incomplete')
|
||||
or 'no loading.wind.asce inputs to compute from')
|
||||
|
||||
def _f(key):
|
||||
v = a.get(key)
|
||||
try:
|
||||
return float(v) if v is not None and v != '' else None
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
v_ult = _f('v_ult_mph')
|
||||
h = _f('mean_roof_height_ft')
|
||||
area = _f('effective_wind_area_ft2')
|
||||
if v_ult is None or h is None or area is None:
|
||||
return None, ('loading.wind.asce needs numeric v_ult_mph, '
|
||||
'mean_roof_height_ft and effective_wind_area_ft2')
|
||||
cc = a.get('cc_chapter')
|
||||
try:
|
||||
cc_chapter = (int(float(cc)) if cc not in (None, '')
|
||||
else windload.CC_CHAPTER)
|
||||
except (TypeError, ValueError):
|
||||
cc_chapter = windload.CC_CHAPTER
|
||||
kzt = _f('kzt')
|
||||
res = windload.compute(
|
||||
v_ult_mph=v_ult,
|
||||
risk_category=a.get('risk_category'),
|
||||
exposure=a.get('exposure'),
|
||||
enclosure=a.get('enclosure'),
|
||||
code_version=a.get('code_version'),
|
||||
mean_roof_height_ft=h,
|
||||
effective_wind_area_ft2=area,
|
||||
cc_chapter=cc_chapter,
|
||||
kzt=kzt if kzt is not None else 1.0)
|
||||
return res, None
|
||||
|
||||
|
||||
def _governing_pressures(res):
|
||||
"""(governing_mag, positive_psf, negative_psf, uncertainty_psf) across the
|
||||
C&C zones, or None where the result is not_judged / empty.
|
||||
|
||||
The in-place model runs a single out-of-plane pressure; a straight wall
|
||||
bends both ways, so the GOVERNING magnitude is what matters -- the largest
|
||||
|p| over Zones 4/5, positive or negative (suction).
|
||||
"""
|
||||
if res is None or res.not_judged or not res.zones:
|
||||
return None
|
||||
pos = max(z.positive_psf for z in res.zones)
|
||||
neg = min(z.negative_psf for z in res.zones) # most negative
|
||||
unc = max(z.uncertainty_psf for z in res.zones)
|
||||
return max(abs(pos), abs(neg)), pos, neg, unc
|
||||
|
||||
|
||||
def resolve_wind(deck):
|
||||
"""The in-place wind-pressure DECISION + provenance, or None.
|
||||
|
||||
None when `loading.wind` is absent/unread, or carries an explicit `zones`
|
||||
list (the golden given-pressure path, handled by `wind_zones` directly).
|
||||
Otherwise a dict from `panelanalysisform.resolve_wind_pressure` (the pure
|
||||
direct > computed > blank precedence) enriched for the page:
|
||||
|
||||
source 'direct' | 'computed' | 'not_judged' | 'blank'
|
||||
pressure_psf governing magnitude fed to the in-place wind case
|
||||
positive_psf, the C&C design values, for display (computed path)
|
||||
negative_psf
|
||||
uncertainty_psf +/- band from the figure-read GCp (computed path)
|
||||
cite, gcpi, windload provenance (computed path)
|
||||
gcpi_cite
|
||||
result the windload.WindResult (computed path) or None
|
||||
reason not-judged / blank reason (else None)
|
||||
"""
|
||||
blk = _block(deck, 'loading').get('wind')
|
||||
if not isinstance(blk, dict) or _is_unread(blk) or 'zones' in blk:
|
||||
return None
|
||||
|
||||
direct_p = blk.get('direct_pressure_psf')
|
||||
direct_s = blk.get('direct_suction_psf')
|
||||
dvals = []
|
||||
for v in (direct_p, direct_s):
|
||||
if v is not None:
|
||||
try:
|
||||
dvals.append(abs(float(v)))
|
||||
except (TypeError, ValueError):
|
||||
pass
|
||||
direct_mag = max(dvals) if dvals else None
|
||||
|
||||
res, reason = compute_asce_pressures(deck)
|
||||
gov = _governing_pressures(res)
|
||||
computed_mag = gov[0] if gov else None
|
||||
computed_reason = None
|
||||
if res is not None and res.not_judged:
|
||||
computed_reason = res.not_judged
|
||||
elif computed_mag is None and reason:
|
||||
computed_reason = reason
|
||||
|
||||
info = dict(_paf.resolve_wind_pressure(direct_mag, computed_mag,
|
||||
computed_reason))
|
||||
info['result'] = res
|
||||
info['positive_psf'] = gov[1] if gov else None
|
||||
info['negative_psf'] = gov[2] if gov else None
|
||||
info['uncertainty_psf'] = gov[3] if gov else None
|
||||
info['direct_pressure_psf'] = _num_or_none(direct_p)
|
||||
info['direct_suction_psf'] = _num_or_none(direct_s)
|
||||
if res is not None and res.gcpi is not None:
|
||||
info['gcpi'], info['gcpi_cite'] = res.gcpi.value, res.gcpi.cite
|
||||
else:
|
||||
info['gcpi'], info['gcpi_cite'] = None, None
|
||||
info['cite'] = res.zones[0].basis if (res is not None and res.zones) else None
|
||||
return info
|
||||
|
||||
|
||||
def _num_or_none(v):
|
||||
if v is None:
|
||||
return None
|
||||
try:
|
||||
return float(v)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def wind_zones(deck, panel=None):
|
||||
"""([LateralPressureZone], tributary_width_ft, reason).
|
||||
|
||||
Three ways a deck states the out-of-plane wind, in priority order:
|
||||
|
||||
* an explicit `zones` list -- {x0_ft, x1_ft, pressure_psf, label} -- the
|
||||
GIVEN-pressure path the goldens and fixtures use, untouched;
|
||||
* the ASCE inputs (`asce`) and/or a direct-entry pressure, which
|
||||
`resolve_wind` turns into ONE governing pressure over the panel length
|
||||
(direct > computed > blank) -- this is the form's path;
|
||||
* the `unread(...)` placeholder -> ([], None, reason).
|
||||
|
||||
The pressure the zone carries is the direct entry if the user gave one, else
|
||||
windload's computed C&C value, else nothing (a reason, no fabricated zone).
|
||||
"""
|
||||
blk = _block(deck, 'loading').get('wind')
|
||||
if blk is None:
|
||||
return [], None, 'no loading.wind on this deck'
|
||||
if _is_unread(blk):
|
||||
return [], None, blk.get('reason') or 'wind not wired yet'
|
||||
|
||||
trib = blk.get('tributary_width_ft')
|
||||
trib = float(trib) if trib is not None else None
|
||||
|
||||
if isinstance(blk, dict) and 'zones' in blk:
|
||||
zdefs = blk.get('zones')
|
||||
if not isinstance(zdefs, list):
|
||||
raise DeckError('loading.wind.zones must be a list, got {0!r}'
|
||||
.format(zdefs))
|
||||
zones = []
|
||||
for i, z in enumerate(zdefs):
|
||||
try:
|
||||
zones.append(inplace.LateralPressureZone(
|
||||
x0_ft=float(z['x0_ft']), x1_ft=float(z['x1_ft']),
|
||||
pressure_psf=float(z['pressure_psf']),
|
||||
label=str(z.get('label') or 'W{0}'.format(i + 1))))
|
||||
except (KeyError, TypeError, ValueError) as exc:
|
||||
raise DeckError('loading.wind.zones[{0}] needs numeric x0_ft, '
|
||||
'x1_ft, pressure_psf ({1})'.format(i, exc))
|
||||
return zones, trib, None
|
||||
|
||||
# ASCE-computed / direct-entry path.
|
||||
info = resolve_wind(deck)
|
||||
if info is None or info.get('pressure_psf') is None:
|
||||
return [], trib, (info or {}).get('reason') or 'no wind pressure supplied'
|
||||
length = getattr(panel, 'length_ft', None) if panel is not None else None
|
||||
x1 = float(length) if length else 1.0
|
||||
zone = inplace.LateralPressureZone(
|
||||
x0_ft=0.0, x1_ft=x1, pressure_psf=float(info['pressure_psf']),
|
||||
label='W-{0}'.format(info['source']))
|
||||
return [zone], trib, None
|
||||
|
||||
|
||||
def applied_axial(deck):
|
||||
"""[AppliedAxial] -- concentrated gravity bearing loads, or []."""
|
||||
items = _block(deck, 'loading').get('applied_axial') or []
|
||||
if not isinstance(items, list):
|
||||
raise DeckError('loading.applied_axial must be a list, got {0!r}'
|
||||
.format(items))
|
||||
out = []
|
||||
for i, a in enumerate(items):
|
||||
try:
|
||||
out.append(inplace.AppliedAxial(
|
||||
x_ft=float(a['x_ft']), dead_lb=float(a.get('dead_lb') or 0.0),
|
||||
live_lb=float(a.get('live_lb') or 0.0),
|
||||
label=str(a.get('label') or 'P{0}'.format(i + 1))))
|
||||
except (KeyError, TypeError, ValueError) as exc:
|
||||
raise DeckError('loading.applied_axial[{0}] needs a numeric x_ft '
|
||||
'({1})'.format(i, exc))
|
||||
return out
|
||||
|
||||
|
||||
def service_options(deck):
|
||||
"""The SERVICE-side inputs (Stage 3), from `loading.service`, or defaults.
|
||||
|
||||
A mapping {deflection_limit_ratio, cracking_class_t, initial_bow_in,
|
||||
axial_eccentricity_in} -- or the `unread(...)` placeholder / absent, which
|
||||
leaves each field at its could-not-judge default (deflection_limit_ratio
|
||||
None => the deflection LIMIT check is NOT JUDGED, never faked). Numbers are
|
||||
validated; a non-number is refused by name.
|
||||
"""
|
||||
blk = _block(deck, 'loading').get('service')
|
||||
if blk is None or _is_unread(blk):
|
||||
return {}
|
||||
if not isinstance(blk, dict):
|
||||
raise DeckError('loading.service must be an unread placeholder or a '
|
||||
'mapping, got {0!r}'.format(blk))
|
||||
out = {}
|
||||
for key in ('deflection_limit_ratio', 'initial_bow_in',
|
||||
'axial_eccentricity_in'):
|
||||
if blk.get(key) is not None:
|
||||
try:
|
||||
out[key] = float(blk[key])
|
||||
except (TypeError, ValueError):
|
||||
raise DeckError('loading.service.{0} is not a number ({1!r})'
|
||||
.format(key, blk[key]))
|
||||
if 'cracking_class_t' in blk:
|
||||
out['cracking_class_t'] = bool(blk['cracking_class_t'])
|
||||
return out
|
||||
|
||||
|
||||
def in_place_inputs(deck, panel):
|
||||
"""(InPlaceInputs, reason). None+reason where the in-place model is unwired.
|
||||
|
||||
Requires at least the lateral restraints; wind may be absent (a gravity-only
|
||||
in-place run is legitimate and the engine flags the zero lateral demand).
|
||||
Returns (None, reason) whenever the restraints are unwired -- which is the
|
||||
Stage-0 default -- so the page keeps its explicit placeholder.
|
||||
"""
|
||||
rxs, r_reason = restraints(deck)
|
||||
if len(rxs) < 2:
|
||||
return None, (r_reason or 'fewer than two lateral restraints')
|
||||
zones, trib, _w_reason = wind_zones(deck, panel)
|
||||
combos = None
|
||||
combo_blk = _block(deck, 'loading').get('combinations')
|
||||
if isinstance(combo_blk, list):
|
||||
combos = _parse_combinations(combo_blk)
|
||||
svc = service_options(deck)
|
||||
return inplace.InPlaceInputs(
|
||||
restraint_xs=rxs, wind_zones=zones,
|
||||
applied_axial=applied_axial(deck), tributary_width_ft=trib,
|
||||
combinations=combos, fc_psi=handling_fc_psi_inplace(panel),
|
||||
deflection_limit_ratio=svc.get('deflection_limit_ratio'),
|
||||
cracking_class_t=svc.get('cracking_class_t', False),
|
||||
initial_bow_in=svc.get('initial_bow_in', 0.0) or 0.0,
|
||||
axial_eccentricity_in=svc.get('axial_eccentricity_in', 0.0) or 0.0), None
|
||||
|
||||
|
||||
def handling_fc_psi_inplace(panel):
|
||||
"""f'c for the in-place model -- the 28-day strength (docs sec.6)."""
|
||||
return panel.fc_psi or panel.fci_psi or 0.0
|
||||
|
||||
|
||||
def _parse_combinations(items):
|
||||
"""Deck-supplied load combinations, or None to take the ASCE default set."""
|
||||
out = []
|
||||
for i, c in enumerate(items):
|
||||
if not isinstance(c, dict) or not isinstance(c.get('factors'), dict):
|
||||
raise DeckError('loading.combinations[{0}] needs a "factors" map'
|
||||
.format(i))
|
||||
out.append(inplace.Combo(
|
||||
name=str(c.get('name') or 'LC{0}'.format(i + 1)),
|
||||
label=str(c.get('label') or ''),
|
||||
factors={str(k): float(v) for k, v in c['factors'].items()},
|
||||
kind=str(c.get('kind') or 'strength'),
|
||||
cite=str(c.get('cite') or '')))
|
||||
return out
|
||||
|
||||
|
||||
# ----------------------------------------------------------------- writing
|
||||
|
||||
def new_deck(mark='PNL101', design_number='PNL1',
|
||||
length_ft=24.0, width_ft=8.0, thickness_in=8.0,
|
||||
fc_psi=6000.0, fci_psi=4000.0, unit_weight_pcf=150.0,
|
||||
prestress_loss_pct=8.0, support_xs=(2.0, 22.0),
|
||||
support_type='dunnage', self_weight_factor=1.0,
|
||||
strands=None, blockouts=None, weight_lb=None,
|
||||
tension_limit_sqrt_fc=5.0, compression_limit_ratio=0.6):
|
||||
"""A well-formed `panelanalysis/1` deck, for fixtures and documentation.
|
||||
|
||||
Every unread block is written as an explicit null + reason via `unread()`,
|
||||
so a deck this builds is a worked example of the file contract as much as a
|
||||
test input. The defaults describe a plain solid panel on two dunnage lines
|
||||
with two straight strands -- the simplest thing the Handling solve has a
|
||||
real answer for.
|
||||
"""
|
||||
if strands is None:
|
||||
strands = [
|
||||
{'y': 1.0, 'z_in': 1.5, 'area_in2': 0.153,
|
||||
'initial_pull_lb': 28900.0, 'label': 'S1'},
|
||||
{'y': width_ft - 1.0, 'z_in': 1.5, 'area_in2': 0.153,
|
||||
'initial_pull_lb': 28900.0, 'label': 'S2'},
|
||||
]
|
||||
reinforcement = {}
|
||||
if strands:
|
||||
reinforcement['prestress'] = {
|
||||
'diameter_in': 0.5,
|
||||
'loss_pct': prestress_loss_pct,
|
||||
'fpu_ksi': 270.0,
|
||||
'strands': list(strands),
|
||||
}
|
||||
else:
|
||||
reinforcement['prestress'] = None
|
||||
reinforcement['_note'] = ('no prestress on this piece; capacity is '
|
||||
'strand-only, so flexural capacity is not '
|
||||
'computed and says so')
|
||||
return {
|
||||
'schema': SCHEMA,
|
||||
'project_information': {
|
||||
'mark': mark,
|
||||
'design_number': design_number,
|
||||
'fc_psi': fc_psi,
|
||||
'fci_psi': fci_psi,
|
||||
'unit_weight_pcf': unit_weight_pcf,
|
||||
'prestress_loss_pct': prestress_loss_pct,
|
||||
},
|
||||
'settings': {
|
||||
'tension_limit_sqrt_fc': tension_limit_sqrt_fc,
|
||||
'compression_limit_ratio': compression_limit_ratio,
|
||||
'phi_flexure': 0.90,
|
||||
'phi_shear': 0.75,
|
||||
'development_length_multiplier': 1.0,
|
||||
'_seeded_from': unread(
|
||||
'settings are seeded from knowledge/defaults in a later '
|
||||
'stage; a Stage 0 fixture hand-authors them'),
|
||||
},
|
||||
'concrete_extents': {
|
||||
'length_ft': length_ft,
|
||||
'width_ft': width_ft,
|
||||
'thickness_in': thickness_in,
|
||||
'finish': 'smooth',
|
||||
'product_type': 'flat',
|
||||
'weight_lb': weight_lb,
|
||||
'blockouts': list(blockouts or []),
|
||||
'recesses': [],
|
||||
},
|
||||
'structural_model': {
|
||||
'supports': [{'x_ft': float(x), 'type': support_type}
|
||||
for x in support_xs],
|
||||
'restraints': unread(
|
||||
'restraints are seeded from clash detection in a later stage '
|
||||
'(Stage 1); a Stage 0 fixture hand-authors supports only'),
|
||||
'bearing': unread(
|
||||
'in-place bearing is not read off any model yet'),
|
||||
},
|
||||
'reinforcement': reinforcement,
|
||||
'loading': {
|
||||
'self_weight': {'included': True,
|
||||
'load_factor': self_weight_factor},
|
||||
'wind': unread('wind / lateral load cases are the in-place engine, '
|
||||
'built in a later stage (Stage 3)'),
|
||||
'combinations': unread('ACI 318 load combinations are built with '
|
||||
'the in-place engine (Stage 3)'),
|
||||
'service': unread('in-place SERVICE inputs (L/n deflection limit, '
|
||||
'Class T cracking, initial bow, axial '
|
||||
'eccentricity) are the in-place engine (Stage '
|
||||
'3); the ASCE 2.4 defaults run without them'),
|
||||
},
|
||||
'handling': {
|
||||
'_note': 'result block; filled by panelanalysis.py from the '
|
||||
'self-weight solve on structural_model.supports',
|
||||
},
|
||||
'in_place': unread('the in-place wind/gravity engine is not built yet '
|
||||
'(Stage 3); its result block is a placeholder, '
|
||||
'never fabricated zeros'),
|
||||
'source': {
|
||||
'synthetic': True,
|
||||
'note': 'Stage 0 hand-authored fixture; not exported from any '
|
||||
'model',
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,338 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Reinforcing suggestions with NO REVIT RUNNING -- answered from the index.
|
||||
|
||||
python tools/reinforce_offline.py WP157
|
||||
python tools/reinforce_offline.py WP157 --index "<...>/reinforce/index.json"
|
||||
python tools/reinforce_offline.py --model sample_job WP157
|
||||
python tools/reinforce_offline.py --list WP
|
||||
python tools/reinforce_offline.py WP157 --all --out report.txt
|
||||
|
||||
WHAT IT PRINTS, AND WHY MOST OF IT IS FOLDED
|
||||
---------------------------------------------
|
||||
The owner's rule for this report, 2026-08-30: the target and the donor
|
||||
proposals are the answer and everything else starts folded, because "除非有啥
|
||||
问题我感觉我不会读" -- unless something is wrong, he will not read it. A
|
||||
section that HAS something wrong in it opens itself and says why, so the page
|
||||
carries a contract: if nothing below the answer is open, nothing needs you.
|
||||
`--all` opens every section; `--out FILE` writes the whole thing unfolded, so
|
||||
a folded section is never the only copy of those words. The folding itself is
|
||||
`vibe.foldreport`, shared with the button.
|
||||
|
||||
WHY THIS CAN EXIST AT ALL
|
||||
-------------------------
|
||||
A Revit model file cannot be parsed outside Revit -- it is a proprietary
|
||||
format and nothing here pretends otherwise. (Spelling out the extension would
|
||||
trip `tools/scrub.py`, which reads `<word>.rvt` as a leaked model filename.)
|
||||
But the model is not what the matcher reads.
|
||||
It reads the INDEX: `<jobs root>/<model>/reinforce/index.json`, roughly a
|
||||
megabyte, which already holds every measurement the search needs. Measured on
|
||||
this project, 2026-08-30: a full index rebuild inside Revit is ~10-16 s, and
|
||||
the match itself is about 0.05 s of ordinary CPython.
|
||||
|
||||
Up to schema 4 exactly one thing on that path still needed a document: the
|
||||
TARGET was measured live, because donor eligibility kept every not-yet-
|
||||
reinforced piece -- which is to say, every ordinary target -- out of the file.
|
||||
Schema 5 measures in-scope non-donors too (`reinforcecore.plan(include=...)`),
|
||||
and with that gone the whole chain runs off the snapshot. Refresh the index
|
||||
once with the model open, close Revit, and suggestions run all day.
|
||||
|
||||
WHAT THAT COSTS, SAID ON EVERY PAGE THIS PRINTS
|
||||
------------------------------------------------
|
||||
The answer is as true as the model was when the index was written and no
|
||||
truer. Inside Revit the cheap signature pass reruns on every press, so a
|
||||
member added, removed or swapped, a piece newly reinforced, held or typed
|
||||
MASTER is caught; only a geometry edit that leaves the member set alone
|
||||
(an opening dragged, a panel stretched) can go stale. Offline NONE of it is
|
||||
caught, because nothing looks at the model at all. Every report this prints
|
||||
opens with the OFFLINE banner and the build stamp, and closes with
|
||||
`reinforcereport.staleness_note(offline=True)` spelling the whole list out.
|
||||
|
||||
NO REVIT IMPORT IS REACHABLE FROM HERE, and that is a property of the call
|
||||
graph, not a hope: `reinforcecore`, `reinforcematch`, `reinforcereport`,
|
||||
`scope` and `compat` touch nothing; `reinforceindex` imports the Revit API
|
||||
only inside the functions that need a document, none of which this calls;
|
||||
`reinforcebind` likewise, and the two functions in it that do need one --
|
||||
`outline_audit` and `suggest_for_piece` -- are the two this file never calls.
|
||||
It calls `suggest_from_index`, which is the same code with the cached record
|
||||
in place of the live measurement. `tests/test_reinforceoffline.py` asserts
|
||||
that no `Autodesk` module was loaded after a full offline run.
|
||||
|
||||
CPython 3.12 -- this file is a `tools/` script, so modern syntax is fine here.
|
||||
The `vibe` modules it imports are IronPython 2.7 and stay that way.
|
||||
"""
|
||||
|
||||
from __future__ import print_function
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import types
|
||||
|
||||
_HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
_LIB = os.path.join(_HERE, '..', 'WillTools.extension', 'lib')
|
||||
sys.path.insert(0, _LIB)
|
||||
|
||||
# The same trick `tools/ticket_delta.py` and the test conftest use, for the
|
||||
# same reason: `vibe/__init__.py` imports the Revit API at module scope, which
|
||||
# is right inside Revit and fatal here. Register a bare `vibe` package pointed
|
||||
# at the directory so the Revit-free submodules import normally and
|
||||
# `__init__.py` never runs.
|
||||
if 'vibe' not in sys.modules:
|
||||
_stub = types.ModuleType('vibe')
|
||||
_stub.__path__ = [os.path.join(_LIB, 'vibe')]
|
||||
sys.modules['vibe'] = _stub
|
||||
|
||||
from vibe import foldreport as fold # noqa: E402
|
||||
from vibe import reinforcebind as bind # noqa: E402
|
||||
from vibe import reinforcecore as core # noqa: E402
|
||||
from vibe import reinforceindex as index # noqa: E402
|
||||
from vibe import reinforcereport as R # noqa: E402
|
||||
|
||||
JOBS_ENV = 'WILLTOOLS_JOBS'
|
||||
WIDTH = 78
|
||||
|
||||
|
||||
# ------------------------------------------------------------ finding the file
|
||||
|
||||
def resolve_index(index_arg=None, jobs=None, model=None):
|
||||
"""(path, how) for the index to read. Never guesses silently.
|
||||
|
||||
Order: an explicit `--index` path; else `<jobs root>/<model>/reinforce/
|
||||
index.json`, which is what WILLTOOLS_JOBS means. With a jobs root and no
|
||||
`--model`, a root holding exactly one model is used and named in the
|
||||
output; a root holding several refuses and lists them, because picking one
|
||||
of somebody's jobs at random is the kind of quiet wrong answer this
|
||||
project spends its comments avoiding.
|
||||
"""
|
||||
if index_arg:
|
||||
return os.path.abspath(index_arg), 'given with --index'
|
||||
root = jobs or os.environ.get(JOBS_ENV)
|
||||
if not root:
|
||||
raise SystemExit(
|
||||
'No index to read. Give one of:\n'
|
||||
' --index <path to reinforce/index.json>\n'
|
||||
' --jobs <jobs root> --model <model name>\n'
|
||||
' set %s and pass --model <model name>' % JOBS_ENV)
|
||||
if not os.path.isdir(root):
|
||||
raise SystemExit('jobs root does not exist: %s' % root)
|
||||
if not model:
|
||||
names = sorted(n for n in os.listdir(root)
|
||||
if os.path.isdir(os.path.join(root, n)))
|
||||
if len(names) != 1:
|
||||
raise SystemExit(
|
||||
'Which model? %s holds %d:\n%s\n\nPass --model <name>.'
|
||||
% (root, len(names), '\n'.join(' ' + n for n in names)))
|
||||
model = names[0]
|
||||
return (index.index_path_in(index.jobs_folder(root, model)),
|
||||
'jobs root %s, model %s' % (root, model))
|
||||
|
||||
|
||||
def load_index(path):
|
||||
"""The cache, or a refusal that says what to do about it.
|
||||
|
||||
A schema mismatch is not a corrupt file and must not be reported as one:
|
||||
it means the index was written by an older version of this tool, and the
|
||||
fix is one refresh inside Revit. The cache is discarded WHOLE -- see
|
||||
`reinforcecore.load` -- because a schema-4 file holds eligible donors only
|
||||
and adopting its pieces would answer "that piece is not in the index" for
|
||||
every ordinary target while looking entirely healthy.
|
||||
"""
|
||||
if not os.path.isfile(path):
|
||||
raise SystemExit(
|
||||
'No index at %s\n\n'
|
||||
'The index is written from inside Revit -- press Reinforcing '
|
||||
'Suggestion once with the model open, then this will work with '
|
||||
'Revit closed.' % path)
|
||||
cache = core.load(path)
|
||||
problem = cache.get('_load_error')
|
||||
if problem:
|
||||
raise SystemExit(
|
||||
'The index at\n %s\ncould not be used: %s\n\n'
|
||||
'Nothing was adopted from it -- an index from another schema is '
|
||||
'discarded whole rather than half-read. Open the model in Revit, '
|
||||
'press Reinforcing Suggestion once to rewrite it (a full rebuild '
|
||||
'measured ~10-16 s), and run this again.' % (path, problem))
|
||||
return cache
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- printing
|
||||
|
||||
# The plain-text renderers live in `vibe.foldreport`, not here, because the
|
||||
# button needs the identical ones to write its unfolded copy to disk. These
|
||||
# are thin names over them so the CLI reads as a CLI.
|
||||
plain = fold.md_text
|
||||
table = fold.table_text
|
||||
|
||||
|
||||
def emit(sections, show_all=False):
|
||||
"""Print the report, folding the routine sections to one line each.
|
||||
|
||||
The owner's rule, applied at a terminal: the answer, then a short list of
|
||||
what else there is, and full text only for the sections that have a
|
||||
problem in them. `--all` prints everything.
|
||||
"""
|
||||
print(fold.text(sections, show_all=show_all, width=WIDTH))
|
||||
|
||||
|
||||
# -------------------------------------------------------------------- the run
|
||||
|
||||
def run(cache, name, folder, include_voids=True, prefixes=None,
|
||||
no_scope=False, max_proposals=5, max_donors=4, full_path=None):
|
||||
"""(suggestion, info, sections) for one target, entirely from the snapshot."""
|
||||
the_scope, stamped = index.scope_in(cache)
|
||||
if no_scope:
|
||||
from vibe import scope as scopelib
|
||||
the_scope = scopelib.Scope()
|
||||
elif prefixes:
|
||||
from vibe import scope as scopelib
|
||||
the_scope = scopelib.Scope(prefixes=prefixes,
|
||||
source='the --prefix option')
|
||||
|
||||
# The outline audit's RESULT, read from the file the live run left beside
|
||||
# the index. It cannot be recomputed here -- it needs each panel's real
|
||||
# solid -- and never having been run is not the same as every outline
|
||||
# being a clean rectangle, which is why `load_audit_at` returns None
|
||||
# rather than an empty set and the report says which it got.
|
||||
approximate = bind.load_audit_at(folder)
|
||||
|
||||
suggestion, info = bind.suggest_from_index(
|
||||
cache, name, approximate=approximate, the_scope=the_scope,
|
||||
include_voids=include_voids, max_proposals=max_proposals,
|
||||
max_donors=max_donors)
|
||||
info['scope_stamped'] = stamped
|
||||
|
||||
target = (cache.get('pieces') or {}).get(name) or {}
|
||||
rows = (bind.explain(suggestion, info, suggestion.best)
|
||||
if suggestion is not None and suggestion.best else [])
|
||||
sections = R.render(suggestion, info, rows, cache,
|
||||
model=cache.get('doc_key'), offline=True,
|
||||
reinforced=target.get('reinforced'),
|
||||
on_hold=target.get('on_hold'),
|
||||
full_path=full_path)
|
||||
if not stamped:
|
||||
# An index that never recorded a scope is a problem, not a footnote:
|
||||
# every count below rests on a pool bounded by nobody knows what.
|
||||
sections[0].md(
|
||||
'> **This index carries no declared scope.** It was built before '
|
||||
'the scope was stamped into the file, so the donor pool below is '
|
||||
'bounded by nothing except the filters named in the rejection '
|
||||
'table. That is "nobody recorded it", not "no scope was '
|
||||
'declared". Pass `--prefix` to bound it yourself.')
|
||||
return suggestion, info, sections
|
||||
|
||||
|
||||
def listing(cache, prefix=None):
|
||||
"""What this snapshot can answer about. Donors and targets kept apart."""
|
||||
donors = [r['name'] for r in core.donors(cache, include_unjudged=True)]
|
||||
targets = [r['name'] for r in core.targets(cache)]
|
||||
if prefix:
|
||||
donors = [n for n in donors if n.startswith(prefix)]
|
||||
targets = [n for n in targets if n.startswith(prefix)]
|
||||
return donors, targets
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(
|
||||
description='Reinforcing suggestions from the cached index, with no '
|
||||
'Revit running.',
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument('mark', nargs='?', help='the piece to suggest donors for')
|
||||
ap.add_argument('--index', help='path to reinforce/index.json')
|
||||
ap.add_argument('--jobs', help='jobs root (default: $%s)' % JOBS_ENV)
|
||||
ap.add_argument('--model', help='model name under the jobs root')
|
||||
ap.add_argument('--list', dest='list_prefix', nargs='?', const='',
|
||||
help='list what the index holds, optionally by prefix')
|
||||
ap.add_argument('--prefix', action='append',
|
||||
help='bound the donor pool to these mark prefixes, '
|
||||
'instead of the scope stamped in the index')
|
||||
ap.add_argument('--no-scope', action='store_true',
|
||||
help='ignore the stamped scope; every donor may donate')
|
||||
ap.add_argument('--no-voids', action='store_true',
|
||||
help='ligaments only -- do not ask a donor to account '
|
||||
'for the target\'s openings as well')
|
||||
ap.add_argument('--max-donors', type=int, default=4)
|
||||
ap.add_argument('--max-proposals', type=int, default=5)
|
||||
ap.add_argument('--all', dest='show_all', action='store_true',
|
||||
help='print every section, including the folded ones')
|
||||
ap.add_argument('--out', dest='out_path',
|
||||
help='write the whole report, nothing folded, to a file')
|
||||
ap.add_argument('--json', dest='json_out',
|
||||
help='write the raw (info minus features) to a file')
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
path, how = resolve_index(args.index, args.jobs, args.model)
|
||||
cache = load_index(path)
|
||||
folder = os.path.dirname(path)
|
||||
|
||||
if args.list_prefix is not None:
|
||||
donors, targets = listing(cache, args.list_prefix or None)
|
||||
print('index: %s (%s)' % (path, how))
|
||||
print('built: %s' % (cache.get('built_at') or 'never'))
|
||||
print('%d donor(s), %d piece(s) indexed only as targets'
|
||||
% (len(donors), len(targets)))
|
||||
print('\ndonors (may be copied from):')
|
||||
print(_columns(donors))
|
||||
print('\ntargets only (not reinforced, or on hold -- ask about these):')
|
||||
print(_columns(targets))
|
||||
if not targets:
|
||||
print(' (none -- this index was built without target pieces. '
|
||||
'Refresh it inside Revit to add them.)')
|
||||
return 0
|
||||
|
||||
if not args.mark:
|
||||
ap.error('give a piece mark, or --list')
|
||||
|
||||
suggestion, info, sections = run(
|
||||
cache, args.mark, folder,
|
||||
include_voids=not args.no_voids,
|
||||
prefixes=args.prefix, no_scope=args.no_scope,
|
||||
max_proposals=args.max_proposals, max_donors=args.max_donors,
|
||||
full_path=args.out_path)
|
||||
if args.out_path:
|
||||
# FOLDED IS NOT GONE. When the reader has asked for a file, it holds
|
||||
# every section unfolded -- so a section the terminal showed as one
|
||||
# line is never the only copy of those words.
|
||||
with open(args.out_path, 'w', encoding='utf-8') as fh:
|
||||
fh.write(fold.text(sections, show_all=True, width=WIDTH))
|
||||
emit(sections, show_all=args.show_all)
|
||||
print('')
|
||||
folded = [s for s in sections if not s.is_open and not s.headline]
|
||||
opened = fold.sections_with_problems(sections)
|
||||
if folded and not args.show_all:
|
||||
print(' %d section(s) folded: %s'
|
||||
% (len(folded), ', '.join(s.key for s in folded)))
|
||||
# THE CONTRACT, SAID OUT LOUD. Claiming "nothing needs you" on a page
|
||||
# where a section opened itself would be exactly the lie the whole rule
|
||||
# exists to prevent, so the two cases get two different sentences.
|
||||
if opened:
|
||||
print(' %d section(s) opened themselves: %s'
|
||||
% (len(opened), ', '.join(s.key for s in opened)))
|
||||
elif not args.show_all:
|
||||
print(' nothing below the answer needs you. --all opens them.')
|
||||
print(' index: %s' % path)
|
||||
print(' found: %s' % how)
|
||||
if args.out_path:
|
||||
print(' full: %s' % args.out_path)
|
||||
|
||||
if args.json_out:
|
||||
slim = dict((k, v) for k, v in info.items()
|
||||
if k not in ('target_features', 'features_by_donor'))
|
||||
slim['status'] = suggestion.status if suggestion else None
|
||||
with open(args.json_out, 'w') as fh:
|
||||
json.dump(slim, fh, indent=1, default=str, sort_keys=True)
|
||||
print(' json: %s' % args.json_out)
|
||||
return 0
|
||||
|
||||
|
||||
def _columns(names, per_line=8):
|
||||
if not names:
|
||||
return ' (none)'
|
||||
out = []
|
||||
for i in range(0, len(names), per_line):
|
||||
out.append(' ' + ' '.join(n.ljust(10) for n in names[i:i + per_line]))
|
||||
return '\n'.join(out).rstrip()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
+616
@@ -0,0 +1,616 @@
|
||||
#!/usr/bin/env python3
|
||||
"""rvt - talk to the live Revit session from your terminal.
|
||||
|
||||
Requires: pyRevit routes server running inside Revit (see ../README.md).
|
||||
Standard library only, Python 3.8+.
|
||||
|
||||
python tools/rvt.py status
|
||||
python tools/rvt.py exec scratch/scratch.py
|
||||
python tools/rvt.py exec -c "print(doc.Title)"
|
||||
python tools/rvt.py exec scratch/rename.py --tx "Rename walls"
|
||||
echo "print(len(list(FilteredElementCollector(doc))))" | python tools/rvt.py exec -
|
||||
python tools/rvt.py sel
|
||||
python tools/rvt.py collect OST_Walls --limit 20
|
||||
python tools/rvt.py params 123456
|
||||
python tools/rvt.py reload
|
||||
python tools/rvt.py --strict status # refuse an unverifiable answer
|
||||
|
||||
Every request carries a correlation id and every answer is checked against it.
|
||||
See `call()` and `REQUEST_CORRELATION` below for why that is not optional.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import uuid
|
||||
|
||||
# pyRevit does not pin the routes port: `serverinfo._get_next_available_port`
|
||||
# starts at the configured one and walks UP past every port a running Revit
|
||||
# already holds. So with two sessions open, 48884 is "whichever Revit started
|
||||
# first" and the second is on 48885. That is not a detail - it is how a sweep
|
||||
# aimed at the sandbox lands in the live model instead.
|
||||
#
|
||||
# The port is therefore settable from the environment rather than hard-coded,
|
||||
# so `tools/snapshot_cycle.py` can scan the small range, find the session that
|
||||
# is actually the sandbox, and hand that port down to every batch command it
|
||||
# runs. Unset, this is exactly what it always was.
|
||||
DEFAULT_PORT = 48884
|
||||
PORT_ENV = "WILLTOOLS_BRIDGE_PORT"
|
||||
|
||||
|
||||
def _port() -> int:
|
||||
try:
|
||||
return int(os.environ.get(PORT_ENV) or DEFAULT_PORT)
|
||||
except (TypeError, ValueError):
|
||||
return DEFAULT_PORT
|
||||
|
||||
|
||||
def base_url() -> str:
|
||||
"""The bridge base URL for THIS call.
|
||||
|
||||
Read per call, not once at import: a caller that sets the variable and
|
||||
then makes a request is the ordinary case, and a module-level constant
|
||||
would silently serve the port that was set when the module loaded.
|
||||
"""
|
||||
return f"http://localhost:{_port()}/vibe"
|
||||
|
||||
|
||||
BASE = base_url()
|
||||
TIMEOUT = 300
|
||||
|
||||
ROOT = pathlib.Path(__file__).resolve().parent.parent
|
||||
SESSION_LOG = ROOT / ".bridge-sessions.log"
|
||||
|
||||
# --------------------------------------------------------- request correlation
|
||||
#
|
||||
# REQUEST_CORRELATION: why every request carries an id, and every answer is
|
||||
# checked against it.
|
||||
#
|
||||
# Measured 2026-09-01, twice, during a 282-piece sweep. One request timed out
|
||||
# client-side ("[260/282] no response from Revit within 300s"), and from then
|
||||
# on the pairing was off by one: `sweep_in_revit.py`'s `open_model_title()`
|
||||
# asked for `doc.Title` and got the SWEEP's progress string back, which it then
|
||||
# printed as the document title; an agent polling `doc.Title` as a drift guard
|
||||
# read the same stolen answer as "the model changed underneath me" and raised a
|
||||
# false alarm. The data on disk was fine. The attribution was not, and **a
|
||||
# response from the wrong request is indistinguishable from a valid answer**.
|
||||
#
|
||||
# WHERE THE DESYNC ACTUALLY LIVES -- not here. This client is stateless per
|
||||
# request: `urlopen` builds a new connection every call and urllib sends
|
||||
# `Connection: close`, so there is no client-side socket to go out of step and
|
||||
# nothing here to "drain". The shared state is inside pyRevit, in
|
||||
# `pyrevit/routes/server/server.py`:
|
||||
#
|
||||
# REQUEST_HNDLR = handler.RequestHandler() # ONE, module level
|
||||
# EVENT_HNDLR = UI.ExternalEvent.Create(REQUEST_HNDLR)
|
||||
#
|
||||
# The HTTP server is a `ThreadingMixIn` server, so each connection gets its own
|
||||
# thread -- but every one of those threads writes the SAME singleton
|
||||
# (`REQUEST_HNDLR.request = request`), calls the SAME `reset()` (clearing one
|
||||
# shared `_response` / `_done` pair), raises the SAME external event, and then
|
||||
# spins in `req_hndlr.join()` waiting on that one `_done` flag before reading
|
||||
# that one `_response`.
|
||||
#
|
||||
# So a client-side timeout does not cancel anything. The server thread is still
|
||||
# spinning, Revit still owes it an answer, and when the next request arrives:
|
||||
#
|
||||
# * it overwrites `REQUEST_HNDLR.request`/`.handler` with its own, so the
|
||||
# abandoned event -- if Revit has not serviced it yet -- executes against
|
||||
# the NEW request; and `ExternalEvent.Raise()` on an already-pending event
|
||||
# coalesces, so the new request's own raise can be swallowed and its
|
||||
# snippet never run at all; and
|
||||
# * both threads then wait on one `_done` flag and read one `_response`, so
|
||||
# whichever handler finishes first answers BOTH sockets. The abandoned
|
||||
# one's write goes to a closed socket and is lost; the live one carries the
|
||||
# abandoned request's body to a client that asked something else.
|
||||
#
|
||||
# That is the whole mechanism, and it is why the fix is in two halves that both
|
||||
# have to exist:
|
||||
#
|
||||
# 1. an id minted here and echoed by `WillTools.extension/startup.py`, so a
|
||||
# stolen answer is REFUSED loudly instead of returned (`_verified`); and
|
||||
# 2. a latch (`_latch_timeout`) written the moment a request times out, which
|
||||
# makes the NEXT call drain that shared slot with a self-identifying probe
|
||||
# before it sends anything real (`_drain_channel`). The latch is a file
|
||||
# because `tools/sweep.py` runs `rvt.py` as a fresh subprocess per call --
|
||||
# an in-process flag would not survive to the request it has to protect,
|
||||
# which is precisely the request that got the stolen answer.
|
||||
#
|
||||
# Nothing here can stop the theft happening inside Revit. It can stop the
|
||||
# result being mistaken for data, which is the part that cost real time.
|
||||
|
||||
CORRELATION_KEY = "_rid"
|
||||
UNVERIFIED_KEY = "_unverified"
|
||||
STRICT_ENV = "WILLTOOLS_BRIDGE_STRICT"
|
||||
|
||||
# Where the "the last request timed out" latch lives. One file per port: which
|
||||
# Revit owns which port is decided by start order (see DEFAULT_PORT above), so
|
||||
# a latch left by a call to 48885 must not gate a call to 48884.
|
||||
LATCH_DIR = ROOT
|
||||
|
||||
# Escalating waits for the drain probes. Short on purpose: the point is to find
|
||||
# out whether the channel is clear, not to sit out another long request. One
|
||||
# abandoned request leaves at most one stale answer, so two probes are the
|
||||
# minimum that can both consume it and then confirm - the third is headroom for
|
||||
# a first probe that gets no answer at all because Revit is still busy.
|
||||
DRAIN_TIMEOUTS = (15, 30, 60)
|
||||
|
||||
_STRICT = False
|
||||
|
||||
|
||||
class BridgeTimeout(Exception):
|
||||
"""A request that got no answer.
|
||||
|
||||
Deliberately an exception and never a value: "Revit did not answer" and
|
||||
"Revit answered" are different states, and the whole incident above is what
|
||||
happens when something downstream reads one as the other.
|
||||
"""
|
||||
|
||||
|
||||
def _strict() -> bool:
|
||||
"""Whether an unverifiable answer is refused rather than flagged."""
|
||||
return bool(_STRICT or (os.environ.get(STRICT_ENV) or "").strip())
|
||||
|
||||
|
||||
def warn(msg: str):
|
||||
"""Say something on stderr.
|
||||
|
||||
stderr, always: `tools/sweep.py` parses this process's STDOUT as JSON, so a
|
||||
warning printed there would corrupt the answer it is warning about.
|
||||
"""
|
||||
print(msg, file=sys.stderr)
|
||||
|
||||
|
||||
def new_request_id() -> str:
|
||||
"""A request id that is unique across processes and across time.
|
||||
|
||||
The pid alone is not enough (`sweep.py` runs one `rvt.py` per call, and pids
|
||||
are reused); a counter alone is not enough for the same reason. This is
|
||||
pid + wall clock + random, which is unique enough to make a match mean
|
||||
something and short enough to read in an error message.
|
||||
"""
|
||||
return "{0}-{1:x}-{2}".format(os.getpid(), int(time.time() * 1000),
|
||||
uuid.uuid4().hex[:8])
|
||||
|
||||
|
||||
def _latch_path() -> pathlib.Path:
|
||||
return LATCH_DIR / ".bridge-timeout-{0}.json".format(_port())
|
||||
|
||||
|
||||
def _latch_timeout(rid: str, path: str, reason: str):
|
||||
"""Record that this bridge may still owe an answer to a dead request."""
|
||||
body = {"rid": rid, "path": path, "reason": reason, "port": _port(),
|
||||
"at": time.time(), "pid": os.getpid()}
|
||||
try:
|
||||
_latch_path().write_text(json.dumps(body, indent=1), encoding="utf-8")
|
||||
except OSError as exc:
|
||||
# A guard that silently failed to arm is worse than no guard: say so
|
||||
# here rather than letting the next call believe the channel is clean.
|
||||
warn("WARNING: could not write the bridge timeout latch at {0} ({1}).\n"
|
||||
" The NEXT request will not know to drain the channel first, so\n"
|
||||
" it may be handed this request's late answer. Treat the next\n"
|
||||
" answer as unattributed."
|
||||
.format(_latch_path(), exc))
|
||||
|
||||
|
||||
def _read_latch():
|
||||
"""The latch for this port, or None. An unreadable latch still counts."""
|
||||
path = _latch_path()
|
||||
if not path.exists():
|
||||
return None
|
||||
try:
|
||||
body = json.loads(path.read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError) as exc:
|
||||
return {"rid": None, "path": None, "reason": "unreadable latch: {0}"
|
||||
.format(exc), "at": None}
|
||||
return body if isinstance(body, dict) else {"rid": None, "path": None,
|
||||
"reason": "malformed latch",
|
||||
"at": None}
|
||||
|
||||
|
||||
def _clear_latch():
|
||||
try:
|
||||
_latch_path().unlink()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def _drain_channel():
|
||||
"""Consume any late reply still sitting in the bridge's one response slot.
|
||||
|
||||
Returns (clear, sentence).
|
||||
|
||||
The probe is `/exec` with a snippet that returns a nonce this process just
|
||||
invented. That gives the answer TWO independent ways to identify itself:
|
||||
the echoed correlation id (only a current in-Revit half sends it) and the
|
||||
nonce coming back as `result` (works against any half, including the one
|
||||
running right now, which was installed before ids existed). An answer that
|
||||
is neither is somebody else's -- reading it here is what removes it from
|
||||
the shared slot, which is the actual draining.
|
||||
"""
|
||||
stale = 0
|
||||
notes = []
|
||||
for wait in DRAIN_TIMEOUTS:
|
||||
nonce = "vibe-drain-" + uuid.uuid4().hex[:16]
|
||||
rid = new_request_id()
|
||||
body = {"code": "result = {0!r}".format(nonce), CORRELATION_KEY: rid}
|
||||
try:
|
||||
res = _send("/exec", "POST", body, wait)
|
||||
except BridgeTimeout:
|
||||
notes.append("a {0}s probe got no answer at all".format(wait))
|
||||
continue
|
||||
if isinstance(res, dict) and res.get("refused"):
|
||||
return False, ("the bridge is refusing every route, so nothing can "
|
||||
"prove the channel is clean: {0}"
|
||||
.format(res.get("error")))
|
||||
echoed = res.get(CORRELATION_KEY) if isinstance(res, dict) else None
|
||||
if echoed is not None and echoed != rid:
|
||||
stale += 1
|
||||
notes.append("a probe came back stamped {0}, which is another "
|
||||
"request's id".format(echoed))
|
||||
continue
|
||||
if isinstance(res, dict) and res.get("result") == nonce:
|
||||
return True, ("the bridge answered its own probe{0}"
|
||||
.format("" if not stale else
|
||||
" after {0} stale repl{1} drained"
|
||||
.format(stale, "y" if stale == 1 else "ies")))
|
||||
stale += 1
|
||||
notes.append("a probe came back without its own nonce in `result` "
|
||||
"-- that answer belonged to something else")
|
||||
return False, "; ".join(notes) or "no probe could identify itself"
|
||||
|
||||
|
||||
def _resolve_timeout_latch():
|
||||
"""Refuse to reuse a channel that may still be carrying a dead request.
|
||||
|
||||
Called before every real request. Draining is attempted, loudly, and only a
|
||||
probe that identifies itself clears the latch -- "the bridge answered" is
|
||||
not the same fact as "the bridge answered ME", and this whole module exists
|
||||
because those two were once the same code path.
|
||||
"""
|
||||
latch = _read_latch()
|
||||
if latch is None:
|
||||
return
|
||||
warn("NOTE: the last request on port {0} ({1}) did not complete: {2}.\n"
|
||||
" pyRevit serves every route through one shared response slot, so a\n"
|
||||
" late answer to that request would be handed to this one. Draining\n"
|
||||
" before sending anything real."
|
||||
.format(_port(), latch.get("path") or "unknown route",
|
||||
latch.get("reason") or "unknown"))
|
||||
clear, why = _drain_channel()
|
||||
if clear:
|
||||
_clear_latch()
|
||||
warn(" channel clear: {0}. Continuing.".format(why))
|
||||
return
|
||||
die("REFUSING to send - this bridge cannot be proven clean.\n\n"
|
||||
" The previous request ({0}) never came back, and every probe since\n"
|
||||
" has failed to identify itself: {1}.\n\n"
|
||||
" Sending now risks being handed that request's answer and reading it\n"
|
||||
" as an answer to this one - which is not a hypothetical, it is what\n"
|
||||
" happened on 2026-09-01 (see REQUEST_CORRELATION in this file).\n\n"
|
||||
" Wait for Revit to finish what it is doing and run this again; the\n"
|
||||
" drain retries every time. If Revit is idle and this keeps failing,\n"
|
||||
" a modal dialog is blocking the API context, or the session needs a\n"
|
||||
" RESTART (never a pyRevit reload).\n\n"
|
||||
" Latch: {2}".format(latch.get("path") or "unknown route", why,
|
||||
_latch_path()))
|
||||
|
||||
|
||||
def check_bridge_safe():
|
||||
"""Refuse to talk to a bridge that has survived a pyRevit reload.
|
||||
|
||||
pyRevit's routes handler keeps pointing at the scripting engine from before
|
||||
the reload. The next request executes against a dead engine and kills Revit
|
||||
outright - an unhandled exception on a non-main thread, which Revit cannot
|
||||
catch. Our code never runs, so this has to be caught out here, before the
|
||||
request is ever sent.
|
||||
|
||||
startup.py appends one line per pyRevit load. Two lines for the same Revit
|
||||
process means a reload happened and the bridge is now a landmine.
|
||||
"""
|
||||
if not SESSION_LOG.exists():
|
||||
# Either Revit has never loaded the extension, or the tripwire in
|
||||
# startup.py is not firing. Say so - a safety net that silently stopped
|
||||
# working is worse than one that was never there.
|
||||
print(
|
||||
"WARNING: no bridge session log at "
|
||||
f"{SESSION_LOG}\n"
|
||||
" The reload tripwire is not recording. If Revit IS running with\n"
|
||||
" pyRevit loaded, the guard is broken - do not reload pyRevit.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return
|
||||
|
||||
entries = []
|
||||
for raw in SESSION_LOG.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||
parts = raw.strip().split("|")
|
||||
if len(parts) == 3:
|
||||
entries.append((parts[0], parts[1], parts[2])) # when, pid, proc_start
|
||||
if not entries:
|
||||
return
|
||||
|
||||
_, pid, proc_start = entries[-1]
|
||||
loads = sum(1 for e in entries if e[1] == pid and e[2] == proc_start)
|
||||
if loads > 1:
|
||||
die(
|
||||
f"REFUSING to send - the bridge is unsafe right now.\n\n"
|
||||
f" pyRevit has been reloaded {loads - 1}x inside this Revit process (pid {pid}).\n"
|
||||
f" Its request handler still points at the scripting engine from before\n"
|
||||
f" the reload, so the next request would crash Revit outright.\n\n"
|
||||
f" Fix: restart Revit (not 'pyRevit > Reload'). A fresh process is safe,\n"
|
||||
f" and a restart picks up new buttons just as well as a reload does.\n\n"
|
||||
f" Log: {SESSION_LOG}"
|
||||
)
|
||||
|
||||
|
||||
def _send(path: str, method: str, payload, timeout: int):
|
||||
"""One HTTP round trip. Raises BridgeTimeout; never checks correlation.
|
||||
|
||||
Arguments always travel as a JSON body, never a query string, and never a
|
||||
header. pyRevit's routes `Request` does not reliably expose a parsed query
|
||||
string across versions - on pyRevit 5.0.1 it exposes none at all, under any
|
||||
attribute name (see startup.py's `_body` docstring) - so every endpoint
|
||||
reads `request.data` instead, and `payload` is how this client fills it in.
|
||||
A header would be no better: `base.Request.__init__` sets `_headers = {}`
|
||||
and `HttpRequestHandler._prepare_request` never fills it, so the request
|
||||
headers are as unreachable from handler code as the query string is. That
|
||||
is why the correlation id travels in the body, on GET routes too - the
|
||||
server reads a body whenever Content-Length is set, whatever the method.
|
||||
"""
|
||||
base = base_url()
|
||||
url = base + path
|
||||
data = None
|
||||
headers = {}
|
||||
if payload is not None:
|
||||
data = json.dumps(payload).encode("utf-8")
|
||||
headers["Content-Type"] = "application/json"
|
||||
req = urllib.request.Request(url, data=data, headers=headers, method=method)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
body = resp.read().decode("utf-8", "replace")
|
||||
except urllib.error.HTTPError as exc:
|
||||
body = exc.read().decode("utf-8", "replace")
|
||||
except TimeoutError:
|
||||
raise BridgeTimeout(path)
|
||||
except urllib.error.URLError as exc:
|
||||
# A CONNECT timeout arrives wrapped in URLError (socket.timeout is an
|
||||
# OSError, and `do_open` wraps those); a READ timeout is raised bare and
|
||||
# is caught above. Both are the same answer - nobody replied - and must
|
||||
# not be reported as "cannot reach Revit", which reads as "Revit is
|
||||
# closed" and is a different fact.
|
||||
if isinstance(exc.reason, TimeoutError):
|
||||
raise BridgeTimeout(path)
|
||||
die(
|
||||
f"cannot reach Revit at {base}\n"
|
||||
f" -> {exc.reason}\n"
|
||||
" Is Revit open with pyRevit loaded? Routes server must be enabled\n"
|
||||
" (pyrevit configs routes enable) and Revit restarted once after install."
|
||||
)
|
||||
except ConnectionResetError as exc:
|
||||
die(
|
||||
f"Revit closed the connection ({type(exc).__name__}).\n"
|
||||
" Usually means the routes server or Revit itself went down mid-request.\n"
|
||||
" Check Revit is still running, then restart it if the bridge stays dead."
|
||||
)
|
||||
try:
|
||||
return json.loads(body)
|
||||
except json.JSONDecodeError:
|
||||
return body
|
||||
|
||||
|
||||
def _verified(res, rid: str, path: str):
|
||||
"""The answer, but only if it is an answer to THIS request.
|
||||
|
||||
Three outcomes, and they are three, not two:
|
||||
|
||||
matching id the answer, with the id stripped back off so callers see
|
||||
exactly the shape they always saw.
|
||||
wrong id REFUSED. Named loudly, both ids printed, nothing returned -
|
||||
a stolen answer is indistinguishable from a real one, so
|
||||
there is no safe value to hand back.
|
||||
no id at all the in-Revit half predates this and cannot say. Warned and
|
||||
flagged `_unverified`, or refused under --strict; see the
|
||||
warning text for why that is the default.
|
||||
"""
|
||||
got = res.get(CORRELATION_KEY) if isinstance(res, dict) else None
|
||||
|
||||
if got is not None and got != rid:
|
||||
# Proof, not suspicion: the channel is serving somebody else's answers.
|
||||
_latch_timeout(rid, path, "an answer arrived stamped with another "
|
||||
"request's id ({0})".format(got))
|
||||
die("REFUSING this answer - it belongs to a different request.\n\n"
|
||||
" asked as {rid}\n"
|
||||
" answered as {got}\n"
|
||||
" route {path}\n\n"
|
||||
" pyRevit serves every route through one shared response slot, so\n"
|
||||
" an abandoned request's answer can be handed to the next caller.\n"
|
||||
" This is that, caught. Nothing is returned, because a wrong\n"
|
||||
" answer and a right one look identical once they are a value.\n\n"
|
||||
" The channel is now latched; the next call drains it first."
|
||||
.format(rid=rid, got=got, path=path))
|
||||
|
||||
if got is None:
|
||||
if _strict():
|
||||
die("REFUSING this answer - it cannot be attributed ({0}).\n\n"
|
||||
" This client stamped the request {1} and the in-Revit half\n"
|
||||
" echoed nothing back, so there is no way to tell this answer\n"
|
||||
" from an abandoned request's answer served out of pyRevit's\n"
|
||||
" shared response slot.\n\n"
|
||||
" The in-Revit half only echoes ids once Revit has been\n"
|
||||
" RESTARTED after this version of the extension was installed\n"
|
||||
" (never a pyRevit reload). Drop --strict / unset {2} to take\n"
|
||||
" the answer with a warning instead."
|
||||
.format(path, rid, STRICT_ENV))
|
||||
warn("WARNING: this answer could not be attributed.\n"
|
||||
" Asked as {0} on {1}; the in-Revit half echoed no id back, so it\n"
|
||||
" predates request correlation and cannot say which request it is\n"
|
||||
" answering. It is almost certainly right - but if a request has\n"
|
||||
" timed out in this session, an answer like this one is exactly\n"
|
||||
" how the wrong body gets read as data.\n"
|
||||
" Fix: RESTART Revit (never a pyRevit reload) to load the current\n"
|
||||
" extension. Use --strict / {2}=1 to refuse instead of warn."
|
||||
.format(rid, path, STRICT_ENV))
|
||||
if isinstance(res, dict):
|
||||
res[UNVERIFIED_KEY] = True
|
||||
return res
|
||||
|
||||
res.pop(CORRELATION_KEY, None)
|
||||
return res
|
||||
|
||||
|
||||
def call(path: str, method: str = "GET", payload=None):
|
||||
"""Call one bridge endpoint, and prove the answer is this call's answer.
|
||||
|
||||
See REQUEST_CORRELATION at the top of this file for the incident and the
|
||||
mechanism. In short: a correlation id goes out in the body (every route,
|
||||
GET included), the current in-Revit half echoes it, and an answer that does
|
||||
not carry this request's id is refused rather than returned.
|
||||
"""
|
||||
check_bridge_safe()
|
||||
_resolve_timeout_latch()
|
||||
rid = new_request_id()
|
||||
body = dict(payload or {})
|
||||
body[CORRELATION_KEY] = rid
|
||||
try:
|
||||
res = _send(path, method, body, TIMEOUT)
|
||||
except BridgeTimeout:
|
||||
_latch_timeout(rid, path, "no answer within {0}s".format(TIMEOUT))
|
||||
die(
|
||||
f"COULD NOT JUDGE: no response from Revit within {TIMEOUT}s ({path}).\n"
|
||||
" This is not an answer about the model. Nothing was read and\n"
|
||||
" nothing was ruled out - the request may still be running inside\n"
|
||||
" Revit, and its answer may arrive after this process is gone.\n"
|
||||
" A modal dialog in Revit blocks the API context the bridge runs on -\n"
|
||||
" dismiss any open dialog, then retry.\n\n"
|
||||
f" The bridge is now latched ({_latch_path().name}): the next call\n"
|
||||
" drains that late answer out of pyRevit's shared response slot\n"
|
||||
" before it sends anything, so it cannot be served as data to\n"
|
||||
" whoever asks next."
|
||||
)
|
||||
return _verified(res, rid, path)
|
||||
|
||||
|
||||
def die(msg: str, code: int = 1):
|
||||
print(msg, file=sys.stderr)
|
||||
raise SystemExit(code)
|
||||
|
||||
|
||||
def show(res):
|
||||
"""Print an /exec-style response in a human- and agent-friendly way."""
|
||||
if not isinstance(res, dict):
|
||||
print(res)
|
||||
return 0
|
||||
out = res.get("stdout")
|
||||
if out:
|
||||
print(out, end="" if out.endswith("\n") else "\n")
|
||||
if res.get("ok"):
|
||||
result = res.get("result")
|
||||
if result is not None:
|
||||
print(json.dumps(result, indent=2, ensure_ascii=False))
|
||||
return 0
|
||||
print("ERROR: " + str(res.get("error", res)), file=sys.stderr)
|
||||
if res.get("traceback"):
|
||||
print(res["traceback"], file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
def cmd_status(_):
|
||||
return show_json(call("/status"))
|
||||
|
||||
|
||||
def show_json(res):
|
||||
print(json.dumps(res, indent=2, ensure_ascii=False))
|
||||
return 0 if isinstance(res, dict) and res.get("ok") else 1
|
||||
|
||||
|
||||
def cmd_exec(args):
|
||||
if args.code:
|
||||
code = args.code
|
||||
elif args.file == "-":
|
||||
code = sys.stdin.read()
|
||||
elif args.file:
|
||||
with open(args.file, "r", encoding="utf-8") as fh:
|
||||
code = fh.read()
|
||||
else:
|
||||
die("give a file, -c CODE, or - to read stdin")
|
||||
return show(call("/exec", "POST", {"code": code, "transaction": args.tx}))
|
||||
|
||||
|
||||
def cmd_sel(_):
|
||||
return show_json(call("/selection"))
|
||||
|
||||
|
||||
def cmd_collect(args):
|
||||
return show_json(
|
||||
call("/collect", "POST", {"category": args.category,
|
||||
"limit": args.limit,
|
||||
"types": bool(args.types)})
|
||||
)
|
||||
|
||||
|
||||
def cmd_params(args):
|
||||
return show_json(call("/params", "POST", {"id": args.id}))
|
||||
|
||||
|
||||
def cmd_reload(_):
|
||||
die(
|
||||
"Reloading over HTTP is not supported - it crashes Revit.\n"
|
||||
" Reloading rebuilds the pyRevit session, including the routes server\n"
|
||||
" that would be serving this request; the half-dead server then throws\n"
|
||||
" on a background thread and takes Revit with it.\n\n"
|
||||
" Use the ribbon instead: Vibe > System > Reload pyRevit\n"
|
||||
" (only needed after ADDING or RENAMING a button - editing an existing\n"
|
||||
" script.py is picked up on the next click with no reload.)"
|
||||
)
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(prog="rvt", description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
# Before the subcommand: `rvt.py --strict status`. Also settable as
|
||||
# WILLTOOLS_BRIDGE_STRICT=1, which is how it reaches the tools that shell
|
||||
# out to this one (sweep.py, snapshot_cycle.py) without threading a flag
|
||||
# through every command line they build.
|
||||
ap.add_argument("--strict", action="store_true",
|
||||
help="refuse an answer that carries no correlation id "
|
||||
"(an in-Revit half from before this Revit restart) "
|
||||
"instead of warning and returning it")
|
||||
sub = ap.add_subparsers(dest="cmd", required=True)
|
||||
|
||||
sub.add_parser("status", help="Revit + open model info").set_defaults(fn=cmd_status)
|
||||
|
||||
p = sub.add_parser("exec", help="run a Python snippet inside Revit")
|
||||
p.add_argument("file", nargs="?", help="path to a .py file, or - for stdin")
|
||||
p.add_argument("-c", "--code", help="inline code")
|
||||
p.add_argument("--tx", metavar="NAME",
|
||||
help="wrap in a Revit Transaction with this name")
|
||||
p.set_defaults(fn=cmd_exec)
|
||||
|
||||
sub.add_parser("sel", help="current selection").set_defaults(fn=cmd_sel)
|
||||
|
||||
p = sub.add_parser("collect", help="collect elements by BuiltInCategory")
|
||||
p.add_argument("category", help="e.g. OST_Walls")
|
||||
p.add_argument("--limit", type=int, default=100)
|
||||
p.add_argument("--types", action="store_true", help="element types, not instances")
|
||||
p.set_defaults(fn=cmd_collect)
|
||||
|
||||
p = sub.add_parser("params", help="dump all parameters of one element")
|
||||
p.add_argument("id", help="element id")
|
||||
p.set_defaults(fn=cmd_params)
|
||||
|
||||
sub.add_parser("reload", help="reload pyRevit (picks up new buttons)").set_defaults(fn=cmd_reload)
|
||||
|
||||
args = ap.parse_args(argv)
|
||||
global _STRICT
|
||||
_STRICT = bool(args.strict)
|
||||
return args.fn(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
+839
@@ -0,0 +1,839 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Refuse to publish anything carrying a client's name.
|
||||
|
||||
python tools/scrub.py # working tree + full git history
|
||||
python tools/scrub.py --worktree # working tree only, tracked and not
|
||||
python tools/scrub.py --staged # what the pre-commit hook runs
|
||||
python tools/scrub.py --private # the gitignored tree, on its own terms
|
||||
|
||||
Exits non-zero on a hit, so it can gate a commit, a package build, or a push.
|
||||
|
||||
--worktree and --staged look at different sets on purpose. --worktree is an
|
||||
audit: every file git can see, tracked or not, and a hit anywhere is a hit.
|
||||
--staged is a gate on one commit: only what is staged can block it, because
|
||||
only what is staged is about to enter history. Untracked hits are still
|
||||
printed, loudly, every time -- they are one `git add -A` from being the other
|
||||
kind -- but they do not stop a commit they have nothing to do with. A gate
|
||||
that blocks unrelated work is a gate people learn to pass with --no-verify.
|
||||
|
||||
--private is the third set, and it is an inventory rather than a gate: it
|
||||
always exits 0. Every mode above honours .gitignore, so none of them has ever
|
||||
opened `QNA/`, `knowledge/plant/` or `knowledge/job/` -- the other
|
||||
repository's territory, where a name is legitimate, and also the largest body
|
||||
of quoted source material in the checkout. Nothing looking at all is worse
|
||||
than looking on the right terms, so --private looks, and asks a different
|
||||
question: not "is there a name here", which is answered yes by design, but
|
||||
"is there a name here in a shape somebody will copy" -- a fenced block, a
|
||||
proposed key, a quoted literal, a path, a name glued into a longer
|
||||
identifier. Those are aggregated by the distinct string and printed; names in
|
||||
prose and in data values are counted and never listed. See `scan_private`.
|
||||
|
||||
It exits 0 because it has no verdict of its own to give. Every string it
|
||||
prints contains either a denylist term or a structural pattern, so if one of
|
||||
them had already been copied into a tracked file, --worktree would be failing
|
||||
on that file right now. --private cannot find a leak that --worktree misses;
|
||||
what it finds is the material a leak would be made out of, weeks before.
|
||||
A mode that returned non-zero on that would return non-zero every single run,
|
||||
and a check that always fails is a check that gets wrapped in `|| true`.
|
||||
|
||||
The terms to look for live in `.scrub-terms`, one per line, which is itself
|
||||
gitignored -- the list of client names must never become the thing that leaks
|
||||
them. `.scrub-terms.example` shows the format.
|
||||
|
||||
History matters as much as the working tree. Editing a file does not remove a
|
||||
name from the commit that introduced it, and `git push` sends the commits, not
|
||||
the checkout. A repo that has never been pushed can still be rewritten
|
||||
cleanly; one that has been pushed cannot.
|
||||
|
||||
Structural patterns below are always checked, term list or not: a model
|
||||
filename or an absolute user path identifies a job whether or not anyone
|
||||
remembered to add it to the list.
|
||||
|
||||
The one exemption: the sanctioned placeholder names CLAUDE.md's "No client or
|
||||
job information" table tells everyone to write instead of a real one -- the
|
||||
job placeholder plus `.rvt` and friends -- are read out of that table (see
|
||||
`load_placeholders`) and excused from the model/family-filename patterns,
|
||||
whole-name only. Gluing anything else onto the placeholder still fails, on
|
||||
purpose -- see `_is_placeholder_filename`.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
TERMS_FILE = os.path.join(ROOT, '.scrub-terms')
|
||||
CLAUDE_MD = os.path.join(ROOT, 'CLAUDE.md')
|
||||
# CLAUDE.md does not ship in a colleague distribution (tools/make_dist.py) --
|
||||
# CONTRIBUTING.md is the fallback there, carrying the identical heading and
|
||||
# table verbatim so `load_placeholders()` still finds it. Checked in this
|
||||
# order; CLAUDE.md wins where both exist, which is every owner checkout.
|
||||
CONTRIBUTING_MD = os.path.join(ROOT, 'CONTRIBUTING.md')
|
||||
|
||||
# Always suspicious, whatever the term list says.
|
||||
STRUCTURAL = [
|
||||
(r'[A-Za-z]:\\Users\\[A-Za-z0-9._-]+', 'absolute Windows user path'),
|
||||
(r'\b[\w -]+\.rvt\b', 'Revit model filename'),
|
||||
(r'\b[\w -]+\.rfa\b', 'Revit family filename'),
|
||||
(r'\b[\w.+-]+@[\w-]+\.[\w.]+\b', 'email address'),
|
||||
]
|
||||
|
||||
# The two structural categories above whose hit is a whole filename, and so
|
||||
# can be checked against the sanctioned placeholders rather than a real name.
|
||||
# The absolute-path and email patterns get no such exemption: there is no
|
||||
# placeholder for those, and `sample_client`/`sample_job` never look like one.
|
||||
FILENAME_CATEGORIES = {'Revit model filename', 'Revit family filename'}
|
||||
|
||||
# Files whose hits are noise rather than leaks.
|
||||
SKIP = {'.scrub-terms', '.scrub-terms.example'}
|
||||
SKIP_DIRS = {'.git', 'dist', '__pycache__', '_ticketqc'}
|
||||
|
||||
|
||||
def load_terms(path=None) -> list[tuple[str, bool]]:
|
||||
"""(term, case_sensitive) pairs. A leading '=' means case-sensitive.
|
||||
|
||||
Names are matched case-insensitively by default, because a client name is
|
||||
a client name however it is typed. Office parameter names are the
|
||||
exception: SCREAMING_SNAKE literals must not appear, but the lowercase
|
||||
role identifiers that replaced them (panel.design_number) are the whole
|
||||
point of the refactor and must not be flagged. Prefix those with '='.
|
||||
"""
|
||||
path = path or TERMS_FILE
|
||||
if not os.path.isfile(path):
|
||||
return []
|
||||
terms = []
|
||||
with open(path, encoding='utf-8') as fh:
|
||||
for line in fh:
|
||||
line = line.split('#', 1)[0].strip()
|
||||
if not line:
|
||||
continue
|
||||
if line.startswith('='):
|
||||
terms.append((line[1:].strip(), True))
|
||||
else:
|
||||
terms.append((line, False))
|
||||
return [t for t in terms if t[0]]
|
||||
|
||||
|
||||
def load_placeholders(path=None) -> list[str]:
|
||||
"""The sanctioned stand-ins for a real client/job name -- read out of
|
||||
CLAUDE.md's own table rather than retyped here.
|
||||
|
||||
CLAUDE.md Section 3 ("No client or job information in the repo. Ever.")
|
||||
is the one place that defines what a placeholder is allowed to be:
|
||||
|
||||
| a client / owner | `sample_client` |
|
||||
| a job / project / model | `sample_job` |
|
||||
|
||||
A second, hardcoded list in this file would drift the day someone adds or
|
||||
renames a row there -- silently, because nothing would fail, the
|
||||
exemption below would just stop covering the current placeholder. Reading
|
||||
the table instead means there is exactly one place to edit and this file
|
||||
always agrees with it.
|
||||
|
||||
Only the table between that heading and the following "Product codes
|
||||
are..." paragraph is read, and only backtick-quoted tokens on table rows
|
||||
(lines starting with "|") are taken -- the product-code examples two
|
||||
paragraphs down (`PNL1.0`, `PNL101`, `ANCHOR_FACE_4T`) are prose, not a
|
||||
table row, and are deliberately NOT placeholders: CLAUDE.md says outright
|
||||
they "stay as they are", i.e. they must still be scrubbable text, not
|
||||
exempted from anything.
|
||||
|
||||
If neither heading can be found -- both files missing, moved, or
|
||||
reworded -- this returns [] rather than guessing at a fallback list. An
|
||||
empty result only makes the filename check stricter (nothing is exempt),
|
||||
never looser, which is the safe direction for a scrub gate to fail in.
|
||||
|
||||
NEITHER FILE, on a colleague distribution: CLAUDE.md is the owner's own
|
||||
agent operating manual and does not ship (see tools/make_dist.py) --
|
||||
CONTRIBUTING.md carries the identical heading and table there instead,
|
||||
and is checked second when `path` is not given explicitly. Measured
|
||||
2026-09-02: without this fallback, CI on a stripped checkout failed on
|
||||
the sanctioned placeholder itself (`sample_job.rvt` in the test suite),
|
||||
which is exactly the "nothing is exempt" failure mode above -- safe, but
|
||||
a false positive on a totally clean fixture, not a leak.
|
||||
"""
|
||||
paths = [path] if path else [CLAUDE_MD, CONTRIBUTING_MD]
|
||||
marker = 'No client or job information in the repo. Ever.'
|
||||
for candidate in paths:
|
||||
try:
|
||||
with open(candidate, encoding='utf-8') as fh:
|
||||
text = fh.read()
|
||||
except OSError:
|
||||
continue
|
||||
start = text.find(marker)
|
||||
if start == -1:
|
||||
continue
|
||||
end = text.find('Product codes are', start)
|
||||
block = text[start:end] if end != -1 else text[start:start + 2000]
|
||||
names = []
|
||||
for line in block.splitlines():
|
||||
line = line.strip()
|
||||
if not line.startswith('|'):
|
||||
continue
|
||||
for m in re.finditer(r'`([^`]+)`', line):
|
||||
names.append(m.group(1).strip().lower())
|
||||
if names:
|
||||
return names
|
||||
return []
|
||||
|
||||
|
||||
def _is_placeholder_filename(matched_text: str, placeholders: list[str]) -> bool:
|
||||
"""True only if `matched_text` IS a placeholder name plus extension --
|
||||
never if a placeholder merely appears somewhere inside it.
|
||||
|
||||
`os.path.splitext` peels off exactly the trailing `.rvt` / `.rfa`, and
|
||||
the stem that is left is compared to the placeholder list whole. That
|
||||
whole-name comparison is the point: a placeholder with anything else
|
||||
appended to its stem is a DIFFERENT stem, absent from the list, so it is
|
||||
still flagged. Only the placeholder alone, plus the extension, passes.
|
||||
"""
|
||||
if not placeholders:
|
||||
return False
|
||||
stem, ext = os.path.splitext(matched_text.strip())
|
||||
if ext.lower() not in ('.rvt', '.rfa'):
|
||||
return False
|
||||
return stem.lower() in placeholders
|
||||
|
||||
|
||||
|
||||
# `\b` treats an underscore as a word character, the same as a letter, so it
|
||||
# does not sit between them: a term does not match where it is immediately
|
||||
# followed by "_SOMETHING" -- there is no boundary at the position between
|
||||
# the term's last letter and the underscore. A term glued to a role
|
||||
# identifier, a piece mark, or a version number this way slips the scan
|
||||
# silently -- the exact shape of a name that is very much still in the file.
|
||||
#
|
||||
# What should end a match is a LETTER continuing the same word on either
|
||||
# side -- that is the one case that means "this is a substring of a longer
|
||||
# ordinary word, not the term" (a fake example: "Northgate" must not match
|
||||
# inside "Northgateway" or "preNorthgate"). An underscore, a digit,
|
||||
# punctuation, or the edge of the line all end the term just as surely as
|
||||
# whitespace does, so none of those may appear immediately adjacent to the
|
||||
# match either -- which is what these lookarounds check instead of `\b`.
|
||||
_TERM_BOUNDARY_BEFORE = r'(?<![A-Za-z])'
|
||||
_TERM_BOUNDARY_AFTER = r'(?![A-Za-z])'
|
||||
|
||||
|
||||
def patterns(terms: list[tuple[str, bool]]) -> list[tuple[re.Pattern, str]]:
|
||||
out = [(re.compile(p, re.I), why) for p, why in STRUCTURAL]
|
||||
for term, cased in terms:
|
||||
flags = 0 if cased else re.I
|
||||
out.append((re.compile(_TERM_BOUNDARY_BEFORE + re.escape(term) +
|
||||
_TERM_BOUNDARY_AFTER, flags),
|
||||
'term {0!r}'.format(term)))
|
||||
return out
|
||||
|
||||
|
||||
def git(*args: str) -> str:
|
||||
return subprocess.run(['git'] + list(args), cwd=ROOT, check=False,
|
||||
capture_output=True, text=True,
|
||||
errors='replace').stdout
|
||||
|
||||
|
||||
def tracked_files() -> list[str]:
|
||||
return [r for r in git('ls-files').splitlines() if r]
|
||||
|
||||
|
||||
def untracked_files() -> list[str]:
|
||||
"""Files git can see but is not tracking, .gitignore honoured.
|
||||
|
||||
These are where a name arrives: a scratch script somebody pasted a model
|
||||
path into an hour ago is invisible to plain 'git ls-files', which is what
|
||||
this scan used to run on. Ignored files stay out, so the real denylist and
|
||||
the local job folders are not scanned.
|
||||
"""
|
||||
return [r for r in git('ls-files', '--others',
|
||||
'--exclude-standard').splitlines() if r]
|
||||
|
||||
|
||||
def staged_files() -> list[str]:
|
||||
"""What this commit is actually about to write into history."""
|
||||
out = git('diff', '--cached', '--name-only', '--diff-filter=ACMR')
|
||||
return [r for r in out.splitlines() if r]
|
||||
|
||||
|
||||
def scan_worktree(pats, listed=None, placeholders=None) -> list[str]:
|
||||
hits = []
|
||||
if listed is None:
|
||||
listed = tracked_files() + untracked_files()
|
||||
for rel in listed:
|
||||
if os.path.basename(rel) in SKIP:
|
||||
continue
|
||||
path = os.path.join(ROOT, rel)
|
||||
if not os.path.isfile(path):
|
||||
continue
|
||||
try:
|
||||
with open(path, encoding='utf-8', errors='replace') as fh:
|
||||
lines = fh.readlines()
|
||||
except OSError:
|
||||
continue
|
||||
for n, line in enumerate(lines, 1):
|
||||
for pat, why in pats:
|
||||
m = pat.search(line)
|
||||
if not m:
|
||||
continue
|
||||
if why in FILENAME_CATEGORIES and \
|
||||
_is_placeholder_filename(m.group(0), placeholders):
|
||||
continue
|
||||
hits.append('{0}:{1}: {2} -- {3}'.format(
|
||||
rel, n, why, m.group(0)[:60]))
|
||||
return hits
|
||||
|
||||
|
||||
def scan_history(pats, placeholders=None) -> list[str]:
|
||||
"""Every version of every file that has ever been committed."""
|
||||
if not git('rev-parse', '--git-dir').strip():
|
||||
return []
|
||||
log = git('log', '--all', '--patch', '--no-color', '--format=%H%x00%s')
|
||||
hits, commit, subject = [], '?', '?'
|
||||
seen = set()
|
||||
for line in log.splitlines():
|
||||
if '\x00' in line and len(line.split('\x00')[0]) == 40:
|
||||
commit, subject = line.split('\x00', 1)
|
||||
continue
|
||||
if not line.startswith('+'):
|
||||
continue
|
||||
for pat, why in pats:
|
||||
m = pat.search(line)
|
||||
if not m:
|
||||
continue
|
||||
if why in FILENAME_CATEGORIES and \
|
||||
_is_placeholder_filename(m.group(0), placeholders):
|
||||
continue
|
||||
key = (commit, why, m.group(0))
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
hits.append('{0} {1!r}: {2} -- {3}'.format(
|
||||
commit[:8], subject[:40], why, m.group(0)[:60]))
|
||||
return hits
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- private tree
|
||||
#
|
||||
# Everything above honours .gitignore, on purpose and for good reasons -- and
|
||||
# the consequence is that nothing in this file has ever opened `QNA/`,
|
||||
# `knowledge/plant/` or `knowledge/job/`. Those are the other repository's
|
||||
# territory (see the .gitignore comment that says so), a name is legitimate
|
||||
# in them, and they are also the largest body of prose in this working tree
|
||||
# and the only part of it whose job is to quote source material verbatim. A
|
||||
# name is not merely possible there, it is certain.
|
||||
#
|
||||
# So this is deliberately NOT a gate and must never be grown into one. If it
|
||||
# flagged every producer name in a plant file it would print a wall of
|
||||
# expected hits, and a wall of expected hits gets read exactly once. That is
|
||||
# the same failure --staged avoids by not blocking a commit on untracked
|
||||
# files, one directory further out.
|
||||
#
|
||||
# The question --private asks is therefore not "is there a name in here" --
|
||||
# answered yes, obviously, that is what the file is -- but "is there a name
|
||||
# in here in a shape that somebody will COPY". A sentence of prose about a
|
||||
# producer's practice stays in the private repo when the note is read. A
|
||||
# fenced block, a proposed YAML key, a quoted literal, a path: those are the
|
||||
# strings that get lifted out of an extraction note and pasted into a module,
|
||||
# a docs page or a test, and they cross the repo boundary with the name still
|
||||
# inside them. That is the distinction this mode is built on, and it is what
|
||||
# keeps the report short enough to still be read next month. (`_scratch/` and
|
||||
# `sample data/` are gitignored too and get the same treatment, for the same
|
||||
# reason: nothing was looking at them either.)
|
||||
#
|
||||
# Be honest about what it cannot do, because the shape of the blind spot is
|
||||
# the useful part:
|
||||
#
|
||||
# * It cannot judge prose. A name in a sentence is counted, never read, so
|
||||
# a producer's name inside a verbatim quotation that somebody later
|
||||
# pastes whole into a docs page is resident here and invisible. What
|
||||
# catches that is --worktree, on the day it lands in a tracked file.
|
||||
# * It cannot read a PDF, and the exported tickets carry the job name in
|
||||
# the title block. Those are in the "not read" tally, deliberately
|
||||
# visible as a number.
|
||||
# * "Travelling" is a guess about a shape, not a proof about a future.
|
||||
# `key: value` in prose and a Markdown fence that is really a diagram
|
||||
# both look like code to it, and a name written into a sentence that is
|
||||
# about to become a docstring does not.
|
||||
# * It only knows names that are on the term list, plus the structural
|
||||
# patterns. A producer nobody has added to `.scrub-terms` is as
|
||||
# invisible here as it is everywhere else.
|
||||
|
||||
PRIVATE_SKIP_DIRS = {'__pycache__', 'dist', 'build', 'node_modules',
|
||||
'site-packages'}
|
||||
|
||||
# Opaque to a line scanner: binary, or an export whose text this cannot read
|
||||
# anyway. They are counted and reported as a number rather than passed over
|
||||
# silently -- "not scanned" and "scanned, clean" are different answers and a
|
||||
# scrub report may not collapse them. A ticket PDF is the standing example:
|
||||
# it carries the job name in its title block and nothing here can see it.
|
||||
PRIVATE_SKIP_EXT = {'.pdf', '.rvt', '.rfa', '.png', '.jpg', '.jpeg', '.gif',
|
||||
'.bmp', '.ico', '.zip', '.7z', '.gz', '.tar', '.xlsx',
|
||||
'.xls', '.docx', '.doc', '.pptx', '.pyc', '.pyo', '.dll',
|
||||
'.exe', '.so', '.dat', '.db', '.sqlite', '.bin'}
|
||||
|
||||
PRIVATE_MAX_BYTES = 2000000
|
||||
|
||||
# What kind of file this is decides which contexts count as travelling below.
|
||||
PRIVATE_DOC_EXT = {'', '.md', '.markdown', '.txt', '.text', '.rst'}
|
||||
PRIVATE_DATA_EXT = {'.yaml', '.yml', '.json', '.csv', '.tsv', '.toml', '.ini',
|
||||
'.cfg', '.xml'}
|
||||
|
||||
_FENCE = re.compile(r'^\s{0,3}(```|~~~)')
|
||||
_INLINE_CODE = re.compile(r'`[^`]+`')
|
||||
_QUOTED = re.compile(r"'[^']*'|\"[^\"]*\"")
|
||||
# A YAML-shaped line: a space-free lowercase key, a colon, a value. Prose
|
||||
# rarely looks like this -- "Note: the producer says" is capitalised and
|
||||
# "the plant says: x" has a space in the key, and the key pattern excludes
|
||||
# both. What this is for is the proposed shape written INTO a note --
|
||||
# `plant_default: 4` in the middle of a paragraph -- which is a line somebody
|
||||
# is about to paste into knowledge.DEFAULTS.
|
||||
_YAML_LINE = re.compile(r'^\s*(-\s+)?[a-z_][a-z0-9_.\-]*:\s*\S')
|
||||
# A whitespace-and-punctuation-delimited token: the unit a name gets glued
|
||||
# into, and the unit somebody copies. Angle brackets and `=` are delimiters
|
||||
# too, or a name inside a generated HTML report comes back as sixty
|
||||
# characters of markup with the interesting part buried in the middle.
|
||||
_TOKEN = re.compile(r'[^\s`\'"(),;<>=]+')
|
||||
|
||||
|
||||
def ignored_files() -> list[str]:
|
||||
"""Files git can see and is deliberately ignoring.
|
||||
|
||||
The mirror image of `untracked_files()`: the same walk, the other side of
|
||||
.gitignore. Between this, `untracked_files()` and `tracked_files()` there
|
||||
is no longer any part of the working tree that no mode ever opens.
|
||||
"""
|
||||
out = git('ls-files', '--others', '--ignored', '--exclude-standard')
|
||||
return [r for r in out.splitlines() if r]
|
||||
|
||||
|
||||
def private_files(listed=None):
|
||||
"""The gitignored files worth reading, and a tally of what was dropped.
|
||||
|
||||
The tally is returned rather than discarded because the count is part of
|
||||
the answer. A report that says "480 files read" while silently passing
|
||||
over 200 PDFs is claiming a coverage it does not have -- the same reason
|
||||
a check that could not run may not render as OK.
|
||||
|
||||
Dot-prefixed path segments go in the tally, not the scan: one agent's
|
||||
browser profile cache under `QNA/` is 58,000 files on its own, and
|
||||
dot-names in a working tree are tool state rather than documents.
|
||||
"""
|
||||
if listed is None:
|
||||
listed = ignored_files()
|
||||
keep = []
|
||||
dropped = {'tool state': 0, 'build output': 0, 'unreadable': 0,
|
||||
'over size cap': 0, 'gone': 0}
|
||||
for rel in listed:
|
||||
parts = rel.replace('\\', '/').split('/')
|
||||
if any(p.startswith('.') for p in parts):
|
||||
dropped['tool state'] += 1
|
||||
continue
|
||||
if os.path.basename(rel) in SKIP:
|
||||
dropped['tool state'] += 1
|
||||
continue
|
||||
if any(p in PRIVATE_SKIP_DIRS for p in parts):
|
||||
dropped['build output'] += 1
|
||||
continue
|
||||
if os.path.splitext(rel)[1].lower() in PRIVATE_SKIP_EXT:
|
||||
dropped['unreadable'] += 1
|
||||
continue
|
||||
try:
|
||||
size = os.path.getsize(os.path.join(ROOT, rel))
|
||||
except OSError:
|
||||
dropped['gone'] += 1
|
||||
continue
|
||||
if size > PRIVATE_MAX_BYTES:
|
||||
dropped['over size cap'] += 1
|
||||
continue
|
||||
keep.append(rel)
|
||||
return keep, dropped
|
||||
|
||||
|
||||
def _file_kind(rel: str) -> str:
|
||||
ext = os.path.splitext(rel)[1].lower()
|
||||
if ext in PRIVATE_DOC_EXT:
|
||||
return 'doc'
|
||||
if ext in PRIVATE_DATA_EXT:
|
||||
return 'data'
|
||||
return 'code'
|
||||
|
||||
|
||||
def _token_at(line: str, pos: int) -> str:
|
||||
for tok in _TOKEN.finditer(line):
|
||||
if tok.start() <= pos < tok.end():
|
||||
return tok.group(0)
|
||||
return ''
|
||||
|
||||
|
||||
def _inside(pos: int, spans) -> bool:
|
||||
"""True if `pos` is strictly inside one of these delimited spans, i.e.
|
||||
between the delimiters rather than on one."""
|
||||
return any(s.start() < pos < s.end() - 1 for s in spans)
|
||||
|
||||
|
||||
def _travels(kind: str, line: str, start: int, why: str, in_fence: bool):
|
||||
"""Where this hit sits, when that place is one a string travels out of --
|
||||
otherwise None, meaning the name is *resident*: this file's own subject
|
||||
matter, sitting in the repository where it is allowed to be.
|
||||
|
||||
Read the branches as a claim about copy-and-paste, not about danger.
|
||||
Nothing here is unsafe where it is. The only question is which of these
|
||||
strings has a plausible future in a tracked file.
|
||||
"""
|
||||
if not why.startswith('term '):
|
||||
# Structural. A machine's user path, an email address, a model or
|
||||
# family filename: none of those is a plant convention or a
|
||||
# producer's vocabulary. Each identifies one person, one machine or
|
||||
# one job, and reads identically wherever it is pasted, so there is
|
||||
# no "doing its job" reading of it to weigh against travelling.
|
||||
return 'structural'
|
||||
tok = _token_at(line, start)
|
||||
if '/' in tok or '\\' in tok:
|
||||
return 'in a path'
|
||||
if kind == 'code':
|
||||
# A private script is all instruction. There is no prose half of it
|
||||
# for a name to be resident in, and a scratch script is the exact
|
||||
# thing the untracked-file scan already exists for -- this one just
|
||||
# happens to sit under an ignored directory.
|
||||
return 'in code'
|
||||
if kind == 'data':
|
||||
# Values in a plant or job file ARE the file. A comment is not: it
|
||||
# is written for a reader, in sentences, and it is what gets carried
|
||||
# across when the shape of the file is proposed somewhere else.
|
||||
hash_at = line.find('#')
|
||||
if hash_at != -1 and hash_at < start:
|
||||
return 'in a comment'
|
||||
return None
|
||||
if in_fence:
|
||||
return 'in a fenced code block'
|
||||
if _inside(start, _INLINE_CODE.finditer(line)):
|
||||
return 'in inline code'
|
||||
if _inside(start, _QUOTED.finditer(line)):
|
||||
return 'in a quoted literal'
|
||||
if _YAML_LINE.match(line):
|
||||
return 'in a key: value line'
|
||||
return None
|
||||
|
||||
|
||||
def _safe(text: str) -> str:
|
||||
"""Whatever of `text` this console can actually print.
|
||||
|
||||
The private tree is notes, and notes are written in whatever language the
|
||||
note-taker was thinking in. A Windows console defaulting to cp1252 raises
|
||||
UnicodeEncodeError on the first character it cannot render, which kills
|
||||
the process halfway down the report -- a scrub that stops printing looks
|
||||
exactly like a scrub that found nothing more.
|
||||
"""
|
||||
enc = getattr(sys.stdout, 'encoding', None) or 'ascii'
|
||||
return text.encode(enc, 'replace').decode(enc, 'replace')
|
||||
|
||||
|
||||
def scan_private(pats, listed=None, placeholders=None):
|
||||
"""Read the private tree and sort what is found into three piles.
|
||||
|
||||
Returns `(travelling, named_dirs, resident)`:
|
||||
|
||||
travelling {token: [count, example, why, glued, where]} -- names in
|
||||
a shape that gets copied. Keyed on the whole delimited
|
||||
TOKEN the name sits in, not on the line, because the unit
|
||||
that travels is a string, not an occurrence: forty
|
||||
mentions of one parameter name are one thing to decide
|
||||
about, and forty lines of it is the wall this mode exists
|
||||
to avoid printing. `glued` is True when the token is
|
||||
larger than the name itself -- the name buried inside an
|
||||
identifier, a filename, a piece mark. That is the shape a
|
||||
human reader skims past, so it sorts to the top.
|
||||
named_dirs {(why, text): [count, example]} -- names in the tree's own
|
||||
file and folder names, aggregated by the name. A job
|
||||
folder named after the job produces one of these for every
|
||||
file underneath it; 380 identical lines would drown
|
||||
everything above.
|
||||
resident {path: count} -- everything else, reported as numbers.
|
||||
Never listed. A producer's name in a sentence about that
|
||||
producer, in the repository that documents them, is the
|
||||
file working correctly.
|
||||
"""
|
||||
if listed is None:
|
||||
listed = private_files()[0]
|
||||
travelling, named_dirs, resident = {}, {}, {}
|
||||
for rel in listed:
|
||||
slashed = rel.replace('\\', '/')
|
||||
for pat, why in pats:
|
||||
m = pat.search(slashed)
|
||||
if not m:
|
||||
continue
|
||||
if why in FILENAME_CATEGORIES and \
|
||||
_is_placeholder_filename(m.group(0), placeholders):
|
||||
continue
|
||||
entry = named_dirs.setdefault((why, m.group(0)[:40]), [0, slashed])
|
||||
entry[0] += 1
|
||||
try:
|
||||
with open(os.path.join(ROOT, rel), encoding='utf-8',
|
||||
errors='replace') as fh:
|
||||
lines = fh.readlines()
|
||||
except OSError:
|
||||
continue
|
||||
kind = _file_kind(rel)
|
||||
in_fence = False
|
||||
for n, line in enumerate(lines, 1):
|
||||
if kind == 'doc' and _FENCE.match(line):
|
||||
in_fence = not in_fence
|
||||
continue
|
||||
for pat, why in pats:
|
||||
m = pat.search(line)
|
||||
if not m:
|
||||
continue
|
||||
if why in FILENAME_CATEGORIES and \
|
||||
_is_placeholder_filename(m.group(0), placeholders):
|
||||
continue
|
||||
where = _travels(kind, line, m.start(), why, in_fence)
|
||||
if where is None:
|
||||
resident[slashed] = resident.get(slashed, 0) + 1
|
||||
continue
|
||||
if where == 'structural':
|
||||
# The pattern already matched the whole string that
|
||||
# identifies somebody -- a user path, an address, a model
|
||||
# filename. Widening that to the surrounding token only
|
||||
# chops it at the first space and prints half of it.
|
||||
token = m.group(0)
|
||||
else:
|
||||
token = _token_at(line, m.start()) \
|
||||
.strip('`\'"*,.:;()[]{}') or m.group(0)
|
||||
glued = token.lower() != m.group(0).lower()
|
||||
# Keyed on the token alone, not on (where, token): one string
|
||||
# is one thing to decide about, and the same identifier
|
||||
# quoted in a fence in one note, inline in another and in a
|
||||
# scratch script in a third is still that one string. The
|
||||
# first `where` seen labels it, and structural patterns run
|
||||
# before terms, so the more specific label wins.
|
||||
entry = travelling.setdefault(
|
||||
token[:70],
|
||||
[0, '{0}:{1}'.format(slashed, n), why, glued, where])
|
||||
entry[0] += 1
|
||||
return travelling, named_dirs, resident
|
||||
|
||||
|
||||
def report_private(listed, dropped, travelling, named_dirs, resident,
|
||||
show=30) -> None:
|
||||
"""Print the three piles: the decidable one first, the expected ones as
|
||||
numbers. Every section is capped, because a section that can grow without
|
||||
limit is a section that stops being read."""
|
||||
skipped = sum(dropped.values())
|
||||
print('\nprivate tree -- {0} gitignored file(s) read, {1} not read'
|
||||
.format(len(listed), skipped))
|
||||
if skipped:
|
||||
print(' not read: ' + ', '.join(
|
||||
'{0} {1}'.format(v, k) for k, v in sorted(dropped.items()) if v))
|
||||
|
||||
if travelling:
|
||||
# Glued first: the name buried inside a longer identifier is the one
|
||||
# a human eye slides over. A bare mention in a code span is still a
|
||||
# word somebody can see is a name.
|
||||
ordered = sorted(travelling.items(),
|
||||
key=lambda kv: (not kv[1][3], -kv[1][0], kv[0]))
|
||||
glued_n = sum(1 for _, v in ordered if v[3])
|
||||
print('\nTRAVELLING -- {0} distinct string(s) in a shape that gets '
|
||||
'copied, {1} of them with the name glued into a longer token. '
|
||||
'Legitimate where they are; the question is only whether one '
|
||||
'has already been pasted somewhere tracked:'
|
||||
.format(len(ordered), glued_n))
|
||||
for token, (count, example, why, glued, where) in ordered[:show]:
|
||||
print(_safe(' {0}{1}: {2} -- {3} x{4}, e.g. {5}'.format(
|
||||
'*' if glued else ' ', where, token, why, count, example)))
|
||||
if len(ordered) > show:
|
||||
print(' ... and {0} more distinct string(s)'
|
||||
.format(len(ordered) - show))
|
||||
|
||||
if named_dirs:
|
||||
ordered = sorted(named_dirs.items(), key=lambda kv: (-kv[1][0], kv[0]))
|
||||
print('\nnamed paths -- {0} name(s) in the private tree\'s own file '
|
||||
'and folder names. Nothing to fix here, but a tracked file that '
|
||||
'quotes one of these paths publishes it:'.format(len(ordered)))
|
||||
for (why, text), (count, example) in ordered[:show]:
|
||||
print(_safe(' {0}: {1} -- in {2} path(s), e.g. {3}'
|
||||
.format(why, text, count, example)))
|
||||
if len(ordered) > show:
|
||||
print(' ... and {0} more'.format(len(ordered) - show))
|
||||
|
||||
if resident:
|
||||
total = sum(resident.values())
|
||||
ordered = sorted(resident.items(), key=lambda kv: (-kv[1], kv[0]))
|
||||
print('\nresident -- {0} match(es) in {1} file(s), not listed. A name '
|
||||
'in prose, or in a data value, is these files doing their job:'
|
||||
.format(total, len(ordered)))
|
||||
for rel, count in ordered[:show]:
|
||||
print(_safe(' {0:>6} {1}'.format(count, rel)))
|
||||
if len(ordered) > show:
|
||||
print(' ... and {0} more file(s)'.format(len(ordered) - show))
|
||||
|
||||
# This mode has no verdict and deliberately returns 0 whatever it finds:
|
||||
# see the paragraph above `main`'s --private branch.
|
||||
print('\nThis is an inventory, not a gate. Nothing above blocks a commit, '
|
||||
'and nothing above is wrong for being where it is -- the question '
|
||||
'each row asks is whether that exact string has already been '
|
||||
'copied into a tracked file. Check one with:\n'
|
||||
' git grep -i -- "<string>"')
|
||||
|
||||
|
||||
# The interpreter is baked in rather than left as bare `python`, and that is
|
||||
# not fussiness. On Windows `python` usually resolves to the Microsoft Store
|
||||
# alias in WindowsApps: a stub that prints "Python was not found" and exits
|
||||
# non-zero. The hook then blocks every commit on this repository with a
|
||||
# message about the Store, which looks nothing like a scrub failure and sends
|
||||
# you looking in the wrong place entirely.
|
||||
#
|
||||
# A hook is per-clone and not part of the repository, so pinning the
|
||||
# interpreter that installed it is exactly the right scope. The fallback chain
|
||||
# is there for a clone whose Python later moves.
|
||||
HOOK = """#!/bin/sh
|
||||
# Installed by tools/scrub.py --install-hook. Blocks a commit carrying a
|
||||
# client name. Bypass with: git commit --no-verify
|
||||
for py in "{python}" python3 python; do
|
||||
case "$py" in *WindowsApps*|"") continue;; esac
|
||||
command -v "$py" >/dev/null 2>&1 || [ -x "$py" ] || continue
|
||||
exec "$py" tools/scrub.py --staged
|
||||
done
|
||||
echo "scrub: no Python found. Reinstall the hook with:" >&2
|
||||
echo " python tools/scrub.py --install-hook" >&2
|
||||
exit 1
|
||||
"""
|
||||
|
||||
|
||||
def install_hook() -> int:
|
||||
"""Put the worktree scan in front of every commit on this clone.
|
||||
|
||||
Hooks are not part of a repository, so this has to be run once per clone
|
||||
-- including on the office machine after cloning.
|
||||
"""
|
||||
hooks = os.path.join(git('rev-parse', '--git-dir').strip() or '.git',
|
||||
'hooks')
|
||||
if not os.path.isdir(hooks):
|
||||
os.makedirs(hooks)
|
||||
path = os.path.join(hooks, 'pre-commit')
|
||||
with open(path, 'w', encoding='utf-8', newline='\n') as fh:
|
||||
fh.write(HOOK.format(python=sys.executable.replace('\\', '/')))
|
||||
os.chmod(path, 0o755)
|
||||
print('installed pre-commit hook: {0}\n using {1}'
|
||||
.format(path, sys.executable))
|
||||
return 0
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument('--worktree', action='store_true',
|
||||
help='skip the history scan')
|
||||
ap.add_argument('--staged', action='store_true',
|
||||
help='what the pre-commit hook runs: hits in STAGED files '
|
||||
'block the commit, hits in untracked files are '
|
||||
'reported but do not. An untracked file cannot reach '
|
||||
'history without being staged first, and blocking on '
|
||||
'one stops work that has nothing to do with the '
|
||||
'commit -- which is how a gate ends up bypassed with '
|
||||
'--no-verify, and then it guards nothing.')
|
||||
ap.add_argument('--private', action='store_true',
|
||||
help='audit the gitignored tree (QNA/, knowledge/plant/, '
|
||||
'knowledge/job/) -- the other repository, where a '
|
||||
'name is allowed to be. An inventory, not a gate: '
|
||||
'never run by the hook and it always exits 0. It '
|
||||
'lists only names sitting in a shape that travels '
|
||||
'(a fenced block, a proposed key, a quoted literal, '
|
||||
'a path, a name glued into a longer identifier), '
|
||||
'aggregated by the distinct string, and counts the '
|
||||
'rest without listing them -- a wall of expected '
|
||||
'hits is a report nobody reads twice.')
|
||||
ap.add_argument('--terms', default=None,
|
||||
help="terms file to scan for (default .scrub-terms). CI "
|
||||
"passes .scrub-terms.example, the committed generic "
|
||||
"sample file, so it never needs the real denylist.")
|
||||
ap.add_argument('--install-hook', action='store_true',
|
||||
help='install as a pre-commit hook on this clone')
|
||||
args = ap.parse_args()
|
||||
|
||||
if args.install_hook:
|
||||
return install_hook()
|
||||
|
||||
terms = load_terms(args.terms)
|
||||
if not terms:
|
||||
print('note: no .scrub-terms file, only structural patterns checked.\n'
|
||||
' copy .scrub-terms.example and add your client names.',
|
||||
file=sys.stderr)
|
||||
pats = patterns(terms)
|
||||
|
||||
placeholders = load_placeholders()
|
||||
if not placeholders:
|
||||
print("note: could not find CLAUDE.md's placeholder table -- the "
|
||||
'model/family-filename patterns will flag the sanctioned '
|
||||
'placeholder name too, with nothing exempted.',
|
||||
file=sys.stderr)
|
||||
|
||||
if args.private:
|
||||
listed, dropped = private_files()
|
||||
travelling, named_paths, resident = scan_private(
|
||||
pats, listed, placeholders)
|
||||
report_private(listed, dropped, travelling, named_paths, resident)
|
||||
# Always 0, deliberately -- see the paragraph in the module docstring.
|
||||
# Every string this prints carries a denylist term or a structural
|
||||
# pattern, so a copy of one into a tracked file is already a --worktree
|
||||
# failure; there is no leak here for an exit code to catch that is not
|
||||
# caught one directory out. What is here is the raw material, and a
|
||||
# mode that failed on the raw material would fail on every run.
|
||||
return 0
|
||||
|
||||
loose = []
|
||||
if args.staged:
|
||||
worktree = scan_worktree(pats, staged_files(), placeholders)
|
||||
loose = scan_worktree(pats, untracked_files(), placeholders)
|
||||
history = []
|
||||
else:
|
||||
worktree = scan_worktree(pats, placeholders=placeholders)
|
||||
history = [] if args.worktree else scan_history(pats, placeholders)
|
||||
|
||||
label_now = 'staged for this commit' if args.staged else 'working tree'
|
||||
for label, hits in ((label_now, worktree), ('git history', history)):
|
||||
if hits:
|
||||
print('\n{0} -- {1} hit(s):'.format(label, len(hits)))
|
||||
for hit in hits:
|
||||
print(' ' + hit)
|
||||
|
||||
if loose:
|
||||
# Not fatal, and said every time until it is dealt with. These are one
|
||||
# 'git add -A' away from being the blocking kind.
|
||||
print('\nWARNING -- {0} hit(s) in untracked files. Not committed, and '
|
||||
'not blocking this commit, but one "git add -A" away from '
|
||||
'being:'.format(len(loose)))
|
||||
for hit in loose:
|
||||
print(' ' + hit)
|
||||
|
||||
if not args.staged:
|
||||
# The blind spot, named out loud in every audit run rather than left
|
||||
# to be rediscovered. These files are not scanned here and should not
|
||||
# be -- a hit in them is not a leak -- but "nothing looks at all" is
|
||||
# how a name sits in a note for six weeks and then gets pasted into a
|
||||
# module. --private is what looks.
|
||||
n_private = len(private_files()[0])
|
||||
print('\nnote: {0} readable gitignored file(s) were not scanned here '
|
||||
'-- QNA/, knowledge/plant/, knowledge/job/, _scratch/. A name '
|
||||
'is allowed in those, so they are not part of this verdict, '
|
||||
'but nothing else looks at them either:\n'
|
||||
' python tools/scrub.py --private'.format(n_private))
|
||||
|
||||
if not worktree and not history:
|
||||
if args.staged:
|
||||
print('clean: {0} staged file(s) checked against {1} term(s) + {2} '
|
||||
'structural pattern(s)'
|
||||
.format(len(staged_files()), len(terms), len(STRUCTURAL)))
|
||||
return 0
|
||||
scope = 'working tree' if args.worktree else 'working tree and history'
|
||||
print('clean: {0} checked against {1} term(s) + {2} structural pattern(s)'
|
||||
.format(scope, len(terms), len(STRUCTURAL)))
|
||||
return 0
|
||||
|
||||
if history:
|
||||
print('\nA hit in history is not fixed by editing the file. If nothing '
|
||||
'has been pushed yet, rewrite the history; if it has, the name '
|
||||
'is public and rotating the repo is the only real fix.')
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,445 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Phase C's host half: wait for Revit to exit, then make the detached copy.
|
||||
|
||||
python tools/snapshot_defer.py --request <req.json> --runner <run.py> \
|
||||
[--pyrevit <pyrevit.exe>] --wait-pid <revit pid> [--revit 2025]
|
||||
|
||||
`--pyrevit` is auto-discovered at the usual install path when left off
|
||||
(`vibe.snapshotmeta.pyrevit_exe_guess`) -- the same fallback the two callers
|
||||
already made for themselves; passing it explicitly still wins. `--revit` is
|
||||
also worth passing explicitly on any machine with more than one Revit
|
||||
version attached: see WHAT IT GUARANTEES below for why omitting it is now a
|
||||
loud warning rather than a silent maybe.
|
||||
|
||||
python tools/snapshot_defer.py --request <req.json> ... --dry-run
|
||||
python tools/snapshot_defer.py --newest <folder>
|
||||
|
||||
WHY A SEPARATE PROCESS EXISTS AT ALL
|
||||
-------------------------------------
|
||||
The machine cannot hold two copies of this model. 300+ MB on disk, 10+ GB
|
||||
open; the engineer's session and a scanning session cannot both be resident,
|
||||
which is what killed the original concurrent design. So the copy is made
|
||||
AFTER his Revit has exited, and something has to still be running to notice
|
||||
that it has. That something is this: a host process, started by the button,
|
||||
which outlives the Revit that started it, sleeps until the pid is gone, and
|
||||
then launches exactly one Revit of its own.
|
||||
|
||||
THE PARAGRAPH ABOVE IS NO LONGER TRUE, AND `--machine-free` IS THE SWITCH
|
||||
--------------------------------------------------------------------------
|
||||
Measured 2026-09-01: the live cloud session and a detached sandbox session
|
||||
ran side by side on this machine for a whole morning. Two Revits are fine;
|
||||
the untested number is three. So the machine-wide gate -- "wait until NO
|
||||
Revit at all is running" -- is now OPT-IN:
|
||||
|
||||
--machine-free ignore (default) only the watched pid is waited for
|
||||
--machine-free require the old behaviour, for the close-model ritual
|
||||
|
||||
`tools/snapshot_cycle.py` calls this with the default and sequences instead:
|
||||
it closes the sandbox Revit BEFORE launching the one that writes the copy, so
|
||||
two is the most that are ever resident. The `require` mode is kept, not
|
||||
deleted, because the close-model ritual on a slower machine is a real use for
|
||||
it and the give-up-and-journal behaviour underneath it is the tested part.
|
||||
|
||||
It is deliberately small. Everything it decides -- what the copy is called,
|
||||
which snapshot is newest, what goes in the log -- lives in
|
||||
`vibe.snapshotmeta`, where it is unit-tested without Revit. What is left here
|
||||
is waiting, launching, and reporting, and the two of those that can be tested
|
||||
without a licence (`wait_for_exit`, `command_for`) are pure functions with
|
||||
their clock and their liveness check injected.
|
||||
|
||||
WHAT IT GUARANTEES
|
||||
-------------------
|
||||
* It never starts Revit while the watched pid is alive.
|
||||
* Under `--machine-free require` it never starts Revit while ANY Revit is
|
||||
running -- not just that pid. He can close Revit and open it again while
|
||||
this is sleeping, and under the old one-model premise launching into that
|
||||
would have put two 10 GB models in memory.
|
||||
* It gives up after `--wait-minutes` and JOURNALS THE GIVE-UP. A Revit sitting
|
||||
on an unanswered save prompt is the ordinary reason, and a day when the copy
|
||||
silently did not happen must not look like a day when it did.
|
||||
* It checks afterwards that the file it was promised actually exists, and
|
||||
journals a failure if not -- the in-Revit runner can die in ways that take
|
||||
its own logging with it.
|
||||
|
||||
Host CPython 3.12. The extension side is IronPython 2.7; the shared module is
|
||||
written to both, and loaded here by path for the same reason the test suite
|
||||
does it -- importing `vibe` as a package runs `vibe/__init__.py`, which pulls
|
||||
in the Revit API and dies.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import traceback
|
||||
import types
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
VIBE = os.path.join(ROOT, 'WillTools.extension', 'lib', 'vibe')
|
||||
|
||||
if 'vibe' not in sys.modules:
|
||||
_stub = types.ModuleType('vibe')
|
||||
_stub.__path__ = [VIBE]
|
||||
_stub.__file__ = os.path.join(VIBE, '__init__.py')
|
||||
sys.modules['vibe'] = _stub
|
||||
|
||||
from vibe import snapshotmeta as S # noqa: E402
|
||||
|
||||
# Revit does not release the model the instant its window closes: the process
|
||||
# lingers while it flushes journals and drops file locks. Starting a new Revit
|
||||
# into that window is how you get a licence check-out race and a half-freed
|
||||
# 10 GB. Measured nowhere -- this is a guess, deliberately generous, and it is
|
||||
# the first number to turn down if the wait feels long.
|
||||
SETTLE_SECONDS = 30
|
||||
|
||||
POLL_SECONDS = 5
|
||||
|
||||
|
||||
# ------------------------------------------------------------- is it gone yet
|
||||
|
||||
def process_alive(pid):
|
||||
"""True while `pid` is a live process. Windows, no third-party module.
|
||||
|
||||
An unknown answer is reported as ALIVE, never as gone: being wrong in that
|
||||
direction costs a wait, and being wrong the other way starts a second
|
||||
Revit next to a live one -- the exact thing this whole design exists to
|
||||
prevent.
|
||||
"""
|
||||
if not pid:
|
||||
return False
|
||||
try:
|
||||
import ctypes
|
||||
from ctypes import wintypes
|
||||
SYNCHRONIZE = 0x00100000
|
||||
PROCESS_QUERY_LIMITED_INFORMATION = 0x1000
|
||||
STILL_ACTIVE = 259
|
||||
kernel32 = ctypes.windll.kernel32
|
||||
handle = kernel32.OpenProcess(
|
||||
SYNCHRONIZE | PROCESS_QUERY_LIMITED_INFORMATION, False, int(pid))
|
||||
if not handle:
|
||||
return False # no such process, or already exited
|
||||
try:
|
||||
code = wintypes.DWORD()
|
||||
if not kernel32.GetExitCodeProcess(handle, ctypes.byref(code)):
|
||||
return True # could not tell -> assume alive
|
||||
return code.value == STILL_ACTIVE
|
||||
finally:
|
||||
kernel32.CloseHandle(handle)
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
out = subprocess.run(
|
||||
['tasklist', '/FI', 'PID eq {0}'.format(pid), '/NH'],
|
||||
capture_output=True, text=True, timeout=30).stdout
|
||||
return str(pid) in out
|
||||
except Exception:
|
||||
return True # could not tell -> assume alive
|
||||
|
||||
|
||||
def any_revit_running(run=None):
|
||||
"""True while ANY Revit is running on this machine, whoever started it.
|
||||
|
||||
Waiting for the button's own pid is not enough. The engineer can close
|
||||
Revit, get a coffee, and open it again inside the window this tool is
|
||||
sleeping through -- and then launching Revit would put a second 10 GB
|
||||
model next to his live one, which is the exact failure the whole deferred
|
||||
design exists to avoid. So the last check before launching is not "has
|
||||
that process gone" but "is the machine free".
|
||||
|
||||
Unknown answers count as RUNNING, for the same reason as in
|
||||
`process_alive`: the cost of being wrong is a wait in one direction and a
|
||||
thrashed workstation in the other.
|
||||
"""
|
||||
run = run or (lambda argv: subprocess.run(
|
||||
argv, capture_output=True, text=True, timeout=30).stdout)
|
||||
try:
|
||||
out = run(['tasklist', '/FI', 'IMAGENAME eq Revit.exe', '/NH'])
|
||||
except Exception:
|
||||
return True
|
||||
return 'Revit.exe' in (out or '')
|
||||
|
||||
|
||||
def wait_for_exit(pid, timeout_s, alive=None, sleep=None, clock=None,
|
||||
poll=POLL_SECONDS):
|
||||
"""Block until `pid` is gone. True if it went, False on timeout.
|
||||
|
||||
Every dependency is injected so the loop can be tested with no process and
|
||||
no wall clock. It returns rather than raising on timeout because the
|
||||
timeout is a REPORTABLE OUTCOME, not an error: somebody left a save prompt
|
||||
open, and the next press of the button needs to say so.
|
||||
"""
|
||||
if not pid:
|
||||
# Nothing to wait for. Guarded here as well as in `process_alive`
|
||||
# because this loop's only exits are "the process went" and "the clock
|
||||
# advanced past the timeout" -- a caller that passes no pid and a
|
||||
# liveness check that always says yes would otherwise spin forever,
|
||||
# which is exactly how this was first written and exactly what the
|
||||
# test suite caught.
|
||||
return True
|
||||
alive = alive or process_alive
|
||||
sleep = sleep or time.sleep
|
||||
clock = clock or time.time
|
||||
started = clock()
|
||||
while alive(pid):
|
||||
if clock() - started >= timeout_s:
|
||||
return False
|
||||
sleep(poll)
|
||||
return True
|
||||
|
||||
|
||||
# ----------------------------------------------------------- the command line
|
||||
|
||||
def command_for(pyrevit_exe, runner, revit_year=None):
|
||||
"""The exact `pyrevit run` argv. One source for it, so the line printed by
|
||||
`--dry-run` and the line actually executed cannot drift apart."""
|
||||
return S.pyrevit_run_argv(pyrevit_exe, runner, revit_year)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- the run
|
||||
|
||||
def load_request(path):
|
||||
with open(path, 'rb') as handle:
|
||||
return json.loads(handle.read().decode('utf-8'))
|
||||
|
||||
|
||||
def fallback_log_for(request_path):
|
||||
"""`<detached_dir>/snapshots.jsonl`, recovered from the REQUEST'S OWN
|
||||
PATH rather than its contents.
|
||||
|
||||
For the one failure `journal()` cannot report: the request file could not
|
||||
even be read, so there is no `log` field to send the line to. But every
|
||||
request this tool is ever handed lives at
|
||||
`<detached_dir>/_requests/<run>.json` -- the button writes it there, and
|
||||
so does `tools/snapshot_cycle.py`'s own `write_detach_request` -- so the
|
||||
well-known log path can be recovered from the FOLDER STRUCTURE alone,
|
||||
with no JSON parse required. None when the path does not have that
|
||||
shape (a caller pointed `--request` at something else entirely), because
|
||||
a log path guessed at random is worse than no journal line at all.
|
||||
"""
|
||||
try:
|
||||
requests_dir = os.path.dirname(os.path.abspath(request_path))
|
||||
if os.path.basename(requests_dir) != '_requests':
|
||||
return None
|
||||
return S.log_path(os.path.dirname(requests_dir))
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def journal(request, event, **fields):
|
||||
path = (request or {}).get('log')
|
||||
if not path:
|
||||
return
|
||||
row = {'event': event, 'run': (request or {}).get('run')}
|
||||
row.update(fields)
|
||||
S.append(path, row)
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
ap.add_argument('--request', help='the request file the button wrote')
|
||||
ap.add_argument('--runner', help='the script pyrevit run executes')
|
||||
ap.add_argument('--pyrevit', help='path to pyrevit.exe')
|
||||
ap.add_argument('--revit', help='Revit year, e.g. 2025')
|
||||
ap.add_argument('--wait-pid', type=int, default=0,
|
||||
help='wait for this process to exit first')
|
||||
ap.add_argument('--wait-minutes', type=float, default=20.0)
|
||||
ap.add_argument('--settle', type=float, default=SETTLE_SECONDS,
|
||||
help='seconds to wait after the pid is gone')
|
||||
ap.add_argument('--machine-free', choices=('ignore', 'require'),
|
||||
default='ignore',
|
||||
help='require: also wait until no Revit at all is running '
|
||||
'(the old close-model behaviour). Default ignore -- '
|
||||
'two Revits are measured to be fine, three are not, '
|
||||
'and the caller sequences for that.')
|
||||
ap.add_argument('--dry-run', action='store_true',
|
||||
help='print what would happen and exit, launching nothing')
|
||||
ap.add_argument('--newest', metavar='FOLDER',
|
||||
help='print the newest snapshot in FOLDER and exit')
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
if args.newest:
|
||||
rows = S.snapshots(args.newest)
|
||||
if not rows:
|
||||
print('no snapshots in {0}'.format(args.newest))
|
||||
return 1
|
||||
for row in rows:
|
||||
mark = '*' if row is rows[-1] else ' '
|
||||
print('{0} {1} {2:8} {3}'.format(mark, row['stamp'],
|
||||
row['layout'], row['path']))
|
||||
print('\nnewest: {0}'.format(rows[-1]['path']))
|
||||
return 0
|
||||
|
||||
if not (args.request and args.runner):
|
||||
ap.error('--request and --runner are required')
|
||||
|
||||
# A missing or unreadable --request has nowhere of its own to be
|
||||
# journalled -- the request IS where the log path lives. But its own
|
||||
# location still says where the detached_dir is (see
|
||||
# `fallback_log_for`), so a crash here is not silent even so; only a
|
||||
# `--request` that does not even have a `_requests` parent gives up on
|
||||
# a journal line entirely, and that is loud on stderr instead.
|
||||
try:
|
||||
request = load_request(args.request)
|
||||
except Exception:
|
||||
tb = traceback.format_exc()
|
||||
print(tb, file=sys.stderr)
|
||||
fallback = fallback_log_for(args.request)
|
||||
if fallback:
|
||||
S.append(fallback, {
|
||||
'event': S.WATCHER_CRASHED, 'run': None,
|
||||
'error': 'could not read --request {0}: {1}'
|
||||
.format(args.request, tb.splitlines()[-1]),
|
||||
'traceback': tb})
|
||||
return 2
|
||||
|
||||
target = (request.get('plan') or {}).get('path')
|
||||
|
||||
# `--pyrevit` used to be required by `ap.error()` above, with no journal
|
||||
# possible at that point (the request was not loaded yet) -- a run
|
||||
# started with the flag simply missing died silently. It is optional now:
|
||||
# discovered the same way the two callers that already had this guess
|
||||
# discover it (`tools/snapshot_cycle.py`'s `_pyrevit_guess`, the
|
||||
# SnapshotClose button's `_pyrevit_cli`), so a caller that forgets it no
|
||||
# longer has to be the third copy of the same fallback.
|
||||
pyrevit_exe = args.pyrevit or S.pyrevit_exe_guess()
|
||||
if not pyrevit_exe:
|
||||
error = ('the pyRevit CLI was not found -- pass --pyrevit, or set '
|
||||
'snapshot.pyrevit_exe, or install pyRevit at the usual '
|
||||
'path ({0}\\pyRevit-Master\\bin\\pyrevit.exe)'
|
||||
.format(os.environ.get('APPDATA', '%APPDATA%')))
|
||||
journal(request, S.FAILED, path=target, error=error)
|
||||
print(error, file=sys.stderr)
|
||||
return 2
|
||||
|
||||
command = command_for(pyrevit_exe, args.runner, args.revit)
|
||||
|
||||
if args.dry_run:
|
||||
print('request {0}'.format(args.request))
|
||||
print('run id {0}'.format(request.get('run')))
|
||||
print('source {0}'.format(
|
||||
(request.get('source') or {}).get('path') or
|
||||
'cloud GUIDs in the request'))
|
||||
print('target {0}'.format(target))
|
||||
print('log {0}'.format(request.get('log')))
|
||||
print('wait for pid {0} (up to {1} min, then {2} s to settle),'
|
||||
.format(args.wait_pid, args.wait_minutes, args.settle))
|
||||
if args.machine_free == 'require':
|
||||
print(' then until NO Revit at all is running on this '
|
||||
'machine')
|
||||
else:
|
||||
print(' and NOT for other Revits (--machine-free '
|
||||
'ignore)')
|
||||
if not args.revit:
|
||||
print('\nWARNING: no --revit=<year> was given. On a machine '
|
||||
'with more than one Revit version attached, pyRevit has '
|
||||
'to pick one on its own -- confirm this is intended '
|
||||
'(see snapshot.revit_year / the Revit year folded into '
|
||||
'snapshot.revit_exe).')
|
||||
print('\nthen runs:\n {0}'.format(S.quote(command)))
|
||||
print('\nwith {0}={1}'.format('WILLTOOLS_SNAPSHOT_REQUEST',
|
||||
args.request))
|
||||
return 0
|
||||
|
||||
# From here on, this process is the whole reason the "requested and
|
||||
# never reported" question (`vibe.snapshotmeta.outstanding`) can be
|
||||
# answered instead of guessed at: a REQUESTED with no WATCHER_LAUNCHED
|
||||
# after it means this process never made it this far at all (never
|
||||
# spawned, or crashed on the request load above); one with
|
||||
# WATCHER_LAUNCHED but nothing terminal after it means it started and
|
||||
# then something below killed it hard enough to skip even the `except`
|
||||
# net a few lines down -- a signal, a forced kill, the machine itself
|
||||
# going away. Both are now distinguishable from "still running" and
|
||||
# from each other, which the ledger could not do before this line
|
||||
# existed.
|
||||
journal(request, S.WATCHER_LAUNCHED, path=target,
|
||||
wait_pid=args.wait_pid or None, wait_minutes=args.wait_minutes,
|
||||
machine_free=args.machine_free, pyrevit_exe=pyrevit_exe,
|
||||
revit_year=args.revit or None)
|
||||
|
||||
try:
|
||||
deadline = args.wait_minutes * 60.0
|
||||
if args.wait_pid:
|
||||
gone = wait_for_exit(args.wait_pid, deadline)
|
||||
if not gone:
|
||||
journal(request, S.FAILED, path=target,
|
||||
error='Revit (pid {0}) was still running after {1} '
|
||||
'minutes; no copy was made. A save prompt left '
|
||||
'unanswered is the usual reason.'
|
||||
.format(args.wait_pid, args.wait_minutes))
|
||||
return 2
|
||||
time.sleep(args.settle)
|
||||
|
||||
# THE MACHINE, not just that one process -- but only when asked for.
|
||||
# He may have opened Revit again while this was sleeping; under the
|
||||
# retired one-model premise that was fatal, and under
|
||||
# `--machine-free require` it still stands down. `wait_for_exit` is
|
||||
# reused with a machine-wide predicate; the pid is a placeholder it
|
||||
# never inspects.
|
||||
if args.machine_free == 'require' and \
|
||||
not wait_for_exit(1, deadline,
|
||||
alive=lambda _pid: any_revit_running()):
|
||||
journal(request, S.FAILED, path=target,
|
||||
error='a Revit is running again on this machine, so no '
|
||||
'copy was made -- --machine-free require was '
|
||||
'asked for. Press the button again at the end of '
|
||||
'the next session, or run with --machine-free '
|
||||
'ignore.')
|
||||
return 2
|
||||
|
||||
env = dict(os.environ)
|
||||
env['WILLTOOLS_SNAPSHOT_REQUEST'] = args.request
|
||||
try:
|
||||
result = subprocess.run(command, env=env, capture_output=True,
|
||||
text=True)
|
||||
code = result.returncode
|
||||
tail = (result.stderr or result.stdout or '').strip().splitlines()
|
||||
except Exception as exc:
|
||||
journal(request, S.FAILED, path=target,
|
||||
error='could not run the pyRevit CLI: {0}'.format(exc))
|
||||
return 2
|
||||
|
||||
# The runner journals its own DONE. This is the backstop for the
|
||||
# case where it could not -- Revit died, the journal never played,
|
||||
# the CLI refused. An outcome nobody wrote down is the failure this
|
||||
# design is least able to survive.
|
||||
if target and os.path.isfile(target):
|
||||
# The file is the only evidence that counts. `pyrevit run`
|
||||
# returns non-zero for journal-playback noise that has nothing
|
||||
# to do with whether the copy was written, so the exit code is
|
||||
# recorded when it is odd and is not allowed to overrule the
|
||||
# filesystem.
|
||||
if code != 0:
|
||||
journal(request, 'phase-c-note', path=target, exit_code=code,
|
||||
note='the copy was written; the pyRevit CLI still '
|
||||
'exited {0}'.format(code))
|
||||
return 0
|
||||
hint = ''
|
||||
if not args.revit:
|
||||
hint = (' No --revit=<year> was passed for this run -- on a '
|
||||
'machine with more than one Revit version attached, '
|
||||
'that is the first thing to try.')
|
||||
journal(request, S.FAILED, path=target, exit_code=code,
|
||||
error=('the pyRevit run finished but the copy is not there. '
|
||||
'Last line: {0}{1}'
|
||||
.format(tail[-1] if tail else '(nothing)', hint)))
|
||||
return 2
|
||||
except Exception:
|
||||
# The backstop for anything above that was not already one of the
|
||||
# named, journalled failures -- so "requested, watcher launched,
|
||||
# then nothing" can never again mean "go look at a console nobody
|
||||
# kept" (the button spawns this with no window and no output
|
||||
# redirection at all; an unjournalled traceback here is provably
|
||||
# unread by anything).
|
||||
tb = traceback.format_exc()
|
||||
journal(request, S.WATCHER_CRASHED, path=target,
|
||||
error=tb.splitlines()[-1], traceback=tb)
|
||||
print(tb, file=sys.stderr)
|
||||
return 2
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
+425
@@ -0,0 +1,425 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Check every WP assembly in the open model, and rank what is wrong.
|
||||
|
||||
python tools/sweep.py measure in Revit, then check
|
||||
python tools/sweep.py --check re-check the saved job files only
|
||||
python tools/sweep.py --out DIR somewhere other than beside the model
|
||||
|
||||
Two halves, deliberately separable:
|
||||
|
||||
**Measure** drives `tools/sweep_in_revit.py` through the bridge, ten pieces a
|
||||
call, until every assembly has been measured. It only reads the model. It is
|
||||
resumable -- stop it, run it again, it carries on.
|
||||
|
||||
**Check** runs `panellift` over the job files that came out. That half needs
|
||||
no Revit at all, because a job file is the complete input to the check. So a
|
||||
change to the mechanics can be re-checked against the whole model in a couple
|
||||
of minutes, and that is the point of keeping the job files rather than only the
|
||||
reports.
|
||||
|
||||
That promise had quietly stopped being true. The position search grew until it
|
||||
was 93% of a run and this sweep took **42 minutes**, which is long enough that
|
||||
nobody re-checks anything. It passes `--no-recommend` now -- it never read a
|
||||
recommendation in the first place -- and is back to about a minute of compute.
|
||||
|
||||
That separability has a trap in it, and this refuses on it now: a job file is
|
||||
the complete input to the check, so a sweep folder can be re-checked against a
|
||||
model it was never measured from and every report will still render, still
|
||||
reconcile, and still be about the wrong piece. Each job file records
|
||||
`source.model`; if that names a model other than the open document, nothing is
|
||||
checked and the message names both. `--any-model` proceeds anyway and says so
|
||||
at the top of the output; `--model TITLE` states the model when Revit is not
|
||||
running. With neither, the guard reports that it could not judge -- which is
|
||||
not the same answer as agreement, and is printed rather than assumed.
|
||||
|
||||
Writes `problems.txt` and `problems.json` next to the jobs.
|
||||
|
||||
What this does NOT do is choose the inserts. The stages come from
|
||||
`liftexport.default_stages`, which is what the grouping rules imply with
|
||||
nobody having clicked anything -- good enough to find what breaks across a
|
||||
whole model, not good enough to sign. See docs/handoff.md.
|
||||
"""
|
||||
from __future__ import print_function
|
||||
|
||||
import argparse
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
REPO = os.path.abspath(os.path.join(HERE, os.pardir))
|
||||
RVT = os.path.join(HERE, 'rvt.py')
|
||||
IN_REVIT = os.path.join(HERE, 'sweep_in_revit.py')
|
||||
PANELLIFT = os.path.join(REPO, 'panellift')
|
||||
SRC = os.path.join(PANELLIFT, 'src')
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- files
|
||||
|
||||
def read(path):
|
||||
with io.open(path, 'r', encoding='utf-8') as fh:
|
||||
return json.loads(fh.read())
|
||||
|
||||
|
||||
def write(path, obj):
|
||||
with io.open(path, 'w', encoding='utf-8') as fh:
|
||||
fh.write(json.dumps(obj, indent=1, sort_keys=True, default=str))
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- Revit
|
||||
|
||||
def bridge(*args):
|
||||
"""One rvt.py call. Returns (ok, parsed stdout or raw text)."""
|
||||
r = subprocess.run([sys.executable, RVT] + list(args),
|
||||
cwd=REPO, capture_output=True, text=True)
|
||||
text = (r.stdout or '').strip()
|
||||
try:
|
||||
return r.returncode == 0, json.loads(text)
|
||||
except ValueError:
|
||||
return False, text or (r.stderr or '').strip()
|
||||
|
||||
|
||||
def sweep_dir():
|
||||
"""Ask the open model where its sweep folder is."""
|
||||
ok, out = bridge('exec', '-c',
|
||||
"from vibe import designspec; "
|
||||
"result = designspec.sub_dir(doc, 'sweep')")
|
||||
if not ok or not isinstance(out, str):
|
||||
raise SystemExit(
|
||||
'Could not reach Revit: {0}\n\n'
|
||||
'Revit closed, or a modal dialog is open in it -- handlers run in '
|
||||
"Revit's API context and cannot fire behind a dialog.".format(out))
|
||||
return out
|
||||
|
||||
|
||||
def open_model_title():
|
||||
"""`doc.Title` of the open model, or None if Revit is not reachable.
|
||||
|
||||
None is "could not ask", never "it matches" -- see `check_source_model`.
|
||||
"""
|
||||
ok, out = bridge('exec', '-c', 'result = doc.Title')
|
||||
if not ok or not isinstance(out, str) or not out.strip():
|
||||
return None
|
||||
return out.strip()
|
||||
|
||||
|
||||
# ------------------------------------------------- which model is this OF?
|
||||
#
|
||||
# A job file is the complete input to the check, which is what makes the
|
||||
# offline half possible -- and it is also how a whole sweep can be re-checked
|
||||
# against a model it was never measured from without anything looking wrong.
|
||||
# Measured 2026-09-01: every one of the reference sweep's job files carries
|
||||
# `source.model` naming one model, while the document open in Revit was a
|
||||
# detached copy that had since DIVERGED from it -- one notch 3.5 in in one
|
||||
# and 2.5 in in the other. Every report still rendered, every number still
|
||||
# reconciled against the job file it came from, and the two models were only
|
||||
# told apart by hand. `source.model` is the field that would have caught it
|
||||
# immediately, and nothing was reading it.
|
||||
|
||||
def model_stem(name):
|
||||
"""'sample_job.rvt' and 'sample_job' are the same model.
|
||||
|
||||
`source.model` is written as `os.path.basename(doc.PathName)`, so it
|
||||
carries the extension; `doc.Title` does not. Compared case-insensitively
|
||||
because Windows does. Returns None for anything empty -- and None is a
|
||||
question, not a match.
|
||||
"""
|
||||
if not name:
|
||||
return None
|
||||
stem = os.path.basename(str(name).strip())
|
||||
if stem.lower().endswith('.rvt'):
|
||||
stem = stem[:-len('.rvt')]
|
||||
return stem.strip().lower() or None
|
||||
|
||||
|
||||
def check_source_model(jobs, doc_title):
|
||||
"""Which of these job files were exported from a different model.
|
||||
|
||||
`jobs` is [(mark, job dict)]. Returns (mismatched, unstamped, reason):
|
||||
|
||||
mismatched [(mark, the model the job file names)] -- exported from a
|
||||
DIFFERENT model from the one open. This is the refusal.
|
||||
unstamped [mark] for job files carrying no `source.model` at all.
|
||||
Written before the field existed, or by hand: they cannot
|
||||
be judged, and they are NOT thereby cleared.
|
||||
reason a sentence, when nothing could be judged at all because
|
||||
there is no open document to compare against. `mismatched`
|
||||
is then empty and that emptiness means "did not ask" --
|
||||
which is why the caller has to print the reason rather than
|
||||
read an empty list as agreement.
|
||||
"""
|
||||
want = model_stem(doc_title)
|
||||
if want is None:
|
||||
return [], [mark for mark, _job in jobs], (
|
||||
'no open document to compare against, so nothing here says '
|
||||
'whether these job files were measured from the model you mean')
|
||||
mismatched = []
|
||||
unstamped = []
|
||||
for mark, job in jobs:
|
||||
said = (job.get('source') or {}).get('model')
|
||||
stem = model_stem(said)
|
||||
if stem is None:
|
||||
unstamped.append(mark)
|
||||
elif stem != want:
|
||||
mismatched.append((mark, said))
|
||||
return mismatched, unstamped, None
|
||||
|
||||
|
||||
def model_refusal(mismatched, doc_title, total):
|
||||
"""The message the mismatch refuses with. Names BOTH models, always."""
|
||||
said = sorted(set(name for _mark, name in mismatched))
|
||||
return (
|
||||
'These job files were exported from a different model.\n\n'
|
||||
' the open document says {open}\n'
|
||||
' the job files say {said}\n\n'
|
||||
'{n} of {total} job file(s) name a model that is not the one open, '
|
||||
'and the first of them is {first}.\n\n'
|
||||
'Nothing was checked. A job file is the complete input to the check, '
|
||||
'so this would have run to completion and produced a full set of '
|
||||
'plausible reports about a model nobody is looking at -- which is '
|
||||
'exactly what happened before this refusal existed, on two copies of '
|
||||
'one model that had since diverged.\n\n'
|
||||
'Re-export the sweep from the open model, point --out at the sweep '
|
||||
'folder that belongs to it, or pass --any-model if you really do mean '
|
||||
'to check these files against this document.'.format(
|
||||
open=doc_title, said=', '.join(said),
|
||||
n=len(mismatched), total=total,
|
||||
first=sorted(mark for mark, _name in mismatched)[0]))
|
||||
|
||||
|
||||
def measure():
|
||||
"""Drive the in-Revit half until every assembly has a meta file."""
|
||||
root = sweep_dir()
|
||||
print('sweep folder: %s' % root)
|
||||
meta_dir = os.path.join(root, 'meta')
|
||||
todo_path = os.path.join(root, 'todo.json')
|
||||
|
||||
stuck = 0
|
||||
while True:
|
||||
total = len(read(todo_path)) if os.path.isfile(todo_path) else 0
|
||||
done = len(os.listdir(meta_dir)) if os.path.isdir(meta_dir) else 0
|
||||
if total and done >= total:
|
||||
print('measured %d/%d' % (done, total))
|
||||
return root
|
||||
|
||||
ok, out = bridge('exec', IN_REVIT)
|
||||
print('[%d/%d] %s' % (done, total, str(out)[:200]))
|
||||
sys.stdout.flush()
|
||||
if not ok or 'did 0 ' in str(out):
|
||||
stuck += 1
|
||||
if stuck >= 3:
|
||||
print('STOPPING: three calls in a row got nowhere. Revit is '
|
||||
'busy, or a dialog is open in it.')
|
||||
return root
|
||||
time.sleep(10)
|
||||
else:
|
||||
stuck = 0
|
||||
|
||||
|
||||
# --------------------------------------------------------------- the check
|
||||
|
||||
def check(root, doc_title=None, any_model=False):
|
||||
jobs_dir = os.path.join(root, 'jobs')
|
||||
meta_dir = os.path.join(root, 'meta')
|
||||
reports = os.path.join(root, 'reports')
|
||||
if not os.path.isdir(reports):
|
||||
os.makedirs(reports)
|
||||
|
||||
env = dict(os.environ, PYTHONPATH=SRC)
|
||||
rows = []
|
||||
names = sorted(n for n in os.listdir(jobs_dir) if n.endswith('.job.json'))
|
||||
|
||||
# WHICH MODEL, before a single number is computed. See check_source_model.
|
||||
jobs = [(n[:-len('.job.json')], read(os.path.join(jobs_dir, n)))
|
||||
for n in names]
|
||||
mismatched, unstamped, reason = check_source_model(jobs, doc_title)
|
||||
notes = []
|
||||
if mismatched and not any_model:
|
||||
raise SystemExit(model_refusal(mismatched, doc_title, len(jobs)))
|
||||
if mismatched:
|
||||
notes.append(
|
||||
'!! --any-model: %d of %d job file(s) name a model that is not '
|
||||
'%s. These reports are about a different model.'
|
||||
% (len(mismatched), len(jobs), doc_title))
|
||||
if reason:
|
||||
notes.append('!! model NOT verified: %s.' % reason)
|
||||
if unstamped and not reason:
|
||||
notes.append(
|
||||
'!! %d of %d job file(s) carry no source.model, so which model '
|
||||
'they came from was not checked.' % (len(unstamped), len(jobs)))
|
||||
for note in notes:
|
||||
print(note)
|
||||
|
||||
for i, name in enumerate(names, 1):
|
||||
mark = name[:-len('.job.json')]
|
||||
meta = read(os.path.join(meta_dir, mark + '.json'))
|
||||
row = dict((k, meta.get(k)) for k in
|
||||
('mark', 'design', 'axis', 'length_ft', 'width_ft',
|
||||
'thickness_in', 'lifters', 'strands'))
|
||||
# --no-recommend, because this sweep never reads a recommendation:
|
||||
# `row` below takes the verdict, the per-stage ratios and the findings
|
||||
# and nothing else. The position search is 93% of a run -- 41 minutes
|
||||
# of this sweep's 42 -- and every second of it was being computed,
|
||||
# written to an HTML file and thrown away. The verdicts, ratios and
|
||||
# findings are untouched by the flag; the reports say on the page that
|
||||
# the positions were not computed, so nobody reads one as a piece with
|
||||
# nothing to recommend. Re-run a single piece through the button, or
|
||||
# `python -m panellift <job> -o <html>` by hand, to get positions.
|
||||
r = subprocess.run(
|
||||
[sys.executable, '-m', 'panellift', os.path.join(jobs_dir, name),
|
||||
'-o', os.path.join(reports, mark + '.html'), '--json',
|
||||
'--no-recommend'],
|
||||
cwd=PANELLIFT, env=env, capture_output=True, text=True)
|
||||
text = (r.stdout or '').strip()
|
||||
# A FAIL verdict exits non-zero, so the exit code says nothing about
|
||||
# whether the check ran. The JSON does.
|
||||
if not text.startswith('{'):
|
||||
row['verdict'] = 'CHECK CRASHED'
|
||||
row['error'] = (r.stderr or r.stdout or '')[-1200:]
|
||||
else:
|
||||
res = json.loads(text)
|
||||
row['verdict'] = res.get('verdict')
|
||||
row['stages'] = [
|
||||
{'name': s.get('name'), 'ratio': s.get('ratio'),
|
||||
'verdict': s.get('verdict'),
|
||||
'findings': [(f.get('severity'), f.get('code'),
|
||||
f.get('message'))
|
||||
for f in s.get('findings') or []]}
|
||||
for s in res.get('stages') or []]
|
||||
if i % 25 == 0 or i == len(names):
|
||||
print('checked %d/%d' % (i, len(names)))
|
||||
sys.stdout.flush()
|
||||
rows.append(row)
|
||||
return rows, notes
|
||||
|
||||
|
||||
def refused(root):
|
||||
"""The pieces that never produced a job file, and why."""
|
||||
meta_dir = os.path.join(root, 'meta')
|
||||
out = []
|
||||
for name in sorted(os.listdir(meta_dir)):
|
||||
meta = read(os.path.join(meta_dir, name))
|
||||
if meta.get('status') != 'exported':
|
||||
out.append(meta)
|
||||
return out
|
||||
|
||||
|
||||
# -------------------------------------------------------------- the report
|
||||
|
||||
def worst(row):
|
||||
return max([s.get('ratio') or 0.0
|
||||
for s in row.get('stages') or []] or [0.0])
|
||||
|
||||
|
||||
def render(rows, not_exported, notes=()):
|
||||
lines = []
|
||||
|
||||
def say(s=''):
|
||||
lines.append(s)
|
||||
|
||||
# At the top, and inside problems.txt as well as on the terminal: a
|
||||
# caveat that only ever scrolled past is a caveat nobody read.
|
||||
for note in notes or ():
|
||||
say(note)
|
||||
if notes:
|
||||
say()
|
||||
|
||||
say('%d pieces checked, %d never got that far'
|
||||
% (len(rows), len(not_exported)))
|
||||
counts = {}
|
||||
for r in rows:
|
||||
counts[r.get('verdict')] = counts.get(r.get('verdict'), 0) + 1
|
||||
for k in sorted(counts, key=lambda k: -counts[k]):
|
||||
say(' %-14s %d' % (k, counts[k]))
|
||||
say()
|
||||
|
||||
checked = [r for r in rows if r.get('stages')]
|
||||
over = sorted([r for r in checked if worst(r) > 0.8], key=worst,
|
||||
reverse=True)
|
||||
say('== OVERSTRESSED: a stage above 0.80 of its allowable tension ==')
|
||||
if not over:
|
||||
say(' none')
|
||||
for r in over:
|
||||
say(' %-8s %6.1f x %5.1f ft %s'
|
||||
% (r['mark'], r.get('length_ft') or 0, r.get('width_ft') or 0,
|
||||
' '.join('%s %.2f' % (s['name'][:4], s.get('ratio') or 0)
|
||||
for s in r['stages'])))
|
||||
say()
|
||||
|
||||
for severity in ('FAIL', 'WARN'):
|
||||
buckets = {}
|
||||
for r in checked:
|
||||
for stage in r['stages']:
|
||||
for sev, code, message in stage['findings']:
|
||||
if sev == severity:
|
||||
buckets.setdefault(code, {})[r['mark']] = message
|
||||
say('== %s findings, by code ==' % severity)
|
||||
if not buckets:
|
||||
say(' none')
|
||||
for code in sorted(buckets, key=lambda c: -len(buckets[c])):
|
||||
marks = sorted(buckets[code])
|
||||
say(' %-8s %3d %s' % (code, len(marks), ', '.join(marks[:10])))
|
||||
say(' %s' % list(buckets[code].values())[0][:150])
|
||||
say()
|
||||
|
||||
crashed = [r for r in rows if r.get('verdict') == 'CHECK CRASHED']
|
||||
if crashed:
|
||||
say('== CHECK CRASHED: %d ==' % len(crashed))
|
||||
for r in crashed:
|
||||
say(' %-8s lifters=%s %s'
|
||||
% (r['mark'], r.get('lifters'),
|
||||
str(r.get('error'))[-200:].replace(chr(10), ' ')))
|
||||
say()
|
||||
|
||||
if not_exported:
|
||||
reasons = {}
|
||||
for meta in not_exported:
|
||||
why = '; '.join(meta.get('problems') or []) or meta.get('status')
|
||||
reasons.setdefault(str(why)[:200], []).append(meta['mark'])
|
||||
say('== NEVER CHECKED: %d, by reason ==' % len(not_exported))
|
||||
for why in sorted(reasons, key=lambda w: -len(reasons[w])):
|
||||
say(' %d x %s' % (len(reasons[why]), why))
|
||||
say(' %s' % ', '.join(sorted(reasons[why])[:8]))
|
||||
return lines
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument('--check', action='store_true',
|
||||
help='skip the Revit half and re-check saved job files')
|
||||
ap.add_argument('--out', help='sweep folder (default: beside the model)')
|
||||
ap.add_argument('--model', help='the model title the job files should '
|
||||
'name; default is whatever Revit has open')
|
||||
ap.add_argument('--any-model', action='store_true',
|
||||
help='check these job files even though they were '
|
||||
'exported from a different model')
|
||||
args = ap.parse_args()
|
||||
|
||||
if args.out:
|
||||
root = args.out
|
||||
elif args.check:
|
||||
root = sweep_dir()
|
||||
else:
|
||||
root = measure()
|
||||
|
||||
# --model lets the guard work with no Revit at all (an offline re-check of
|
||||
# a sweep folder whose model is not open); with neither, the title is
|
||||
# None and the guard says it could not judge instead of saying it agrees.
|
||||
doc_title = args.model or open_model_title()
|
||||
|
||||
rows, notes = check(root, doc_title, any_model=args.any_model)
|
||||
write(os.path.join(root, 'problems.json'), rows)
|
||||
text = chr(10).join(render(rows, refused(root), notes))
|
||||
with io.open(os.path.join(root, 'problems.txt'), 'w',
|
||||
encoding='utf-8') as fh:
|
||||
fh.write(text + chr(10))
|
||||
print()
|
||||
print(text)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,249 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Measure every in-scope assembly in the open model. Runs INSIDE Revit.
|
||||
|
||||
Not run directly -- `tools/sweep.py` feeds it to the bridge over and over:
|
||||
|
||||
python tools/rvt.py exec tools/sweep_in_revit.py
|
||||
|
||||
Read only. No transaction, no view change, no selection, nothing that would
|
||||
show up in the user's undo stack.
|
||||
|
||||
WHICH ASSEMBLIES, AND WHO SAYS SO
|
||||
---------------------------------
|
||||
The DECLARED SCOPE (`vibe.scope`), read from the job knowledge layer, exactly
|
||||
as Job Recon, Takeoff and Ticket QC read it. Until 2026-08-31 this file
|
||||
filtered on a two-letter mark prefix written into the code, which was wrong
|
||||
twice over: the declaration in the job file was not load-bearing here (delete
|
||||
it and this still filtered; change it and this did not follow), and one job's
|
||||
naming was baked into a repository that must not carry one.
|
||||
|
||||
`run()` already loads the knowledge dict for the measurement itself, so the
|
||||
scope costs nothing to read -- there was never a reason for the literal.
|
||||
|
||||
WHAT IT SAYS ABOUT WHAT IT LEFT ALONE
|
||||
--------------------------------------
|
||||
Out of scope is a third state, not an absence, so this writes the scope line
|
||||
`scope.json` next to the other outputs AND returns it from the enumerating
|
||||
call, which is the call `tools/sweep.py` prints. A sweep that silently
|
||||
measured a subset would look exactly like a sweep that measured everything.
|
||||
|
||||
todo.json every in-scope assembly, its element id and its design
|
||||
scope.json the declared scope, the line, and what was left alone
|
||||
jobs/<mark>.job.json panellift input, default stages, nothing picked
|
||||
meta/<mark>.json what happened: status, axis, weight gate, audit
|
||||
|
||||
One call does CHUNK pieces and returns how many are left. A piece with a meta
|
||||
file is done, so this is resumable and safe to re-run: delete the sweep folder
|
||||
to start over -- including after the declaration changes, because `todo.json`
|
||||
is enumerated once and does not re-read the scope.
|
||||
"""
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import traceback
|
||||
|
||||
from Autodesk.Revit.DB import ElementId
|
||||
|
||||
from vibe import designspec, knowledge, liftexport, ticketqc
|
||||
from vibe import scope as scopelib
|
||||
from vibe.compat import text_type
|
||||
|
||||
CHUNK = 10
|
||||
|
||||
|
||||
def _dump(path, obj):
|
||||
text = json.dumps(obj, indent=1, sort_keys=True, default=repr)
|
||||
if not isinstance(text, text_type):
|
||||
text = text.decode('utf-8')
|
||||
with io.open(path, 'w', encoding='utf-8') as fh:
|
||||
fh.write(text)
|
||||
|
||||
|
||||
def _load(path):
|
||||
with io.open(path, 'r', encoding='utf-8') as fh:
|
||||
return json.loads(fh.read())
|
||||
|
||||
|
||||
def _folders(doc):
|
||||
root = designspec.sub_dir(doc, 'sweep')
|
||||
return root, designspec.sub_dir(doc, os.path.join('sweep', 'jobs')), \
|
||||
designspec.sub_dir(doc, os.path.join('sweep', 'meta'))
|
||||
|
||||
|
||||
def enumerate_pieces(doc, know, todo_path, scope_path=None):
|
||||
"""Every in-scope assembly with the design it belongs to, written once.
|
||||
|
||||
Returns (todo, scope_info). What the DECLARED scope left alone is
|
||||
counted and named in `scope_info` and written to `scope.json` -- it is
|
||||
never simply absent, which is the one thing `vibe.scope` exists to stop.
|
||||
"""
|
||||
the_scope = scopelib.of(know)
|
||||
todo, left_alone, unreadable, total = [], [], [], 0
|
||||
for design, group in ticketqc.assemblies_by_design(doc, know).items():
|
||||
for asm in group:
|
||||
name = str(asm.Name)
|
||||
total += 1
|
||||
if the_scope.is_unreadable(name):
|
||||
# No readable prefix: 'I could not tell' never removes work.
|
||||
unreadable.append(name)
|
||||
if not the_scope.includes(name):
|
||||
left_alone.append(name)
|
||||
continue
|
||||
todo.append({'id': ticketqc.eid(asm), 'mark': name,
|
||||
'design': design})
|
||||
todo.sort(key=lambda r: (r['design'], r['mark']))
|
||||
_dump(todo_path, todo)
|
||||
|
||||
scope_info = {
|
||||
'declared': the_scope.is_declared,
|
||||
'label': the_scope.label(),
|
||||
'declaration': the_scope.declaration_lines(),
|
||||
'line': the_scope.line(left_alone, total=total, unit='assembly',
|
||||
unreadable_names=unreadable),
|
||||
# The headline is written HERE, at enumeration, and read back
|
||||
# verbatim on every later call. See `_scope_headline`.
|
||||
'headline': the_scope.short_line(left_alone, total=total,
|
||||
unit='assembly'),
|
||||
'near_miss': the_scope.near_miss_line(left_alone, unit='assembly'),
|
||||
'left_alone': the_scope.tally(left_alone),
|
||||
'left_alone_count': len(left_alone),
|
||||
'measured_count': len(todo),
|
||||
'assemblies_seen': total,
|
||||
}
|
||||
if scope_path:
|
||||
_dump(scope_path, scope_info)
|
||||
return todo, scope_info
|
||||
|
||||
|
||||
def measure(doc, know, sheets, item, jobs_dir):
|
||||
"""One assembly, as far as it gets. Never raises."""
|
||||
meta = {'mark': item['mark'], 'design': item['design'],
|
||||
'status': 'unknown'}
|
||||
try:
|
||||
asm = doc.GetElement(ElementId(item['id']))
|
||||
# Unattended: take the time and locate the inserts properly.
|
||||
job, problems = liftexport.export(doc, asm, know, sheets,
|
||||
item['design'], batch=True)
|
||||
meta['problems'] = problems
|
||||
if job is None:
|
||||
meta['status'] = 'refused'
|
||||
return meta
|
||||
|
||||
src = job.get('source') or {}
|
||||
panel = job.get('panel') or {}
|
||||
meta['axis'] = src.get('erected_vertical_axis')
|
||||
meta['length_ft'] = panel.get('length_ft')
|
||||
meta['width_ft'] = panel.get('width_ft')
|
||||
meta['thickness_in'] = panel.get('thickness_in')
|
||||
meta['lifters'] = len(panel.get('lifters') or [])
|
||||
meta['strands'] = len(
|
||||
((panel.get('prestress') or {}).get('strands')) or [])
|
||||
meta['fc_psi'] = panel.get('fc_psi')
|
||||
meta['fci_psi'] = panel.get('fci_psi')
|
||||
|
||||
ok, message, _numbers = liftexport.weight_check(job, know)
|
||||
meta['weight_ok'] = bool(ok)
|
||||
if not ok:
|
||||
meta['weight_msg'] = message
|
||||
|
||||
stages, _rows, stage_problems = liftexport.default_stages(job, know)
|
||||
if not stages:
|
||||
meta['status'] = 'no stages'
|
||||
return meta
|
||||
if stage_problems:
|
||||
meta['status'] = 'dunnage misconfigured'
|
||||
meta['stage_problems'] = stage_problems
|
||||
return meta
|
||||
|
||||
# A stage carried on dunnage has no 'lifters' key at all -- it is
|
||||
# dropped rather than emptied, because panellift refuses a stage that
|
||||
# gives both a support line and an insert.
|
||||
meta['stage_lifters'] = dict(
|
||||
(s['name'], len(s.get('lifters') or [])) for s in stages)
|
||||
meta['stage_span'] = dict(
|
||||
(s['name'], s.get('span')) for s in stages)
|
||||
meta['stage_supports'] = dict(
|
||||
(s['name'], s.get('supports')) for s in stages
|
||||
if s.get('supports'))
|
||||
try:
|
||||
meta['audit'] = repr(liftexport.stage_audit(stages))
|
||||
except Exception as exc:
|
||||
meta['audit'] = 'audit failed: {0}'.format(exc)
|
||||
|
||||
# The stages here are the DEFAULTS -- what the grouping rules imply,
|
||||
# with nobody having clicked anything. That is the point of a sweep
|
||||
# and it is also its limit: see docs/handoff.md.
|
||||
missing = [s['name'] for s in stages if not s.get('fc_psi')]
|
||||
if missing:
|
||||
meta['status'] = 'no strength'
|
||||
meta['missing_fc'] = missing
|
||||
return meta
|
||||
|
||||
job['stages'] = stages
|
||||
_dump(os.path.join(jobs_dir, item['mark'] + '.job.json'),
|
||||
liftexport.strip_private(job))
|
||||
meta['status'] = 'exported'
|
||||
except Exception:
|
||||
meta['status'] = 'crash'
|
||||
meta['error'] = traceback.format_exc()
|
||||
return meta
|
||||
|
||||
|
||||
def _scope_headline(info):
|
||||
"""The recorded headline, for the FRONT of every return value.
|
||||
|
||||
Short and first on purpose. `tools/sweep.py` prints `str(out)[:200]` --
|
||||
it truncates -- so the full sentence lives in `scope.json` and what a
|
||||
reader sees at the console is the count, which is what tells him there is
|
||||
a file to open. A scope line cut off the end of a progress line would be
|
||||
a scope line nobody read.
|
||||
|
||||
A sweep folder enumerated before this existed has no `scope.json`, and
|
||||
that is reported rather than filled in: what such a run filtered on is
|
||||
genuinely unknown to this code.
|
||||
"""
|
||||
if not info:
|
||||
return ('SCOPE: NOT RECORDED -- this sweep folder was enumerated '
|
||||
'before the scope was read, so what it left out is unknown. '
|
||||
'Delete the sweep folder and re-run.')
|
||||
return (info.get('headline') or 'SCOPE: recorded without a headline') \
|
||||
+ ' (scope.json)'
|
||||
|
||||
|
||||
def run(doc):
|
||||
know = knowledge.load(doc)
|
||||
root, jobs_dir, meta_dir = _folders(doc)
|
||||
todo_path = os.path.join(root, 'todo.json')
|
||||
scope_path = os.path.join(root, 'scope.json')
|
||||
|
||||
if not os.path.isfile(todo_path):
|
||||
todo, scope_info = enumerate_pieces(doc, know, todo_path, scope_path)
|
||||
return '{0} | enumerated {1} assemblies into {2}'.format(
|
||||
_scope_headline(scope_info), len(todo), root)
|
||||
|
||||
# Re-read rather than re-derive: the enumeration is what the scope was
|
||||
# applied to, and that happened on an earlier call. Saying it again from
|
||||
# a knowledge file that may have been edited since would describe a
|
||||
# filter this run did not use.
|
||||
scope_info = _load(scope_path) if os.path.isfile(scope_path) else None
|
||||
|
||||
todo = _load(todo_path)
|
||||
_summaries, sheets = ticketqc.collect_context(doc)
|
||||
|
||||
did, remaining = [], 0
|
||||
for item in todo:
|
||||
meta_path = os.path.join(meta_dir, item['mark'] + '.json')
|
||||
if os.path.isfile(meta_path):
|
||||
continue
|
||||
if len(did) >= CHUNK:
|
||||
remaining += 1
|
||||
continue
|
||||
meta = measure(doc, know, sheets, item, jobs_dir)
|
||||
_dump(meta_path, meta)
|
||||
did.append('{0}:{1}'.format(item['mark'], meta['status']))
|
||||
|
||||
return '{0} | did {1} [{2}] remaining {3}'.format(
|
||||
_scope_headline(scope_info), len(did), ', '.join(did), remaining)
|
||||
|
||||
|
||||
result = run(doc) # noqa: F821 -- doc is injected by tools/rvt.py
|
||||
@@ -0,0 +1,226 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Revision delta for precast shop tickets -- CLI.
|
||||
|
||||
python tools/ticket_delta.py OLD NEW [--crops DIR] [--json OUT.json]
|
||||
|
||||
OLD and NEW may each be a single PDF or a folder of them. Folders are paired
|
||||
by mark, because the two issues of a real set were named differently (old by
|
||||
mark, new by job-control-mark) and the mark is the only identifier both carry.
|
||||
|
||||
Needs pymupdf on the path. It is not a dependency of the extension -- the
|
||||
Revit side runs IronPython and cannot import it -- so this script is the only
|
||||
thing in the repo that touches a PDF.
|
||||
"""
|
||||
|
||||
from __future__ import print_function
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
import types
|
||||
|
||||
_HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
_LIB = os.path.join(_HERE, '..', 'WillTools.extension', 'lib')
|
||||
sys.path.insert(0, _LIB)
|
||||
|
||||
# Same trick the test conftest uses, and for the same reason: vibe/__init__.py
|
||||
# imports the Revit API at module scope, which is right inside Revit and fatal
|
||||
# on host CPython. Register a bare `vibe` package pointed at the directory so
|
||||
# the Revit-free submodules import normally and __init__.py never runs.
|
||||
if 'vibe' not in sys.modules:
|
||||
_stub = types.ModuleType('vibe')
|
||||
_stub.__path__ = [os.path.join(_LIB, 'vibe')]
|
||||
sys.modules['vibe'] = _stub
|
||||
|
||||
from vibe import ticketdelta # noqa: E402
|
||||
|
||||
|
||||
def _load_extract():
|
||||
try:
|
||||
from vibe import ticketextract
|
||||
import pymupdf # noqa: F401
|
||||
except ImportError as exc:
|
||||
raise SystemExit(
|
||||
'ticket_delta needs pymupdf on PYTHONPATH (import name pymupdf, '
|
||||
'not fitz). %s' % exc)
|
||||
return ticketextract
|
||||
|
||||
|
||||
SEV_MARK = {
|
||||
ticketdelta.MATERIAL: '!!',
|
||||
ticketdelta.UNCERTAIN: '??',
|
||||
ticketdelta.ADVISORY: ' *',
|
||||
ticketdelta.HOUSEKEEP: ' ',
|
||||
}
|
||||
|
||||
|
||||
def _pdfs(path):
|
||||
if os.path.isdir(path):
|
||||
return sorted(f for f in os.listdir(path) if f.lower().endswith('.pdf'))
|
||||
return [os.path.basename(path)]
|
||||
|
||||
|
||||
def _dir(path):
|
||||
return path if os.path.isdir(path) else os.path.dirname(os.path.abspath(path))
|
||||
|
||||
|
||||
def render_crop(pdf, page, x, y, out_png, dpi=300, pad=90):
|
||||
"""Render the neighbourhood of a finding so a human can read the number.
|
||||
|
||||
This exists because the text layer is not the sheet. Four independent
|
||||
text-layer failure modes have been measured on this project's sources, and
|
||||
each one produces a confident wrong number rather than an error. Every
|
||||
reported change ships with the picture it was read from; a reviewer who
|
||||
disagrees with the tool can see immediately which of them is wrong.
|
||||
"""
|
||||
import pymupdf
|
||||
doc = pymupdf.open(pdf)
|
||||
pg = doc[page]
|
||||
clip = pymupdf.Rect(max(0, x - pad), max(0, y - pad),
|
||||
min(pg.rect.x1, x + pad), min(pg.rect.y1, y + pad))
|
||||
pix = pg.get_pixmap(dpi=dpi, clip=clip)
|
||||
pix.save(out_png)
|
||||
doc.close()
|
||||
return out_png
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument('old')
|
||||
ap.add_argument('new')
|
||||
ap.add_argument('--crops', help='directory to render a PNG per material '
|
||||
'finding into')
|
||||
ap.add_argument('--json', dest='json_out')
|
||||
ap.add_argument('--all', action='store_true',
|
||||
help='print housekeeping findings too')
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
tx = _load_extract()
|
||||
|
||||
old_dir, new_dir = _dir(args.old), _dir(args.new)
|
||||
plan = ticketdelta.pair_sets(_pdfs(args.old), _pdfs(args.new))
|
||||
|
||||
print('=' * 78)
|
||||
print('REVISION DELTA')
|
||||
print(' old issue: %s (%d file(s))' % (args.old, plan['naming']['old_count']))
|
||||
print(' new issue: %s (%d file(s))' % (args.new, plan['naming']['new_count']))
|
||||
if plan['naming']['changed']:
|
||||
print('')
|
||||
print(' !! THE FILE NAMING CHANGED BETWEEN ISSUES.')
|
||||
print(' old files carry a control number on %d of %d;'
|
||||
' new on %d of %d.'
|
||||
% (plan['naming']['old_has_control'], plan['naming']['old_count'],
|
||||
plan['naming']['new_has_control'], plan['naming']['new_count']))
|
||||
print(' These two issues therefore CANNOT be paired by filename.')
|
||||
print(' Pairing below is on the mark, the only identifier both'
|
||||
' namings carry.')
|
||||
if plan['naming']['fan_out']:
|
||||
print('')
|
||||
print(' !! ONE OLD FILE MAPS TO SEVERAL NEW ONES:')
|
||||
for m, n in sorted(plan['naming']['fan_out'].items()):
|
||||
print(' %s: 1 old file -> %d new files' % (m, n))
|
||||
print(' One ticket per control number: several identical panels'
|
||||
' now get')
|
||||
print(' several tickets. The comparison below runs against each'
|
||||
' of them.')
|
||||
for f in plan['old_only']:
|
||||
print(' !! WITHDRAWN: %s is in the old issue and has no counterpart'
|
||||
' in the new' % f)
|
||||
for f in plan['new_only']:
|
||||
print(' !! ADDED: %s is in the new issue and has no counterpart'
|
||||
' in the old' % f)
|
||||
print('=' * 78)
|
||||
|
||||
cache = {}
|
||||
|
||||
def get(path):
|
||||
if path not in cache:
|
||||
cache[path] = tx.extract(path)
|
||||
return cache[path]
|
||||
|
||||
results = []
|
||||
for orec, nrecs in plan['pairs']:
|
||||
a = get(os.path.join(old_dir, orec['file']))
|
||||
for nrec in nrecs:
|
||||
b = get(os.path.join(new_dir, nrec['file']))
|
||||
fs = ticketdelta.compare(a, b, tx.parse_dim, tx.format_inches,
|
||||
tx.locate)
|
||||
results.append({'old': orec['file'], 'new': nrec['file'],
|
||||
'mark': orec['mark'], 'control': nrec['control'],
|
||||
'findings': fs})
|
||||
|
||||
for r in results:
|
||||
counts = ticketdelta.summarise(r['findings'])
|
||||
print('')
|
||||
print('-' * 78)
|
||||
print('%s %s -> %s' % (r['mark'], r['old'], r['new']))
|
||||
print(' %d material, %d uncertain, %d advisory, %d housekeeping'
|
||||
% (counts[ticketdelta.MATERIAL], counts[ticketdelta.UNCERTAIN],
|
||||
counts[ticketdelta.ADVISORY], counts[ticketdelta.HOUSEKEEP]))
|
||||
shown = 0
|
||||
for i, f in enumerate(r['findings']):
|
||||
if f['severity'] == ticketdelta.HOUSEKEEP and not args.all:
|
||||
continue
|
||||
shown += 1
|
||||
print(' %s [%s] %s' % (SEV_MARK[f['severity']], f['kind'], f['where']))
|
||||
print(' %s' % f['what'])
|
||||
if f['old'] is not None or f['new'] is not None:
|
||||
print(' %s -> %s'
|
||||
% (_short(f['old']), _short(f['new'])))
|
||||
if f['note']:
|
||||
print(' (%s)' % f['note'])
|
||||
if not shown:
|
||||
print(' nothing above housekeeping.')
|
||||
|
||||
if args.crops:
|
||||
if not os.path.isdir(args.crops):
|
||||
os.makedirs(args.crops)
|
||||
n = 0
|
||||
for r in results:
|
||||
for i, f in enumerate(r['findings']):
|
||||
if f['severity'] != ticketdelta.MATERIAL:
|
||||
continue
|
||||
for side, key, pdfdir, fname in (
|
||||
('old', 'loc_old', old_dir, r['old']),
|
||||
('new', 'loc_new', new_dir, r['new'])):
|
||||
loc = f.get(key)
|
||||
if not loc:
|
||||
continue
|
||||
stem = os.path.splitext(fname)[0]
|
||||
out = os.path.join(args.crops, '%s_%02d_%s.png'
|
||||
% (stem, i, side))
|
||||
try:
|
||||
render_crop(os.path.join(pdfdir, fname), loc['page'],
|
||||
loc['x'], loc['y'], out)
|
||||
f.setdefault('crops', {})[side] = out
|
||||
n += 1
|
||||
except Exception as exc:
|
||||
print(' crop failed: %s' % exc)
|
||||
print('\n%d crop(s) written to %s' % (n, args.crops))
|
||||
|
||||
if args.json_out:
|
||||
with open(args.json_out, 'w') as fh:
|
||||
json.dump({'plan': {k: v for k, v in plan.items() if k != 'pairs'},
|
||||
'results': results}, fh, indent=1, default=str)
|
||||
print('\njson written to %s' % args.json_out)
|
||||
|
||||
total_material = sum(ticketdelta.summarise(r['findings'])[ticketdelta.MATERIAL]
|
||||
for r in results)
|
||||
print('')
|
||||
print('=' * 78)
|
||||
print('%d pair(s) compared, %d material change(s) in total.'
|
||||
% (len(results), total_material))
|
||||
return 0
|
||||
|
||||
|
||||
def _short(v, n=110):
|
||||
if v is None:
|
||||
return '(none)'
|
||||
v = str(v)
|
||||
return v if len(v) <= n else v[:n - 3] + '...'
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,287 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Ask a small vision model one question about one rendered ticket view.
|
||||
|
||||
python tools/vision_read.py <request.json> <response.json>
|
||||
|
||||
This is the CPython half of the Ticket QC vision escalation. It exists as a
|
||||
separate process for the same two reasons the other out-of-process tools do:
|
||||
|
||||
* the extension runs on IronPython 2.7 inside Revit's UI thread, and there
|
||||
is no HTTP client there worth trusting with a model call;
|
||||
* nothing here touches the Revit API. It reads PNGs and JSON and writes
|
||||
JSON, so it has no business being inside Revit.
|
||||
|
||||
The two halves talk through files, never imports. `vibe.ticketvision` builds
|
||||
the request, this writes the response, and neither imports the other.
|
||||
|
||||
WHAT THIS MAY AND MAY NOT DO
|
||||
----------------------------
|
||||
It may return a verdict from a model that looked at a picture. It may not
|
||||
return anything that renders like a computed result. Every row it writes
|
||||
carries `source: "model-read"`, and a row it could not obtain is
|
||||
`status: "NOT_RUN"` with a reason -- never a pass, never silence. Reaching
|
||||
the model is not the same as the model being right, and neither is the same
|
||||
as this program having run at all; the three are kept separate on purpose.
|
||||
|
||||
ENDPOINT
|
||||
--------
|
||||
The model server lives on the office LAN (default `http://192.168.99.151`,
|
||||
override with `VISION_ENDPOINT` or an `endpoint` key in the request). The
|
||||
endpoint speaks the OpenAI chat/completions wire format; the model name
|
||||
(`will-tool-ai` by default) is whatever that server advertises. There is no
|
||||
key in this file, no key in the request JSON, no key in the response JSON,
|
||||
and no key in anything printed. Nothing here ever reads a key into a
|
||||
variable, so nothing here can leak one. An unreachable server is a NOT_RUN
|
||||
with that reason.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
# The LAN vision server this tool talks to. Nothing here calls a cloud
|
||||
# provider: the endpoint is a local model server on the office network, and
|
||||
# the model name is the one that server advertises. Override either with the
|
||||
# environment when the server moves.
|
||||
DEFAULT_MODEL = "will-tool-ai"
|
||||
DEFAULT_ENDPOINT = "http://192.168.99.151/v1/chat/completions"
|
||||
|
||||
# The three answers the skill allows. Anything else the model says is not an
|
||||
# answer, and is treated as a refusal rather than coerced into one.
|
||||
ANSWERS = ("clear", "problem_seen", "cannot_tell")
|
||||
|
||||
# One image, one question, one JSON object back. 400 output tokens is roughly
|
||||
# four times the longest legal answer -- enough that a well-formed reply is
|
||||
# never truncated, small enough that a runaway one costs nothing.
|
||||
MAX_TOKENS = 400
|
||||
|
||||
# The API downscales any image whose long edge exceeds 1568 px, so a bigger
|
||||
# render buys nothing and costs upload time. The extension renders to 1500;
|
||||
# this is the ceiling it is staying under, restated here because the two
|
||||
# halves cannot import each other and a silent drift between them would show
|
||||
# up as unexplained illegibility.
|
||||
IMAGE_LONG_EDGE_CEILING_PX = 1568
|
||||
|
||||
|
||||
def _not_run(item, reason):
|
||||
return {
|
||||
"id": item.get("id"),
|
||||
"question_id": item.get("question_id"),
|
||||
"status": "NOT_RUN",
|
||||
"reason": reason,
|
||||
"answer": None,
|
||||
"where": None,
|
||||
"why": None,
|
||||
"source": "model-read",
|
||||
}
|
||||
|
||||
|
||||
def _read_json(path):
|
||||
with open(path, encoding="utf-8") as fh:
|
||||
return json.load(fh)
|
||||
|
||||
|
||||
def _write_json(path, obj):
|
||||
with open(path, "w", encoding="utf-8") as fh:
|
||||
json.dump(obj, fh, indent=2, sort_keys=True)
|
||||
|
||||
|
||||
def load_skill(path):
|
||||
"""The instructions the model reads, from the file that is versioned with
|
||||
the code. Deliberately not a string in this module: a prompt buried in a
|
||||
function is a prompt nobody reviews."""
|
||||
with open(path, encoding="utf-8") as fh:
|
||||
return fh.read()
|
||||
|
||||
|
||||
def parse_answer(text, question_id):
|
||||
"""The model's reply -> a result row, or None when it is not an answer.
|
||||
|
||||
Strict on purpose. A reply that is prose, or JSON with an answer word the
|
||||
skill does not define, is NOT coerced into the nearest legal value -- it
|
||||
is a refusal, because a coerced verdict is exactly the hallucination this
|
||||
whole path is built to keep out of the report.
|
||||
"""
|
||||
if not text:
|
||||
return None
|
||||
body = text.strip()
|
||||
if body.startswith("```"):
|
||||
body = body.split("\n", 1)[-1]
|
||||
if body.rstrip().endswith("```"):
|
||||
body = body.rstrip()[:-3]
|
||||
start, end = body.find("{"), body.rfind("}")
|
||||
if start < 0 or end <= start:
|
||||
return None
|
||||
try:
|
||||
obj = json.loads(body[start:end + 1])
|
||||
except ValueError:
|
||||
return None
|
||||
if not isinstance(obj, dict):
|
||||
return None
|
||||
answer = obj.get("answer")
|
||||
if answer not in ANSWERS:
|
||||
return None
|
||||
if obj.get("question_id") not in (None, question_id):
|
||||
# It answered a different question than the one it was asked. That is
|
||||
# not a near-miss to be salvaged; it means the reply is not about this
|
||||
# image.
|
||||
return None
|
||||
why = obj.get("why")
|
||||
where = obj.get("where")
|
||||
return {
|
||||
"answer": answer,
|
||||
"where": where if isinstance(where, str) and where.strip() else None,
|
||||
"why": why if isinstance(why, str) and why.strip() else None,
|
||||
}
|
||||
|
||||
|
||||
def build_user_content(item):
|
||||
"""One image block, then the question. Image first: the guidance is that a
|
||||
question about an image reads better after the image."""
|
||||
with open(item["image"], "rb") as fh:
|
||||
data = base64.standard_b64encode(fh.read()).decode("utf-8")
|
||||
lines = [
|
||||
"question_id: " + str(item.get("question_id")),
|
||||
"",
|
||||
"This is one view rendered off a precast shop ticket.",
|
||||
]
|
||||
if item.get("view_kind"):
|
||||
lines.append("The view is a " + str(item["view_kind"]) + ".")
|
||||
if item.get("effective_dpi"):
|
||||
lines.append(
|
||||
"It was rendered at about {0:.0f} dots per inch of the printed "
|
||||
"sheet. If that was not enough to read it, say so."
|
||||
.format(float(item["effective_dpi"])))
|
||||
lines.append("")
|
||||
lines.append("Answer the question named above, as one JSON object.")
|
||||
# OpenAI-compatible chat format: image_url carries the base64 data URI.
|
||||
mime = "image/png"
|
||||
if item.get("image", "").lower().endswith((".jpg", ".jpeg")):
|
||||
mime = "image/jpeg"
|
||||
return [
|
||||
{"type": "image_url",
|
||||
"image_url": {"url": "data:{0};base64,{1}".format(mime, data)}},
|
||||
{"type": "text", "text": "\n".join(lines)},
|
||||
]
|
||||
|
||||
|
||||
def ask(endpoint, model, skill_text, item):
|
||||
"""One call. Returns a result row; never raises for an API problem."""
|
||||
try:
|
||||
content = build_user_content(item)
|
||||
except OSError as ex:
|
||||
return _not_run(item, "the rendered image could not be read: "
|
||||
"{0}".format(type(ex).__name__))
|
||||
|
||||
started = time.time()
|
||||
try:
|
||||
body = json.dumps({
|
||||
"model": model,
|
||||
"max_tokens": MAX_TOKENS,
|
||||
"messages": [
|
||||
{"role": "system", "content": skill_text},
|
||||
{"role": "user", "content": content},
|
||||
],
|
||||
}).encode("utf-8")
|
||||
req = urllib.request.Request(endpoint, data=body, headers={
|
||||
"Content-Type": "application/json",
|
||||
})
|
||||
with urllib.request.urlopen(req, timeout=180) as resp:
|
||||
reply = json.loads(resp.read().decode("utf-8"))
|
||||
except Exception as ex: # noqa: BLE001 - see below
|
||||
# Every failure to REACH the model is one outcome: NOT RUN, with the
|
||||
# reason. Never a pass. The exception type is reported and the message
|
||||
# is not, because an error string can carry request detail and
|
||||
# this file's whole discipline is that nothing sensitive is written
|
||||
# to a file the tool keeps.
|
||||
return _not_run(item, "the model could not be reached: {0}"
|
||||
.format(type(ex).__name__))
|
||||
|
||||
try:
|
||||
text = "".join(
|
||||
c.get("text", "") for c in reply["choices"][0]["message"]["content"]
|
||||
if isinstance(c, dict) and c.get("type") == "text")
|
||||
except (KeyError, IndexError, TypeError):
|
||||
text = reply.get("choices", [{}])[0].get("message", {}).get("content", "")
|
||||
if isinstance(text, str):
|
||||
pass
|
||||
else:
|
||||
text = ""
|
||||
parsed = parse_answer(text, item.get("question_id"))
|
||||
if parsed is None:
|
||||
return _not_run(item, "the model's reply was not one of the three "
|
||||
"answers the skill defines")
|
||||
|
||||
usage = reply.get("usage", {})
|
||||
row = _not_run(item, None)
|
||||
row.update(parsed)
|
||||
row["status"] = "OK"
|
||||
row["reason"] = None
|
||||
row["model"] = model
|
||||
row["elapsed_s"] = round(time.time() - started, 2)
|
||||
row["input_tokens"] = usage.get("prompt_tokens")
|
||||
row["output_tokens"] = usage.get("completion_tokens")
|
||||
return row
|
||||
|
||||
|
||||
def run(request):
|
||||
"""The request dict -> the response dict. No I/O beyond the images."""
|
||||
items = request.get("items") or []
|
||||
model = request.get("model") or DEFAULT_MODEL
|
||||
out = {"model": model, "source": "model-read", "results": []}
|
||||
|
||||
max_calls = request.get("max_calls")
|
||||
if isinstance(max_calls, int) and max_calls >= 0 and len(items) > max_calls:
|
||||
# The bound is the caller's, and it is enforced here as well as there.
|
||||
# Over-budget items are NOT RUN naming the bound -- a dropped item
|
||||
# would read as an item that passed.
|
||||
for item in items[max_calls:]:
|
||||
out["results"].append(_not_run(
|
||||
item, "this run's vision call budget ({0}) was already spent"
|
||||
.format(max_calls)))
|
||||
items = items[:max_calls]
|
||||
|
||||
try:
|
||||
skill_text = load_skill(request["skill"])
|
||||
except (OSError, KeyError) as ex:
|
||||
out["results"][:0] = [_not_run(
|
||||
item, "the ticketqc-vision skill file could not be read "
|
||||
"({0})".format(type(ex).__name__)) for item in items]
|
||||
return out
|
||||
|
||||
endpoint = (request.get("endpoint") or
|
||||
os.environ.get("VISION_ENDPOINT") or DEFAULT_ENDPOINT)
|
||||
# The endpoint is a bare host[:port] in some configs; accept both that
|
||||
# and a full URL, and always hit the chat/completions path.
|
||||
if not endpoint.startswith("http"):
|
||||
endpoint = "http://{0}/v1/chat/completions".format(endpoint)
|
||||
|
||||
for i, item in enumerate(items):
|
||||
out["results"].insert(i, ask(endpoint, model, skill_text, item))
|
||||
return out
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument("request", help="request JSON written by ticketvision")
|
||||
ap.add_argument("response", help="where to write the response JSON")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
request = _read_json(args.request)
|
||||
response = run(request)
|
||||
_write_json(args.response, response)
|
||||
|
||||
ok = len([r for r in response["results"] if r["status"] == "OK"])
|
||||
print("{0} answered, {1} NOT RUN".format(
|
||||
ok, len(response["results"]) - ok))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user