will/willtools-demo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

WillTools: precast shop-ticket QC and panel handling tools
ci / test (push) Has been cancelled
Details

Browse Source 
Public demo snapshot of an in-house pyRevit extension for precast concrete
shop-ticket QC (37 checks / 46 checklist rows) and a standalone panel
handling solver (PCI Chapter 8).

- No client, job, plant or product data: knowledge layers ship as
  structural templates only.
- Ticket QC's model-read escalation targets a LAN model server endpoint
  (configurable), not a cloud provider.
- Built from the private development repo via tools/make_dist.py.
This commit is contained in:
will123song
2026-09-03 16:33:47 -05:00
commit 52d94303bc
281 changed files with 155656 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
WillTools.extension/lib/vibe/takeoffcore.py
+597
View File
@@ -0,0 +1,597 @@
# -*- coding: utf-8 -*-
"""The takeoff: what a job actually contains, per piece and in total. No Revit.
WHAT THIS IS BUILT ON, AND WHY IT IS NOT ANOTHER MODEL SWEEP
-------------------------------------------------------------
`vibe.reinforceindex` already walks every assembly, resolves its members and
records a `families` dict -- family name -> instance count -- on every piece
record. That is a takeoff's raw material and nothing was reading it. This
module takes the index cache as its INPUT and adds no model access of its
own, so a whole-job takeoff costs whatever `reinforceindex.read()` costs and
nothing more.
The consequence is a limit, stated once here because every honest line of the
output depends on it: THIS MODULE CAN ONLY SEE WHAT THE INDEX RECORDED.
Three limits follow and each is reported rather than papered over.
1. The index builds only ELIGIBLE pieces -- reinforced, not on hold. A job
has more assemblies than that. The caller passes `population` so the
report can say what fraction of the job it covered instead of implying
the whole of it.
2. The index keys `families` by FAMILY NAME, and at this plant a
reinforcing family is a BEND SHAPE, not a bar mark: one shape family
carries one bar mark on one piece and a different one on the next. So
hardware resolves to a part number off the family name and REINFORCING
LARGELY DOES NOT. Those instances are counted and reported as
`bar_shape` with the mark unresolved -- never dropped, and never
silently rolled into a bar mark that was guessed.
The fix is one field on the index record and it is written up in
`docs/takeoff.md`: a `marks` counter built from
`ticketqc._member_mark(member, know)`, which already returns the mark an
engineer says out loud on a bar and on an anchor alike. `piece_lines()`
uses it the moment it appears -- see `_bar_lines_from_marks` -- and
needs no other change here.
3. `features is None` on an index record means the piece could not be
MEASURED. It does not mean the piece is empty, and it does not stop the
takeoff: the `families` dict is recorded on nearly every failure path.
A record with no families dict at all is the real could-not-judge, and
it is listed as one with its reason rather than counted as zero.
THE COUNTING RULES THIS JOB HAS ALREADY PAID TO LEARN
------------------------------------------------------
**Control numbers.** An assembly is a MARK. A mark is a drawing. The pieces
on the floor are CONTROL NUMBERS. In one sampled ticket set 151 tickets
carried only 71 distinct marks, and one mark alone accounted for eighteen of
them. So a takeoff that counts marks under-orders every material in the job,
and this one multiplies -- but it prints both figures side by side, with the
multiplier and where the multiplier came from, because per-mark and per-job
answer different questions and a reader will want to check one against the
other. A multiplier the caller could not source is 1 with basis
`assumed_one`, and that is reported as an UNDERCOUNT, loudly, not as a
number.
This module never decides that multiplier and never reads a knowledge key for
it: `pieces` arrives per mark, already sourced and already ranked. What one
exported ticket covers -- `nomenclature.ticket_unit` -- is declared per job,
read in `takeoffgather.run`, and acted on by `takeoffcount.counting_plan`.
Both files carry the reasoning.
**Nesting.** A member whose `SuperComponent` is set is a child and still
appears in `GetMemberIds()`, so a family nested inside another is counted
twice: once as itself, once inside its parent's instance. This module does
not guess. A nesting DECLARED in `takeoff.nested_in` is honoured -- the child
leaves the order list for its own section. A nesting nobody declared is
DETECTED and reported as a suspicion for a person to rule on:
`suspected_nesting()` finds family pairs whose counts are equal on every
piece where both occur. It never subtracts on a suspicion. An automatic
correction that is wrong is invisible in the total; a suspicion that is wrong
costs one sentence to dismiss.
**Attached bars.** `drawings.annotation_covered_by_parent` already records
which bar marks arrive welded to which part. Those bars are real steel and
are always COUNTED. Whether they are ORDERED is a purchasing question, not a
geometry one, so it is a switch -- `takeoff.order_attached_bars_separately`,
default false -- and either way they appear in their own section with the
parent named. Silently either way is the one thing not on offer.
WHAT IT REFUSES TO COUNT
-------------------------
A family the rules cannot classify goes to `unclassified` with its name, its
count and how many pieces carry it. It is never bucketed into 'other'. Every
classification carries a `basis`: `declared` when a knowledge key named the
family outright, `rule:<name>` when a pattern decided it, and the report
groups by basis so every inference this module made is on the page.
"""
# pylint: disable=invalid-name
import re
# Every class this takeoff knows. `unclassified` is a real answer, not a
# failure mode, and it is the only one whose contents are a work list.
BAR = 'bar' # reinforcing, mark resolved
BAR_SHAPE = 'bar_shape' # reinforcing, family is a bend shape, mark unknown
STRAND = 'strand'
MESH = 'mesh'
HARDWARE = 'hardware' # an orderable part with a part number
CAST = 'cast' # separately modelled cast concrete, not an insert
NESTED = 'nested' # declared as a child of another family
NOT_ORDERED = 'not_ordered' # the piece itself, or a modelling construct
UNCLASSIFIED = 'unclassified'
ORDERABLE = (BAR, BAR_SHAPE, STRAND, MESH, HARDWARE, CAST)
# Defaults live here rather than in `knowledge.DEFAULTS` so that this module
# owns them and no other file has to change to add a takeoff. Any knowledge
# layer overrides them under `takeoff:`; the plant or job layer is where the
# real prefixes belong, and `docs/takeoff.md` carries the block to paste.
#
# `not_ordered` DEFAULTS TO EMPTY on purpose. A default list of things to
# ignore is a default under-count, and the families it would name -- the
# panel itself, a modelling zone -- are plant nomenclature that this module
# has no business knowing. Left empty they surface as `unclassified`, which
# is the loud version of the same fact and the one a person can act on.
DEFAULTS = {
'bar_prefixes': ['SR'],
'part_prefixes': ['SP', 'P'],
'shape_markers': ['(SCHEDULED)'],
'not_ordered': [],
'family_class': {},
'nested_in': {},
'attached_to': {},
'order_attached_bars_separately': False,
}
def settings(know, get=None):
"""The `takeoff:` block, defaults filled in. `get` is `knowledge.get`.
`get` is passed in rather than imported so this module needs nothing but
the standard library, which is what keeps it testable off a model.
"""
out = dict(DEFAULTS)
for key in DEFAULTS:
if get is None:
value = ((know or {}).get('takeoff') or {}).get(key)
else:
value = get(know, 'takeoff.' + key)
if value is not None:
out[key] = value
return out
# ------------------------------------------------------------ classification
def _up(text):
return (text or '').upper()
def _prefixed(name, prefixes):
"""The longest `<PREFIX><digits>` token in `name`, or None.
Longest match wins so a part in an `SP` series is never read as the `P`
series inside it, and the lookbehind is what stops an embed family whose
name wraps a part number from resolving to anything but that number.
"""
up = _up(name)
best = None
for prefix in sorted([_up(p) for p in (prefixes or [])],
key=len, reverse=True):
if not prefix:
continue
found = re.search(r'(?<![A-Z0-9])' + re.escape(prefix) + r'(\d+)', up)
if found is None:
continue
token = prefix + found.group(1)
if best is None or len(token) > len(best):
best = token
return best
def classify(family, cfg, is_strand=None, is_mesh=None, is_cast=None):
"""(class, key, basis) for one family name.
`key` is what a person orders by: a part number, a bar mark, a bend
shape, or -- when nothing resolved -- the family name itself, so that an
unclassified row still names the thing it could not classify.
Order matters and is deliberate:
1. a family named outright in knowledge beats every pattern;
2. a declared nesting, so a child never reaches the order list;
3. a declared not-ordered family;
4. strand, mesh and cast concrete, which the existing `families:`
rules already decide and which this module must not re-decide;
5. a bend-shape marker, BEFORE the bar-prefix rule -- a shape family
has no mark in its name and must not borrow one;
6. bar prefixes, then part prefixes;
7. unclassified.
"""
up = _up(family)
declared = cfg.get('family_class') or {}
for key in declared:
if _up(key) == up:
return (str(declared[key]), family, 'declared')
for child in (cfg.get('nested_in') or {}):
if _up(child) == up:
return (NESTED, family, 'declared')
for token in (cfg.get('not_ordered') or []):
if _up(token) and _up(token) in up:
return (NOT_ORDERED, family, 'declared')
if is_strand is not None and is_strand(family):
return (STRAND, family, 'rule:strand')
if is_mesh is not None and is_mesh(family):
return (MESH, family, 'rule:mesh')
if is_cast is not None and is_cast(family):
return (CAST, family, 'rule:cast')
for marker in (cfg.get('shape_markers') or []):
if _up(marker) and _up(marker) in up:
return (BAR_SHAPE, family, 'rule:shape_marker')
mark = _prefixed(family, cfg.get('bar_prefixes'))
if mark:
return (BAR, mark, 'rule:bar_prefix')
part = _prefixed(family, cfg.get('part_prefixes'))
if part:
return (HARDWARE, part, 'rule:part_prefix')
return (UNCLASSIFIED, family, 'none')
# ------------------------------------------------------------- attached bars
def attached_parents(cfg, know, get=None):
"""{BAR MARK: [parent marks]} -- bars that ship welded to a part.
Read from `drawings.annotation_covered_by_parent`, which the drawing
checks already maintain, so a bar cannot be attached for the purpose of
one check and loose for another. `takeoff.attached_to` overrides it for a
job where the two genuinely differ.
"""
if get is None:
source = ((know or {}).get('drawings') or {}).get(
'annotation_covered_by_parent') or {}
else:
source = get(know, 'drawings.annotation_covered_by_parent', {}) or {}
override = cfg.get('attached_to')
if override:
source = override
out = {}
for child in source:
parents = source[child]
if isinstance(parents, (list, tuple)):
out[_up(child)] = [str(p) for p in parents]
elif parents:
out[_up(child)] = [str(parents)]
return out
# ----------------------------------------------------------------- one piece
def piece_lines(record, cfg, attached=None, is_strand=None, is_mesh=None,
pieces=1, is_cast=None):
"""Every takeoff line for one index record, per-mark and per-job.
`pieces` is the control-number multiplier: how many physical pieces this
one mark issues. `per_mark` is what one of them contains; `per_job` is
what the job needs.
"""
attached = attached or {}
lines = []
families = (record or {}).get('families') or {}
marks = (record or {}).get('marks')
skip = set()
if marks:
lines.extend(_bar_lines_from_marks(
record, marks, families, cfg, attached, pieces))
skip = set(_up(f) for f in _shape_families(families, cfg))
for family in sorted(families):
if _up(family) in skip:
continue
klass, key, basis = classify(family, cfg, is_strand, is_mesh, is_cast)
count = int(families[family] or 0)
lines.append(_line(record, klass, key, family, basis, count, pieces,
attached))
return sorted(lines, key=lambda r: (r['class'], r['key']))
def _shape_families(families, cfg):
"""Families that are bend shapes -- the ones a `marks` counter replaces."""
out = []
for family in families:
for marker in (cfg.get('shape_markers') or []):
if _up(marker) and _up(marker) in _up(family):
out.append(family)
break
return out
def _bar_lines_from_marks(record, marks, families, cfg, attached, pieces):
"""Bar lines built from a per-instance mark counter, when one exists.
THIS IS THE PATH THAT DOES NOT EXIST YET. `reinforceindex.extract` does
not record `marks`; the moment it does, every `bar_shape` row in the
output turns into a real bar mark and nothing else in this module
changes. Written now so that the index change is a one-field change and
not a rewrite, and exercised by the tests with a synthetic record.
A mark counter carries hardware marks too, because a hardware family and
its mark are the same string. Only marks that read as a BAR and are not
already a family we counted are taken from here; anything else would be
counted once as a family and once again as a mark.
"""
known = set(_up(f) for f in families)
lines = []
for mark in sorted(marks):
bar = _prefixed(mark, cfg.get('bar_prefixes'))
if not bar or _up(bar) in known:
continue
lines.append(_line(record, BAR, bar, None, 'index:marks',
int(marks[mark] or 0), pieces, attached))
return lines
def _line(record, klass, key, family, basis, count, pieces, attached):
parents = attached.get(_up(key)) if klass in (BAR, BAR_SHAPE) else None
return {
'piece': (record or {}).get('name'),
'design_number': (record or {}).get('design_number'),
'class': klass,
'key': key,
'family': family,
'basis': basis,
'per_mark': count,
'pieces': int(pieces),
'per_job': count * int(pieces),
'attached_to': list(parents) if parents else None,
}
# ------------------------------------------------------------------- the job
def build(cache, cfg, multipliers=None, attached=None, is_strand=None,
is_mesh=None, population=None, is_cast=None):
"""The whole takeoff. `cache` is a `reinforcecore` cache, read, not touched.
`multipliers` is {piece name: (count, basis)}. A piece missing from it is
1 with basis `assumed_one` and lands in `undercounted`, because a
multiplier nobody could source is not a multiplier of one.
"""
multipliers = multipliers or {}
attached = attached or {}
pieces = (cache or {}).get('pieces') or {}
by_piece = {}
unjudged = []
undercounted = []
all_lines = []
for name in sorted(pieces):
record = pieces[name] or {}
count, basis = multipliers.get(name, (1, 'assumed_one'))
count = int(count or 1)
if basis == 'assumed_one':
undercounted.append(name)
families = record.get('families') or {}
if not families:
unjudged.append({
'piece': name,
'reason': record.get('error') or 'no families recorded',
})
by_piece[name] = {'piece': name, 'pieces': count,
'multiplier_basis': basis, 'lines': [],
'judged': False,
'design_number': record.get('design_number')}
continue
lines = piece_lines(record, cfg, attached, is_strand, is_mesh,
pieces=count, is_cast=is_cast)
by_piece[name] = {
'piece': name,
'design_number': record.get('design_number'),
'pieces': count,
'multiplier_basis': basis,
'lines': lines,
'judged': True,
}
all_lines.extend(lines)
return {
'by_piece': by_piece,
'totals': roll_up(all_lines),
'unclassified': unclassified_rows(all_lines),
'unjudged': unjudged,
'undercounted': sorted(undercounted),
'nesting_suspects': suspected_nesting(pieces),
'attached': attached,
'order_attached_separately': bool(
cfg.get('order_attached_bars_separately')),
'coverage': coverage(cache, population, unjudged),
'bases': bases(all_lines),
'pieces_on_the_floor': sum(
by_piece[n]['pieces'] for n in by_piece),
}
def roll_up(lines):
"""Job totals, keyed (class, key). Both figures, always both."""
out = {}
for line in lines:
slot = (line['class'], line['key'])
row = out.get(slot)
if row is None:
row = {'class': line['class'], 'key': line['key'],
'per_mark_total': 0, 'per_job_total': 0,
'marks': 0, 'families': [], 'bases': [],
'attached_to': line.get('attached_to')}
out[slot] = row
row['per_mark_total'] += line['per_mark']
row['per_job_total'] += line['per_job']
row['marks'] += 1
if line.get('family') and line['family'] not in row['families']:
row['families'].append(line['family'])
if line['basis'] not in row['bases']:
row['bases'].append(line['basis'])
rows = list(out.values())
for row in rows:
row['families'].sort()
row['bases'].sort()
row['multiplier'] = (
round(row['per_job_total'] / float(row['per_mark_total']), 3)
if row['per_mark_total'] else None)
rows.sort(key=lambda r: (r['class'], -r['per_job_total'], r['key']))
return rows
def order_rows(totals, order_attached_separately=False):
"""(order, ships_with_parent, not_ordered) -- the split a buyer needs.
An attached bar is always counted. Whether it appears on the ORDER is the
switch; either way it appears, with its parent named.
"""
order, ships, other = [], [], []
for row in totals:
if row['class'] not in ORDERABLE:
other.append(row)
elif row.get('attached_to') and not order_attached_separately:
ships.append(row)
else:
order.append(row)
return order, ships, other
def unclassified_rows(lines):
"""Every family no rule could place, with what it would have cost.
Never merged into an 'other' row. An under-count that looks tidy is worse
than a messy honest one, so this is the section that has to be read.
"""
out = {}
for line in lines:
if line['class'] != UNCLASSIFIED:
continue
row = out.get(line['key'])
if row is None:
row = {'family': line['key'], 'per_mark_total': 0,
'per_job_total': 0, 'pieces': 0, 'examples': []}
out[line['key']] = row
row['per_mark_total'] += line['per_mark']
row['per_job_total'] += line['per_job']
row['pieces'] += 1
if line.get('piece') and len(row['examples']) < 5:
row['examples'].append(line['piece'])
rows = list(out.values())
rows.sort(key=lambda r: (-r['per_job_total'], r['family']))
return rows
def bases(lines):
"""How many rows each basis decided. The inference budget, on the page."""
out = {}
for line in lines:
out[line['basis']] = out.get(line['basis'], 0) + 1
return out
def coverage(cache, population, unjudged):
"""What the takeoff saw, against what the job has. Never implied.
`assemblies_in_model` is the caller's honest denominator -- the index
only ever builds reinforced, not-on-hold pieces, so `marks_not_indexed`
is a real hole in the job and not a rounding.
"""
pieces = (cache or {}).get('pieces') or {}
population = population or {}
indexed = len(pieces)
assemblies = population.get('assemblies')
return {
'indexed_marks': indexed,
'judged_marks': indexed - len(unjudged),
'unjudged_marks': len(unjudged),
'assemblies_in_model': assemblies,
'marks_not_indexed': (None if assemblies is None
else max(0, assemblies - indexed)),
'built_at': (cache or {}).get('built_at'),
'complete': bool(assemblies is not None and assemblies == indexed
and not unjudged),
}
# ---------------------------------------------------------- nesting suspects
def suspected_nesting(pieces, min_pieces=3, min_agreement=0.9,
min_distinct_counts=2):
"""Family pairs that look like child-inside-parent. A SUSPICION only.
Two families whose counts are EQUAL on the pieces where the first occurs
is the signature of a nested family: the child is counted once on its own
and once inside its parent's instance.
WHY A LOCKSTEP OF ONES IS NOT EVIDENCE. Two families that each appear
exactly once on the same forty pieces agree perfectly and mean nothing --
that is what a pair of unrelated parts on one panel type looks like. What
a nested family does is FOLLOW its parent's count as that count VARIES:
four of one, four of the other; eight and eight. So a pair only qualifies
when the child's count takes at least `min_distinct_counts` different
values across the pieces. Without that filter this list came back with
forty rows of coincidence and the three real pairs buried in them.
WHY THE AGREEMENT IS NOT REQUIRED TO BE PERFECT. The strongest declared
nesting on this job -- a bar that lives inside an embed whose family name
says "with bar" -- has 938 instances of the bar against 943 of the embed.
Five pieces disagree, for reasons that are somebody's to explain, and an
exact-equality test finds nothing at all. So agreement is a RATIO, it is
reported on every row, and `1.0` is visibly different from `0.95`.
DIRECTION IS OFTEN UNDECIDABLE FROM COUNTS ALONE, and saying so is the
honest part. When A and B occur on the same pieces in the same numbers,
the arithmetic cannot tell which sits inside which -- only that one of
the two is very probably being counted twice. Such a pair is reported
ONCE with `direction: undecided` rather than twice with a direction each
way, because two rows reads like two findings.
Nothing is subtracted on this evidence. It is printed so that a person
can move the pair into `takeoff.nested_in`, which IS acted on. An
automatic correction that is wrong is invisible in the total; a suspicion
that is wrong costs one sentence to dismiss.
"""
where = {}
for name in pieces:
record = pieces[name] or {}
for family in (record.get('families') or {}):
where.setdefault(family, {})[name] = int(
record['families'][family] or 0)
def agreement(child, parent):
child_on, parent_on = where[child], where[parent]
if not child_on:
return 0.0
same = 0
for piece in child_on:
if parent_on.get(piece) == child_on[piece]:
same += 1
return same / float(len(child_on))
names = sorted(where)
found = {}
for child in names:
if len(where[child]) < min_pieces:
continue
if len(set(where[child].values())) < min_distinct_counts:
continue
for parent in names:
if parent == child:
continue
if len(where[parent]) < len(where[child]) * min_agreement:
continue
forward = agreement(child, parent)
if forward < min_agreement:
continue
back = (len(where[parent]) >= min_pieces
and agreement(parent, child) >= min_agreement)
key = tuple(sorted((child, parent))) if back else (child, parent)
if key in found:
continue
first, second = (key if back else (child, parent))
found[key] = {
'child': first,
'parent': second,
'direction': 'undecided' if back else 'child_in_parent',
'agreement': round(forward if not back
else min(forward, agreement(parent, child)),
3),
'pieces': len(where[child]),
'instances': sum(where[child].values()),
'parent_pieces': len(where[parent]),
}
out = list(found.values())
out.sort(key=lambda r: (-r['agreement'], -r['instances'], r['child'],
r['parent']))
return out