will/willtools-demo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

WillTools 2026-09-09 -- clean snapshot of the development repo for review
ci / test (push) Successful in 6m14s
Details

Browse Source 
History-free snapshot of the private development repository at 77a82eb.
Code, docs and the public catalogue only: no plant or job knowledge
layers, no client, job or producer names (tools/scrub.py clean against
the widened term list; a second word-level grep clean). The intro deck is
left out because its slide images cannot be checked for job drawings.
This commit is contained in:
will123song
2026-09-09 08:35:15 -05:00
commit 25592fa6c2
406 changed files with 223435 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
WillTools.extension/lib/vibe/takeoffcount.py
+470
View File
@@ -0,0 +1,470 @@
# -*- coding: utf-8 -*-
"""How many physical pieces one mark is. The control-number multiplier. No Revit.
WHY THIS IS ITS OWN MODULE
---------------------------
It is the single number that decides whether the takeoff is right, and there
is more than one place it can come from. An assembly is a MARK, a mark is one
drawing, and the pieces on the floor are the pieces the mark issues.
Identical pieces share a mark, so counting marks under-orders every material
in the job.
How badly is measurable off an exported ticket set without opening Revit:
151 tickets, 71 distinct marks. The heaviest mark in that sample issued
eighteen tickets. A per-mark takeoff of that sample would have ordered 71
pieces' worth of hardware for 151 pieces -- IF one ticket is one piece there,
which is a declared fact and not this module's to assume. See the ticket-unit
section below; it is the reason a ticket set can be read to two different
numbers.
THREE SOURCES, RANKED, AND THE RANKING IS THE POINT
----------------------------------------------------
control_numbers the control numbers carried by the model's own
elements, grouped by mark. AUTHORITATIVE: a control
number is one physical piece, and this is where they
actually live. Measured on this job by the job
reconciliation: 2480 control-numbered elements ->
1776 pieces, against 755 assemblies. They sit on
STRUCTURAL FRAMING; walls, floors, columns,
foundations and generic models were swept -- 17,239
elements -- and carried none.
ticket_files the exported tickets, named <job>-<control>_<mark>.
Only covers marks that have been ticketed, so it is a
cross-check rather than a source -- and a very good
one, because it is ground truth for the marks it does
cover.
assembly_instances how many assembly instances share the mark's name.
MEASURED AND FOUND USELESS ON THIS JOB, and kept only
because it is useless in a way worth recording: 755
assembly instances, 755 distinct names, not one
duplicate. It therefore returned 1 for every mark,
which made every per-job figure silently equal its
per-mark figure -- the exact undercount this module
exists to prevent, wearing the costume of a sourced
number. It now REFUSES to emit a count of one (see
`from_assembly_names`), so on a job like this one it
contributes nothing at all and the marks it cannot
speak for stay visibly `assumed_one`.
Anything none of them covers is 1 with basis `assumed_one`, which the report
prints as an UNDERCOUNT rather than as a quantity. A multiplier nobody could
source is not a multiplier of one -- and the reason that rule was worth
writing is that it is what caught `assembly_instances` being wrong.
WHAT ONE TICKET IS, AND WHY THIS MODULE MAY NOT DECIDE IT
----------------------------------------------------------
`ticket_files` is a piece count only where ONE TICKET IS ONE PIECE. That is
not universal and it is not this module's to assume: some plants issue one
ticket per control number, and some issue one ticket covering a whole mark --
one drawing, one sheet, a QUANITY field and several control numbers printed
on it. The same export folder read under the second arrangement gives one
filename per mark, and counting those filenames as pieces would return a
confident 1 for a mark that is really seven. That is exactly the
`assembly_instances` failure in different clothes, and it is why the ranking
above stops being a constant here.
So the fact is DECLARED, in `nomenclature.ticket_unit`, and read out of the
knowledge layers rather than compiled in. `takeoffgather.run` does the read,
with the literal path, and hands the value to `counting_plan()` below, which
turns it into the ranking this run is allowed to use and one sentence saying
why. Three states, and all three are printed:
control_number one ticket per control number. A ticket IS a piece, so
all three sources rank exactly as `ORDER` says.
piece_mark one ticket covers the mark. A ticket filename is not a
piece count, so `ticket_files` is WITHDRAWN and the
ticket-against-model cross-check stands down with a
sentence rather than reporting every mark as a
disagreement.
not declared nobody has said, so this run does not know which of the
two it is looking at. `ticket_files` is withdrawn HERE
TOO, and the refusal names the key to declare.
That last one is the only judgement call in this file and it goes the way
this codebase's other unset switches go -- `rails.left_rail_is_fixed`,
`takeoff.not_ordered`, `notes.instruction_views`. Falling back to
"one ticket is one piece" would be a guess that can only ever UNDER-count,
and it would arrive wearing a source's name (`ticket_files`) rather than
`assumed_one`. Withdrawing the source instead costs nothing that was known:
the marks it covered fall to `assumed_one`, which the report already prints
as an undercount in those words. A number that says "could not judge" is a
different thing from a number that says "judged, found one", and the whole
point of this module is that the two must never look alike.
ONE PARSER FOR ONE RULE
------------------------
A control field may hold a range: `50120-50123` is four pieces, not one
token. `jobrecon.expand_control_field` already implements that expansion,
with its dash normalisation and its 500-wide sanity cap, and this module
CALLS IT rather than reimplementing it. Three implementations of one parsing
rule is how they come to disagree.
"""
# pylint: disable=invalid-name
import re
from vibe import jobrecon
# <job>-<control>_<mark>.pdf, with an optional revision suffix on the mark.
# The older export was named by mark alone, which is exactly how one file per
# mark came to be mistaken for one piece per mark; a name with no control
# number in it is therefore IGNORED rather than counted as one piece.
TICKET_RE = re.compile(r'^(?P<job>[^-_]+)-(?P<control>[^_]+)_(?P<mark>.+)$')
# Precedence, best first. A mark found by two sources takes the better one
# and the report can still show what the other said.
#
# It is the ranking FOR A JOB WHERE ONE TICKET IS ONE PIECE. Which sources a
# given job may actually use is `counting_plan()`, off the declared
# `nomenclature.ticket_unit`; this tuple is the full ranking those plans are
# drawn from, and no caller should reach for it instead of a plan.
ORDER = ('control_numbers', 'ticket_files', 'assembly_instances')
# `nomenclature.ticket_unit` -- what one exported ticket covers. A JOB fact:
# the same plant can do it either way and the manual shows both, so it is
# declared in the job layer and never inferred here.
TICKET_UNIT_KEY = 'nomenclature.ticket_unit'
CONTROL_NUMBER = 'control_number'
PIECE_MARK = 'piece_mark'
# Which sources are a PIECE COUNT under each declared unit. Only
# `ticket_files` turns on this: a control number off a model element is one
# physical piece however the plant tickets it, and a duplicated assembly name
# is a duplicated piece either way.
NO_TICKETS = tuple(b for b in ORDER if b != 'ticket_files')
UNIT_SOURCES = {
CONTROL_NUMBER: ORDER,
PIECE_MARK: NO_TICKETS,
}
def _stem(filename):
name = (filename or '').replace('\\', '/').rsplit('/', 1)[-1]
if '.' in name:
name = name.rsplit('.', 1)[0]
return name
def parse_ticket_name(filename):
"""(mark, control) from an exported ticket filename, or (None, None).
A revision suffix belongs to the ticket, not to the piece: two tickets
for the same control number are one piece, so the control number is what
is counted and the mark keeps its revision stripped.
A CONTROL NUMBER CONTAINS AT LEAST ONE DIGIT, and that one rule is the
difference between reading an export folder and reading a ticket set.
The folder also holds the detail sheets, whose names take the identical
three-part shape -- `<job>-SHEET_X4-1.pdf` parses out a "control number"
of SHEET and a "mark" of X4-1 -- and twelve of those were being counted
as marks. They cost nothing in quantity, because no real mark is named
after a detail sheet, but they inflated what the source claimed to know
and they were caught by `join_quality` rather than by anyone reading the
code. Letters in a control number are still fine; letters ALONE are not.
"""
found = TICKET_RE.match(_stem(filename))
if found is None:
return (None, None)
control = found.group('control').strip()
if not any(ch.isdigit() for ch in control):
return (None, None)
mark = found.group('mark').strip()
mark = re.sub(r'[_-]R\d+$', '', mark, flags=re.IGNORECASE)
return (mark or None, control or None)
def unparsed_ticket_files(filenames):
"""The filenames `from_ticket_files` could not read. Never silent.
A folder of exports is somebody's directory and holds whatever they put
in it. What was skipped is a fact about the source's coverage, so it is
returned rather than assumed to be rubbish.
"""
return sorted(name for name in (filenames or [])
if parse_ticket_name(name) == (None, None))
def from_ticket_files(filenames):
"""{mark: (distinct control numbers, 'ticket_files')}.
Distinct, not total: a mark reissued after a revision has two files and
one piece.
"""
seen = {}
for filename in filenames or []:
mark, control = parse_ticket_name(filename)
if mark and control:
seen.setdefault(mark, set()).add(control)
return dict((mark, (len(seen[mark]), 'ticket_files')) for mark in seen)
def from_assembly_names(names, min_instances=2):
"""{mark: (instances, 'assembly_instances')} -- DUPLICATED NAMES ONLY.
Wants the names WITH duplicates, because the duplication is the whole
signal. A name that appears once is NOT evidence that the mark is one
piece; it is the absence of evidence, and the two must not produce the
same output.
That distinction is not academic. Measured on this job through the
bridge: 755 assembly instances, 755 distinct names, zero duplicates. An
earlier version of this function returned 1 for each of them, every mark
got a confident `assembly_instances` basis, and every per_job figure
silently equalled its per_mark figure -- a whole-job undercount that
looked sourced. `min_instances=2` is what makes that impossible: with no
duplicates this returns {} and every mark falls through to
`assumed_one`, which the report prints as an undercount in those words.
"""
out = {}
for name in names or []:
if not name:
continue
out[name] = out.get(name, 0) + 1
return dict((name, (out[name], 'assembly_instances')) for name in out
if out[name] >= min_instances)
def from_piece_records(pieces):
"""({mark: (pieces, 'control_numbers')}, unreadable) from model elements.
`pieces` is the list `jobreconscan.gather` produces -- the same data
contract the job reconciliation eats, deliberately, so that the takeoff
and the reconciliation cannot end up disagreeing about how many pieces
the job has. Each record is {'control', 'mark', 'id', ...}.
Control numbers are counted DISTINCT per mark, because a control number
shared by several elements is still one piece: the reconciliation reports
2480 control-numbered elements resolving to 1776 pieces, and the
difference between those two numbers is exactly that sharing.
A control field is expanded through `jobrecon.expand_control_field`, so a
range is four pieces rather than one token and an unreadable slot is
RETURNED rather than dropped. An element whose mark is missing cannot be
attributed to a mark and is returned as unreadable too -- it is a real
piece this multiplier cannot place, and losing it silently would
under-count in exactly the way this module exists to prevent.
"""
by_mark = {}
unreadable = []
for piece in pieces or []:
mark = (piece.get('mark') or '').strip()
controls, bad = jobrecon.expand_control_field(piece.get('control'))
for token in bad:
unreadable.append({'id': piece.get('id'), 'mark': mark or None,
'token': token,
'why': 'not readable as a control number'})
if not controls:
continue
if not mark:
unreadable.append({'id': piece.get('id'), 'mark': None,
'token': ', '.join(controls),
'why': 'the element carries no mark, so its '
'pieces cannot be attributed'})
continue
by_mark.setdefault(mark, set()).update(controls)
return (dict((mark, (len(by_mark[mark]), 'control_numbers'))
for mark in by_mark),
unreadable)
def from_control_numbers(by_mark):
"""{mark: (distinct control numbers, 'control_numbers')}.
The already-grouped form, for a caller that has the mapping in hand.
`from_piece_records` is the one that takes the reconciliation's own
element records and does the grouping and the range expansion.
`by_mark` is {mark: [control numbers]} read off the model. Distinct
again, and an empty list is NOT a zero: a mark whose control numbers
could not be read is left out entirely so it falls through to the next
source rather than being multiplied by nothing.
"""
out = {}
for mark in by_mark or {}:
controls = set(str(c).strip() for c in (by_mark[mark] or [])
if str(c).strip())
if controls:
out[mark] = (len(controls), 'control_numbers')
return out
def counting_plan(unit, sources=None):
"""Which sources may count pieces, given the DECLARED `ticket_unit`.
`unit` is the value of `nomenclature.ticket_unit` as the knowledge layers
hold it -- read by the caller, so this module needs no knowledge access
and stays testable with a string. `sources` is {basis: {mark: (n, basis)}}
and is optional: it is used only to say what withdrawing a source
actually COST on this job, which is the difference between a rule and a
finding.
Returns a plan the report prints whole:
unit what was declared, or None
declared somebody said
recognised and this module knows what they said
order the bases that may count pieces, best first
withdrawn the bases that may not, and why is in `reason`
withdrawn_marks marks a withdrawn source knew
cost marks that lose their ONLY source to the withdrawal
reason one sentence, ALWAYS, including when nothing changed
refusal the same sentence when the run COULD NOT JUDGE --
undeclared or unrecognised. None when somebody
declared it, whichever way they declared it.
An unrecognised value is a refusal and not a fallback. A key set to
something nobody implemented is somebody's intention, misspelt; reading
it as the default would silently do the opposite of what they asked for
on exactly the jobs where it matters.
"""
unit = (str(unit).strip() or None) if unit is not None else None
order = UNIT_SOURCES.get(unit)
plan = {
'key': TICKET_UNIT_KEY,
'unit': unit,
'declared': unit is not None,
'recognised': order is not None,
'order': tuple(order if order is not None else NO_TICKETS),
'reason': None,
'refusal': None,
}
plan['withdrawn'] = tuple(b for b in ORDER if b not in plan['order'])
if unit == CONTROL_NUMBER:
plan['reason'] = (
'`{0}: {1}` -- this job issues one ticket per control number, so '
'one exported ticket is one piece and the ticket files count as '
'pieces, ranked behind the model\'s own control '
'numbers.'.format(TICKET_UNIT_KEY, CONTROL_NUMBER))
elif unit == PIECE_MARK:
plan['reason'] = (
'`{0}: {1}` -- one ticket covers a whole mark, so a ticket '
'filename names one control number and NOT one piece. The '
'exported tickets were therefore NOT counted as pieces, and the '
'ticket-against-model cross-check stands down rather than '
'reporting every mark as a disagreement. Marks that only the '
'tickets covered are counted once and shown as an '
'undercount.'.format(TICKET_UNIT_KEY, PIECE_MARK))
elif unit is None:
plan['refusal'] = (
'`{0}` is not declared, so this run does not know whether one '
'exported ticket is one piece or a whole mark of them. The '
'exported tickets were NOT used as a piece count: taking one '
'ticket as one piece would under-count a job ticketed the other '
'way, and it would do it wearing a source\'s name instead of '
'`assumed_one`. Declare `{0}: {1}` (one ticket per control '
'number) or `{0}: {2}` (one ticket covering the mark) in the job '
'knowledge file and press again.'.format(
TICKET_UNIT_KEY, CONTROL_NUMBER, PIECE_MARK))
else:
plan['refusal'] = (
'`{0}` is set to "{1}", which this takeoff does not recognise -- '
'it knows `{2}` and `{3}`. A value nobody can read is not a '
'declaration, so the exported tickets were NOT used as a piece '
'count and nothing was assumed on your behalf.'.format(
TICKET_UNIT_KEY, unit, CONTROL_NUMBER, PIECE_MARK))
if plan['reason'] is None:
plan['reason'] = plan['refusal']
known = {}
for basis in (sources or {}):
for mark in (sources[basis] or {}):
known.setdefault(mark, set()).add(basis)
plan['withdrawn_marks'] = len(
[m for m in known if known[m] & set(plan['withdrawn'])])
plan['cost'] = len([m for m in known
if not known[m] & set(plan['order'])])
return plan
def merge(*sources, **kwargs):
"""Fold the sources together, best basis first. Later never overrides.
`order` (keyword) is the ranking this job is allowed to use --
`counting_plan()['order']`. It defaults to `ORDER`, all three sources, so
a caller that has not asked the question gets exactly the behaviour this
function has always had.
A basis NOT in `order` is DROPPED, never ranked last: a source that is not
a piece count on this job must not become one merely because nothing
better covered the mark. Ranking it 99th would let it through on exactly
the marks where nothing can check it, which is where a wrong multiplier
does its damage.
"""
order = kwargs.pop('order', None) or ORDER
if kwargs:
raise TypeError('merge() got an unexpected keyword argument {0}'
.format(sorted(kwargs)[0]))
rank = dict((basis, i) for i, basis in enumerate(order))
out = {}
for source in sources:
for mark in source or {}:
count, basis = source[mark]
if basis not in rank:
continue
have = out.get(mark)
if have is None or rank[basis] < rank.get(have[1], 99):
out[mark] = (int(count), basis)
return out
def disagreements(a, b):
"""Marks where two sources give different counts. Worth printing.
A mismatch between the exported tickets and whatever the model says is
the cheapest possible audit of the multiplier, and it is the one number
in this takeoff that nothing else can check.
"""
out = []
for mark in sorted(set(a or {}) & set(b or {})):
if a[mark][0] != b[mark][0]:
out.append({'mark': mark,
'left': a[mark][0], 'left_basis': a[mark][1],
'right': b[mark][0], 'right_basis': b[mark][1]})
return out
def join_quality(multipliers, marks):
"""Do the multiplier's marks and the takeoff's marks name the same things?
The takeoff keys on the ASSEMBLY NAME; the multiplier keys on the piece
MARK read off a framing element. Measured, those are the same string --
an assembly named for its mark -- but "measured" is not "guaranteed", and
the failure if they ever diverge is silent and total: every mark falls
through to `assumed_one` and the job is under-ordered wholesale while the
report shows a source that found plenty.
So the join is counted, both ways, and printed. `unmatched_source` is the
tell: a source that found hundreds of marks the takeoff has never heard
of is not a source, it is a different naming scheme.
"""
marks = set(marks or [])
source = set(multipliers or {})
return {
'takeoff_marks': len(marks),
'source_marks': len(source),
'matched': len(marks & source),
'unsourced': sorted(marks - source),
'unmatched_source': sorted(source - marks),
}
def summarise(multipliers, marks=None):
"""The multiplier, as a reader has to be able to check it.
`marks` is every mark the takeoff covered, so that a mark with no
multiplier is counted into `assumed_one` rather than vanishing.
"""
marks = list(marks) if marks is not None else list(multipliers or {})
by_basis = {}
total = 0
for mark in marks:
count, basis = (multipliers or {}).get(mark, (1, 'assumed_one'))
by_basis[basis] = by_basis.get(basis, 0) + 1
total += int(count)
return {
'marks': len(marks),
'pieces': total,
'multiplier': (round(total / float(len(marks)), 3) if marks else None),
'by_basis': by_basis,
'assumed_one': by_basis.get('assumed_one', 0),
}