will/willtools-demo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

WillTools: precast shop-ticket QC and panel handling tools
ci / test (push) Has been cancelled
Details

Browse Source 
Public demo snapshot of an in-house pyRevit extension for precast concrete
shop-ticket QC (37 checks / 46 checklist rows) and a standalone panel
handling solver (PCI Chapter 8).

- No client, job, plant or product data: knowledge layers ship as
  structural templates only.
- Ticket QC's model-read escalation targets a LAN model server endpoint
  (configurable), not a cloud provider.
- Built from the private development repo via tools/make_dist.py.
This commit is contained in:
will123song
2026-09-03 16:33:47 -05:00
commit 52d94303bc
281 changed files with 155656 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
WillTools.extension/lib/vibe/takeoffgather.py
+364
View File
@@ -0,0 +1,364 @@
# -*- coding: utf-8 -*-
"""The Revit half of the takeoff. Gathering only, and READ ONLY.
The split is the one `reinforceindex` / `reinforcecore` already models:
everything decidable without a document is in `takeoffcore`, `takeoffcount`
and `takeoffreport`, all three unit-tested on host CPython. This file does
the gathering and nothing else. There is no Transaction here, no parameter
write, no delete, and no reason for that to change -- a takeoff reports what
a model contains and has no business changing it.
WHAT IT ACTUALLY READS, AND WHAT IT COSTS
------------------------------------------
Almost nothing, because the expensive work is already cached. The takeoff's
input is `reinforceindex.read(doc)` -- the index as it stands, measured at
1.4 s with no rebuild -- plus one cheap pass over the assembly instances for
the multiplier. No view is generated, no geometry is solved, and none of the
30-60 s per piece this project was warned about is paid.
`refresh` is deliberately NOT called from here. A takeoff off a stale index
is a takeoff of the model as it was, which is a knowable and reportable
thing: `coverage.built_at` says when. A takeoff that silently spends six
seconds rebuilding is a button that sometimes freezes Revit and never says
why. Refreshing the index is the reinforcing tool's job and its button.
WHERE THE MULTIPLIER COMES FROM, AFTER IT WAS MEASURED AND WAS WRONG
--------------------------------------------------------------------
The first version of this file counted how many assembly instances share a
name, on the reasonable-sounding claim that three identical pieces of one
mark are three assemblies. It was written down as UNVERIFIED and it was
verified, through the bridge, and it was WRONG: 755 assembly instances, 755
distinct names, zero duplicates. That source returns 1 for every mark, so
every per-job figure silently equalled its per-mark figure -- a whole-job
undercount wearing the costume of a sourced number.
Control numbers do not live on assemblies at all. They live on STRUCTURAL
FRAMING elements, and there are far more of them than there are assemblies:
2480 control-numbered elements resolving to 1776 pieces, against 755
assemblies. Walls, floors, columns, foundations and generic models were swept
-- 17,239 elements -- and carry none.
So the multiplier now comes from `vibe.jobreconscan.gather`, the job
reconciliation's own gatherer, reusing its data contract rather than sweeping
the model a second time with a second opinion about what a piece is. The
takeoff and the reconciliation cannot disagree about how many pieces the job
has, because they are reading the same records through the same expander.
WHAT THAT COSTS, AND THE ONE NARROWING MADE HERE
-------------------------------------------------
`jobreconscan` sweeps six categories at about 6 s. This asks it for
STRUCTURAL FRAMING ONLY, on the measured evidence above that the other five
carry no control numbers -- so the takeoff's multiplier costs roughly a
second rather than six.
That is a deliberate narrowing of somebody else's careful default and it is
therefore reported, not assumed: `extras['multiplier']['categories']` names
what was searched and the report prints it. If a piece is ever modelled as a
wall, this run will not see it, and the honest place for that sentence is on
the page rather than in this comment. Pass `categories=None` to sweep all six.
WHAT ONE EXPORTED TICKET IS, AND WHERE THAT IS READ
----------------------------------------------------
`run()` reads `nomenclature.ticket_unit` out of the knowledge layers and
hands it to `takeoffcount.counting_plan`. It decides one thing and it decides
it for a good reason: the exported tickets are a count of PIECES only where
one ticket is one control number. A plant that puts a whole mark on one
ticket -- one drawing, a QUANITY field, several control numbers listed --
exports one filename per mark, and reading that as one piece per mark is the
`assembly_instances` undercount again.
Declared `piece_mark`, or not declared at all, the ticket source is WITHDRAWN
rather than demoted, and the ticket-against-model cross-check stands down
with a sentence. Undeclared is a refusal that names the key: this run cannot
tell which kind of folder it is looking at, and assuming the generous reading
would under-count while wearing a source's name.
A ROLE NOBODY HAS MAPPED IS A REFUSAL, NOT AN EMPTY JOB
--------------------------------------------------------
`member.control_number` is a knowledge role and it is not mapped in the plant
layer yet. Unmapped, this returns no multipliers at all -- and that must read
as "the takeoff could not find out", never as "every mark is one piece". The
refusal is carried out in `extras['multiplier']['refusal']` and the report
leads the multiplier section with it.
"""
# pylint: disable=import-error,broad-except,invalid-name
import os
from vibe import takeoffcore as core
from vibe import takeoffcount as count
# Every control number in this model sits on structural framing. Measured by
# the job reconciliation across six categories and 17,239 elements; the other
# five carry none. See the module docstring for why this narrowing is
# reported rather than assumed.
FRAMING_ONLY = ('OST_StructuralFraming',)
def population(doc):
"""How many assemblies the model actually has. The honest denominator.
Cheap: `reinforceindex.assemblies` is a scoped collector over assembly
instances with no geometry, measured at 0.29 s over 755 of them.
"""
from vibe import reinforceindex
pairs = reinforceindex.assemblies(doc)
names = [name for name, _el in pairs]
return {
'assemblies': len(set(names)),
'assembly_instances': len(names),
'names': names,
}
def assembly_multipliers(names):
"""{mark: (instances, 'assembly_instances')} -- duplicated names only.
Kept as a last-resort source and expected to return {} on this job: 755
assemblies, 755 distinct names. It can no longer emit a count of one, so
it cannot turn "no source" into a confident 1. See `takeoffcount`.
"""
return count.from_assembly_names(names)
def control_multipliers(doc, know=None, categories=FRAMING_ONLY):
"""({mark: (pieces, 'control_numbers')}, info) off the model's elements.
Delegates the sweep to `vibe.jobreconscan.gather`, which is the module
that established where control numbers live and how a control field is
expanded. Nothing here re-derives either.
`info` always says what happened, including when nothing did: an unmapped
role produces an empty mapping AND a `refusal` string, and the two are
not the same output as a job with no control numbers in it.
"""
from vibe import jobreconscan
from vibe import knowledge as K
info = {'categories': list(categories) if categories else None,
'refusal': None, 'unreadable': [], 'elements': 0, 'notes': []}
if not K.param_names(know, 'member.control_number'):
info['refusal'] = (
'The knowledge role `member.control_number` is not mapped, so the '
'model was not asked how many pieces each mark is. Every mark '
'below is counted ONCE and every per-job figure equals its '
'per-mark figure. This is a refusal, not a finding: map '
'`parameters.member.control_number` in the plant knowledge file '
'(the same role Job Recon needs) and press again.')
return {}, info
try:
pieces, _sheets, notes, _timing = jobreconscan.gather(
doc, know=know, categories=categories)
except Exception as exc:
info['refusal'] = ('The model could not be read for control numbers: '
'{0}. Nothing was multiplied.'.format(exc))
return {}, info
info['elements'] = len(pieces)
info['notes'] = list(notes or [])
multipliers, unreadable = count.from_piece_records(pieces)
info['unreadable'] = unreadable
return multipliers, info
def ticket_multipliers(folder):
"""{mark: (controls, 'ticket_files')} from a folder of exported tickets.
Ground truth for the marks it covers and silent about the rest. Reads
filenames only -- it never opens a PDF, and the folder is a job path that
belongs in knowledge, never in this file.
"""
if not folder or not os.path.isdir(folder):
return {}
try:
names = os.listdir(folder)
except Exception:
return {}
return count.from_ticket_files(names)
# -------------------------------------------------------- the declared scope
def _apply_scope(the_scope, cache, pop):
"""(cache, population, info) narrowed to the declared scope.
Pure dict work, no document. Returns COPIES -- the index cache is read
from disk and shared, and quietly editing it would leave a narrowed index
behind for the next caller who never asked for one.
`info` always carries the line, including when the scope is undeclared or
set nothing aside. A takeoff that counted everything must say so just as
loudly as one that did not, because the reader cannot otherwise tell a
complete count from a filtered one.
"""
pieces = (cache or {}).get('pieces') or {}
names = sorted(pieces.keys())
kept, left = the_scope.split_names(names)
pop_kept, pop_left = the_scope.split_names((pop or {}).get('names') or [])
info = {
'declared': the_scope.is_declared,
'label': the_scope.label(),
'declaration': the_scope.declaration_lines(),
'left_alone': the_scope.tally(left),
'left_alone_marks': len(left),
'counted_marks': len(kept),
'line': the_scope.line(left, total=len(names), unit='indexed mark'),
'assemblies_line': the_scope.line(
pop_left, total=len(pop_kept) + len(pop_left), unit='assembly')
if the_scope.is_declared else None,
}
if not the_scope.is_declared or not left:
return cache, pop, info
new_cache = dict(cache or {})
new_cache['pieces'] = dict((n, pieces[n]) for n in kept)
new_pop = dict(pop or {})
new_pop['names'] = pop_kept
new_pop['assemblies'] = len(set(pop_kept))
new_pop['assembly_instances'] = len(pop_kept)
return new_cache, new_pop, info
# ------------------------------------------------------------------- run it
def run(doc, know=None, ticket_folder=None):
"""The whole takeoff for the open model. READ ONLY.
Returns (takeoff, counts, extras) where `extras` carries the multiplier
sources separately so the report can show them disagreeing. A
disagreement between the exported tickets and the model is the only audit
the multiplier has, and it costs a directory listing.
"""
from vibe import reinforceindex
if know is None:
from vibe import knowledge as K
know = K.load(doc)
from vibe import knowledge as K
cfg = core.settings(know, K.get)
attached = core.attached_parents(cfg, know, K.get)
cache = reinforceindex.read(doc)
pop = population(doc)
# ---- the declared scope ---------------------------------------------
#
# A takeoff is a job document and a job has several engineers on it. What
# is counted here is what the declared scope says is this engineer's, and
# what was left out is COUNTED AND NAMED rather than dropped -- `extras`
# carries the line and the report prints it above the quantities. No
# declaration means everything is counted, which is the honest default.
#
# The index, the assembly population and the multipliers are all narrowed
# together. Narrowing one of them would leave the report claiming a hole
# in the job ('marks in the model but NOT indexed') that is really just
# somebody else's work.
from vibe import scope as scopelib
the_scope = scopelib.of(know, K.get)
cache, pop, scope_info = _apply_scope(the_scope, cache, pop)
if ticket_folder is None:
ticket_folder = K.get(know, 'takeoff.ticket_folder')
# ---- what one exported ticket is ------------------------------------
#
# DECLARED, not decided here: `nomenclature.ticket_unit`. One ticket per
# control number makes the exported tickets a piece count; one ticket per
# mark makes them a count of drawings, and reading a folder of the second
# kind as the first returns a confident 1 for a mark that is really
# seven. The read is done HERE, with the literal path, because this is
# the file that already holds a document and a knowledge module;
# `takeoffcount.counting_plan` owns everything that follows from the
# value and is testable on a string.
unit = K.get(know, 'nomenclature.ticket_unit')
from_control, multiplier_info = control_multipliers(doc, know)
from_assembly = assembly_multipliers(pop['names'])
from_tickets = ticket_multipliers(ticket_folder)
# Order matters and is the plan's, drawn from `takeoffcount.ORDER`:
# control numbers off the model, then the exported tickets IF this job
# tickets one piece at a time, then -- last, and empty on a job whose
# assembly names are all distinct -- duplicated assembly names. A source
# the plan withdrew is dropped, not demoted.
plan = count.counting_plan(unit, {
'control_numbers': from_control,
'ticket_files': from_tickets,
'assembly_instances': from_assembly})
multipliers = count.merge(from_control, from_tickets, from_assembly,
order=plan['order'])
if the_scope.is_declared:
multipliers = dict((m, v) for m, v in multipliers.items()
if the_scope.includes(m))
takeoff = core.build(
cache, cfg,
multipliers=multipliers,
attached=attached,
is_strand=lambda fam: K.is_strand(know, fam),
is_mesh=lambda fam: _is_mesh(K, know, fam),
is_cast=lambda fam: K.matches(
fam, K.get(know, 'families.cast_include', []), []),
population=pop)
marks = sorted((cache.get('pieces') or {}).keys())
counts = count.summarise(multipliers, marks)
counts['refusal'] = multiplier_info.get('refusal')
counts['join'] = count.join_quality(multipliers, marks)
counts['sources'] = {
'control_numbers': len(from_control),
'ticket_files': len(from_tickets),
'assembly_instances': len(from_assembly),
}
counts['ticket_unit'] = plan
extras = {
'scope': scope_info,
'multiplier': multiplier_info,
# The cross-check compares two counts of PIECES. Where the tickets
# are not a piece count -- declared `piece_mark`, or nobody has said
# -- every mark would "disagree" for a reason that is not a finding,
# so the comparison is stood down and the plan's sentence says so
# instead. A cross-check nobody may believe is worse than none.
'disagreements': (count.disagreements(from_control, from_tickets)
if 'ticket_files' in plan['order'] else []),
'index_path': reinforceindex.index_path(doc),
}
return takeoff, counts, extras
def _is_mesh(K, know, family):
prefix = K.get(know, 'families.wwf_prefix', '')
return bool(prefix) and (family or '').upper().startswith(
str(prefix).upper())
def write_reports(doc, takeoff, counts, folder=None, stem='takeoff',
extras=None):
"""Write the markdown and both CSVs beside the model. Returns the paths.
Beside the MODEL, under the same `_ticketqc` root the index and the
ticket evidence already use -- never into the repository. A takeoff is
job data: it names marks, part numbers and quantities for one client's
job, and the repository must never see it.
"""
from vibe import designspec
from vibe import takeoffreport as report
if folder is None:
folder = designspec.sub_dir(doc, 'takeoff')
if not os.path.isdir(folder):
os.makedirs(folder)
paths = {}
for suffix, text in (
('.md', report.markdown(takeoff, counts, extras=extras)),
('-totals.csv', report.csv_totals(takeoff)),
('-by-piece.csv', report.csv_by_piece(takeoff))):
path = os.path.join(folder, stem + suffix)
handle = open(path, 'wb')
try:
handle.write(text.encode('utf-8'))
finally:
handle.close()
paths[suffix] = path
return paths