will
25592fa6c2
ci / test (push) Successful in 6m14s
History-free snapshot of the private development repository at 77a82eb. Code, docs and the public catalogue only: no plant or job knowledge layers, no client, job or producer names (tools/scrub.py clean against the widened term list; a second word-level grep clean). The intro deck is left out because its slide images cannot be checked for job drawings.
365 lines
16 KiB
Python
365 lines
16 KiB
Python
# -*- coding: utf-8 -*-
|
|
"""The Revit half of the takeoff. Gathering only, and READ ONLY.
|
|
|
|
The split is the one `reinforceindex` / `reinforcecore` already models:
|
|
everything decidable without a document is in `takeoffcore`, `takeoffcount`
|
|
and `takeoffreport`, all three unit-tested on host CPython. This file does
|
|
the gathering and nothing else. There is no Transaction here, no parameter
|
|
write, no delete, and no reason for that to change -- a takeoff reports what
|
|
a model contains and has no business changing it.
|
|
|
|
WHAT IT ACTUALLY READS, AND WHAT IT COSTS
|
|
------------------------------------------
|
|
Almost nothing, because the expensive work is already cached. The takeoff's
|
|
input is `reinforceindex.read(doc)` -- the index as it stands, measured at
|
|
1.4 s with no rebuild -- plus one cheap pass over the assembly instances for
|
|
the multiplier. No view is generated, no geometry is solved, and none of the
|
|
30-60 s per piece this project was warned about is paid.
|
|
|
|
`refresh` is deliberately NOT called from here. A takeoff off a stale index
|
|
is a takeoff of the model as it was, which is a knowable and reportable
|
|
thing: `coverage.built_at` says when. A takeoff that silently spends six
|
|
seconds rebuilding is a button that sometimes freezes Revit and never says
|
|
why. Refreshing the index is the reinforcing tool's job and its button.
|
|
|
|
WHERE THE MULTIPLIER COMES FROM, AFTER IT WAS MEASURED AND WAS WRONG
|
|
--------------------------------------------------------------------
|
|
The first version of this file counted how many assembly instances share a
|
|
name, on the reasonable-sounding claim that three identical pieces of one
|
|
mark are three assemblies. It was written down as UNVERIFIED and it was
|
|
verified, through the bridge, and it was WRONG: 755 assembly instances, 755
|
|
distinct names, zero duplicates. That source returns 1 for every mark, so
|
|
every per-job figure silently equalled its per-mark figure -- a whole-job
|
|
undercount wearing the costume of a sourced number.
|
|
|
|
Control numbers do not live on assemblies at all. They live on STRUCTURAL
|
|
FRAMING elements, and there are far more of them than there are assemblies:
|
|
2480 control-numbered elements resolving to 1776 pieces, against 755
|
|
assemblies. Walls, floors, columns, foundations and generic models were swept
|
|
-- 17,239 elements -- and carry none.
|
|
|
|
So the multiplier now comes from `vibe.jobreconscan.gather`, the job
|
|
reconciliation's own gatherer, reusing its data contract rather than sweeping
|
|
the model a second time with a second opinion about what a piece is. The
|
|
takeoff and the reconciliation cannot disagree about how many pieces the job
|
|
has, because they are reading the same records through the same expander.
|
|
|
|
WHAT THAT COSTS, AND THE ONE NARROWING MADE HERE
|
|
-------------------------------------------------
|
|
`jobreconscan` sweeps six categories at about 6 s. This asks it for
|
|
STRUCTURAL FRAMING ONLY, on the measured evidence above that the other five
|
|
carry no control numbers -- so the takeoff's multiplier costs roughly a
|
|
second rather than six.
|
|
|
|
That is a deliberate narrowing of somebody else's careful default and it is
|
|
therefore reported, not assumed: `extras['multiplier']['categories']` names
|
|
what was searched and the report prints it. If a piece is ever modelled as a
|
|
wall, this run will not see it, and the honest place for that sentence is on
|
|
the page rather than in this comment. Pass `categories=None` to sweep all six.
|
|
|
|
WHAT ONE EXPORTED TICKET IS, AND WHERE THAT IS READ
|
|
----------------------------------------------------
|
|
`run()` reads `nomenclature.ticket_unit` out of the knowledge layers and
|
|
hands it to `takeoffcount.counting_plan`. It decides one thing and it decides
|
|
it for a good reason: the exported tickets are a count of PIECES only where
|
|
one ticket is one control number. A plant that puts a whole mark on one
|
|
ticket -- one drawing, a QUANITY field, several control numbers listed --
|
|
exports one filename per mark, and reading that as one piece per mark is the
|
|
`assembly_instances` undercount again.
|
|
|
|
Declared `piece_mark`, or not declared at all, the ticket source is WITHDRAWN
|
|
rather than demoted, and the ticket-against-model cross-check stands down
|
|
with a sentence. Undeclared is a refusal that names the key: this run cannot
|
|
tell which kind of folder it is looking at, and assuming the generous reading
|
|
would under-count while wearing a source's name.
|
|
|
|
A ROLE NOBODY HAS MAPPED IS A REFUSAL, NOT AN EMPTY JOB
|
|
--------------------------------------------------------
|
|
`member.control_number` is a knowledge role and it is not mapped in the plant
|
|
layer yet. Unmapped, this returns no multipliers at all -- and that must read
|
|
as "the takeoff could not find out", never as "every mark is one piece". The
|
|
refusal is carried out in `extras['multiplier']['refusal']` and the report
|
|
leads the multiplier section with it.
|
|
"""
|
|
# pylint: disable=import-error,broad-except,invalid-name
|
|
import os
|
|
|
|
from vibe import takeoffcore as core
|
|
from vibe import takeoffcount as count
|
|
|
|
# Every control number in this model sits on structural framing. Measured by
|
|
# the job reconciliation across six categories and 17,239 elements; the other
|
|
# five carry none. See the module docstring for why this narrowing is
|
|
# reported rather than assumed.
|
|
FRAMING_ONLY = ('OST_StructuralFraming',)
|
|
|
|
|
|
def population(doc):
|
|
"""How many assemblies the model actually has. The honest denominator.
|
|
|
|
Cheap: `reinforceindex.assemblies` is a scoped collector over assembly
|
|
instances with no geometry, measured at 0.29 s over 755 of them.
|
|
"""
|
|
from vibe import reinforceindex
|
|
pairs = reinforceindex.assemblies(doc)
|
|
names = [name for name, _el in pairs]
|
|
return {
|
|
'assemblies': len(set(names)),
|
|
'assembly_instances': len(names),
|
|
'names': names,
|
|
}
|
|
|
|
|
|
def assembly_multipliers(names):
|
|
"""{mark: (instances, 'assembly_instances')} -- duplicated names only.
|
|
|
|
Kept as a last-resort source and expected to return {} on this job: 755
|
|
assemblies, 755 distinct names. It can no longer emit a count of one, so
|
|
it cannot turn "no source" into a confident 1. See `takeoffcount`.
|
|
"""
|
|
return count.from_assembly_names(names)
|
|
|
|
|
|
def control_multipliers(doc, know=None, categories=FRAMING_ONLY):
|
|
"""({mark: (pieces, 'control_numbers')}, info) off the model's elements.
|
|
|
|
Delegates the sweep to `vibe.jobreconscan.gather`, which is the module
|
|
that established where control numbers live and how a control field is
|
|
expanded. Nothing here re-derives either.
|
|
|
|
`info` always says what happened, including when nothing did: an unmapped
|
|
role produces an empty mapping AND a `refusal` string, and the two are
|
|
not the same output as a job with no control numbers in it.
|
|
"""
|
|
from vibe import jobreconscan
|
|
from vibe import knowledge as K
|
|
|
|
info = {'categories': list(categories) if categories else None,
|
|
'refusal': None, 'unreadable': [], 'elements': 0, 'notes': []}
|
|
|
|
if not K.param_names(know, 'member.control_number'):
|
|
info['refusal'] = (
|
|
'The knowledge role `member.control_number` is not mapped, so the '
|
|
'model was not asked how many pieces each mark is. Every mark '
|
|
'below is counted ONCE and every per-job figure equals its '
|
|
'per-mark figure. This is a refusal, not a finding: map '
|
|
'`parameters.member.control_number` in the plant knowledge file '
|
|
'(the same role Job Recon needs) and press again.')
|
|
return {}, info
|
|
|
|
try:
|
|
pieces, _sheets, notes, _timing = jobreconscan.gather(
|
|
doc, know=know, categories=categories)
|
|
except Exception as exc:
|
|
info['refusal'] = ('The model could not be read for control numbers: '
|
|
'{0}. Nothing was multiplied.'.format(exc))
|
|
return {}, info
|
|
|
|
info['elements'] = len(pieces)
|
|
info['notes'] = list(notes or [])
|
|
multipliers, unreadable = count.from_piece_records(pieces)
|
|
info['unreadable'] = unreadable
|
|
return multipliers, info
|
|
|
|
|
|
def ticket_multipliers(folder):
|
|
"""{mark: (controls, 'ticket_files')} from a folder of exported tickets.
|
|
|
|
Ground truth for the marks it covers and silent about the rest. Reads
|
|
filenames only -- it never opens a PDF, and the folder is a job path that
|
|
belongs in knowledge, never in this file.
|
|
"""
|
|
if not folder or not os.path.isdir(folder):
|
|
return {}
|
|
try:
|
|
names = os.listdir(folder)
|
|
except Exception:
|
|
return {}
|
|
return count.from_ticket_files(names)
|
|
|
|
|
|
# -------------------------------------------------------- the declared scope
|
|
|
|
def _apply_scope(the_scope, cache, pop):
|
|
"""(cache, population, info) narrowed to the declared scope.
|
|
|
|
Pure dict work, no document. Returns COPIES -- the index cache is read
|
|
from disk and shared, and quietly editing it would leave a narrowed index
|
|
behind for the next caller who never asked for one.
|
|
|
|
`info` always carries the line, including when the scope is undeclared or
|
|
set nothing aside. A takeoff that counted everything must say so just as
|
|
loudly as one that did not, because the reader cannot otherwise tell a
|
|
complete count from a filtered one.
|
|
"""
|
|
pieces = (cache or {}).get('pieces') or {}
|
|
names = sorted(pieces.keys())
|
|
kept, left = the_scope.split_names(names)
|
|
pop_kept, pop_left = the_scope.split_names((pop or {}).get('names') or [])
|
|
|
|
info = {
|
|
'declared': the_scope.is_declared,
|
|
'label': the_scope.label(),
|
|
'declaration': the_scope.declaration_lines(),
|
|
'left_alone': the_scope.tally(left),
|
|
'left_alone_marks': len(left),
|
|
'counted_marks': len(kept),
|
|
'line': the_scope.line(left, total=len(names), unit='indexed mark'),
|
|
'assemblies_line': the_scope.line(
|
|
pop_left, total=len(pop_kept) + len(pop_left), unit='assembly')
|
|
if the_scope.is_declared else None,
|
|
}
|
|
if not the_scope.is_declared or not left:
|
|
return cache, pop, info
|
|
|
|
new_cache = dict(cache or {})
|
|
new_cache['pieces'] = dict((n, pieces[n]) for n in kept)
|
|
new_pop = dict(pop or {})
|
|
new_pop['names'] = pop_kept
|
|
new_pop['assemblies'] = len(set(pop_kept))
|
|
new_pop['assembly_instances'] = len(pop_kept)
|
|
return new_cache, new_pop, info
|
|
|
|
|
|
# ------------------------------------------------------------------- run it
|
|
|
|
def run(doc, know=None, ticket_folder=None):
|
|
"""The whole takeoff for the open model. READ ONLY.
|
|
|
|
Returns (takeoff, counts, extras) where `extras` carries the multiplier
|
|
sources separately so the report can show them disagreeing. A
|
|
disagreement between the exported tickets and the model is the only audit
|
|
the multiplier has, and it costs a directory listing.
|
|
"""
|
|
from vibe import reinforceindex
|
|
|
|
if know is None:
|
|
from vibe import knowledge as K
|
|
know = K.load(doc)
|
|
from vibe import knowledge as K
|
|
|
|
cfg = core.settings(know, K.get)
|
|
attached = core.attached_parents(cfg, know, K.get)
|
|
|
|
cache = reinforceindex.read(doc)
|
|
pop = population(doc)
|
|
|
|
# ---- the declared scope ---------------------------------------------
|
|
#
|
|
# A takeoff is a job document and a job has several engineers on it. What
|
|
# is counted here is what the declared scope says is this engineer's, and
|
|
# what was left out is COUNTED AND NAMED rather than dropped -- `extras`
|
|
# carries the line and the report prints it above the quantities. No
|
|
# declaration means everything is counted, which is the honest default.
|
|
#
|
|
# The index, the assembly population and the multipliers are all narrowed
|
|
# together. Narrowing one of them would leave the report claiming a hole
|
|
# in the job ('marks in the model but NOT indexed') that is really just
|
|
# somebody else's work.
|
|
from vibe import scope as scopelib
|
|
the_scope = scopelib.of(know, K.get)
|
|
cache, pop, scope_info = _apply_scope(the_scope, cache, pop)
|
|
|
|
if ticket_folder is None:
|
|
ticket_folder = K.get(know, 'takeoff.ticket_folder')
|
|
|
|
# ---- what one exported ticket is ------------------------------------
|
|
#
|
|
# DECLARED, not decided here: `nomenclature.ticket_unit`. One ticket per
|
|
# control number makes the exported tickets a piece count; one ticket per
|
|
# mark makes them a count of drawings, and reading a folder of the second
|
|
# kind as the first returns a confident 1 for a mark that is really
|
|
# seven. The read is done HERE, with the literal path, because this is
|
|
# the file that already holds a document and a knowledge module;
|
|
# `takeoffcount.counting_plan` owns everything that follows from the
|
|
# value and is testable on a string.
|
|
unit = K.get(know, 'nomenclature.ticket_unit')
|
|
|
|
from_control, multiplier_info = control_multipliers(doc, know)
|
|
from_assembly = assembly_multipliers(pop['names'])
|
|
from_tickets = ticket_multipliers(ticket_folder)
|
|
# Order matters and is the plan's, drawn from `takeoffcount.ORDER`:
|
|
# control numbers off the model, then the exported tickets IF this job
|
|
# tickets one piece at a time, then -- last, and empty on a job whose
|
|
# assembly names are all distinct -- duplicated assembly names. A source
|
|
# the plan withdrew is dropped, not demoted.
|
|
plan = count.counting_plan(unit, {
|
|
'control_numbers': from_control,
|
|
'ticket_files': from_tickets,
|
|
'assembly_instances': from_assembly})
|
|
multipliers = count.merge(from_control, from_tickets, from_assembly,
|
|
order=plan['order'])
|
|
if the_scope.is_declared:
|
|
multipliers = dict((m, v) for m, v in multipliers.items()
|
|
if the_scope.includes(m))
|
|
|
|
takeoff = core.build(
|
|
cache, cfg,
|
|
multipliers=multipliers,
|
|
attached=attached,
|
|
is_strand=lambda fam: K.is_strand(know, fam),
|
|
is_mesh=lambda fam: _is_mesh(K, know, fam),
|
|
is_cast=lambda fam: K.matches(
|
|
fam, K.get(know, 'families.cast_include', []), []),
|
|
population=pop)
|
|
|
|
marks = sorted((cache.get('pieces') or {}).keys())
|
|
counts = count.summarise(multipliers, marks)
|
|
counts['refusal'] = multiplier_info.get('refusal')
|
|
counts['join'] = count.join_quality(multipliers, marks)
|
|
counts['sources'] = {
|
|
'control_numbers': len(from_control),
|
|
'ticket_files': len(from_tickets),
|
|
'assembly_instances': len(from_assembly),
|
|
}
|
|
counts['ticket_unit'] = plan
|
|
extras = {
|
|
'scope': scope_info,
|
|
'multiplier': multiplier_info,
|
|
# The cross-check compares two counts of PIECES. Where the tickets
|
|
# are not a piece count -- declared `piece_mark`, or nobody has said
|
|
# -- every mark would "disagree" for a reason that is not a finding,
|
|
# so the comparison is stood down and the plan's sentence says so
|
|
# instead. A cross-check nobody may believe is worse than none.
|
|
'disagreements': (count.disagreements(from_control, from_tickets)
|
|
if 'ticket_files' in plan['order'] else []),
|
|
'index_path': reinforceindex.index_path(doc),
|
|
}
|
|
return takeoff, counts, extras
|
|
|
|
|
|
def _is_mesh(K, know, family):
|
|
prefix = K.get(know, 'families.wwf_prefix', '')
|
|
return bool(prefix) and (family or '').upper().startswith(
|
|
str(prefix).upper())
|
|
|
|
|
|
def write_reports(doc, takeoff, counts, folder=None, stem='takeoff',
|
|
extras=None):
|
|
"""Write the markdown and both CSVs beside the model. Returns the paths.
|
|
|
|
Beside the MODEL, under the same `_ticketqc` root the index and the
|
|
ticket evidence already use -- never into the repository. A takeoff is
|
|
job data: it names marks, part numbers and quantities for one client's
|
|
job, and the repository must never see it.
|
|
"""
|
|
from vibe import designspec
|
|
from vibe import takeoffreport as report
|
|
if folder is None:
|
|
folder = designspec.sub_dir(doc, 'takeoff')
|
|
if not os.path.isdir(folder):
|
|
os.makedirs(folder)
|
|
paths = {}
|
|
for suffix, text in (
|
|
('.md', report.markdown(takeoff, counts, extras=extras)),
|
|
('-totals.csv', report.csv_totals(takeoff)),
|
|
('-by-piece.csv', report.csv_by_piece(takeoff))):
|
|
path = os.path.join(folder, stem + suffix)
|
|
handle = open(path, 'wb')
|
|
try:
|
|
handle.write(text.encode('utf-8'))
|
|
finally:
|
|
handle.close()
|
|
paths[suffix] = path
|
|
return paths
|