# -*- coding: utf-8 -*- """The Revit half of the takeoff. Gathering only, and READ ONLY. The split is the one `reinforceindex` / `reinforcecore` already models: everything decidable without a document is in `takeoffcore`, `takeoffcount` and `takeoffreport`, all three unit-tested on host CPython. This file does the gathering and nothing else. There is no Transaction here, no parameter write, no delete, and no reason for that to change -- a takeoff reports what a model contains and has no business changing it. WHAT IT ACTUALLY READS, AND WHAT IT COSTS ------------------------------------------ Almost nothing, because the expensive work is already cached. The takeoff's input is `reinforceindex.read(doc)` -- the index as it stands, measured at 1.4 s with no rebuild -- plus one cheap pass over the assembly instances for the multiplier. No view is generated, no geometry is solved, and none of the 30-60 s per piece this project was warned about is paid. `refresh` is deliberately NOT called from here. A takeoff off a stale index is a takeoff of the model as it was, which is a knowable and reportable thing: `coverage.built_at` says when. A takeoff that silently spends six seconds rebuilding is a button that sometimes freezes Revit and never says why. Refreshing the index is the reinforcing tool's job and its button. WHERE THE MULTIPLIER COMES FROM, AFTER IT WAS MEASURED AND WAS WRONG -------------------------------------------------------------------- The first version of this file counted how many assembly instances share a name, on the reasonable-sounding claim that three identical pieces of one mark are three assemblies. It was written down as UNVERIFIED and it was verified, through the bridge, and it was WRONG: 755 assembly instances, 755 distinct names, zero duplicates. That source returns 1 for every mark, so every per-job figure silently equalled its per-mark figure -- a whole-job undercount wearing the costume of a sourced number. Control numbers do not live on assemblies at all. They live on STRUCTURAL FRAMING elements, and there are far more of them than there are assemblies: 2480 control-numbered elements resolving to 1776 pieces, against 755 assemblies. Walls, floors, columns, foundations and generic models were swept -- 17,239 elements -- and carry none. So the multiplier now comes from `vibe.jobreconscan.gather`, the job reconciliation's own gatherer, reusing its data contract rather than sweeping the model a second time with a second opinion about what a piece is. The takeoff and the reconciliation cannot disagree about how many pieces the job has, because they are reading the same records through the same expander. WHAT THAT COSTS, AND THE ONE NARROWING MADE HERE ------------------------------------------------- `jobreconscan` sweeps six categories at about 6 s. This asks it for STRUCTURAL FRAMING ONLY, on the measured evidence above that the other five carry no control numbers -- so the takeoff's multiplier costs roughly a second rather than six. That is a deliberate narrowing of somebody else's careful default and it is therefore reported, not assumed: `extras['multiplier']['categories']` names what was searched and the report prints it. If a piece is ever modelled as a wall, this run will not see it, and the honest place for that sentence is on the page rather than in this comment. Pass `categories=None` to sweep all six. WHAT ONE EXPORTED TICKET IS, AND WHERE THAT IS READ ---------------------------------------------------- `run()` reads `nomenclature.ticket_unit` out of the knowledge layers and hands it to `takeoffcount.counting_plan`. It decides one thing and it decides it for a good reason: the exported tickets are a count of PIECES only where one ticket is one control number. A plant that puts a whole mark on one ticket -- one drawing, a QUANITY field, several control numbers listed -- exports one filename per mark, and reading that as one piece per mark is the `assembly_instances` undercount again. Declared `piece_mark`, or not declared at all, the ticket source is WITHDRAWN rather than demoted, and the ticket-against-model cross-check stands down with a sentence. Undeclared is a refusal that names the key: this run cannot tell which kind of folder it is looking at, and assuming the generous reading would under-count while wearing a source's name. A ROLE NOBODY HAS MAPPED IS A REFUSAL, NOT AN EMPTY JOB -------------------------------------------------------- `member.control_number` is a knowledge role and it is not mapped in the plant layer yet. Unmapped, this returns no multipliers at all -- and that must read as "the takeoff could not find out", never as "every mark is one piece". The refusal is carried out in `extras['multiplier']['refusal']` and the report leads the multiplier section with it. """ # pylint: disable=import-error,broad-except,invalid-name import os from vibe import takeoffcore as core from vibe import takeoffcount as count # Every control number in this model sits on structural framing. Measured by # the job reconciliation across six categories and 17,239 elements; the other # five carry none. See the module docstring for why this narrowing is # reported rather than assumed. FRAMING_ONLY = ('OST_StructuralFraming',) def population(doc): """How many assemblies the model actually has. The honest denominator. Cheap: `reinforceindex.assemblies` is a scoped collector over assembly instances with no geometry, measured at 0.29 s over 755 of them. """ from vibe import reinforceindex pairs = reinforceindex.assemblies(doc) names = [name for name, _el in pairs] return { 'assemblies': len(set(names)), 'assembly_instances': len(names), 'names': names, } def assembly_multipliers(names): """{mark: (instances, 'assembly_instances')} -- duplicated names only. Kept as a last-resort source and expected to return {} on this job: 755 assemblies, 755 distinct names. It can no longer emit a count of one, so it cannot turn "no source" into a confident 1. See `takeoffcount`. """ return count.from_assembly_names(names) def control_multipliers(doc, know=None, categories=FRAMING_ONLY): """({mark: (pieces, 'control_numbers')}, info) off the model's elements. Delegates the sweep to `vibe.jobreconscan.gather`, which is the module that established where control numbers live and how a control field is expanded. Nothing here re-derives either. `info` always says what happened, including when nothing did: an unmapped role produces an empty mapping AND a `refusal` string, and the two are not the same output as a job with no control numbers in it. """ from vibe import jobreconscan from vibe import knowledge as K info = {'categories': list(categories) if categories else None, 'refusal': None, 'unreadable': [], 'elements': 0, 'notes': []} if not K.param_names(know, 'member.control_number'): info['refusal'] = ( 'The knowledge role `member.control_number` is not mapped, so the ' 'model was not asked how many pieces each mark is. Every mark ' 'below is counted ONCE and every per-job figure equals its ' 'per-mark figure. This is a refusal, not a finding: map ' '`parameters.member.control_number` in the plant knowledge file ' '(the same role Job Recon needs) and press again.') return {}, info try: pieces, _sheets, notes, _timing = jobreconscan.gather( doc, know=know, categories=categories) except Exception as exc: info['refusal'] = ('The model could not be read for control numbers: ' '{0}. Nothing was multiplied.'.format(exc)) return {}, info info['elements'] = len(pieces) info['notes'] = list(notes or []) multipliers, unreadable = count.from_piece_records(pieces) info['unreadable'] = unreadable return multipliers, info def ticket_multipliers(folder): """{mark: (controls, 'ticket_files')} from a folder of exported tickets. Ground truth for the marks it covers and silent about the rest. Reads filenames only -- it never opens a PDF, and the folder is a job path that belongs in knowledge, never in this file. """ if not folder or not os.path.isdir(folder): return {} try: names = os.listdir(folder) except Exception: return {} return count.from_ticket_files(names) # -------------------------------------------------------- the declared scope def _apply_scope(the_scope, cache, pop): """(cache, population, info) narrowed to the declared scope. Pure dict work, no document. Returns COPIES -- the index cache is read from disk and shared, and quietly editing it would leave a narrowed index behind for the next caller who never asked for one. `info` always carries the line, including when the scope is undeclared or set nothing aside. A takeoff that counted everything must say so just as loudly as one that did not, because the reader cannot otherwise tell a complete count from a filtered one. """ pieces = (cache or {}).get('pieces') or {} names = sorted(pieces.keys()) kept, left = the_scope.split_names(names) pop_kept, pop_left = the_scope.split_names((pop or {}).get('names') or []) info = { 'declared': the_scope.is_declared, 'label': the_scope.label(), 'declaration': the_scope.declaration_lines(), 'left_alone': the_scope.tally(left), 'left_alone_marks': len(left), 'counted_marks': len(kept), 'line': the_scope.line(left, total=len(names), unit='indexed mark'), 'assemblies_line': the_scope.line( pop_left, total=len(pop_kept) + len(pop_left), unit='assembly') if the_scope.is_declared else None, } if not the_scope.is_declared or not left: return cache, pop, info new_cache = dict(cache or {}) new_cache['pieces'] = dict((n, pieces[n]) for n in kept) new_pop = dict(pop or {}) new_pop['names'] = pop_kept new_pop['assemblies'] = len(set(pop_kept)) new_pop['assembly_instances'] = len(pop_kept) return new_cache, new_pop, info # ------------------------------------------------------------------- run it def run(doc, know=None, ticket_folder=None): """The whole takeoff for the open model. READ ONLY. Returns (takeoff, counts, extras) where `extras` carries the multiplier sources separately so the report can show them disagreeing. A disagreement between the exported tickets and the model is the only audit the multiplier has, and it costs a directory listing. """ from vibe import reinforceindex if know is None: from vibe import knowledge as K know = K.load(doc) from vibe import knowledge as K cfg = core.settings(know, K.get) attached = core.attached_parents(cfg, know, K.get) cache = reinforceindex.read(doc) pop = population(doc) # ---- the declared scope --------------------------------------------- # # A takeoff is a job document and a job has several engineers on it. What # is counted here is what the declared scope says is this engineer's, and # what was left out is COUNTED AND NAMED rather than dropped -- `extras` # carries the line and the report prints it above the quantities. No # declaration means everything is counted, which is the honest default. # # The index, the assembly population and the multipliers are all narrowed # together. Narrowing one of them would leave the report claiming a hole # in the job ('marks in the model but NOT indexed') that is really just # somebody else's work. from vibe import scope as scopelib the_scope = scopelib.of(know, K.get) cache, pop, scope_info = _apply_scope(the_scope, cache, pop) if ticket_folder is None: ticket_folder = K.get(know, 'takeoff.ticket_folder') # ---- what one exported ticket is ------------------------------------ # # DECLARED, not decided here: `nomenclature.ticket_unit`. One ticket per # control number makes the exported tickets a piece count; one ticket per # mark makes them a count of drawings, and reading a folder of the second # kind as the first returns a confident 1 for a mark that is really # seven. The read is done HERE, with the literal path, because this is # the file that already holds a document and a knowledge module; # `takeoffcount.counting_plan` owns everything that follows from the # value and is testable on a string. unit = K.get(know, 'nomenclature.ticket_unit') from_control, multiplier_info = control_multipliers(doc, know) from_assembly = assembly_multipliers(pop['names']) from_tickets = ticket_multipliers(ticket_folder) # Order matters and is the plan's, drawn from `takeoffcount.ORDER`: # control numbers off the model, then the exported tickets IF this job # tickets one piece at a time, then -- last, and empty on a job whose # assembly names are all distinct -- duplicated assembly names. A source # the plan withdrew is dropped, not demoted. plan = count.counting_plan(unit, { 'control_numbers': from_control, 'ticket_files': from_tickets, 'assembly_instances': from_assembly}) multipliers = count.merge(from_control, from_tickets, from_assembly, order=plan['order']) if the_scope.is_declared: multipliers = dict((m, v) for m, v in multipliers.items() if the_scope.includes(m)) takeoff = core.build( cache, cfg, multipliers=multipliers, attached=attached, is_strand=lambda fam: K.is_strand(know, fam), is_mesh=lambda fam: _is_mesh(K, know, fam), is_cast=lambda fam: K.matches( fam, K.get(know, 'families.cast_include', []), []), population=pop) marks = sorted((cache.get('pieces') or {}).keys()) counts = count.summarise(multipliers, marks) counts['refusal'] = multiplier_info.get('refusal') counts['join'] = count.join_quality(multipliers, marks) counts['sources'] = { 'control_numbers': len(from_control), 'ticket_files': len(from_tickets), 'assembly_instances': len(from_assembly), } counts['ticket_unit'] = plan extras = { 'scope': scope_info, 'multiplier': multiplier_info, # The cross-check compares two counts of PIECES. Where the tickets # are not a piece count -- declared `piece_mark`, or nobody has said # -- every mark would "disagree" for a reason that is not a finding, # so the comparison is stood down and the plan's sentence says so # instead. A cross-check nobody may believe is worse than none. 'disagreements': (count.disagreements(from_control, from_tickets) if 'ticket_files' in plan['order'] else []), 'index_path': reinforceindex.index_path(doc), } return takeoff, counts, extras def _is_mesh(K, know, family): prefix = K.get(know, 'families.wwf_prefix', '') return bool(prefix) and (family or '').upper().startswith( str(prefix).upper()) def write_reports(doc, takeoff, counts, folder=None, stem='takeoff', extras=None): """Write the markdown and both CSVs beside the model. Returns the paths. Beside the MODEL, under the same `_ticketqc` root the index and the ticket evidence already use -- never into the repository. A takeoff is job data: it names marks, part numbers and quantities for one client's job, and the repository must never see it. """ from vibe import designspec from vibe import takeoffreport as report if folder is None: folder = designspec.sub_dir(doc, 'takeoff') if not os.path.isdir(folder): os.makedirs(folder) paths = {} for suffix, text in ( ('.md', report.markdown(takeoff, counts, extras=extras)), ('-totals.csv', report.csv_totals(takeoff)), ('-by-piece.csv', report.csv_by_piece(takeoff))): path = os.path.join(folder, stem + suffix) handle = open(path, 'wb') try: handle.write(text.encode('utf-8')) finally: handle.close() paths[suffix] = path return paths