# -*- coding: utf-8 -*- """The DECLARED SCOPE: which pieces of a job this engineer is answerable for. WHY THIS EXISTS --------------- Several engineers work one job. The owner of this tool details the wall panels; the beams, the slabs and the columns belong to other people. The plant-level knowledge every check in here runs on -- naming, hardware, bed layout, tolerances -- was validated against the panels and NOT against the rest, and the owner cannot validate it against the rest, because he does not hold the job knowledge for somebody else's pieces. So a tool that judges all of them is not being thorough. It is producing verdicts on work it has no basis to judge, in the same report and the same typeface as the verdicts it can stand behind. WHY THIS IS NOT AN IGNORE LIST ------------------------------ An exclusion that silently drops things is the exact failure this project has already paid for: one unauthorised exclusion hid forty tickets and every check that ran came back clean. Nothing here may repeat it, so: * OUT OF SCOPE IS A THIRD STATE. Not a pass, not a failure, not an absence. Every report that leaves work alone says so, in a line, naming how much and of what kind. * THE SCOPE IS DECLARED, NEVER INFERRED. It is read from the JOB knowledge layer -- `scope.mark_prefixes` -- because who is responsible for what is a fact about this job and this office, not about the plant and certainly not about the code. No prefix is written into this file. * NO DECLARATION MEANS JUDGE EVERYTHING. An undeclared scope is not an empty scope. `Scope()` includes every piece, and its report line says that nothing was set aside AND that the knowledge behind those verdicts may never have been validated outside the usual work. * A PIECE WHOSE KIND CANNOT BE READ IS NEVER SET ASIDE. If a mark carries no readable letter prefix, the tool does not know whether it is in scope, and "I could not tell" may only ever resolve towards judging it. Those are counted and named on the same line. * A NEAR MISS IS REPORTED, NEVER RESOLVED. See below. THE NEAR MISS, AND WHY THE MATCH IS NOT WIDENED ------------------------------------------------ `prefix_of` reads the WHOLE leading letter run, so a declaration of `AAA` does not reach a mark `AAAB001` -- its prefix is `AAAB`, which is not `AAA`, and the piece is set aside as somebody else's. That is a real hazard: on a job whose marks carry a suffix letter for a variant, a genuine piece of this engineer's is quietly handed to another engineer. The obvious repair -- match a declared prefix as a PREFIX, so `AAA` reaches `AAAB` -- is not made here, and deliberately: * It would also reach every OTHER letter run that starts the same way, pulling pieces that really are somebody else's INTO the checked set. * A false in-scope is worse than a false out-of-scope. Out of scope is reported, in a line, by kind. In scope is judged silently against knowledge that was never validated for it -- which is the exact failure this module exists to prevent. * Which of the two a given job means is a JOB FACT. It belongs in the job file, declared by a person, not in a rule this code guessed. So the behaviour is unchanged and the near miss is REPORTED: a set-aside kind that is a prefix of a declared one, or that a declared one is a prefix of, is named on the report with both spellings and the sentence saying the tool did not guess. A human widens the declaration -- by adding the longer prefix to `scope.mark_prefixes` -- or leaves it alone. Nothing here decides. WHAT THE SCOPE IS EXPRESSED IN, AND WHY --------------------------------------- The MARK PREFIX -- the letters a piece mark starts with. Reasons, in order: * Every entry point already has a mark, or an assembly name that is the mark, in its hand. Sheets are named for their mark too. One rule reaches all of them. * The prefix is the ONE stable classifier. A design number moves -- new designs are added to a live job weekly, and a scope listed as design numbers goes stale silently, which is the failure mode this file exists to prevent. * It is what the reinforcing donor filter already discriminated on, so adopting it is not a new concept in this codebase, it is the existing one lifted out of the code and into the job file. A job whose marks do NOT encode the discipline in their prefix cannot use this key, and must not be given a guessed answer: it declares nothing, gets "no scope declared", and every report says every piece was judged. Adding a second expression -- design numbers, a category, an explicit mark list -- is a new key beside this one, never a reinterpretation of this one. IronPython 2.7: no f-strings, no pathlib, no typing, no dataclasses. """ import re from vibe.compat import string_types # Where the declaration lives. JOB layer -- see the module docstring. KEY = 'scope.mark_prefixes' NOTE_KEY = 'scope.note' BY_KEY = 'scope.declared_by' # The leading letters of a mark. Deliberately letters-only: a mark is # at every plant seen so far, and anything else is # 'could not read', which is a reported state rather than a guess. _PREFIX_RE = re.compile(r'^([A-Za-z]+)') # Letters followed by a number: the shape of a piece mark or a design number. # Used where a caller holds a name that MIGHT be a mark -- a sheet name, say # -- and must not treat 'the general arrangement drawing' as somebody else's # piece. Anything that is not this shape is left alone by the scope entirely. _MARK_RE = re.compile(r'^[A-Za-z]+\s*\d') # What a piece with no readable prefix is called on a report line. UNREADABLE = '(no readable mark prefix)' def prefix_of(name): """The leading letters of a mark, upper case, or None if there are none.""" if not isinstance(name, string_types): name = '' if name is None else str(name) m = _PREFIX_RE.match(name.strip()) return m.group(1).upper() if m else None def looks_like_mark(name): """True when this text has the shape of a piece mark or design number. A caller holding something that is only SOMETIMES a mark (a sheet name) passes names through this first, so a drawing that is not a piece is never counted as another engineer's piece. """ if not isinstance(name, string_types): name = '' if name is None else str(name) return bool(_MARK_RE.match(name.strip())) def as_mark(name): """The name if it reads as a mark, else '' -- which every predicate here treats as 'could not tell', i.e. left in.""" return name if looks_like_mark(name) else '' class Scope(object): """The declared scope as a predicate. Construct with no argument for 'nobody has declared one', which includes everything.""" def __init__(self, prefixes=(), note=None, declared_by=None, source='job knowledge: ' + KEY): cleaned = [] for p in (prefixes or ()): text = ('' if p is None else str(p)).strip().upper() if text and text not in cleaned: cleaned.append(text) self.prefixes = tuple(cleaned) self.note = note or None self.declared_by = declared_by or None self.source = source # -------------------------------------------------------------- state @property def is_declared(self): return bool(self.prefixes) def label(self): """What the scope IS, in words a reader can check against reality.""" if not self.is_declared: return 'no scope declared' return 'piece marks beginning {0}'.format( ' or '.join(self.prefixes)) # ---------------------------------------------------------- predicate def includes(self, name): """True when this mark is the engineer's to judge. True for everything when no scope is declared, and true for a mark whose prefix cannot be read -- 'I could not tell' never removes work. """ if not self.is_declared: return True prefix = prefix_of(name) if prefix is None: return True return prefix in self.prefixes def is_unreadable(self, name): return prefix_of(name) is None def kind(self, name): """The group a set-aside item is reported under.""" return prefix_of(name) or UNREADABLE # ------------------------------------------------------------ splits def split(self, items, key=None): """(inside, outside) preserving order. `key` reads the mark.""" key = key or (lambda x: x) inside, outside = [], [] for item in (items or []): (inside if self.includes(key(item)) else outside).append(item) return inside, outside def split_names(self, names): return self.split(list(names or [])) def tally(self, names): """[(kind, count)] for the things set aside, biggest first.""" counts = {} for name in (names or []): kind = self.kind(name) counts[kind] = counts.get(kind, 0) + 1 return sorted(counts.items(), key=lambda kv: (-kv[1], kv[0])) # -------------------------------------------------------- the near miss def nearly(self, kind): """The declared prefix `kind` NEARLY is, or None. 'Nearly' is deliberately narrow: one of the two spellings is a prefix of the other. `AAAB` nearly matches a declared `AAA`, and a bare `AAA` nearly matches a declared `AAAB` -- either way somebody wrote one and meant, perhaps, the other. `BBB` nearly matches nothing. An exact match is not a near miss, it is a match. """ if not self.is_declared or not kind or kind == UNREADABLE: return None for declared in self.prefixes: if kind == declared: return None if kind.startswith(declared) or declared.startswith(kind): return declared return None def near_misses(self, names): """[(kind, count, declared)] for set-aside kinds that nearly match. Biggest first. Empty when the scope is undeclared, when nothing was set aside, or when nothing set aside resembles the declaration -- which is the ordinary case and prints nothing. """ out = [] for kind, count in self.tally(names): declared = self.nearly(kind) if declared: out.append((kind, count, declared)) return out def near_miss_line(self, outside_names, unit='piece'): """The sentence a human has to read, or None. Says both spellings and says the tool did not choose between them. Never a verdict: a near miss is a question for the person who holds the job knowledge, and the only two answers -- widen the declaration or leave it -- are both his. """ misses = self.near_misses(outside_names) if not misses: return None bits = ', '.join( '{0} {1}(s) with mark prefix {2} (the declaration says {3})' .format(count, unit, kind, declared) for kind, count, declared in misses) return ('NEAR MISS -- SOMEBODY SHOULD LOOK: {0} were set aside. A ' 'prefix that only NEARLY matches is either work of yours the ' 'declaration does not reach, or somebody else\'s work that ' 'happens to start the same way, and this tool will not guess ' 'which -- guessing wrong towards "yours" would judge another ' 'engineer\'s piece silently. If they are yours, add the ' 'longer prefix to the declaration ({1}); if they are not, ' 'nothing needs doing and this line is the record that it was ' 'looked at.'.format(bits, self.source)) # ------------------------------------------------------------ report def line(self, outside_names, total=None, unit='piece', unreadable_names=None): """ONE line saying what was left alone and why. Never a pass. `outside_names` are the marks that were NOT judged. `total` is how many there were altogether, so the reader can see the proportion; omit it and the line simply does not claim one. A NEAR MISS RIDES ON THIS LINE rather than on a line of its own, so that every consumer of `line()` reports it without having to know it exists. A report that wants it on a shut fold's summary as well asks for `near_miss_line()` directly. """ outside = list(outside_names or []) unreadable = list(unreadable_names or []) tail = '' if unreadable: tail = (' {0} {1}(s) carry no readable mark prefix; they could ' 'not be placed inside or outside the scope and were ' 'judged rather than set aside.' .format(len(unreadable), unit)) near = self.near_miss_line(outside, unit=unit) if near: tail += ' ' + near if not self.is_declared: return ('NOT JUDGED, OUT OF SCOPE: none set aside. No scope is ' 'declared for this job ({0} is empty), so every {1} was ' 'judged -- including work this plant knowledge has never ' 'been validated against. Declare a scope to be told what ' 'is being taken on trust.'.format(self.source, unit) + tail) if not outside: # Deliberately does NOT say 'everything here was in scope'. The # user may have selected an out-of-scope piece himself, which is # judged and annotated rather than set aside; this line's job is # only to say that nothing was removed from the run. return ('NOT JUDGED, OUT OF SCOPE: none -- nothing was set aside ' 'from this run. The declared scope is {0} ({1}).' .format(self.label(), self.source) + tail) kinds = ', '.join('{0} {1}'.format(k, n) for k, n in self.tally(outside)) of_total = ' of {0}'.format(total) if total else '' return ('NOT JUDGED, OUT OF SCOPE: {0}{1} {2}(s) were left alone -- ' '{3}. The declared scope is {4} ({5}); everything else ' 'belongs to another engineer and the knowledge this tool ' 'runs on is not validated for it. Not judged is not clean, ' 'and not a failure either.' .format(len(outside), of_total, unit, kinds, self.label(), self.source) + tail) def short_line(self, outside_names, total=None, unit='piece'): """The same three states in ONE SHORT CLAUSE, for a console. `line()` is the sentence; this is the headline. It exists because some consumers truncate -- `tools/sweep.py` prints the first 200 characters of what the in-Revit half returns -- and a scope line that gets cut off the end of a progress line is a scope line nobody read. It says the COUNT, which is what tells a reader there is a full sentence somewhere worth opening. It never replaces `line()`. Anything printing this must also write `line()` where the reader can reach it. """ outside = list(outside_names or []) if not self.is_declared: return ('SCOPE: none declared -- every {0} was judged' .format(unit)) of_total = ' of {0}'.format(total) if total else '' if not outside: return ('SCOPE {0}: no {1}{2} was set aside' .format(self.label(), unit, of_total)) kinds = self.tally(outside) shown = ', '.join('{0} {1}'.format(k, n) for k, n in kinds[:3]) if len(kinds) > 3: shown += ', +{0} more'.format(len(kinds) - 3) text = ('NOT JUDGED, OUT OF SCOPE: {0}{1} {2}(s) left alone [{3}]' .format(len(outside), of_total, unit, shown)) if self.near_misses(outside): text += ' -- NEAR MISS' return text def annotation(self, outside_names, unit='piece'): """The line for work the USER picked that falls outside the scope. Nothing the user selected is ever dropped -- see `honoured_selection` in the design notes. It is judged, and this says the verdict rests on knowledge nobody has validated for that kind of piece. """ outside = list(outside_names or []) if not self.is_declared or not outside: return None kinds = ', '.join('{0} {1}'.format(k, n) for k, n in self.tally(outside)) # A near miss matters MORE here than on `line()`: the reader is being # told his own selection rests on unvalidated knowledge, and if the # prefix only nearly missed, that warning may be the wrong one. near = self.near_miss_line(outside, unit=unit) return ('JUDGED OUTSIDE THE DECLARED SCOPE, AT YOUR REQUEST: {0} ' '{1}(s) -- {2}. You selected them, so nothing was dropped; ' 'but the declared scope is {3}, and these verdicts rest on ' 'knowledge that has not been validated for those pieces. ' 'Read them as opinions, not as checks.' .format(len(outside), unit, kinds, self.label()) + ((' ' + near) if near else '')) def declaration_lines(self): """Who declared it and what they said, when they said anything.""" out = [] if self.declared_by: out.append('scope declared by {0}'.format(self.declared_by)) if self.note: # A YAML folded scalar (`note: >`) arrives with a trailing # newline, which puts a blank line into every report that prints # it. Stripped here rather than at four call sites. out.append(' '.join(str(self.note).split())) return out def _as_list(value): if value is None: return [] if isinstance(value, string_types): return [value] return list(value) def of(know, get=None): """The declared scope out of a loaded knowledge dict. `get` is `knowledge.get`. It is a parameter so a test can pass its own and this module never has to import `knowledge` at module scope; the ordinary path imports it lazily, and reads the three keys through the alias `K` with LITERAL paths so `tools/knowledge_drift.py` can see that they are genuinely read. A key nobody can prove is read is a key the file only claims to honour. """ if not know: return Scope() if get is None: from vibe import knowledge as K return Scope(prefixes=_as_list(K.get(know, 'scope.mark_prefixes', None)), note=K.get(know, 'scope.note', None), declared_by=K.get(know, 'scope.declared_by', None)) return Scope(prefixes=_as_list(get(know, 'scope.mark_prefixes', None)), note=get(know, 'scope.note', None), declared_by=get(know, 'scope.declared_by', None))