will
52d94303bc
ci / test (push) Has been cancelled
Public demo snapshot of an in-house pyRevit extension for precast concrete shop-ticket QC (37 checks / 46 checklist rows) and a standalone panel handling solver (PCI Chapter 8). - No client, job, plant or product data: knowledge layers ship as structural templates only. - Ticket QC's model-read escalation targets a LAN model server endpoint (configurable), not a cloud provider. - Built from the private development repo via tools/make_dist.py.
423 lines
19 KiB
Python
423 lines
19 KiB
Python
# -*- coding: utf-8 -*-
|
|
"""The DECLARED SCOPE: which pieces of a job this engineer is answerable for.
|
|
|
|
WHY THIS EXISTS
|
|
---------------
|
|
Several engineers work one job. The owner of this tool details the wall
|
|
panels; the beams, the slabs and the columns belong to other people. The
|
|
plant-level knowledge every check in here runs on -- naming, hardware,
|
|
bed layout, tolerances -- was validated against the panels and NOT against
|
|
the rest, and the owner cannot validate it against the rest, because he does
|
|
not hold the job knowledge for somebody else's pieces.
|
|
|
|
So a tool that judges all of them is not being thorough. It is producing
|
|
verdicts on work it has no basis to judge, in the same report and the same
|
|
typeface as the verdicts it can stand behind.
|
|
|
|
WHY THIS IS NOT AN IGNORE LIST
|
|
------------------------------
|
|
An exclusion that silently drops things is the exact failure this project has
|
|
already paid for: one unauthorised exclusion hid forty tickets and every
|
|
check that ran came back clean. Nothing here may repeat it, so:
|
|
|
|
* OUT OF SCOPE IS A THIRD STATE. Not a pass, not a failure, not an absence.
|
|
Every report that leaves work alone says so, in a line, naming how much
|
|
and of what kind.
|
|
|
|
* THE SCOPE IS DECLARED, NEVER INFERRED. It is read from the JOB knowledge
|
|
layer -- `scope.mark_prefixes` -- because who is responsible for what is
|
|
a fact about this job and this office, not about the plant and certainly
|
|
not about the code. No prefix is written into this file.
|
|
|
|
* NO DECLARATION MEANS JUDGE EVERYTHING. An undeclared scope is not an
|
|
empty scope. `Scope()` includes every piece, and its report line says
|
|
that nothing was set aside AND that the knowledge behind those verdicts
|
|
may never have been validated outside the usual work.
|
|
|
|
* A PIECE WHOSE KIND CANNOT BE READ IS NEVER SET ASIDE. If a mark carries
|
|
no readable letter prefix, the tool does not know whether it is in scope,
|
|
and "I could not tell" may only ever resolve towards judging it. Those
|
|
are counted and named on the same line.
|
|
|
|
* A NEAR MISS IS REPORTED, NEVER RESOLVED. See below.
|
|
|
|
THE NEAR MISS, AND WHY THE MATCH IS NOT WIDENED
|
|
------------------------------------------------
|
|
`prefix_of` reads the WHOLE leading letter run, so a declaration of `AAA`
|
|
does not reach a mark `AAAB001` -- its prefix is `AAAB`, which is not `AAA`,
|
|
and the piece is set aside as somebody else's. That is a real hazard: on a
|
|
job whose marks carry a suffix letter for a variant, a genuine piece of this
|
|
engineer's is quietly handed to another engineer.
|
|
|
|
The obvious repair -- match a declared prefix as a PREFIX, so `AAA` reaches
|
|
`AAAB` -- is not made here, and deliberately:
|
|
|
|
* It would also reach every OTHER letter run that starts the same way,
|
|
pulling pieces that really are somebody else's INTO the checked set.
|
|
* A false in-scope is worse than a false out-of-scope. Out of scope is
|
|
reported, in a line, by kind. In scope is judged silently against
|
|
knowledge that was never validated for it -- which is the exact failure
|
|
this module exists to prevent.
|
|
* Which of the two a given job means is a JOB FACT. It belongs in the job
|
|
file, declared by a person, not in a rule this code guessed.
|
|
|
|
So the behaviour is unchanged and the near miss is REPORTED: a set-aside
|
|
kind that is a prefix of a declared one, or that a declared one is a prefix
|
|
of, is named on the report with both spellings and the sentence saying the
|
|
tool did not guess. A human widens the declaration -- by adding the longer
|
|
prefix to `scope.mark_prefixes` -- or leaves it alone. Nothing here decides.
|
|
|
|
WHAT THE SCOPE IS EXPRESSED IN, AND WHY
|
|
---------------------------------------
|
|
The MARK PREFIX -- the letters a piece mark starts with. Reasons, in order:
|
|
|
|
* Every entry point already has a mark, or an assembly name that is the
|
|
mark, in its hand. Sheets are named for their mark too. One rule reaches
|
|
all of them.
|
|
* The prefix is the ONE stable classifier. A design number moves -- new
|
|
designs are added to a live job weekly, and a scope listed as design
|
|
numbers goes stale silently, which is the failure mode this file exists
|
|
to prevent.
|
|
* It is what the reinforcing donor filter already discriminated on, so
|
|
adopting it is not a new concept in this codebase, it is the existing
|
|
one lifted out of the code and into the job file.
|
|
|
|
A job whose marks do NOT encode the discipline in their prefix cannot use
|
|
this key, and must not be given a guessed answer: it declares nothing, gets
|
|
"no scope declared", and every report says every piece was judged. Adding a
|
|
second expression -- design numbers, a category, an explicit mark list -- is
|
|
a new key beside this one, never a reinterpretation of this one.
|
|
|
|
IronPython 2.7: no f-strings, no pathlib, no typing, no dataclasses.
|
|
"""
|
|
import re
|
|
|
|
from vibe.compat import string_types
|
|
|
|
# Where the declaration lives. JOB layer -- see the module docstring.
|
|
KEY = 'scope.mark_prefixes'
|
|
NOTE_KEY = 'scope.note'
|
|
BY_KEY = 'scope.declared_by'
|
|
|
|
# The leading letters of a mark. Deliberately letters-only: a mark is
|
|
# <letters><number> at every plant seen so far, and anything else is
|
|
# 'could not read', which is a reported state rather than a guess.
|
|
_PREFIX_RE = re.compile(r'^([A-Za-z]+)')
|
|
|
|
# Letters followed by a number: the shape of a piece mark or a design number.
|
|
# Used where a caller holds a name that MIGHT be a mark -- a sheet name, say
|
|
# -- and must not treat 'the general arrangement drawing' as somebody else's
|
|
# piece. Anything that is not this shape is left alone by the scope entirely.
|
|
_MARK_RE = re.compile(r'^[A-Za-z]+\s*\d')
|
|
|
|
# What a piece with no readable prefix is called on a report line.
|
|
UNREADABLE = '(no readable mark prefix)'
|
|
|
|
|
|
def prefix_of(name):
|
|
"""The leading letters of a mark, upper case, or None if there are none."""
|
|
if not isinstance(name, string_types):
|
|
name = '' if name is None else str(name)
|
|
m = _PREFIX_RE.match(name.strip())
|
|
return m.group(1).upper() if m else None
|
|
|
|
|
|
def looks_like_mark(name):
|
|
"""True when this text has the shape of a piece mark or design number.
|
|
|
|
A caller holding something that is only SOMETIMES a mark (a sheet name)
|
|
passes names through this first, so a drawing that is not a piece is
|
|
never counted as another engineer's piece.
|
|
"""
|
|
if not isinstance(name, string_types):
|
|
name = '' if name is None else str(name)
|
|
return bool(_MARK_RE.match(name.strip()))
|
|
|
|
|
|
def as_mark(name):
|
|
"""The name if it reads as a mark, else '' -- which every predicate here
|
|
treats as 'could not tell', i.e. left in."""
|
|
return name if looks_like_mark(name) else ''
|
|
|
|
|
|
class Scope(object):
|
|
"""The declared scope as a predicate. Construct with no argument for
|
|
'nobody has declared one', which includes everything."""
|
|
|
|
def __init__(self, prefixes=(), note=None, declared_by=None,
|
|
source='job knowledge: ' + KEY):
|
|
cleaned = []
|
|
for p in (prefixes or ()):
|
|
text = ('' if p is None else str(p)).strip().upper()
|
|
if text and text not in cleaned:
|
|
cleaned.append(text)
|
|
self.prefixes = tuple(cleaned)
|
|
self.note = note or None
|
|
self.declared_by = declared_by or None
|
|
self.source = source
|
|
|
|
# -------------------------------------------------------------- state
|
|
|
|
@property
|
|
def is_declared(self):
|
|
return bool(self.prefixes)
|
|
|
|
def label(self):
|
|
"""What the scope IS, in words a reader can check against reality."""
|
|
if not self.is_declared:
|
|
return 'no scope declared'
|
|
return 'piece marks beginning {0}'.format(
|
|
' or '.join(self.prefixes))
|
|
|
|
# ---------------------------------------------------------- predicate
|
|
|
|
def includes(self, name):
|
|
"""True when this mark is the engineer's to judge.
|
|
|
|
True for everything when no scope is declared, and true for a mark
|
|
whose prefix cannot be read -- 'I could not tell' never removes work.
|
|
"""
|
|
if not self.is_declared:
|
|
return True
|
|
prefix = prefix_of(name)
|
|
if prefix is None:
|
|
return True
|
|
return prefix in self.prefixes
|
|
|
|
def is_unreadable(self, name):
|
|
return prefix_of(name) is None
|
|
|
|
def kind(self, name):
|
|
"""The group a set-aside item is reported under."""
|
|
return prefix_of(name) or UNREADABLE
|
|
|
|
# ------------------------------------------------------------ splits
|
|
|
|
def split(self, items, key=None):
|
|
"""(inside, outside) preserving order. `key` reads the mark."""
|
|
key = key or (lambda x: x)
|
|
inside, outside = [], []
|
|
for item in (items or []):
|
|
(inside if self.includes(key(item)) else outside).append(item)
|
|
return inside, outside
|
|
|
|
def split_names(self, names):
|
|
return self.split(list(names or []))
|
|
|
|
def tally(self, names):
|
|
"""[(kind, count)] for the things set aside, biggest first."""
|
|
counts = {}
|
|
for name in (names or []):
|
|
kind = self.kind(name)
|
|
counts[kind] = counts.get(kind, 0) + 1
|
|
return sorted(counts.items(), key=lambda kv: (-kv[1], kv[0]))
|
|
|
|
# -------------------------------------------------------- the near miss
|
|
|
|
def nearly(self, kind):
|
|
"""The declared prefix `kind` NEARLY is, or None.
|
|
|
|
'Nearly' is deliberately narrow: one of the two spellings is a
|
|
prefix of the other. `AAAB` nearly matches a declared `AAA`, and a
|
|
bare `AAA` nearly matches a declared `AAAB` -- either way somebody
|
|
wrote one and meant, perhaps, the other. `BBB` nearly matches
|
|
nothing. An exact match is not a near miss, it is a match.
|
|
"""
|
|
if not self.is_declared or not kind or kind == UNREADABLE:
|
|
return None
|
|
for declared in self.prefixes:
|
|
if kind == declared:
|
|
return None
|
|
if kind.startswith(declared) or declared.startswith(kind):
|
|
return declared
|
|
return None
|
|
|
|
def near_misses(self, names):
|
|
"""[(kind, count, declared)] for set-aside kinds that nearly match.
|
|
|
|
Biggest first. Empty when the scope is undeclared, when nothing was
|
|
set aside, or when nothing set aside resembles the declaration --
|
|
which is the ordinary case and prints nothing.
|
|
"""
|
|
out = []
|
|
for kind, count in self.tally(names):
|
|
declared = self.nearly(kind)
|
|
if declared:
|
|
out.append((kind, count, declared))
|
|
return out
|
|
|
|
def near_miss_line(self, outside_names, unit='piece'):
|
|
"""The sentence a human has to read, or None.
|
|
|
|
Says both spellings and says the tool did not choose between them.
|
|
Never a verdict: a near miss is a question for the person who holds
|
|
the job knowledge, and the only two answers -- widen the declaration
|
|
or leave it -- are both his.
|
|
"""
|
|
misses = self.near_misses(outside_names)
|
|
if not misses:
|
|
return None
|
|
bits = ', '.join(
|
|
'{0} {1}(s) with mark prefix {2} (the declaration says {3})'
|
|
.format(count, unit, kind, declared)
|
|
for kind, count, declared in misses)
|
|
return ('NEAR MISS -- SOMEBODY SHOULD LOOK: {0} were set aside. A '
|
|
'prefix that only NEARLY matches is either work of yours the '
|
|
'declaration does not reach, or somebody else\'s work that '
|
|
'happens to start the same way, and this tool will not guess '
|
|
'which -- guessing wrong towards "yours" would judge another '
|
|
'engineer\'s piece silently. If they are yours, add the '
|
|
'longer prefix to the declaration ({1}); if they are not, '
|
|
'nothing needs doing and this line is the record that it was '
|
|
'looked at.'.format(bits, self.source))
|
|
|
|
# ------------------------------------------------------------ report
|
|
|
|
def line(self, outside_names, total=None, unit='piece',
|
|
unreadable_names=None):
|
|
"""ONE line saying what was left alone and why. Never a pass.
|
|
|
|
`outside_names` are the marks that were NOT judged. `total` is how
|
|
many there were altogether, so the reader can see the proportion;
|
|
omit it and the line simply does not claim one.
|
|
|
|
A NEAR MISS RIDES ON THIS LINE rather than on a line of its own, so
|
|
that every consumer of `line()` reports it without having to know it
|
|
exists. A report that wants it on a shut fold's summary as well asks
|
|
for `near_miss_line()` directly.
|
|
"""
|
|
outside = list(outside_names or [])
|
|
unreadable = list(unreadable_names or [])
|
|
tail = ''
|
|
if unreadable:
|
|
tail = (' {0} {1}(s) carry no readable mark prefix; they could '
|
|
'not be placed inside or outside the scope and were '
|
|
'judged rather than set aside.'
|
|
.format(len(unreadable), unit))
|
|
near = self.near_miss_line(outside, unit=unit)
|
|
if near:
|
|
tail += ' ' + near
|
|
|
|
if not self.is_declared:
|
|
return ('NOT JUDGED, OUT OF SCOPE: none set aside. No scope is '
|
|
'declared for this job ({0} is empty), so every {1} was '
|
|
'judged -- including work this plant knowledge has never '
|
|
'been validated against. Declare a scope to be told what '
|
|
'is being taken on trust.'.format(self.source, unit)
|
|
+ tail)
|
|
|
|
if not outside:
|
|
# Deliberately does NOT say 'everything here was in scope'. The
|
|
# user may have selected an out-of-scope piece himself, which is
|
|
# judged and annotated rather than set aside; this line's job is
|
|
# only to say that nothing was removed from the run.
|
|
return ('NOT JUDGED, OUT OF SCOPE: none -- nothing was set aside '
|
|
'from this run. The declared scope is {0} ({1}).'
|
|
.format(self.label(), self.source) + tail)
|
|
|
|
kinds = ', '.join('{0} {1}'.format(k, n) for k, n in self.tally(outside))
|
|
of_total = ' of {0}'.format(total) if total else ''
|
|
return ('NOT JUDGED, OUT OF SCOPE: {0}{1} {2}(s) were left alone -- '
|
|
'{3}. The declared scope is {4} ({5}); everything else '
|
|
'belongs to another engineer and the knowledge this tool '
|
|
'runs on is not validated for it. Not judged is not clean, '
|
|
'and not a failure either.'
|
|
.format(len(outside), of_total, unit, kinds, self.label(),
|
|
self.source) + tail)
|
|
|
|
def short_line(self, outside_names, total=None, unit='piece'):
|
|
"""The same three states in ONE SHORT CLAUSE, for a console.
|
|
|
|
`line()` is the sentence; this is the headline. It exists because
|
|
some consumers truncate -- `tools/sweep.py` prints the first 200
|
|
characters of what the in-Revit half returns -- and a scope line
|
|
that gets cut off the end of a progress line is a scope line nobody
|
|
read. It says the COUNT, which is what tells a reader there is a
|
|
full sentence somewhere worth opening.
|
|
|
|
It never replaces `line()`. Anything printing this must also write
|
|
`line()` where the reader can reach it.
|
|
"""
|
|
outside = list(outside_names or [])
|
|
if not self.is_declared:
|
|
return ('SCOPE: none declared -- every {0} was judged'
|
|
.format(unit))
|
|
of_total = ' of {0}'.format(total) if total else ''
|
|
if not outside:
|
|
return ('SCOPE {0}: no {1}{2} was set aside'
|
|
.format(self.label(), unit, of_total))
|
|
kinds = self.tally(outside)
|
|
shown = ', '.join('{0} {1}'.format(k, n) for k, n in kinds[:3])
|
|
if len(kinds) > 3:
|
|
shown += ', +{0} more'.format(len(kinds) - 3)
|
|
text = ('NOT JUDGED, OUT OF SCOPE: {0}{1} {2}(s) left alone [{3}]'
|
|
.format(len(outside), of_total, unit, shown))
|
|
if self.near_misses(outside):
|
|
text += ' -- NEAR MISS'
|
|
return text
|
|
|
|
def annotation(self, outside_names, unit='piece'):
|
|
"""The line for work the USER picked that falls outside the scope.
|
|
|
|
Nothing the user selected is ever dropped -- see `honoured_selection`
|
|
in the design notes. It is judged, and this says the verdict rests on
|
|
knowledge nobody has validated for that kind of piece.
|
|
"""
|
|
outside = list(outside_names or [])
|
|
if not self.is_declared or not outside:
|
|
return None
|
|
kinds = ', '.join('{0} {1}'.format(k, n) for k, n in self.tally(outside))
|
|
# A near miss matters MORE here than on `line()`: the reader is being
|
|
# told his own selection rests on unvalidated knowledge, and if the
|
|
# prefix only nearly missed, that warning may be the wrong one.
|
|
near = self.near_miss_line(outside, unit=unit)
|
|
return ('JUDGED OUTSIDE THE DECLARED SCOPE, AT YOUR REQUEST: {0} '
|
|
'{1}(s) -- {2}. You selected them, so nothing was dropped; '
|
|
'but the declared scope is {3}, and these verdicts rest on '
|
|
'knowledge that has not been validated for those pieces. '
|
|
'Read them as opinions, not as checks.'
|
|
.format(len(outside), unit, kinds, self.label())
|
|
+ ((' ' + near) if near else ''))
|
|
|
|
def declaration_lines(self):
|
|
"""Who declared it and what they said, when they said anything."""
|
|
out = []
|
|
if self.declared_by:
|
|
out.append('scope declared by {0}'.format(self.declared_by))
|
|
if self.note:
|
|
# A YAML folded scalar (`note: >`) arrives with a trailing
|
|
# newline, which puts a blank line into every report that prints
|
|
# it. Stripped here rather than at four call sites.
|
|
out.append(' '.join(str(self.note).split()))
|
|
return out
|
|
|
|
|
|
def _as_list(value):
|
|
if value is None:
|
|
return []
|
|
if isinstance(value, string_types):
|
|
return [value]
|
|
return list(value)
|
|
|
|
|
|
def of(know, get=None):
|
|
"""The declared scope out of a loaded knowledge dict.
|
|
|
|
`get` is `knowledge.get`. It is a parameter so a test can pass its own
|
|
and this module never has to import `knowledge` at module scope; the
|
|
ordinary path imports it lazily, and reads the three keys through the
|
|
alias `K` with LITERAL paths so `tools/knowledge_drift.py` can see that
|
|
they are genuinely read. A key nobody can prove is read is a key the
|
|
file only claims to honour.
|
|
"""
|
|
if not know:
|
|
return Scope()
|
|
if get is None:
|
|
from vibe import knowledge as K
|
|
return Scope(prefixes=_as_list(K.get(know, 'scope.mark_prefixes', None)),
|
|
note=K.get(know, 'scope.note', None),
|
|
declared_by=K.get(know, 'scope.declared_by', None))
|
|
return Scope(prefixes=_as_list(get(know, 'scope.mark_prefixes', None)),
|
|
note=get(know, 'scope.note', None),
|
|
declared_by=get(know, 'scope.declared_by', None))
|