# -*- coding: utf-8 -*-
"""How many physical pieces one mark is. The control-number multiplier. No Revit.

WHY THIS IS ITS OWN MODULE
---------------------------
It is the single number that decides whether the takeoff is right, and there
is more than one place it can come from. An assembly is a MARK, a mark is one
drawing, and the pieces on the floor are the pieces the mark issues.
Identical pieces share a mark, so counting marks under-orders every material
in the job.

How badly is measurable off an exported ticket set without opening Revit:
151 tickets, 71 distinct marks. The heaviest mark in that sample issued
eighteen tickets. A per-mark takeoff of that sample would have ordered 71
pieces' worth of hardware for 151 pieces -- IF one ticket is one piece there,
which is a declared fact and not this module's to assume. See the ticket-unit
section below; it is the reason a ticket set can be read to two different
numbers.

THREE SOURCES, RANKED, AND THE RANKING IS THE POINT
----------------------------------------------------
    control_numbers     the control numbers carried by the model's own
                        elements, grouped by mark. AUTHORITATIVE: a control
                        number is one physical piece, and this is where they
                        actually live. Measured on this job by the job
                        reconciliation: 2480 control-numbered elements ->
                        1776 pieces, against 755 assemblies. They sit on
                        STRUCTURAL FRAMING; walls, floors, columns,
                        foundations and generic models were swept -- 17,239
                        elements -- and carried none.
    ticket_files        the exported tickets, named <job>-<control>_<mark>.
                        Only covers marks that have been ticketed, so it is a
                        cross-check rather than a source -- and a very good
                        one, because it is ground truth for the marks it does
                        cover.
    assembly_instances  how many assembly instances share the mark's name.
                        MEASURED AND FOUND USELESS ON THIS JOB, and kept only
                        because it is useless in a way worth recording: 755
                        assembly instances, 755 distinct names, not one
                        duplicate. It therefore returned 1 for every mark,
                        which made every per-job figure silently equal its
                        per-mark figure -- the exact undercount this module
                        exists to prevent, wearing the costume of a sourced
                        number. It now REFUSES to emit a count of one (see
                        `from_assembly_names`), so on a job like this one it
                        contributes nothing at all and the marks it cannot
                        speak for stay visibly `assumed_one`.

Anything none of them covers is 1 with basis `assumed_one`, which the report
prints as an UNDERCOUNT rather than as a quantity. A multiplier nobody could
source is not a multiplier of one -- and the reason that rule was worth
writing is that it is what caught `assembly_instances` being wrong.

WHAT ONE TICKET IS, AND WHY THIS MODULE MAY NOT DECIDE IT
----------------------------------------------------------
`ticket_files` is a piece count only where ONE TICKET IS ONE PIECE. That is
not universal and it is not this module's to assume: some plants issue one
ticket per control number, and some issue one ticket covering a whole mark --
one drawing, one sheet, a QUANITY field and several control numbers printed
on it. The same export folder read under the second arrangement gives one
filename per mark, and counting those filenames as pieces would return a
confident 1 for a mark that is really seven. That is exactly the
`assembly_instances` failure in different clothes, and it is why the ranking
above stops being a constant here.

So the fact is DECLARED, in `nomenclature.ticket_unit`, and read out of the
knowledge layers rather than compiled in. `takeoffgather.run` does the read,
with the literal path, and hands the value to `counting_plan()` below, which
turns it into the ranking this run is allowed to use and one sentence saying
why. Three states, and all three are printed:

    control_number   one ticket per control number. A ticket IS a piece, so
                     all three sources rank exactly as `ORDER` says.
    piece_mark       one ticket covers the mark. A ticket filename is not a
                     piece count, so `ticket_files` is WITHDRAWN and the
                     ticket-against-model cross-check stands down with a
                     sentence rather than reporting every mark as a
                     disagreement.
    not declared     nobody has said, so this run does not know which of the
                     two it is looking at. `ticket_files` is withdrawn HERE
                     TOO, and the refusal names the key to declare.

That last one is the only judgement call in this file and it goes the way
this codebase's other unset switches go -- `rails.left_rail_is_fixed`,
`takeoff.not_ordered`, `notes.instruction_views`. Falling back to
"one ticket is one piece" would be a guess that can only ever UNDER-count,
and it would arrive wearing a source's name (`ticket_files`) rather than
`assumed_one`. Withdrawing the source instead costs nothing that was known:
the marks it covered fall to `assumed_one`, which the report already prints
as an undercount in those words. A number that says "could not judge" is a
different thing from a number that says "judged, found one", and the whole
point of this module is that the two must never look alike.

ONE PARSER FOR ONE RULE
------------------------
A control field may hold a range: `50120-50123` is four pieces, not one
token. `jobrecon.expand_control_field` already implements that expansion,
with its dash normalisation and its 500-wide sanity cap, and this module
CALLS IT rather than reimplementing it. Three implementations of one parsing
rule is how they come to disagree.
"""
# pylint: disable=invalid-name
import re

from vibe import jobrecon

# <job>-<control>_<mark>.pdf, with an optional revision suffix on the mark.
# The older export was named by mark alone, which is exactly how one file per
# mark came to be mistaken for one piece per mark; a name with no control
# number in it is therefore IGNORED rather than counted as one piece.
TICKET_RE = re.compile(r'^(?P<job>[^-_]+)-(?P<control>[^_]+)_(?P<mark>.+)$')

# Precedence, best first. A mark found by two sources takes the better one
# and the report can still show what the other said.
#
# It is the ranking FOR A JOB WHERE ONE TICKET IS ONE PIECE. Which sources a
# given job may actually use is `counting_plan()`, off the declared
# `nomenclature.ticket_unit`; this tuple is the full ranking those plans are
# drawn from, and no caller should reach for it instead of a plan.
ORDER = ('control_numbers', 'ticket_files', 'assembly_instances')

# `nomenclature.ticket_unit` -- what one exported ticket covers. A JOB fact:
# the same plant can do it either way and the manual shows both, so it is
# declared in the job layer and never inferred here.
TICKET_UNIT_KEY = 'nomenclature.ticket_unit'
CONTROL_NUMBER = 'control_number'
PIECE_MARK = 'piece_mark'

# Which sources are a PIECE COUNT under each declared unit. Only
# `ticket_files` turns on this: a control number off a model element is one
# physical piece however the plant tickets it, and a duplicated assembly name
# is a duplicated piece either way.
NO_TICKETS = tuple(b for b in ORDER if b != 'ticket_files')
UNIT_SOURCES = {
    CONTROL_NUMBER: ORDER,
    PIECE_MARK: NO_TICKETS,
}


def _stem(filename):
    name = (filename or '').replace('\\', '/').rsplit('/', 1)[-1]
    if '.' in name:
        name = name.rsplit('.', 1)[0]
    return name


def parse_ticket_name(filename):
    """(mark, control) from an exported ticket filename, or (None, None).

    A revision suffix belongs to the ticket, not to the piece: two tickets
    for the same control number are one piece, so the control number is what
    is counted and the mark keeps its revision stripped.

    A CONTROL NUMBER CONTAINS AT LEAST ONE DIGIT, and that one rule is the
    difference between reading an export folder and reading a ticket set.
    The folder also holds the detail sheets, whose names take the identical
    three-part shape -- `<job>-SHEET_X4-1.pdf` parses out a "control number"
    of SHEET and a "mark" of X4-1 -- and twelve of those were being counted
    as marks. They cost nothing in quantity, because no real mark is named
    after a detail sheet, but they inflated what the source claimed to know
    and they were caught by `join_quality` rather than by anyone reading the
    code. Letters in a control number are still fine; letters ALONE are not.
    """
    found = TICKET_RE.match(_stem(filename))
    if found is None:
        return (None, None)
    control = found.group('control').strip()
    if not any(ch.isdigit() for ch in control):
        return (None, None)
    mark = found.group('mark').strip()
    mark = re.sub(r'[_-]R\d+$', '', mark, flags=re.IGNORECASE)
    return (mark or None, control or None)


def unparsed_ticket_files(filenames):
    """The filenames `from_ticket_files` could not read. Never silent.

    A folder of exports is somebody's directory and holds whatever they put
    in it. What was skipped is a fact about the source's coverage, so it is
    returned rather than assumed to be rubbish.
    """
    return sorted(name for name in (filenames or [])
                  if parse_ticket_name(name) == (None, None))


def from_ticket_files(filenames):
    """{mark: (distinct control numbers, 'ticket_files')}.

    Distinct, not total: a mark reissued after a revision has two files and
    one piece.
    """
    seen = {}
    for filename in filenames or []:
        mark, control = parse_ticket_name(filename)
        if mark and control:
            seen.setdefault(mark, set()).add(control)
    return dict((mark, (len(seen[mark]), 'ticket_files')) for mark in seen)


def from_assembly_names(names, min_instances=2):
    """{mark: (instances, 'assembly_instances')} -- DUPLICATED NAMES ONLY.

    Wants the names WITH duplicates, because the duplication is the whole
    signal. A name that appears once is NOT evidence that the mark is one
    piece; it is the absence of evidence, and the two must not produce the
    same output.

    That distinction is not academic. Measured on this job through the
    bridge: 755 assembly instances, 755 distinct names, zero duplicates. An
    earlier version of this function returned 1 for each of them, every mark
    got a confident `assembly_instances` basis, and every per_job figure
    silently equalled its per_mark figure -- a whole-job undercount that
    looked sourced. `min_instances=2` is what makes that impossible: with no
    duplicates this returns {} and every mark falls through to
    `assumed_one`, which the report prints as an undercount in those words.
    """
    out = {}
    for name in names or []:
        if not name:
            continue
        out[name] = out.get(name, 0) + 1
    return dict((name, (out[name], 'assembly_instances')) for name in out
                if out[name] >= min_instances)


def from_piece_records(pieces):
    """({mark: (pieces, 'control_numbers')}, unreadable) from model elements.

    `pieces` is the list `jobreconscan.gather` produces -- the same data
    contract the job reconciliation eats, deliberately, so that the takeoff
    and the reconciliation cannot end up disagreeing about how many pieces
    the job has. Each record is {'control', 'mark', 'id', ...}.

    Control numbers are counted DISTINCT per mark, because a control number
    shared by several elements is still one piece: the reconciliation reports
    2480 control-numbered elements resolving to 1776 pieces, and the
    difference between those two numbers is exactly that sharing.

    A control field is expanded through `jobrecon.expand_control_field`, so a
    range is four pieces rather than one token and an unreadable slot is
    RETURNED rather than dropped. An element whose mark is missing cannot be
    attributed to a mark and is returned as unreadable too -- it is a real
    piece this multiplier cannot place, and losing it silently would
    under-count in exactly the way this module exists to prevent.
    """
    by_mark = {}
    unreadable = []
    for piece in pieces or []:
        mark = (piece.get('mark') or '').strip()
        controls, bad = jobrecon.expand_control_field(piece.get('control'))
        for token in bad:
            unreadable.append({'id': piece.get('id'), 'mark': mark or None,
                               'token': token,
                               'why': 'not readable as a control number'})
        if not controls:
            continue
        if not mark:
            unreadable.append({'id': piece.get('id'), 'mark': None,
                               'token': ', '.join(controls),
                               'why': 'the element carries no mark, so its '
                                      'pieces cannot be attributed'})
            continue
        by_mark.setdefault(mark, set()).update(controls)
    return (dict((mark, (len(by_mark[mark]), 'control_numbers'))
                 for mark in by_mark),
            unreadable)


def from_control_numbers(by_mark):
    """{mark: (distinct control numbers, 'control_numbers')}.

    The already-grouped form, for a caller that has the mapping in hand.
    `from_piece_records` is the one that takes the reconciliation's own
    element records and does the grouping and the range expansion.

    `by_mark` is {mark: [control numbers]} read off the model. Distinct
    again, and an empty list is NOT a zero: a mark whose control numbers
    could not be read is left out entirely so it falls through to the next
    source rather than being multiplied by nothing.
    """
    out = {}
    for mark in by_mark or {}:
        controls = set(str(c).strip() for c in (by_mark[mark] or [])
                       if str(c).strip())
        if controls:
            out[mark] = (len(controls), 'control_numbers')
    return out


def counting_plan(unit, sources=None):
    """Which sources may count pieces, given the DECLARED `ticket_unit`.

    `unit` is the value of `nomenclature.ticket_unit` as the knowledge layers
    hold it -- read by the caller, so this module needs no knowledge access
    and stays testable with a string. `sources` is {basis: {mark: (n, basis)}}
    and is optional: it is used only to say what withdrawing a source
    actually COST on this job, which is the difference between a rule and a
    finding.

    Returns a plan the report prints whole:

        unit             what was declared, or None
        declared         somebody said
        recognised       and this module knows what they said
        order            the bases that may count pieces, best first
        withdrawn        the bases that may not, and why is in `reason`
        withdrawn_marks  marks a withdrawn source knew
        cost             marks that lose their ONLY source to the withdrawal
        reason           one sentence, ALWAYS, including when nothing changed
        refusal          the same sentence when the run COULD NOT JUDGE --
                         undeclared or unrecognised. None when somebody
                         declared it, whichever way they declared it.

    An unrecognised value is a refusal and not a fallback. A key set to
    something nobody implemented is somebody's intention, misspelt; reading
    it as the default would silently do the opposite of what they asked for
    on exactly the jobs where it matters.
    """
    unit = (str(unit).strip() or None) if unit is not None else None
    order = UNIT_SOURCES.get(unit)
    plan = {
        'key': TICKET_UNIT_KEY,
        'unit': unit,
        'declared': unit is not None,
        'recognised': order is not None,
        'order': tuple(order if order is not None else NO_TICKETS),
        'reason': None,
        'refusal': None,
    }
    plan['withdrawn'] = tuple(b for b in ORDER if b not in plan['order'])

    if unit == CONTROL_NUMBER:
        plan['reason'] = (
            '`{0}: {1}` -- this job issues one ticket per control number, so '
            'one exported ticket is one piece and the ticket files count as '
            'pieces, ranked behind the model\'s own control '
            'numbers.'.format(TICKET_UNIT_KEY, CONTROL_NUMBER))
    elif unit == PIECE_MARK:
        plan['reason'] = (
            '`{0}: {1}` -- one ticket covers a whole mark, so a ticket '
            'filename names one control number and NOT one piece. The '
            'exported tickets were therefore NOT counted as pieces, and the '
            'ticket-against-model cross-check stands down rather than '
            'reporting every mark as a disagreement. Marks that only the '
            'tickets covered are counted once and shown as an '
            'undercount.'.format(TICKET_UNIT_KEY, PIECE_MARK))
    elif unit is None:
        plan['refusal'] = (
            '`{0}` is not declared, so this run does not know whether one '
            'exported ticket is one piece or a whole mark of them. The '
            'exported tickets were NOT used as a piece count: taking one '
            'ticket as one piece would under-count a job ticketed the other '
            'way, and it would do it wearing a source\'s name instead of '
            '`assumed_one`. Declare `{0}: {1}` (one ticket per control '
            'number) or `{0}: {2}` (one ticket covering the mark) in the job '
            'knowledge file and press again.'.format(
                TICKET_UNIT_KEY, CONTROL_NUMBER, PIECE_MARK))
    else:
        plan['refusal'] = (
            '`{0}` is set to "{1}", which this takeoff does not recognise -- '
            'it knows `{2}` and `{3}`. A value nobody can read is not a '
            'declaration, so the exported tickets were NOT used as a piece '
            'count and nothing was assumed on your behalf.'.format(
                TICKET_UNIT_KEY, unit, CONTROL_NUMBER, PIECE_MARK))
    if plan['reason'] is None:
        plan['reason'] = plan['refusal']

    known = {}
    for basis in (sources or {}):
        for mark in (sources[basis] or {}):
            known.setdefault(mark, set()).add(basis)
    plan['withdrawn_marks'] = len(
        [m for m in known if known[m] & set(plan['withdrawn'])])
    plan['cost'] = len([m for m in known
                        if not known[m] & set(plan['order'])])
    return plan


def merge(*sources, **kwargs):
    """Fold the sources together, best basis first. Later never overrides.

    `order` (keyword) is the ranking this job is allowed to use --
    `counting_plan()['order']`. It defaults to `ORDER`, all three sources, so
    a caller that has not asked the question gets exactly the behaviour this
    function has always had.

    A basis NOT in `order` is DROPPED, never ranked last: a source that is not
    a piece count on this job must not become one merely because nothing
    better covered the mark. Ranking it 99th would let it through on exactly
    the marks where nothing can check it, which is where a wrong multiplier
    does its damage.
    """
    order = kwargs.pop('order', None) or ORDER
    if kwargs:
        raise TypeError('merge() got an unexpected keyword argument {0}'
                        .format(sorted(kwargs)[0]))
    rank = dict((basis, i) for i, basis in enumerate(order))
    out = {}
    for source in sources:
        for mark in source or {}:
            count, basis = source[mark]
            if basis not in rank:
                continue
            have = out.get(mark)
            if have is None or rank[basis] < rank.get(have[1], 99):
                out[mark] = (int(count), basis)
    return out


def disagreements(a, b):
    """Marks where two sources give different counts. Worth printing.

    A mismatch between the exported tickets and whatever the model says is
    the cheapest possible audit of the multiplier, and it is the one number
    in this takeoff that nothing else can check.
    """
    out = []
    for mark in sorted(set(a or {}) & set(b or {})):
        if a[mark][0] != b[mark][0]:
            out.append({'mark': mark,
                        'left': a[mark][0], 'left_basis': a[mark][1],
                        'right': b[mark][0], 'right_basis': b[mark][1]})
    return out


def join_quality(multipliers, marks):
    """Do the multiplier's marks and the takeoff's marks name the same things?

    The takeoff keys on the ASSEMBLY NAME; the multiplier keys on the piece
    MARK read off a framing element. Measured, those are the same string --
    an assembly named for its mark -- but "measured" is not "guaranteed", and
    the failure if they ever diverge is silent and total: every mark falls
    through to `assumed_one` and the job is under-ordered wholesale while the
    report shows a source that found plenty.

    So the join is counted, both ways, and printed. `unmatched_source` is the
    tell: a source that found hundreds of marks the takeoff has never heard
    of is not a source, it is a different naming scheme.
    """
    marks = set(marks or [])
    source = set(multipliers or {})
    return {
        'takeoff_marks': len(marks),
        'source_marks': len(source),
        'matched': len(marks & source),
        'unsourced': sorted(marks - source),
        'unmatched_source': sorted(source - marks),
    }


def summarise(multipliers, marks=None):
    """The multiplier, as a reader has to be able to check it.

    `marks` is every mark the takeoff covered, so that a mark with no
    multiplier is counted into `assumed_one` rather than vanishing.
    """
    marks = list(marks) if marks is not None else list(multipliers or {})
    by_basis = {}
    total = 0
    for mark in marks:
        count, basis = (multipliers or {}).get(mark, (1, 'assumed_one'))
        by_basis[basis] = by_basis.get(basis, 0) + 1
        total += int(count)
    return {
        'marks': len(marks),
        'pieces': total,
        'multiplier': (round(total / float(len(marks)), 3) if marks else None),
        'by_basis': by_basis,
        'assumed_one': by_basis.get('assumed_one', 0),
    }