WillTools 2026-09-09 -- clean snapshot of the development repo for review
ci / test (push) Successful in 6m14s
ci / test (push) Successful in 6m14s
History-free snapshot of the private development repository at 77a82eb. Code, docs and the public catalogue only: no plant or job knowledge layers, no client, job or producer names (tools/scrub.py clean against the widened term list; a second word-level grep clean). The intro deck is left out because its slide images cannot be checked for job drawings.
This commit is contained in:
@@ -0,0 +1,445 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Phase C's host half: wait for Revit to exit, then make the detached copy.
|
||||
|
||||
python tools/snapshot_defer.py --request <req.json> --runner <run.py> \
|
||||
[--pyrevit <pyrevit.exe>] --wait-pid <revit pid> [--revit 2025]
|
||||
|
||||
`--pyrevit` is auto-discovered at the usual install path when left off
|
||||
(`vibe.snapshotmeta.pyrevit_exe_guess`) -- the same fallback the two callers
|
||||
already made for themselves; passing it explicitly still wins. `--revit` is
|
||||
also worth passing explicitly on any machine with more than one Revit
|
||||
version attached: see WHAT IT GUARANTEES below for why omitting it is now a
|
||||
loud warning rather than a silent maybe.
|
||||
|
||||
python tools/snapshot_defer.py --request <req.json> ... --dry-run
|
||||
python tools/snapshot_defer.py --newest <folder>
|
||||
|
||||
WHY A SEPARATE PROCESS EXISTS AT ALL
|
||||
-------------------------------------
|
||||
The machine cannot hold two copies of this model. 300+ MB on disk, 10+ GB
|
||||
open; the engineer's session and a scanning session cannot both be resident,
|
||||
which is what killed the original concurrent design. So the copy is made
|
||||
AFTER his Revit has exited, and something has to still be running to notice
|
||||
that it has. That something is this: a host process, started by the button,
|
||||
which outlives the Revit that started it, sleeps until the pid is gone, and
|
||||
then launches exactly one Revit of its own.
|
||||
|
||||
THE PARAGRAPH ABOVE IS NO LONGER TRUE, AND `--machine-free` IS THE SWITCH
|
||||
--------------------------------------------------------------------------
|
||||
Measured 2026-09-01: the live cloud session and a detached sandbox session
|
||||
ran side by side on this machine for a whole morning. Two Revits are fine;
|
||||
the untested number is three. So the machine-wide gate -- "wait until NO
|
||||
Revit at all is running" -- is now OPT-IN:
|
||||
|
||||
--machine-free ignore (default) only the watched pid is waited for
|
||||
--machine-free require the old behaviour, for the close-model ritual
|
||||
|
||||
`tools/snapshot_cycle.py` calls this with the default and sequences instead:
|
||||
it closes the sandbox Revit BEFORE launching the one that writes the copy, so
|
||||
two is the most that are ever resident. The `require` mode is kept, not
|
||||
deleted, because the close-model ritual on a slower machine is a real use for
|
||||
it and the give-up-and-journal behaviour underneath it is the tested part.
|
||||
|
||||
It is deliberately small. Everything it decides -- what the copy is called,
|
||||
which snapshot is newest, what goes in the log -- lives in
|
||||
`vibe.snapshotmeta`, where it is unit-tested without Revit. What is left here
|
||||
is waiting, launching, and reporting, and the two of those that can be tested
|
||||
without a licence (`wait_for_exit`, `command_for`) are pure functions with
|
||||
their clock and their liveness check injected.
|
||||
|
||||
WHAT IT GUARANTEES
|
||||
-------------------
|
||||
* It never starts Revit while the watched pid is alive.
|
||||
* Under `--machine-free require` it never starts Revit while ANY Revit is
|
||||
running -- not just that pid. He can close Revit and open it again while
|
||||
this is sleeping, and under the old one-model premise launching into that
|
||||
would have put two 10 GB models in memory.
|
||||
* It gives up after `--wait-minutes` and JOURNALS THE GIVE-UP. A Revit sitting
|
||||
on an unanswered save prompt is the ordinary reason, and a day when the copy
|
||||
silently did not happen must not look like a day when it did.
|
||||
* It checks afterwards that the file it was promised actually exists, and
|
||||
journals a failure if not -- the in-Revit runner can die in ways that take
|
||||
its own logging with it.
|
||||
|
||||
Host CPython 3.12. The extension side is IronPython 2.7; the shared module is
|
||||
written to both, and loaded here by path for the same reason the test suite
|
||||
does it -- importing `vibe` as a package runs `vibe/__init__.py`, which pulls
|
||||
in the Revit API and dies.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import traceback
|
||||
import types
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
VIBE = os.path.join(ROOT, 'WillTools.extension', 'lib', 'vibe')
|
||||
|
||||
if 'vibe' not in sys.modules:
|
||||
_stub = types.ModuleType('vibe')
|
||||
_stub.__path__ = [VIBE]
|
||||
_stub.__file__ = os.path.join(VIBE, '__init__.py')
|
||||
sys.modules['vibe'] = _stub
|
||||
|
||||
from vibe import snapshotmeta as S # noqa: E402
|
||||
|
||||
# Revit does not release the model the instant its window closes: the process
|
||||
# lingers while it flushes journals and drops file locks. Starting a new Revit
|
||||
# into that window is how you get a licence check-out race and a half-freed
|
||||
# 10 GB. Measured nowhere -- this is a guess, deliberately generous, and it is
|
||||
# the first number to turn down if the wait feels long.
|
||||
SETTLE_SECONDS = 30
|
||||
|
||||
POLL_SECONDS = 5
|
||||
|
||||
|
||||
# ------------------------------------------------------------- is it gone yet
|
||||
|
||||
def process_alive(pid):
|
||||
"""True while `pid` is a live process. Windows, no third-party module.
|
||||
|
||||
An unknown answer is reported as ALIVE, never as gone: being wrong in that
|
||||
direction costs a wait, and being wrong the other way starts a second
|
||||
Revit next to a live one -- the exact thing this whole design exists to
|
||||
prevent.
|
||||
"""
|
||||
if not pid:
|
||||
return False
|
||||
try:
|
||||
import ctypes
|
||||
from ctypes import wintypes
|
||||
SYNCHRONIZE = 0x00100000
|
||||
PROCESS_QUERY_LIMITED_INFORMATION = 0x1000
|
||||
STILL_ACTIVE = 259
|
||||
kernel32 = ctypes.windll.kernel32
|
||||
handle = kernel32.OpenProcess(
|
||||
SYNCHRONIZE | PROCESS_QUERY_LIMITED_INFORMATION, False, int(pid))
|
||||
if not handle:
|
||||
return False # no such process, or already exited
|
||||
try:
|
||||
code = wintypes.DWORD()
|
||||
if not kernel32.GetExitCodeProcess(handle, ctypes.byref(code)):
|
||||
return True # could not tell -> assume alive
|
||||
return code.value == STILL_ACTIVE
|
||||
finally:
|
||||
kernel32.CloseHandle(handle)
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
out = subprocess.run(
|
||||
['tasklist', '/FI', 'PID eq {0}'.format(pid), '/NH'],
|
||||
capture_output=True, text=True, timeout=30).stdout
|
||||
return str(pid) in out
|
||||
except Exception:
|
||||
return True # could not tell -> assume alive
|
||||
|
||||
|
||||
def any_revit_running(run=None):
|
||||
"""True while ANY Revit is running on this machine, whoever started it.
|
||||
|
||||
Waiting for the button's own pid is not enough. The engineer can close
|
||||
Revit, get a coffee, and open it again inside the window this tool is
|
||||
sleeping through -- and then launching Revit would put a second 10 GB
|
||||
model next to his live one, which is the exact failure the whole deferred
|
||||
design exists to avoid. So the last check before launching is not "has
|
||||
that process gone" but "is the machine free".
|
||||
|
||||
Unknown answers count as RUNNING, for the same reason as in
|
||||
`process_alive`: the cost of being wrong is a wait in one direction and a
|
||||
thrashed workstation in the other.
|
||||
"""
|
||||
run = run or (lambda argv: subprocess.run(
|
||||
argv, capture_output=True, text=True, timeout=30).stdout)
|
||||
try:
|
||||
out = run(['tasklist', '/FI', 'IMAGENAME eq Revit.exe', '/NH'])
|
||||
except Exception:
|
||||
return True
|
||||
return 'Revit.exe' in (out or '')
|
||||
|
||||
|
||||
def wait_for_exit(pid, timeout_s, alive=None, sleep=None, clock=None,
|
||||
poll=POLL_SECONDS):
|
||||
"""Block until `pid` is gone. True if it went, False on timeout.
|
||||
|
||||
Every dependency is injected so the loop can be tested with no process and
|
||||
no wall clock. It returns rather than raising on timeout because the
|
||||
timeout is a REPORTABLE OUTCOME, not an error: somebody left a save prompt
|
||||
open, and the next press of the button needs to say so.
|
||||
"""
|
||||
if not pid:
|
||||
# Nothing to wait for. Guarded here as well as in `process_alive`
|
||||
# because this loop's only exits are "the process went" and "the clock
|
||||
# advanced past the timeout" -- a caller that passes no pid and a
|
||||
# liveness check that always says yes would otherwise spin forever,
|
||||
# which is exactly how this was first written and exactly what the
|
||||
# test suite caught.
|
||||
return True
|
||||
alive = alive or process_alive
|
||||
sleep = sleep or time.sleep
|
||||
clock = clock or time.time
|
||||
started = clock()
|
||||
while alive(pid):
|
||||
if clock() - started >= timeout_s:
|
||||
return False
|
||||
sleep(poll)
|
||||
return True
|
||||
|
||||
|
||||
# ----------------------------------------------------------- the command line
|
||||
|
||||
def command_for(pyrevit_exe, runner, revit_year=None):
|
||||
"""The exact `pyrevit run` argv. One source for it, so the line printed by
|
||||
`--dry-run` and the line actually executed cannot drift apart."""
|
||||
return S.pyrevit_run_argv(pyrevit_exe, runner, revit_year)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- the run
|
||||
|
||||
def load_request(path):
|
||||
with open(path, 'rb') as handle:
|
||||
return json.loads(handle.read().decode('utf-8'))
|
||||
|
||||
|
||||
def fallback_log_for(request_path):
|
||||
"""`<detached_dir>/snapshots.jsonl`, recovered from the REQUEST'S OWN
|
||||
PATH rather than its contents.
|
||||
|
||||
For the one failure `journal()` cannot report: the request file could not
|
||||
even be read, so there is no `log` field to send the line to. But every
|
||||
request this tool is ever handed lives at
|
||||
`<detached_dir>/_requests/<run>.json` -- the button writes it there, and
|
||||
so does `tools/snapshot_cycle.py`'s own `write_detach_request` -- so the
|
||||
well-known log path can be recovered from the FOLDER STRUCTURE alone,
|
||||
with no JSON parse required. None when the path does not have that
|
||||
shape (a caller pointed `--request` at something else entirely), because
|
||||
a log path guessed at random is worse than no journal line at all.
|
||||
"""
|
||||
try:
|
||||
requests_dir = os.path.dirname(os.path.abspath(request_path))
|
||||
if os.path.basename(requests_dir) != '_requests':
|
||||
return None
|
||||
return S.log_path(os.path.dirname(requests_dir))
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def journal(request, event, **fields):
|
||||
path = (request or {}).get('log')
|
||||
if not path:
|
||||
return
|
||||
row = {'event': event, 'run': (request or {}).get('run')}
|
||||
row.update(fields)
|
||||
S.append(path, row)
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
ap.add_argument('--request', help='the request file the button wrote')
|
||||
ap.add_argument('--runner', help='the script pyrevit run executes')
|
||||
ap.add_argument('--pyrevit', help='path to pyrevit.exe')
|
||||
ap.add_argument('--revit', help='Revit year, e.g. 2025')
|
||||
ap.add_argument('--wait-pid', type=int, default=0,
|
||||
help='wait for this process to exit first')
|
||||
ap.add_argument('--wait-minutes', type=float, default=20.0)
|
||||
ap.add_argument('--settle', type=float, default=SETTLE_SECONDS,
|
||||
help='seconds to wait after the pid is gone')
|
||||
ap.add_argument('--machine-free', choices=('ignore', 'require'),
|
||||
default='ignore',
|
||||
help='require: also wait until no Revit at all is running '
|
||||
'(the old close-model behaviour). Default ignore -- '
|
||||
'two Revits are measured to be fine, three are not, '
|
||||
'and the caller sequences for that.')
|
||||
ap.add_argument('--dry-run', action='store_true',
|
||||
help='print what would happen and exit, launching nothing')
|
||||
ap.add_argument('--newest', metavar='FOLDER',
|
||||
help='print the newest snapshot in FOLDER and exit')
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
if args.newest:
|
||||
rows = S.snapshots(args.newest)
|
||||
if not rows:
|
||||
print('no snapshots in {0}'.format(args.newest))
|
||||
return 1
|
||||
for row in rows:
|
||||
mark = '*' if row is rows[-1] else ' '
|
||||
print('{0} {1} {2:8} {3}'.format(mark, row['stamp'],
|
||||
row['layout'], row['path']))
|
||||
print('\nnewest: {0}'.format(rows[-1]['path']))
|
||||
return 0
|
||||
|
||||
if not (args.request and args.runner):
|
||||
ap.error('--request and --runner are required')
|
||||
|
||||
# A missing or unreadable --request has nowhere of its own to be
|
||||
# journalled -- the request IS where the log path lives. But its own
|
||||
# location still says where the detached_dir is (see
|
||||
# `fallback_log_for`), so a crash here is not silent even so; only a
|
||||
# `--request` that does not even have a `_requests` parent gives up on
|
||||
# a journal line entirely, and that is loud on stderr instead.
|
||||
try:
|
||||
request = load_request(args.request)
|
||||
except Exception:
|
||||
tb = traceback.format_exc()
|
||||
print(tb, file=sys.stderr)
|
||||
fallback = fallback_log_for(args.request)
|
||||
if fallback:
|
||||
S.append(fallback, {
|
||||
'event': S.WATCHER_CRASHED, 'run': None,
|
||||
'error': 'could not read --request {0}: {1}'
|
||||
.format(args.request, tb.splitlines()[-1]),
|
||||
'traceback': tb})
|
||||
return 2
|
||||
|
||||
target = (request.get('plan') or {}).get('path')
|
||||
|
||||
# `--pyrevit` used to be required by `ap.error()` above, with no journal
|
||||
# possible at that point (the request was not loaded yet) -- a run
|
||||
# started with the flag simply missing died silently. It is optional now:
|
||||
# discovered the same way the two callers that already had this guess
|
||||
# discover it (`tools/snapshot_cycle.py`'s `_pyrevit_guess`, the
|
||||
# SnapshotClose button's `_pyrevit_cli`), so a caller that forgets it no
|
||||
# longer has to be the third copy of the same fallback.
|
||||
pyrevit_exe = args.pyrevit or S.pyrevit_exe_guess()
|
||||
if not pyrevit_exe:
|
||||
error = ('the pyRevit CLI was not found -- pass --pyrevit, or set '
|
||||
'snapshot.pyrevit_exe, or install pyRevit at the usual '
|
||||
'path ({0}\\pyRevit-Master\\bin\\pyrevit.exe)'
|
||||
.format(os.environ.get('APPDATA', '%APPDATA%')))
|
||||
journal(request, S.FAILED, path=target, error=error)
|
||||
print(error, file=sys.stderr)
|
||||
return 2
|
||||
|
||||
command = command_for(pyrevit_exe, args.runner, args.revit)
|
||||
|
||||
if args.dry_run:
|
||||
print('request {0}'.format(args.request))
|
||||
print('run id {0}'.format(request.get('run')))
|
||||
print('source {0}'.format(
|
||||
(request.get('source') or {}).get('path') or
|
||||
'cloud GUIDs in the request'))
|
||||
print('target {0}'.format(target))
|
||||
print('log {0}'.format(request.get('log')))
|
||||
print('wait for pid {0} (up to {1} min, then {2} s to settle),'
|
||||
.format(args.wait_pid, args.wait_minutes, args.settle))
|
||||
if args.machine_free == 'require':
|
||||
print(' then until NO Revit at all is running on this '
|
||||
'machine')
|
||||
else:
|
||||
print(' and NOT for other Revits (--machine-free '
|
||||
'ignore)')
|
||||
if not args.revit:
|
||||
print('\nWARNING: no --revit=<year> was given. On a machine '
|
||||
'with more than one Revit version attached, pyRevit has '
|
||||
'to pick one on its own -- confirm this is intended '
|
||||
'(see snapshot.revit_year / the Revit year folded into '
|
||||
'snapshot.revit_exe).')
|
||||
print('\nthen runs:\n {0}'.format(S.quote(command)))
|
||||
print('\nwith {0}={1}'.format('WILLTOOLS_SNAPSHOT_REQUEST',
|
||||
args.request))
|
||||
return 0
|
||||
|
||||
# From here on, this process is the whole reason the "requested and
|
||||
# never reported" question (`vibe.snapshotmeta.outstanding`) can be
|
||||
# answered instead of guessed at: a REQUESTED with no WATCHER_LAUNCHED
|
||||
# after it means this process never made it this far at all (never
|
||||
# spawned, or crashed on the request load above); one with
|
||||
# WATCHER_LAUNCHED but nothing terminal after it means it started and
|
||||
# then something below killed it hard enough to skip even the `except`
|
||||
# net a few lines down -- a signal, a forced kill, the machine itself
|
||||
# going away. Both are now distinguishable from "still running" and
|
||||
# from each other, which the ledger could not do before this line
|
||||
# existed.
|
||||
journal(request, S.WATCHER_LAUNCHED, path=target,
|
||||
wait_pid=args.wait_pid or None, wait_minutes=args.wait_minutes,
|
||||
machine_free=args.machine_free, pyrevit_exe=pyrevit_exe,
|
||||
revit_year=args.revit or None)
|
||||
|
||||
try:
|
||||
deadline = args.wait_minutes * 60.0
|
||||
if args.wait_pid:
|
||||
gone = wait_for_exit(args.wait_pid, deadline)
|
||||
if not gone:
|
||||
journal(request, S.FAILED, path=target,
|
||||
error='Revit (pid {0}) was still running after {1} '
|
||||
'minutes; no copy was made. A save prompt left '
|
||||
'unanswered is the usual reason.'
|
||||
.format(args.wait_pid, args.wait_minutes))
|
||||
return 2
|
||||
time.sleep(args.settle)
|
||||
|
||||
# THE MACHINE, not just that one process -- but only when asked for.
|
||||
# He may have opened Revit again while this was sleeping; under the
|
||||
# retired one-model premise that was fatal, and under
|
||||
# `--machine-free require` it still stands down. `wait_for_exit` is
|
||||
# reused with a machine-wide predicate; the pid is a placeholder it
|
||||
# never inspects.
|
||||
if args.machine_free == 'require' and \
|
||||
not wait_for_exit(1, deadline,
|
||||
alive=lambda _pid: any_revit_running()):
|
||||
journal(request, S.FAILED, path=target,
|
||||
error='a Revit is running again on this machine, so no '
|
||||
'copy was made -- --machine-free require was '
|
||||
'asked for. Press the button again at the end of '
|
||||
'the next session, or run with --machine-free '
|
||||
'ignore.')
|
||||
return 2
|
||||
|
||||
env = dict(os.environ)
|
||||
env['WILLTOOLS_SNAPSHOT_REQUEST'] = args.request
|
||||
try:
|
||||
result = subprocess.run(command, env=env, capture_output=True,
|
||||
text=True)
|
||||
code = result.returncode
|
||||
tail = (result.stderr or result.stdout or '').strip().splitlines()
|
||||
except Exception as exc:
|
||||
journal(request, S.FAILED, path=target,
|
||||
error='could not run the pyRevit CLI: {0}'.format(exc))
|
||||
return 2
|
||||
|
||||
# The runner journals its own DONE. This is the backstop for the
|
||||
# case where it could not -- Revit died, the journal never played,
|
||||
# the CLI refused. An outcome nobody wrote down is the failure this
|
||||
# design is least able to survive.
|
||||
if target and os.path.isfile(target):
|
||||
# The file is the only evidence that counts. `pyrevit run`
|
||||
# returns non-zero for journal-playback noise that has nothing
|
||||
# to do with whether the copy was written, so the exit code is
|
||||
# recorded when it is odd and is not allowed to overrule the
|
||||
# filesystem.
|
||||
if code != 0:
|
||||
journal(request, 'phase-c-note', path=target, exit_code=code,
|
||||
note='the copy was written; the pyRevit CLI still '
|
||||
'exited {0}'.format(code))
|
||||
return 0
|
||||
hint = ''
|
||||
if not args.revit:
|
||||
hint = (' No --revit=<year> was passed for this run -- on a '
|
||||
'machine with more than one Revit version attached, '
|
||||
'that is the first thing to try.')
|
||||
journal(request, S.FAILED, path=target, exit_code=code,
|
||||
error=('the pyRevit run finished but the copy is not there. '
|
||||
'Last line: {0}{1}'
|
||||
.format(tail[-1] if tail else '(nothing)', hint)))
|
||||
return 2
|
||||
except Exception:
|
||||
# The backstop for anything above that was not already one of the
|
||||
# named, journalled failures -- so "requested, watcher launched,
|
||||
# then nothing" can never again mean "go look at a console nobody
|
||||
# kept" (the button spawns this with no window and no output
|
||||
# redirection at all; an unjournalled traceback here is provably
|
||||
# unread by anything).
|
||||
tb = traceback.format_exc()
|
||||
journal(request, S.WATCHER_CRASHED, path=target,
|
||||
error=tb.splitlines()[-1], traceback=tb)
|
||||
print(tb, file=sys.stderr)
|
||||
return 2
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user