Compare commits

..

2 Commits

Author SHA1 Message Date
AmanTahiliani
2d6fa531f2 fix(compare): gap deltas beyond reference laps 2026-07-04 01:16:01 -04:00
AmanTahiliani
ad379e0f07 fix(compare): gap delta graph when reference lap is null
Reference-null laps now emit null deltas for all drivers and exclude that
lap window from aligned cumulative totals so later comparisons stay sane.
Challenger-null behavior is unchanged. Compare tab meta documents the policy.

Closes #46

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-04 01:04:55 -04:00
8 changed files with 118 additions and 112 deletions

View File

@@ -24,7 +24,7 @@ bin/dev CLI: `dev implement <issue#> --harness <name> [--dry-run]`
- **Groom** (interactive, Claude): `/groom <issue#>` runs a seeded grill-me → writes a - **Groom** (interactive, Claude): `/groom <issue#>` runs a seeded grill-me → writes a
Ready spec into the issue body → sets Effort/Priority → leaves Stage at `Research`. Ready spec into the issue body → sets Effort/Priority → leaves Stage at `Research`.
You review and flip to `Ready`. You review and flip to `Ready`.
- **Implement** (recommended harnesses: `codex` or `cursor`): `.agents/bin/dev implement <issue#> --harness <name>` - **Implement** (any harness): `.agents/bin/dev implement <issue#> --harness <name>`
(or `/implement …` in Claude to supervise) → isolated worktree → runs the harness (or `/implement …` in Claude to supervise) → isolated worktree → runs the harness
headless on the spec → build gate → opens a PR → sets Stage `In Review`. headless on the spec → build gate → opens a PR → sets Stage `In Review`.
- **Review + merge**: use the `review` skill from a harness different from the - **Review + merge**: use the `review` skill from a harness different from the
@@ -42,17 +42,8 @@ local adapter state.
## Adding / fixing a harness ## Adding / fixing a harness
Edit one function in `harnesses.sh`: `harness_<name> <workdir> <promptfile>`, running the Edit one function in `harnesses.sh`: `harness_<name> <workdir> <promptfile>`, running the
tool non-interactively in `<workdir>` on the prompt. For current implementation tool non-interactively in `<workdir>` on the prompt. `claude`/`codex`/`opencode` are
dispatch, prefer `codex` or `cursor`; `claude` and `opencode` remain available, and wired; `pi`/`cursor` are stubs — confirm their headless flags before trusting.
`pi` still needs flag verification before trusting.
`agy` / Antigravity is deliberately disabled for non-dry-run dispatch as of
2026-07-04. Phase 1 testing found the headless path unreliable: with
`--new-project` it ignored the prompt and tried to scaffold, while without it the CLI
could resume a stale conversation and hang past the print timeout. Keep using
`.agents/bin/dev implement <issue#> --harness agy --dry-run` for prompt inspection
only; real dispatch should use `codex` or `cursor` until a fresh Antigravity
headless invocation is verified and documented.
Always `--dry-run` a new harness first: it renders the exact prompt and plan, touching Always `--dry-run` a new harness first: it renders the exact prompt and plan, touching
nothing (no worktree, PR, or state change). nothing (no worktree, PR, or state change).

View File

@@ -1,7 +1,7 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# .agents/bin/dev — box-box agentic dev CLI. Works from any harness's shell. # .agents/bin/dev — box-box agentic dev CLI. Works from any harness's shell.
# #
# dev implement <issue#> --harness <codex|cursor|claude|opencode|pi|agy> [--dry-run] [--base <branch>] # dev implement <issue#> --harness <claude|codex|opencode|pi|cursor> [--dry-run] [--base <branch>]
# #
# Grooming is driven interactively via the Claude Code /groom skill; this CLI covers # Grooming is driven interactively via the Claude Code /groom skill; this CLI covers
# the implement lane (dispatch a Ready issue to a harness → worktree → gate → PR). # the implement lane (dispatch a Ready issue to a harness → worktree → gate → PR).
@@ -15,7 +15,7 @@ box-box dev CLI
dev implement <issue#> --harness <name> [--dry-run] [--base <branch>] dev implement <issue#> --harness <name> [--dry-run] [--base <branch>]
harnesses: codex, cursor (recommended) · claude, opencode (available) · agy (dry-run only, disabled for dispatch) · pi (verify flags) harnesses: claude, codex, opencode (supported) · pi, cursor (verify flags in .agents/harnesses.sh)
--dry-run render the prompt + plan, touch nothing (no worktree/PR/state change) --dry-run render the prompt + plan, touch nothing (no worktree/PR/state change)
--base base branch for the worktree/PR (default: main) --base base branch for the worktree/PR (default: main)
EOF EOF

View File

@@ -11,18 +11,6 @@
# ---- MUST-HAVE ---- # ---- MUST-HAVE ----
harness_disabled_reason() { # <name> -> reason on stdout; 0 means disabled
case "$1" in
agy)
cat <<'EOF'
agy is disabled for implementation dispatch as of 2026-07-04: Antigravity headless mode was observed to ignore prompts with --new-project, resume stale conversations without it, and hang past print timeouts. Use codex or cursor until a fresh headless invocation is verified.
EOF
return 0
;;
*) return 1 ;;
esac
}
harness_claude() { # Claude Code — print mode, auto-accept edits harness_claude() { # Claude Code — print mode, auto-accept edits
local dir="$1" prompt="$2" local dir="$1" prompt="$2"
( cd "$dir" && claude -p "$(cat "$prompt")" --permission-mode acceptEdits ) ( cd "$dir" && claude -p "$(cat "$prompt")" --permission-mode acceptEdits )
@@ -43,11 +31,14 @@ harness_cursor() { # Cursor CLI agent — composer-2.5, hea
( cd "$dir" && cursor-agent -p "$(cat "$prompt")" --model composer-2.5 --force --trust ) ( cd "$dir" && cursor-agent -p "$(cat "$prompt")" --model composer-2.5 --force --trust )
} }
harness_agy() { # Antigravity CLI — disabled until headless is verified harness_agy() { # Antigravity CLI — UNRELIABLE headless (2026-07): with
local reason # --new-project it ignores the prompt and asks to scaffold a project; without it, it
reason="$(harness_disabled_reason agy)" # resumes the previous conversation (silently keeping its old model — --model only
echo "harness_agy: $reason" >&2 # applies to new conversations) and can hang past the print timeout. Do not trust for
return 2 # dispatch until fixed upstream; verify with a trivial prompt first.
local dir="$1" prompt="$2"
( cd "$dir" && agy --print --print-timeout 60m \
--model="Gemini 3.1 Pro (High)" --dangerously-skip-permissions "$(cat "$prompt")" )
} }
# ---- NICE-TO-HAVE (verify the exact invocation for your version before trusting) ---- # ---- NICE-TO-HAVE (verify the exact invocation for your version before trusting) ----

View File

@@ -52,15 +52,6 @@ dispatch() { # <issue#> <harness> [--dry-run] [--base <branch>]
if ! declare -f "harness_$harness" >/dev/null 2>&1; then if ! declare -f "harness_$harness" >/dev/null 2>&1; then
echo "no adapter for harness '$harness' — add harness_$harness() to .agents/harnesses.sh" >&2; return 2 echo "no adapter for harness '$harness' — add harness_$harness() to .agents/harnesses.sh" >&2; return 2
fi fi
if [ "$dry" != 1 ] && declare -f harness_disabled_reason >/dev/null 2>&1; then
local disabled_reason
if disabled_reason="$(harness_disabled_reason "$harness")"; then
echo "harness '$harness' is disabled for non-dry-run dispatch." >&2
echo " $disabled_reason" >&2
echo " Use --dry-run for prompt inspection, or dispatch with --harness codex/cursor." >&2
return 2
fi
fi
local repo_root title body slug branch wt prompt local repo_root title body slug branch wt prompt
repo_root="$(git rev-parse --show-toplevel)" || return 1 repo_root="$(git rev-parse --show-toplevel)" || return 1

View File

@@ -1,59 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
cd "$ROOT"
# shellcheck source=/dev/null
source "$ROOT/.agents/lib/dispatch.sh"
issue_title() { echo "Disable agy harness smoke"; }
issue_body() { echo "## Spec"; echo; echo "Smoke prompt body"; }
get_field() { echo "Ready"; }
set_stage() { echo "unexpected set_stage $*" >&2; return 99; }
run_gate() { echo "unexpected run_gate $*" >&2; return 99; }
unexpected_git_file="$(mktemp "${TMPDIR:-/tmp}/boxbox-agy-git.XXXX")"
rm -f "$unexpected_git_file"
git() {
if [ "${1:-}" = "rev-parse" ]; then
command git "$@"
return
fi
echo "unexpected git $*" >&2
touch "$unexpected_git_file"
return 99
}
set +e
non_dry_output="$(dispatch 47 agy 2>&1)"
non_dry_status=$?
set -e
[ "$non_dry_status" -eq 2 ] || {
echo "expected agy non-dry-run to exit 2, got $non_dry_status" >&2
echo "$non_dry_output" >&2
exit 1
}
[[ "$non_dry_output" == *"harness 'agy' is disabled"* ]] || {
echo "expected disabled-harness message" >&2
echo "$non_dry_output" >&2
exit 1
}
[ ! -e "$unexpected_git_file" ] || {
echo "agy non-dry-run reached git before failing" >&2
echo "$non_dry_output" >&2
exit 1
}
dry_output="$(dispatch 47 agy --dry-run 2>&1)"
[[ "$dry_output" == *"[dry-run] no worktree / harness / PR / state change"* ]] || {
echo "expected agy dry-run to render dispatch preview" >&2
echo "$dry_output" >&2
exit 1
}
[[ "$dry_output" == *"Smoke prompt body"* ]] || {
echo "expected agy dry-run prompt body" >&2
echo "$dry_output" >&2
exit 1
}

View File

@@ -229,7 +229,9 @@ export function CompareView({ sessionKey, results, drivers }: Props) {
<div> <div>
<div className="compare-section-title">Race pace</div> <div className="compare-section-title">Race pace</div>
<div className="compare-section-meta"> <div className="compare-section-meta">
Cumulative lap-time delta vs {referenceLabel ?? 'reference'} Cumulative lap-time delta vs {referenceLabel ?? 'reference'}. Deltas are plotted
only where the reference lap is valid; gaps appear when the reference has no lap
time.
</div> </div>
</div> </div>
<SectionState <SectionState

View File

@@ -21,15 +21,31 @@ export function formatDeltaSeconds(delta: number): string {
return `${sign}${delta.toFixed(1)}s` return `${sign}${delta.toFixed(1)}s`
} }
function buildCumulative(lapTimes: ReadonlyArray<number | null>): number[] { /**
* Cumulative lap time aligned to reference-valid laps only.
* Laps where the reference is null are skipped for every series so later deltas
* do not compare against a frozen baseline while challengers keep accumulating.
*/
function buildAlignedCumulative(
lapTimes: ReadonlyArray<number | null>,
referenceLapTimes: ReadonlyArray<number | null>,
): number[] {
const cumulative: number[] = [] const cumulative: number[] = []
let running = 0 let running = 0
for (const lap of lapTimes) { const length = Math.max(lapTimes.length, referenceLapTimes.length)
if (lap !== null) {
for (let i = 0; i < length; i++) {
if (referenceLapTimes[i] == null) {
cumulative.push(running)
continue
}
const lap = lapTimes[i]
if (lap != null) {
running += lap running += lap
} }
cumulative.push(running) cumulative.push(running)
} }
return cumulative return cumulative
} }
@@ -47,7 +63,8 @@ function resolveReference(
/** /**
* Compute per-lap cumulative time delta for each non-reference driver. * Compute per-lap cumulative time delta for each non-reference driver.
* Positive = behind reference; negative = ahead. * Positive = behind reference; negative = ahead.
* Null laps carry cumulative forward but emit null in deltas (skip when plotting). * Deltas are only emitted where the reference lap is valid; reference-null laps
* gap every series. Challenger-null laps gap only that driver's line.
*/ */
export function computeCumulativeDeltas( export function computeCumulativeDeltas(
series: ReadonlyArray<DeltaSeries>, series: ReadonlyArray<DeltaSeries>,
@@ -56,24 +73,22 @@ export function computeCumulativeDeltas(
const reference = resolveReference(series, referenceLabel) const reference = resolveReference(series, referenceLabel)
if (!reference) return [] if (!reference) return []
const refCumulative = buildCumulative(reference.lapTimes) const refLapTimes = reference.lapTimes
const refCumulative = buildAlignedCumulative(refLapTimes, refLapTimes)
return series return series
.filter((s) => s.label !== reference.label) .filter((s) => s.label !== reference.label)
.map((driver) => { .map((driver) => {
const driverCumulative = buildCumulative(driver.lapTimes) const driverCumulative = buildAlignedCumulative(driver.lapTimes, refLapTimes)
const lapCount = Math.max(driver.lapTimes.length, refCumulative.length) const lapCount = Math.max(driver.lapTimes.length, refCumulative.length)
const deltas: (number | null)[] = [] const deltas: (number | null)[] = []
for (let i = 0; i < lapCount; i++) { for (let i = 0; i < lapCount; i++) {
if (driver.lapTimes[i] === null) { if (refLapTimes[i] == null || driver.lapTimes[i] == null) {
deltas.push(null) deltas.push(null)
continue continue
} }
const refValue = refCumulative[i] ?? refCumulative[refCumulative.length - 1] ?? 0 deltas.push(driverCumulative[i] - refCumulative[i])
const driverValue =
driverCumulative[i] ?? driverCumulative[driverCumulative.length - 1] ?? 0
deltas.push(driverValue - refValue)
} }
return { return {

View File

@@ -47,7 +47,7 @@ describe('computeCumulativeDeltas', () => {
expect(result[0].deltas[2]).toBeCloseTo(2) expect(result[0].deltas[2]).toBeCloseTo(2)
}) })
it('emits null for missing lap times while carrying cumulative forward', () => { it('emits null for challenger missing lap times while carrying cumulative forward', () => {
const withNull: DeltaSeries = { const withNull: DeltaSeries = {
label: 'NOR', label: 'NOR',
color: '#FF8000', color: '#FF8000',
@@ -60,6 +60,41 @@ describe('computeCumulativeDeltas', () => {
expect(result[0].deltas[2]).toBeCloseTo(-92) expect(result[0].deltas[2]).toBeCloseTo(-92)
}) })
it('gaps all drivers when the reference lap is null and resumes without that window', () => {
const refWithNull: DeltaSeries = {
label: 'VER',
color: '#3671C6',
lapTimes: [90, null, 92],
}
const validChallenger: DeltaSeries = {
label: 'HAM',
color: '#E8002D',
lapTimes: [89, 91, 90],
}
const result = computeCumulativeDeltas([refWithNull, validChallenger])
expect(result[0].deltas[0]).toBeCloseTo(-1)
expect(result[0].deltas[1]).toBeNull()
// Lap 3 excludes the reference-null window for both: (89+90) - (90+92) = -3
expect(result[0].deltas[2]).toBeCloseTo(-3)
})
it('gaps challenger laps beyond a shorter reference series', () => {
const shortReference: DeltaSeries = {
label: 'VER',
color: '#3671C6',
lapTimes: [90, 91],
}
const longerChallenger: DeltaSeries = {
label: 'HAM',
color: '#E8002D',
lapTimes: [89, 92, 90],
}
const result = computeCumulativeDeltas([shortReference, longerChallenger])
expect(result[0].deltas[0]).toBeCloseTo(-1)
expect(result[0].deltas[1]).toBeCloseTo(0)
expect(result[0].deltas[2]).toBeNull()
})
it('returns an empty array when only one series is provided', () => { it('returns an empty array when only one series is provided', () => {
expect(computeCumulativeDeltas([reference])).toEqual([]) expect(computeCumulativeDeltas([reference])).toEqual([])
}) })
@@ -97,6 +132,19 @@ describe('DeltaTimeGraph', () => {
expect(screen.queryByTestId('delta-line-VER')).not.toBeInTheDocument() expect(screen.queryByTestId('delta-line-VER')).not.toBeInTheDocument()
}) })
it('splits polylines at reference-null laps', () => {
const refWithNull: DeltaSeries = {
label: 'VER',
color: '#3671C6',
lapTimes: [90, null, 92],
}
const { container } = render(
<DeltaTimeGraph series={[refWithNull, challenger]} />,
)
const lines = container.querySelectorAll('.delta-graph-driver-line')
expect(lines.length).toBeGreaterThan(1)
})
it('shows a crosshair tooltip on hover', () => { it('shows a crosshair tooltip on hover', () => {
vi.spyOn(SVGSVGElement.prototype, 'getBoundingClientRect').mockReturnValue({ vi.spyOn(SVGSVGElement.prototype, 'getBoundingClientRect').mockReturnValue({
x: 0, x: 0,
@@ -118,4 +166,31 @@ describe('DeltaTimeGraph', () => {
expect(screen.getByText(/Lap 1/)).toBeInTheDocument() expect(screen.getByText(/Lap 1/)).toBeInTheDocument()
vi.restoreAllMocks() vi.restoreAllMocks()
}) })
it('omits tooltip rows on reference-null laps', () => {
vi.spyOn(SVGSVGElement.prototype, 'getBoundingClientRect').mockReturnValue({
x: 0,
y: 0,
left: 0,
top: 0,
width: 640,
height: 220,
right: 640,
bottom: 220,
toJSON: () => ({}),
})
const refWithNull: DeltaSeries = {
label: 'VER',
color: '#3671C6',
lapTimes: [90, null, 92],
}
const { container } = render(
<DeltaTimeGraph series={[refWithNull, challenger]} />,
)
const hoverLayer = container.querySelector('.delta-graph-hover-layer')
fireEvent.mouseMove(hoverLayer!, { clientX: 352, clientY: 100 })
expect(screen.getByTestId('delta-crosshair')).toBeInTheDocument()
expect(screen.queryByTestId('delta-tooltip')).not.toBeInTheDocument()
vi.restoreAllMocks()
})
}) })