Add tooling for automatic upstream sync (#83)
This commit is contained in:
@@ -447,6 +447,22 @@ publish dry="false":
|
||||
print "\n✅ All crates published!"
|
||||
|
||||
|
||||
[doc('Sync upstream Zed GPUI changes into this fork (local-only, never pushes)')]
|
||||
[group('sync')]
|
||||
sync-upstream *args:
|
||||
@python3 {{ project_root }}/scripts/sync-upstream/sync_upstream.py sync {{ args }}
|
||||
|
||||
[doc('One-time: record the upstream baseline to sync from (defaults to the pinned zed dep rev)')]
|
||||
[group('sync')]
|
||||
sync-upstream-bootstrap *args:
|
||||
@python3 {{ project_root }}/scripts/sync-upstream/sync_upstream.py bootstrap {{ args }}
|
||||
|
||||
[doc('Show how far behind upstream Zed GPUI this fork is')]
|
||||
[group('sync')]
|
||||
sync-upstream-status:
|
||||
@python3 {{ project_root }}/scripts/sync-upstream/sync_upstream.py status
|
||||
|
||||
|
||||
[doc('Generate project documentation')]
|
||||
[group('docs')]
|
||||
doc *flags:
|
||||
@@ -479,3 +495,4 @@ alias l := lint
|
||||
alias d := doc
|
||||
alias up := update
|
||||
alias cl := clean
|
||||
alias sync := sync-upstream
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
# Upstream sync
|
||||
|
||||
Automated, local-only tooling to pull GPUI changes from the upstream Zed monorepo
|
||||
(`zed-industries/zed`) into this standalone fork. Conflicts and resulting build
|
||||
breakage are resolved with `claude -p`. **Nothing is ever pushed.**
|
||||
|
||||
## TL;DR
|
||||
|
||||
```sh
|
||||
just sync-upstream-bootstrap # ONE TIME — records the baseline to sync from
|
||||
just sync-upstream # pull upstream changes onto a fresh sync/ branch
|
||||
just sync-upstream-status # how far behind upstream are we?
|
||||
```
|
||||
|
||||
`just sync-upstream` leaves a `sync/zed-<date>-<sha>` branch with the upstream delta
|
||||
merged, conflicts resolved, and the workspace compiling (or a clear report of what's
|
||||
left). Review it, then fast-forward `main` onto it or open a PR — by hand.
|
||||
|
||||
## How it works
|
||||
|
||||
Upstream keeps the GPUI crates **in-tree** in the monorepo under `crates/gpui*`. This
|
||||
fork keeps the same crates at the **same relative paths**, so upstream changes apply at
|
||||
identical paths here. The script uses a **vendor-branch 3-way merge** (a generalized
|
||||
`git subtree` merge):
|
||||
|
||||
1. A local branch `vendor/zed-gpui` holds a *filtered replay* of upstream's actual
|
||||
gpui-touching commits — each keeps its original author/date/message (plus a
|
||||
`zed-upstream: <sha>` trailer) but carries only the tracked `crates/gpui*` trees,
|
||||
built via a throwaway git index (no working-tree churn). Non-gpui and merge commits
|
||||
are dropped.
|
||||
2. Each sync extends that chain with the new commits and `git merge`s its tip. Git's
|
||||
merge base is the previous tip, so the merge replays exactly the upstream delta since
|
||||
the last sync — anything this fork already cherry-picked upstream produces **no
|
||||
conflict** — and every upstream commit is preserved in the merge's **second-parent
|
||||
history** (so `git log` shows both histories; `git log --first-parent` shows just the
|
||||
-ce line).
|
||||
3. The merge is committed as **two commits** for reviewability:
|
||||
- **Commit 1 — raw merge:** git's auto-merges applied, conflict markers committed in
|
||||
as-is (deterministic add/delete conflicts settled by policy: gpui-ce's deletions
|
||||
kept). This captures exactly what git could *not* resolve.
|
||||
- **Commit 2 — resolution:** `claude -p` (`resolve-conflicts.prompt.md`, looped up to
|
||||
`--retries`) edits out the markers. Because it's a separate commit, its diff shows
|
||||
*exactly* what was chosen — auditable in isolation, distinct from git's auto-merge.
|
||||
(A conflict-free sync is just one clean merge commit.)
|
||||
4. The pinned `zed-industries/zed` git-dep revs in the root `Cargo.toml` are bumped to
|
||||
the synced commit, then the **verify-fix loop** runs: the build gate (`just check`,
|
||||
which also treats compile **warnings** as fixable so the branch stays CI-clean), then
|
||||
the test gate (`just test`). Any failure — compile errors, warnings, or test failures —
|
||||
is handed to `claude -p` (`fix-build.prompt.md`), looped up to `--retries` times, and
|
||||
committed as a **third** commit. Disable tests with `SYNC_RUN_TESTS=0`, or warning
|
||||
enforcement with `SYNC_FAIL_ON_WARNINGS=0`.
|
||||
|
||||
### Tracked crates (fork dir ← upstream dir)
|
||||
|
||||
Synced 1:1 (same path): `gpui`, `gpui_linux`, `gpui_macos`, `gpui_macros`, `gpui_platform`,
|
||||
`gpui_shared_string`, `gpui_tokio`, `gpui_web`, `gpui_wgpu`, `gpui_windows`.
|
||||
|
||||
Synced with **path remapping** — vendored + renamed by the fork (PR #91 removed the git sources):
|
||||
`gpui_collections`←`collections`, `gpui_sum_tree`←`sum_tree`, `gpui_refineable`←`refineable`,
|
||||
`gpui_derive_refineable`←`refineable/derive_refineable`, `gpui_scheduler`←`scheduler`,
|
||||
`gpui_media`←`media`, `gpui_zed_util`←`util`, `gpui_ce_util`←`gpui_util`, `gpui_path`←`path`.
|
||||
The merge preserves each
|
||||
crate's gpui-ce adaptations (package rename, path deps, `ztracing`→`tracing`, `zlog` removal) via
|
||||
conflict resolution while taking upstream's real changes — so upstream API additions land through the
|
||||
merge instead of being hand-ported during the build-fix pass.
|
||||
|
||||
Left untouched: `crates/gpui_elements` (fork-only stub), `tooling/perf` (fork-only); `util_macros`
|
||||
is no longer used by the fork. The mapping lives in `TRACKED_CRATES` in `sync_upstream.py`.
|
||||
|
||||
### Cross-crate file moves
|
||||
|
||||
Upstream relocates code between crates (e.g. #61029 split `util/src/rel_path.rs` out into the new
|
||||
`crates/path`). Naively that reads as "upstream deleted a file the fork had modified", and the
|
||||
delete/modify policy would resurrect a stale duplicate of code that now lives elsewhere.
|
||||
|
||||
Because **both sides of a vendor-history diff are already remapped to fork paths**, git's rename
|
||||
detection reports such a move directly in gpui-ce terms — `detect_moves()` runs
|
||||
`git diff -M<similarity> --diff-filter=R <prev vendor tip> <new tip>` and keeps the cross-directory
|
||||
hits (`crates/gpui_zed_util/src/rel_path.rs` → `crates/gpui_path/src/rel_path.rs`, detected at 61%
|
||||
similarity for #61029). Those moves then:
|
||||
|
||||
- settle the resulting `UD` conflict as **accept the deletion** (the content arrived at the new path
|
||||
in the same merge) instead of keeping the fork's now-orphaned copy, and
|
||||
- get handed to the resolution prompt (rule 8), which re-applies the fork's adaptations at the **new**
|
||||
location and repoints `use`/`mod` references.
|
||||
|
||||
Tune with `SYNC_MOVE_SIMILARITY` (default `40%`). A move that falls below the threshold simply
|
||||
degrades to the old behaviour — the fork's copy is kept and flagged for review, never lost.
|
||||
|
||||
## One-time bootstrap
|
||||
|
||||
The script needs to know which upstream commit this fork currently corresponds to, to use as the
|
||||
merge base. Historically this was auto-read from the `zed-industries/zed` rev pinned in `Cargo.toml`,
|
||||
but PR #91 removed those git sources, so there is no longer a pinned rev to default to — **pass the
|
||||
baseline explicitly**:
|
||||
|
||||
```sh
|
||||
just sync-upstream-bootstrap <upstream-sha> # e.g. 876ec5a8a074 (the last rev pinned before #91)
|
||||
```
|
||||
|
||||
Bootstrap adds the `zed` remote, builds the baseline vendor snapshot (all tracked crates remapped to
|
||||
their fork paths at that rev), records it in the current branch's history with a **no-op** `-s ours`
|
||||
merge (no files change), and writes `state.json`. Run it once. A too-old baseline just means the
|
||||
first real sync has more to merge; correctness is unaffected. After the first successful sync the
|
||||
recorded baseline advances automatically, so bootstrap is not needed again.
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `sync_upstream.py` | orchestrator (git plumbing + `claude -p` loops); stdlib-only, fully typed |
|
||||
| `resolve-conflicts.prompt.md` | rules for the conflict-resolution `claude -p` pass |
|
||||
| `fix-build.prompt.md` | rules for the build-fix `claude -p` pass |
|
||||
| `state.json` | committed: last synced upstream sha + vendor tip |
|
||||
|
||||
## Config / env overrides
|
||||
|
||||
Every default near the top of `sync_upstream.py` is overridable via a `SYNC_*` env var:
|
||||
|
||||
```sh
|
||||
SYNC_MODEL=sonnet just sync-upstream # cheaper model
|
||||
SYNC_RETRIES=5 just sync-upstream # more claude passes
|
||||
SYNC_VERIFY_CMD="just ci-test" just sync-upstream
|
||||
just sync-upstream --ref some-tag --no-bump --dry-run
|
||||
```
|
||||
|
||||
## Caveats
|
||||
|
||||
- The compile gate is host-only (`just check` / `cargo check --workspace`). macOS- and
|
||||
Windows-specific changes can't be fully verified on a Linux host — verify those on the
|
||||
platform or in CI. The build-fix prompt asks claude to flag such changes.
|
||||
- Requires Python 3 (stdlib only — no pip installs), the `claude` CLI on `PATH`, and a
|
||||
working `just` + Rust toolchain. The `just` recipes shell out to `sync_upstream.py`.
|
||||
- If conflict resolution exhausts its retries, the branch is left mid-merge for you to
|
||||
finish. If the build can't be fixed in time, the merge is committed and the branch is
|
||||
left with the remaining errors plus a clear report.
|
||||
- `--allowedTools` is passed as a single space-separated string; if your `claude` CLI
|
||||
version expects a different format, adjust `SYNC_CLAUDE_ALLOWED_TOOLS`.
|
||||
@@ -0,0 +1,63 @@
|
||||
## Repository context
|
||||
|
||||
`gpui-ce` is a standalone fork of Zed's GPUI. A 3-way merge of upstream Zed GPUI changes was just
|
||||
committed, and the pinned `zed-industries/zed` git-dependency revisions were bumped to match the
|
||||
synced commit. The result has a problem the sync introduced — a compile error, a compile warning,
|
||||
or a test failure (see the output above). Fix it so the gate passes.
|
||||
|
||||
## Rules
|
||||
|
||||
1. **Fix only what the merge/sync caused.** Address the issues in the output: items moved/renamed
|
||||
upstream, changed function signatures or trait bounds, added/removed enum variants, and any
|
||||
fallout in gpui-ce's own patches. NOTE: the util crates (`collections`, `util`, `gpui_util`,
|
||||
`sum_tree`, `refineable`, `scheduler`, `media`) are now **vendored in-tree** as `gpui_collections`,
|
||||
`gpui_zed_util`, `gpui_ce_util`, `gpui_sum_tree`, `gpui_refineable`, `gpui_scheduler`,
|
||||
`gpui_media` and are **synced by this same tool** — so if gpui needs a new API from one of them,
|
||||
it should already be present from the merge. Prefer using that API; only hand-add to a vendored
|
||||
crate if the merge genuinely didn't bring it (and say so in your summary).
|
||||
|
||||
2. **Compile warnings.** Fix every compile warning the merge introduced (unused imports/variables,
|
||||
unreachable code, deprecated APIs, etc.) by addressing the **root cause** — the synced branch
|
||||
must be warning-clean to pass CI. Do **not** silence warnings with `#[allow(...)]`, `_`-prefixes,
|
||||
or `#[allow(dead_code)]` unless that is genuinely the correct fix.
|
||||
|
||||
3. **Test failures.** Fix the underlying cause. Do **not** delete tests, add `#[ignore]`, weaken or
|
||||
delete assertions, or otherwise change a test just to make it pass. If an upstream change
|
||||
legitimately changes behavior, update the test to match upstream's intent — and call that out in
|
||||
your summary. Note that some tests may fail for environmental reasons (e.g. no display); flag
|
||||
those rather than "fixing" them.
|
||||
|
||||
4. **Prefer minimal, idiomatic changes** consistent with how upstream intends the new API to be
|
||||
used, and matching the surrounding gpui-ce code style. Preserve gpui-ce's existing features
|
||||
(blur, kinetic scrolling, wgpu device-loss API, etc.); if an upstream API change requires
|
||||
updating a gpui-ce patch, update the patch correctly.
|
||||
|
||||
5. **Do not** edit `tooling/perf` or `crates/gpui_elements` unless one of them is the actual source
|
||||
of an issue. Do not run `git commit`, `git merge`, or `git push` (the surrounding script commits
|
||||
and re-runs the gate). You may run `cargo check` / `cargo build` / `cargo test` to verify. If you
|
||||
need scratch space, use `/tmp` — never write scratch files into the working tree (they would be
|
||||
committed).
|
||||
|
||||
6. If an issue stems from the **root `Cargo.toml`** (a workspace dependency that must be added or
|
||||
updated to match upstream's new requirements — the sync merges crate trees but not the root
|
||||
manifest, so new `[workspace.dependencies]` entries upstream added often need adding here), fix it
|
||||
there using gpui-ce's sourcing convention: **path deps** (`{ path = "crates/gpui_*", package =
|
||||
"gpui_*" }`) for the vendored crates, `zed-font-kit` for font-kit, and crates.io versions
|
||||
otherwise. There are no longer any `zed-industries/zed` git deps.
|
||||
|
||||
7. **Newly vendored crates need fork packaging applied.** When upstream adds a crate that this tool
|
||||
tracks, it arrives as a *clean add* — no conflict, so no resolution pass adapted it, and it still
|
||||
carries upstream's packaging verbatim. If a tracked crate directory is new in this merge, bring it
|
||||
in line with its siblings before anything else: set `name` to the fork's `gpui_*` name (keeping
|
||||
`[lib] name` as the upstream crate name so `use` sites are unchanged), match the siblings'
|
||||
`version`/`edition`/`publish`/`description`/`repository`, convert workspace/git deps to gpui-ce's
|
||||
sourcing convention (rule 6), add the crate to the root `Cargo.toml` members, and **set
|
||||
`license = "Apache-2.0"`** — gpui-ce is Apache-only, and an upstream manifest may declare another
|
||||
license (or contradict its own bundled license file). Also copy a sibling's `LICENSE-APACHE` file
|
||||
into the new crate dir; upstream ships that as a symlink to a root file gpui-ce doesn't have, so
|
||||
the link would dangle. Call out in your summary anything whose license looks non-Apache, and do
|
||||
NOT silently vendor code that genuinely is — flag it for a human instead.
|
||||
|
||||
When finished, briefly summarize the fixes and anything a human should double-check (especially
|
||||
changes to macOS/Windows-only code that this host can't fully compile, and any tests you judged to
|
||||
be failing for environmental rather than correctness reasons).
|
||||
@@ -0,0 +1,89 @@
|
||||
## Repository context
|
||||
|
||||
`gpui-ce` is a standalone community fork of Zed's GPUI. It vendors crates from the upstream Zed
|
||||
monorepo (`zed-industries/zed`). Two groups:
|
||||
|
||||
**Same relative path** (upstream dir == fork dir): `crates/gpui`, `crates/gpui_linux`,
|
||||
`crates/gpui_macos`, `crates/gpui_macros`, `crates/gpui_platform`, `crates/gpui_shared_string`,
|
||||
`crates/gpui_tokio`, `crates/gpui_web`, `crates/gpui_wgpu`, `crates/gpui_windows`.
|
||||
|
||||
**Vendored + renamed** (upstream dir → fork dir), formerly pulled as `zed-industries/zed` git
|
||||
deps but now vendored in-tree by the fork:
|
||||
`collections`→`gpui_collections`, `sum_tree`→`gpui_sum_tree`, `refineable`→`gpui_refineable`,
|
||||
`refineable/derive_refineable`→`gpui_derive_refineable`, `scheduler`→`gpui_scheduler`,
|
||||
`media`→`gpui_media`, `util`→`gpui_zed_util`, `gpui_util`→`gpui_ce_util`, `path`→`gpui_path`.
|
||||
The sync remaps upstream's
|
||||
content into these fork dirs, so a conflict here is upstream's version of the crate vs. the fork's
|
||||
vendored+adapted version. Preserve the fork's adaptations (see rule 4) while taking upstream's real
|
||||
changes. (`util_macros` is no longer used by the fork; `gpui_elements` and `tooling/perf` are
|
||||
fork-only and never synced.)
|
||||
|
||||
A 3-way `git merge` of the upstream delta produced conflicts, and the raw merge — with conflict
|
||||
markers committed in — is already its own commit. Your job is to resolve **every** marker in the
|
||||
listed files so the result is correct gpui-ce code that incorporates the upstream changes. Your
|
||||
edits land as a **separate, reviewable resolution commit** diffed against that raw merge.
|
||||
|
||||
## Rules
|
||||
|
||||
1. **Resolve every conflict marker** (`<<<<<<<`, `=======`, `>>>>>>>`, `|||||||`) in the listed
|
||||
files. Leave no markers behind. Do not touch files that aren't conflicted.
|
||||
|
||||
2. **gpui-ce keeps its own patches.** gpui-ce carries features/fixes not yet upstream (e.g. blur
|
||||
filters, kinetic scrolling on Wayland, the wgpu device-loss API). When a conflict pits an
|
||||
upstream change against a gpui-ce patch, **keep both behaviours** — integrate the upstream
|
||||
change around gpui-ce's additions rather than dropping either. Only drop a gpui-ce line if the
|
||||
upstream change genuinely supersedes it.
|
||||
|
||||
3. **Already-present (cherry-picked) changes.** gpui-ce frequently contributes to and cherry-picks
|
||||
from upstream, so an upstream commit may already be present here under a different hash. If a
|
||||
conflict exists *only* because the change is **already applied** in gpui-ce (semantically
|
||||
equivalent, even if worded differently), keep gpui-ce's version and do **not** duplicate the code.
|
||||
|
||||
4. **Vendored crate adaptations — preserve them.** The vendored+renamed crates carry mechanical
|
||||
gpui-ce adaptations on top of upstream. When a conflict pits upstream against one of these, KEEP
|
||||
the adaptation and take upstream's real code change around it:
|
||||
- **Package rename:** the package name is the fork's `gpui_*` name (e.g. `gpui_collections`, not
|
||||
`collections`), with the fork's `publish`/version/workspace metadata. Do **not** revert to
|
||||
upstream's package name.
|
||||
- **Path deps:** intra-fork deps are `{ path = "crates/gpui_*", package = "gpui_*" }`, not git or
|
||||
crates.io deps. Upstream referring to a sibling as `collections`/`util`/`sum_tree`/etc. maps to
|
||||
the fork's `gpui_*` path dep. Do not convert fork path deps back to git/registry deps.
|
||||
- **Stripped zed-internal crates:** the fork replaces zed-only infra with std/community crates —
|
||||
e.g. `ztracing`→`tracing`, and `zlog`/`zlog::init_test()` test-logger blocks are removed. Keep
|
||||
these substitutions; don't reintroduce `ztracing`/`zlog`.
|
||||
|
||||
5. **`Cargo.toml` (per-crate and root):**
|
||||
- KEEP gpui-ce packaging (names, `publish`, `edition`, workspace metadata) and the fork's dep
|
||||
*sources* (path deps for the vendored crates; `zed-font-kit` for font-kit).
|
||||
- **ADOPT upstream's real changes: newly added/removed dependencies, new features, new
|
||||
`[target.'cfg(...)']` blocks, and — importantly — dependency VERSION BUMPS.** If upstream bumped
|
||||
a crate (e.g. `resvg`/`usvg` 0.45→0.46, `taffy`, `accesskit`), take the new version; a bumped
|
||||
dep is often paired with a regression test that only passes on the new version. Wire any newly
|
||||
required workspace dependency through the fork's convention.
|
||||
|
||||
6. **Removed Zed-app / AGPL code.** gpui-ce stripped Zed-application-specific and non-Apache code.
|
||||
If an upstream change references a crate or module that doesn't exist in gpui-ce (e.g.
|
||||
`http_client`, `reqwest_client`, `util_macros`), drop that reference rather than reintroducing the
|
||||
removed code.
|
||||
|
||||
7. **Add/delete conflicts are handled for you** — the script settles `modify/delete` cases
|
||||
(files gpui-ce deleted that upstream changed are kept deleted) before calling you, so you
|
||||
only ever see content conflicts. Don't recreate a deleted file.
|
||||
|
||||
8. **Relocated files.** When upstream MOVES a file to another crate, the prompt lists it above as
|
||||
`old path → new path`. The script has already deleted the old path and the merge has already
|
||||
brought in the new one, so the code is not lost — but any **gpui-ce adaptation that lived in the
|
||||
old file is**. For each listed move: diff what the fork had at the old path against what arrived
|
||||
at the new one, re-apply the fork's adaptations (rule 4) at the NEW location, and update every
|
||||
`use`/`mod`/path reference to point there. Never leave the old copy behind alongside the new one —
|
||||
a duplicated type is worse than either version alone. Do **not** re-create the old file.
|
||||
|
||||
9. **Do not** run `git commit`, `git merge`, `git rebase`, or `git push`. Only edit files to
|
||||
resolve the conflicts — the surrounding script stages and commits. Do not change anything
|
||||
unrelated to the conflicts.
|
||||
|
||||
10. **No scratch files in the repo.** If you need to save a base/upstream/fork copy of a file to
|
||||
diff it, write it under `/tmp`, never inside the working tree (a stray `.merge_tmp/` or similar
|
||||
would be committed). Resolve strictly by editing the conflicted files in place.
|
||||
|
||||
When finished, briefly summarize what you resolved and any decisions worth a human's attention.
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"last_synced_sha": "",
|
||||
"vendor_tip": "",
|
||||
"last_synced_date": "",
|
||||
"upstream_url": "https://github.com/zed-industries/zed.git",
|
||||
"upstream_ref": "main"
|
||||
}
|
||||
Executable
+826
@@ -0,0 +1,826 @@
|
||||
#!/usr/bin/env python3
|
||||
"""sync_upstream.py — bring upstream Zed GPUI changes into the standalone gpui-ce fork.
|
||||
|
||||
Strategy: a vendor-branch 3-way (subtree-style) merge that preserves upstream history.
|
||||
A local branch (vendor/zed-gpui) holds a *filtered replay* of upstream's actual
|
||||
gpui-touching commits — each preserving its author/date/message (plus a `zed-upstream:`
|
||||
trailer) but carrying only the tracked crates/gpui* trees. Merging its tip replays exactly
|
||||
the upstream delta since the last sync (already-cherry-picked changes no-op) while every
|
||||
upstream commit stays in the merge's second-parent history. A conflicted merge is committed
|
||||
as TWO commits for reviewability: the raw merge with conflict markers committed in, then
|
||||
claude's resolution as a separate, auditable diff. Build fixes land in a third commit.
|
||||
|
||||
Local-only: this never pushes, and never lets claude push.
|
||||
|
||||
Usage:
|
||||
sync_upstream.py bootstrap [BASELINE_SHA] # one-time setup (see README)
|
||||
sync_upstream.py sync [REF] [options] # default subcommand
|
||||
sync_upstream.py status
|
||||
|
||||
Options (sync):
|
||||
--ref REF upstream ref to sync to (default: main / SYNC_ZED_REF)
|
||||
--model NAME claude model (default: opus / SYNC_MODEL)
|
||||
--retries N max claude passes per phase (default: 3 / SYNC_RETRIES)
|
||||
--no-bump don't bump pinned zed git-dep revs
|
||||
--dry-run show what would be synced, then stop
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import NoReturn, cast
|
||||
|
||||
# locate repo
|
||||
SCRIPT_DIR: Path = Path(__file__).resolve().parent
|
||||
REPO_ROOT: Path = Path(
|
||||
subprocess.run(
|
||||
["git", "-C", str(SCRIPT_DIR), "rev-parse", "--show-toplevel"],
|
||||
capture_output=True, text=True, check=True,
|
||||
).stdout.strip()
|
||||
)
|
||||
|
||||
|
||||
# static config (env-overridable)
|
||||
def _env(name: str, default: str) -> str:
|
||||
return os.environ.get(name, default)
|
||||
|
||||
|
||||
# NOTE: `zed` is a SEPARATE remote from `upstream` (which points at gpui-ce/gpui-ce).
|
||||
ZED_REMOTE_NAME: str = _env("SYNC_ZED_REMOTE_NAME", "zed")
|
||||
ZED_REMOTE_URL: str = _env("SYNC_ZED_REMOTE_URL", "https://github.com/zed-industries/zed.git")
|
||||
|
||||
# Fork crate dir (under crates/) -> upstream crate path (under crates/). The in-tree gpui*
|
||||
# crates map to themselves; the rest were vendored + renamed from the Zed monorepo by PR #91
|
||||
# ("removed all of the git sources") and are tracked via path remapping. Their gpui-ce
|
||||
# adaptations (package rename, path deps, ztracing->tracing, zlog removal, etc.) are
|
||||
# preserved through the 3-way merge's conflict resolution — never re-applied by hand.
|
||||
# Left untouched: gpui_elements (fork-only stub), tooling/perf (fork-only). util_macros is
|
||||
# no longer used by the fork.
|
||||
TRACKED_CRATES: dict[str, str] = {
|
||||
# in-tree gpui crates (identity mapping)
|
||||
"gpui": "gpui",
|
||||
"gpui_linux": "gpui_linux",
|
||||
"gpui_macos": "gpui_macos",
|
||||
"gpui_macros": "gpui_macros",
|
||||
"gpui_platform": "gpui_platform",
|
||||
"gpui_shared_string": "gpui_shared_string",
|
||||
"gpui_tokio": "gpui_tokio",
|
||||
"gpui_web": "gpui_web",
|
||||
"gpui_wgpu": "gpui_wgpu",
|
||||
"gpui_windows": "gpui_windows",
|
||||
# vendored + renamed from upstream (fork dir -> upstream path)
|
||||
"gpui_collections": "collections",
|
||||
"gpui_sum_tree": "sum_tree",
|
||||
"gpui_refineable": "refineable",
|
||||
"gpui_derive_refineable": "refineable/derive_refineable", # nested upstream
|
||||
"gpui_scheduler": "scheduler",
|
||||
"gpui_media": "media",
|
||||
"gpui_zed_util": "util",
|
||||
"gpui_ce_util": "gpui_util",
|
||||
# Created mid-range by upstream's "Split out `RelPath` into a separate crate" (#61029),
|
||||
# which MOVED crates/util/src/rel_path.rs into it. Tracked so the split arrives through the
|
||||
# merge (see MOVE detection below) instead of leaving a stale duplicate in gpui_zed_util.
|
||||
"gpui_path": "path",
|
||||
}
|
||||
|
||||
VENDOR_BRANCH: str = _env("SYNC_VENDOR_BRANCH", "vendor/zed-gpui")
|
||||
|
||||
CLAUDE_BIN: str = _env("SYNC_CLAUDE_BIN", "claude")
|
||||
# Per-invocation wall-clock cap (seconds); 0 disables. The real safety bound — hitting it
|
||||
# is non-fatal (the loop re-checks progress and retries).
|
||||
CLAUDE_TIMEOUT: int = int(_env("SYNC_CLAUDE_TIMEOUT", "1800"))
|
||||
# Optional cap on claude's agentic turns per invocation (0 = no cap, the default).
|
||||
CLAUDE_MAX_TURNS: int = int(_env("SYNC_CLAUDE_MAX_TURNS", "0"))
|
||||
# Edits/writes are auto-accepted via --permission-mode acceptEdits; the rest are
|
||||
# read-only/build helpers. Git staging/commits are done here, never by claude; push is
|
||||
# never allowed.
|
||||
_DEFAULT_ALLOWED_TOOLS: str = (
|
||||
"Read Edit Write Grep Glob Bash(cargo check:*) Bash(cargo build:*) "
|
||||
"Bash(cargo metadata:*) Bash(git status:*) Bash(git diff:*) Bash(git log:*) "
|
||||
"Bash(rg:*) Bash(grep:*) Bash(ls:*) Bash(cat:*) Bash(sed:*) Bash(find:*)"
|
||||
)
|
||||
CLAUDE_ALLOWED_TOOLS: str = _env("SYNC_CLAUDE_ALLOWED_TOOLS", _DEFAULT_ALLOWED_TOOLS)
|
||||
|
||||
# Post-merge gates (host-buildable crates only; macOS/Windows changes need their own
|
||||
# platform or CI). The verify-fix loop runs the build gate, then the test gate.
|
||||
VERIFY_CMD: str = _env("SYNC_VERIFY_CMD", "just check")
|
||||
TEST_CMD: str = _env("SYNC_TEST_CMD", "just test")
|
||||
RUN_TESTS: bool = _env("SYNC_RUN_TESTS", "1") == "1"
|
||||
# Treat compile warnings as a fixable condition (the synced branch must be warning-clean to
|
||||
# pass CI, which denies warnings). Set 0 if pre-existing warnings cause churn.
|
||||
FAIL_ON_WARNINGS: bool = _env("SYNC_FAIL_ON_WARNINGS", "1") == "1"
|
||||
|
||||
STATE_FILE: Path = REPO_ROOT / "scripts" / "sync-upstream" / "state.json"
|
||||
|
||||
|
||||
# runtime config (env defaults; overridable by CLI flags / during a run)
|
||||
@dataclass
|
||||
class Runtime:
|
||||
zed_ref: str
|
||||
model: str
|
||||
retries: int
|
||||
bump_zed_deps: bool
|
||||
dry_run: bool
|
||||
build_log: str = ""
|
||||
|
||||
|
||||
RT = Runtime(
|
||||
zed_ref=_env("SYNC_ZED_REF", "main"),
|
||||
model=_env("SYNC_MODEL", "opus"),
|
||||
retries=int(_env("SYNC_RETRIES", "3")),
|
||||
bump_zed_deps=_env("SYNC_BUMP_ZED_DEPS", "1") == "1",
|
||||
dry_run=False,
|
||||
)
|
||||
|
||||
|
||||
# output
|
||||
_COLOR: bool = sys.stdout.isatty()
|
||||
|
||||
|
||||
def _c(code: str) -> str:
|
||||
return code if _COLOR else ""
|
||||
|
||||
|
||||
_B, _G, _Y, _R, _D, _Z = (
|
||||
_c("\033[34m"), _c("\033[32m"), _c("\033[33m"), _c("\033[31m"), _c("\033[2m"), _c("\033[0m"),
|
||||
)
|
||||
|
||||
|
||||
def log(msg: str) -> None:
|
||||
print(f"{_B}▶{_Z} {msg}", flush=True)
|
||||
|
||||
|
||||
def ok(msg: str) -> None:
|
||||
print(f"{_G}✓{_Z} {msg}", flush=True)
|
||||
|
||||
|
||||
def warn(msg: str) -> None:
|
||||
print(f"{_Y}⚠{_Z} {msg}", file=sys.stderr, flush=True)
|
||||
|
||||
|
||||
class SyncError(Exception):
|
||||
"""Fatal, user-facing error; caught in main() and printed without a traceback."""
|
||||
|
||||
|
||||
def die(msg: str) -> NoReturn:
|
||||
raise SyncError(msg)
|
||||
|
||||
|
||||
# git helpers
|
||||
def git(
|
||||
*args: str,
|
||||
check: bool = True,
|
||||
capture: bool = True,
|
||||
env: dict[str, str] | None = None,
|
||||
input_text: str | None = None,
|
||||
) -> subprocess.CompletedProcess[str]:
|
||||
full_env: dict[str, str] = dict(os.environ)
|
||||
if env:
|
||||
full_env.update(env)
|
||||
result = subprocess.run(
|
||||
["git", *args],
|
||||
cwd=REPO_ROOT,
|
||||
env=full_env,
|
||||
input=input_text,
|
||||
text=True,
|
||||
stdout=subprocess.PIPE if capture else None,
|
||||
stderr=subprocess.PIPE if capture else None,
|
||||
)
|
||||
if check and result.returncode != 0:
|
||||
detail = (result.stderr or "").strip()
|
||||
die(f"`git {' '.join(args)}` failed (exit {result.returncode})"
|
||||
+ (f":\n{detail}" if detail else ""))
|
||||
return result
|
||||
|
||||
|
||||
def run_git(
|
||||
*args: str,
|
||||
check: bool = True,
|
||||
capture: bool = True,
|
||||
env: dict[str, str] | None = None,
|
||||
input_text: str | None = None,
|
||||
) -> None:
|
||||
"""git() for side-effect-only calls (result intentionally discarded)."""
|
||||
_ = git(*args, check=check, capture=capture, env=env, input_text=input_text)
|
||||
|
||||
|
||||
def gout(*args: str, env: dict[str, str] | None = None) -> str:
|
||||
return git(*args, env=env).stdout.strip()
|
||||
|
||||
|
||||
def gok(*args: str) -> bool:
|
||||
return git(*args, check=False).returncode == 0
|
||||
|
||||
|
||||
# vendor history (filtered replay of upstream commits)
|
||||
def tracked_pathspec() -> list[str]:
|
||||
"""Upstream pathspecs for the tracked crates (used to filter upstream rev-lists)."""
|
||||
return [f"crates/{up}" for up in TRACKED_CRATES.values()]
|
||||
|
||||
|
||||
def filtered_tree(sha: str) -> str | None:
|
||||
"""Tree object holding ONLY the tracked crates from <sha>, each placed at its FORK path
|
||||
(remapping upstream dirs -> gpui-ce's renamed dirs). None if none are present.
|
||||
|
||||
Uses a throwaway index, so there's no working-tree churn.
|
||||
"""
|
||||
tmp = tempfile.mkdtemp()
|
||||
env = {"GIT_INDEX_FILE": os.path.join(tmp, "index")}
|
||||
try:
|
||||
run_git("read-tree", "--empty", env=env)
|
||||
added = 0
|
||||
for fork_path, up_path in TRACKED_CRATES.items():
|
||||
if gok("cat-file", "-e", f"{sha}:crates/{up_path}"):
|
||||
run_git("read-tree", f"--prefix=crates/{fork_path}/", f"{sha}:crates/{up_path}", env=env)
|
||||
added += 1
|
||||
if added == 0:
|
||||
return None
|
||||
# A tracked upstream path can nest inside another (refineable/derive_refineable lives
|
||||
# under refineable). The parent read pulled the nested subtree into the parent's fork
|
||||
# prefix; drop it so each crate holds only its own files.
|
||||
for fork_path, up_path in TRACKED_CRATES.items():
|
||||
for other_up in TRACKED_CRATES.values():
|
||||
if other_up != up_path and other_up.startswith(up_path + "/"):
|
||||
rel = other_up[len(up_path) + 1:]
|
||||
run_git("rm", "-r", "-q", "--cached", "--ignore-unmatch", "--",
|
||||
f"crates/{fork_path}/{rel}", env=env, check=False)
|
||||
return gout("write-tree", env=env)
|
||||
finally:
|
||||
shutil.rmtree(tmp, ignore_errors=True)
|
||||
|
||||
|
||||
def build_vendor_snapshot(sha: str, parent: str | None = None) -> str:
|
||||
"""Bootstrap baseline: a single filtered snapshot commit. Echoes the commit sha."""
|
||||
tree = filtered_tree(sha)
|
||||
if tree is None:
|
||||
die(f"no tracked crates found at {sha[:12]}")
|
||||
parent_args = ["-p", parent] if parent else []
|
||||
msg = f"vendor: zed gpui baseline @ {sha[:12]}\n"
|
||||
return git("commit-tree", tree, *parent_args, input_text=msg).stdout.strip()
|
||||
|
||||
|
||||
def replay_commit(tree: str, parent: str, original: str) -> str:
|
||||
"""Filtered commit preserving <original>'s author/committer/dates/message + trailer."""
|
||||
message = gout("show", "-s", "--format=%B", original)
|
||||
env = {
|
||||
"GIT_AUTHOR_NAME": gout("show", "-s", "--format=%an", original),
|
||||
"GIT_AUTHOR_EMAIL": gout("show", "-s", "--format=%ae", original),
|
||||
"GIT_AUTHOR_DATE": gout("show", "-s", "--format=%aI", original),
|
||||
"GIT_COMMITTER_NAME": gout("show", "-s", "--format=%cn", original),
|
||||
"GIT_COMMITTER_EMAIL": gout("show", "-s", "--format=%ce", original),
|
||||
"GIT_COMMITTER_DATE": gout("show", "-s", "--format=%cI", original),
|
||||
}
|
||||
body = f"{message}\n\nzed-upstream: {original}\n"
|
||||
return git("commit-tree", tree, "-p", parent, env=env, input_text=body).stdout.strip()
|
||||
|
||||
|
||||
def build_vendor_history(parent: str, frm: str, to: str) -> str:
|
||||
"""Replay upstream gpui-touching commits frm..to as a filtered chain onto <parent>.
|
||||
|
||||
Merge commits and commits with no net change inside the tracked crates are dropped.
|
||||
Echoes the new tip sha.
|
||||
"""
|
||||
prev = parent
|
||||
replayed = 0
|
||||
listing = gout("rev-list", "--reverse", "--topo-order", "--no-merges",
|
||||
f"{frm}..{to}", "--", *tracked_pathspec())
|
||||
for commit in listing.split():
|
||||
tree = filtered_tree(commit)
|
||||
if tree is None:
|
||||
continue
|
||||
if tree == gout("rev-parse", f"{prev}^{{tree}}"):
|
||||
continue # no net change inside the tracked crates
|
||||
prev = replay_commit(tree, prev, commit)
|
||||
replayed += 1
|
||||
log(f"replayed {replayed} upstream gpui commit(s) into vendor history")
|
||||
return prev
|
||||
|
||||
|
||||
# cross-crate file moves
|
||||
# Upstream regularly relocates code between crates (e.g. #61029 split crates/util/src/rel_path.rs
|
||||
# out into the new crates/path). Because BOTH sides of a vendor-history diff are already remapped
|
||||
# to this fork's paths, git's rename detection reports such a move in fork terms
|
||||
# (crates/gpui_zed_util/src/rel_path.rs -> crates/gpui_path/src/rel_path.rs) — which is exactly
|
||||
# what's needed to settle the resulting delete/modify conflict correctly.
|
||||
MOVE_SIMILARITY: str = _env("SYNC_MOVE_SIMILARITY", "40%")
|
||||
|
||||
|
||||
def detect_moves(base: str, tip: str) -> dict[str, str]:
|
||||
"""Map old fork path -> new fork path for files upstream relocated across tracked crates.
|
||||
|
||||
Only cross-directory moves are reported; a pure in-place rewrite isn't a move.
|
||||
"""
|
||||
out = gout("diff", f"-M{MOVE_SIMILARITY}", "--name-status", "--diff-filter=R", base, tip)
|
||||
moves: dict[str, str] = {}
|
||||
for line in out.splitlines():
|
||||
parts = line.split("\t")
|
||||
if len(parts) != 3:
|
||||
continue
|
||||
_, old, new = parts
|
||||
if os.path.dirname(old) != os.path.dirname(new):
|
||||
moves[old] = new
|
||||
for old, new in moves.items():
|
||||
log(f" upstream moved {old} → {new}")
|
||||
if moves:
|
||||
ok(f"detected {len(moves)} cross-crate file move(s) in this range")
|
||||
return moves
|
||||
|
||||
|
||||
# claude invocation
|
||||
def render_prompt(kind: str, files: list[str], issue: str = "",
|
||||
moves: dict[str, str] | None = None) -> str:
|
||||
if kind == "resolve":
|
||||
head = (
|
||||
"You are resolving git merge conflicts from syncing upstream Zed's GPUI crates "
|
||||
"into the standalone `gpui-ce` fork. A merge commit with the conflict markers "
|
||||
"committed in already exists; edit the working-tree files to remove every marker. "
|
||||
"Your edits become a SEPARATE resolution commit that will be reviewed in "
|
||||
"isolation, so resolve faithfully."
|
||||
)
|
||||
listing = "\n".join(f" - {f}" for f in files)
|
||||
moves_block = ""
|
||||
if moves:
|
||||
relocated = "\n".join(f" - {old} → {new}" for old, new in moves.items())
|
||||
moves_block = (
|
||||
"\n\nUpstream RELOCATED these files across crates in this range (the old path has\n"
|
||||
"already been removed for you; the new one arrived via the merge). Carry any gpui-ce\n"
|
||||
"adaptation that lived in the old file over to the new location, and make sure nothing\n"
|
||||
"still refers to the old path:\n" + relocated
|
||||
)
|
||||
body = (SCRIPT_DIR / "resolve-conflicts.prompt.md").read_text()
|
||||
return f"{head}\n\nConflicted / unresolved files:\n{listing}{moves_block}\n\n{body}"
|
||||
head = (
|
||||
f"You are fixing {issue or 'build issues'} that the upstream Zed GPUI merge introduced "
|
||||
"in `gpui-ce`. The merge is already committed; fix only what the merge/sync caused."
|
||||
)
|
||||
cmd_used = TEST_CMD if issue == "test failures" else VERIFY_CMD
|
||||
tail = "\n".join(RT.build_log.splitlines()[-300:])
|
||||
body = (SCRIPT_DIR / "fix-build.prompt.md").read_text()
|
||||
block = f"```\n{tail}\n```"
|
||||
return f"{head}\n\nCommand: `{cmd_used}`\nRecent output (tail):\n{block}\n\n{body}"
|
||||
|
||||
|
||||
def run_claude(prompt: str) -> None:
|
||||
"""Invoke claude -p. Never fatal: max-turns/timeout exit non-zero AFTER useful work, so
|
||||
the surrounding loop re-checks real progress and retries up to RT.retries."""
|
||||
cmd = [
|
||||
CLAUDE_BIN, "-p", prompt,
|
||||
"--model", RT.model,
|
||||
"--permission-mode", "acceptEdits",
|
||||
"--allowedTools", CLAUDE_ALLOWED_TOOLS,
|
||||
]
|
||||
if CLAUDE_MAX_TURNS > 0:
|
||||
cmd += ["--max-turns", str(CLAUDE_MAX_TURNS)]
|
||||
try:
|
||||
result = subprocess.run(cmd, cwd=REPO_ROOT, timeout=CLAUDE_TIMEOUT or None)
|
||||
if result.returncode != 0:
|
||||
warn(f"claude exited non-zero ({result.returncode}; likely --max-turns) — re-checking progress, may retry")
|
||||
except subprocess.TimeoutExpired:
|
||||
warn(f"claude hit the {CLAUDE_TIMEOUT}s timeout — re-checking progress, may retry")
|
||||
|
||||
|
||||
# conflict detection / resolution
|
||||
_MARKER_RE = r"^(<{7}|>{7}|\|{7})"
|
||||
|
||||
|
||||
def _unmerged() -> list[str]:
|
||||
out = gout("diff", "--name-only", "--diff-filter=U")
|
||||
return out.split() if out else []
|
||||
|
||||
|
||||
def _marker_files() -> list[str]:
|
||||
result = git("grep", "-lE", _MARKER_RE, "--", "crates/", check=False)
|
||||
out = result.stdout.strip()
|
||||
return out.split() if (result.returncode == 0 and out) else []
|
||||
|
||||
|
||||
def conflicted_files() -> list[str]:
|
||||
return sorted(set(_unmerged()) | set(_marker_files()))
|
||||
|
||||
|
||||
def has_unresolved() -> bool:
|
||||
return bool(_unmerged()) or bool(_marker_files())
|
||||
|
||||
|
||||
def auto_resolve_add_delete(moves: dict[str, str] | None = None) -> None:
|
||||
"""Settle add/delete conflicts claude can't express via edits:
|
||||
DU = deleted by us (gpui-ce), modified by them -> keep gpui-ce's deletion;
|
||||
UD = modified by us, deleted by them -> keep gpui-ce's version (flag),
|
||||
UNLESS upstream merely RELOCATED the file to another tracked crate, in which case
|
||||
the deletion is accepted (the content arrived at the new path via this same merge)
|
||||
and claude is told to carry the fork's adaptations across.
|
||||
"""
|
||||
moves = moves or {}
|
||||
handled = False
|
||||
for line in gout("status", "--porcelain").splitlines():
|
||||
if len(line) < 4:
|
||||
continue
|
||||
code, path = line[:2], line[3:]
|
||||
if code == "DU":
|
||||
log(f" modify/delete — keeping gpui-ce's deletion of {path}")
|
||||
run_git("rm", "-q", "--force", "--", path, check=False)
|
||||
handled = True
|
||||
elif code == "UD":
|
||||
dest = moves.get(path)
|
||||
if dest and (REPO_ROOT / dest).exists():
|
||||
log(f" delete/modify — upstream MOVED {path} → {dest}; accepting the deletion")
|
||||
run_git("rm", "-q", "--force", "--", path, check=False)
|
||||
else:
|
||||
warn(f" delete/modify — upstream deleted {path}; keeping gpui-ce's version (review)")
|
||||
run_git("add", "--", path, check=False)
|
||||
handled = True
|
||||
if handled:
|
||||
ok("auto-resolved add/delete conflicts")
|
||||
|
||||
|
||||
def resolve_conflicts_loop(branch: str, moves: dict[str, str] | None = None) -> None:
|
||||
attempt = 0
|
||||
while has_unresolved():
|
||||
attempt += 1
|
||||
if attempt > RT.retries:
|
||||
warn(f"still unresolved after {RT.retries} attempt(s):")
|
||||
for path in conflicted_files():
|
||||
print(f" {path}", file=sys.stderr)
|
||||
die(f"conflict resolution failed; branch '{branch}' left for manual finishing")
|
||||
files = conflicted_files()
|
||||
before = len(files)
|
||||
log(f"claude conflict-resolution pass {attempt}/{RT.retries} ({before} file(s) remaining)")
|
||||
for path in files:
|
||||
print(f" {_D}conflict:{_Z} {path}")
|
||||
run_claude(render_prompt("resolve", files, moves=moves))
|
||||
# Stage only tracked updates (resolves the conflicted files). NOT `-A`: conflict
|
||||
# resolution edits existing files, so any new untracked file is claude's scratch
|
||||
# (e.g. saved base/upstream copies for diffing) and must not be swept into the commit.
|
||||
run_git("add", "-u")
|
||||
after = len(conflicted_files())
|
||||
if after > 0 and after >= before:
|
||||
warn(f"no progress this pass ({before} → {after} files) — claude may be stuck")
|
||||
|
||||
|
||||
# dependency bump
|
||||
def bump_zed_deps(target: str) -> None:
|
||||
log(f"bumping zed-industries/zed git-dep revs → {target[:12]}")
|
||||
path = REPO_ROOT / "Cargo.toml"
|
||||
lines = path.read_text().splitlines(keepends=True)
|
||||
bumped = 0
|
||||
for i, line in enumerate(lines):
|
||||
if 'github.com/zed-industries/zed"' in line:
|
||||
new_line, count = re.subn(r'rev = "[0-9a-fA-F]+"', f'rev = "{target}"', line)
|
||||
if count:
|
||||
lines[i] = new_line
|
||||
bumped += 1
|
||||
_ = path.write_text("".join(lines))
|
||||
ok(f"rewrote {bumped} zed dep rev(s) in Cargo.toml")
|
||||
|
||||
|
||||
# build / test verification + fixing
|
||||
# A compile warning line: `warning: ...` or `warning[...]` at the start of a line.
|
||||
_WARNING_RE = r"(?m)^warning(:|\[)"
|
||||
|
||||
|
||||
def _run_gate(cmd: str) -> tuple[int, str]:
|
||||
result = subprocess.run(
|
||||
cmd, cwd=REPO_ROOT, shell=True, text=True,
|
||||
stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
|
||||
)
|
||||
out = result.stdout or ""
|
||||
print(out, end="")
|
||||
return result.returncode, out
|
||||
|
||||
|
||||
def _build_issue() -> str:
|
||||
"""Run the compile gate; '' if clean, else a short issue label (errors or warnings)."""
|
||||
log(f"verifying build: {VERIFY_CMD}")
|
||||
rc, out = _run_gate(VERIFY_CMD)
|
||||
RT.build_log = out
|
||||
if rc != 0:
|
||||
return "compile errors"
|
||||
if FAIL_ON_WARNINGS and re.search(_WARNING_RE, out) is not None:
|
||||
return "compile warnings"
|
||||
return ""
|
||||
|
||||
|
||||
def _test_issue() -> str:
|
||||
"""Run the test gate; '' if it passes, else 'test failures'."""
|
||||
log(f"running tests: {TEST_CMD}")
|
||||
rc, out = _run_gate(TEST_CMD)
|
||||
RT.build_log = out
|
||||
return "test failures" if rc != 0 else ""
|
||||
|
||||
|
||||
def verify_fix_loop() -> bool:
|
||||
"""Loop the build (compile errors + warnings) and test gates, fixing failures with claude.
|
||||
|
||||
The build gate runs first; only once it's clean do we run tests. Any failing gate's
|
||||
output is handed to claude, bounded by RT.retries total fix passes.
|
||||
"""
|
||||
attempt = 0
|
||||
while True:
|
||||
issue = _build_issue()
|
||||
if not issue and RUN_TESTS:
|
||||
issue = _test_issue()
|
||||
if not issue:
|
||||
ok("verification passed (build + tests)" if RUN_TESTS else "verification passed (build)")
|
||||
return True
|
||||
attempt += 1
|
||||
if attempt > RT.retries:
|
||||
warn(f"{issue} remain after {RT.retries} fix attempt(s); leaving branch for manual finishing")
|
||||
return False
|
||||
log(f"claude fix pass {attempt}/{RT.retries} ({issue})")
|
||||
run_claude(render_prompt("verify", [], issue))
|
||||
|
||||
|
||||
# state
|
||||
def state_get(key: str) -> str:
|
||||
if not STATE_FILE.exists():
|
||||
return ""
|
||||
try:
|
||||
data = cast("dict[str, object]", json.loads(STATE_FILE.read_text()))
|
||||
except (json.JSONDecodeError, OSError):
|
||||
return ""
|
||||
value = data.get(key)
|
||||
return value if isinstance(value, str) else ""
|
||||
|
||||
|
||||
def write_state(last_synced: str, vendor_tip: str) -> None:
|
||||
payload: dict[str, str] = {
|
||||
"last_synced_sha": last_synced,
|
||||
"vendor_tip": vendor_tip,
|
||||
"last_synced_date": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||
"upstream_url": ZED_REMOTE_URL,
|
||||
"upstream_ref": RT.zed_ref,
|
||||
}
|
||||
_ = STATE_FILE.write_text(json.dumps(payload, indent=2) + "\n")
|
||||
|
||||
|
||||
# upstream remote
|
||||
def require_clean_tree() -> None:
|
||||
if gout("status", "--porcelain"):
|
||||
die("working tree is not clean; commit or stash changes first")
|
||||
|
||||
|
||||
def ensure_remote() -> None:
|
||||
if not gok("remote", "get-url", ZED_REMOTE_NAME):
|
||||
log(f"adding remote '{ZED_REMOTE_NAME}' → {ZED_REMOTE_URL}")
|
||||
run_git("remote", "add", ZED_REMOTE_NAME, ZED_REMOTE_URL)
|
||||
|
||||
|
||||
def fetch_ref(ref: str) -> None:
|
||||
log(f"fetching {ZED_REMOTE_NAME}/{ref} (blobless partial fetch)…")
|
||||
run_git("fetch", "--filter=blob:none", ZED_REMOTE_NAME, ref, capture=False)
|
||||
|
||||
|
||||
def fetch_object(sha: str) -> None:
|
||||
if gok("cat-file", "-e", f"{sha}^{{commit}}"):
|
||||
return
|
||||
log(f"fetching object {sha[:12]} from {ZED_REMOTE_NAME}…")
|
||||
run_git("fetch", "--filter=blob:none", ZED_REMOTE_NAME, sha, check=False, capture=False)
|
||||
if not gok("cat-file", "-e", f"{sha}^{{commit}}"):
|
||||
die(f"could not fetch {sha} from upstream")
|
||||
|
||||
|
||||
def default_baseline() -> str:
|
||||
"""The zed rev currently pinned in the root Cargo.toml (best automatic guess)."""
|
||||
for line in (REPO_ROOT / "Cargo.toml").read_text().splitlines():
|
||||
if 'github.com/zed-industries/zed"' in line:
|
||||
match = re.search(r'rev = "([0-9a-fA-F]{7,40})"', line)
|
||||
if match:
|
||||
return match.group(1)
|
||||
return ""
|
||||
|
||||
|
||||
# subcommands
|
||||
def cmd_bootstrap(baseline: str | None) -> None:
|
||||
require_clean_tree()
|
||||
ensure_remote()
|
||||
base = baseline or default_baseline()
|
||||
if not base:
|
||||
die("no baseline sha given and none pinned in Cargo.toml; pass one explicitly")
|
||||
log(f"bootstrapping sync baseline at {base[:12]}")
|
||||
fetch_object(base)
|
||||
base = gout("rev-parse", f"{base}^{{commit}}")
|
||||
|
||||
v0 = build_vendor_snapshot(base)
|
||||
run_git("update-ref", f"refs/heads/{VENDOR_BRANCH}", v0)
|
||||
ok(f"vendor branch '{VENDOR_BRANCH}' → {v0[:12]}")
|
||||
|
||||
if gok("merge-base", "--is-ancestor", v0, "HEAD"):
|
||||
warn("baseline already recorded in this branch's history; skipping ours-merge")
|
||||
else:
|
||||
msg = (f"chore(sync): record zed gpui baseline {base[:12]}\n\n"
|
||||
"Establishes the merge base for automated upstream syncs. No file changes.")
|
||||
run_git("merge", "-s", "ours", "--allow-unrelated-histories", "--no-edit", "-m", msg, v0,
|
||||
capture=False)
|
||||
ok("recorded baseline in history (no file changes)")
|
||||
|
||||
write_state(base, v0)
|
||||
run_git("add", str(STATE_FILE))
|
||||
run_git("commit", "-m", f"chore(sync): initialize upstream sync state at {base[:12]}")
|
||||
ok("bootstrap complete — run 'just sync-upstream' to pull upstream changes")
|
||||
|
||||
|
||||
def cmd_sync(ref: str | None) -> None:
|
||||
require_clean_tree()
|
||||
ensure_remote()
|
||||
if ref:
|
||||
RT.zed_ref = ref
|
||||
|
||||
last = state_get("last_synced_sha")
|
||||
vendor_tip = state_get("vendor_tip")
|
||||
if not last:
|
||||
die("no sync baseline recorded — run: just sync-upstream-bootstrap [SHA]")
|
||||
if not vendor_tip:
|
||||
die("state is missing vendor_tip — re-run bootstrap")
|
||||
|
||||
fetch_ref(RT.zed_ref)
|
||||
target = gout("rev-parse", "FETCH_HEAD^{commit}")
|
||||
log(f"last synced {last[:12]} → target {target[:12]} ({RT.zed_ref})")
|
||||
if target == last:
|
||||
ok(f"already up to date with {RT.zed_ref}")
|
||||
return
|
||||
|
||||
log("upstream commits touching tracked crates:")
|
||||
listing = git("log", "--oneline", f"{last}..{target}", "--", *tracked_pathspec(), check=False)
|
||||
for line in listing.stdout.splitlines():
|
||||
print(f" {line}")
|
||||
|
||||
if RT.dry_run:
|
||||
ok("dry run — no changes made")
|
||||
return
|
||||
|
||||
if not gok("cat-file", "-e", f"{vendor_tip}^{{commit}}"):
|
||||
die(f"vendor_tip {vendor_tip[:12]} missing locally — re-run bootstrap")
|
||||
if not gok("merge-base", "--is-ancestor", vendor_tip, "HEAD"):
|
||||
die("vendor_tip not in current branch history — run sync from the branch holding the last sync (usually main)")
|
||||
|
||||
vnew = build_vendor_history(vendor_tip, last, target)
|
||||
run_git("update-ref", f"refs/heads/{VENDOR_BRANCH}", vnew)
|
||||
if vnew == vendor_tip:
|
||||
warn("no gpui-touching upstream commits in range — only deps will be updated")
|
||||
|
||||
# Both sides are already remapped to fork paths, so this reports upstream's cross-crate
|
||||
# relocations in gpui-ce terms — used to settle delete/modify conflicts as moves.
|
||||
moves = detect_moves(vendor_tip, vnew)
|
||||
|
||||
branch = f"sync/zed-{datetime.now(timezone.utc):%Y%m%d}-{target[:7]}"
|
||||
run_git("switch", "-C", branch, capture=False)
|
||||
ok(f"working on branch '{branch}'")
|
||||
|
||||
upstream_note = (
|
||||
"The individual upstream commits are preserved in this merge's second-parent\n"
|
||||
"history (filtered to the tracked crates)."
|
||||
)
|
||||
merge_clean_msg = (
|
||||
f"merge: sync zed gpui {last[:12]}..{target[:12]}\n\n"
|
||||
f"Synced tracked GPUI crates from zed-industries/zed ({RT.zed_ref}); no conflicts.\n"
|
||||
f"{upstream_note}\nUpstream range: {last}..{target}"
|
||||
)
|
||||
merge_raw_msg = (
|
||||
f"merge: sync zed gpui {last[:12]}..{target[:12]} (raw, conflict markers)\n\n"
|
||||
"git's automatic 3-way merge of the filtered upstream history. Files git could\n"
|
||||
"NOT auto-merge are committed here WITH their conflict markers intact; the very\n"
|
||||
"next commit contains the resolution, so it can be reviewed as an isolated diff\n"
|
||||
"against this raw state. Deterministic add/delete conflicts were settled by\n"
|
||||
f"policy (gpui-ce's deletions kept).\n{upstream_note}\nUpstream range: {last}..{target}"
|
||||
)
|
||||
resolution_msg = (
|
||||
f"resolve conflicts from zed gpui sync {last[:12]}..{target[:12]}\n\n"
|
||||
"Resolution of the conflict markers left by the preceding merge commit, performed\n"
|
||||
"by claude -p. Review THIS diff in isolation to audit the resolution — it shows\n"
|
||||
"exactly which side/lines were chosen, distinct from what git auto-merged."
|
||||
)
|
||||
|
||||
log("merging upstream delta…")
|
||||
# 3-way merge but DON'T auto-commit, so we can split raw conflicts from the resolution.
|
||||
merged = git("merge", "--no-ff", "--no-commit", vnew, check=False, capture=False)
|
||||
if merged.returncode == 0:
|
||||
run_git("commit", "--no-edit", "-m", merge_clean_msg)
|
||||
ok("merged cleanly (no conflicts)")
|
||||
else:
|
||||
if not gok("rev-parse", "-q", "--verify", "MERGE_HEAD"):
|
||||
die("git merge failed without conflicts to resolve; aborting")
|
||||
# Settle deterministic add/delete conflicts (kept out of the LLM step).
|
||||
auto_resolve_add_delete(moves)
|
||||
nconf = len(conflicted_files())
|
||||
# Commit 1: the RAW merge — auto-merges applied, conflict markers committed in.
|
||||
run_git("add", "-A")
|
||||
run_git("commit", "--no-edit", "-m", merge_raw_msg)
|
||||
ok(f"committed raw merge with conflict markers ({nconf} file(s) to resolve)")
|
||||
if nconf > 0:
|
||||
# Commit 2: claude's resolution of the markers — reviewable as an isolated diff.
|
||||
warn("resolving conflict markers with claude…")
|
||||
resolve_conflicts_loop(branch, moves)
|
||||
run_git("commit", "-m", resolution_msg)
|
||||
ok("committed conflict resolution (separate, reviewable commit)")
|
||||
else:
|
||||
ok("no content conflicts (only add/delete, settled in the merge commit)")
|
||||
|
||||
if RT.bump_zed_deps:
|
||||
bump_zed_deps(target)
|
||||
|
||||
verify_ok = verify_fix_loop()
|
||||
|
||||
if gout("status", "--porcelain"):
|
||||
run_git("add", "-A")
|
||||
run_git("commit", "-m",
|
||||
"chore(sync): post-merge fixes (zed deps bump + build/test/warning fixes)",
|
||||
check=False)
|
||||
ok("committed post-merge fixes")
|
||||
|
||||
write_state(target, vnew)
|
||||
run_git("add", str(STATE_FILE))
|
||||
run_git("commit", "-m", f"chore(sync): advance sync state to {target[:12]}")
|
||||
summary(branch, last, target, verify_ok)
|
||||
|
||||
|
||||
def cmd_status() -> None:
|
||||
ensure_remote()
|
||||
last = state_get("last_synced_sha")
|
||||
print(f"last synced : {last or '<none — run bootstrap>'}")
|
||||
print(f"vendor tip : {state_get('vendor_tip')}")
|
||||
print(f"synced date : {state_get('last_synced_date')}")
|
||||
run_git("fetch", "--filter=blob:none", ZED_REMOTE_NAME, RT.zed_ref, check=False)
|
||||
target = gout("rev-parse", "FETCH_HEAD^{commit}") if gok("rev-parse", "FETCH_HEAD^{commit}") else ""
|
||||
if target:
|
||||
print(f"upstream {RT.zed_ref} : {target[:12]}")
|
||||
if last and target and last != target:
|
||||
count = gout("rev-list", "--count", f"{last}..{target}") if gok(
|
||||
"rev-list", "--count", f"{last}..{target}") else "?"
|
||||
print(f"\nbehind by {count} upstream commit(s) on {RT.zed_ref}")
|
||||
elif last and target and last == target:
|
||||
ok(f"up to date with {RT.zed_ref}")
|
||||
|
||||
|
||||
def summary(branch: str, last: str, target: str, verify_ok: bool) -> None:
|
||||
gates = f"{VERIFY_CMD} + {TEST_CMD}" if RUN_TESTS else VERIFY_CMD
|
||||
print()
|
||||
if verify_ok:
|
||||
ok("sync complete — build + tests pass" if RUN_TESTS else "sync complete — build passes")
|
||||
else:
|
||||
warn("sync complete — VERIFICATION FAILED, finish manually")
|
||||
print(f" branch : {branch}")
|
||||
print(f" upstream range : {last[:12]}..{target[:12]}")
|
||||
print(f" verify ({gates}) : {'OK' if verify_ok else 'FAILED'}")
|
||||
print(f"\nReview : git log --oneline --stat main..{branch}")
|
||||
print(f"Merge : git switch main && git merge --ff-only {branch} (or open a PR)")
|
||||
print(f"{_D}Nothing was pushed.{_Z}")
|
||||
|
||||
|
||||
# entrypoint
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="sync_upstream.py",
|
||||
description="Sync upstream Zed GPUI changes into gpui-ce (local-only, never pushes).",
|
||||
)
|
||||
_ = parser.add_argument("command", nargs="?", default="sync",
|
||||
choices=["sync", "bootstrap", "status"])
|
||||
_ = parser.add_argument("arg", nargs="?", default=None,
|
||||
help="bootstrap: baseline sha; sync: upstream ref")
|
||||
_ = parser.add_argument("--ref", default=None, help="upstream ref to sync to")
|
||||
_ = parser.add_argument("--model", default=None, help="claude model")
|
||||
_ = parser.add_argument("--retries", type=int, default=None, help="max claude passes per phase")
|
||||
_ = parser.add_argument("--no-bump", action="store_true", help="don't bump pinned zed git-dep revs")
|
||||
_ = parser.add_argument("--dry-run", action="store_true", help="show what would be synced, then stop")
|
||||
ns = parser.parse_args()
|
||||
|
||||
command = cast(str, ns.command)
|
||||
positional = cast("str | None", ns.arg)
|
||||
model = cast("str | None", ns.model)
|
||||
retries = cast("int | None", ns.retries)
|
||||
ref = cast("str | None", ns.ref)
|
||||
|
||||
if model is not None:
|
||||
RT.model = model
|
||||
if retries is not None:
|
||||
RT.retries = retries
|
||||
if cast(bool, ns.no_bump):
|
||||
RT.bump_zed_deps = False
|
||||
RT.dry_run = cast(bool, ns.dry_run)
|
||||
|
||||
try:
|
||||
if command == "bootstrap":
|
||||
cmd_bootstrap(positional)
|
||||
elif command == "status":
|
||||
cmd_status()
|
||||
else:
|
||||
cmd_sync(ref or positional)
|
||||
except SyncError as exc:
|
||||
print(f"{_R}✗ {exc}{_Z}", file=sys.stderr)
|
||||
return 1
|
||||
except KeyboardInterrupt:
|
||||
return 130
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user