Add Apple Silicon modernization design spec

Phases 1 and 2: refactor build script to drop universal2 enforcement and run natively on Apple Silicon, bump upstream SHAs and Python versions, add 3.14, final-release 3.9 and 3.10. CI/CD overhaul deferred to Phase 3. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
macadmins · erikng · May 12, 2026 · May 11, 2026 · May 11, 2026 · May 12, 2026
commit b060a5813092222ad0f9e3870b0615adba0cf8de
diff --git a/docs/superpowers/specs/2026-05-11-apple-silicon-modernization-design.md b/docs/superpowers/specs/2026-05-11-apple-silicon-modernization-design.md
@@ -0,0 +1,194 @@
+# Apple Silicon Modernization — Design
+
+**Date:** 2026-05-11
+**Author:** Erik Gomez
+**Status:** Draft
+
+## Summary
+
+Modernize the macadmins Python framework build pipeline by:
+
+1. Refactoring `build_python_framework_pkgs.zsh` to drop universal2 enforcement, run natively on Apple Silicon, and trim to a single build type (`recommended`).
+2. Bumping all upstream dependencies — `relocatable-python`, `munki-pkg`, Python interpreter patches, and Python package pins — and adding Python 3.14 as a supported branch.
+
+CI/CD overhaul (Phase 3) is **deferred to a follow-up spec**; this work targets local-only changes plus the requirement and version updates that ride on top of them.
+
+## Background
+
+`build_python_framework_pkgs.zsh` produces a relocatable `Python3.framework` installed under `/Library/ManagedFrameworks/Python/`, packaged via `munki-pkg`, signed, notarized, and shipped as a `.pkg`. It is consumed by Munki 6, Autopkg, InstallApplications, Nudge, and similar fleet tools.
+
+Today the script:
+
+- Requires an Intel build host. Lines 200–223 enumerate every `.dylib` and `.so` under the built framework and fail the build unless each contains two architectures. This forces `--no-binary` on `cffi`, `charset-normalizer`, `PyYAML`, `tomli`, `xattr`, and `black` in `requirements_recommended.txt` so wheels are compiled fat instead of installed pre-built.
+- Supports three build types (`minimal`, `no_customization`, `recommended`) but CI only builds `recommended`.
+- Pins `relocatable-python` to `fb4dd9b…` and `munki-pkg` to `96cffb4e…`, both stale.
+- Hardcodes `Xcode_15.2.app` and `macos-13` (Intel) GitHub runners.
+- Carries a latent path bug in the ad-hoc codesign branch (`build_python_framework_pkgs.zsh:241` — missing `/` between `${FRAMEWORKDIR}` and `Python3.framework`).
+
+The team is shipping macOS deployments to Apple Silicon hardware; the universal2 contract is no longer worth the build-host constraints, the `--no-binary` slowdowns, or the maintenance surface.
+
+## Goals
+
+- Build cleanly from an Apple Silicon Mac with no Intel host required.
+- Use pre-built arm64 wheels from PyPI.
+- Single supported build flavor (`recommended`); delete the other two.
+- Current upstream SHAs and Python patch versions.
+- Add Python 3.14 to the supported set.
+- One last release of Python 3.9 and 3.10, then retire them.
+- Script is readable, factored into clear functions, and easy to invoke locally without simulating a CI environment.
+
+## Non-Goals
+
+- CI/CD changes (workflow consolidation, action bumps, runner migration, release trigger changes). Deferred to Phase 3.
+- Compiling CPython from source for EOL branches. 3.9 / 3.10 final releases continue to use python.org's last published `.pkg` (`3.9.13` / `3.10.11`).
+- Changing the install location, package identifier, or signing identity.
+- Producing an explicitly arm64-thinned framework via `lipo`. The python.org base remains universal2; arm64-only wheels make the *added* content single-arch. We do not strip x86_64 slices from upstream binaries.
+
+## Phase 1 — Script Refactor
+
+### Script signature
+
+Current:
+
+```
+build_python_framework_pkgs.zsh <TYPE> <INSTALLER_ID> <APPLICATION_ID> <PYTHON_VERSION> <PYTHON_MAJOR_VERSION> <NOTARY_PASSWORD>
+```
+
+New (positional args dropped in favor of long flags for clarity; major version derived from full version):
+
+```
+build_python_framework_pkgs.zsh \
+  --python-version 3.13.13 \
+  [--installer-id "Developer ID Installer: ..."] \
+  [--application-id "Developer ID Application: ..."] \
+  [--notary-password "$NOTARY_APP_PASSWORD"] \
+  [--xcode-path /Applications/Xcode_16.x.app]
+```
+
+When `--installer-id` / `--application-id` are omitted, the script falls back to ad-hoc signing and does not produce a signed `.pkg`. When `--notary-password` is omitted, notarization is skipped. This makes local invocation `./build_python_framework_pkgs.zsh --python-version 3.13.13` and nothing more.
+
+### Behavioral changes
+
+1. **Drop the universal2 validation block.** Remove the `find … "2 architectures"` checks for `.dylib` and `.so` files.
+2. **Drop `--no-binary` markers** in `requirements_recommended.txt` for `black`, `cffi`, `charset-normalizer`, `PyYAML`, `tomli`, `xattr`.
+3. **Guard CI-only steps.** The `brew remove --force …` and the `sudo xcode-select -s "$XCODE_PATH"` calls run only when `$CI` or `$GITHUB_ACTIONS` is set. Locally they're no-ops.
+4. **Parametrize Xcode path.** `--xcode-path` (or `XCODE_PATH` env var) replaces the `Xcode_15.2.app` hardcode. Default to `xcode-select -p`'s active developer dir when not specified.
+5. **Collapse per-version symlink block.** The five identical `if [[ "${PYTHON_MAJOR_VERSION}" == "3.x" ]]` blocks at lines 144–158 become a single unconditional `ln -s`.
+6. **Single build type.** Delete the `minimal` / `no_customization` branches. `TYPE` becomes an internal constant (`recommended`) used for path naming only.
+7. **Fix the codesign path bug** at line 241.
+8. **Bump pinned SHAs:**
+   - `RP_SHA` → `8ee72fe3a5dbef733365370ebf44f25022b895ef`
+   - `MP_SHA` → `bbd07730d1b93ed3828246575ef5676bba74b5d1`
+
+### Refactor for readability
+
+Pull discrete steps into named functions so the top-level script reads as a sequence of intents rather than a wall of inline shell:
+
+- `parse_args` — long-flag parsing, validation, defaults.
+- `prepare_build_dirs` — framework dir setup, payload skeleton.
+- `download_tool <name> <sha> <url> <dest>` — generic curl-and-unzip used for both `relocatable-python` and `munki-pkg`.
+- `build_framework` — runs `make_relocatable_python_framework.py`.
+- `codesign_framework` — single implementation that takes the identity (or `-` for ad-hoc) as an argument, replacing the duplicate signed-vs-ad-hoc branches.
+- `build_pkg` — emits `build-info.json`, runs `munkipkg`.
+- `notarize_and_staple` — runs only when notary credentials are present.
+- `zip_framework` — emits the standalone framework zip.
+- `cleanup` — removes temp dirs.
+
+Use `set -eu` (zsh equivalent) at the top of the script so a step failure short-circuits the run instead of relying on per-step exit-code checks.
+
+### Files removed
+
+- `requirements_minimal.txt`
+- `requirements_no_customization.txt` (empty)
+- `requirement_files/requirements_minimal.txt`
+- `requirement_files/requirements_opinionated.txt` (no longer referenced)
+- `build_all_python_frameworks.zsh` (only purpose was to invoke all three build types)
+
+### Files updated
+
+- `build_python_framework_pkgs.zsh` — full refactor per above.
+- `requirements_recommended.txt` — drop `--no-binary` markers.
+- `README.md` — drop the "Flavors of Python", "No Customization", "Minimal" sections; drop the "build on an Intel macOS device" note; update interactive-use snippet for Apple Silicon.
+- `requirement_files/requirements_recommended.txt` — remains as the human-curated source for which package families are included (no behavior change in Phase 1).
+
+### Local validation steps
+
+After the refactor:
+
+1. `./build_python_framework_pkgs.zsh --python-version 3.13.13` on the Apple Silicon Mac. Expect: an unsigned framework zip in `outputs/`, no signed `.pkg`.
+2. Manually install the framework to `/Library/ManagedFrameworks/Python/Python3.framework` and smoke-test:
+   - `managed_python3 --version` reports `3.13.13`.
+   - `managed_python3 -c "import objc; import xattr; import requests; print('ok')"` succeeds.
+   - `managed_python3 -c "import platform; print(platform.machine())"` prints `arm64`.
+3. Repeat for `--python-version 3.14.5`.
+
+## Phase 2 — Dependency and Version Bumps
+
+### Python interpreter versions
+
+| Branch | Old | New | Notes |
+|---|---|---|---|
+| 3.9  | 3.9.13  | 3.9.13  | Final release. No upstream `.pkg` past this. |
+| 3.10 | 3.10.11 | 3.10.11 | Final release. No upstream `.pkg` past this. |
+| 3.11 | 3.11.7  | 3.11.9  | |
+| 3.12 | 3.12.1  | 3.12.10 | |
+| 3.13 | 3.13.5  | 3.13.13 | |
+| 3.14 | — | 3.14.5  | New supported branch. |
+
+### Python package pin sweep
+
+For each pinned package in `requirements_recommended.txt`:
+
+1. Check if a newer release exists on PyPI.
+2. Verify an `arm64` macOS wheel exists for **every** supported Python branch (3.9–3.14). If a version lacks a wheel for 3.9 / 3.10 (likely for newer pyobjc, cffi, etc.), pin a per-branch override or hold the package at the last version that supports all branches. Document the holdback inline.
+3. The packages most likely to need attention: `pyobjc` (currently 11.1; check arm64 wheel coverage for 3.14), `cffi`, `xattr`, `cryptography` if pulled transitively.
+
+### 3.9 / 3.10 final-release release notes
+
+Each gets a one-time release, kicked off manually via `workflow_dispatch` (no CI restructuring needed for this in Phase 2), with:
+
+- Updated package pins (from the sweep above).
+- Release-notes call-out: "**This is the final release of the Python <X.Y> framework. Future updates will target 3.11 and newer. Plan your migration.**"
+
+The workflow files themselves (`build_python_3.9.yml`, `build_python_3.10.yml`) stay in place during Phase 2 so the final builds can run. They are moved to `.github/workflows/archived/` as part of Phase 3.
+
+### 3.14 enablement
+
+`.github/workflows/build_python_3.14.yml` is added as a near-copy of `build_python_3.13.yml`, parametrized for 3.14.5. The file lives alongside the existing per-version workflows; consolidation happens in Phase 3.
+
+### Dependency-update tooling
+
+- **Dependabot for GitHub Actions:** enable in `.github/dependabot.yml`. Low noise; catches stale `apple-actions/import-codesign-certs`, `actions/checkout`, etc. without us thinking about it.
+- **Pip pins:** keep manual. We want intentional bumps with smoke testing, not auto-merged churn.
+
+## Deferred to Phase 3
+
+Captured here for context — not in scope for this spec:
+
+- Consolidate the five per-version workflows into one reusable workflow + thin callers.
+- Migrate runner from `macos-13` (Intel) → `macos-14` (Apple Silicon).
+- Bump action versions: `actions/checkout@v3 → v5`, `softprops/[email protected] → v2`, `actions/[email protected] → latest v4`, `apple-actions/import-codesign-certs → latest`, `metcalfc/changelog-generator → latest` (or replace with a `gh api` shell step).
+- Switch release trigger from `pull_request` (currently cuts a prerelease per PR) to `push` on a release tag plus `workflow_dispatch`. PR runs build artifacts but don't publish releases.
+- Archive `build_python_3.9.yml` and `build_python_3.10.yml`.
+
+## Risks and Open Questions
+
+- **arm64 wheel coverage for 3.9 / 3.10.** Older Pythons may not have arm64 wheels for the newest pin of every package. Mitigation: per-branch pin overrides, or hold the package at a known-good version. The pin sweep in Phase 2 will surface this.
+- **First arm64 build may expose latent assumptions** in `relocatable-python`. The pinned-SHA bump pulls in a year+ of upstream changes; we may need to file/patch downstream issues. Mitigation: local validation gate before merging.
+- **3.9 / 3.10 final builds depend on the new script working with old Python branches.** The bumped `relocatable-python` should still accept the older `--python-version` values, but it's a small contract worth verifying as part of local validation.
+
+## Acceptance Criteria
+
+Phase 1 is done when:
+
+- `./build_python_framework_pkgs.zsh --python-version 3.13.13` succeeds end-to-end on an Apple Silicon Mac with no `sudo` other than the existing `mkdir`/`chown` steps and produces an installable framework zip.
+- The resulting `managed_python3` runs `import objc; import xattr; import requests` on Apple Silicon.
+- `git grep -- '--no-binary'` returns no matches in `requirements_recommended.txt`.
+- `minimal` / `no_customization` are gone from the repo and README.
+
+Phase 2 is done when:
+
+- All non-EOL workflows pin the patch versions in the table above.
+- `build_python_3.14.yml` exists and runs green end-to-end.
+- 3.9.13 and 3.10.11 each have one final release published with "final release" notes.
+- `.github/dependabot.yml` is enabled for GitHub Actions.