fix: Windows UTF-8 sdist + Darwin libomp / lightgbm CI paths by ababber · Pull Request #3 · sqlmath/sqlmath

ababber · 2026-03-27T21:04:30Z

Summary

Separate from the README PR (#1) so this can be merged or replaced independently.

setup.py

Open pyproject.toml, README.md, MANIFEST.in, and the sdist temp script with encoding="utf-8" so build_pkg_info / python setup.py sdist do not use Windows cp1252 (fixes UnicodeDecodeError when README contains UTF-8).

.ci.sh

Resolve libomp for Homebrew on Apple Silicon (/opt/homebrew), Intel (/usr/local), or $HOMEBREW_PREFIX.
rm -f before copying dylibs so read-only artifacts do not break cp.
Use pip show lightgbm (not ruff) for site-packages when copying lib_lightgbm.

Context: Windows traceback from build_pkg_info reading README; Intel Mac libomp path (issue #2).

Made with Cursor

…htgbm in CI - setup.py: open text files with encoding=utf-8 in build_pkg_info and build_sdist (avoids cp1252 UnicodeDecodeError embedding README on Windows). - .ci.sh: resolve libomp for Apple Silicon, Intel, and HOMEBREW_PREFIX; rm stale dylibs before copy; use pip show lightgbm for site-packages path. Made-with: Cursor

Made-with: Cursor

… exit code - sqlmath.mjs: coerce waitAsync timeout with Number/isFinite so test waitAsync() no-arg does not pass NaN to setTimeout under npm_config_mode_test (fixes TimeoutNaNWarning and flaky coverage). - jslint.mjs, jslint_ci.sh: v8CoverageReportCreate waits on child close and maps non-numeric code + signal to an integer exit code (Node can report null code on exit in some stdio/inherit cases). Made-with: Cursor

After sqlmath#3 is on beta, merge or rebase docs/readme-rewrite so CI and build_pkg_info see UTF-8-safe setup.py and updated .ci.sh.

Adds DEMO-MERGE-STRATEGY.md for Kai. PKG-INFO from build_pkg_info() after README + PR sqlmath#3 stack. Made-with: Cursor

…erification

ababber · 2026-03-27T21:57:57Z

Merge strategy and demo for #3 -> #1

Summary

This PR is intentionally split from the README PR (#1) so you can land tooling/CI fixes independently or replace them with your own approach.

setup.py — Opens pyproject.toml, README.md, MANIFEST.in, PKG-INFO, and the sdist temp script with encoding="utf-8", so build_pkg_info / python setup.py sdist do not use the Windows default (e.g. cp1252) and hit UnicodeDecodeError once the README contains UTF-8 (smart quotes, etc.).

.ci.sh — Resolves libomp for Homebrew on Apple Silicon (/opt/homebrew), Intel (/usr/local), or $HOMEBREW_PREFIX; rm -f before copying dylibs; uses pip show lightgbm (not ruff) for the LightGBM library path when GITHUB_ACTION is unset.

sqlmath.mjs / jslint.mjs / jslint_ci.sh — waitAsync() with no argument no longer passes NaN into setTimeout under test mode; v8CoverageReportCreate waits on the child close event and normalizes exit codes so local/CI Node runs do not fail with exitCode === null after tests pass.

Style — One pycodestyle E501 wrap for the PKG-INFO open() call.

Suggested merge order with PR #1

If you want the README rewrite without re-breaking Windows or macOS CI, merge this PR (#3) into beta first, then merge or refresh #1 on top of updated beta. If Git reports conflicts on setup.py, .ci.sh, jslint.mjs, jslint_ci.sh, or sqlmath.mjs, keep the versions from #3.

Demo on my fork (optional)

On ababber/sqlmath there are read-only demo branches so you can compare diffs or run CI locally:

Branch	Intent
`demo/merge-pr3-then-pr1`	`beta` → #3 → #1 + regenerated `PKG-INFO` — “happy path”
`demo/merge-pr1-only-on-beta`	`beta` + README only — contrast without #3 fixes

Doc: DEMO-MERGE-STRATEGY.md (on demo/merge-pr3-then-pr1)
Compare (golden path vs fork beta): ababber/sqlmath@beta...demo/merge-pr3-then-pr1
Compare (docs: rewrite README with examples, API reference, use cases #1-only contrast): ababber/sqlmath@beta...demo/merge-pr1-only-on-beta

Demo branch: local CI check (`demo/merge-pr3-then-pr1`)

I did not run the full shCiPre + shCiBase path on the demo branch until after opening it; fix/windows-utf8-and-ci was what I validated first.

What I did:

First run on demo/merge-pr3-then-pr1 failed in shCiBase: package.json.fileCount was 46 but the repo had 47 tracked files because of demo-only DEMO-MERGE-STRATEGY.md.
After bumping fileCount to 47, run-staging-ci.sh (same entrypoints as workflow: lint, build_ext, Node tests + V8 coverage, Python tests) completed successfully on macOS, Node 24, and a venv used for pip/LightGBM.
shCiBase also refreshed MANIFEST.in to include DEMO-MERGE-STRATEGY.md. That was committed and pushed as c8986fde on origin/demo/merge-pr3-then-pr1.
A second full run after c8986fde: run-staging-ci: shCiPre + shCiBase completed with no trailing git diff (clean tree).

Upstream caveat: After merging #3 then #1 on sqlmath/sqlmath beta without the demo markdown file, fileCount should match whatever git ls-tree -r HEAD | wc -l is after that merge (still 46 today if nothing else is added), and MANIFEST.in will not mention DEMO-MERGE-STRATEGY.md unless you add it on purpose.

Happy to adjust anything to match your preferred release flow.

kaizhu256 · 2026-03-28T05:47:01Z

thx for all the hard-work, and figuring out the local-ci !
can you add macos-x86_64 to github's ci-workflow as follows:

apply following patch:

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 0780456..846bdee 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -24,6 +24,8 @@ jobs:
         node_version:
           - "24"
         os:
+          - macos-15-intel
           - macos-latest
           - ubuntu-latest
           - windows-latest

enable github-workflow for branch alpha (https://docs.github.com/en/actions/how-tos/manage-workflow-runs/disable-and-enable-workflows)
push this pull-request-branch to alpha:

git push origin HEAD:alpha -f

verify the github-ci passes @ https://github.com/ababber/sqlmath/actions

Kai (kaichu250) requested an Intel macOS runner alongside macos-latest.

Hard-coded /opt/homebrew-first lookup picked the wrong prefix on GitHub macos-15-intel (Intel Homebrew under /usr/local). Use `brew --prefix libomp` so the copy matches the keg brew installed.

Sync .ci.sh for the Actions step that checks out CI scripts from origin/alpha before shCiBase. Matches fix/windows-utf8-and-ci / PR sqlmath#3. Made-with: Cursor

ababber · 2026-03-30T16:20:24Z

I added the ci-workflow, here are the run details:

successful ci run

- Wrap libomp comment and error printf in .ci.sh to 80 columns - Restore jslint.mjs and jslint_ci.sh to beta (exit handler, not close) Made-with: Cursor

ababber · 2026-03-30T20:43:04Z

@kaizhu256

quick follow-up...

on the items you called out, and how I applied them:

`.ci.sh` (80-column + libomp)

The whole file is now within 80 columns (verified with awk), not just the lines I touched, so it matches the style you asked for.
The long error line is split using printf "%s%s\n" with continuations — that’s valid POSIX sh (two string args, one newline), and 1>&2 still applies to the whole printf.
Comment breaks are on natural phrase boundaries so it stays readable.
libomp uses brew --prefix libomp, checks the dylib exists, then rm -f + cp -L — same intent for Apple Silicon and Intel Homebrew prefixes.
rm -f "sqlmath/$FILE" before copying LightGBM avoids leaving a stale arch-specific binary in place.

`jslint.mjs` / `jslint_ci.sh` (revert)

Both are back to .on("exit", resolve) in v8CoverageReportCreate, per your preference not to touch that unless it’s causing real pain. The two files stay in sync on that spawn.

successful ci run

Thanks again for the clear feedback.

Made-with: Cursor

… to lib_lightgbm_platform_arch.xxx, libomp_platform_arch.xxx, to prevent name-collision under darwin_arm64 and darwin_x64 builds.

ababber mentioned this pull request Mar 27, 2026

fix: Windows UTF-8 sdist + Darwin libomp / lightgbm CI paths ababber/sqlmath#1

Closed

ababber added 2 commits March 27, 2026 16:07

style: wrap PKG-INFO open() for pycodestyle E501 (79 cols)

86ddd64

Made-with: Cursor

ababber added a commit to ababber/sqlmath that referenced this pull request Mar 27, 2026

demo: document merge order; sync PKG-INFO with merged README

0a7c106

Adds DEMO-MERGE-STRATEGY.md for Kai. PKG-INFO from build_pkg_info() after README + PR sqlmath#3 stack. Made-with: Cursor

ababber added a commit to ababber/sqlmath that referenced this pull request Mar 27, 2026

focus: update PR sqlmath#3 GitHub comment draft with demo branch CI v…

3550e86

…erification

ababber mentioned this pull request Mar 27, 2026

build: libomp path assumes ARM homebrew (/opt/homebrew) - fails on x86 Mac #2

Closed

ababber added 2 commits March 29, 2026 23:19

ci: add macos-15-intel to Actions matrix

053131a

Kai (kaichu250) requested an Intel macOS runner alongside macos-latest.

fix(ci): resolve libomp via brew --prefix on Darwin

1dfe8e1

Hard-coded /opt/homebrew-first lookup picked the wrong prefix on GitHub macos-15-intel (Intel Homebrew under /usr/local). Use `brew --prefix libomp` so the copy matches the keg brew installed.

fix(ci): Kai follow-up — 80-col .ci.sh, revert jslint coverage spawn

196128e

- Wrap libomp comment and error printf in .ci.sh to 80 columns - Restore jslint.mjs and jslint_ci.sh to beta (exit handler, not close) Made-with: Cursor

ababber and others added 4 commits March 30, 2026 15:52

ci: run workflow on pull_request to alpha, beta, master

a78bcca

Made-with: Cursor

Merge branch 'beta' into fix/windows-utf8-and-ci

420a50c

- sqlmath-ci - Cleanup merge.

1f1cd58

- sqlmath-ci - Rename pre-built-binaries lib_lightgbm.xxx, libomp.xxx…

bb2321f

… to lib_lightgbm_platform_arch.xxx, libomp_platform_arch.xxx, to prevent name-collision under darwin_arm64 and darwin_x64 builds.

kaizhu256 merged commit f4a5c4e into sqlmath:beta Apr 1, 2026
8 checks passed

ababber mentioned this pull request Apr 7, 2026

docs: rewrite README with examples, API reference, use cases #1

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Windows UTF-8 sdist + Darwin libomp / lightgbm CI paths#3

fix: Windows UTF-8 sdist + Darwin libomp / lightgbm CI paths#3
kaizhu256 merged 10 commits into
sqlmath:betafrom
ababber:fix/windows-utf8-and-ci

ababber commented Mar 27, 2026

Uh oh!

ababber commented Mar 27, 2026

Uh oh!

kaizhu256 commented Mar 28, 2026

Uh oh!

ababber commented Mar 30, 2026

Uh oh!

ababber commented Mar 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ababber commented Mar 27, 2026

Summary

setup.py

.ci.sh

Uh oh!

ababber commented Mar 27, 2026

Merge strategy and demo for #3 -> #1

Summary

Suggested merge order with PR #1

Demo on my fork (optional)

Demo branch: local CI check (demo/merge-pr3-then-pr1)

Uh oh!

kaizhu256 commented Mar 28, 2026

Uh oh!

ababber commented Mar 30, 2026

Uh oh!

ababber commented Mar 30, 2026

quick follow-up...

.ci.sh (80-column + libomp)

jslint.mjs / jslint_ci.sh (revert)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Demo branch: local CI check (`demo/merge-pr3-then-pr1`)

`.ci.sh` (80-column + libomp)

`jslint.mjs` / `jslint_ci.sh` (revert)