Skip to content

gh-126016: Force crossinterpreter thread states to get cleaned up #126026

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ZeroIntensity wants to merge 11 commits into from
Closed

gh-126016: Force crossinterpreter thread states to get cleaned up #126026

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
7a67cf5
Force thread states to get cleaned up.
ZeroIntensity Oct 27, 2024
30e654d
Add some comments.
ZeroIntensity Oct 27, 2024
d4b0908
Add blurb
ZeroIntensity Oct 27, 2024
f7b7eff
Make the blurb entry prettier
ZeroIntensity Oct 27, 2024
4dff63c
Add TODO.
ZeroIntensity Oct 27, 2024
48150be
Simplify fix.
ZeroIntensity Oct 27, 2024
4012aa6
Add a test.
ZeroIntensity Oct 27, 2024
94b6d09
Add some comments.
ZeroIntensity Oct 27, 2024
bd9f754
Skip test on Windows because it doesn't support SIGINT.
ZeroIntensity Oct 27, 2024
75e93dd
Fix it for Windows (I hope)
ZeroIntensity Oct 27, 2024
61c9299
Revert "Fix it for Windows (I hope)"
ZeroIntensity Oct 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Force thread states to get cleaned up.
  • Loading branch information
@ZeroIntensity
ZeroIntensity committed Oct 27, 2024
commit 7a67cf5c302cb44956d47d5cd75bc6a58b953dde
3 changes: 3 additions & 0 deletions Include/internal/pycore_pystate.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,9 @@ extern PyThreadState * _PyThreadState_Swap(

extern PyStatus _PyInterpreterState_Enable(_PyRuntimeState *runtime);

extern PyThreadState *
_PyThreadState_SwapAttached(PyThreadState *tstate);

#ifdef HAVE_FORK
extern PyStatus _PyInterpreterState_DeleteExceptMain(_PyRuntimeState *runtime);
extern void _PySignal_AfterFork(void);
Expand Down
34 changes: 34 additions & 0 deletions Python/crossinterp.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1692,6 +1692,39 @@ _PyXI_HasCapturedException(_PyXI_session *session)
return session->error != NULL;
}

static int
_PyXI_ThreadStateRecovery(void *ptr)
Comment thread
ZeroIntensity marked this conversation as resolved.
Outdated
{
if ((&_PyRuntime)->_main_interpreter.finalizing != 1)
{
// If the interpreter isn't finalizing, then
// it's not our job to clean anything up.
return 0;
}

PyThreadState *interp_tstate = (PyThreadState *)ptr;
if (interp_tstate->interp == NULL)
{
// Interpreter was cleaned up, do nothing.
return 0;
}

if (!_PyInterpreterState_IsRunningMain(interp_tstate->interp))
{
// Main thread was cleaned up, nothing to fix.
return 0;
}

// Subinterpreter is in a thread that suspended early!
PyThreadState *return_tstate = _PyThreadState_SwapAttached(interp_tstate);
_PyInterpreterState_SetNotRunningMain(interp_tstate->interp);

PyThreadState_Clear(interp_tstate);
PyThreadState_Swap(return_tstate);
PyThreadState_Delete(interp_tstate);
return 0;
}

int
_PyXI_Enter(_PyXI_session *session,
PyInterpreterState *interp, PyObject *nsupdates)
Expand All @@ -1718,6 +1751,7 @@ _PyXI_Enter(_PyXI_session *session,
errcode = _PyXI_ERR_ALREADY_RUNNING;
goto error;
}
Py_AddPendingCall(_PyXI_ThreadStateRecovery, _PyThreadState_GET());
session->running = 1;

// Cache __main__.__dict__.
Expand Down
13 changes: 13 additions & 0 deletions Python/pystate.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2406,6 +2406,19 @@ PyThreadState_Swap(PyThreadState *newts)
return _PyThreadState_Swap(&_PyRuntime, newts);
}

/*
* Calls PyThreadState_Swap() on the a bound thread state.
* This breaks the GIL, so it should only be used if certain that
* it's impossible for the thread to be running code.
*/
PyThreadState *
_PyThreadState_SwapAttached(PyThreadState *tstate)
{
tstate->_status.bound_gilstate = 0;
tstate->_status.holds_gil = 0;
tstate->_status.active = 0;
return PyThreadState_Swap(tstate);
}

void
_PyThreadState_Bind(PyThreadState *tstate)
Expand Down