clide/docs/macos-pty-problem.md

macOS PTY Problem — Diagnosis Complete

⚠ HISTORICAL — ptyc is retired. PTY spawning moved to Dart FFI posix_openpt() + posix_spawn() (D-5 amendments, T-96). The race described below was specific to the old C helper's SCM_RIGHTS hand-off and no longer applies.

Status: Root cause found

The PTY master fd is valid (isatty=1), SCM_RIGHTS transfer is correct, all struct layouts are correct. The problem is timing: the reader isolate starts too late and the shell has already exited by the time read() is called on the master fd. macOS returns EOF (n=0) immediately when the slave side is closed — unlike Linux which buffers data.

The timing sequence (what happens)

1. Dart: socketpair()
2. Dart: Process.start(ptyc)
3. ptyc: posix_openpt + grantpt + unlockpt + open(slave)
4. ptyc: fork() → child starts zsh
5. ptyc: sendmsg(SCM_RIGHTS, master_fd)  ← master fd sent
6. ptyc: printf(ok) + exit (or block if diagnostic)
7. Dart: recvmsg() → receives master_fd    ← fd is valid, isatty=1
8. Dart: PtySession._() constructor
9. Dart: _startReader() → Isolate.spawn()  ← ASYNC, does not start immediately
10. ... event loop yields ...
11. Reader isolate starts, calls read(masterFd)
12. But zsh already exited → slave closed → read() returns 0 (EOF)

The gap between step 9 (Isolate.spawn scheduled) and step 11 (reader actually runs) is where the data is lost. On macOS, the kernel doesn't buffer PTY master data after the slave closes.

Why is zsh exiting?

zsh (/bin/zsh) is an interactive shell. It should NOT exit immediately — it should show a prompt and wait for input. But in the test, it does exit. Possible reasons:

1. No controlling terminal at spawn time. ptyc's child does setsid() + TIOCSCTTY + dup2(slave, 0/1/2) — this should work. But if the slave fd is already closed or invalid by the time dup2 runs, zsh gets no tty and exits.

2. Environment. The env passed to ptyc may be empty or missing TERM, HOME, SHELL etc. Without TERM, zsh may fail to initialize and exit.

3. The PTYC_SOCK_FD inherits open fds. When ptyc forks, the child inherits ALL open fds (master, slave, socket, exec-failure pipe). ptyc closes master and pipe in the child, but the socket fd stays open. This shouldn't cause an exit, but it's worth checking.

4. stdin is connected to the slave. After dup2(slave, 0), zsh reads from the PTY slave. If Dart hasn't written anything to the master AND there's no PTY echo (because the master isn't being read), zsh might get SIGHUP or detect a broken pipe.

Verified facts

Test	Result
ptyc compiles on macOS	✓
socketpair fd inherited by ptyc	✓ (verified with test binary)
ptyc creates PTY and forks	✓ ({"ok":true,"pid":N})
SCM_RIGHTS transfer	✓ (correct cmsg layout)
cmsghdr struct (Dart)	✓ (CmsghdrDarwin, 12 bytes, correct offsets)
msghdr struct (Dart)	✓ (MsghdrDarwin, correct field sizes)
SOL_SOCKET	✓ (0xffff on macOS)
TIOCSWINSZ	✓ (0x80087467 on macOS)
Received fd is a tty	✓ (isatty=1)
Reader isolate starts	✓ (prints "started, fd=N")
Reader gets data	✗ — EOF immediately (n=0)
Keeping ptyc alive helps	✗ — still EOF
close(master) in ptyc is the cause	✗ — disproven

Next steps

1. Investigate why zsh exits immediately. Add logging to ptyc's child process to verify it reaches execvp. Check if the child gets a signal (SIGHUP, SIGTERM) right after exec.

2. Check the environment passed to ptyc. If env is empty, the child shell has no TERM, HOME, etc. and may exit immediately.

3. Try a long-running command instead of zsh — e.g. sleep 10 — to rule out shell-specific init failures.

4. Consider forkpty() on macOS — eliminates the timing gap entirely. forkpty() creates the PTY, forks, and returns the master fd all in one call from the same process. The reader can start before the child is even exec'd.

Environment

• macOS 26.3.1 (Darwin 25.3.0), Apple Silicon (arm64)
• Flutter 3.41.7, Developer ID signed (no sandbox)
• pql 1.4.4, dugite-native git 2.53.0
• ptyc compiled with: cc -std=c11 -Wall -Wextra -Wpedantic -Werror -O2 -D_FORTIFY_SOURCE=2 -D_DARWIN_C_SOURCE