sift/vendor/modernc.org/libc/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

What this is

modernc.org/libc is a partial reimplementation of C libc in pure Go. It is the runtime for C programs translated to Go by the modernc.org/ccgo transpiler (notably modernc.org/sqlite). It is not a standalone library for general use; its API tracks the needs of ccgo-generated code and may change incompatibly between versions.

Compatibility note: callers of this package (e.g. modernc.org/sqlite) must use the libc version that matches their generated code (see the consumer's go.mod). Do not bump the libc version of a downstream consumer in isolation without re-translating its C sources.

Architectural split: musl-derived vs hand-written

This is the most important thing to understand before editing any file. There are two completely separate code paths selected by build constraint:

1. musl-derived (linux/amd64, arm64, loong64, ppc64le, s390x, riscv64, 386, arm) — the libc implementation is generated by transpiling musl-libc's C source to Go via ccgo/v4. The canonical build tag is:

   //go:build linux && (amd64 || arm64 || loong64 || ppc64le || s390x || riscv64 || 386 || arm)
   

   Generated files for this path: ccgo_linux_*.go (~4 MB each, the bulk of the musl translation), the per-arch musl_*.go files written by older tooling, and on amd64 the assembly stubs in abi0_linux_amd64.{go,s} (generated by qbecc --abi0wrap). The hand-written glue lives in libc_musl.go, native_musl.go (native Go replacements for the hot musl routines memcpy/memmove/memset/memcmp/fabs/strcspn on all musl targets; the transpiled originals survive in the generated files as ___musl_*, renamed by the generator), etc_musl.go, mem_brk_musl.go, mem_musl.go, memgrind_musl.go, pthread_musl.go, syscall_musl.go, aliases.go, atomic.go, atomic{32,64}.go, builtin{32,64}.go, libc_linux_statfs.go.

2. Hand-written (darwin, freebsd, netbsd, openbsd, illumos, windows, linux/mips64le) — the libc surface is implemented by hand in Go, with the negated build constraint:

   //go:build !(linux && (amd64 || arm64 || loong64 || ppc64le || s390x || riscv64 || 386 || arm))
   

   Main files: libc.go, libc_unix.go, libc_unix{1,2,3}.go, libc_<goos>.go, libc_<goos>_<goarch>.go, libc_windows*.go, pthread.go, mem.go, memgrind.go, mem_brk.go. These ports also pull in an older set of ccgo/v3-generated musl_<goos>_<goarch>.go files for the platform-independent pieces (strtod, math, network, etc.).

When you make a change you almost always need to make it in both code paths. Look at the existing build tag of the file you're editing — if it has the long linux && (amd64 || ...) form, you're in the musl path, and there's a sibling file with the negated tag that needs the parallel change.

Common commands

Build / sanity check:

• go build ./... — current platform only
• make editor — fast iteration: compile tests to /dev/null, build generator, lint generated asm
• make build_all_targets — full cross-compile sweep across all supported GOOS/GOARCH × build tags (slow; required before submitting per CONTRIBUTING.md)
• make vet — vet, filtering out known abi0 false positives

Tests:

• make short-test — go test -short plus log-grep summary
• make test — full test, 24h timeout
• make xtest — full test with -tags=ccgo.dmesg,ccgo.assert (verbose ccgo debugging)
• make libc-test — runs only TestLibc, which transpiles and runs the musl libc-test suite under testdata/nsz.repo.hu/libc-test
• make xlibc — TestLibc with debug tags
• make membrk-test — full test with -tags=libc.membrk (the sbrk-style debug allocator)
• Run one test: go test -v -run TestStrtod -count=1
• Filter TestLibc subtests: go test -v -run TestLibc -re '<regexp>' (the -re flag is defined in all_musl_test.go)

Regenerate musl-derived code:

• make generate — downloads the pinned musl tarball if needed, re-translates it via go run generator.go into $DIR (default /tmp/libc), then rebuilds and short-tests. Only works on linux. Output files (ccgo_linux_*.go, etc.) are committed to the repo and should not be hand-edited.
• make dev — same as generate but with GO_GENERATE_DEV=1 and -tags=ccgo.dmesg,ccgo.assert for ccgo debugging
• Cross-arch generation: set GO_GENERATE_GOARCH=arm64 etc. The builder.json autogen regex lists which arches CI auto-regenerates; recent commits like pi32 auto generate are these.
• Pinned musl version: internal/archive/archive.go (currently musl-7ada6dde6f9dc6a2836c3d92c2f762d35fd229e0).

Build tags

Functional tags (passed via -tags=):

• libc.membrk — replaces malloc with a fixed-size sbrk-style allocator (1 GiB on 64-bit), free becomes a no-op, heap pre-filled with PRNG bytes. Used to surface use-after-free in downstream code.
• libc.memgrind — wraps the allocator with an audit table; MemAuditStart / MemAuditReport detect leaks. Adds overhead.
• libc.dmesg — appends a per-pid log to /tmp/libc.log of major libc events.
• libc.strace — flips __ccgo_strace to true so every Xfoo entry prints a trace line. Heavy.
• libc.memexpvar — exposes allocator stats through expvar.
• ccgo.dmesg, ccgo.assert — consumed by ccgo during generation, not at runtime.

libc.membrk and libc.memgrind are mutually exclusive (the file build tags enforce it).

Symbol naming conventions

ccgo encodes C identifiers with prefixes; the same scheme is used in hand-written files for consistency:

• Xfoo — externally visible C function foo (e.g. Xmalloc, Xputchar)
• Yfoo — abi0-wrapped entry to Xfoo (linux/amd64 only, in abi0_linux_amd64.go)
• Tfoo_t — typedef'd C type foo_t (e.g. Tsize_t, Tssize_t)
• Sfoo — struct tag foo, Ufoo — union tag, Ffield — struct field
• X__foo / X___foo — C identifiers starting with _ get folded; both __errno_location and ___errno_location exist
• Lowercase xfoo, _foo — internal helpers

Ignore capi_<goos>_<goarch>.go

They are vestigial. Very old ccgo versions consulted them years ago; they survive only so that existing code in the wild does not break, and that is their entire purpose. Current ccgo discovers this libc's API by parsing the libc sources, so the manifests are not consulted and not authoritative. Adding a new Xfoo needs no capi_*.go change.

Treat them as carrying zero information about what this libc exports today. Most were last touched in 2023 (capi_windows_amd64.go 2023-02-22, capi_linux_amd64.go 2023-06-08), so they are years stale and will disagree with reality.

In particular, do not use them as evidence when diagnosing a symbol-resolution problem — grep the actual func X... definitions instead. Because they are stale uniformly, they compare equal across libc versions that genuinely differ, so "the capi files are identical, therefore libc is not the variable" is a false conclusion. That reasoning has already sent one real investigation down the wrong path.

TLS — thread-local state

A *TLS represents a C "thread". Every Xfoo takes one as the first argument. TLS is not safe for concurrent use — one goroutine per TLS.

• Inside an Xmain-style entry point: Start(main) creates the main TLS, locks the goroutine to the OS thread, and runs main.
• Library callers (e.g. sqlite): call NewTLS() per goroutine, defer tls.Close(). The first NewTLS() is the "main" thread and is not OS-thread-locked.
• Pthread support: Xpthread_create spawns a goroutine with its own TLS. The musl pthread path (pthread_musl.go) and the hand-written path (pthread.go) implement this separately.
• tls_linux_amd64.{go,s} — assembly stubs that let abi0-translated code reach tls.Alloc / tls.Free without going through Go calling conventions.

Subpackages

The repo contains a constellation of subpackages under errno/, fcntl/, pthread/, signal/, stdio/, stdlib/, sys/{types,socket,stat,vfs,random}, time/, unistd/, pwd/, grp/, netdb/, netinet/, etc. They are thin: mostly typedefs and integer constants for use by ccgo-generated callers. They are not meant to be imported by general Go code.

internal/archive — declares the pinned musl version; internal/autogen and internal/overlay — supporting files for the generator.

What is generated vs hand-written

Code generated (do not hand-edit; regenerate via make generate):

• ccgo_linux_*.go — the bulk of the musl translation, one per Linux arch
• musl_<goos>_<goarch>.go — older ccgo/v3 generation for non-Linux ports and some helpers
• abi0_linux_amd64.{go,s} — qbecc-generated abi0 wrappers
• capi_*.go — vestigial exported-symbol manifests. Not regenerated, not consulted by current ccgo, years stale; see "Ignore capi_<goos>_<goarch>.go" above. Kept only for old code in the wild.

Anything else (libc.go, libc_musl.go, libc_<goos>.go, libc_unix*.go, mem*.go, pthread*.go, aliases.go, etc.) is hand-written.

generate.go and genheaders.go are old (//go:build ignore) ccgo/v3 helpers, kept around but mostly superseded by generator.go (ccgo/v4).

Debugging artifacts in the tree

Several large files in the repo root are debug logs / surface dumps (log, log-test, log-generate, surface.old, surface.new). They are tracked but stale; do not depend on their contents.