Skip to content

COMPILER.md — sun compile Pipeline & Cache Model

Living reference for the compilation subsystem (internal/compiler). Tier-1 authority remains docs/ARCHITECTURE.md §5.2; this file documents the concrete implementation contracts that later sprints (S5 in particular) depend on.

Binary management (solcmgr.go)

  • The solc-tron version is pinned in sail.toml ([solc] version); the binary lives at <SAIL_HOME>/bin/solc-<version>[.exe].
  • Download source: https://github.com/tronprotocol/solidity/releases/download/tv_<version>/<asset> with one asset per OS — solc-static-linux, solc-macos (universal x86_64+arm64), solc-windows.exe. The SAIL_SOLC_URL environment variable overrides the full URL (mirrors, air-gapped setups, hermetic tests).
  • Checksum before execution, always. The host platform's SHA-256 digest must be pinned under [solc.checksums] keyed "<goos>/<goarch>". The digest is verified while streaming the download (a failing transfer is never installed) and re-verified on every EnsureSolc call. No digest → no execution, with an instructive error. There are no exceptions.
  • Concurrent sun processes are serialized by a lock file (solc-<version>.lock, O_CREATE|O_EXCL) with stale-lock recovery (10 min).

Invocation (pipeline.go)

Modern solc (Standard JSON, filesystem resolution):

  • solc --standard-json --base-path . --include-path lib, executed with the project root as working directory.
  • sources references only the unit's root file via urls; solc's Host Filesystem Loader resolves every transitive import. solc is the only Solidity parser in the system — no regular expressions, no import scanning anywhere in internal/compiler.
  • settings.remappings comes from sail.toml.
  • outputSelection (the S5 contract — never trimmed): abi, metadata, evm.bytecode, evm.deployedBytecode.sourceMap.

Old solc-tron fallback (no --include-path):

  • Capability is detected from the observed runtime error (the flag rejection on stderr), never from the version string, and memoized per process.
  • Inline-retry loop: the root source is sent as literal content; every Source "X" not found diagnostic makes the pipeline read X from disk (project root, then lib/) and retry with it added. Converges in O(import-graph depth), hard-capped at 256 iterations.

Compilation units & parallelism (compile.go)

  • Roots: every *.sol under src/, test/, and script/ (Foundry parity). Files under lib/ participate only as imports and are never roots.
  • Independent units run under golang.org/x/sync/errgroup bounded by GOMAXPROCS. All artifact/depfile writes are atomic (temp file + rename) and each unit owns its own paths, so the cache is safe under concurrency.
  • The solc binary is resolved (and checksum-verified) only when at least one unit is stale — a fully cached run never touches solc at all.

Depfile & incremental cache (depgraph.go, cache.go)

Per-unit depfile at out/.cache/<target-with-__-separators>.json (e.g. src__Counter.sol.json):

{
  "solcVersion": "0.8.23",
  "settingsHash": "<sha256 of the canonical settings JSON>",
  "sources": { "src/Counter.sol": "<sha256>", "...": "..." },
  "artifacts": [ "out/Counter.sol/Counter.json" ]
}
  • sources enumerates every participating file as reported by solc's output — the import graph is never derived by SunSail itself.
  • Cache hit (solc not executed) requires all of: same solc version, same settings hash, every recorded source hash intact, every recorded artifact present on disk. Anything else is a miss.
  • Transitive invalidation is structural: an import's hash is recorded in the depfile of every unit that consumed it, so touching one file stales exactly its dependent closure — no graph walk needed. Recompiling refreshes the depfile.
  • Deleting out/ removes artifacts and cache state together (full rebuild).

Artifacts (artifacts.go)

Layout: out/<File>.sol/<Contract>.json, one JSON per contract defined in the unit's root file (each project file is its own unit, so every artifact has exactly one writer; lib/ sources get no artifacts of their own):

{
  "abi": [ ... ],
  "bytecode": "6080...",
  "deployedSourceMap": "...",
  "metadata": "{...solc metadata JSON string...}"
}
  • bytecode is the creation bytecode object.
  • deployedSourceMap is the source map of the deployed bytecode — this is the exact field runner/trace.go consumes in S5.

Testing

  • make check / make e2e are hermetic: unit tests inject an in-process fake Runner; the e2e harness builds a deterministic solc stub (test/e2e/testdata/solcstub) that emulates modern and legacy solc, logs every invocation, and is installed pre-verified under a temp SAIL_HOME (with SAIL_SOLC_URL pointed at an unroutable guard address).
  • The real download + real solc compile of the scaffolded project runs under //go:build differential (make differential).