dhilst/sx

Structural compleXity optimizator

Go

0

28 commits

updated Sep 18, 2026

See the code

See what people are saying (1)

README

sx

sx is an automated refactoring tool for minimizing Go codebases.

LLMs are good at adding code: wrappers, fallback paths, adapters, one-off helpers, defensive branches, and repeated special cases. The main goal of sx is to shrink code generated by AI after implementation rounds. It measures Go code by counting AST nodes, detects refactoring candidates, applies them one at a time, and keeps only changes that still build, pass tests, and make the measured program smaller.

Every change sx makes is a type-aware AST transformation performed by a maintained Go tool: gopls for inlining and extraction, eg for example-based rewrites, and deadcode for reachability. Before a change is tried, sx predicts it and refuses the patterns the tool cannot transform correctly. After a change, sx gofmts it, builds it, runs the tests of every package that could be affected, and measures it. A change that fails any of these steps is reverted, so every change sx keeps builds and passes the tests.

sx optimizes for small code, so review its diff the way you review any refactoring, for naming and style. Correctness comes from the type-aware tools and the gates, not from the review.

It is intended to be used inside a coding agent through a skill, after an implementation round, before push as a git hook, or inside CI. You can also run it manually, but the main loop is designed for repeated use after an LLM makes the code grow.

Agent and Automation Use

The primary workflow is agent-driven. A coding agent implements a change, then runs sx to take out the bloat that round added, reviews the patch, and applies it to your tree.

This repository ships the assistant-facing instructions for that workflow:

FileClientWhat it defines
.codex/skills/sx/SKILL.mdCodexThe $sx skill: when to use sx, min, bake, and adding eg rules by hand
.claude/commands/sx.mdClaude CodeThe /sx <cmd> [param] dispatcher
.claude/commands/sx/min.mdClaude Code/sx min [auto|all]
.claude/commands/sx/bake.mdClaude Code/sx bake [path]

To use them in your own repository, copy the files to the same paths there. Both clients use the same command shape, with $ in Codex and / in slash-command clients:

{$|/}sx <cmd> [param]
CommandDefaultWhat it does
sx min autoyesMinimize in a temporary worktree, review the diff, and apply only the changes that read well
sx min allMinimize in a temporary worktree and apply every change that passed the gates
sx bakepath sx/examples/egWrite new eg rewrite templates from patterns found in your code
sx bake <path>The same, writing the templates to <path>

/sx with no command, or with an unknown one, prints the summary above.

Before you start

The agent runs sx from your module, so set it up once:

go get -tool github.com/dhilst/sx/cmd/sx
go install golang.org/x/tools/cmd/deadcode@latest
go install golang.org/x/tools/gopls@latest
go install golang.org/x/tools/cmd/eg@latest

If the helpers are missing, the agent installs them when your policy allows it. Without go get -tool, it uses an installed sx binary, or asks you for one.

Tutorial: sx min auto

Use this after an implementation round, when you want the bloat removed and want the agent to decide what is worth keeping.

  1. Commit your work. min runs on a copy of HEAD. If your tree has uncommitted or untracked files, the agent stops and asks you to either commit them first or abort. It never minimizes HEAD while changes sit beside it, because the resulting diff would not match the code you have.

  2. Ask for it:

    /sx min
    

    auto is the default, so /sx min auto is the same. In Codex, use $sx min.

  3. The agent works in a disposable worktree. It runs, roughly:

    tmp=$(mktemp -d)
    git worktree add -d "$tmp/worktree" HEAD
    go tool sx refactor -apply -n 100 "$tmp/worktree"
    

    Every change is gated there: gofmt, build, the tests of each affected package, and a re-count. Anything that fails a gate or does not shrink the tree is reverted. Your checkout is not touched.

  4. The agent tests and reviews the result. It runs your project's test command in the worktree (or go test ./...) and reads the diff.

  5. It applies what reads well. In auto mode the agent keeps the changes it judges maintainable and briefly explains each one it drops. For example, it might keep an inlined one-use helper and drop an inline that left a bare { ... } block behind.

  6. The worktree is removed, and the accepted changes are left uncommitted in your tree for you to look over:

    git diff
    

An example of what the agent reports (the numbers are illustrative):

sx: 10099 -> 9871 nodes (-228) in 14 changes, 17 attempts.
Applied 12 changes:
  - inlined parseFlags, loadConfig, newClient (each called once)
  - removed unreachable legacyHandler and its "net/http/httputil" import
  - extracted a repeated retry loop in fetch.go into one function
  - strings.Index(s, "/") >= 0 -> strings.Contains(s, "/")
Dropped 2:
  - inline of render(): left a bare block with a renamed variable
  - extraction in handlers.go: the new function needs five parameters

Tutorial: sx min all

Use this when you want every gated reduction applied, for example on generated scaffolding or on a branch you will squash, and you will review the diff yourself.

/sx min all

The steps are the same as auto, except for step 5: every change that passed the gates is applied, and none is dropped for style. It is a good fit for a first pass on a large AI-written change. Follow it with a normal code review, or run /sx min auto next time.

Tutorial: sx bake

eg templates teach sx expression rewrites specific to your codebase. bake has the agent find them for you.

  1. Ask for it:

    /sx bake
    
  2. The agent looks for repeated larger-than-necessary expressions, such as a comparison against a constant, a manual loop that a standard library call replaces, or a helper wrapped in a conversion it does not need.

  3. It writes one template per rule to sx/examples/eg:

    //go:build ignore
    
    package template
    
    import "strings"
    
    func before(s, prefix string) bool { return strings.Index(s, prefix) == 0 }
    func after(s, prefix string) bool  { return strings.HasPrefix(s, prefix) }
    

    Each template's before and after have the same type, and a template never drops, duplicates, or reorders an argument that could have side effects.

  4. It validates them without writing to your code:

    go tool sx refactor -check -eg sx/examples/eg .
    

    Templates that do not parse, do not type-check under eg, or do not shrink the tree when they match are discarded.

  5. Later min runs use them automatically. sx searches examples/eg and sx/examples/eg by default, so the next /sx min applies the new rules along with everything else.

Commit the templates like any other code. They are ordinary Go files kept out of your build by //go:build ignore.

Tutorial: sx bake <path>

Use a path when your team keeps rules somewhere else, or keeps several rule sets:

/sx bake ./tools/eg

The steps are the same, except the templates are written to and validated in ./tools/eg. Because that is not a default search path, point sx at it yourself:

go tool sx refactor -check -eg ./tools/eg .
go tool sx refactor -apply -eg ./tools/eg -eg sx/examples/eg .

-eg can be repeated or take comma-separated paths. Passing it replaces the default search paths, so list every directory you want.

More examples

Minimize right after a feature lands:

Implement the export-to-CSV command, commit it, then run /sx min.

Grow the rule set before minimizing:

/sx bake
/sx min auto

Keep a shared rule set in the repository and use it in every run:

/sx bake ./examples/eg
/sx min all

Minimize a single package (ask the agent directly):

Run sx min on ./internal/storage only.

Git hook

Outside an agent, sx works as a pre-push hook. -check never writes files. It exits non-zero when a shrinking candidate exists, so the push stops until the bloat is handled:

#!/bin/sh
# .git/hooks/pre-push
exec go tool sx refactor -check -n 30 .
chmod +x .git/hooks/pre-push

When the hook fires, run /sx min (or go tool sx refactor -apply .), commit, and push again.

CI

The same check works as a CI step; see the CI Example:

go tool sx refactor -check -n 30 .

Structural Complexity

sx defines structural complexity as |AST|: the number of Go AST nodes needed to express a program. This is deliberately naive. It ignores formatting, comments, and taste so the tool has a simple objective function to optimize.

sx works by detecting refactoring candidates: dead code, deduplication, inlining, eg rewrites, and other AST/type-safe transformations. In apply mode, it applies candidates eagerly, then measures, builds, tests, and redetects after each kept change. The loop continues until there are no more candidates or the configured -n attempt limit is reached.

Who This Is For

Use sx when you want to:

  • shrink AI-generated code after an LLM-assisted development round
  • push back against wrapper-heavy, branch-heavy, duplicated LLM output
  • find the largest functions in a Go package or module
  • remove unreachable functions detected by deadcode
  • inline one-use helpers through gopls
  • factor repeated code when doing so reduces AST size
  • apply small, example-based expression rewrites with eg
  • fail CI when a shrinking candidate is available

How sx Keeps Changes Safe

Safety comes in layers, and each one can only reject a change:

LayerWhat it guarantees
Type-aware toolsEdits are made on the type-checked AST, never by text substitution. gopls preserves semantics when it inlines or extracts: it binds arguments that cannot be substituted and keeps conversions explicit. eg rewrites only expressions whose types match the template
Conservative scopeOnly unexported functions are inlined, only unreachable functions are removed, and only identical code within one package is deduplicated. Generated files and files outside the current build are never edited. Inlining also skips functions that carry //go: directives, use unsafe, or are named by assembly or //go:linkname
Predictor refusalsPatterns the tools get wrong are refused before anything is written, such as values copied after their address is taken, lost loop-carried writes, and control flow that leaves an extracted run. See Appendix A
GatesEach change must parse, gofmt, build, and pass the tests of its package and every package that imports it
RevertA change that fails any gate, or does not make the tree smaller, is undone before the next one is tried
Read-only CI mode-check never writes, and -check -apply is rejected

The tests are the final word on behaviour, so the stronger your test suite, the stronger that guarantee.

Install

In the Go module you want to shrink, add sx as a tool dependency:

go get -tool github.com/dhilst/sx/cmd/sx

Then run it with:

go tool sx .

Install the helper tools for refactoring candidates:

go install golang.org/x/tools/cmd/deadcode@latest
go install golang.org/x/tools/gopls@latest
go install golang.org/x/tools/cmd/eg@latest

You do not need every helper installed, but each missing helper disables one class of candidate. If no usable helper is available, sx refactor exits with an install message. eg counts as usable only when at least one template exists.

Inside this repository, go tool sx and go run ./cmd/sx build the same local program.

Helper Tools

sx decides which changes are worth trying, but it delegates the actual Go-aware work to maintained Go tools:

ToolLinkUsed for
deadcodegolang.org/x/tools/cmd/deadcodeFinding unreachable functions that can be deleted
goplsgolang.org/x/tools/goplsInlining calls, extracting duplicated statement runs, and repairing imports
eggolang.org/x/tools/cmd/egApplying example-based expression rewrites from template files

The tools are optional in the sense that sx can run with only the helpers you have installed. Missing helpers simply remove candidate classes:

  • without deadcode, unreachable functions are not proposed
  • without gopls, inline and deduplication candidates are not proposed
  • without eg, example rewrite templates are not applied

At least one candidate source must be available. For eg, that means both the eg binary and at least one template under examples/eg, sx/examples/eg, or a path passed with -eg.

Quick Start

Start with a clean git working tree so rejected or unwanted patches are easy to inspect and undo.

1. Measure the current module

go tool sx .

This prints the total node count and the largest functions.

2. Preview one candidate without editing files

go tool sx refactor .

Without -apply, sx refactor only reports the best candidate it would try.

3. Apply a bounded pass

go tool sx refactor -apply -n 30 .

For each attempted change, sx formats, rebuilds and repairs unused imports, re-counts nodes, runs the relevant tests, and reverts the change unless it builds, the tests pass, and the final count is smaller.

4. Review before committing

go test ./...
git diff

Every change in the diff has already been built and tested. What remains is a style review: keep the changes that read well, and drop any you would name or structure differently.

Typical Workflows

Local minimization pass

git status --short
go tool sx refactor -apply -n 30 .
go test ./...
git diff

To review the changes in smaller batches, run fewer attempts at a time:

git restore .
go tool sx refactor -apply -n 5 .

CI check

Use -check to fail when sx can see at least one likely shrinking candidate. It never writes files.

go tool sx refactor -check -n 30 .

A -check candidate is a prediction from the transformation models. -apply then confirms it with the real build, test, and measurement gates.

Disable tests for a fast exploratory run

go tool sx refactor -apply -test=false -n 30 .

This skips the per-change test gate, which makes the loop faster. Run go test ./... once at the end to restore the same guarantee.

What sx Changes

sx currently looks for four kinds of reduction.

KindHelperWhat it tries
Dead codedeadcodeRemove unreachable plain functions
InlininggoplsInline unexported functions called once
DeduplicationgoplsExtract repeated statement runs when the extraction is smaller
eg examplesegRewrite expressions using example templates

Dead Code

sx asks deadcode which plain functions are unreachable from the current program. It then tries deleting one candidate at a time, together with any import only that function used, and keeps the deletion only if the build, tests, and measured AST count all pass.

Inlining

sx finds unexported plain functions that are called exactly once in their package, then asks gopls to run the actual inline refactor and deletes the declaration once nothing refers to it. gopls owns the type-aware edit; sx owns the decision about whether the resulting patch is smaller and still valid.

The predictor works out which inlining strategy gopls will use and what it will write: the substituted expression, a var binding for any parameter that cannot be replaced by its argument, braces when names would clash, and explicit conversions where a type would otherwise be lost. A call that gopls could only inline by wrapping the body in a function literal is not attempted.

Deduplication

sx looks for repeated statement runs in a package. It prices extracting each run the way gopls would do it, with parameters for the variables the run reads, results for the ones the code after it needs, and a returned flag or error check when the run contains return. When that makes the tree smaller, sx asks gopls to extract the first copy and replaces every other copy with the same call site gopls wrote.

This is deliberately conservative. It does not try to invent arbitrary abstractions; it only attempts repeated code that can be represented as a normal Go extraction and accepted by the same build, test, and measurement gates. Runs that gopls would extract incorrectly are refused before anything is written:

  • a variable whose address the run takes, which would be copied into or out of the helper
  • a write the run makes that a loop or closure reads later
  • a break, continue, or goto that leaves the run
  • a type parameter of the enclosing function in the new signature
  • a type from a package the file does not import
  • two parameters with the same name

eg Rewrites

sx loads eg templates from configured directories, asks eg whether each template matches, and prices the rewrite over every match. A wildcard that after uses fewer times than before, as in s[:len(s)] -> s, also saves the expression it matched. The rewrite is applied only when it is selected as a candidate.

Every attempted change follows this loop:

  1. choose the highest predicted saving not already tried
  2. apply the candidate
  3. format the touched files
  4. rebuild, repairing unused imports if that is the only problem
  5. count AST nodes again
  6. run the tests of the packages that could be affected
  7. keep the change only if it builds, the tests pass, and the count went down

The predicted saving decides what to try first and whether a candidate is offered at all. Each model rebuilds what the helper tool will write and counts it, so the prediction is meant to equal the measured change; see Appendix A. The final measured count is still what decides whether the change stays.

Code Reduction Examples

These are examples of the small expression rewrites included in examples/eg.

// before
if strings.Index(name, "/") >= 0 {
	return true
}

// after
if strings.Contains(name, "/") {
	return true
}
// before
return fmt.Sprintf("%s", value)

// after
return value
// before
return time.Now().Sub(start)

// after
return time.Since(start)
// before
return bytes.Compare(a, b) == 0

// after
return bytes.Equal(a, b)
// before
return enabled == true

// after
return enabled

Checked-in templates currently cover:

fmt.Errorf("%s", s)       -> errors.New(s)
fmt.Sprintf("%s", s)      -> s
time.Now().Sub(t)         -> time.Since(t)
s[:len(s)]                -> s
x == true                 -> x
x != false                -> x
!!x                       -> x
bytes.Compare(a, b) == 0  -> bytes.Equal(a, b)
strings.Index(s, sub)>=0  -> strings.Contains(s, sub)
strings.Index(s, sub)==-1 -> !strings.Contains(s, sub)

Add Your Own eg Rewrites

eg is the Go example-based refactoring tool from golang.org/x/tools. An eg template is a Go file with a before function and an after function. Both functions must have the same type.

Minimal template

//go:build ignore

package template

func before(s string) string { return s[:len(s)] }
func after(s string) string  { return s }

To copy this into your own module:

mkdir -p sx/examples/eg
$EDITOR sx/examples/eg/full-string-slice.go
go tool sx refactor -check -eg sx/examples/eg .
go tool sx refactor -apply -eg sx/examples/eg .
go test ./...
git diff

You can also pass any template directory to sx:

go tool sx refactor -check -eg ./examples/eg .
go tool sx refactor -apply -eg ./examples/eg .

The -eg flag can be repeated:

go tool sx refactor -apply -eg ./examples/eg -eg ./team/eg .

It can also take comma-separated paths:

go tool sx refactor -apply -eg ./examples/eg,./team/eg .

Disable eg rewrites by passing an empty -eg value:

go tool sx refactor -apply -eg "" .

By default, sx searches these directories if they exist:

examples/eg
sx/examples/eg

Template-writing checklist

  • Keep before and after the same type.
  • Prefer one returned expression in each function.
  • Add imports normally when the expressions need them.
  • Use //go:build ignore so templates are not compiled into your module.
  • Avoid rules that duplicate, remove, or reorder expressions with side effects.
  • Start with narrow, obvious rewrites and let sx prove the measured saving.

Command Reference

Measure Go files:

go tool sx [-json] [-n 20] [-tests] <paths...>

Refactor a module:

go tool sx refactor [-apply] [-check] [-n 10] [-test=false] [-eg path] <dir>

Useful flags:

FlagCommandMeaning
-jsonmeasureEmit measurement output as JSON
-nmeasureNumber of functions to list; 0 lists all
-testsmeasureInclude _test.go files when measuring
-applyrefactorWrite accepted changes; without it, preview one candidate
-checkrefactorExit non-zero when a shrinking candidate is found; never writes files; cannot be combined with -apply
-nrefactorNumber of candidates to attempt; default is 10
-test=falserefactorSkip tests after each accepted-looking change
-egrefactorFile or directory of eg templates; repeatable

CI Example

name: sx

on:
  pull_request:
  push:
    branches: [main]

jobs:
  minimize:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with:
          go-version-file: go.mod
      - run: go install golang.org/x/tools/cmd/deadcode@v0.50.0
      - run: go install golang.org/x/tools/gopls@v0.23.0
      - run: go install golang.org/x/tools/cmd/eg@v0.50.0
      - run: go tool sx refactor -check -n 30 .

The versions are the ones the models in Appendix A were checked against. A newer gopls may extract or inline differently, which costs wasted attempts, not wrong results: the measured count still decides.

Troubleshooting

SymptomWhat to do
sx refactor says helper tools are missingInstall at least one of deadcode, gopls, or eg
eg templates are ignoredConfirm eg is installed and templates are in a searched path or passed with -eg
A candidate appears in -check but is not kept by -applyA gate rejected it and sx reverted it; the tree is unchanged
The diff is larger than you want to review at onceRerun with a smaller -n
A change is smaller but reads worseDrop it from the diff; sx optimizes for size, and naming and style are yours

Algorithm and Layers

The minimization loop is greedy and measurement-gated:

  1. Parse the current Go files and measure the tree as C = |AST|. The measure covers non-test .go files that match the current build constraints, skipping vendor, testdata, and directories starting with . or _. _test.go files are not counted.
  2. Compute the test scope once: the package in the target directory plus every package in its module that depends on it, directly or transitively. If go list cannot answer, the scope falls back to ./... under the target.
  3. For each attempt, run the available detectors fresh against the current tree, in this order: deadcode, inline, deduplication (both need gopls), and eg templates. A detector that errors contributes no candidates.
  4. Price each candidate with its transformation model (Appendix A) and pick the one with the highest predicted saving, −ΔN > 0, whose key has not already been tried; ties go to the earlier detector. Candidates the model refuses, or predicts would not shrink the tree, are never offered. The chosen key is marked as tried whatever the outcome.
  5. Without -apply, report that candidate and stop without writing. With -check, report it and exit non-zero. If no candidate remains, both modes exit successfully. -check and -apply are rejected together before anything is measured or written.
  6. With -apply, apply one candidate to the working tree. If the edit cannot be produced or is rejected by the apply step's own checks, report it as skipped and continue; a skipped candidate still counts as an attempt.
  7. gofmt the files whose content the change altered (a formatting failure reverts the change and aborts the run), then go build ./.... If every build error is an unused import, run gopls imports on those files and build again.
  8. Parse again and measure C'. If tests are enabled, run go test on the precomputed scope only after the build and measurement succeed.
  9. Keep the change iff it builds, tests pass, and C' < C. Otherwise, restore the files the apply step recorded.
  10. If kept, set C = C'. Either way, redetect candidates on the next attempt and repeat until no candidate remains or -n attempts have been made.

Candidate keys make retries stable across edits that move code: dead-code candidates are keyed by position, inline candidates by package and function, deduplication candidates by the content hash of the repeated run, and eg candidates by template.

Each prediction comes from a model of the transformation, computed from the type-checked AST:

KindModel
Dead code$\Delta N = -N(D) + \Delta I$
Inline$\Delta N = N(R) - N(S) - N(D) + \Delta I$
Deduplication$\Delta N = F + D\,C - (D-1)\,B$
eg$\Delta N = \sum_{m}[\,N(\mathit{after}) - N(\mathit{before}) + \sum_w (a_w - b_w)(N(m_w) - 1)\,] + \Delta I$

The terms are defined in Appendix A.

The layers are intentionally separate:

LayerResponsibility
ParserReads Go files with the standard Go parser, respecting build constraints; generated files are measured and read for references but never edited
MeasurerCounts AST nodes and reports `
DetectorFinds possible reductions: unreachable plain functions, unexported plain functions referenced exactly once in their package, identical statement runs of at least 12 nodes within one package, and matching eg templates
PredictorModels each transformation to compute its exact ΔN, and refuses the ones the helper tool would get wrong; checked against test/examples, but the gate still decides
Refactor toolPerforms the edit. gopls inlines the call and extracts the first duplicate; eg rewrites every match of one template across the tree. sx itself deletes dead functions, deletes an inlined function once nothing refers to it, and replaces the remaining duplicate copies with the call gopls generated
GateFormats touched files, builds, repairs unused imports, remeasures, and by default tests the precomputed package scope
ReverterRestores the recorded files whenever the gate fails or the measured tree is not smaller

The apply step also refuses some edits before the gate runs: an inline whose function is still referenced afterwards, is exported, or is named by assembly or //go:linkname; and an eg rewrite that touches a generated file or a file outside the current build. For a deduplication, the call site gopls writes for the first copy (declarations, the call, and any return check) is copied to every other copy.

The quality of sx depends mostly on detection quality and rewrite coverage. Better detectors produce fewer doomed candidates and find more real reductions. Better predictors waste fewer attempts. A richer, conservative eg example library gives the tool more AST/type-safe expression rewrites to try. Improving sx usually means adding one of those: a detector, a predictor filter, or an eg template that captures a common larger-to-smaller Go idiom.

Test Examples

test/examples is a library of small programs, each isolating one behaviour: <name>_before.go is the input and <name>_after.go is what sx refactor -apply makes of it. The prefix names the transformation under test (dead_, inline_, dedup_, eg_). Each file carries //go:build ignore, so the examples are not part of this module's build.

Two tests use the library:

TestWhat it checks
TestModelsMatchReality (internal/refactor)Every candidate each detector finds in each example, including the ones its model says would grow the tree, is applied for real. The measured ΔN must equal the predicted one exactly, and the result must build.
TestExamples (cmd/sx)The whole loop runs on each _before.go and must produce _after.go byte for byte.

To add an example, write <name>_before.go, then generate its expected result and review the diff before committing:

go test ./cmd/sx -run Examples -update
git diff test/examples

Both tests need deadcode, gopls, and eg. They are skipped when those are not installed, and they run in CI with the pinned versions.

Appendix A: Transformation Models

Every candidate is priced by a model of what the helper tool will write. Each model reconstructs the AST after the transformation and counts nodes; it does not estimate from source length. A candidate is offered only when $\Delta N < 0$, and it is ranked by $-\Delta N$.

A.0 Notation

  • $N(x)$: the number of ast.Node values in the subtree $x$, the same count the measure uses.
  • $\Delta N = N(\text{tree after}) - N(\text{tree before})$, over the files the measure counts.
  • $[P]$: 1 when $P$ holds and 0 otherwise.
  • $T_x$: the syntax of $x$'s type as the tool writes it.

Imports. Every transformation can make an import unused, which the repair step removes, or need one the file lacks, which gopls and eg add. $\Delta I$ is the resulting signed change in nodes: negative when imports go, positive when they arrive. Let $U$ be the imports referenced before the change and not after, and $A$ the packages referenced after it and not imported:

\Delta I = -\sum_{s \in U} N(s) \;-\; [\text{an import declaration is left empty}]
\;+\; 2\,|A| \;+\; [A \neq \emptyset \wedge \text{the file has no import declaration}]

An import spec costs 2 nodes (the path literal and the spec), or 3 when it is renamed.

A.1 Dead code

Deleting an unreachable declaration $D$ removes it and the imports only it used. Deleting code can only remove imports, never add them, so $A = \emptyset$ and the import term is never positive:

\Delta N_{\text{dead}} = -N(D) + \Delta I
= -N(D) - \sum_{s \in U} N(s) - [\text{an import declaration is left empty}]

$\Delta I$ is a signed change in nodes, not a count of imports, so it is added rather than subtracted. For example, deleting a 10-node function that was the only user of "strings" gives $\Delta N = -10 + (-2) = -12$. The doc comment is removed too, but comments are not nodes.

A.2 Inlining

Inlining the only call to $D$ replaces the syntax $S$ the call occupied with a replacement $R$, then deletes $D$:

\Delta N_{\text{inline}} = N(R) - N(S) - N(D) + \Delta I

Substitution. Parameter $p$, with argument $a_p$ and $r_p$ references in the body, is replaced by its argument iff

\neg\text{assigned}(p) \wedge \neg\text{addressed}(p) \wedge \neg\text{shadowed}(a_p)
\wedge \big(r_p \le 1 \vee \text{dup}(a_p)\big)
\wedge \neg\big(r_p = 0 \wedge (\text{effects}(a_p) \vee \text{lastref}(a_p))\big)

where $\text{dup}$ holds for identifiers, integer literals, "", 0.0, 1.0, T{}, conversions, and selections that do not indirect a pointer. Otherwise $p$ is kept in the set $K$ and bound in a declaration. With $\pi_p$ the parentheses substitution needs and $\kappa_p$ the references at which an explicit conversion $T_p(a_p)$ is needed, the substitution term is

\sigma = \sum_{p \notin K} \Big[\, r_p\,\big(N(a_p) - 1\big) + \pi_p + \kappa_p\,\big(1 + N(T_p)\big) \Big]

A conversion is needed at a reference that is not assigned to a value of the parameter's type, is assigned to an interface, or feeds type inference, when the argument's own type differs from $T_p$.

Binding. When $K \neq \emptyset$, one var declaration holds one spec for each parameter field $f$ with kept names $K_f$:

\beta = 2 + \sum_{f : K_f \neq \emptyset} \Big( 1 + |K_f| + N(T_f) + \sum_{p \in K_f} N(a_p) \Big)

Strategies. gopls chooses one strategy, which fixes $S$ and $R$:

  1. Returned expression. The body is return e and the call is inside an expression, with $K = \emptyset$. $S$ is the call:

    N(R) = N(e) + \sigma + [\text{non-trivial}]\,\big(1 + N(T_r)\big)
    
  2. Returned call as a statement. The body is return e, $e$ is itself a call, and the call is a statement, with $K = \emptyset$. $S$ is the call:

    N(R) = N(e) + \sigma
    
  3. Statements. The call is a statement, and the body has no return, defer, or labels. $S$ is the whole call statement:

    N(R) = \sum_i N(s_i) + \sigma + \beta + [\text{clash}]
    
  4. Empty body. The call is a statement. $S$ is the statement, and only arguments with effects survive:

    N(R) = [K \neq \emptyset]\,\Big(1 + |K| + \sum_{p \in K} N(a_p)\Big)
    
  5. Anything else is refused.

"Non-trivial" means the returned expression's type, or its default type for a constant, is not the declared result type $T_r$. "Clash" means the inlined statements or the binding declare a name the enclosing block already declares, so gopls keeps the braces. The refused case is literalization, func(...){...}(...). It saves only the name and the call's two nodes, and leaves an immediately invoked closure where the call was.

A.3 Extraction (deduplication)

For a run of statements with $B$ nodes repeated $D$ times, where gopls extracts the first copy and the call it writes there replaces every copy:

\Delta N_{\text{extract}} = F + D\,C - (D - 1)\,B

$F$ is what the new declaration adds besides the body it takes over, and $C$ is what replaces each copy. They follow from the run's variables and control flow:

  • $P$: parameters. Variables declared before the run and read in it.
  • $V$: results. Variables the run declares or assigns whose value the code after the run reads before overwriting it.
  • $Q$: return plumbing. When the run contains return, the enclosing function's results, plus a bool flag unless every return is if err != nil { return ..., err }.
  • $n_{ret}$: the return statements in the run.
  • $\tau$: the run ends in a top-level return, so it always returns.
  • $\epsilon$: every return in the run is an error check.
  • $Z(T)$: the size of $T$'s zero value. It is 1 for 0, "", false, or nil; $1 + N(T)$ for T{}; and 4 for *new(T).

The declaration is func newFunction(p T, ...) (R, ...) { body }, with a return appended when values come back, and every return in the body padded with zero values:

\begin{aligned}
F ={}& 5 + \sum_{p \in P} \big(2 + N(T_p)\big)
 + \big[|V| + |Q| > 0\big]\Big(1 + \sum_{v \in V}\big(1 + N(T_v)\big) + \sum_{q \in Q}\big(1 + N(T_q)\big)\Big) \\
 &+ \big[|V| + |Q| > 0 \wedge \neg\tau\big]\Big(1 + |V| + \sum_{q \in Q} Z(T_q)\Big)
 + \big[\,n_{ret} > 0 \wedge \neg\tau\,\big]\; n_{ret}\Big(\sum_{v \in V} Z(T_v) + [\neg\epsilon]\Big)
\end{aligned}

The call site is the call, an assignment when values come back, and the check that carries a return out. When the assignment cannot use := (some result was declared before the run, in a scope it cannot be redeclared in), gopls first declares with var the set $L$: the results the run declares, together with $Q$.

C = \underbrace{1 + (2 + |P|)}_{\text{statement and call}}
 + \big[\neg\tau\big]\,\big(|V| + |Q|\big)
 + [\text{no }{:=}]\sum_{t \in L}\big(4 + N(t)\big)
 + \begin{cases}
 6 + |Q| & \text{if } \epsilon \wedge \neg\tau \quad (\texttt{if err != nil \{ return ... \}}) \\
 4 + |Q| - 1 & \text{if } n_{ret} > 0 \wedge \neg\epsilon \wedge \neg\tau \quad (\texttt{if shouldReturn \{ return ... \}}) \\
 0 & \text{otherwise}
 \end{cases}

In the flag case, $|Q| - 1$ counts the enclosing function's results, without the flag. With $\tau$ the call site is return newFunction(...).

The model refuses a run, and it is never offered, when gopls would extract it into code that does not compile or that behaves differently:

RefusalWhy
A parameter or result has its address taken in the run, by &v, a pointer method, or a closuregopls passes and returns by value, so whatever holds the address keeps the helper's copy
A write in the run is not returned, but a loop or closure reads it laterThe write lands on the helper's copy and is lost
A break, continue, or goto leaves the rungopls threads it through a control value, which is not modelled
The signature needs a type parametergopls does not carry the enclosing function's type parameters over
The signature names a package the file does not importgopls does not add the import
Two parameters would share a nameA type switch declares its variable once per clause

A.4 eg rewrites

A template rewrites before(w...) to after(w...). Each match $m$ binds each wildcard $w$ to an expression $m_w$ whose type is assignable to $w$'s. With $b_w$ and $a_w$ the number of times $w$ appears in before and in after:

\Delta N_{\text{eg}} = \sum_{m \in M} \Big[\, N(\mathit{after}) - N(\mathit{before}) + \sum_{w} (a_w - b_w)\,\big(N(m_w) - 1\big) \Big] + \Delta I

Here $M$ is the set of matches in files the measure counts. eg also rewrites tests, but the measure does not see them. eg adds the imports after needs, and the repair step removes the ones before no longer uses.

A.5 Validation

The models follow gopls v0.23.0 and golang.org/x/tools v0.50.0 (eg, deadcode). TestModelsMatchReality applies every candidate in test/examples and requires the measured $\Delta N$ to equal the prediction. At the time of writing that is 55 real transformations with no mismatch: 37 extractions, 9 inlines, 6 eg rewrites, and 3 dead-code removals.

License

Apache License 2.0. See LICENSE.

Contributors

dhilst

28 commits

dhilst/sx

Structural compleXity optimizator

Go

0

28 commits

updated Sep 18, 2026

See the code

See what people are saying (1)

README

sx

sx is an automated refactoring tool for minimizing Go codebases.

LLMs are good at adding code: wrappers, fallback paths, adapters, one-off helpers, defensive branches, and repeated special cases. The main goal of sx is to shrink code generated by AI after implementation rounds. It measures Go code by counting AST nodes, detects refactoring candidates, applies them one at a time, and keeps only changes that still build, pass tests, and make the measured program smaller.

Every change sx makes is a type-aware AST transformation performed by a maintained Go tool: gopls for inlining and extraction, eg for example-based rewrites, and deadcode for reachability. Before a change is tried, sx predicts it and refuses the patterns the tool cannot transform correctly. After a change, sx gofmts it, builds it, runs the tests of every package that could be affected, and measures it. A change that fails any of these steps is reverted, so every change sx keeps builds and passes the tests.

sx optimizes for small code, so review its diff the way you review any refactoring, for naming and style. Correctness comes from the type-aware tools and the gates, not from the review.

It is intended to be used inside a coding agent through a skill, after an implementation round, before push as a git hook, or inside CI. You can also run it manually, but the main loop is designed for repeated use after an LLM makes the code grow.

Agent and Automation Use

The primary workflow is agent-driven. A coding agent implements a change, then runs sx to take out the bloat that round added, reviews the patch, and applies it to your tree.

This repository ships the assistant-facing instructions for that workflow:

FileClientWhat it defines
.codex/skills/sx/SKILL.mdCodexThe $sx skill: when to use sx, min, bake, and adding eg rules by hand
.claude/commands/sx.mdClaude CodeThe /sx <cmd> [param] dispatcher
.claude/commands/sx/min.mdClaude Code/sx min [auto|all]
.claude/commands/sx/bake.mdClaude Code/sx bake [path]

To use them in your own repository, copy the files to the same paths there. Both clients use the same command shape, with $ in Codex and / in slash-command clients:

{$|/}sx <cmd> [param]
CommandDefaultWhat it does
sx min autoyesMinimize in a temporary worktree, review the diff, and apply only the changes that read well
sx min allMinimize in a temporary worktree and apply every change that passed the gates
sx bakepath sx/examples/egWrite new eg rewrite templates from patterns found in your code
sx bake <path>The same, writing the templates to <path>

/sx with no command, or with an unknown one, prints the summary above.

Before you start

The agent runs sx from your module, so set it up once:

go get -tool github.com/dhilst/sx/cmd/sx
go install golang.org/x/tools/cmd/deadcode@latest
go install golang.org/x/tools/gopls@latest
go install golang.org/x/tools/cmd/eg@latest

If the helpers are missing, the agent installs them when your policy allows it. Without go get -tool, it uses an installed sx binary, or asks you for one.

Tutorial: sx min auto

Use this after an implementation round, when you want the bloat removed and want the agent to decide what is worth keeping.

  1. Commit your work. min runs on a copy of HEAD. If your tree has uncommitted or untracked files, the agent stops and asks you to either commit them first or abort. It never minimizes HEAD while changes sit beside it, because the resulting diff would not match the code you have.

  2. Ask for it:

    /sx min
    

    auto is the default, so /sx min auto is the same. In Codex, use $sx min.

  3. The agent works in a disposable worktree. It runs, roughly:

    tmp=$(mktemp -d)
    git worktree add -d "$tmp/worktree" HEAD
    go tool sx refactor -apply -n 100 "$tmp/worktree"
    

    Every change is gated there: gofmt, build, the tests of each affected package, and a re-count. Anything that fails a gate or does not shrink the tree is reverted. Your checkout is not touched.

  4. The agent tests and reviews the result. It runs your project's test command in the worktree (or go test ./...) and reads the diff.

  5. It applies what reads well. In auto mode the agent keeps the changes it judges maintainable and briefly explains each one it drops. For example, it might keep an inlined one-use helper and drop an inline that left a bare { ... } block behind.

  6. The worktree is removed, and the accepted changes are left uncommitted in your tree for you to look over:

    git diff
    

An example of what the agent reports (the numbers are illustrative):

sx: 10099 -> 9871 nodes (-228) in 14 changes, 17 attempts.
Applied 12 changes:
  - inlined parseFlags, loadConfig, newClient (each called once)
  - removed unreachable legacyHandler and its "net/http/httputil" import
  - extracted a repeated retry loop in fetch.go into one function
  - strings.Index(s, "/") >= 0 -> strings.Contains(s, "/")
Dropped 2:
  - inline of render(): left a bare block with a renamed variable
  - extraction in handlers.go: the new function needs five parameters

Tutorial: sx min all

Use this when you want every gated reduction applied, for example on generated scaffolding or on a branch you will squash, and you will review the diff yourself.

/sx min all

The steps are the same as auto, except for step 5: every change that passed the gates is applied, and none is dropped for style. It is a good fit for a first pass on a large AI-written change. Follow it with a normal code review, or run /sx min auto next time.

Tutorial: sx bake

eg templates teach sx expression rewrites specific to your codebase. bake has the agent find them for you.

  1. Ask for it:

    /sx bake
    
  2. The agent looks for repeated larger-than-necessary expressions, such as a comparison against a constant, a manual loop that a standard library call replaces, or a helper wrapped in a conversion it does not need.

  3. It writes one template per rule to sx/examples/eg:

    //go:build ignore
    
    package template
    
    import "strings"
    
    func before(s, prefix string) bool { return strings.Index(s, prefix) == 0 }
    func after(s, prefix string) bool  { return strings.HasPrefix(s, prefix) }
    

    Each template's before and after have the same type, and a template never drops, duplicates, or reorders an argument that could have side effects.

  4. It validates them without writing to your code:

    go tool sx refactor -check -eg sx/examples/eg .
    

    Templates that do not parse, do not type-check under eg, or do not shrink the tree when they match are discarded.

  5. Later min runs use them automatically. sx searches examples/eg and sx/examples/eg by default, so the next /sx min applies the new rules along with everything else.

Commit the templates like any other code. They are ordinary Go files kept out of your build by //go:build ignore.

Tutorial: sx bake <path>

Use a path when your team keeps rules somewhere else, or keeps several rule sets:

/sx bake ./tools/eg

The steps are the same, except the templates are written to and validated in ./tools/eg. Because that is not a default search path, point sx at it yourself:

go tool sx refactor -check -eg ./tools/eg .
go tool sx refactor -apply -eg ./tools/eg -eg sx/examples/eg .

-eg can be repeated or take comma-separated paths. Passing it replaces the default search paths, so list every directory you want.

More examples

Minimize right after a feature lands:

Implement the export-to-CSV command, commit it, then run /sx min.

Grow the rule set before minimizing:

/sx bake
/sx min auto

Keep a shared rule set in the repository and use it in every run:

/sx bake ./examples/eg
/sx min all

Minimize a single package (ask the agent directly):

Run sx min on ./internal/storage only.

Git hook

Outside an agent, sx works as a pre-push hook. -check never writes files. It exits non-zero when a shrinking candidate exists, so the push stops until the bloat is handled:

#!/bin/sh
# .git/hooks/pre-push
exec go tool sx refactor -check -n 30 .
chmod +x .git/hooks/pre-push

When the hook fires, run /sx min (or go tool sx refactor -apply .), commit, and push again.

CI

The same check works as a CI step; see the CI Example:

go tool sx refactor -check -n 30 .

Structural Complexity

sx defines structural complexity as |AST|: the number of Go AST nodes needed to express a program. This is deliberately naive. It ignores formatting, comments, and taste so the tool has a simple objective function to optimize.

sx works by detecting refactoring candidates: dead code, deduplication, inlining, eg rewrites, and other AST/type-safe transformations. In apply mode, it applies candidates eagerly, then measures, builds, tests, and redetects after each kept change. The loop continues until there are no more candidates or the configured -n attempt limit is reached.

Who This Is For

Use sx when you want to:

  • shrink AI-generated code after an LLM-assisted development round
  • push back against wrapper-heavy, branch-heavy, duplicated LLM output
  • find the largest functions in a Go package or module
  • remove unreachable functions detected by deadcode
  • inline one-use helpers through gopls
  • factor repeated code when doing so reduces AST size
  • apply small, example-based expression rewrites with eg
  • fail CI when a shrinking candidate is available

How sx Keeps Changes Safe

Safety comes in layers, and each one can only reject a change:

LayerWhat it guarantees
Type-aware toolsEdits are made on the type-checked AST, never by text substitution. gopls preserves semantics when it inlines or extracts: it binds arguments that cannot be substituted and keeps conversions explicit. eg rewrites only expressions whose types match the template
Conservative scopeOnly unexported functions are inlined, only unreachable functions are removed, and only identical code within one package is deduplicated. Generated files and files outside the current build are never edited. Inlining also skips functions that carry //go: directives, use unsafe, or are named by assembly or //go:linkname
Predictor refusalsPatterns the tools get wrong are refused before anything is written, such as values copied after their address is taken, lost loop-carried writes, and control flow that leaves an extracted run. See Appendix A
GatesEach change must parse, gofmt, build, and pass the tests of its package and every package that imports it
RevertA change that fails any gate, or does not make the tree smaller, is undone before the next one is tried
Read-only CI mode-check never writes, and -check -apply is rejected

The tests are the final word on behaviour, so the stronger your test suite, the stronger that guarantee.

Install

In the Go module you want to shrink, add sx as a tool dependency:

go get -tool github.com/dhilst/sx/cmd/sx

Then run it with:

go tool sx .

Install the helper tools for refactoring candidates:

go install golang.org/x/tools/cmd/deadcode@latest
go install golang.org/x/tools/gopls@latest
go install golang.org/x/tools/cmd/eg@latest

You do not need every helper installed, but each missing helper disables one class of candidate. If no usable helper is available, sx refactor exits with an install message. eg counts as usable only when at least one template exists.

Inside this repository, go tool sx and go run ./cmd/sx build the same local program.

Helper Tools

sx decides which changes are worth trying, but it delegates the actual Go-aware work to maintained Go tools:

ToolLinkUsed for
deadcodegolang.org/x/tools/cmd/deadcodeFinding unreachable functions that can be deleted
goplsgolang.org/x/tools/goplsInlining calls, extracting duplicated statement runs, and repairing imports
eggolang.org/x/tools/cmd/egApplying example-based expression rewrites from template files

The tools are optional in the sense that sx can run with only the helpers you have installed. Missing helpers simply remove candidate classes:

  • without deadcode, unreachable functions are not proposed
  • without gopls, inline and deduplication candidates are not proposed
  • without eg, example rewrite templates are not applied

At least one candidate source must be available. For eg, that means both the eg binary and at least one template under examples/eg, sx/examples/eg, or a path passed with -eg.

Quick Start

Start with a clean git working tree so rejected or unwanted patches are easy to inspect and undo.

1. Measure the current module

go tool sx .

This prints the total node count and the largest functions.

2. Preview one candidate without editing files

go tool sx refactor .

Without -apply, sx refactor only reports the best candidate it would try.

3. Apply a bounded pass

go tool sx refactor -apply -n 30 .

For each attempted change, sx formats, rebuilds and repairs unused imports, re-counts nodes, runs the relevant tests, and reverts the change unless it builds, the tests pass, and the final count is smaller.

4. Review before committing

go test ./...
git diff

Every change in the diff has already been built and tested. What remains is a style review: keep the changes that read well, and drop any you would name or structure differently.

Typical Workflows

Local minimization pass

git status --short
go tool sx refactor -apply -n 30 .
go test ./...
git diff

To review the changes in smaller batches, run fewer attempts at a time:

git restore .
go tool sx refactor -apply -n 5 .

CI check

Use -check to fail when sx can see at least one likely shrinking candidate. It never writes files.

go tool sx refactor -check -n 30 .

A -check candidate is a prediction from the transformation models. -apply then confirms it with the real build, test, and measurement gates.

Disable tests for a fast exploratory run

go tool sx refactor -apply -test=false -n 30 .

This skips the per-change test gate, which makes the loop faster. Run go test ./... once at the end to restore the same guarantee.

What sx Changes

sx currently looks for four kinds of reduction.

KindHelperWhat it tries
Dead codedeadcodeRemove unreachable plain functions
InlininggoplsInline unexported functions called once
DeduplicationgoplsExtract repeated statement runs when the extraction is smaller
eg examplesegRewrite expressions using example templates

Dead Code

sx asks deadcode which plain functions are unreachable from the current program. It then tries deleting one candidate at a time, together with any import only that function used, and keeps the deletion only if the build, tests, and measured AST count all pass.

Inlining

sx finds unexported plain functions that are called exactly once in their package, then asks gopls to run the actual inline refactor and deletes the declaration once nothing refers to it. gopls owns the type-aware edit; sx owns the decision about whether the resulting patch is smaller and still valid.

The predictor works out which inlining strategy gopls will use and what it will write: the substituted expression, a var binding for any parameter that cannot be replaced by its argument, braces when names would clash, and explicit conversions where a type would otherwise be lost. A call that gopls could only inline by wrapping the body in a function literal is not attempted.

Deduplication

sx looks for repeated statement runs in a package. It prices extracting each run the way gopls would do it, with parameters for the variables the run reads, results for the ones the code after it needs, and a returned flag or error check when the run contains return. When that makes the tree smaller, sx asks gopls to extract the first copy and replaces every other copy with the same call site gopls wrote.

This is deliberately conservative. It does not try to invent arbitrary abstractions; it only attempts repeated code that can be represented as a normal Go extraction and accepted by the same build, test, and measurement gates. Runs that gopls would extract incorrectly are refused before anything is written:

  • a variable whose address the run takes, which would be copied into or out of the helper
  • a write the run makes that a loop or closure reads later
  • a break, continue, or goto that leaves the run
  • a type parameter of the enclosing function in the new signature
  • a type from a package the file does not import
  • two parameters with the same name

eg Rewrites

sx loads eg templates from configured directories, asks eg whether each template matches, and prices the rewrite over every match. A wildcard that after uses fewer times than before, as in s[:len(s)] -> s, also saves the expression it matched. The rewrite is applied only when it is selected as a candidate.

Every attempted change follows this loop:

  1. choose the highest predicted saving not already tried
  2. apply the candidate
  3. format the touched files
  4. rebuild, repairing unused imports if that is the only problem
  5. count AST nodes again
  6. run the tests of the packages that could be affected
  7. keep the change only if it builds, the tests pass, and the count went down

The predicted saving decides what to try first and whether a candidate is offered at all. Each model rebuilds what the helper tool will write and counts it, so the prediction is meant to equal the measured change; see Appendix A. The final measured count is still what decides whether the change stays.

Code Reduction Examples

These are examples of the small expression rewrites included in examples/eg.

// before
if strings.Index(name, "/") >= 0 {
	return true
}

// after
if strings.Contains(name, "/") {
	return true
}
// before
return fmt.Sprintf("%s", value)

// after
return value
// before
return time.Now().Sub(start)

// after
return time.Since(start)
// before
return bytes.Compare(a, b) == 0

// after
return bytes.Equal(a, b)
// before
return enabled == true

// after
return enabled

Checked-in templates currently cover:

fmt.Errorf("%s", s)       -> errors.New(s)
fmt.Sprintf("%s", s)      -> s
time.Now().Sub(t)         -> time.Since(t)
s[:len(s)]                -> s
x == true                 -> x
x != false                -> x
!!x                       -> x
bytes.Compare(a, b) == 0  -> bytes.Equal(a, b)
strings.Index(s, sub)>=0  -> strings.Contains(s, sub)
strings.Index(s, sub)==-1 -> !strings.Contains(s, sub)

Add Your Own eg Rewrites

eg is the Go example-based refactoring tool from golang.org/x/tools. An eg template is a Go file with a before function and an after function. Both functions must have the same type.

Minimal template

//go:build ignore

package template

func before(s string) string { return s[:len(s)] }
func after(s string) string  { return s }

To copy this into your own module:

mkdir -p sx/examples/eg
$EDITOR sx/examples/eg/full-string-slice.go
go tool sx refactor -check -eg sx/examples/eg .
go tool sx refactor -apply -eg sx/examples/eg .
go test ./...
git diff

You can also pass any template directory to sx:

go tool sx refactor -check -eg ./examples/eg .
go tool sx refactor -apply -eg ./examples/eg .

The -eg flag can be repeated:

go tool sx refactor -apply -eg ./examples/eg -eg ./team/eg .

It can also take comma-separated paths:

go tool sx refactor -apply -eg ./examples/eg,./team/eg .

Disable eg rewrites by passing an empty -eg value:

go tool sx refactor -apply -eg "" .

By default, sx searches these directories if they exist:

examples/eg
sx/examples/eg

Template-writing checklist

  • Keep before and after the same type.
  • Prefer one returned expression in each function.
  • Add imports normally when the expressions need them.
  • Use //go:build ignore so templates are not compiled into your module.
  • Avoid rules that duplicate, remove, or reorder expressions with side effects.
  • Start with narrow, obvious rewrites and let sx prove the measured saving.

Command Reference

Measure Go files:

go tool sx [-json] [-n 20] [-tests] <paths...>

Refactor a module:

go tool sx refactor [-apply] [-check] [-n 10] [-test=false] [-eg path] <dir>

Useful flags:

FlagCommandMeaning
-jsonmeasureEmit measurement output as JSON
-nmeasureNumber of functions to list; 0 lists all
-testsmeasureInclude _test.go files when measuring
-applyrefactorWrite accepted changes; without it, preview one candidate
-checkrefactorExit non-zero when a shrinking candidate is found; never writes files; cannot be combined with -apply
-nrefactorNumber of candidates to attempt; default is 10
-test=falserefactorSkip tests after each accepted-looking change
-egrefactorFile or directory of eg templates; repeatable

CI Example

name: sx

on:
  pull_request:
  push:
    branches: [main]

jobs:
  minimize:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with:
          go-version-file: go.mod
      - run: go install golang.org/x/tools/cmd/deadcode@v0.50.0
      - run: go install golang.org/x/tools/gopls@v0.23.0
      - run: go install golang.org/x/tools/cmd/eg@v0.50.0
      - run: go tool sx refactor -check -n 30 .

The versions are the ones the models in Appendix A were checked against. A newer gopls may extract or inline differently, which costs wasted attempts, not wrong results: the measured count still decides.

Troubleshooting

SymptomWhat to do
sx refactor says helper tools are missingInstall at least one of deadcode, gopls, or eg
eg templates are ignoredConfirm eg is installed and templates are in a searched path or passed with -eg
A candidate appears in -check but is not kept by -applyA gate rejected it and sx reverted it; the tree is unchanged
The diff is larger than you want to review at onceRerun with a smaller -n
A change is smaller but reads worseDrop it from the diff; sx optimizes for size, and naming and style are yours

Algorithm and Layers

The minimization loop is greedy and measurement-gated:

  1. Parse the current Go files and measure the tree as C = |AST|. The measure covers non-test .go files that match the current build constraints, skipping vendor, testdata, and directories starting with . or _. _test.go files are not counted.
  2. Compute the test scope once: the package in the target directory plus every package in its module that depends on it, directly or transitively. If go list cannot answer, the scope falls back to ./... under the target.
  3. For each attempt, run the available detectors fresh against the current tree, in this order: deadcode, inline, deduplication (both need gopls), and eg templates. A detector that errors contributes no candidates.
  4. Price each candidate with its transformation model (Appendix A) and pick the one with the highest predicted saving, −ΔN > 0, whose key has not already been tried; ties go to the earlier detector. Candidates the model refuses, or predicts would not shrink the tree, are never offered. The chosen key is marked as tried whatever the outcome.
  5. Without -apply, report that candidate and stop without writing. With -check, report it and exit non-zero. If no candidate remains, both modes exit successfully. -check and -apply are rejected together before anything is measured or written.
  6. With -apply, apply one candidate to the working tree. If the edit cannot be produced or is rejected by the apply step's own checks, report it as skipped and continue; a skipped candidate still counts as an attempt.
  7. gofmt the files whose content the change altered (a formatting failure reverts the change and aborts the run), then go build ./.... If every build error is an unused import, run gopls imports on those files and build again.
  8. Parse again and measure C'. If tests are enabled, run go test on the precomputed scope only after the build and measurement succeed.
  9. Keep the change iff it builds, tests pass, and C' < C. Otherwise, restore the files the apply step recorded.
  10. If kept, set C = C'. Either way, redetect candidates on the next attempt and repeat until no candidate remains or -n attempts have been made.

Candidate keys make retries stable across edits that move code: dead-code candidates are keyed by position, inline candidates by package and function, deduplication candidates by the content hash of the repeated run, and eg candidates by template.

Each prediction comes from a model of the transformation, computed from the type-checked AST:

KindModel
Dead code$\Delta N = -N(D) + \Delta I$
Inline$\Delta N = N(R) - N(S) - N(D) + \Delta I$
Deduplication$\Delta N = F + D\,C - (D-1)\,B$
eg$\Delta N = \sum_{m}[\,N(\mathit{after}) - N(\mathit{before}) + \sum_w (a_w - b_w)(N(m_w) - 1)\,] + \Delta I$

The terms are defined in Appendix A.

The layers are intentionally separate:

LayerResponsibility
ParserReads Go files with the standard Go parser, respecting build constraints; generated files are measured and read for references but never edited
MeasurerCounts AST nodes and reports `
DetectorFinds possible reductions: unreachable plain functions, unexported plain functions referenced exactly once in their package, identical statement runs of at least 12 nodes within one package, and matching eg templates
PredictorModels each transformation to compute its exact ΔN, and refuses the ones the helper tool would get wrong; checked against test/examples, but the gate still decides
Refactor toolPerforms the edit. gopls inlines the call and extracts the first duplicate; eg rewrites every match of one template across the tree. sx itself deletes dead functions, deletes an inlined function once nothing refers to it, and replaces the remaining duplicate copies with the call gopls generated
GateFormats touched files, builds, repairs unused imports, remeasures, and by default tests the precomputed package scope
ReverterRestores the recorded files whenever the gate fails or the measured tree is not smaller

The apply step also refuses some edits before the gate runs: an inline whose function is still referenced afterwards, is exported, or is named by assembly or //go:linkname; and an eg rewrite that touches a generated file or a file outside the current build. For a deduplication, the call site gopls writes for the first copy (declarations, the call, and any return check) is copied to every other copy.

The quality of sx depends mostly on detection quality and rewrite coverage. Better detectors produce fewer doomed candidates and find more real reductions. Better predictors waste fewer attempts. A richer, conservative eg example library gives the tool more AST/type-safe expression rewrites to try. Improving sx usually means adding one of those: a detector, a predictor filter, or an eg template that captures a common larger-to-smaller Go idiom.

Test Examples

test/examples is a library of small programs, each isolating one behaviour: <name>_before.go is the input and <name>_after.go is what sx refactor -apply makes of it. The prefix names the transformation under test (dead_, inline_, dedup_, eg_). Each file carries //go:build ignore, so the examples are not part of this module's build.

Two tests use the library:

TestWhat it checks
TestModelsMatchReality (internal/refactor)Every candidate each detector finds in each example, including the ones its model says would grow the tree, is applied for real. The measured ΔN must equal the predicted one exactly, and the result must build.
TestExamples (cmd/sx)The whole loop runs on each _before.go and must produce _after.go byte for byte.

To add an example, write <name>_before.go, then generate its expected result and review the diff before committing:

go test ./cmd/sx -run Examples -update
git diff test/examples

Both tests need deadcode, gopls, and eg. They are skipped when those are not installed, and they run in CI with the pinned versions.

Appendix A: Transformation Models

Every candidate is priced by a model of what the helper tool will write. Each model reconstructs the AST after the transformation and counts nodes; it does not estimate from source length. A candidate is offered only when $\Delta N < 0$, and it is ranked by $-\Delta N$.

A.0 Notation

  • $N(x)$: the number of ast.Node values in the subtree $x$, the same count the measure uses.
  • $\Delta N = N(\text{tree after}) - N(\text{tree before})$, over the files the measure counts.
  • $[P]$: 1 when $P$ holds and 0 otherwise.
  • $T_x$: the syntax of $x$'s type as the tool writes it.

Imports. Every transformation can make an import unused, which the repair step removes, or need one the file lacks, which gopls and eg add. $\Delta I$ is the resulting signed change in nodes: negative when imports go, positive when they arrive. Let $U$ be the imports referenced before the change and not after, and $A$ the packages referenced after it and not imported:

\Delta I = -\sum_{s \in U} N(s) \;-\; [\text{an import declaration is left empty}]
\;+\; 2\,|A| \;+\; [A \neq \emptyset \wedge \text{the file has no import declaration}]

An import spec costs 2 nodes (the path literal and the spec), or 3 when it is renamed.

A.1 Dead code

Deleting an unreachable declaration $D$ removes it and the imports only it used. Deleting code can only remove imports, never add them, so $A = \emptyset$ and the import term is never positive:

\Delta N_{\text{dead}} = -N(D) + \Delta I
= -N(D) - \sum_{s \in U} N(s) - [\text{an import declaration is left empty}]

$\Delta I$ is a signed change in nodes, not a count of imports, so it is added rather than subtracted. For example, deleting a 10-node function that was the only user of "strings" gives $\Delta N = -10 + (-2) = -12$. The doc comment is removed too, but comments are not nodes.

A.2 Inlining

Inlining the only call to $D$ replaces the syntax $S$ the call occupied with a replacement $R$, then deletes $D$:

\Delta N_{\text{inline}} = N(R) - N(S) - N(D) + \Delta I

Substitution. Parameter $p$, with argument $a_p$ and $r_p$ references in the body, is replaced by its argument iff

\neg\text{assigned}(p) \wedge \neg\text{addressed}(p) \wedge \neg\text{shadowed}(a_p)
\wedge \big(r_p \le 1 \vee \text{dup}(a_p)\big)
\wedge \neg\big(r_p = 0 \wedge (\text{effects}(a_p) \vee \text{lastref}(a_p))\big)

where $\text{dup}$ holds for identifiers, integer literals, "", 0.0, 1.0, T{}, conversions, and selections that do not indirect a pointer. Otherwise $p$ is kept in the set $K$ and bound in a declaration. With $\pi_p$ the parentheses substitution needs and $\kappa_p$ the references at which an explicit conversion $T_p(a_p)$ is needed, the substitution term is

\sigma = \sum_{p \notin K} \Big[\, r_p\,\big(N(a_p) - 1\big) + \pi_p + \kappa_p\,\big(1 + N(T_p)\big) \Big]

A conversion is needed at a reference that is not assigned to a value of the parameter's type, is assigned to an interface, or feeds type inference, when the argument's own type differs from $T_p$.

Binding. When $K \neq \emptyset$, one var declaration holds one spec for each parameter field $f$ with kept names $K_f$:

\beta = 2 + \sum_{f : K_f \neq \emptyset} \Big( 1 + |K_f| + N(T_f) + \sum_{p \in K_f} N(a_p) \Big)

Strategies. gopls chooses one strategy, which fixes $S$ and $R$:

  1. Returned expression. The body is return e and the call is inside an expression, with $K = \emptyset$. $S$ is the call:

    N(R) = N(e) + \sigma + [\text{non-trivial}]\,\big(1 + N(T_r)\big)
    
  2. Returned call as a statement. The body is return e, $e$ is itself a call, and the call is a statement, with $K = \emptyset$. $S$ is the call:

    N(R) = N(e) + \sigma
    
  3. Statements. The call is a statement, and the body has no return, defer, or labels. $S$ is the whole call statement:

    N(R) = \sum_i N(s_i) + \sigma + \beta + [\text{clash}]
    
  4. Empty body. The call is a statement. $S$ is the statement, and only arguments with effects survive:

    N(R) = [K \neq \emptyset]\,\Big(1 + |K| + \sum_{p \in K} N(a_p)\Big)
    
  5. Anything else is refused.

"Non-trivial" means the returned expression's type, or its default type for a constant, is not the declared result type $T_r$. "Clash" means the inlined statements or the binding declare a name the enclosing block already declares, so gopls keeps the braces. The refused case is literalization, func(...){...}(...). It saves only the name and the call's two nodes, and leaves an immediately invoked closure where the call was.

A.3 Extraction (deduplication)

For a run of statements with $B$ nodes repeated $D$ times, where gopls extracts the first copy and the call it writes there replaces every copy:

\Delta N_{\text{extract}} = F + D\,C - (D - 1)\,B

$F$ is what the new declaration adds besides the body it takes over, and $C$ is what replaces each copy. They follow from the run's variables and control flow:

  • $P$: parameters. Variables declared before the run and read in it.
  • $V$: results. Variables the run declares or assigns whose value the code after the run reads before overwriting it.
  • $Q$: return plumbing. When the run contains return, the enclosing function's results, plus a bool flag unless every return is if err != nil { return ..., err }.
  • $n_{ret}$: the return statements in the run.
  • $\tau$: the run ends in a top-level return, so it always returns.
  • $\epsilon$: every return in the run is an error check.
  • $Z(T)$: the size of $T$'s zero value. It is 1 for 0, "", false, or nil; $1 + N(T)$ for T{}; and 4 for *new(T).

The declaration is func newFunction(p T, ...) (R, ...) { body }, with a return appended when values come back, and every return in the body padded with zero values:

\begin{aligned}
F ={}& 5 + \sum_{p \in P} \big(2 + N(T_p)\big)
 + \big[|V| + |Q| > 0\big]\Big(1 + \sum_{v \in V}\big(1 + N(T_v)\big) + \sum_{q \in Q}\big(1 + N(T_q)\big)\Big) \\
 &+ \big[|V| + |Q| > 0 \wedge \neg\tau\big]\Big(1 + |V| + \sum_{q \in Q} Z(T_q)\Big)
 + \big[\,n_{ret} > 0 \wedge \neg\tau\,\big]\; n_{ret}\Big(\sum_{v \in V} Z(T_v) + [\neg\epsilon]\Big)
\end{aligned}

The call site is the call, an assignment when values come back, and the check that carries a return out. When the assignment cannot use := (some result was declared before the run, in a scope it cannot be redeclared in), gopls first declares with var the set $L$: the results the run declares, together with $Q$.

C = \underbrace{1 + (2 + |P|)}_{\text{statement and call}}
 + \big[\neg\tau\big]\,\big(|V| + |Q|\big)
 + [\text{no }{:=}]\sum_{t \in L}\big(4 + N(t)\big)
 + \begin{cases}
 6 + |Q| & \text{if } \epsilon \wedge \neg\tau \quad (\texttt{if err != nil \{ return ... \}}) \\
 4 + |Q| - 1 & \text{if } n_{ret} > 0 \wedge \neg\epsilon \wedge \neg\tau \quad (\texttt{if shouldReturn \{ return ... \}}) \\
 0 & \text{otherwise}
 \end{cases}

In the flag case, $|Q| - 1$ counts the enclosing function's results, without the flag. With $\tau$ the call site is return newFunction(...).

The model refuses a run, and it is never offered, when gopls would extract it into code that does not compile or that behaves differently:

RefusalWhy
A parameter or result has its address taken in the run, by &v, a pointer method, or a closuregopls passes and returns by value, so whatever holds the address keeps the helper's copy
A write in the run is not returned, but a loop or closure reads it laterThe write lands on the helper's copy and is lost
A break, continue, or goto leaves the rungopls threads it through a control value, which is not modelled
The signature needs a type parametergopls does not carry the enclosing function's type parameters over
The signature names a package the file does not importgopls does not add the import
Two parameters would share a nameA type switch declares its variable once per clause

A.4 eg rewrites

A template rewrites before(w...) to after(w...). Each match $m$ binds each wildcard $w$ to an expression $m_w$ whose type is assignable to $w$'s. With $b_w$ and $a_w$ the number of times $w$ appears in before and in after:

\Delta N_{\text{eg}} = \sum_{m \in M} \Big[\, N(\mathit{after}) - N(\mathit{before}) + \sum_{w} (a_w - b_w)\,\big(N(m_w) - 1\big) \Big] + \Delta I

Here $M$ is the set of matches in files the measure counts. eg also rewrites tests, but the measure does not see them. eg adds the imports after needs, and the repair step removes the ones before no longer uses.

A.5 Validation

The models follow gopls v0.23.0 and golang.org/x/tools v0.50.0 (eg, deadcode). TestModelsMatchReality applies every candidate in test/examples and requires the measured $\Delta N$ to equal the prediction. At the time of writing that is 55 real transformations with no mismatch: 37 extractions, 9 inlines, 6 eg rewrites, and 3 dead-code removals.

License

Apache License 2.0. See LICENSE.

Contributors

dhilst

28 commits

Languages

Go

100.0%