dvislobokov/go-ml-complation

Go full-line completion dataset (go/types)

Caret-based full-line completion samples extracted from permissively licensed Go repositories with the goflc builder (Go go/parser + go/types). Each sample is an exact editor position: left_context ends at the caret, target_text is the rest of the physical line (no newline, trailing whitespace excluded), right_context follows it. Original source is reconstructable from offsets (byte offsets as used by Go tooling, plus UTF-16 offsets for IDEs); every row carries repository_id, revision, relative_path and license for attribution.

Splits are assigned per repository group (forks/near-copies together); cross-repository duplicate files are kept once. Generated code (// Code generated ... DO NOT EDIT., *.pb.go, ...), vendor/ and testdata/ are excluded. No code was built or executed (no go build, go generate, cgo or module download).

How the semantic context is produced

For every sample the builder removes the hidden target from the file (editor-equivalent snapshot) and asks go/types what is visible at the caret: RET result type, EXPECT expected type, ARG parameters (incl. receiver and closure parameters), LOCAL locals declared before the caret, FIELD/METHOD members of the method receiver, RECV/MEMBER the selector base (package x or a value type) and its accessible members after ., CALL the callee signature inside an argument list, TYPE contracts of nearby repository types. Analysis is offline: the standard library and packages of the same repository resolve; external modules are not downloaded, so their symbols are reported as unresolved (partially_resolved). Facts never come from the hidden target (per-record leakage audit; a record that fails it is emitted without facts). The samples table is the canonical, model-agnostic record without facts; the facts live in semantic (join by sample_id), and prompts already contains both rendered as training pairs.

from datasets import load_dataset
repo = "dvislobokov/go-ml-complation"
prompts = load_dataset(repo, "prompts", split="train")    # ready prompt/completion pairs
samples = load_dataset(repo, "samples", split="train")    # canonical samples (no semantic columns)
semantic = load_dataset(repo, "semantic", split="train")  # structured go/types facts, same sample_id

Examples (real rows from the prompts config)

prompt ends at the caret with <|complete|>; the model must produce completion (rest of the line + <|eol|>). Code is shortened here to its last lines; stored prompts keep up to 4,000 characters of code.

Empty line (start of a statement)

github.com/alecthomas/units · util.go:60 · sample_id 00053d44f0688a4f0e1875fd15d9ba81

<|go|><|path|>util.go
<|sem|>
RET (int64, error)
ARG s:string unitMap:map[string]float64
LOCAL f:float64 neg:bool orig:string
<|code|>
…
	return x, s[i:], nil
}

func ParseUnit(s string, unitMap map[string]float64) (int64, error) {
	// [-+]?([0-9]*(\.[0-9]*)?[a-z]+)+
	orig := s
	f := float64(0)
	neg := false

	// Consume [-+]?
	if s != "" {
		<|complete|>

completion: c := s[0]<|eol|>

After . (package or receiver members)

github.com/noiseonwires/labean · monitor.go:82 · sample_id 0001f8f6fff81b6802e7b4da4178916e

<|go|><|path|>monitor.go
<|sem|>
ARG m:taskMonitor
LOCAL PollPeriod:untyped int
FIELD scheduleStop:chan scheduledCancel cancelStop:chan scheduledCancel terminate:chan os.Signal queue:[]scheduledCancel env:*state
METHOD CancelTask(cmd string); Process(); ScheduleTaskToStop(cancelCmd string, timeout uint16)
RECV *state
MEMBER config:*appConfig; monitor:*taskMonitor; log:logger; authorized(r *http.Request)->bool
TYPE scheduledCancel: cmd:string; startTime:time.Time; timeout:time.Duration
<|code|>
…
		case task := <-m.scheduleStop:
			m.queue = append(m.queue, task)
		case task := <-m.cancelStop:
			tmp := m.queue[:0]
			for _, p := range m.queue {
				if p.cmd != task.cmd {
					tmp = append(tmp, p)
				}
			}
			m.queue = tmp
		case <-m.terminate:
			m.env.<|complete|>

completion: log.Info("Terminate request received, shutting down tasks...")<|eol|>

Inside an argument list (callee signature)

github.com/bingcicle/friendly-potato · main.go:61 · sample_id 0001f020e1be1e3f3801e287ffc06bfd

<|go|><|path|>main.go
<|sem|>
EXPECT string
LOCAL all:[]anymap eq:*string field:*string inputs:*string mode:*string p:string rex:*string
CALL strings.TrimSpace(s string)->string @0 s:string
<|code|>
…
	mode := flag.String("mode", "merge", "merge|filter")
	inputs := flag.String("in", "", "comma-separated JSONL files")
	field := flag.String("field", "", "field name for filter")
	eq := flag.String("eq", "", "value equals (string compare)")
	rex := flag.String("rex", "", "regex match")
	flag.Parse()

	if *inputs == "" { log.Fatal("--in required") }

	var all []anymap
	for _, p := range strings.Split(*inputs, ",") {
		items, err := readJSONL(strings.TrimSpace(<|complete|>

completion: p))<|eol|>

After a keyword (nearby project types)

github.com/csyezheng/a2fa · cmd/commands/remove.go:76 · sample_id 002a49707aabc942bc3575db21dd96ef

<|go|><|path|>cmd/commands/remove.go
<|sem|>
RET *removeCommand
EXPECT *removeCommand
LOCAL removeCmd:*removeCommand
TYPE removeCommand: r:*rootCommand; name:string; use:string; commands:[]Commander; Args(ctx context.Context, cd *Commandeer, args []string)->error; Commands()->[]Commander; Init(cd *Commandeer)->error; Name()->string; PreRun(cd *Commandeer, runner *Commandeer)->error; Run(ctx context.Context, cd *Commandeer, args []string)->error
<|code|>
…
}

func (c *removeCommand) Commands() []Commander {
	return c.commands
}

func newRemoveCommand() *removeCommand {
	removeCmd := &removeCommand{
		name: "remove",
		use:  "remove <account name> [user name]",
	}
	return <|complete|>

completion: removeCmd<|eol|>

Progress (updated each batch)

Last update: 2026-10-10T04:11:34Z