RegionIO/CLAUDE.md
Master290 e0fdddd887 Write chunk NBT vanilla reads: flat root, byte section Y, four-bit block palettes
Our region files were not Anvil. Three things stood between them and vanilla's
deserializer, and each is enough on its own:

  * everything was nested under a "Level" compound. Chunk data lived there until
    1.18; SerializableChunkData builds a flat root and never looks for the key.
  * a section's Y was an Int. Vanilla writes putByte and reads getByteOr, so
    every section of ours decodes as index 0 and overwrites the one before it.
  * block palettes were packed at ceil(log2(size)) bits. Strategy's tableswitch
    routes bit counts 1 through 4 to the same four-bit configuration, so a
    palette of 2..16 states is four bits wide on disk. Ours were one to three,
    which makes the long array a quarter of the length vanilla computes, and
    SimpleBitStorage rejects the section outright rather than misreading it.

Biome containers were already right: Strategy has no such floor for them, and a
Global configuration above three bits still stores palette indices, just at its
own width. The suspicion that biomes collapsed on save/load was unfounded --
what let it stand is that every round-trip test in this package set blocks and
asserted blocks, so nothing proved biomes survived. They do now, per cell, for
every palette width a section can hold.

Verified against vanilla rather than against our reading of it:
tools/VanillaChunkFormatCheck.java opens a region file we wrote using vanilla's
RegionFile, NbtIo, Strategy and SimpleBitStorage. Sixteen generated chunks pass.
Reverting either the Y type or the palette floor makes it fail with vanilla's
own message -- "Invalid length given for storage, got: 64 but expected: 256" --
so the check can fail, which is the only reason to trust it passing.

Reading a world the official server generated is what the surface-height parity
capture in CLAUDE.md has always needed, and this is half of it.
2026-07-27 03:44:48 +03:00

10 KiB
Raw Blame History

RegionIO — working notes

A Minecraft Java Edition server core in Go, targeting 26.1.2 / protocol 775. The goal is vanilla fidelity, not a lookalike: where vanilla behaviour is known, match it exactly.

README.md describes what the server does. This file is about how to work on it.

Commands

go build ./... && go vet ./...
make test          # go test ./...
make test-race     # the race-sensitive subset
make verify        # both

go run ./cmd/regionio -seed 12345            # serves on 0.0.0.0:25565
go run ./cmd/regionio -seed 12345 -world ""  # in-memory world, nothing read from or written to disk
go run ./cmd/gendump                         # client-free generator diagnostics

Hard rules

No third-party dependencies. go.mod has none. NBT, zlib framing, MD5 seeding, noise, the density-function interpreter — all in-tree. Keep it that way.

Bump generatorVersion whenever the generator's output changes. It lives in internal/world/store.go and is stamped into every saved chunk; a mismatch makes the chunk regenerate. Without a bump, the chunks already on disk keep their old terrain and your change looks like it did nothing in exactly the area you are standing in — chunkAt prefers the store over the generator. TestGeneratorVersionStampRejectsStaleChunks covers the mechanism. For quick iteration, -world "" sidesteps persistence entirely. (The seed is guarded separately, by the world metadata file, and a seed mismatch is a hard error rather than a regeneration.)

Verify vanilla behaviour against the jar; don't recall it. See below.

Vanilla ground truth

None of these are redistributable, so all are gitignored. Obtain server.jar from Mojang.

Source What it gives you
.refjava/ Decompiled classes for the parts we port: Climate, SurfaceRules, SurfaceSystem, Aquifer, DensityFunctions, NoiseRouterData, NoiseBasedChunkGenerator, TerrainProvider, PalettedContainer, LevelChunkSection, ClientboundLevelChunkPacketData
versions/26.1.2/server-26.1.2.jar The real (deobfuscated) server. The outer server.jar is only a bundler
generated/reports/ Datagen output: blocks.json, registries.json, packets.json, biome_parameters/

The inner jar also carries the complete worldgen datapack, which is the source for everything we still approximate: 259 placed_feature, 222 configured_feature, 66 biome (with per-stage features and carvers), 5 configured_carver, 35 structure, 1359 structure NBTs — 648 KB for the feature/biome/carver set.

Three ways in, cheapest first:

unzip -p versions/26.1.2/server-26.1.2.jar data/minecraft/worldgen/biome/plains.json
javap -p -c -classpath versions/26.1.2/server-26.1.2.jar net.minecraft.world.level.chunk.Strategy

javap settles questions the decompiled subset does not cover. It is how the biome palette threshold was pinned down: Strategy$2 switches {0..3} and everything above falls through to the global palette, which .refjava/ alone could not show.

When a constant has to come from vanilla's runtime rather than its source or reports, dump it with a Java program run against the jar. tools/VanillaBlockStateDump.java is the one that exists — it walks the block-state registry and emits, per state, light opacity and emission, voxel face-occlusion masks, and the blocksMotion / fluid / leaves flags the heightmaps need, into internal/world/block_properties.bin. Rebuild it with:

CP="versions/26.1.2/server-26.1.2.jar;$(find libraries -name '*.jar' | tr '\n' ';')"
javac -nowarn -cp "$CP" -d <out> tools/VanillaBlockStateDump.java
java -cp "<out>;$CP" VanillaBlockStateDump > internal/world/block_properties.bin

The 39 jars under libraries/ are required; the server jar alone will not boot the registry. Bump the format version in both the Java and internal/world/block_properties.go whenever the layout or a flag's meaning changes.

The same trick verifies output, not just constants. tools/VanillaChunkFormatCheck.java opens a region file we wrote with vanilla's own RegionFile, NbtIo, Strategy and SimpleBitStorage and fails if the root is not flat, a section Y is not a byte, or a palette array is not the width vanilla derives from its palette size. Note the server jar is signed, so a helper cannot be declared inside a net.minecraft.* package — reach protected members by reflection instead.

Substring-matching block names is how the light table was wrong before (grass_block matched "grass", bedrock matched "bed"); don't reintroduce that shape of guess anywhere.

Layout

cmd/regionio/     entry point (flags, listener, graceful shutdown)
cmd/gendump/      client-free generator diagnostics — biome spread, surface blocks,
                  subsurface banding, deep-layer composition, bedrock band, fluid census,
                  cross-section
cmd/genblocks/    generates internal/worldgen/generated_blocks.go from the block report
cmd/genlight/     legacy light-table generator, superseded by tools/VanillaBlockStateDump.java
tools/            Java dumpers run against the jar, plus their Go-side fixtures
internal/protocol/  VarInt, framing, compression, packet IDs
internal/nbt/       NBT codec (modified UTF-8)
internal/registry/  28 embedded synced registries + tags, verbatim from vanilla
internal/world/     chunk model, wire encoder, cache + tickets, Anvil store, lighting,
                    the chunk generator itself (vanilla.go), decoration
internal/worldgen/  the library: noise, density-function interpreter, surface rules,
                    aquifer, climate/biome finder, embedded datapack under data/
internal/network/   per-connection state machine
internal/server/    shared core: config, sessions, status, profiles, entity loops

Note the split: internal/worldgen is a library over the datapack; the chunk generator that drives it is internal/world/vanilla.go.

Fidelity status

Bit-exact and parity-tested — treat as settled, change only with a vanilla reference in hand: random.go (Xoroshiro128++, upgradeSeedTo128bit, MD5 seeding), improved_noise.go, perlin_noise.go, normal_noise.go, blended.go, spline.go, density.go, the 4×8×4 cell grid with trilinear interpolation, the climate/biome finder, and the chunk wire encoder (TestGoldenAgainstVanilla compares bytes against a real vanilla chunk).

Ported from the decompiled source and checked by behaviour rather than by bytes — faithful as far as we can tell, but no vanilla capture confirms them: the aquifer (worldgen/aquifer.go, the whole of Aquifer.NoiseBasedAquifer bar shouldScheduleFluidUpdate), the surface-rule interpreter (worldgen/surface.go, every condition the overworld tree uses), the badlands clay bands (worldgen/bandlands.go), and the column pass in world/vanilla.go that mirrors SurfaceSystem.

The whole noise_router is parsed. preliminary_surface_level is reachable through od.PreliminarySurfaceLevelAt, which quart-aligns and memoises across chunks the way NoiseChunk does; the vein_* keys are parsed but nothing reads them yet.

The rule tree is seed-bound: od.SurfaceRule() returns a *SurfaceRuleSet compiled against the world's RandomState, because noise_threshold and vertical_gradient cannot work without it. Get a context from NewContext, call BeginColumn per column, then Apply per block.

Two things a surface rule cannot do silently: name a block that is not in worldgen/blockids.go (that is a parse error now — it used to resolve to 0 and get dropped, which is how deepslate went missing from the whole world), and be added without the world's height bounds (anchors resolve at parse time).

Known gaps, roughly in order of how visible they are:

  • No carvers and no ore veins. Caves come only from the density router; configured_carver is not extracted and the OreVeinifier over the parsed vein_* keys is not written.
  • No PerlinSimplexNoise, so two corners of Biome.coldEnoughToSnow are missing: the height adjustment that cools a column above sea level + 17, and the frozen temperature modifier that warms patches of frozen ocean. Base temperatures are real (worldgen/biome_temperature.go, extracted from the jar's 65 biome JSONs). The overworld tree reaches minecraft:temperature from exactly one rule — whether a hole in a frozen ocean floor ices over — so neither omission is visible; snowy peaks come from biome selection, not from this condition.
  • erodedBadlandsExtension and frozenOceanExtension are not ported. SurfaceSystem runs both outside the rule tree, for eroded badlands spires and frozen-ocean icebergs.
  • Decoration is hand-written heuristics, not the vanilla feature system: oak trees only and without a biome check (so oaks grow in deserts), ores that cannot generate below y≈0 because they only replace stone and never deepslate, and no grass, flowers, lakes or springs.

Testing worldgen

make verify is the gate, but most generator defects are invisible to it — they show up as terrain that looks wrong. cmd/gendump exists for that: biome distribution, top surface blocks, subsurface banding, deep-layer composition, the bedrock band, the underground fluid census, the badlands clay bands, and an ASCII cross-section, with no client involved. Add an assertion to it whenever you fix a class of defect; the bedrock-band check is the model — it prints per-layer counts and fails loudly on any air or water in the floor. The fluid census is the same shape: it prints water as a share of the open volume under inland chunks (3.8% now, 100% before the aquifer) and fails if caves flood again.

Anything gendump can assert on, prefer to also assert in a test. internal/world carries four that started as gendump checks and run under make verify: TestCavesAreDry, TestNoFluidUnderBedrock, TestGrassColumnsHaveDirt, TestDeepslateLayer.

go test -race needs cgo and a C toolchain; on a Windows box without gcc, make test-race cannot run at all.

internal/world/vanilla_parity_test.go compares surface heights against a capture from the official server and skips when the capture is absent. Note it reads a hardcoded /tmp path, so on Windows it never runs. A capture is produced by running the vanilla server headless at a known seed and reading its region files back with our own regionfile.go + nbt.