diff options
Diffstat (limited to 'doc/worldformat.txt')
| -rw-r--r-- | doc/worldformat.txt | 591 |
1 files changed, 591 insertions, 0 deletions
diff --git a/doc/worldformat.txt b/doc/worldformat.txt new file mode 100644 index 000000000..d3b1b1189 --- /dev/null +++ b/doc/worldformat.txt @@ -0,0 +1,591 @@ +============================= +Minetest World Format 22...25 +============================= + +This applies to a world format carrying the block serialization version +22...25, used at least in +- 0.4.dev-20120322 ... 0.4.dev-20120606 (22...23) +- 0.4.0 (23) +- 24 was never released as stable and existed for ~2 days + +The block serialization version does not fully specify every aspect of this +format; if compliance with this format is to be checked, it needs to be +done by detecting if the files and data indeed follows it. + +Legacy stuff +============= +Data can, in theory, be contained in the flat file directory structure +described below in Version 17, but it is not officially supported. Also you +may stumble upon all kinds of oddities in not-so-recent formats. + +Files +====== +Everything is contained in a directory, the name of which is freeform, but +often serves as the name of the world. + +Currently the authentication and ban data is stored on a per-world basis. +It can be copied over from an old world to a newly created world. + +World +|-- auth.txt ----- Authentication data +|-- env_meta.txt - Environment metadata +|-- ipban.txt ---- Banned ips/users +|-- map_meta.txt - Map metadata +|-- map.sqlite --- Map data +|-- players ------ Player directory +| |-- player1 -- Player file +| '-- Foo ------ Player file +`-- world.mt ----- World metadata + +auth.txt +--------- +Contains authentication data, player per line. + <name>:<password hash>:<privilege1,...> + +Legacy format (until 0.4.12) of password hash is <name><password> SHA1'd, +in the base64 encoding. + +Format (since 0.4.13) of password hash is #1#<salt>#<verifier>, with the parts inside <> encoded in the base64 encoding. +<verifier> is an RFC 5054 compatible SRP-2048-SHA1 verifier +of the given salt, password, and the player's name lowercased. + +Example lines: +- Player "celeron55", no password, privileges "interact" and "shout": + celeron55::interact,shout +- Player "Foo", password "bar", privilege "shout", with a legacy password hash: + foo:iEPX+SQWIR3p67lj/0zigSWTKHg:shout +- Player "Foo", password "bar", privilege "shout", with an up to date pw hash (yes it is THAT long): + foo:#1#hPpy4O3IAn1hsNK00A6wNw#Kpu6rj7McsrPCt4euTb5RA5ltF7wdcWGoYMcRngwDi11cZhPuuR9i5Bo7o6A877TgcEwoc//HNrj9EjR/CGjdyTFmNhiermZOADvd8eu32FYK1kf7RMC0rXWxCenYuOQCG4WF9mMGiyTPxC63VAjAMuc1nCZzmy6D9zt0SIKxOmteI75pAEAIee2hx4OkSXRIiU4Zrxo1Xf7QFxkMY4x77vgaPcvfmuzom0y/fU1EdSnZeopGPvzMpFx80ODFx1P34R52nmVl0W8h4GNo0k8ZiWtRCdrJxs8xIg7z5P1h3Th/BJ0lwexpdK8sQZWng8xaO5ElthNuhO8UQx1l6FgEA:shout +- Player "bar", no password, no privileges: + bar:: + +env_meta.txt +------------- +Simple global environment variables. +Example content (added indentation): + game_time = 73471 + time_of_day = 19118 + EnvArgsEnd + +ipban.txt +---------- +Banned IP addresses and usernames. +Example content (added indentation): + 123.456.78.9|foo + 123.456.78.10|bar + +map_meta.txt +------------- +Simple global map variables. +Example content (added indentation): + seed = 7980462765762429666 + [end_of_params] + +map.sqlite +----------- +Map data. +See Map File Format below. + +player1, Foo +------------- +Player data. +Filename can be anything. +See Player File Format below. + +world.mt +--------- +World metadata. +Example content (added indentation): + gameid = mesetint + +Player File Format +=================== + +- Should be pretty self-explanatory. +- Note: position is in nodes * 10 + +Example content (added indentation): + hp = 11 + name = celeron55 + pitch = 39.77 + position = (-5231.97,15,1961.41) + version = 1 + yaw = 101.37 + PlayerArgsEnd + List main 32 + Item default:torch 13 + Item default:pick_steel 1 50112 + Item experimental:tnt + Item default:cobble 99 + Item default:pick_stone 1 13104 + Item default:shovel_steel 1 51838 + Item default:dirt 61 + Item default:rail 78 + Item default:coal_lump 3 + Item default:cobble 99 + Item default:leaves 22 + Item default:gravel 52 + Item default:axe_steel 1 2045 + Item default:cobble 98 + Item default:sand 61 + Item default:water_source 94 + Item default:glass 2 + Item default:mossycobble + Item default:pick_steel 1 64428 + Item animalmaterials:bone + Item default:sword_steel + Item default:sapling + Item default:sword_stone 1 10647 + Item default:dirt 99 + Empty + Empty + Empty + Empty + Empty + Empty + Empty + Empty + EndInventoryList + List craft 9 + Empty + Empty + Empty + Empty + Empty + Empty + Empty + Empty + Empty + EndInventoryList + List craftpreview 1 + Empty + EndInventoryList + List craftresult 1 + Empty + EndInventoryList + EndInventory + +Map File Format +================ + +Minetest maps consist of MapBlocks, chunks of 16x16x16 nodes. + +In addition to the bulk node data, MapBlocks stored on disk also contain +other things. + +History +-------- +We need a bit of history in here. Initially Minetest stored maps in a +format called the "sectors" format. It was a directory/file structure like +this: + sectors2/XXX/ZZZ/YYYY +For example, the MapBlock at (0,1,-2) was this file: + sectors2/000/ffd/0001 + +Eventually Minetest outgrow this directory structure, as filesystems were +struggling under the amount of files and directories. + +Large servers seriously needed a new format, and thus the base of the +current format was invented, suggested by celeron55 and implemented by +JacobF. + +SQLite3 was slammed in, and blocks files were directly inserted as blobs +in a single table, indexed by integer primary keys, oddly mangled from +coordinates. + +Today we know that SQLite3 allows multiple primary keys (which would allow +storing coordinates separately), but the format has been kept unchanged for +that part. So, this is where it has come. +</history> + +So here goes +------------- +map.sqlite is an sqlite3 database, containing a single table, called +"blocks". It looks like this: + + CREATE TABLE `blocks` (`pos` INT NOT NULL PRIMARY KEY,`data` BLOB); + +The key +-------- +"pos" is created from the three coordinates of a MapBlock using this +algorithm, defined here in Python: + + def getBlockAsInteger(p): + return int64(p[2]*16777216 + p[1]*4096 + p[0]) + + def int64(u): + while u >= 2**63: + u -= 2**64 + while u <= -2**63: + u += 2**64 + return u + +It can be converted the other way by using this code: + + def getIntegerAsBlock(i): + x = unsignedToSigned(i % 4096, 2048) + i = int((i - x) / 4096) + y = unsignedToSigned(i % 4096, 2048) + i = int((i - y) / 4096) + z = unsignedToSigned(i % 4096, 2048) + return x,y,z + + def unsignedToSigned(i, max_positive): + if i < max_positive: + return i + else: + return i - 2*max_positive + +The blob +--------- +The blob is the data that would have otherwise gone into the file. + +See below for description. + +MapBlock serialization format +============================== +NOTE: Byte order is MSB first (big-endian). +NOTE: Zlib data is in such a format that Python's zlib at least can + directly decompress. + +u8 version +- map format version number, this one is version 22 + +u8 flags +- Flag bitmasks: + - 0x01: is_underground: Should be set to 0 if there will be no light + obstructions above the block. If/when sunlight of a block is updated + and there is no block above it, this value is checked for determining + whether sunlight comes from the top. + - 0x02: day_night_differs: Whether the lighting of the block is different + on day and night. Only blocks that have this bit set are updated when + day transforms to night. + - 0x04: lighting_expired: If true, lighting is invalid and should be + updated. If you can't calculate lighting in your generator properly, + you could try setting this 1 to everything and setting the uppermost + block in every sector as is_underground=0. I am quite sure it doesn't + work properly, though. + - 0x08: generated: True if the block has been generated. If false, block + is mostly filled with CONTENT_IGNORE and is likely to contain eg. parts + of trees of neighboring blocks. + +u8 content_width +- Number of bytes in the content (param0) fields of nodes +if map format version <= 23: + - Always 1 +if map format version >= 24: + - Always 2 + +u8 params_width +- Number of bytes used for parameters per node +- Always 2 + +zlib-compressed node data: +if content_width == 1: + - content: + u8[4096]: param0 fields + u8[4096]: param1 fields + u8[4096]: param2 fields +if content_width == 2: + - content: + u16[4096]: param0 fields + u8[4096]: param1 fields + u8[4096]: param2 fields +- The location of a node in each of those arrays is (z*16*16 + y*16 + x). + +zlib-compressed node metadata list +- content: + u16 version (=1) + u16 count of metadata + foreach count: + u16 position (p.Z*MAP_BLOCKSIZE*MAP_BLOCKSIZE + p.Y*MAP_BLOCKSIZE + p.X) + u16 type_id + u16 content_size + u8[content_size] (content of metadata) + +- Node timers +if map format version == 23: + u8 unused version (always 0) +if map format version == 24: (NOTE: Not released as stable) + u8 nodetimer_version + if nodetimer_version == 0: + (nothing else) + if nodetimer_version == 1: + u16 num_of_timers + foreach num_of_timers: + u16 timer position (z*16*16 + y*16 + x) + s32 timeout*1000 + s32 elapsed*1000 + +u8 static object version: +- Always 0 + +u16 static_object_count + +foreach static_object_count: + u8 type (object type-id) + s32 pos_x_nodes * 10000 + s32 pos_y_nodes * 10000 + s32 pos_z_nodes * 10000 + u16 data_size + u8[data_size] data + +u32 timestamp +- Timestamp when last saved, as seconds from starting the game. +- 0xffffffff = invalid/unknown timestamp, nothing should be done with the time + difference when loaded + +u8 name-id-mapping version +- Always 0 + +u16 num_name_id_mappings + +foreach num_name_id_mappings + u16 id + u16 name_len + u8[name_len] name + +- Node timers +if map format version == 25: + u8 length of the data of a single timer (always 2+4+4=10) + u16 num_of_timers + foreach num_of_timers: + u16 timer position (z*16*16 + y*16 + x) + s32 timeout*1000 + s32 elapsed*1000 + +EOF. + +Format of nodes +---------------- +A node is composed of the u8 fields param0, param1 and param2. + +if map format version <= 23: + The content id of a node is determined as so: + - If param0 < 0x80, + content_id = param0 + - Otherwise + content_id = (param0<<4) + (param2>>4) +if map format version >= 24: + The content id of a node is param0. + +The purpose of param1 and param2 depend on the definition of the node. + +The name-id-mapping +-------------------- +The mapping maps node content ids to node names. + +Node metadata format +--------------------- + +1: Generic metadata + serialized inventory + u32 len + u8[len] text + u16 len + u8[len] owner + u16 len + u8[len] infotext + u16 len + u8[len] inventory drawspec + u8 allow_text_input (bool) + u8 removal_disabled (bool) + u8 enforce_owner (bool) + u32 num_vars + foreach num_vars + u16 len + u8[len] name + u32 len + u8[len] value + +14: Sign metadata + u16 text_len + u8[text_len] text + +15: Chest metadata + serialized inventory + +16: Furnace metadata + TBD + +17: Locked Chest metadata + u16 len + u8[len] owner + serialized inventory + +Static objects +--------------- +Static objects are persistent freely moving objects in the world. + +Object types: +1: Test object +2: Item +3: Rat (deprecated) +4: Oerkki (deprecated) +5: Firefly (deprecated) +6: MobV2 (deprecated) +7: LuaEntity + +1: Item: + u8 version + version 0: + u16 len + u8[len] itemstring + +7: LuaEntity: + u8 version + version 1: + u16 len + u8[len] entity name + u32 len + u8[len] static data + s16 hp + s32 velocity.x * 10000 + s32 velocity.y * 10000 + s32 velocity.z * 10000 + s32 yaw * 1000 + +Itemstring format +------------------ +eg. 'default:dirt 5' +eg. 'default:pick_wood 21323' +eg. '"default:apple" 2' +eg. 'default:apple' +- The wear value in tools is 0...65535 +- There are also a number of older formats that you might stumble upon: +eg. 'node "default:dirt" 5' +eg. 'NodeItem default:dirt 5' +eg. 'ToolItem WPick 21323' + +Inventory serialization format +------------------------------- +- The inventory serialization format is line-based +- The newline character used is "\n" +- The end condition of a serialized inventory is always "EndInventory\n" +- All the slots in a list must always be serialized. + +Example (format does not include "---"): +--- +List foo 4 +Item default:sapling +Item default:sword_stone 1 10647 +Item default:dirt 99 +Empty +EndInventoryList +List bar 9 +Empty +Empty +Empty +Empty +Empty +Empty +Empty +Empty +Empty +EndInventoryList +EndInventory +--- + +============================================== +Minetest World Format used as of 2011-05 or so +============================================== + +Map data serialization format version 17. + +0.3.1 does not use this format, but a more recent one. This exists here for +historical reasons. + +Directory structure: +sectors/XXXXZZZZ or sectors2/XXX/ZZZ +XXXX, ZZZZ, XXX and ZZZ being the hexadecimal X and Z coordinates. +Under these, the block files are stored, called YYYY. + +There also exists files map_meta.txt and chunk_meta, that are used by the +generator. If they are not found or invalid, the generator will currently +behave quite strangely. + +The MapBlock file format (sectors2/XXX/ZZZ/YYYY): +------------------------------------------------- + +NOTE: Byte order is MSB first. + +u8 version +- map format version number, this one is version 17 + +u8 flags +- Flag bitmasks: + - 0x01: is_underground: Should be set to 0 if there will be no light + obstructions above the block. If/when sunlight of a block is updated and + there is no block above it, this value is checked for determining whether + sunlight comes from the top. + - 0x02: day_night_differs: Whether the lighting of the block is different on + day and night. Only blocks that have this bit set are updated when day + transforms to night. + - 0x04: lighting_expired: If true, lighting is invalid and should be updated. + If you can't calculate lighting in your generator properly, you could try + setting this 1 to everything and setting the uppermost block in every + sector as is_underground=0. I am quite sure it doesn't work properly, + though. + +zlib-compressed map data: +- content: + u8[4096]: content types + u8[4096]: param1 values + u8[4096]: param2 values + +zlib-compressed node metadata +- content: + u16 version (=1) + u16 count of metadata + foreach count: + u16 position (= p.Z*MAP_BLOCKSIZE*MAP_BLOCKSIZE + p.Y*MAP_BLOCKSIZE + p.X) + u16 type_id + u16 content_size + u8[content_size] misc. stuff contained in the metadata + +u16 mapblockobject_count +- always write as 0. +- if read != 0, just fail. + +foreach mapblockobject_count: + - deprecated, should not be used. Length of this data can only be known by + properly parsing it. Just hope not to run into any of this. + +u8 static object version: +- currently 0 + +u16 static_object_count + +foreach static_object_count: + u8 type (object type-id) + s32 pos_x * 1000 + s32 pos_y * 1000 + s32 pos_z * 1000 + u16 data_size + u8[data_size] data + +u32 timestamp +- Timestamp when last saved, as seconds from starting the game. +- 0xffffffff = invalid/unknown timestamp, nothing will be done with the time + difference when loaded (recommended) + +Node metadata format: +--------------------- + +Sign metadata: + u16 string_len + u8[string_len] string + +Furnace metadata: + TBD + +Chest metadata: + TBD + +Locking Chest metadata: + u16 string_len + u8[string_len] string + TBD + +// END + |