summaryrefslogtreecommitdiff
path: root/doc/mapformat.txt
blob: 2b049adcdec43647fd0dd08ae26d6689d02c1acb (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
=============================
Minetest World Format 22...25
=============================

This applies to a world format carrying the block serialization version
22...25, used at least in
- 0.4.dev-20120322 ... 0.4.dev-20120606 (22...23)
- 0.4.0 (23)
- 24 was never released as stable and existed for ~2 days

The block serialization version does not fully specify every aspect of this
format; if compliance with this format is to be checked, it needs to be
done by detecting if the files and data indeed follows it.

Legacy stuff
=============
Data can, in theory, be contained in the flat file directory structure
described below in Version 17, but it is not officially supported. Also you
may stumble upon all kinds of oddities in not-so-recent formats.

Files
======
Everything is contained in a directory, the name of which is freeform, but
often serves as the name of the world.

Currently the authentication and ban data is stored on a per-world basis.
It can be copied over from an old world to a newly created world.

World
|-- auth.txt ----- Authentication data
|-- env_meta.txt - Environment metadata
|-- ipban.txt ---- Banned ips/users
|-- map_meta.txt - Map metadata
|-- map.sqlite --- Map data
|-- players ------ Player directory
|   |-- player1 -- Player file
|   '-- Foo ------ Player file
`-- world.mt ----- World metadata

auth.txt
---------
Contains authentication data, player per line.
  <name>:<password hash>:<privilege1,...>
Format of password hash is <name><password> SHA1'd, in the base64 encoding.

Example lines:
- Player "celeron55", no password, privileges "interact" and "shout":
    celeron55::interact,shout
- Player "Foo", password "bar", privilege "shout":
    foo:iEPX+SQWIR3p67lj/0zigSWTKHg:shout
- Player "bar", no password, no privileges:
    bar::

env_meta.txt
-------------
Simple global environment variables.
Example content (added indentation):
  game_time = 73471
  time_of_day = 19118
  EnvArgsEnd

ipban.txt
----------
Banned IP addresses and usernames.
Example content (added indentation):
  123.456.78.9|foo
  123.456.78.10|bar

map_meta.txt
-------------
Simple global map variables.
Example content (added indentation):
  seed = 7980462765762429666
  [end_of_params]

map.sqlite
-----------
Map data.
See Map File Format below.

player1, Foo
-------------
Player data.
Filename can be anything.
See Player File Format below.

world.mt
---------
World metadata.
Example content (added indentation):
  gameid = mesetint

Player File Format
===================

- Should be pretty self-explanatory.
- Note: position is in nodes * 10

Example content (added indentation):
  hp = 11
  name = celeron55
  pitch = 39.77
  position = (-5231.97,15,1961.41)
  version = 1
  yaw = 101.37
  PlayerArgsEnd
  List main 32
  Item default:torch 13
  Item default:pick_steel 1 50112
  Item experimental:tnt
  Item default:cobble 99
  Item default:pick_stone 1 13104
  Item default:shovel_steel 1 51838
  Item default:dirt 61
  Item default:rail 78
  Item default:coal_lump 3
  Item default:cobble 99
  Item default:leaves 22
  Item default:gravel 52
  Item default:axe_steel 1 2045
  Item default:cobble 98
  Item default:sand 61
  Item default:water_source 94
  Item default:glass 2
  Item default:mossycobble
  Item default:pick_steel 1 64428
  Item animalmaterials:bone
  Item default:sword_steel
  Item default:sapling
  Item default:sword_stone 1 10647
  Item default:dirt 99
  Empty
  Empty
  Empty
  Empty
  Empty
  Empty
  Empty
  Empty
  EndInventoryList
  List craft 9
  Empty
  Empty
  Empty
  Empty
  Empty
  Empty
  Empty
  Empty
  Empty
  EndInventoryList
  List craftpreview 1
  Empty
  EndInventoryList
  List craftresult 1
  Empty
  EndInventoryList
  EndInventory

Map File Format
================

Minetest maps consist of MapBlocks, chunks of 16x16x16 nodes.

In addition to the bulk node data, MapBlocks stored on disk also contain
other things.

History
--------
We need a bit of history in here. Initially Minetest stored maps in a
format called the "sectors" format. It was a directory/file structure like
this:
  sectors2/XXX/ZZZ/YYYY
For example, the MapBlock at (0,1,-2) was this file:
  sectors2/000/ffd/0001

Eventually Minetest outgrow this directory structure, as filesystems were
struggling under the amount of files and directories.

Large servers seriously needed a new format, and thus the base of the
current format was invented, suggested by celeron55 and implemented by
JacobF.

SQLite3 was slammed in, and blocks files were directly inserted as blobs
in a single table, indexed by integer primary keys, oddly mangled from
coordinates.

Today we know that SQLite3 allows multiple primary keys (which would allow
storing coordinates separately), but the format has been kept unchanged for
that part. So, this is where it has come.
</history>

So here goes
-------------
map.sqlite is an sqlite3 database, containing a single table, called
"blocks". It looks like this:

  CREATE TABLE `blocks` (`pos` INT NOT NULL PRIMARY KEY,`data` BLOB);

The key
--------
"pos" is created from the three coordinates of a MapBlock using this
algorithm, defined here in Python:

  def getBlockAsInteger(p):
      return int64(p[2]*16777216 + p[1]*4096 + p[0])
  
  def int64(u):
      while u >= 2**63:
          u -= 2**64
      while u <= -2**63:
          u += 2**64
      return u
  
It can be converted the other way by using this code:

  def getIntegerAsBlock(i):
      x = unsignedToSigned(i % 4096, 2048)
      i = int((i - x) / 4096)
      y = unsignedToSigned(i % 4096, 2048)
      i = int((i - y) / 4096)
      z = unsignedToSigned(i % 4096, 2048)
      return x,y,z
  
  def unsignedToSigned(i, max_positive):
      if i < max_positive:
          return i
      else:
          return i - 2*max_positive

The blob
---------
The blob is the data that would have otherwise gone into the file.

See below for description.

MapBlock serialization format
==============================
NOTE: Byte order is MSB first (big-endian).
NOTE: Zlib data is in such a format that Python's zlib at least can
      directly decompress.

u8 version
- map format version number, this one is version 22

u8 flags
- Flag bitmasks:
  - 0x01: is_underground: Should be set to 0 if there will be no light
    obstructions above the block. If/when sunlight of a block is updated
    and there is no block above it, this value is checked for determining
    whether sunlight comes from the top.
  - 0x02: day_night_differs: Whether the lighting of the block is different
    on day and night. Only blocks that have this bit set are updated when
    day transforms to night.
  - 0x04: lighting_expired: If true, lighting is invalid and should be
    updated.  If you can't calculate lighting in your generator properly,
    you could try setting this 1 to everything and setting the uppermost
    block in every sector as is_underground=0. I am quite sure it doesn't
    work properly, though.
  - 0x08: generated: True if the block has been generated. If false, block
    is mostly filled with CONTENT_IGNORE and is likely to contain eg. parts
    of trees of neighboring blocks.

u8 content_width
- Number of bytes in the content (param0) fields of nodes
if map format version <= 23:
    - Always 1
if map format version >= 24:
    - Always 2

u8 params_width
- Number of bytes used for parameters per node
- Always 2

zlib-compressed node data:
if content_width == 1:
    - content:
      u8[4096]: param0 fields
      u8[4096]: param1 fields
      u8[4096]: param2 fields
if content_width == 2:
    - content:
      u16[4096]: param0 fields
      u8[4096]: param1 fields
      u8[4096]: param2 fields
- The location of a node in each of those arrays is (z*16*16 + y*16 + x).

zlib-compressed node metadata list
- content:
  u16 version (=1)
  u16 count of metadata
  foreach count:
    u16 position (p.Z*MAP_BLOCKSIZE*MAP_BLOCKSIZE + p.Y*MAP_BLOCKSIZE + p.X)
    u16 type_id
    u16 content_size
    u8[content_size] (content of metadata)

- Node timers
if map format version == 23:
  u8 unused version (always 0)
if map format version == 24: (NOTE: Not released as stable)
  u8 nodetimer_version
  if nodetimer_version == 0:
    (nothing else)
  if nodetimer_version == 1:
    u16 num_of_timers
    foreach num_of_timers:
      u16 timer position (z*16*16 + y*16 + x)
      s32 timeout*1000
      s32 elapsed*1000

u8 static object version:
- Always 0

u16 static_object_count

foreach static_object_count:
  u8 type (object type-id)
  s32 pos_x_nodes * 10000
  s32 pos_y_nodes * 10000
  s32 pos_z_nodes * 10000
  u16 data_size
  u8[data_size] data

u32 timestamp
- Timestamp when last saved, as seconds from starting the game.
- 0xffffffff = invalid/unknown timestamp, nothing should be done with the time
               difference when loaded

u8 name-id-mapping version
- Always 0

u16 num_name_id_mappings

foreach num_name_id_mappings
  u16 id
  u16 name_len
  u8[name_len] name

- Node timers
if map format version == 25:
  u8 length of the data of a single timer (always 2+4+4=10)
  u16 num_of_timers
  foreach num_of_timers:
    u16 timer position (z*16*16 + y*16 + x)
    s32 timeout*1000
    s32 elapsed*1000

EOF.

Format of nodes
----------------
A node is composed of the u8 fields param0, param1 and param2.

if map format version <= 23:
    The content id of a node is determined as so:
    - If param0 < 0x80,
        content_id = param0
    - Otherwise
        content_id = (param0<<4) + (param2>>4)
if map format version >= 24:
    The content id of a node is param0.

The purpose of param1 and param2 depend on the definition of the node.

The name-id-mapping
--------------------
The mapping maps node content ids to node names.

Node metadata format
---------------------

1: Generic metadata
  serialized inventory
  u32 len
  u8[len] text
  u16 len
  u8[len] owner
  u16 len
  u8[len] infotext
  u16 len
  u8[len] inventory drawspec
  u8 allow_text_input (bool)
  u8 removal_disabled (bool)
  u8 enforce_owner (bool)
  u32 num_vars
  foreach num_vars
    u16 len
    u8[len] name
    u32 len
    u8[len] value

14: Sign metadata
  u16 text_len
  u8[text_len] text

15: Chest metadata
  serialized inventory

16: Furnace metadata
  TBD

17: Locked Chest metadata
  u16 len
  u8[len] owner
  serialized inventory

Static objects
---------------
Static objects are persistent freely moving objects in the world.

Object types:
1: Test object
2: Item
3: Rat (deprecated)
4: Oerkki (deprecated)
5: Firefly (deprecated)
6: MobV2 (deprecated)
7: LuaEntity

1: Item:
  u8 version
  version 0:
    u16 len
    u8[len] itemstring

7: LuaEntity:
  u8 version
  version 1:
    u16 len
    u8[len] entity name
    u32 len
    u8[len] static data
    s16 hp
    s32 velocity.x * 10000
    s32 velocity.y * 10000
    s32 velocity.z * 10000
    s32 yaw * 1000

Itemstring format
------------------
eg. 'default:dirt 5'
eg. 'default:pick_wood 21323'
eg. '"default:apple" 2'
eg. 'default:apple'
- The wear value in tools is 0...65535
- There are also a number of older formats that you might stumble upon:
eg. 'node "default:dirt" 5'
eg. 'NodeItem default:dirt 5'
eg. 'ToolItem WPick 21323'

Inventory serialization format
-------------------------------
- The inventory serialization format is line-based
- The newline character used is "\n"
- The end condition of a serialized inventory is always "EndInventory\n"
- All the slots in a list must always be serialized.

Example (format does not include "---"):
---
List foo 4
Item default:sapling
Item default:sword_stone 1 10647
Item default:dirt 99
Empty
EndInventoryList
List bar 9
Empty
Empty
Empty
Empty
Empty
Empty
Empty
Empty
Empty
EndInventoryList
EndInventory
---

==============================================
Minetest World Format used as of 2011-05 or so
==============================================

Map data serialization format version 17.

0.3.1 does not use this format, but a more recent one. This exists here for
historical reasons.

Directory structure:
sectors/XXXXZZZZ or sectors2/XXX/ZZZ
XXXX, ZZZZ, XXX and ZZZ being the hexadecimal X and Z coordinates.
Under these, the block files are stored, called YYYY.

There also exists files map_meta.txt and chunk_meta, that are used by the
generator. If they are not found or invalid, the generator will currently
behave quite strangely.

The MapBlock file format (sectors2/XXX/ZZZ/YYYY):
-------------------------------------------------

NOTE: Byte order is MSB first.

u8 version
- map format version number, this one is version 17

u8 flags
- Flag bitmasks:
  - 0x01: is_underground: Should be set to 0 if there will be no light
    obstructions above the block. If/when sunlight of a block is updated and
	there is no block above it, this value is checked for determining whether
	sunlight comes from the top.
  - 0x02: day_night_differs: Whether the lighting of the block is different on
    day and night. Only blocks that have this bit set are updated when day
	transforms to night.
  - 0x04: lighting_expired: If true, lighting is invalid and should be updated.
    If you can't calculate lighting in your generator properly, you could try
	setting this 1 to everything and setting the uppermost block in every
	sector as is_underground=0. I am quite sure it doesn't work properly,
	though.

zlib-compressed map data:
- content:
  u8[4096]: content types
  u8[4096]: param1 values
  u8[4096]: param2 values

zlib-compressed node metadata
- content:
  u16 version (=1)
  u16 count of metadata
  foreach count:
    u16 position (= p.Z*MAP_BLOCKSIZE*MAP_BLOCKSIZE + p.Y*MAP_BLOCKSIZE + p.X)
	u16 type_id
	u16 content_size
	u8[content_size] misc. stuff contained in the metadata

u16 mapblockobject_count
- always write as 0.
- if read != 0, just fail.

foreach mapblockobject_count:
  - deprecated, should not be used. Length of this data can only be known by
    properly parsing it. Just hope not to run into any of this.

u8 static object version:
- currently 0

u16 static_object_count

foreach static_object_count:
  u8 type (object type-id)
  s32 pos_x * 1000
  s32 pos_y * 1000
  s32 pos_z * 1000
  u16 data_size
  u8[data_size] data

u32 timestamp
- Timestamp when last saved, as seconds from starting the game.
- 0xffffffff = invalid/unknown timestamp, nothing will be done with the time
               difference when loaded (recommended)

Node metadata format:
---------------------

Sign metadata:
  u16 string_len
  u8[string_len] string

Furnace metadata:
  TBD

Chest metadata:
  TBD

Locking Chest metadata:
  u16 string_len
  u8[string_len] string
  TBD

// END