aboutsummaryrefslogtreecommitdiff
path: root/old/ancient_main_comment.txt
blob: d7b0e307fc0299b9fc9e8a24e4266e7198876661 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
------------------------------------------------------------------
The ancient comment from the beginning of main.cpp is stored here.
------------------------------------------------------------------

/*
=============================== NOTES ==============================
NOTE: Things starting with TODO are sometimes only suggestions.

NOTE: iostream.imbue(std::locale("C")) is very slow
NOTE: Global locale is now set at initialization

NOTE: If VBO (EHM_STATIC) is used, remember to explicitly free the
      hardware buffer (it is not freed automatically)

NOTE: A random to-do list saved here as documentation:
A list of "active blocks" in which stuff happens. (+=done)
	+ Add a never-resetted game timer to the server
	+ Add a timestamp value to blocks
	+ The simple rule: All blocks near some player are "active"
	- Do stuff in real time in active blocks
		+ Handle objects
		- Grow grass, delete leaves without a tree
		- Spawn some mobs based on some rules
		- Transform cobble to mossy cobble near water
		- Run a custom script
		- ...And all kinds of other dynamic stuff
	+ Keep track of when a block becomes active and becomes inactive
	+ When a block goes inactive:
		+ Store objects statically to block
		+ Store timer value as the timestamp
	+ When a block goes active:
		+ Create active objects out of static objects
		- Simulate the results of what would have happened if it would have
		  been active for all the time
		  	- Grow a lot of grass and so on
	+ Initially it is fine to send information about every active object
	  to every player. Eventually it should be modified to only send info
	  about the nearest ones.
	  	+ This was left to be done by the old system and it sends only the
		  nearest ones.

NOTE: Seeds in 1260:6c77e7dbfd29:
5721858502589302589:
	Spawns you on a small sand island with a surface dungeon
2983455799928051958:
	Enormous jungle + a surface dungeon at ~(250,0,0)

Old, wild and random suggestions that probably won't be done:
-------------------------------------------------------------

SUGG: If player is on ground, mainly fetch ground-level blocks

SUGG: Expose Connection's seqnums and ACKs to server and client.
      - This enables saving many packets and making a faster connection
	  - This also enables server to check if client has received the
	    most recent block sent, for example.
SUGG: Add a sane bandwidth throttling system to Connection

SUGG: More fine-grained control of client's dumping of blocks from
      memory
	  - ...What does this mean in the first place?

SUGG: A map editing mode (similar to dedicated server mode)

SUGG: Transfer more blocks in a single packet
SUGG: A blockdata combiner class, to which blocks are added and at
      destruction it sends all the stuff in as few packets as possible.
SUGG: Make a PACKET_COMBINED which contains many subpackets. Utilize
      it by sending more stuff in a single packet.
	  - Add a packet queue to RemoteClient, from which packets will be
	    combined with object data packets
		- This is not exactly trivial: the object data packets are
		  sometimes very big by themselves
	  - This might not give much network performance gain though.

SUGG: Precalculate lighting translation table at runtime (at startup)
      - This is not doable because it is currently hand-made and not
	    based on some mathematical function.
		- Note: This has been changing lately

SUGG: A version number to blocks, which increments when the block is
      modified (node add/remove, water update, lighting update)
	  - This can then be used to make sure the most recent version of
	    a block has been sent to client, for example

SUGG: Make the amount of blocks sending to client and the total
	  amount of blocks dynamically limited. Transferring blocks is the
	  main network eater of this system, so it is the one that has
	  to be throttled so that RTTs stay low.

SUGG: Meshes of blocks could be split into 6 meshes facing into
      different directions and then only those drawn that need to be

SUGG: Background music based on cellular automata?
      http://www.earslap.com/projectslab/otomata

SUGG: Simple light color information to air

SUGG: Server-side objects could be moved based on nodes to enable very
      lightweight operation and simple AI
	- Not practical; client would still need to show smooth movement.

SUGG: Make a system for pregenerating quick information for mapblocks, so
	  that the client can show them as cubes before they are actually sent
	  or even generated.

SUGG: Erosion simulation at map generation time
    - This might be plausible if larger areas of map were pregenerated
	  without lighting (which is slow)
	- Simulate water flows, which would carve out dirt fast and
	  then turn stone into gravel and sand and relocate it.
	- How about relocating minerals, too? Coal and gold in
	  downstream sand and gravel would be kind of cool
	  - This would need a better way of handling minerals, mainly
		to have mineral content as a separate field. the first
		parameter field is free for this.
	- Simulate rock falling from cliffs when water has removed
	  enough solid rock from the bottom

SUGG: For non-mapgen FarMesh: Add a per-sector database to store surface
      stuff as simple flags/values
      - Light?
	  - A building?
	  And at some point make the server send this data to the client too,
	  instead of referring to the noise functions
	  - Ground height
	  - Surface ground type
	  - Trees?

Gaming ideas:
-------------

- Aim for something like controlling a single dwarf in Dwarf Fortress
- The player could go faster by a crafting a boat, or riding an animal
- Random NPC traders. what else?

Game content:
-------------

- When furnace is destroyed, move items to player's inventory
- Add lots of stuff
- Glass blocks
- Growing grass, decaying leaves
	- This can be done in the active blocks I guess.
	- Lots of stuff can be done in the active blocks.
	- Uh, is there an active block list somewhere? I think not. Add it.
- Breaking weak structures
	- This can probably be accomplished in the same way as grass
- Player health points
	- When player dies, throw items on map (needs better item-on-map
	  implementation)
- Cobble to get mossy if near water
- More slots in furnace source list, so that multiple ingredients
  are possible.
- Keys to chests?

- The Treasure Guard; a big monster with a hammer
	- The hammer does great damage, shakes the ground and removes a block
	- You can drop on top of it, and have some time to attack there
	  before he shakes you off

- Maybe the difficulty could come from monsters getting tougher in
  far-away places, and the player starting to need something from
  there when time goes by.
  - The player would have some of that stuff at the beginning, and
    would need new supplies of it when it runs out

- A bomb
- A spread-items-on-map routine for the bomb, and for dying players

- Fighting:
  - Proper sword swing simulation
  - Player should get damage from colliding to a wall at high speed

Documentation:
--------------

Build system / running:
-----------------------

Networking and serialization:
-----------------------------

SUGG: Fix address to be ipv6 compatible

User Interface:
---------------

Graphics:
---------

SUGG: Combine MapBlock's face caches to so big pieces that VBO
      can be used
      - That is >500 vertices
	  - This is not easy; all the MapBlocks close to the player would
	    still need to be drawn separately and combining the blocks
		would have to happen in a background thread

SUGG: Make fetching sector's blocks more efficient when rendering
      sectors that have very large amounts of blocks (on client)
	  - Is this necessary at all?

SUGG: Draw cubes in inventory directly with 3D drawing commands, so that
      animating them is easier.

SUGG: Option for enabling proper alpha channel for textures

TODO: Flowing water animation

TODO: A setting for enabling bilinear filtering for textures

TODO: Better control of draw_control.wanted_max_blocks

TODO: Further investigate the use of GPU lighting in addition to the
      current one

TODO: Artificial (night) light could be more yellow colored than sunlight.
      - This is technically doable.
	  - Also the actual colors of the textures could be made less colorful
	    in the dark but it's a bit more difficult.

SUGG: Somehow make the night less colorful

TODO: Occlusion culling
      - At the same time, move some of the renderMap() block choosing code
        to the same place as where the new culling happens.
      - Shoot some rays per frame and when ready, make a new list of
	    blocks for usage of renderMap and give it a new pointer to it.

Configuration:
--------------

Client:
-------

TODO: Untie client network operations from framerate
      - Needs some input queues or something
	  - This won't give much performance boost because calculating block
	    meshes takes so long

SUGG: Make morning and evening transition more smooth and maybe shorter

TODO: Don't update all meshes always on single node changes, but
      check which ones should be updated
	  - implement Map::updateNodeMeshes() and the usage of it
	  - It will give almost always a 4x boost in mesh update performance.

- A weapon engine

- Tool/weapon visualization

FIXME: When disconnected to the menu, memory is not freed properly

TODO: Investigate how much the mesh generator thread gets used when
      transferring map data

Server:
-------

SUGG: Make an option to the server to disable building and digging near
      the starting position

FIXME: Server sometimes goes into some infinite PeerNotFoundException loop

* Fix the problem with the server constantly saving one or a few
  blocks? List the first saved block, maybe it explains.
  - It is probably caused by oscillating water
  - TODO: Investigate if this still happens (this is a very old one)
* Make a small history check to transformLiquids to detect and log
  continuous oscillations, in such detail that they can be fixed.

FIXME: The new optimized map sending doesn't sometimes send enough blocks
       from big caves and such
FIXME: Block send distance configuration does not take effect for some reason

Environment:
------------

TODO: Add proper hooks to when adding and removing active blocks

TODO: Finish the ActiveBlockModifier stuff and use it for something

Objects:
--------

TODO: Get rid of MapBlockObjects and use only ActiveObjects
	- Skipping the MapBlockObject data is nasty - there is no "total
	  length" stored; have to make a SkipMBOs function which contains
	  enough of the current code to skip them properly.

SUGG: MovingObject::move and Player::move are basically the same.
      combine them.
	- NOTE: This is a bit tricky because player has the sneaking ability
	- NOTE: Player::move is more up-to-date.
	- NOTE: There is a simple move implementation now in collision.{h,cpp}
	- NOTE: MovingObject will be deleted (MapBlockObject)

TODO: Add a long step function to objects that is called with the time
      difference when block activates

Map:
----

TODO: Flowing water to actually contain flow direction information
      - There is a space for this - it just has to be implemented.

TODO: Consider smoothening cave floors after generating them

TODO: Fix make_tree, make_* to use seed-position-consistent pseudorandom
	  - delta also

Misc. stuff:
------------
TODO: Make sure server handles removing grass when a block is placed (etc)
      - The client should not do it by itself
	  - NOTE: I think nobody does it currently...
TODO: Block cube placement around player's head
TODO: Protocol version field
TODO: Think about using same bits for material for fences and doors, for
	  example

SUGG: Restart irrlicht completely when coming back to main menu from game.
	- This gets rid of everything that is stored in irrlicht's caches.
	- This might be needed for texture pack selection in menu

TODO: Merge bahamada's audio stuff (clean patch available)

Making it more portable:
------------------------
 
Stuff to do before release:
---------------------------

Fixes to the current release:
-----------------------------

Stuff to do after release:
---------------------------

Doing currently:
----------------

======================================================================

*/
we know that SQLite3 allows multiple primary keys (which would allow storing coordinates separately), but the format has been kept unchanged for that part. So, this is where it has come. </history> So here goes ------------- map.sqlite is an sqlite3 database, containg a single table, called "blocks". It looks like this: CREATE TABLE `blocks` (`pos` INT NOT NULL PRIMARY KEY,`data` BLOB); The key -------- "pos" is created from the three coordinates of a MapBlock using this algorithm, defined here in Python: def getBlockAsInteger(p): return int64(p[2]*16777216 + p[1]*4096 + p[0]) def int64(u): while u >= 2**63: u -= 2**64 while u <= -2**63: u += 2**64 return u It can be converted the other way by using this code: def getIntegerAsBlock(i): x = unsignedToSigned(i % 4096, 2048) i = int((i - x) / 4096) y = unsignedToSigned(i % 4096, 2048) i = int((i - y) / 4096) z = unsignedToSigned(i % 4096, 2048) return x,y,z def unsignedToSigned(i, max_positive): if i < max_positive: return i else: return i - 2*max_positive The blob --------- The blob is the data that would have otherwise gone into the file. See below for description. MapBlock serialization format ============================== NOTE: Byte order is MSB first (big-endian). NOTE: Zlib data is in such a format that Python's zlib at least can directly decompress. u8 version - map format version number, this one is version 22 u8 flags - Flag bitmasks: - 0x01: is_underground: Should be set to 0 if there will be no light obstructions above the block. If/when sunlight of a block is updated and there is no block above it, this value is checked for determining whether sunlight comes from the top. - 0x02: day_night_differs: Whether the lighting of the block is different on day and night. Only blocks that have this bit set are updated when day transforms to night. - 0x04: lighting_expired: If true, lighting is invalid and should be updated. If you can't calculate lighting in your generator properly, you could try setting this 1 to everything and setting the uppermost block in every sector as is_underground=0. I am quite sure it doesn't work properly, though. - 0x08: generated: True if the block has been generated. If false, block is mostly filled with CONTENT_IGNORE and is likely to contain eg. parts of trees of neighboring blocks. u8 content_width - Number of bytes in the content (param0) fields of nodes if map format version <= 23: - Always 1 if map format version >= 24: - Always 2 u8 params_width - Number of bytes used for parameters per node - Always 2 zlib-compressed node data: if content_width == 1: - content: u8[4096]: param0 fields u8[4096]: param1 fields u8[4096]: param2 fields if content_width == 2: - content: u16[4096]: param0 fields u8[4096]: param1 fields u8[4096]: param2 fields - The location of a node in each of those arrays is (z*16*16 + y*16 + x). zlib-compressed node metadata list - content: u16 version (=1) u16 count of metadata foreach count: u16 position (p.Z*MAP_BLOCKSIZE*MAP_BLOCKSIZE + p.Y*MAP_BLOCKSIZE + p.X) u16 type_id u16 content_size u8[content_size] (content of metadata) - Node timers if map format version == 23: u8 unused version (always 0) if map format version == 24: (NOTE: Not released as stable) u8 nodetimer_version if nodetimer_version == 0: (nothing else) if nodetimer_version == 1: u16 num_of_timers foreach num_of_timers: u16 timer position (z*16*16 + y*16 + x) s32 timeout*1000 s32 elapsed*1000 u8 static object version: - Always 0 u16 static_object_count foreach static_object_count: u8 type (object type-id) s32 pos_x_nodes * 10000 s32 pos_y_nodes * 10000 s32 pos_z_nodes * 10000 u16 data_size u8[data_size] data u32 timestamp - Timestamp when last saved, as seconds from starting the game. - 0xffffffff = invalid/unknown timestamp, nothing should be done with the time difference when loaded u8 name-id-mapping version - Always 0 u16 num_name_id_mappings foreach num_name_id_mappings u16 id u16 name_len u8[name_len] name - Node timers if map format version == 25: u8 length of the data of a single timer (always 2+4+4=10) u16 num_of_timers foreach num_of_timers: u16 timer position (z*16*16 + y*16 + x) s32 timeout*1000 s32 elapsed*1000 EOF. Format of nodes ---------------- A node is composed of the u8 fields param0, param1 and param2. if map format version <= 23: The content id of a node is determined as so: - If param0 < 0x80, content_id = param0 - Otherwise content_id = (param0<<4) + (param2>>4) if map format version >= 24: The content id of a node is param0. The purpose of param1 and param2 depend on the definition of the node. The name-id-mapping -------------------- The mapping maps node content ids to node names. Node metadata format --------------------- 1: Generic metadata serialized inventory u32 len u8[len] text u16 len u8[len] owner u16 len u8[len] infotext u16 len u8[len] inventory drawspec u8 allow_text_input (bool) u8 removal_disabled (bool) u8 enforce_owner (bool) u32 num_vars foreach num_vars u16 len u8[len] name u32 len u8[len] value 14: Sign metadata u16 text_len u8[text_len] text 15: Chest metadata serialized inventory 16: Furnace metadata TBD 17: Locked Chest metadata u16 len u8[len] owner serialized inventory Static objects --------------- Static objects are persistent freely moving objects in the world. Object types: 1: Test object 2: Item 3: Rat (deprecated) 4: Oerkki (deprecated) 5: Firefly (deprecated) 6: MobV2 (deprecated) 7: LuaEntity 1: Item: u8 version version 0: u16 len u8[len] itemstring 7: LuaEntity: u8 version version 1: u16 len u8[len] entity name u32 len u8[len] static data s16 hp s32 velocity.x * 10000 s32 velocity.y * 10000 s32 velocity.z * 10000 s32 yaw * 1000 Itemstring format ------------------ eg. 'default:dirt 5' eg. 'default:pick_wood 21323' eg. '"default:apple" 2' eg. 'default:apple' - The wear value in tools is 0...65535 - There are also a number of older formats that you might stumble upon: eg. 'node "default:dirt" 5' eg. 'NodeItem default:dirt 5' eg. 'ToolItem WPick 21323' Inventory serialization format ------------------------------- - The inventory serialization format is line-based - The newline character used is "\n" - The end condition of a serialized inventory is always "EndInventory\n" - All the slots in a list must always be serialized. Example (format does not include "---"): --- List foo 4 Item default:sapling Item default:sword_stone 1 10647 Item default:dirt 99 Empty EndInventoryList List bar 9 Empty Empty Empty Empty Empty Empty Empty Empty Empty EndInventoryList EndInventory --- ============================================== Minetest World Format used as of 2011-05 or so ============================================== Map data serialization format version 17. 0.3.1 does not use this format, but a more recent one. This exists here for historical reasons. Directory structure: sectors/XXXXZZZZ or sectors2/XXX/ZZZ XXXX, ZZZZ, XXX and ZZZ being the hexadecimal X and Z coordinates. Under these, the block files are stored, called YYYY. There also exists files map_meta.txt and chunk_meta, that are used by the generator. If they are not found or invalid, the generator will currently behave quite strangely. The MapBlock file format (sectors2/XXX/ZZZ/YYYY): ------------------------------------------------- NOTE: Byte order is MSB first. u8 version - map format version number, this one is version 17 u8 flags - Flag bitmasks: - 0x01: is_underground: Should be set to 0 if there will be no light obstructions above the block. If/when sunlight of a block is updated and there is no block above it, this value is checked for determining whether sunlight comes from the top. - 0x02: day_night_differs: Whether the lighting of the block is different on day and night. Only blocks that have this bit set are updated when day transforms to night. - 0x04: lighting_expired: If true, lighting is invalid and should be updated. If you can't calculate lighting in your generator properly, you could try setting this 1 to everything and setting the uppermost block in every sector as is_underground=0. I am quite sure it doesn't work properly, though. zlib-compressed map data: - content: u8[4096]: content types u8[4096]: param1 values u8[4096]: param2 values zlib-compressed node metadata - content: u16 version (=1) u16 count of metadata foreach count: u16 position (= p.Z*MAP_BLOCKSIZE*MAP_BLOCKSIZE + p.Y*MAP_BLOCKSIZE + p.X) u16 type_id u16 content_size u8[content_size] misc. stuff contained in the metadata u16 mapblockobject_count - always write as 0. - if read != 0, just fail. foreach mapblockobject_count: - deprecated, should not be used. Length of this data can only be known by properly parsing it. Just hope not to run into any of this. u8 static object version: - currently 0 u16 static_object_count foreach static_object_count: u8 type (object type-id) s32 pos_x * 1000 s32 pos_y * 1000 s32 pos_z * 1000 u16 data_size u8[data_size] data u32 timestamp - Timestamp when last saved, as seconds from starting the game. - 0xffffffff = invalid/unknown timestamp, nothing will be done with the time difference when loaded (recommended) Node metadata format: --------------------- Sign metadata: u16 string_len u8[string_len] string Furnace metadata: TBD Chest metadata: TBD Locking Chest metadata: u16 string_len u8[string_len] string TBD // END