dendrite/roomserver
Neil Alexander fc1d8e479b
Ensure that all state event IDs are included in the added section when rewriting state (#2725)
This should hopefully fix an entire class of problems where components
downstream from the roomserver (i.e. the sync API) could just lose a
whole bunch of state after a rewrite operation like a federated join.

The root of the bug is that we set `RewritesState` in the output event
which instructs downstream components to purge their copy of any room
state, but then didn't send the entire state snapshot in
`adds_state_event_ids` so the downstream state ends up being incomplete
as a result.
2022-09-16 10:35:32 +01:00
..
acls Pass pointers to events — reloaded (#1583) 2020-11-16 15:44:53 +00:00
api Tweak LoadMembershipAtEvent behaviour when state not known (#2716) 2022-09-13 12:52:09 +01:00
auth Pass pointers to events — reloaded (#1583) 2020-11-16 15:44:53 +00:00
internal Ensure that all state event IDs are included in the added section when rewriting state (#2725) 2022-09-16 10:35:32 +01:00
inthttp Restore the room version cache in the roomserver internal API HTTP client 2022-08-11 17:34:09 +01:00
producers Roomserver producers package (#2546) 2022-07-01 10:54:07 +01:00
state Tweak LoadMembershipAtEvent behaviour when state not known (#2716) 2022-09-13 12:52:09 +01:00
storage Check unique constraint errors when manually inserting migrations (#2712) 2022-09-13 08:07:43 +02:00
types Various roominfo tweaks (#2607) 2022-08-02 12:27:15 +01:00
version Set default room version to 9 2022-08-11 16:31:44 +01:00
README.md use go module for dependencies (#594) 2019-05-21 21:56:55 +01:00
roomserver_test.go Fix QuerySharedUsers for the SyncAPI keychange consumer (#2554) 2022-07-05 14:50:56 +02:00
roomserver.go Fix QuerySharedUsers for the SyncAPI keychange consumer (#2554) 2022-07-05 14:50:56 +02:00

RoomServer

RoomServer Internals

Numeric IDs

To save space matrix string identifiers are mapped to local numeric IDs. The numeric IDs are more efficient to manipulate and use less space to store. The numeric IDs are never exposed in the API the room server exposes. The numeric IDs are converted to string IDs before they leave the room server. The numeric ID for a string ID is never 0 to avoid being confused with go's default zero value. Zero is used to indicate that there was no corresponding string ID. Well-known event types and event state keys are preassigned numeric IDs.

State Snapshot Storage

The room server stores the state of the matrix room at each event. For efficiency the state is stored as blocks of 3-tuples of numeric IDs for the event type, event state key and event ID. For further efficiency the state snapshots are stored as the combination of up to 64 these blocks. This allows blocks of the room state to be reused in multiple snapshots.

The resulting database tables look something like this:

+-------------------------------------------------------------------+
| Events                                                            |
+---------+-------------------+------------------+------------------+
| EventNID| EventTypeNID      | EventStateKeyNID | StateSnapshotNID |
+---------+-------------------+------------------+------------------+
|       1 | m.room.create   1 | ""             1 | <nil>          0 |
|       2 | m.room.member   2 | "@user:foo"    2 | <nil>          0 |
|       3 | m.room.member   2 | "@user:bar"    3 | {1,2}          1 |
|       4 | m.room.message  3 | <nil>          0 | {1,2,3}        2 |
|       5 | m.room.member   2 | "@user:foo"    2 | {1,2,3}        2 |
|       6 | m.room.message  3 | <nil>          0 | {1,3,6}        3 |
+---------+-------------------+------------------+------------------+

+----------------------------------------+
| State Snapshots                        |
+-----------------------+----------------+
| EventStateSnapshotNID | StateBlockNIDs |
+-----------------------+----------------|
|                     1 |           {1}  |
|                     2 |         {1,2}  |
|                     3 |       {1,2,3}  |
+-----------------------+----------------+

+-----------------------------------------------------------------+
| State Blocks                                                    |
+---------------+-------------------+------------------+----------+
| StateBlockNID | EventTypeNID      | EventStateKeyNID | EventNID |
+---------------+-------------------+------------------+----------+
|             1 | m.room.create   1 | ""             1 |        1 |
|             1 | m.room.member   2 | "@user:foo"    2 |        2 |
|             2 | m.room.member   2 | "@user:bar"    3 |        3 |
|             3 | m.room.member   2 | "@user:foo"    2 |        6 |
+---------------+-------------------+------------------+----------+