dendrite/roomserver
Neil Alexander a572f4db03
Fix bugs that could wedge rooms (#2154)
* Don't flake so badly for rejected events

* Moar

* Fix panic

* Don't count rejected events as missing

* Don't treat rejected events without state as missing

* Revert "Don't count rejected events as missing"

This reverts commit 4b6139b62eb91ba059b47415b0275964b37d9b43.

* Missing events should be KindOld

* If we have state, use it, regardless of memberships which could be stale now

* Fetch missing state for KindOld too

* Tweak the condition again

* Clean up a bit

* Use room updater to get latest events in a race-free way

* Return the correct error

* Improve errors
2022-02-07 19:10:01 +00:00
..
acls Pass pointers to events — reloaded (#1583) 2020-11-16 15:44:53 +00:00
api Improve roomserver logging 2022-01-31 15:33:00 +00:00
auth Pass pointers to events — reloaded (#1583) 2020-11-16 15:44:53 +00:00
internal Fix bugs that could wedge rooms (#2154) 2022-02-07 19:10:01 +00:00
inthttp Fix keyring regressions in previous P2P demo 2021-12-13 13:24:49 +00:00
state Full roomserver input transactional isolation (#2141) 2022-02-04 10:39:34 +00:00
storage Full roomserver input transactional isolation (#2141) 2022-02-04 10:39:34 +00:00
types Fix bugs that could wedge rooms (#2154) 2022-02-07 19:10:01 +00:00
version Set default room version to v6 (#1438) 2020-09-25 12:59:57 +01:00
README.md use go module for dependencies (#594) 2019-05-21 21:56:55 +01:00
roomserver.go Remove sarama/saramajetstream dependencies (#2138) 2022-02-04 13:08:13 +00:00

RoomServer

RoomServer Internals

Numeric IDs

To save space matrix string identifiers are mapped to local numeric IDs. The numeric IDs are more efficient to manipulate and use less space to store. The numeric IDs are never exposed in the API the room server exposes. The numeric IDs are converted to string IDs before they leave the room server. The numeric ID for a string ID is never 0 to avoid being confused with go's default zero value. Zero is used to indicate that there was no corresponding string ID. Well-known event types and event state keys are preassigned numeric IDs.

State Snapshot Storage

The room server stores the state of the matrix room at each event. For efficiency the state is stored as blocks of 3-tuples of numeric IDs for the event type, event state key and event ID. For further efficiency the state snapshots are stored as the combination of up to 64 these blocks. This allows blocks of the room state to be reused in multiple snapshots.

The resulting database tables look something like this:

+-------------------------------------------------------------------+
| Events                                                            |
+---------+-------------------+------------------+------------------+
| EventNID| EventTypeNID      | EventStateKeyNID | StateSnapshotNID |
+---------+-------------------+------------------+------------------+
|       1 | m.room.create   1 | ""             1 | <nil>          0 |
|       2 | m.room.member   2 | "@user:foo"    2 | <nil>          0 |
|       3 | m.room.member   2 | "@user:bar"    3 | {1,2}          1 |
|       4 | m.room.message  3 | <nil>          0 | {1,2,3}        2 |
|       5 | m.room.member   2 | "@user:foo"    2 | {1,2,3}        2 |
|       6 | m.room.message  3 | <nil>          0 | {1,3,6}        3 |
+---------+-------------------+------------------+------------------+

+----------------------------------------+
| State Snapshots                        |
+-----------------------+----------------+
| EventStateSnapshotNID | StateBlockNIDs |
+-----------------------+----------------|
|                     1 |           {1}  |
|                     2 |         {1,2}  |
|                     3 |       {1,2,3}  |
+-----------------------+----------------+

+-----------------------------------------------------------------+
| State Blocks                                                    |
+---------------+-------------------+------------------+----------+
| StateBlockNID | EventTypeNID      | EventStateKeyNID | EventNID |
+---------------+-------------------+------------------+----------+
|             1 | m.room.create   1 | ""             1 |        1 |
|             1 | m.room.member   2 | "@user:foo"    2 |        2 |
|             2 | m.room.member   2 | "@user:bar"    3 |        3 |
|             3 | m.room.member   2 | "@user:foo"    2 |        6 |
+---------------+-------------------+------------------+----------+