Commit Graph

414 Commits

Author SHA1 Message Date
Till
caf310fd79
AWSY missing federation tests (#2943)
In an attempt to fix the missing AWSY tests and to get to 100%
server-server compliance.
2023-01-20 15:18:06 +01:00
Neil
738686ae68
Add /_dendrite/admin/purgeRoom/{roomID} (#2662)
This adds a new admin endpoint `/_dendrite/admin/purgeRoom/{roomID}`. It
completely erases all database entries for a given room ID.

The roomserver will start by clearing all data for that room and then
will generate an output event to notify downstream components (i.e. the
sync API and federation API) to do the same.

It does not currently clear media and it is currently not implemented
for SQLite since it relies on SQL array operations right now.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Co-authored-by: Till Faelligen <2353100+S7evinK@users.noreply.github.com>
2023-01-19 21:02:32 +01:00
Till
2e1fe58937
Fix backfilling (#2926)
This should fix https://github.com/matrix-org/dendrite/issues/2923
2023-01-05 09:24:00 +01:00
Till
5eed31fea3
Handle guest access [1/2?] (#2872)
Needs https://github.com/matrix-org/sytest/pull/1315, as otherwise the
membership events aren't persisted yet when hitting `/state` after
kicking guest users.

Makes the following tests pass:
```
Guest users denied access over federation if guest access prohibited
Guest users are kicked from guest_access rooms on revocation of guest_access
Guest users are kicked from guest_access rooms on revocation of guest_access over federation
```

Todo (in a follow up PR):
- Restrict access to CS API Endpoints as per
https://spec.matrix.org/v1.4/client-server-api/#client-behaviour-14

Co-authored-by: kegsay <kegan@matrix.org>
2022-12-22 13:05:59 +01:00
Till
7d2344049d
Cleanup stale device lists for users we don't share a room with anymore (#2857)
The stale device lists table might contain entries for users we don't
share a room with anymore. This now asks the roomserver about left users
and removes those entries from the table.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-12-12 08:20:59 +01:00
Till
e245a26f6b
Enable/Disable internal metrics (#2899)
Basically enables us to use `test.WithAllDatabases` when testing
internal HTTP APIs, as this would otherwise result in Prometheus
complaining about already registered metric names.
2022-12-05 13:53:36 +01:00
Neil Alexander
9b8bb55430
Don't get blacklisted hosts when querying joined servers (#2880)
Otherwise we just waste time/CPU.
2022-11-15 17:21:16 +00:00
Neil Alexander
6650712a1c
Federation fixes for virtual hosting 2022-11-15 15:05:23 +00:00
Till
2a77a910eb
Handle remote room upgrades (#2866)
Makes the following tests pass
```
/upgrade moves remote aliases to the new room
Local and remote users' homeservers remove a room from their public directory on upgrade
```
2022-11-14 12:07:13 +00:00
Till
1e79b0557e
Use a writer to assign state key NIDs (#2877) 2022-11-14 12:06:27 +00:00
Till Faelligen
e177e0ae73
Fix oops, add simple UT 2022-11-11 16:44:59 +01:00
Till
c648c671a3
Fix issue with missing user NIDs (#2874)
This should fix #2696 and possibly other related issues regarding
missing user NIDs.
(https://github.com/matrix-org/dendrite/issues/2094?)
2022-11-11 10:52:43 +01:00
Till
0193549201
Send presence to newly added servers (#2869)
This should make `New federated private chats get full presence
information (SYN-115)` happy.
2022-11-11 10:35:17 +01:00
Neil Alexander
16c2a95900
Improve logging for processEventWithMissingState 2022-11-02 11:30:49 +00:00
Neil Alexander
6663728eb1
Fix SQLite roomserver_published migration 2022-11-01 16:08:13 +00:00
Till
2acc1d65fb
Optimize history visibility checks (#2848)
This optimizes history visibility checks by (mostly) avoiding database
hits.
Possibly solves https://github.com/matrix-org/dendrite/issues/2777

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-11-01 15:07:17 +00:00
Neil Alexander
7bd6631935
Move code for calculating auth difference into GMSL 2022-11-01 10:12:11 +00:00
Neil Alexander
f10c6f26e5
Add /_dendrite/admin/downloadState/{serverName}/{roomID} admin endpoint 2022-10-31 09:13:28 +00:00
Till Faelligen
a785532463
Fix upgrade appservices 2022-10-27 16:01:51 +02:00
Till
444b4bbdb8
Add AS specific public room list endpoints (#2836)
Adds `PUT
/_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}` and
`DELTE
/_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}`
support, as well as the ability to filter `/publicRooms` on networkID
and including all networks.
2022-10-27 14:40:35 +02:00
Neil Alexander
f6dea712d2
Initial support for multiple server names (#2829)
This PR is the first step towards virtual hosting by laying the
groundwork for multiple server names being configured.
2022-10-26 12:59:19 +01:00
Neboer
2a4c7f45b3
Add support for config "auto_join_rooms" (#2823)
Add support for config "auto_join_rooms". Now new accounts can join the
rooms in config file automatically.

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have justified why this PR doesn't need tests.
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)

Signed-off-by: `Rubin Poster <rubinposter@gmail.com>`
2022-10-26 11:04:53 +02:00
Neil Alexander
cd8f7e1251
Set inactivity threshold on durable consumers in the roomserver input API (#2795)
This prevents us from holding onto durable consumers indefinitely for
rooms that have long since turned inactive, since they do have a bit of
a processing overhead in the NATS Server. If we clear up a consumer and
then a room becomes active again, the consumer gets recreated as needed.

The threshold is set to 24 hours for now, we can tweak it later if needs
be.
2022-10-14 15:14:29 +01:00
Till
088ad1dd21
Fix outliers whose auth_events are in a different room are correctly rejected (#2791)
Fixes `outliers whose auth_events are in a different room are correctly
rejected`, by validating that auth events are all from the same room and
not using rejected events for event auth.
2022-10-14 09:14:54 +02:00
Till
3c1474f68f
Fix /get_missing_events for rooms with joined/invited history_visibility (#2787)
Sytest was using a wrong `history_visibility` for `invited`
(https://github.com/matrix-org/sytest/pull/1303), so `invited` was
passing for the wrong reason (-> defaulted to `shared`, as `invite`
wasn't understood).
This change now handles missing events like Synapse, if a server isn't
allowed to see the event, it gets a redacted version of it, making the
`get_missing_events` tests pass.
2022-10-11 16:04:02 +02:00
Till
1ca3f3efb5
Fix issue with DMs shown as normal rooms (#2776)
Fixes #2121, test added in
https://github.com/matrix-org/complement/pull/494
2022-10-07 16:00:12 +02:00
Neil Alexander
8e231130e9
Revert "tDatabase transaction tweaks in roomserver"
This reverts commit 8d8f4689a0.
2022-10-07 14:05:06 +01:00
Neil Alexander
8d8f4689a0
tDatabase transaction tweaks in roomserver 2022-10-07 12:21:55 +01:00
Neil Alexander
3f82bceb70
Don't try to talk to ourselves when finding missing events 2022-10-06 10:51:06 +01:00
Neil Alexander
c85bc3434f
Optimise QuerySharedUsers so that we can only work on local users (#2766)
Otherwise the sync API key change consumer wastes a lot of time trying
to wake up the notifiers for non-local users.
2022-10-05 12:47:53 +01:00
Till
8c0c3441d8
Add RoomEventType nats.Header to avoid unneeded unmarshalling (#2765) 2022-10-05 12:12:42 +02:00
Neil Alexander
ae10aac456
Don't perform a federated join after invite if we are already joined to the room (#2762)
If we are already joined to the room then it shouldn't matter if you
were invited or not, so this looks like a bug.
2022-10-04 15:40:04 +01:00
Neil Alexander
f022fc1397
Remove origin field from PDUs (#2737)
This nukes the `origin` field from PDUs as per
matrix-org/matrix-spec#998, matrix-org/gomatrixserverlib#341.
2022-09-26 17:35:35 +01:00
Neil Alexander
fc1d8e479b
Ensure that all state event IDs are included in the added section when rewriting state (#2725)
This should hopefully fix an entire class of problems where components
downstream from the roomserver (i.e. the sync API) could just lose a
whole bunch of state after a rewrite operation like a federated join.

The root of the bug is that we set `RewritesState` in the output event
which instructs downstream components to purge their copy of any room
state, but then didn't send the entire state snapshot in
`adds_state_event_ids` so the downstream state ends up being incomplete
as a result.
2022-09-16 10:35:32 +01:00
Neil Alexander
7f89fed1e4
Revert 482914aef4 2022-09-14 09:55:50 +01:00
Neil Alexander
482914aef4
Use AckNone on the ephemeral room input consumer 2022-09-13 15:25:02 +01:00
Neil Alexander
b05e028f7d
Tweak LoadMembershipAtEvent behaviour when state not known (#2716)
Previously `LoadMembershipAtEvent` would fail if the state before one of
the events was not known, i.e. because it was an outlier. This modifies
it so that it gracefully handles not knowing the state and returns no
memberships instead, so that history visibility doesn't freak out and
kill `/sync` requests dead.
2022-09-13 12:52:09 +01:00
Neil Alexander
3e55856254
Always resolve state in QueryStateAfterEvents 2022-09-13 09:37:38 +01:00
Till
100fa9b235
Check unique constraint errors when manually inserting migrations (#2712)
This should avoid unnecessary logging on startup if the migration (were
we need `InsertMigration`) was already executed.
This now checks for "unique constraint errors" for SQLite and Postgres
and fails the startup process if the migration couldn't be manually
inserted for some other reason.
2022-09-13 08:07:43 +02:00
Neil Alexander
2792d0490f
Fix missing signature check on the /get_missing_events response 2022-09-12 13:30:51 +01:00
Neil Alexander
c0e17bbe1b
Fix transactions around assigning NIDs 2022-09-09 13:30:09 +01:00
Till
8196b29657
Change detection of already executed migrations (#2665)
This changes the detection of already executed migrations for the
roomserver state block and keychange refactor. It now uses schema tables
provided by the database engine to check if the column was already
removed. We now also store the migration in the migrations table.

This should stop e.g. Postgres from logging errors like `ERROR: column
"event_nid" does not exist at character 8`.
2022-09-09 13:14:52 +01:00
Neil Alexander
31f4ae8997
Use a single context instead 2022-09-07 16:24:43 +01:00
Neil Alexander
5014b35bd7
Update state reset capture to Sentry 2022-09-07 16:23:22 +01:00
Neil Alexander
cd22ba22b0
Improve Sentry reporting 2022-09-05 17:25:11 +01:00
Neil Alexander
ecee5f10f4
Tweak logging for detected state resets 2022-09-05 17:08:54 +01:00
Neil Alexander
d1f87e63f1
Move SetLatestEvents call 2022-09-05 13:16:14 +01:00
Neil Alexander
8ff3f1a7c9
Remove a couple unnecessary Sentry captures from backfill 2022-08-25 11:01:07 +01:00
Neil Alexander
cd7fa34595
Tweak logging and Sentry reporting for roomserver input 2022-08-25 10:57:27 +01:00
Neil Alexander
16156b0b09
Fix 500s on /state, /state_ids when state not known (#2672)
This was due to bad error bubbling.
2022-08-25 09:51:36 +01:00