Commit Graph

2915 Commits

Author SHA1 Message Date
Till
3c1474f68f
Fix /get_missing_events for rooms with joined/invited history_visibility (#2787)
Sytest was using a wrong `history_visibility` for `invited`
(https://github.com/matrix-org/sytest/pull/1303), so `invited` was
passing for the wrong reason (-> defaulted to `shared`, as `invite`
wasn't understood).
This change now handles missing events like Synapse, if a server isn't
allowed to see the event, it gets a redacted version of it, making the
`get_missing_events` tests pass.
2022-10-11 16:04:02 +02:00
Neil Alexander
0a9aebdf01
Private read receipts (#2789)
Implement behaviours for `m.read.private` receipts.
2022-10-11 12:27:21 +01:00
Neil Alexander
3920b9f9b6
Tweak GetStateDeltas behaviour (#2788)
Improves the control flow of `GetStateDeltas` for clarity and possibly
also fixes a bug where duplicate state delta entries could be inserted
with different memberships instead of being correctly overridden by
`join`.
2022-10-11 10:58:34 +01:00
Neil Alexander
9ed8ff6b93
Tweak federation M_NOT_FOUND errors 2022-10-11 10:48:36 +01:00
Neil Alexander
6bf1912525
Fix joined hosts with RewritesState (#2785)
This ensures that the joined hosts in the federation API are correct
after the state is rewritten. This might fix some races around the time
of joining federated rooms.
2022-10-10 16:54:04 +01:00
Till
b000db81ca
Send E2EE related errors to sentry (#2784)
Only sends errors if we're not retrying them in NATS.
Not sure if those should be scoped/tagged with something like "E2EE".
2022-10-10 17:36:26 +02:00
Till Faelligen
39581af3ba
CI update 2022-10-10 15:49:56 +02:00
Till Faelligen
dcc0116287
SyTest List Maintenance 2022-10-10 15:38:00 +02:00
Till
0f09e9d196
Move /event to the SyncAPI (#2782)
This allows us to apply history visibility without having to recalculate
it in the roomserver.
Unblocks https://github.com/matrix-org/complement/pull/495, fix missing
part of https://github.com/matrix-org/dendrite/issues/617
2022-10-10 12:19:16 +02:00
Neil Alexander
fb6cb2dbcb
Tweak GetAccountByPassword more 2022-10-10 11:14:16 +01:00
Neil Alexander
80a0ab6246
Further tweak to the issue template 2022-10-10 11:09:40 +01:00
Neil Alexander
b32b6d6e8e
Update issue and pull request templates 2022-10-10 11:03:52 +01:00
Neil Alexander
04bab14290
Add regression test for 980fa55846 2022-10-10 10:45:15 +01:00
Neil Alexander
980fa55846
Stronger passwordless account checks (fixes #2780) 2022-10-10 10:39:29 +01:00
Neil Alexander
f1b8df0f49
Version 0.10.2 (#2778)
Changelog and version bump.
2022-10-07 15:55:29 +01:00
Till
1ca3f3efb5
Fix issue with DMs shown as normal rooms (#2776)
Fixes #2121, test added in
https://github.com/matrix-org/complement/pull/494
2022-10-07 16:00:12 +02:00
Neil Alexander
8e231130e9
Revert "tDatabase transaction tweaks in roomserver"
This reverts commit 8d8f4689a0.
2022-10-07 14:05:06 +01:00
Neil Alexander
1b5460a920
Ensure we only wake up a given user once (#2775)
This ensures that the sync API notifier only wakes up a given user once
for a given stream position.
2022-10-07 13:42:35 +01:00
Neil Alexander
8d8f4689a0
tDatabase transaction tweaks in roomserver 2022-10-07 12:21:55 +01:00
Till
b9d0e9f7ed
Add test for QueryDeviceMessages (#2773)
Adds tests for `QueryDeviceMessages` and also includes some
optimizations to reduce allocations in the DB layer.
2022-10-07 10:54:42 +02:00
Till Faelligen
453b50e1d3
Update README 2022-10-07 07:32:27 +02:00
Till
8c5b166784
Use the stream positions of the notifier (#2768)
Use the stream positions of the notifier, which might have advanced
since setting it at the beginning of the loop. This possibly helps in
reducing roundtrips to the SyncAPI, just because we didn't fetch the
latest data.
Also fixes a minor oversight in the receipts stream.
2022-10-06 11:57:13 +01:00
Neil Alexander
d605d928bc
Allow specifying old signing keys with the public key and key ID only (#2770)
If the private key file is lost, it's often possible to retrieve the
public key from another server elsewhere, so we should make it possible
to configure it in that way.
2022-10-06 11:56:00 +01:00
Till
ec5d1d681d
Always return one_time_key_counts on /keys/upload (#2769)
The OTK count is
[required](https://spec.matrix.org/v1.4/client-server-api/#post_matrixclientv3keysupload)
in responses to `/keys/upload`, so return those.
2022-10-06 12:30:24 +02:00
Neil Alexander
3f82bceb70
Don't try to talk to ourselves when finding missing events 2022-10-06 10:51:06 +01:00
Neil Alexander
e53dcb25a9
Tweak logging for federated room joins 2022-10-06 10:07:13 +01:00
Neil Alexander
9ba3103f88
Document database connection limits 2022-10-05 14:54:19 +01:00
Till
0f777d421c
Remove empty fields from /sync response (#2755)
First attempt at removing empty fields from `/sync` responses. Needs
https://github.com/matrix-org/sytest/pull/1298 to keep Sytest happy.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-10-05 13:47:13 +01:00
Neil Alexander
c85bc3434f
Optimise QuerySharedUsers so that we can only work on local users (#2766)
Otherwise the sync API key change consumer wastes a lot of time trying
to wake up the notifiers for non-local users.
2022-10-05 12:47:53 +01:00
Neil Alexander
6f602bb096
Demote Failed to query device keys for some users warning to level=debug
Many of these warnings are due to dead servers and are quite annoying when they fill up the logs.
2022-10-05 11:16:05 +01:00
Till
8c0c3441d8
Add RoomEventType nats.Header to avoid unneeded unmarshalling (#2765) 2022-10-05 12:12:42 +02:00
Neil Alexander
ebd137cf6b
Check PostgreSQL connection count (#2760)
This PR queries PostgreSQL for the `max_connections` and
`superuser_reserved_connections` settings and then ensures that
Dendrite's `max_open_conns` doesn't exceed the allowed value.

This is a really common source of configuration problems and can either
result in blocking queries or deadlocks, so it seems reasonable that we
complain as loudly as possible when it happens.
2022-10-05 11:07:17 +01:00
Till
e070352293
Side effect import bleve analyzer languages (#2763)
... to actually allow different languages. Fixes #2761 
Binary size increases by ~1MB.
2022-10-05 11:14:33 +02:00
Neil Alexander
21f8881985
Add indexes that optimise selectStateInRangeSQL (#2764)
This gets rid of some expensive scans on `add_state_ids` and
`remove_state_ids`, turning them into much cheaper and faster index
scans instead.
2022-10-04 16:43:10 +01:00
Neil Alexander
ae10aac456
Don't perform a federated join after invite if we are already joined to the room (#2762)
If we are already joined to the room then it shouldn't matter if you
were invited or not, so this looks like a bug.
2022-10-04 15:40:04 +01:00
Neil Alexander
3da182212e
Track reasons why the process is in a degraded state 2022-10-04 13:02:41 +01:00
Neil Alexander
a767102f8a
Reduce max_open_conns in monolith sample config 2022-10-04 11:34:52 +01:00
Neil Alexander
085bf5e28b
Revert Docker changes 2022-10-04 11:33:05 +01:00
Neil Alexander
98b73652e0
Try to populate -ldflags in Docker builds 2022-10-04 11:07:54 +01:00
Neil Alexander
ede4632835
Fix Docker GHA 2022-10-04 10:43:58 +01:00
Till
e6c992ba8b
Update Dockerfile (#2342)
Updates/adds a new multistage (build-kit) Dockerfile. (if accepted,
could make `Dockerfile.monolith` and `Dockerfile.polylith` in
`build/docker` obsolete)
There's no huge difference between the dockerfiles, except this uses a
non-root user when running the container, also doesn't copy the working
directory to the image when building.
Also adds vulnerabilities scans using
[Trivy](https://github.com/aquasecurity/trivy) for the created docker
images. (untested)

Building images is done using 
```
docker build . --target image-monolith -t dendrite-monolith
docker build . --target image-polylith -t dendrite-polylith
```

As noted in the comments, only adds `dendrite-polylith-multi` to the
polylith image and all required binaries to the monolith image.
Probably needs some docs updating, if this is accepted.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-10-04 10:41:06 +01:00
Neil Alexander
34ed316584
Fix docs 2022-10-03 19:35:26 +01:00
Neil Alexander
ba66b5a3b9
Allow multiple static peers in Pinecone iOS/Android demos 2022-10-03 14:43:38 +01:00
Neil Alexander
34451d21b8
P2P demo tweaks 2022-10-03 14:35:10 +01:00
Neil Alexander
50fa50a343
Update P2P base directories 2022-10-03 13:05:58 +01:00
Neil Alexander
fec3ee384b
Stop CPU burn in PerformMarkAsStaleIfNeeded 2022-10-03 12:59:56 +01:00
Ashley Nelson
c1e16fd41e
Fix fragility of selectEventsWithEventIDsSQL queries (#2757)
This fixes a temporary workaround with the `selectEventsWithEventIDsSQL`
queries where fields need to be artificially added to the queries so the
row results match the format of the `syncapi_output_room_events` table.
I made similar functions that accept row results from the
`syncapi_current_room_state` table and convert them into StreamEvents
without the fields that are specific to output room events.

There is also a unit test in the first commit to ensure the resulting
behavior doesn't change from the modified queries and functions.

Fixes #601.

### Pull Request Checklist

<!-- Please read docs/CONTRIBUTING.md before submitting your pull
request -->

* [x] I have added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x] Pull request includes a [sign
off](https://github.com/matrix-org/dendrite/blob/main/docs/CONTRIBUTING.md#sign-off)

Signed-off-by: `Ashley Nelson <fant@shley.email>`

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-10-03 11:57:21 +01:00
Neil Alexander
b7f4bab358
Hopefully fix P2P --config error (re. #2756) 2022-10-03 11:38:31 +01:00
Neil Alexander
d32f60249d
Modify sync transaction behaviour (#2758)
This now uses a transaction per stream, so that errors in one stream
don't propagate to another, and we therefore no longer need to do hacks
to reopen a new transaction after aborting a failed one.
2022-10-03 11:38:20 +01:00
Till Faelligen
d4710217f8
Use non-HTTPS as default URL, as most people will be running behind a
reverse proxy
2022-10-02 11:31:40 +02:00