Commit Graph

2890 Commits

Author SHA1 Message Date
Till
297479ea49
Use pointer when passing the connection manager around (#3152)
As otherwise existing connections aren't reused.
2023-07-19 13:37:04 +02:00
devonh
a01faee17c
Extend context timeout on send_join to allow for joining complex rooms (#3153)
Background federated joins are currently broken since they timeout after
30s. This timeout didn't exist before the refactor. It should still exist but it needs to be extended to allow for the additional time it can take a server to generate the /send_join response when joining a complex room.
2023-07-18 18:48:05 +00:00
Till Faelligen
33ff309572
Don't HTTP500 if a profile does't exist 2023-07-14 14:24:31 +02:00
Till Faelligen
6011ddc0a8
Discard "illegal base64 data at input byte 0" errors in the SyncAPI 2023-07-14 08:28:30 +02:00
Till Faelligen
3e314e028e
Avoid panic due to being unable to query the userID 2023-07-14 08:04:25 +02:00
Till
5267cc0f54
Optimise getting local members and membership counts (#3150)
The previous version was getting **ALL** membership events (as
`ClientEvents`, so going through `NewEventFromTrustedJSONWithID`) for a
given room.
Now we are querying only locally joined users as `ClientEvents`, which
should **significantly** reduce allocations.

Take for example a large room with 2k membership events, but only 1
local user - avoiding 1999 `NewEventFromTrustedJSONWithID` calls just to
calculate the `roomSize` which we can also query by other means.

This is also getting called for every `OutputRoomEvent` in the userAPI.

Benchmark with 1 local user and 100 remote users.
```
pkg: github.com/matrix-org/dendrite/userapi/consumers
cpu: 12th Gen Intel(R) Core(TM) i5-12500H
                    │   old.txt   │               new.txt               │
                    │   sec/op    │   sec/op     vs base                │
LocalRoomMembers-16   375.9µ ± 7%   327.6µ ± 6%  -12.85% (p=0.000 n=10)

                    │    old.txt    │               new.txt                │
                    │     B/op      │     B/op      vs base                │
LocalRoomMembers-16   79.426Ki ± 0%   8.507Ki ± 0%  -89.29% (p=0.000 n=10)

                    │   old.txt   │              new.txt               │
                    │  allocs/op  │ allocs/op   vs base                │
LocalRoomMembers-16   1015.0 ± 0%   277.0 ± 0%  -72.71% (p=0.000 n=10)
```
2023-07-13 14:19:08 +02:00
Till
f12982472c
Tweaks around /messages (#3149)
Try to mitigate some issues with `/messages`
2023-07-13 14:18:37 +02:00
Till Faelligen
0df982a2e5
Update NATS again [skip ci] 2023-07-13 14:17:48 +02:00
Till
99f94fc735
Add revision to version string (#3147)
Since the removal of `build.sh`, we don't include any information about
the revision Dendrite was build from. Since go1.18, the revision a
binary was build from is automatically included, so we can try to get
that instead.

This also adds a `dendrite_up` metric showing the current version
(`dendrite_up{version="0.13.1+c796f20"} 1`)

Closes #2993
2023-07-11 13:56:25 +02:00
Till
69b2069dea
Avoid loops by setting end to an empty string if start == end (#3146) 2023-07-08 11:45:44 +02:00
Till Faelligen
b965a08faa
Unknown issue 2023-07-07 22:52:23 +02:00
Till Faelligen
ef32de928d
[NATS] Issue identified and fixed applied, workaround known. 2023-07-07 22:10:52 +02:00
Till
74a5ab6c24
Fix issues reported by Sentry (#3143)
This should fix a few issues reported by Sentry
2023-07-07 22:00:10 +02:00
Till
eb9e90379d
Add event size checks similar to Synapse (#3140)
Companion to https://github.com/matrix-org/gomatrixserverlib/pull/400
This tries to mimic the logic found in Synapse, as dropping events can
break rooms (and we may end up in endless loops..)
2023-07-07 20:37:23 +02:00
Neil
e93bdd56fd
Set max age for roomserver input stream to avoid excessive interior deletes (#3145)
If old messages build up in the input stream and do not get processed
successfully, this can create a significant drift between the stream
first sequence and the consumer ack floors, which results in a slow and
expensive start-up when interest-based retention is in use.

If a message is sat in the stream for 24 hours, it's probably not going
to get processed successfully, so let NATS drop them instead. Dendrite
can reconcile by fetching missing events later if it needs to.

---------

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2023-07-07 19:59:34 +02:00
Till
c08c7405db
Prepare statement on an existing transaction (#3144)
This should fix an issue with the database being locked for SQLite.
2023-07-07 13:09:39 +02:00
devonh
cc9b695c1e
Populate syncapi state event prev_sender with userID (#3142) 2023-07-06 23:54:35 +00:00
devonh
3a125fd8fa
Fix prev event lookup in syncapi (#3141)
The syncapi operates using userID's so when querying for the previous
state event we need to lookup the userID from the given senderID before
the state query.
2023-07-06 19:50:28 +00:00
devonh
d507c5fc95
Add pseudoID compatibility to Invites (#3126) 2023-07-06 15:15:24 +00:00
Till Faelligen
fea946d914
Don't spam the logs - downgrade sentry 2023-07-06 10:55:21 +02:00
Till Faelligen
9f7e14e4d0
Back to the original version for now 2023-07-06 10:44:11 +02:00
Till Faelligen
4a666932f5
[debug] Downgrade NATS 2023-07-06 10:31:32 +02:00
Till Faelligen
e1d76de6c6
Increase NATS server startup timeout 2023-07-06 10:04:46 +02:00
Till
49d75d3cf6
Version 0.13.1 (#3136) 2023-07-06 09:28:39 +02:00
Till Faelligen
5a87c703fa
Fix metrics.. 2023-07-05 12:34:53 +02:00
Till
4c3a526e1b
Fix adding state events to the database (#3133)
When we're adding state to the database, we check which eventNIDs are
already in a block, if we already have that eventNID, we remove it from
the list. In its current form we would skip over eventNIDs in the case
we already found a match (we're decrementing `i` twice)
My theory is, that when we later get the state blocks, we are receiving
"too many" eventNIDs (well, yea, we stored too many), which may or may
not can result in state resets when comparing different state snapshots.
(e.g. when adding state we stored a eventNID by accident because we
skipped it, later we add more state and are not adding it because we
don't skip it)
2023-07-04 17:15:44 +02:00
Till
2ee03fd657
Version 0.13.0 (#3127) 2023-06-30 08:49:37 +02:00
Omar Pakker
de1ed9d486
Extend Dendrite Helm chart with some additional config options (#3077)
This set of changes introduces a few (compatible) changes to the Helm
chart:
- Allow PVC class to be set on each PVC, not only one-for-all.
- Allow Prometheus servicemonitor and rules labels to be empty.
- Have the option to generate the ingress (incl. TLS config) based on
dendrite_config.


* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: Omar Pakker <Omar007@users.noreply.github.com>

---------

Signed-off-by: Omar Pakker <Omar007@users.noreply.github.com>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
2023-06-30 08:26:06 +02:00
Till Faelligen
939ee325f8
Actually use the parameter 2023-06-29 18:02:11 +02:00
Till
23cd7877a1
Add MXIDMapping for pseudoID rooms (#3112)
Add `MXIDMapping` on membership events when
creating/joining rooms.
2023-06-28 20:29:49 +02:00
Till
4722f12fab
Fix setting displayname and avatar_url (#3125)
As per the spec, `displayname` and `avatar_url` may be empty.
2023-06-28 20:18:07 +02:00
Till
a5ea928d0f
Fix syncAPI redactions (#3118)
Previously we were setting `redacted_because` to the PDU event, but as
per the spec it should really be a client event.
This fixes it.
2023-06-28 10:05:00 +02:00
santhoshivan23
45082d4dce
feat: admin APIs for token authenticated registration (#3101)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Santhoshivan Amudhan santhoshivan23@gmail.com`
2023-06-22 16:37:21 +00:00
Till
a734b112c6
Fix backfilling (#3117)
This should fix two issues with backfilling:
1. right after creating and joining a room over federation, we are doing
a `/backfill` request, which would return redacted events, because the
`authEvents` are empty. Even though the spec states that, in the absence
of a history visibility event, it should be handled as `shared`.
2. `gomatrixserverlib: unsupported room version ''` - because, well, we
were never setting the `roomInfo` field..
2023-06-20 16:52:29 +02:00
CicadaCinema
d13466c1ee
rearrange order of sections about signing keys and configuring dendrite, fix a dead link (#3114)
I thought I would rearrange these pages since the configuration step
requires that a signing key has been generated.

Co-authored-by: kegsay <kegan@matrix.org>
2023-06-18 22:54:16 +01:00
Josh Qou
420e7ec81f
Fix unsafe hotserving behaviour for multimedia uploads. (#3113)
Return multimedia with a disposition type of attachment instead of
inline. NVT#1548992

Signed-off-by: Josh Qou [jqou@icloud.com](mailto:jqou@icloud.com)

Co-authored-by: Jon <haddock.05.roast@icloud.com>
2023-06-15 12:28:34 +01:00
Devon Hudson
8cf6c381e2
Fix senderID/key conversion unit tests 2023-06-14 17:11:27 +01:00
Devon Hudson
3f4df25b31
Add missing dep 2023-06-14 17:04:19 +01:00
Devon Hudson
5aaa539e3e
Fix senderID/key conversions 2023-06-14 16:42:09 +01:00
devonh
e4665979bf
Merge SenderID & Per Room User Key work (#3109) 2023-06-14 14:23:46 +00:00
Till
7a2e325d10
Add AssignRoomNID to pre-assign roomNIDs (#3111) 2023-06-13 16:28:41 +02:00
Till
2c87972a3a
Create user room key if needed (#3108) 2023-06-13 14:19:31 +02:00
Till
82b73a4906
Add sender_key to ClientEvent (#3110) 2023-06-13 12:50:22 +02:00
devonh
77d9e4e93d
Cleanup remaining statekey usage for senderIDs (#3106) 2023-06-12 11:19:25 +00:00
Till
832ccc32f6
Add initial support for storing user room keys (#3098) 2023-06-12 12:45:42 +02:00
Antonio Cheong
5713c5715c
Update sample link (#3107)
Leftover work by f956a8c1d9

Signed-off-by: `Antonio Cheong <acheong@student.dalat.org>`

[skip ci]
2023-06-12 10:51:26 +02:00
devonh
8ea1a11105
Use SenderID Type (#3105) 2023-06-07 17:14:35 +00:00
devonh
7a1fd7f512
PDU Sender split (#3100)
Initial cut of splitting PDU Sender into SenderID & looking up UserID where required.
2023-06-06 20:55:18 +00:00
Till
725ff5567d
Make StrictValidityChecking a function (#3092)
Companion PR to https://github.com/matrix-org/gomatrixserverlib/pull/388
2023-06-06 15:16:55 +02:00
Till
d11da6ec7c
Fix newly found linter issues (#3099)
Fixes the issues found in
https://github.com/matrix-org/dendrite/actions/runs/5155539352/jobs/9285342056#step:5:22.
Only naked returns in longer functions.
2023-06-02 15:48:04 +02:00