Commit Graph

2586 Commits

Author SHA1 Message Date
Brandon
015465d496
Update 1_planning.md (#2467)
* Update 1_planning.md

Modes section of the planning component of the documentation rewritten for grammar and clarity.

* Update 1_planning.md

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-05-25 17:17:02 +01:00
Neil Alexander
6940c7c7dd
Try to spot state deletions when they happen (#2489) 2022-05-25 16:40:31 +01:00
Neil Alexander
ff53398635
Mark room versions 8 and 9 as stable 2022-05-25 16:39:51 +01:00
Neil Alexander
ac4d0072cf
Fix using the remote provided event from /send_join (#2490) 2022-05-25 14:31:07 +01:00
Neil Alexander
9ce3898d03
Fix the resolve-state tool 2022-05-25 13:37:15 +01:00
Neil Alexander
81843e8836
Restricted join support on /make_join, /send_join (#2478)
* Add `QueryRestrictedJoinAllowed`

* Add `Resident` flag to `QueryRestrictedJoinAllowedResponse`

* Check restricted joins on federation API

* Return `Restricted` to determine if the room was restricted or not

* Populate `AuthorisedVia` properly

* Sign the event on `/send_join`, return it in the `/send_join` response in the `"event"` key

* Kick back joins with invalid authorising user IDs, use event from `"event"` key if returned in `RespSendJoin`

* Use invite helper in `QueryRestrictedJoinAllowed`

* Only use users with the power to invite, change error bubbling a bit

* Placate the almighty linter

One day I will nuke `gocyclo` from orbit and everything in the world will be much better for it.

* Review comments
2022-05-25 10:05:30 +01:00
Neil Alexander
d621dd2986
Allow authing invited user in rooms with "restricted" join rule (update to matrix-org/gomatrixserverlib#313) 2022-05-24 11:08:28 +01:00
Neil Alexander
ead0112aa1
Fix join_authorised_via_users_server key name in SendEvent 2022-05-24 10:22:26 +01:00
Neil Alexander
7379b02b70
Give PL100 to invited users in trusted_private_chat preset (#2485) 2022-05-24 10:10:25 +01:00
Neil Alexander
c83837e684
Strip join_authorised_by_users_server when updating membership events using /rooms/{roomID}/state (#2482) 2022-05-24 09:11:23 +01:00
Aaron Ogle
20844942a8
Handle invite in appservice roomserver consumer (#2476)
Signed-off-by: Aaron Ogle <aaron@geekgonecrazy.com>
2022-05-24 09:07:33 +01:00
Neil Alexander
4472267901
Preserve content from /make_join as well as supplied content in the request (#2481) 2022-05-23 17:54:01 +01:00
Neil Alexander
8a7567c652
Fix control flow related to joining restricted v8/v9 rooms (update to matrix-org/gomatrixserverlib#312) 2022-05-23 17:49:48 +01:00
Neil Alexander
b5a497a0c0
Allow defers to run in TestMain in federation API tests 2022-05-23 14:54:43 +01:00
Neil Alexander
028963af1c
Update to matrix-org/gomatrixserverlib#311 (#2479)
* Update to matrix-org/gomatrixserverlib#311

* Update to squash commit
2022-05-23 14:26:56 +01:00
Till
a53c9300aa
Fix /deactivate (#2474)
* Fix /deactivate

* Update test to correctly check the expected response
2022-05-20 13:27:11 +02:00
kegsay
21dd5a7176
syncapi: don't return early for no-op incremental syncs (#2473)
* syncapi: don't return early for no-op incremental syncs

Comments explain why, but basically it's an inefficient use
of bandwidth and some sytests rely on /sync to block.

* Honour timeouts

* Actually return a response with timeout=0
2022-05-19 09:00:56 +01:00
Till
f321a7d55e
Really SKIP_NODB (#2472)
* Really SKIP_NODB

* Use fatalError in createLocalDB

* Check if createdb exists

* Revert change

* Remove !Quiet
2022-05-18 15:17:23 +02:00
kegsay
b3162755a9
bugfix: fix race condition when updating presence via /sync (#2470)
* bugfix: fix race condition when updating presence via /sync

Previously when presence is updated via /sync, we would send the presence update
asyncly via NATS. This created a race condition:
 - If the presence update is processed quickly, the /sync which triggered the presence
   update would see an online presence.
 - If the presence update was processed slowly, the /sync which triggered the presence
   update would see an offline presence.

This is the root cause behind the flakey sytest: 'User sees their own presence in a sync'.

The fix is to ensure we update the database/advance the stream position synchronously
for local users.

* Bugfix for test
2022-05-17 15:53:08 +01:00
Kegan Dougal
ac92e04772 Remove debug logging 2022-05-17 13:31:48 +01:00
kegsay
6de29c1cd2
bugfix: E2EE device keys could sometimes not be sent to remote servers (#2466)
* Fix flakey sytest 'Local device key changes get to remote servers'

* Debug logs

* Remove internal/test and use /test only

Remove a lot of ancient code too.

* Use FederationRoomserverAPI in more places

* Use more interfaces in federationapi; begin adding regression test

* Linting

* Add regression test

* Unbreak tests

* ALL THE LOGS

* Fix a race condition which could cause events to not be sent to servers

If a new room event which rewrites state arrives, we remove all joined hosts
then re-calculate them. This wasn't done in a transaction so for a brief period
we would have no joined hosts. During this interim, key change events which arrive
would not be sent to destination servers. This would sporadically fail on sytest.

* Unbreak new tests

* Linting
2022-05-17 13:23:35 +01:00
kegsay
cd82460513
Add docs which explain how to calculate coverage (#2468) 2022-05-17 10:45:50 +01:00
Till
05607d6b87
Add roomserver tests (3/4) (#2447)
* Add Room Aliases tests

* Add Rooms table test

* Move StateKeyTuplerSorter to the types package

* Add StateBlock tests
Some optimizations

* Add State Snapshot tests
Some optimization

* Return []int64 and convert to pq.Int64Array for postgres

* Move []types.EventNID back to rows.Next()

* Update tests, rename SelectRoomIDs
2022-05-16 19:33:16 +02:00
Neil Alexander
6af35385ba
Version 0.8.5 (#2461)
* Version 0.8.5

* Update changelog

* Update changelog
2022-05-13 13:17:15 +01:00
Neil Alexander
b40b548432
The Pinecone gobind demo must listen on localhost for baseURL to be correct 2022-05-13 12:06:47 +01:00
Neil Alexander
1698c39579
Update README.md 2022-05-13 11:52:42 +01:00
Neil Alexander
be9be2553f
Resolve over old and new extremities (#2457)
* Feed existing state into state res when calculating state from new extremities

* Remove duplicates

* Fix bug

* Sort and unique

* Update to matrix-org/gomatrixserverlib#308

* Trim the slice properly

* Update gomatrixserverlib again

* Update to matrix-org/gomatrixserverlib#308
2022-05-13 11:52:04 +01:00
Neil Alexander
cafc2d2c10
Update NATS Server to version 2.8.2 (#2460) 2022-05-13 11:36:04 +01:00
Till Faelligen
b57fdcc82d Only try to get OTKs if the context isn't done yet 2022-05-13 10:28:00 +02:00
Till
870f9b0c3f
Shuffle config Verify/Defaults a bit around (#2459) 2022-05-13 09:33:55 +02:00
Neil Alexander
fc670f03a2
Separate sample configs for monolith and polylith (#2456)
* Update sample configs

* Update references

* Remove sections that are dead in the monolith sample
2022-05-12 12:05:55 +01:00
Till
0d1505a4c1
Fix create-account with global database settings (#2455)
* Fix create-account with global database settings

* Avoid warning about open registration
2022-05-12 11:35:35 +02:00
Kegan Dougal
3437adf597 Wait 100ms for events to be processed by syncapi 2022-05-12 10:11:46 +01:00
Till
58af7f61b6
Fix OTK upload spam (#2448)
* Fix OTK spam

* Update comment

* Optimize selectKeysCountSQL to only return max 100 keys

* Return CurrentPosition if the request timed out

* Revert "Return CurrentPosition if the request timed out"

This reverts commit 7dbdda964189f5542048c06ce5ffc6d4da1814e6.

Co-authored-by: kegsay <kegan@matrix.org>
2022-05-11 17:15:18 +01:00
Neil Alexander
24f7be968d
Fix link 2022-05-11 15:46:45 +01:00
Neil Alexander
19a9166eb0
New documentation: https://matrix-org.github.io/dendrite/ 2022-05-11 15:39:36 +01:00
kegsay
9599b3686e
More syncapi tests (#2451)
* WIP tests for flakey create event

* Uncomment all database test
2022-05-11 13:44:32 +01:00
kegsay
c15bfefd0d
Add RoomExists flag to QueryMembershipForUser (#2450)
Fixes https://github.com/matrix-org/complement/pull/369
2022-05-11 11:29:23 +01:00
Till
6db08b2874
Add roomserver tests (2/?) (#2445)
* Add invite table tests; move variable declarations

* Add Membership table tests

* Move variable declarations

* Add PrevEvents table tests

* Add Published table test

* Add Redactions tests
Fix bug in SQLite markRedactionValidatedSQL

* PR comments, better readability for invite tests
2022-05-10 14:41:12 +02:00
Neil Alexander
1897e2f1c0
Version 0.8.4 2022-05-10 12:44:29 +01:00
Neil Alexander
e2a932ec0b
Add indexes to syncapi_output_room_events table that satisfy the filters (#2446) 2022-05-10 11:23:36 +01:00
Neil Alexander
77722c5a4f
Back out matrix-org/dendrite#2421 by restoring http.Clients
This creates problems with non-HTTPS endpoints and should fix #2444.
2022-05-10 11:08:10 +01:00
database64128
1b3fa9689c
🏗️ mediaapi/thumbnailer: fix build with bimg (#2440)
Co-authored-by: kegsay <kegan@matrix.org>
2022-05-09 17:51:30 +01:00
Neil Alexander
6b3c183396
Version 0.8.3 (#2431)
* Version 0.8.3

* Update changelog
2022-05-09 17:31:14 +01:00
kegsay
236b16aa6c
Begin adding syncapi component tests (#2442)
* Add very basic syncapi tests

* Add a way to inject jetstream messages

* implement add_state_ids

* bugfixes

* Unbreak tests

* Remove now un-needed API call

* Linting
2022-05-09 17:23:02 +01:00
Neil Alexander
a443d1e5f3
Don't store invites in sync API that aren't relevant to local users (#2439) 2022-05-09 16:25:22 +01:00
Neil Alexander
79da75d483
Federation consumer adds_state_event_ids tweak (#2441)
* Don't ask roomserver for events we already have in federation API

* Check number of events returned is as expected

* Preallocate array

* Improve shape a bit
2022-05-09 16:19:35 +01:00
Neil Alexander
1a7f4c8aa9
Don't try to re-fetch the event if it is listed in adds_state_event_ids (#2437)
* Don't try to re-fetch the event in the output message

* Try that again

* Add the initial event into the set
2022-05-09 15:22:33 +01:00
Till
f69ebc6af2
Add roomserver tests (1/?) (#2434)
* Add EventJSONTable tests

* Add eventJSON tests

* Add EventStateKeysTable tests

* Add EventTypesTable tests

* Add Events Table tests
Move variable declaration outside loops
Switch to testify/assert for tests

* Move variable declaration outside loop

* Remove random data

* Fix issue where the EventReferenceSHA256 is not set

* Add more tests

* Revert "Fix issue where the EventReferenceSHA256 is not set"

This reverts commit 8ae34c4e5f78584f0edb479f5a893556d2b95d19.

* Update GMSL

* Add tests for duplicate entries

* Test what happens if we select non-existing NIDs

* Add test for non-existing eventType

* Really update GMSL
2022-05-09 15:30:32 +02:00
Neil Alexander
09d754cfbf
One NATS instance per BaseDendrite (#2438)
* One NATS instance per `BaseDendrite`

* Fix roomserver
2022-05-09 14:15:24 +01:00