Commit Graph

187 Commits

Author SHA1 Message Date
Neil Alexander
5912429d53
Return a more useful error on /register spec compliance violation (#1792) 2021-03-08 13:57:15 +00:00
Neil Alexander
1ad96e2e2d
Tweak AS registration check and AS component HTTP clients (#1785)
* Tweak AS registration check

* Check appservice usernames using correct function

* Update sytest-whitelist

* Use gomatrixserverlib.Client since that allows us to disable TLS validation using the config

* Add appservice-specific client and ability to control TLS validation for appservices only

* Set timeout on appservice client

* Review comments

* Remove dead code

* Enforce LoginTypeApplicationService after all

* Check correct auth type field
2021-03-05 10:40:27 +00:00
Will Hunt
9557ccada4
Fix appsevice alias queries part 2 (#1684)
* Check membership of room

* Use QueryStateAfterEventsResponse

* Fix complexity

* Add field ShouldHitAppservice to GetRoomIDForAlias

* Hit appservice when trying to join a non-existent alias

* remove unused

* Changes that I made a long time ago

* Rename to appserviceJoinedAtEvent

* Check membership in GetMemberships

* Update QueryMembershipsForRoom

* Tweaks in client API

* Update appserviceJoinedAtEvent

* Comments

* Try QueryMembershipForUser instead

* Undo some changes to client API that shouldn't be needed

* More /event tweaks

* Refactor /event bit

* Go back to QueryMembershipsForRoom because appservices are hard

* Fix bugs in onMessage

* Add comments

* More logical naming, clean up a bit

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-03-03 17:00:31 +00:00
Will Hunt
a2773922d2
Send events to appservice based on room membership (#1680)
* Check membership of room

* Use QueryStateAfterEventsResponse

* Fix complexity

* Changes that I made a long time ago

* Rename to appserviceJoinedAtEvent

* Check membership in GetMemberships

* Update QueryMembershipsForRoom

* Tweaks in client API

* Update appserviceJoinedAtEvent

* Comments

* Try QueryMembershipForUser instead

* Undo some changes to client API that shouldn't be needed

* More /event tweaks

* Refactor /event bit

* Go back to QueryMembershipsForRoom because appservices are hard

* Fix bugs in onMessage

* Add comments

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-03-03 16:27:44 +00:00
Neil Alexander
d15836e260
Increase gocyclo complexity to 25 (and remove all but 2 golint directives related to it) (#1783) 2021-03-03 14:35:57 +00:00
Neil Alexander
81312b8a78
Return the current OTK count on an empty upload request (#1774)
* Always return OTK counts

* Fix parameter ordering

* Send IDs over to keyserver internal API

* Review comments

* Fix syntax error

* Fix panic, hopefully

* Require user ID to be set

* Fix user API call
2021-03-02 11:40:20 +00:00
Michael Telatynski
f448e8972a
Implement /createRoom power_level_content_override (#1761)
* Implement /createRoom power_level_content_override

* delint

* Skip having to re-marshal PowerLevelContentOverride by making use of `json.RawMessage`
2021-02-12 09:43:31 +00:00
Kegsay
93942f8ab6
Gate peeking behind msc flags (#1731) 2021-01-22 16:08:47 +00:00
sumitks866
dd1e31bee7
Mention unstable features in /_matrix/client/versions (#1710)
* Mention unstable features in /_matrix/client/versions

* list enabled mscs

* Don't update go.mod/go.sum

* goimports

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-01-18 13:09:28 +00:00
Neil Alexander
932ff6b166
Fix /register 2020-12-18 14:51:30 +00:00
Will Hunt
e4d1f0958f
Do not check if a username is exclusive if the request is for an appservice in /register (#1652)
* Do not check if a username is exclusive if the request is for an appservice in /register

* remove useless comment

* Move statements

* fix broken test

* Also fix the senderLocalpart problem

* Check domain name is ours

* Handle accessTokenErr

* Return unauthorised instead of forbidden

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-12-18 14:36:59 +00:00
Neil Alexander
65ff5c9a2c
Don't send null in joined_rooms response (#1659) 2020-12-18 13:33:08 +00:00
Neil Alexander
9a1ef81fa2
Add CS sendevent metrics
Squashed commit of the following:

commit c38c39107b6dda0c8e6320d61da2365c47eea4e9
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Thu Dec 17 10:13:09 2020 +0000

    Update buckets

commit 5a3bcc8bd4167150374827b0e42a0dea0366beff
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Thu Dec 17 09:41:43 2020 +0000

    Update buckets

commit 78dff8165ddf596e28af04faf56466752ebc17af
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Thu Dec 17 09:37:27 2020 +0000

    Register sendEventDuration

commit 612b0f3d84f1547ff30131f7b084baf609edab52
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Thu Dec 17 09:35:59 2020 +0000

    sendevent metrics
2020-12-17 10:52:31 +00:00
Neil Alexander
be7d8595be
Peeking updates (#1607)
* Add unpeek

* Don't allow peeks into encrypted rooms

* Fix send tests

* Update consumers
2020-12-03 11:11:46 +00:00
alexkursell
2b03d24358
Fix /joined_members API response (#1606)
* Fix /joined_members API response

* Fix golint issue
2020-12-03 11:01:49 +00:00
Neil Alexander
b5aa7ca3ab
Top-level setup package (#1605)
* Move config, setup, mscs into "setup" top-level folder

* oops, forgot the EDU server

* Add setup

* goimports
2020-12-02 17:41:00 +00:00
alexkursell
13cbd50dc2
Add last_seen_ip and last_seen_ts to /devices response (#1592) 2020-11-20 09:26:50 +00:00
bodqhrohro
1cf9f20d2f
Allow = in user identifiers (#1590)
While I was breaking through all the TDD bureaucracy to finally push this fix,
it turned out that it already got fixed in #1578. Still I push the rest of the
changes (basically, everything except of actually fixing the bug ;)).

`=` is crucial for matrix-bifrost which turns XMPP @'s into =40.

Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>
2020-11-18 22:56:06 +00:00
David Spenler
35ea55e70b
Implemented whois endpoint (#1573)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-11-17 10:07:03 +00:00
Neil Alexander
20a01bceb2
Pass pointers to events — reloaded (#1583)
* Pass events as pointers

* Fix lint errors

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update to matrix-org/gomatrixserverlib#240
2020-11-16 15:44:53 +00:00
Neil Alexander
8ce740d949
Fix username regex to allow = character (#1578) 2020-11-12 10:36:54 +00:00
S7evinK
bcb89ada5e
Implement read receipts (#1528)
* fix conversion from int to string yields a string of one rune, not a string of digits

* Add receipts table to syncapi

* Use StreamingToken as the since value

* Add required method to testEDUProducer

* Make receipt json creation "easier" to read

* Add receipts api to the eduserver

* Add receipts endpoint

* Add eduserver kafka consumer

* Add missing kafka config

* Add passing tests to whitelist

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Fix copy & paste error

* Fix column count error

* Make outbound federation receipts pass

* Make "Inbound federation rejects receipts from wrong remote" pass

* Don't use errors package

* - Add TODO for batching requests
- Rename variable

* Return a better error message

* - Use OutputReceiptEvent instead of InputReceiptEvent as result
- Don't use the errors package for errors
- Defer CloseAndLogIfError to close rows
- Fix Copyright

* Better creation/usage of JoinResponse

* Query all joined rooms instead of just one

* Update gomatrixserverlib

* Add sqlite3 migration

* Add postgres migration

* Ensure required sequence exists before running migrations

* Clarification on comment

* - Fix a bug when creating client receipts
- Use concrete types instead of interface{}

* Remove dead code
Use key for timestamp

* Fix postgres query...

* Remove single purpose struct

* Use key/value directly

* Only apply receipts on initial sync or if edu positions differ,
otherwise we'll be sending the same receipts over and over again.

* Actually update the id, so it is correctly send in syncs

* Set receipt on request to /read_markers

* Fix issue with receipts getting overwritten

* Use fmt.Errorf instead of pkg/errors

* Revert "Add postgres migration"

This reverts commit 722fe5a04628882b787d096942459961db159b06.

* Revert "Add sqlite3 migration"

This reverts commit d113b03f6495a4b8f8bcf158a3d00b510b4240cc.

* Fix selectRoomReceipts query

* Make golangci-lint happy

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-11-09 18:46:11 +00:00
S7evinK
eccd0d2c1b
Implement forgetting about rooms (#1572)
* Add basic storage methods

* Add internal api handler

* Add check for forgotten room

* Add /rooms/{roomID}/forget endpoint

* Add missing rsAPI method

* Remove unused parameters

* Add passing tests

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Add missing file

* Add postgres migration

* Add sqlite migration

* Use Forgetter to forget room

* Remove empty line

* Update HTTP status codes

It looks like the spec calls for these to be 400, rather than 403: https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-rooms-roomid-forget

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-11-05 10:19:23 +00:00
Devon Johnson
45abdcaeb9
Send state after event, not current
Signed-off-by: Devon Johnson <djohnson1865@gmail.com>
2020-10-19 15:12:37 +01:00
Neil Alexander
6e63df1d9a
KindOld (#1531)
* Add KindOld

* Don't process latest events/memberships for old events

* Allow federationsender to ignore duplicate key entries when LatestEventIDs is duplicated by RS output events

* Signal to downstream components if an event has become a forward extremity

* Don't exclude from sync

* Soft-fail checks on KindNew

* Don't run the latest events updater at all for KindOld

* Don't make federation sender change after all

* Kind in federation sender join

* Don't send isForwardExtremity

* Fix syncapi

* Update comments

* Fix SendEventWithState

* Update sytest-whitelist

* Generate old output events

* Sync API consumes old room events

* Update comments
2020-10-19 14:59:13 +01:00
Neil Alexander
640e8c50ec
Take write lock for rate limit map (#1532)
* Take write lock for rate limit map

* Fix potential race condition
2020-10-16 15:44:39 +01:00
Neil Alexander
49abe359e6
Start Kafka connections for each component that needs them (#1527)
* Start Kafka connection for each component that needs one

* Fix roomserver unit tests

* Rename to naffkaInstance (@Kegsay review comment)

* Fix import cycle
2020-10-15 13:27:13 +01:00
Neil Alexander
20aec70ead
Send cumulative state when creating room (#1519)
* Send state with new room events

* lookupEvent sends outliers

* Revert "lookupEvent sends outliers"

This reverts commit 3e1655644105a542b806e28d6d2536fbd23ecc83.
2020-10-14 09:57:27 +01:00
S7evinK
1cd525ef0d
Extend device_devices table (#1471)
* Add last_used_ts and IP to database

* Add migrations

* Rename column
Prepare statements

* Add interface method and implement it

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Rename struct fields

* Add user_agent to database

* Add userAgent to registration calls

* Add missing "IF NOT EXISTS"

* use txn writer

* Add UserAgent to Device

Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-09 09:17:23 +01:00
Pestdoktor
c4c8bfd027
reject invalid UTF-8 (#1472)
* reject invalid UTF-8

Signed-off-by: Jonas Fentker <jonas@fentker.eu>

* update sytest-whitelist

Signed-off-by: Jonas Fentker <jonas@fentker.eu>

Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-09 09:15:51 +01:00
Loïck Bonniot
f3e8ae01ef
Implement fully read markers (#1475)
See #653

Signed-off-by: Loïck Bonniot <git@lesterpig.com>

Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-09 09:15:35 +01:00
Neil Alexander
533006141e
Return 200 on join before time out (#1493)
* Return 200 on join afer 15 seconds if nothing better has happened by that point

* Return 202 instead, 20 second timeout
2020-10-07 15:29:14 +01:00
Loïck Bonniot
4e8c484618
Implement account deactivation (#1455)
* Implement account deactivation

See #610
Signed-off-by: Loïck Bonniot <git@lesterpig.com>

* Rename 'is_active' to 'is_deactivated'

Signed-off-by: Loïck Bonniot <git@lesterpig.com>

Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-02 17:18:20 +01:00
Loïck Bonniot
439bc235d7
Add 'completed' field in Interactive Auth API (#1469)
This field must be present even after authentication failure, as
tested by sytest. This is needed by #1455.

Signed-off-by: Loïck Bonniot <git@lesterpig.com>
2020-10-02 15:05:23 +01:00
bn4t
da89f2bd1f
Finish implementing the capabilities endpoint (#1449)
Closes #1310

Signed-off-by: Benjamin Nater <me@bn4t.me>
2020-09-29 10:06:59 +01:00
Matthew Hodgson
5bbe73747c unbreak CORS on /capabilities.
fixes https://github.com/vector-im/element-web/issues/15297
2020-09-27 22:23:42 +01:00
Matthew Hodgson
39507bacc3
Peeking via MSC2753 (#1370)
Initial implementation of MSC2753, as tested by https://github.com/matrix-org/sytest/pull/944.
Doesn't yet handle unpeeks, peeked EDUs, or history viz changing during a peek - these will follow.
https://github.com/matrix-org/dendrite/pull/1370 has full details.
2020-09-10 14:39:18 +01:00
Kegsay
c992f4f1f4
Remove current state server (#1405)
* Remove current state server

Closes #1365 #1272 #1357

* Remove current state server from scripts/docs
2020-09-07 14:47:59 +01:00
Kegsay
7913759921
Remove QueryBulkStateContent from current state server (#1404)
* Remove QueryBulkStateContent from current state server

Expected fail due to db impl not existing

* Implement query bulk state content

* Fix up rejecting invites over federation

* Fix bulk content marshalling
2020-09-07 12:38:09 +01:00
Kegsay
088294ee65
Remove QueryRoomsForUser from current state server (#1398) 2020-09-04 15:58:30 +01:00
Neil Alexander
5076925c18
Password changes (#1397)
* User API support for password changes

* Password changes in client API

* Update sytest-whitelist

* Remove debug logging

* Default logout_devices to true

* Fix deleting devices by local part
2020-09-04 15:16:13 +01:00
Kegsay
81688d6bde
Remove QueryCurrentState from current-state-server (#1395) 2020-09-04 12:30:56 +01:00
Kegsay
67cc8619d3
Remove QueryKnownUsers from current state server (#1393)
* Remove QueryKnownUsers from current state server

* Fix HTTP mode
2020-09-04 11:46:01 +01:00
Neil Alexander
6150de6cb3
FIFO ordering of input events (#1386)
* Initial FIFOing of roomserver inputs

* Remove EventID response from api.InputRoomEventsResponse

* Don't send back event ID unnecessarily

* Fix ordering hopefully

* Reduce copies, use buffered task channel to reduce contention on other rooms

* Fix error handling
2020-09-03 15:22:16 +01:00
Neil Alexander
74743ac8ae
Rate limiting (#1385)
* Initial rate limiting

* Move rate limiting to client API

* Update rate limits to hopefully be self-cleaning

* Use X-Forwarded-For, add comments

* Reduce rate limit threshold

* Tweak interval

* Configurable backoff

* Review comments, set cleanup interval to 30 seconds

* Allow generate-config to produce sane CI config

* Fix Complement dockerfile
2020-09-03 10:12:11 +01:00
Kegsay
9d9e854fe0
Add Queryer and Inputer and factor out more RSAPI stuff (#1382)
* Add Queryer and use embedded structs

* Add Inputer and factor out more RS API stuff

This neatly splits up the RS API based on the functionality it provides,
whilst providing a useful place for code sharing via the `helpers` package.
2020-09-02 17:13:15 +01:00
Neil Alexander
f06637435b
Fix #1381 (#1384) 2020-09-02 16:52:06 +01:00
Rohit Mohan
3f9b829bc5
Public room client API changes (#1368)
Signed-off-by: Rohit Mohan <rohitmohan96@gmail.com>
2020-09-01 10:26:34 +01:00
Kegsay
9af2f5f1f2
Remove device DB from clientapi (#1352)
* Remove device DB from clientapi

* Remove device DB from startup configuration

It's all an impl detail now in user API
2020-08-27 18:53:40 +01:00
Neil Alexander
42183bef50
Allow join to accept server_name in query parameters (#1346) 2020-08-26 18:23:08 +01:00