Commit Graph

725 Commits

Author SHA1 Message Date
Andrew Morgan
a56752f3f6
implement AS timestamp massaging (#542) 2018-07-17 08:45:30 -07:00
Andrew Morgan
99005d6a91
Allow appservices to register without a login type (#529)
* Allow appservices to register without a logintype

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Reduce cyclomatic complexity on Register function

* Fix some grammar
2018-07-17 08:39:49 -07:00
Andrew Morgan
d2b349a75a
Check userID against AS namespace instead of localpart (#533) 2018-07-17 08:36:23 -07:00
Andrew Morgan
40b1dd6f12
Have TravisCI run with go 1.10 as well (#527) 2018-07-17 07:58:32 -07:00
Andrew Morgan
e3b9754d59
Internal API for Appservices (#541) 2018-07-17 07:36:04 -07:00
Anant Prakash
2fb2f7ca7b
Add AS support to common.MakeAuthAPI (#427)
* Add AS support to MakeAuthAPI
Make clientapi utilize the same

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Add user parameter support to MakeAuthAPI

* Make VerifyAccessToken private, let VerifyUserFromRequest return the
device if present

* Make a dummy device for AS users

* Refactor arguments into auth.Data

* Update routing of all components

* Update code comment

* Use const AppServiceDeviceID

* Handle cases when AS is not masquerading
2018-07-17 03:47:03 +05:30
Andrew Morgan
3ac23f9063
Remove trailing slashes from appservice url (#546)
* Remove trailing slashes from appservice url

* Use TrimRight instead
2018-07-16 08:16:01 -07:00
Andrew Morgan
49b63089f5
Consolidate AS interest checking (#539)
* Methods for checking if an AS is interested in events

* Look through rooms namespace for matching room IDs
2018-07-16 05:30:04 -07:00
Anant Prakash
d4b24462d1
return err when err != nil (#545) 2018-07-11 17:46:38 +05:30
Anant Prakash
730c4f74db
[federation] Fix state API endpoints (#518)
* [federation] Fix state API endpoints

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Use parseEventIDParam instead
2018-07-11 15:43:04 +05:30
Anant Prakash
c8feee7354
[roomserver] Add internal API QueryMembershipForUser (#544)
* Add internal API QueryMembershipForUser

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Change sender to user_id
2018-07-11 15:40:37 +05:30
Anant Prakash
74b319c42a
[federation] Add make_leave & send_leave (#535)
* [federation] Add make_leave & send_leave

* Remove redundant parameters

* Check membership is set to leave

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>
2018-07-11 15:37:44 +05:30
Anant Prakash
c87fb4d894
Remove unnecessary req.Close() (#537)
Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>
2018-07-06 17:06:19 +05:30
Andrew Morgan
385cab8dc9
Actually check if alias is reserved (#534) 2018-07-06 03:29:15 -07:00
Andrew Morgan
ae19db60e3
Check userID instead of username from application services (#500)
* Check UserID instead of username from AS's. Tests.

* add tests to validateApplicationService

* Use some literals, organize URLs & checks

* Fix error messages and incorrect test
2018-07-06 03:28:49 -07:00
Andrew Morgan
20af8a6786
Allow multiple regexes per as namespace (#528)
Signed-off-by: Andrew Morgan <andrewm@matrix.org>
2018-07-06 02:46:12 -07:00
Andrew Morgan
8da54352e5
Document internal APIs and Application Services component (#499)
* Document internal APIs, app services in WIRING.md

* Add application services component to WIRING diagram

* Fix typo and clarify what mode internal APIs use HTTP
2018-07-06 02:33:53 -07:00
Andrew Morgan
83b3df762b
Apply the same username checks to AS's, without '_' check (#501) 2018-07-06 02:27:11 -07:00
Andrew Morgan
7f7ae65098
Don't require preset to create room (#496)
* Don't require preset to create room

* Set defaults if no preset. Use switch instead of if

* Add default case and comment to r.Preset switch
2018-07-06 02:20:17 -07:00
Andrew Morgan
7736e247b7
Send Application Service Events (#477)
* Prevent sql scanning into nil value in accounts_table

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Remove uneccessary logging, null checking

* Don't forget to set the localpart

* Simplify error checking

* Store And Send Application Service Events

* Modify INSTALL.md and dendrite-config.yaml for the new appservice database
* Correct all instances of casing on 'application service' to align with
spec
* Store incoming events that an app service is interested in in the
database to be later read by transaction workers.
* Retrieve these events from transaction workers, one per AS.
* Minimal transaction ID data is stored as well to recover after
server failure.
* Send events to AS and exponentially backoff on failure.

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Finish my own sentences.

* Fix up database interaction

* Change to event-based AS sending

* Reduce cyclomatic complexity

* Appease the errcheck gods

* Delete by int ID instead of string.

This was causing some events to not be deleted, as < an eventID doesn't
really make much sense.

* Check if there are more events to send before sleeping

* Send same transaction if last send attempt failed

* Don't backoff on non-200s, tight send loop, 1 event query

* Remove tight send loop. Fix events not being deleted

* Additionally order by event id, track main.go

* Return the last txnID, which our events are using

* Remove old main.go file

* Prevent duplicate events from being sent...

* Strip event content if it doesn't contain anything

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Update gomatrixserverlib and use Unsigned AS event prop

* Fixes

* Fix sync server comment
* Remove unnecessary printlns
* Use logrus Fields
* Worker state methods
* Remove sillyness

* Fix up event filtering

* Handle transaction event limit in loop

* Switch to using a sequence for transaction IDs

* Don't verify self-signed AS certificates

* Fix logging

* Use gmsl.Event instead of AS-only event in transactions

Also clear up the logic on lookupStateEvents a little bit.

* Change invalid_txn_id to global (for efficiency)

* Use a bool for EventsReady instead of an int
2018-07-05 09:34:59 -07:00
Anant Prakash
8f5526763c Remove unused params (#538)
Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>
2018-07-04 03:05:20 -07:00
Anant Prakash
142fbcde21
Return 404 when profile is not found (#524)
* Return 404 when profile is not found

* Handler sql.ErrNoRows and avoid returning nil

update federation GetProfile as well

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>
2018-07-03 19:27:52 +05:30
Andrew Morgan
1cc21d6bcd
Prevent alias registration in AS exclusive namespace (#480)
* Prevent alias registration in AS exclusive namespace

* Fix M_Exclusive errors so they return status 400 instead of 403.

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Directly access namespace map instead of searching for key
2018-06-29 04:09:00 -07:00
Andrew Morgan
141fd91537
Prevent sql scanning into nil value in accounts_table (#479)
* Prevent sql scanning into nil value in accounts_table

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Remove uneccessary logging, null checking

* Don't forget to set the localpart

* Simplify error checking
2018-06-29 03:55:29 -07:00
Anant Prakash
af08eea46d
[federation] Add user_devices query (#498)
* [federation] Add user_devices query
2018-06-26 16:02:43 +05:30
Anant Prakash
262fc25aca
[federation] Implement get missing events api (#516)
* [federation] implement get_missing_events

* Check that events are viewable by the server

* Explain filterEvents
2018-06-26 15:55:49 +05:30
Anant Prakash
853e82506b
[federation] Implement state APIs (#486)
* Refactor GetEvents

* [federationapi] Add state APIs
2018-06-22 16:16:19 +05:30
Andrew Morgan
a76e31f1b3
Remove dependency errors (#507) 2018-06-18 05:44:25 -07:00
Andrew Morgan
06338b06d7
Prevent index out of bounds error (#503) 2018-06-18 04:03:08 -07:00
Andrew Morgan
93b7b18646
Add group_id, rate_limit and protocol AS config options (#478)
* Add group_id, rate_limit and protocol AS config options

* We currently just record and error check these options. There are not
currently implemented.

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Clean things up and fix yaml declaration

* Warn loudly when app service requests unimplemented options

* Fix comments

* Remove high cyclomatic complexity of appservice checkErrors

* Set default rate limited to true
2018-06-18 02:43:15 -07:00
Anant Prakash
78440083df
Add AS dummy device ID (#487)
Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>
2018-06-07 19:30:04 +05:30
Andrew Morgan
241b1b5ace
Update gomatrixserverlib version (#476)
Signed-off-by: Andrew Morgan <andrewm@matrix.org>
2018-06-01 17:42:55 +01:00
Andrew Morgan
63dc2141ba Fix typo that caused ASTokens to not be uniquely checked
Signed-off-by: Andrew Morgan <andrewm@matrix.org>
2018-06-01 16:38:38 +01:00
Anant Prakash
b86b55360a Add app service authentication functions (#433)
* Add support for AS ?user= parameter in auth

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Fix typo
2018-06-01 12:16:19 +01:00
Anant Prakash
1f570d0e92 Auto-generate username if none provided during registration (#470)
* Auto-generate username if none provided during registration

* Remove rogue backtick

* Add appropriate log msg
2018-05-31 15:36:15 +01:00
Anant Prakash
05be8d1c99 Move makeUserID into userutil and prevent code duplication (#475)
* Move makeUserID into userutil and prevent code duplication

* Correct ungraceful merge
2018-05-31 15:21:13 +01:00
Anant Prakash
7ac1efc4b3 Scan AppServiceID from accounts table, fix typo (#474) 2018-05-31 10:46:50 +01:00
Andrew Morgan
04551becb4
Filter Roomserver Events Based on Application Service (#467)
* Compile room and alias namespace regexs

We'll be needing these for event filtering in the appservice component.

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* App service filters roomserver events

Doing so based on namespace regexes that each app service has defined.

To get the aliases for a roomID a new aliasAPI endpoint was defined,
GetAliasesFromRoomID, which does exactly what it says on the tin.

Next step is to queue events to be sent off to each homeserver.

* Additionally filter state events for app services

* Fixed context, logging, derps, config handling

* Prevented user from creating more than one regex per namespace type

Got caught out by realizing I had an extra '-' in the config file. This
prevents anyone from making the same mistake :)

* Removed exclusive RoomID namespace regex, as we won't need to check
upon room creation if the ID is reserved exclusively by an AS (as this
is silly and horribly inefficient).

* Fixed all else mentioned
2018-05-30 13:43:13 +01:00
Anant Prakash
60e77959ee [roomserver] Store transaction ID and prevent reprocessing of events (#446) 2018-05-26 12:03:35 +01:00
Andrew Morgan
ed388a32b7
Add Application Service component (#462)
* Add Application Service component

The component subscribes to the Roomserver kafka stream, filtering
events to be eventually sent off to relevant app services, as well as
handling incoming HTTP traffic from app services on the
`/matrix/app/r0/*` route.

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Make linting happy

Signed-off-by: Andrew Morgan <andrewm@matrix.org>
2018-05-24 13:54:42 +01:00
Andrew Morgan
1d7278d950
Fix typo and bug in /devices/{deviceID} (#465)
* Fix typo in API Endpoint

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Fixed another typo and bug

Use a sql.NullInt64 instead of an Int64 as that allows for values to
sometimes be null when pulling from a postgres table. Can result in
error otherwise.

Signed-off-by: Andrew Morgan <andrewm@matrix.org>
2018-05-24 13:53:22 +01:00
Andrew Morgan
c238048599
Update Prometheus metrics tracking (#459)
Signed-off-by: Andrew Morgan <andrewm@matrix.org>
2018-05-23 15:42:08 +01:00
Andrew Morgan
93a6178d45 Fix typo in register.go
All in a hard day's work.

Signed-off-by: Andrew Morgan <andrewm@matrix.org>
2018-05-22 18:44:07 +01:00
Andrew Morgan
6fed3a1be2 Small cleanup to WIRING.md 2018-05-22 14:13:26 +01:00
Andrew Morgan
f5cc5bd9bb
Revert "Add token generation using go macaroon (#437)" (#448)
This reverts commit afeab7b2d4.

Signed-off-by: Andrew Morgan <andrewm@matrix.org>
2018-05-22 11:29:19 +01:00
Anant Prakash
afeab7b2d4 Add token generation using go macaroon (#437)
* Add Go macaroon library

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Add macaroon generation and serialization, for login token.

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Remove copyright, trim empty lines

* Make Serialize functions private

* Fix typos
2018-05-22 10:13:58 +01:00
Zero King
89e0a9e812 Fix typo (#445)
Signed-off-by: Zero King <l2dy@icloud.com>
2018-05-21 05:34:04 -07:00
Anant Prakash
29532e7bc3 Make clientapi:sendevents idempotent (#444)
* Add transactions.Cache to clientapi setup

* Add idempotency to clientapi/SendEvent
2018-05-18 10:49:40 +01:00
Anant Prakash
8861437c26 Add module for transactions cache (#440)
* Add transactions cache, write tests.

Add a transactions module in dendrite/common. This is needed for idempotent APIs.

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Use cycling double map instead, improve code logic, remove unneeded test

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Update code comments

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Use two constructors for default and custom cleanupPeriod
Add code comments

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>
2018-05-16 11:36:40 +01:00
Zero King
4c807e082f Fix typo (#441)
Signed-off-by: Zero King <l2dy@icloud.com>
2018-05-15 10:14:26 +01:00