Commit Graph

68 Commits

Author SHA1 Message Date
Andrew Morgan
e3b9754d59
Internal API for Appservices (#541) 2018-07-17 07:36:04 -07:00
Andrew Morgan
7736e247b7
Send Application Service Events (#477)
* Prevent sql scanning into nil value in accounts_table

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Remove uneccessary logging, null checking

* Don't forget to set the localpart

* Simplify error checking

* Store And Send Application Service Events

* Modify INSTALL.md and dendrite-config.yaml for the new appservice database
* Correct all instances of casing on 'application service' to align with
spec
* Store incoming events that an app service is interested in in the
database to be later read by transaction workers.
* Retrieve these events from transaction workers, one per AS.
* Minimal transaction ID data is stored as well to recover after
server failure.
* Send events to AS and exponentially backoff on failure.

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Finish my own sentences.

* Fix up database interaction

* Change to event-based AS sending

* Reduce cyclomatic complexity

* Appease the errcheck gods

* Delete by int ID instead of string.

This was causing some events to not be deleted, as < an eventID doesn't
really make much sense.

* Check if there are more events to send before sleeping

* Send same transaction if last send attempt failed

* Don't backoff on non-200s, tight send loop, 1 event query

* Remove tight send loop. Fix events not being deleted

* Additionally order by event id, track main.go

* Return the last txnID, which our events are using

* Remove old main.go file

* Prevent duplicate events from being sent...

* Strip event content if it doesn't contain anything

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Update gomatrixserverlib and use Unsigned AS event prop

* Fixes

* Fix sync server comment
* Remove unnecessary printlns
* Use logrus Fields
* Worker state methods
* Remove sillyness

* Fix up event filtering

* Handle transaction event limit in loop

* Switch to using a sequence for transaction IDs

* Don't verify self-signed AS certificates

* Fix logging

* Use gmsl.Event instead of AS-only event in transactions

Also clear up the logic on lookupStateEvents a little bit.

* Change invalid_txn_id to global (for efficiency)

* Use a bool for EventsReady instead of an int
2018-07-05 09:34:59 -07:00
Andrew Morgan
ed388a32b7
Add Application Service component (#462)
* Add Application Service component

The component subscribes to the Roomserver kafka stream, filtering
events to be eventually sent off to relevant app services, as well as
handling incoming HTTP traffic from app services on the
`/matrix/app/r0/*` route.

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Make linting happy

Signed-off-by: Andrew Morgan <andrewm@matrix.org>
2018-05-24 13:54:42 +01:00
Tristan Claverie
58c10c6d54 Enable debug logs using a configuration parameter and put all logs in a single file (#379) 2018-04-10 13:21:20 +01:00
Andrew Morgan
fa362ecef2 Load Application Service Configuration Files (#377)
Signed-off-by: Andrew Morgan (https://amorgan.xyz) <andrew@amorgan.xyz>
2017-12-19 17:00:44 +00:00
Erik Johnston
9476a266bd
Add database.naffka to example config (#350)
* Add database.naffka to example config

* Update dendrite-config.yaml

* Add naffka creation to INSTALL
2017-11-22 15:46:49 +00:00
Michael Telatynski
90396b5620 implement voip/turnServer API endpoint (#323)
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2017-11-09 09:58:45 +00:00
Erik Johnston
996b7d4198 Update dendrite-config.yaml to use server.crt, not .pem (#299) 2017-10-10 16:04:39 +01:00
Mark Haines
847621bc5d Add config for setting up a jaeger opentracing reporter (#271)
* Add config for setting up a jaeger opentracing reporter

* Remove redundant comment
2017-09-28 17:00:23 +01:00
Richard van der Hoff
3dd30858d1 Update the install instructions (#255)
* Update the install instructions

To include some of the newer components.

* update INSTALL diagram

client_data goes to sync-api-server, not room-server.
2017-09-25 11:20:36 +01:00
Brendan Abolivier
f1fce55697 Add config and checks for trusted ID servers (#206)
* Add config for trusted ID servers

* Add new error

* Implement check for trusted ID server

* Complete unfinished comment

* Make comment more explicit in the config file

* Use go standard errors in membership.go

* Use standard errors instead of JSON responses in threepid

* Doc errors

* Remove unused parameter
2017-09-11 19:18:19 +01:00
Brendan Abolivier
b15ce900ab Implement public rooms APIs (#185)
* Move events contents to common

* Basic database structure

* Complete database update

* Support visibility update and retrieval

* Add HTTP methods for visibility update and retrieval

* Add the database for the new component

* Add a listener for the new component

* Fix attribute update statements

* Create public rooms component

* Fix failing test

* Add roomserver consumer

* Fix a bug in aliases creation

* Add a check on type

* Implement public rooms directory

* Use auth API for visibility update

* Support filtering

* Add component to monolith

* Various fixes

* Fix computation of next public rooms batch

* Retrieve state events from the roomserver query API + avoid dupes on join

* Split update of string or boolean attribute in two separate functions

* Use event type to detect duplicate joins

* Improve the joined members counter computation

* Use event.RoomID()
2017-08-22 11:12:51 +01:00
Richard van der Hoff
46877b6baa Add installation instructions to repo (#192) 2017-08-21 17:34:05 +01:00
Mark Haines
c27d1fdfb4 Optionally use naffka in the monolithic server (#183)
* dependency injection for the kafka consumers/producers

* Optionally use naffka in the monolithic server

* remember to call setupKafka()

* tweak imports

* fix integration tests

* Add use_naffka to the example config

* Update comment on the listen APIs
2017-08-16 13:36:41 +01:00
Brendan Abolivier
0fbb8b7824 Make account data sync incremental (#170)
* Clean roomserver consumer

* Make account data sync incremental

* Use a different name for the sync AD table

* Improved error logging

* Created missing topic in tests

* Add client API topic to tests

* Add client API topic to common

* Move data batch retrieval

* Add database index for data retrieval

* Fix typo in table name

* Fix indentation
2017-08-02 16:21:35 +01:00
Brendan Abolivier
f40b776aa3 Updated config file example with federation sender database (#157) 2017-07-12 14:48:12 +01:00
Brendan Abolivier
355ab5eedf Notify profile update (#153)
* Profile retrieval

* Saving avatar (without propagating it)

* Saving display name (without propagating it)

* Getters for display name and avatar URL

* Doc'd

* Introduced new Kafka topic and producer

* Updated config with new kafka topic

* Switched to samara producer and now sending messages

* Doc'd

* Put kafka update after the database insert

* Doc'd profileUpdate structure

* Removed unused parameter

* Moved user updates producer to clientapi/producers
2017-07-11 14:14:06 +01:00
Mark Haines
b2f9b8c641 Update the example configs with the new unified config (#146)
* Update the example configs with the new unified config

* Review comments

* Use an int for the config version
2017-07-05 13:10:28 +01:00