Neil Alexander
92982a402f
Update Docker ( #1542 )
...
* Separate Docker images, rather than tags
* Allow specifying tag to build/push/pull
* Include goose in Docker builds
2020-10-20 11:34:22 +01:00
Neil Alexander
a71360d099
Update INSTALL.md ( #1549 )
2020-10-20 11:22:46 +01:00
Neil Alexander
c7bf122a26
Update sytest lists
2020-10-19 15:38:42 +01:00
Neil Alexander
5d80ff11a0
Update sytest-whitelist
2020-10-19 15:17:28 +01:00
Devon Johnson
45abdcaeb9
Send state after event, not current
...
Signed-off-by: Devon Johnson <djohnson1865@gmail.com>
2020-10-19 15:12:37 +01:00
Neil Alexander
6e63df1d9a
KindOld ( #1531 )
...
* Add KindOld
* Don't process latest events/memberships for old events
* Allow federationsender to ignore duplicate key entries when LatestEventIDs is duplicated by RS output events
* Signal to downstream components if an event has become a forward extremity
* Don't exclude from sync
* Soft-fail checks on KindNew
* Don't run the latest events updater at all for KindOld
* Don't make federation sender change after all
* Kind in federation sender join
* Don't send isForwardExtremity
* Fix syncapi
* Update comments
* Fix SendEventWithState
* Update sytest-whitelist
* Generate old output events
* Sync API consumes old room events
* Update comments
2020-10-19 14:59:13 +01:00
S7evinK
0974f6e2c0
Fix internal http api ( #1535 )
...
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2020-10-19 09:38:10 +01:00
Neil Alexander
640e8c50ec
Take write lock for rate limit map ( #1532 )
...
* Take write lock for rate limit map
* Fix potential race condition
2020-10-16 15:44:39 +01:00
Kegsay
4a7fb9c045
Automatically upgrade databases on startup ( #1529 )
...
* Support auto-upgrading accounts DB
* Auto-upgrade device DB deltas
* Support up/downgrading from cmd/goose
* Linting
* Create tables then do migrations then prepare statements
To avoid failing due to some things not existing
* Linting
2020-10-15 18:09:41 +01:00
Neil Alexander
3e5d38e284
Improve state resolution v2 performance ( matrix-org/gomatrixserverlib#237 )
2020-10-15 16:20:19 +01:00
Neil Alexander
e154c45b51
Better logging around db.StoreEvent
2020-10-15 14:14:17 +01:00
Neil Alexander
49abe359e6
Start Kafka connections for each component that needs them ( #1527 )
...
* Start Kafka connection for each component that needs one
* Fix roomserver unit tests
* Rename to naffkaInstance (@Kegsay review comment)
* Fix import cycle
2020-10-15 13:27:13 +01:00
Neil Alexander
10f1beb0de
Don't re-run state resolution on a single trusted state snapshot ( #1526 )
...
* Don't re-run state resolution on a single trusted state snapshot
* Lint
* Check if backward extremity is create event before checking missing state
2020-10-15 12:08:49 +01:00
Kegsay
e3c2b081c7
txn nil guard when rolling back LatestEventsUpdater ( #1524 )
...
* txn nil guard when rolling back LatestEventsUpdater
* Spell lint correctly
2020-10-14 17:05:09 +01:00
Neil Alexander
6f12b8f85c
Ignore typing events where sender doesn't match origin ( #1523 )
...
* Ignore typing notifications where the sender doesn't match the origin
* Update sytest-whitelist
* Fix formatting directives
2020-10-14 16:49:25 +01:00
Neil Alexander
e3a3908654
Update sytest-whitelist ( closes #1315 )
2020-10-14 16:07:15 +01:00
Neil Alexander
8d9ecb3996
Ignore duplicate redaction entries ( #1522 )
2020-10-14 15:24:43 +01:00
Neil Alexander
286dd408ae
Better semver compliance for version reporting
2020-10-14 12:51:06 +01:00
Neil Alexander
7a1fd123de
Improved state handling in /send ( #1521 )
...
* Capture errors
* Don't request only state key tuples needed for auth (we end up discarding room state this way)
* QueryStateAfterEvent returns all state when no tuples supplied
* Resolve state
* Comments
2020-10-14 12:39:37 +01:00
Neil Alexander
20aec70ead
Send cumulative state when creating room ( #1519 )
...
* Send state with new room events
* lookupEvent sends outliers
* Revert "lookupEvent sends outliers"
This reverts commit 3e1655644105a542b806e28d6d2536fbd23ecc83.
2020-10-14 09:57:27 +01:00
Neil Alexander
9d6b77c58a
Try to retrieve missing auth events from multiple servers ( #1516 )
...
* Recursively fetch auth events if needed
* Fix processEvent call
* Ask more servers in lookupEvent
* Don't panic!
* Panic at the Disco
* Find servers more aggressively
* Add getServers
* Fix number of servers to 5, don't bail making RespState if auth events missing
* Fix panic
* Ignore missing state events too
* Report number of servers correctly
* Don't reuse request context for /send_join
* Update federation API tests
* Don't recurse processEvents
* Implement getEvents differently
2020-10-13 11:53:20 +01:00
Kegsay
d7ea814fa8
Wrap NewMembershipUpdater in a db writer ( #1515 )
2020-10-13 10:20:27 +01:00
Neil Alexander
73bc28b11f
Update docker-compose.deps.yml
2020-10-13 09:54:07 +01:00
Neil Alexander
2f578531ba
Update Docker samples
2020-10-13 09:50:11 +01:00
Neil Alexander
8001627cfc
Get missing event tweaks ( #1514 )
...
* Adjust backfill to send backward extremity with state before other backfilled events, include prev_events with no state amongst missing events
* Not finished refactor
* Fix test
* Remove isInboundTxn
* Remove debug logging
2020-10-12 15:56:15 +01:00
Pieter
0804594a61
Add systemd example service to docs ( #1512 )
...
Signed-off-by: Pieter Hollander <git@pieterhollander.nl>
2020-10-12 10:06:56 +01:00
Neil Alexander
6b579e6aba
Don't ignore local keys from fetchers ( #1513 )
2020-10-12 10:02:20 +01:00
Kegsay
9096bfcee8
Validate m.room.create events in send_join responses ( #1505 )
...
* Validate m.room.create events in send_join responses
For sytest compliance, refs #1315 and #1317
Fixes #1317
* Linting
2020-10-10 00:21:15 +01:00
Neil Alexander
fe5d1400bf
Update federation timeouts ( #1504 )
...
* Update to matrix-org/gomatrixserverlib#234
* Update gomatrixserverlib
* Update federation timeouts
* Fix dendritejs
* Increase /send context time in destination queue
2020-10-09 17:08:32 +01:00
Neil Alexander
4df7e345bb
Only return 500 on /send if a database error occurs ( #1503 )
2020-10-09 15:06:43 +01:00
S7evinK
2bd0449c5b
Use default value when adding column which is not null ( #1501 )
...
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2020-10-09 12:41:53 +01:00
S7evinK
1cd525ef0d
Extend device_devices table ( #1471 )
...
* Add last_used_ts and IP to database
* Add migrations
* Rename column
Prepare statements
* Add interface method and implement it
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
* Rename struct fields
* Add user_agent to database
* Add userAgent to registration calls
* Add missing "IF NOT EXISTS"
* use txn writer
* Add UserAgent to Device
Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-09 09:17:23 +01:00
Pestdoktor
c4c8bfd027
reject invalid UTF-8 ( #1472 )
...
* reject invalid UTF-8
Signed-off-by: Jonas Fentker <jonas@fentker.eu>
* update sytest-whitelist
Signed-off-by: Jonas Fentker <jonas@fentker.eu>
Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-09 09:15:51 +01:00
Loïck Bonniot
f3e8ae01ef
Implement fully read markers ( #1475 )
...
See #653
Signed-off-by: Loïck Bonniot <git@lesterpig.com>
Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-09 09:15:35 +01:00
Kegsay
009401ad4d
Version 0.1.0
...
Beta README (#1466 )
* Beta docs
* More tweaks
* More docs
* Update README.md (#1497 )
* Call out missing features
* Add CHANGES
* Call out CHANGES
* Update INSTALL.md
* Update README.md
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-10-08 17:47:43 +01:00
Kegan Dougal
b12b7abcc0
v0.1.0
...
Squashed commit of the following:
commit 570528e5f19d76fe2fd7ea646246eea9650b1b60
Author: Kegan Dougal <kegan@matrix.org>
Date: Thu Oct 8 14:52:10 2020 +0100
v0.1.0
commit 02c020bced3be6d1a63e1001ab9c0ef6231ba087
Merge: db840f02 8035c50c78f6e1a33e12f6e98bca7a8352ddded7
2020-10-08 14:53:46 +01:00
Neil Alexander
8035c50c06
Don't get into situations where we have no forward extremities
2020-10-08 14:46:25 +01:00
Neil Alexander
78f6e1a31e
Don't set new state NID if state regression
2020-10-08 13:30:30 +01:00
Kegsay
a846dad0e1
Return what we have when we encounter missing events when servicing backfill/gme ( #1499 )
...
We expect to have missing events as we walk back in the DAG over federation
as we didn't always create the room. When checking if the server is allowed
to see those events, just give up and stop rather than fail the request.
2020-10-08 12:22:00 +01:00
Neil Alexander
429bd48129
Return a non-fatal error to the federation API on a state regression ( #1498 )
...
* Return a non-fatal error to the federation API on a state regression
* Return no error but don't morph state
2020-10-08 12:13:50 +01:00
Kegsay
3e12f6e9c2
Remove notifs about key changes in syncapi ( #1496 )
...
The join/leave events themselves will wake up the right people so we
needn't do it twice.
2020-10-08 10:27:10 +01:00
Neil Alexander
8b880be57e
Include a stripped version of the invite itself ( #1495 )
2020-10-08 10:03:37 +01:00
Neil Alexander
8bca7a83a9
Update monolith -api behaviour ( #1484 )
...
* Update monolith -api mode listeners
* Fix check
* Fix another check
* Update HTTP API addr behaviour
* Redefine NoExternalListener
* NoListener
2020-10-07 16:59:22 +01:00
Kegsay
bf7e85848b
Rename serverkeyapi to signingkeyserver ( #1492 )
...
* Rename serverkeyapi to signingkeyserver
We use "api" for public facing stuff and "server" for internal stuff.
As the server key API is internal only, we call it 'signing key server',
which also clarifies the type of key (as opposed to TLS keys, E2E keys, etc)
* Convert docker/scripts to use signing-key-server
* Rename missed bits
2020-10-07 16:23:18 +01:00
Neil Alexander
533006141e
Return 200 on join before time out ( #1493 )
...
* Return 200 on join afer 15 seconds if nothing better has happened by that point
* Return 202 instead, 20 second timeout
2020-10-07 15:29:14 +01:00
Neil Alexander
d821f9d3c9
Deep checking of forward extremities ( #1491 )
...
* Deep forward extremity calculation
* Use updater txn
* Update error
* Update error
* Create previous event references in StoreEvent
* Use latest events updater to row-lock prev events
* Fix unexpected fallthrough
* Fix deadlock
* Don't roll back
* Update comments in calculateLatest
* Don't include events that we can't find references for in the forward extremities
* Add another passing test
2020-10-07 14:05:33 +01:00
Neil Alexander
f7c15071de
Don't return 500s on checking to see if a remote server is allowed to see an event we don't know about ( #1490 )
2020-10-07 10:30:27 +01:00
Kegsay
0f7e707f39
Optimise servers to backfill from ( #1485 )
...
- Prefer perspective servers if they are in the room.
- Limit the number of backfill servers to 5 to avoid taking too long.
2020-10-06 18:09:02 +01:00
Neil Alexander
4feff8e8d9
Don't give up if we fail to fetch a key ( #1483 )
...
* Don't give up if we fail to fetch a key
* Fix logging line
* furl nolint
2020-10-06 17:59:08 +01:00
Neil Alexander
ee79d662e7
Update to matrix-org/gomatrixserverlib#233
2020-10-06 15:37:31 +01:00
