Commit Graph

1820 Commits

Author SHA1 Message Date
Neil Alexander
f64c8822bc
Federation sender refactor (#1621)
* Refactor federation sender, again

* Clean up better

* Missing operators

* Try to get overflowed events from database

* Fix queries

* Log less

* Comments

* nil PDUs/EDUs shouldn't happen but guard against them for safety

* Tweak logging

* Fix transaction coalescing

* Update comments

* Check nils more

* Remove channels as they add extra complexity and possibly will deadlock

* Don't hold lock while sending transaction

* Less spam about sleeping queues

* Comments

* Bug-fixing

* Don't try to rehydrate twice

* Don't queue in memory for blacklisted destinations

* Don't queue in memory for blacklisted destinations

* Fix a couple of bugs

* Check for duplicates when pulling things out of the database

* Durable transactions, some more refactoring

* Revert "Durable transactions, some more refactoring"

This reverts commit 5daf924eaaefec5e4f7c12c16ca24e898de4adbb.

* Fix deadlock
2020-12-09 10:03:22 +00:00
Neil Alexander
e1d32e2ff1
Update to matrix-org/gomatrixserverlib@232bace 2020-12-07 16:21:55 +00:00
bodqhrohro
45dec20593
Update links to golangci-lint documentation (#1617)
Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>
2020-12-07 11:23:01 +00:00
Neil Alexander
8f5393db5f
Update FAQ 2020-12-07 10:20:19 +00:00
Neil Alexander
5d65a879a5
Federation sender event cache (#1614)
* Cache federation sender events

* Store in the correct cache

* Update federation event cache

* Fix Unset

* Give EDUs same caching treatment as PDUs

* Make federationsender_cache_size configurable

* Default caches configuration

* Fix unit tests

* Revert "Fix unit tests"

This reverts commit 24eb5d22524f20e1024b1475debe61ae20538a5a.

* Revert "Default caches configuration"

This reverts commit 464ecd1e64b9d2983f6fd5430e9607519d543cb3.

* Revert "Make federationsender_cache_size configurable"

This reverts commit 4631f5324151e006a15d6f19008f06361b994607.
2020-12-04 14:52:10 +00:00
Kegsay
b507312d4c
MSC2836 threading: part 2 (#1596)
* Update GMSL

* Add MSC2836EventRelationships to fedsender

* Call MSC2836EventRelationships in reqCtx

* auth remote servers

* Extract room ID and servers from previous events; refactor a bit

* initial cut of federated threading

* Use the right client/fed struct in the response

* Add QueryAuthChain for use with MSC2836

* Add auth chain to federated response

* Fix pointers

* under CI: more logging and enable mscs, nil fix

* Handle direction: up

* Actually send message events to the roomserver..

* Add children and children_hash to unsigned, with tests

* Add logic for exploring threads and tracking children; missing storage functions

* Implement storage functions for children

* Add fetchUnknownEvent

* Do federated hits for include_children if we have unexplored children

* Use /ev_rel rather than /event as the former includes child metadata

* Remove cross-room threading impl

* Enable MSC2836 in the p2p demo

* Namespace mscs db

* Enable msc2836 for ygg

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-12-04 14:11:01 +00:00
TR_SLimey
c052edafdd
Fix broken image in CONTRIBUTING.md (#1612) 2020-12-04 14:10:01 +00:00
Neil Alexander
68d92fd3b4
Update sytest-whitelist/blacklist 2020-12-04 14:09:21 +00:00
Ronnie Ebrin
a677a288bd
federationsender/roomserver: don't panic while federation is disabled (#1615) 2020-12-04 14:08:17 +00:00
Neil Alexander
1ce9c52442
Don't recalculate event IDs so often (#1610)
* Don't recalculate event IDs so often

* Revert invite change

* Make sure we're using the right NIDs

* Update gomatrixserverlib

* Update to NewEventFromTrustedJSONWithEventID

* Fix go.mod

* Update gomatrixserverlib to matrix-org/gomatrixserverlib#243

* Use BulkSelectEventID
2020-12-04 10:41:07 +00:00
Petter Jakub Økland
19b1d40d64
Hiawatha reverse proxy sample for monolith and fixes to polylith sample (#1609)
* Created polylith reverse proxy sample for Hiawatha

* Create monolith-sample.conf

* Added timeout to proxies and binding explanation.

* Fixed typo.

* Consistency with regards to polylith-sample.conf.
2020-12-03 13:52:00 +00:00
Neil Alexander
52905ffb82
Update PROFILING.md 2020-12-03 13:49:34 +00:00
Neil Alexander
253b05ccde
Update PROFILING.md 2020-12-03 13:48:02 +00:00
Neil Alexander
246866a131
Add PROFILING.md 2020-12-03 13:46:56 +00:00
Neil Alexander
eef8f88092
Update FAQ.md 2020-12-03 13:28:41 +00:00
Neil Alexander
a4bf9921ad
Update FAQ.md 2020-12-03 13:27:48 +00:00
Neil Alexander
71327b8efa
Add FAQ.md 2020-12-03 13:22:32 +00:00
Neil Alexander
be7d8595be
Peeking updates (#1607)
* Add unpeek

* Don't allow peeks into encrypted rooms

* Fix send tests

* Update consumers
2020-12-03 11:11:46 +00:00
alexkursell
2b03d24358
Fix /joined_members API response (#1606)
* Fix /joined_members API response

* Fix golint issue
2020-12-03 11:01:49 +00:00
Neil Alexander
ec7a0e42ae
Simplify create-account (#1608) 2020-12-03 10:55:17 +00:00
Neil Alexander
1f3a498601
Version 0.3.2 2020-12-02 17:46:28 +00:00
Neil Alexander
b5aa7ca3ab
Top-level setup package (#1605)
* Move config, setup, mscs into "setup" top-level folder

* oops, forgot the EDU server

* Add setup

* goimports
2020-12-02 17:41:00 +00:00
Neil Alexander
3ef6187e96
Revert "Update quic-go"
This reverts commit 93c3f1a7b3.
2020-12-02 17:03:36 +00:00
Neil Alexander
93c3f1a7b3
Update quic-go 2020-12-02 16:39:41 +00:00
Neil Alexander
417c7d3569
Send client events to appservices (#1603)
* Send client events to appservices

* FormatSync instead of FormatAll
2020-12-02 15:14:12 +00:00
Neil Alexander
bdf6490375
Add ability to disable federation (#1604)
* Allow disabling federation

* Don't start federation queues if disabled

* Fix for Go 1.13
2020-12-02 15:10:03 +00:00
Neil Alexander
b4c3692dcc
Optimise CheckServerAllowedToSeeEvent (#1602)
* Try to limit how many state events we have to unmarshal

* Comments
2020-12-02 11:45:50 +00:00
Ariadne Conill
d5b8260196
syncapi/requestpool: fix initial sync logic error in appendAccountData() (#1594)
* requestpool: fix initial sync logic error in appendAccountData()

In initial sync, req.since is no longer nil, but instead,
req.since.PDUPosition() and req.since.EDUPosition() returns 0.

This ensures forgotten rooms do not come back as zombies.

* syncapi/requestpool: reintroduce req.since == nil check
2020-12-01 17:01:34 +00:00
bodqhrohro
c667a19775
Assign CREATEDB permission to the test user (#1591)
During the tests databases get recreated, and this fails despite of the
user being the owner of a dropped database. Maybe related to certain
PostgreSQL version.

Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>
2020-12-01 16:59:47 +00:00
Neil Alexander
c16abb089c
Give fsAPI to keyserver in polylith/gobind 2020-12-01 10:52:02 +00:00
Neil Alexander
9c52f82736
Version 0.3.1 2020-11-20 11:38:58 +00:00
Neil Alexander
c636be5070
Update last seen on sync requests (#1593)
* Update last seen on sync requests

* Fix MSC2836 unit tests

* Only update once per minute

* Remove debug logging

* Configurable option

* Simplify updateLastSeen/cleanLastSeen
2020-11-20 11:29:02 +00:00
alexkursell
13cbd50dc2
Add last_seen_ip and last_seen_ts to /devices response (#1592) 2020-11-20 09:26:50 +00:00
Kegsay
6353b0b7e4
MSC2836: Threading - part one (#1589)
* Add mscs/hooks package, begin work for msc2836

* Flesh out hooks and add SQL schema

* Begin implementing core msc2836 logic

* Add test harness

* Linting

* Implement visibility checks; stub out APIs for tests

* Flesh out testing

* Flesh out walkThread a bit

* Persist the origin_server_ts as well

* Edges table instead of relationships

* Add nodes table for event metadata

* LEFT JOIN to extract origin_server_ts for children

* Add graph walking structs

* Implement walking algorithm

* Add more graph walking tests

* Add auto_join for local rooms

* Fix create table syntax on postgres

* Add relationship_room_id|servers to the unsigned section of events

* Persist the parent room_id/servers in edge metadata

Other events cannot assert the true room_id/servers for the
parent event, only make claims to them, hence why this is
edge metadata.

* guts to pass through room_id/servers

* Refactor msc2836 to allow handling from federation

* Add JoinedVia to PerformJoin responses

* Fix tests; review comments
2020-11-19 11:34:59 +00:00
bodqhrohro
1cf9f20d2f
Allow = in user identifiers (#1590)
While I was breaking through all the TDD bureaucracy to finally push this fix,
it turned out that it already got fixed in #1578. Still I push the rest of the
changes (basically, everything except of actually fixing the bug ;)).

`=` is crucial for matrix-bifrost which turns XMPP @'s into =40.

Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>
2020-11-18 22:56:06 +00:00
Neil Alexander
e65a3443a4
Fix paths in quickstart 2020-11-18 15:11:48 +00:00
Neil Alexander
265cf5e835
Protect txnReq.newEvents with mutex (#1587)
* Protect txnReq.newEvents and txnReq.haveEvents with mutex

* Missing defer

* Remove t.haveEventsMutex
2020-11-18 11:31:58 +00:00
TR_SLimey
3bcb003248
Add IPv6 setting to nginx conf (#1582)
* Added IPv6 setting to nginx conf

https://matrix.to/#/!yomrOFwgFXzmeMAbzX:matrix.org/$sIkSxL002BKottY1Fop-bkusuyCJ1Y0zIFzqSWDYRrc?via=matrix.org&via=privacytools.io&via=dendrite.link

* Add IPv6 support to polylith nginx conf also
2020-11-18 10:33:17 +00:00
David Spenler
35ea55e70b
Implemented whois endpoint (#1573)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-11-17 10:07:03 +00:00
Aaron Raimist
d3b3371856
Update README to remove mention that read receipts are not implemented (#1586)
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2020-11-17 08:43:22 +00:00
Neil Alexander
20a01bceb2
Pass pointers to events — reloaded (#1583)
* Pass events as pointers

* Fix lint errors

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update to matrix-org/gomatrixserverlib#240
2020-11-16 15:44:53 +00:00
Neil Alexander
d8b526b603
Update docker readme 2020-11-16 11:27:59 +00:00
Neil Alexander
bfa9df4a5a
Acknowledgements in changelog 2020-11-16 11:25:24 +00:00
Neil Alexander
3856e547de
Version 0.3.0 2020-11-16 11:13:05 +00:00
Neil Alexander
3034d8e805
Update sytest lists to blacklist flaky 'Forgotten room messages cannot be paginated' test 2020-11-16 10:51:55 +00:00
Mayeul Cantan
af41fcadc4
Fix Dendrite not backfilling on world_readable rooms (#1575)
The previous implementation was only checking if room history was
"shared", which it wasn't for rooms where a user was invited, or world
readable rooms.
This implementation leverages the IsServerAllowed method, which already
implements the complete verification algorithm.

Signed-off-by: `Mayeul Cantan <oss+matrix@mayeul.net>`

Co-authored-by: Kegsay <kegan@matrix.org>
2020-11-16 10:47:16 +00:00
Felix Hanley
41c0b5acfb
Update build script to run on any POSIX (#1577)
Signed-off-by: Felix Hanley <felix@userspace.com.au>
2020-11-16 10:09:53 +00:00
S7evinK
cb41c992f7
Add -version flag (#1581)
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2020-11-16 09:54:58 +00:00
Neil Alexander
b7e288084c
Update README.md 2020-11-15 12:39:11 +00:00
Neil Alexander
8ce740d949
Fix username regex to allow = character (#1578) 2020-11-12 10:36:54 +00:00