Commit Graph

66 Commits

Author SHA1 Message Date
Emanuele Aliberti
fb10633a80
build.sh to build.cmd (#2319)
* build.sh to build.cmd

convert and adapt for Windows users

* remove hardwired GO ARCH

* silence cmd.exe

* update directions for Windows

* Update INSTALL.md

Co-authored-by: emanuele.aliberti <emanuele.aliberti@mtka.eu>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-04-05 11:27:29 +01:00
Neil Alexander
6748a2a823
Fix sign-off link in CONTRIBUTING.md 2022-04-04 11:02:16 +01:00
S7evinK
49dc49b232
Remove eduserver (#2306)
* Move receipt sending to own JetStream producer

* Move SendToDevice to producer

* Remove most parts of the EDU server

* Fix SendToDevice & copyrights

* Move structs, cleanup EDU Server traces

* Use HeadersOnly subscription

* Missing file

* Fix linter issues

* Move consumers to own files

* Rename durable consumer; Consumer cleanup

* Docs/config cleanup
2022-03-29 14:14:35 +02:00
Neil Alexander
a96098051e
Update FAQ.md 2022-03-18 11:44:13 +00:00
Neil Alexander
d78e0a33e0
Remove references to userapi_devices (fixes #2259) 2022-03-09 16:38:50 +00:00
Neil Alexander
264165eb8c
Update systemd example to set LimitNOFILE 2022-02-25 17:35:10 +00:00
Neil Alexander
8a1dfffe3d
Various updates for renaming the master branch to main 2022-02-08 16:16:01 +00:00
Neil Alexander
2c3dd48bb2
Require Go 1.16 (#2122) 2022-01-28 16:24:01 +00:00
S7evinK
e7f9f2e6f6
Update bridge FAQ & README (#2106)
* Update bridge FAQ

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Update README
2022-01-24 10:29:45 +00:00
FORCHA
3c92b55bec
Update monolith-sample.conf (#2087)
* Update monolith-sample.conf

-Replaced undefined monolith value  with server_name  (my.hostname.com) value in reference tho ths issue https://github.com/matrix-org/dendrite/issues/2078

* Update monolith-sample.conf

Changed IP to location of monolith server

Co-authored-by: kegsay <kegan@matrix.org>
2022-01-21 14:37:59 +00:00
S7evinK
161f145176
Add NATS JetStream support (#1866)
* Add NATS JetStream support
Update shopify/sarama

* Fix addresses

* Don't change Addresses in Defaults

* Update saramajetstream

* Add missing error check

Keep typing events for at least one minute

* Use all configured NATS addresses

* Update saramajetstream

* Try setting up with NATS

* Make sure NATS uses own persistent directory (TODO: make this configurable)

* Update go.mod/go.sum

* Jetstream package

* Various other refactoring

* Build fixes

* Config tweaks, make random jetstream storage path for CI

* Disable interest policies

* Try to sane default on jetstream base path

* Try to use in-memory for CI

* Restore storage/retention

* Update nats.go dependency

* Adapt changes to config

* Remove unneeded TopicFor

* Dep update

* Revert "Remove unneeded TopicFor"

This reverts commit f5a4e4a339b6f94ec215778dca22204adaa893d1.

* Revert changes made to streams

* Fix build problems

* Update nats-server

* Update go.mod/go.sum

* Roomserver input API queuing using NATS

* Fix topic naming

* Prometheus metrics

* More refactoring to remove saramajetstream

* Add missing topic

* Don't try to populate map that doesn't exist

* Roomserver output topic

* Update go.mod/go.sum

* Message acknowledgements

* Ack tweaks

* Try to resume transaction re-sends

* Try to resume transaction re-sends

* Update to matrix-org/gomatrixserverlib@91dadfb

* Remove internal.PartitionStorer from components that don't consume keychanges

* Try to reduce re-allocations a bit in resolveConflictsV2

* Tweak delivery options on RS input

* Publish send-to-device messages into correct JetStream subject

* Async and sync roomserver input

* Update dendrite-config.yaml

* Remove roomserver tests for now (they need rewriting)

* Remove roomserver test again (was merged back in)

* Update documentation

* Docker updates

* More Docker updates

* Update Docker readme again

* Fix lint issues

* Send final event in `processEvent` synchronously (since this might stop Sytest from being so upset)

* Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that

* Go 1.16 instead of Go 1.13 for upgrade tests and Complement

* Revert "Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that"

This reverts commit 368675283fc44501f227639811bdb16dd5deef8c.

* Don't report any errors on `/send` to see what fun that creates

* Fix panics on closed channel sends

* Enforce state key matches sender

* Do the same for leave

* Various tweaks to make tests happier

Squashed commit of the following:

commit 13f9028e7a63662759ce7c55504a9d2423058668
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Tue Jan 4 15:47:14 2022 +0000

    Do the same for leave

commit e6be7f05c349fafbdddfe818337a17a60c867be1
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Tue Jan 4 15:33:42 2022 +0000

    Enforce state key matches sender

commit 85ede6d64bf10ce9b91cdd6d80f87350ee55242f
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Tue Jan 4 14:07:04 2022 +0000

    Fix panics on closed channel sends

commit 9755494a98bed62450f8001d8128e40481d27e15
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Tue Jan 4 13:38:22 2022 +0000

    Don't report any errors on `/send` to see what fun that creates

commit 3bb4f87b5dd56882febb4db5621db484c8789b7c
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Tue Jan 4 13:00:26 2022 +0000

    Revert "Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that"

    This reverts commit 368675283fc44501f227639811bdb16dd5deef8c.

commit fe2673ed7be9559eaca134424e403a4faca100b0
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Tue Jan 4 12:09:34 2022 +0000

    Go 1.16 instead of Go 1.13 for upgrade tests and Complement

commit 368675283fc44501f227639811bdb16dd5deef8c
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Tue Jan 4 11:51:45 2022 +0000

    Don't report event rejection errors via `/send`, since apparently this is upsetting tests that don't expect that

commit b028dfc08577bcf52e6cb498026e15fa5d46d07c
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Tue Jan 4 10:29:08 2022 +0000

    Send final event in `processEvent` synchronously (since this might stop Sytest from being so upset)

* Merge in NATS Server v2.6.6 and nats.go v1.13 into the in-process connection fork

* Add `jetstream.WithJetStreamMessage` to make ack/nak-ing less messy, use process context in consumers

* Fix consumer component name in  federation API

* Add comment explaining where streams are defined

* Tweaks to roomserver input with comments

* Finish that sentence that I apparently forgot to finish in INSTALL.md

* Bump version number of config to 2

* Add comments around asynchronous sends to roomserver in processEventWithMissingState

* More useful error message when the config version does not match

* Set version in generate-config

* Fix version in config.Defaults

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-01-05 17:44:49 +00:00
Neil Alexander
da101469fa
Minimum version Go 1.15 (#1957) 2021-08-03 11:35:29 +01:00
J. Ryan Stinnett
034f5ae3f1
Add startup testing for Wasm Pinecone build (#1910)
* Only include go-sqlite3 on the relevant binaries

* The driver name is always sqlite3 now

* Update to matrix-org/go-sqlite3-js@e537baa

* Add initial Wasm test harness

* Upgrade go-sqlite3-js

This fixes an error about semicolons in single statements.

* Add browser-like WebSocket API for testing

* Upgrade go-sqlite3-js

This upgrade includes printing panic messages next to stacks.

* Run for all PRs targeting any branch

* Use manual Node caching

* Temporarily run for all pushes

* Use npm ci instead of install

* Use HTTPS auth for repo packages

* Match path style from build.sh

* update utp

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-07-20 12:14:58 +01:00
Rasmus Thomsen
2c9a390fa6
Add CORS to nginx config (#1791)
Without this entry, setups where users have the homeserver on the URL
matrix.myurl.com but want the servername to be myurl.com don't work by default
since clients like element.io can't connect to the homeserver
2021-06-07 11:06:57 +01:00
Arnaud Venturi
6fa5743ea0
Add missing IPv6 "ssl" keyword in nginx config examples (#1854)
Signed-off-by: Arnaud Venturi <git@toadjaune.eu>
2021-06-07 09:29:54 +01:00
Neil Alexander
1002e87b60
Pinecone P2P demo (#1856)
* Pinecone demo

* Enable multicast, fix HTTP routing

* Fix multicast import

* Fix build

* Update Pinecone demo

* Fix the keys

* Tweaks

* Pinecone room directory support (early)

* Fix gobind-pinecone

* Add pinecone listener

* Fix public key value

* Use AuthenticatedConnect for dial

* Fix gobind-pinecone

* Stop panics

* Give fsAPI to keyserver

* Pinecone demo fixes

* Update gobind build scripts

* Account creation

* Tweaks

* Setup tweaks

* API tweaks

* API tweaks

* API tweaks

* Port mutex

* Re-enable multicast

* Add ReadCopy

* Update quic-go, fixes

* Shutdowns fixed for iOS

* Update build script

* Add WebSocket support

* Bug fixes

* Netconn context

* Fix WebSocket connectivity

* Fixes to gobind API

* Strip frameworks

* Configurability updates

* Update go.mod

* Update go.mod/go.sum

* Update go.mod/go.sum

* Update go.mod/go.sum

* Try to stay connected tto static peer

* Update gobind-pinecone

* Update go.mod/go.sum

* Test uTP+TLS

* Use HTTP/2

* Don't use HTTP/2

* Update go.mod/go.sum

* Attempt to reconnect to the static peer if it drops

* Stay connected to static peers more stickily

* Retry room directory lookups if they fail

* NewQUIC -> NewSessions

* Storage updates

* Don't return immediately when there's nothing to sync

* Updates

* Try to reconnect to static peer more

* Update go.mod/go.sum

* Require Go 1.14

* Update go.mod/go.sum

* Update go.mod/go.sum
2021-05-06 12:00:42 +01:00
Marco Kundt
d1496793b9
fix database names to reflect renaming (#1636)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-02-17 15:20:06 +00:00
Matthew Hodgson
0571d395b5
Peeking over federation via MSC2444 (#1391)
* a very very WIP first cut of peeking via MSC2753.

doesn't yet compile or work.
needs to actually add the peeking block into the sync response.
checking in now before it gets any bigger, and to gather any initial feedback on the vague shape of it.

* make PeekingDeviceSet private

* add server_name param

* blind stab at adding a `peek` section to /sync

* make it build

* make it launch

* add peeking to getResponseWithPDUsForCompleteSync

* cancel any peeks when we join a room

* spell out how to runoutside of docker if you want speed

* fix SQL

* remove unnecessary txn for SelectPeeks

* fix s/join/peek/ cargocult fail

* HACK: Track goroutine IDs to determine when we write by the wrong thread

To use: set `DENDRITE_TRACE_SQL=1` then grep for `unsafe`

* Track partition offsets and only log unsafe for non-selects

* Put redactions in the writer goroutine

* Update filters on writer goroutine

* wrap peek storage in goid hack

* use exclusive writer, and MarkPeeksAsOld more efficiently

* don't log ascii in binary at sql trace...

* strip out empty roomd deltas

* re-add txn to SelectPeeks

* re-add accidentally deleted field

* reject peeks for non-worldreadable rooms

* move perform_peek

* fix package

* correctly refactor perform_peek

* WIP of implementing MSC2444

* typo

* Revert "Merge branch 'kegan/HACK-goid-sqlite-db-is-locked' into matthew/peeking"

This reverts commit 3cebd8dbfbccdf82b7930b7b6eda92095ca6ef41, reversing
changes made to ed4b3a58a7855acc43530693cc855b439edf9c7c.

* (almost) make it build

* clean up bad merge

* support SendEventWithState with optional event

* fix build & lint

* fix build & lint

* reinstate federated peeks in the roomserver (doh)

* fix sql thinko

* todo for authenticating state returned by /peek

* support returning current state from QueryStateAndAuthChain

* handle SS /peek

* reimplement SS /peek to prod the RS to tell the FS about the peek

* rename RemotePeeks as OutboundPeeks

* rename remote_peeks_table as outbound_peeks_table

* add perform_handle_remote_peek.go

* flesh out federation doc

* add inbound peeks table and hook it up

* rename ambiguous RemotePeek as InboundPeek

* rename FSAPI's PerformPeek as PerformOutboundPeek

* setup inbound peeks db correctly

* fix api.SendEventWithState with no event

* track latestevent on /peek

* go fmt

* document the peek send stream race better

* fix SendEventWithRewrite not to bail if handed a non-state event

* add fixme

* switch SS /peek to use SendEventWithRewrite

* fix comment

* use reverse topo ordering to find latest extrem

* support postgres for federated peeking

* go fmt

* back out bogus go.mod change

* Fix performOutboundPeekUsingServer

* Fix getAuthChain -> GetAuthChain

* Fix build issues

* Fix build again

* Fix getAuthChain -> GetAuthChain

* Don't repeat outbound peeks for the same room ID to the same servers

* Fix lint

* Don't omitempty to appease sytest

Co-authored-by: Kegan Dougal <kegan@matrix.org>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-01-22 14:55:08 +00:00
rany
ef24ea2678
Add madvdontneed=1 to decrease RAM consumption for systemd service (#1690)
* Add madvdontneed=1 to decrease RAM consumption for systemd service

* Update monolith-example.service

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-01-14 10:31:45 +00:00
Neil Alexander
d7824ed5b1
Update INSTALL.md (#1637) 2020-12-14 09:42:27 +00:00
Neil Alexander
b794873fa0
Update INSTALL.md 2020-12-12 20:55:34 +00:00
bodqhrohro
45dec20593
Update links to golangci-lint documentation (#1617)
Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>
2020-12-07 11:23:01 +00:00
Neil Alexander
8f5393db5f
Update FAQ 2020-12-07 10:20:19 +00:00
TR_SLimey
c052edafdd
Fix broken image in CONTRIBUTING.md (#1612) 2020-12-04 14:10:01 +00:00
Petter Jakub Økland
19b1d40d64
Hiawatha reverse proxy sample for monolith and fixes to polylith sample (#1609)
* Created polylith reverse proxy sample for Hiawatha

* Create monolith-sample.conf

* Added timeout to proxies and binding explanation.

* Fixed typo.

* Consistency with regards to polylith-sample.conf.
2020-12-03 13:52:00 +00:00
Neil Alexander
52905ffb82
Update PROFILING.md 2020-12-03 13:49:34 +00:00
Neil Alexander
253b05ccde
Update PROFILING.md 2020-12-03 13:48:02 +00:00
Neil Alexander
246866a131
Add PROFILING.md 2020-12-03 13:46:56 +00:00
Neil Alexander
eef8f88092
Update FAQ.md 2020-12-03 13:28:41 +00:00
Neil Alexander
a4bf9921ad
Update FAQ.md 2020-12-03 13:27:48 +00:00
Neil Alexander
71327b8efa
Add FAQ.md 2020-12-03 13:22:32 +00:00
bodqhrohro
c667a19775
Assign CREATEDB permission to the test user (#1591)
During the tests databases get recreated, and this fails despite of the
user being the owner of a dropped database. Maybe related to certain
PostgreSQL version.

Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>
2020-12-01 16:59:47 +00:00
TR_SLimey
3bcb003248
Add IPv6 setting to nginx conf (#1582)
* Added IPv6 setting to nginx conf

https://matrix.to/#/!yomrOFwgFXzmeMAbzX:matrix.org/$sIkSxL002BKottY1Fop-bkusuyCJ1Y0zIFzqSWDYRrc?via=matrix.org&via=privacytools.io&via=dendrite.link

* Add IPv6 support to polylith nginx conf also
2020-11-18 10:33:17 +00:00
Petter Jakub Økland
fe339c8b98
Created polylith reverse proxy sample for Hiawatha (#1571) 2020-10-28 16:37:56 +00:00
Neil Alexander
0af35bec1a
Update documentation (#1569)
* Update dendrite-config.yaml

* Update README.md

* Update INSTALL.md

* Update INSTALL.md
2020-10-26 22:09:13 +00:00
Adit Sachde
b66b2eab32
Document dendrite server key format (#1520)
Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-22 12:21:31 +01:00
Neil Alexander
39c7a8915c
Multi-personality polylith binary (#1552)
* Initial work oon multipersonality binary

* Remove old binaries

* Monolith and polylith binaries

* Better logging

* dendrite-poly-multi

* Fix path

* Copyright notices etc

* Tweaks

* Update Docker, INSTALL.md

* Take first argument if flags package doesn't find any args

* Postgres 9.6 or later, fix some more Docker stuff

* Don't create unnecessary e2ekey DB

* Run go mod tidy
2020-10-20 16:11:24 +01:00
Neil Alexander
a71360d099
Update INSTALL.md (#1549) 2020-10-20 11:22:46 +01:00
Pieter
0804594a61
Add systemd example service to docs (#1512)
Signed-off-by: Pieter Hollander <git@pieterhollander.nl>
2020-10-12 10:06:56 +01:00
Kegsay
009401ad4d
Version 0.1.0
Beta README (#1466)

* Beta docs

* More tweaks

* More docs

* Update README.md (#1497)

* Call out missing features

* Add CHANGES

* Call out CHANGES

* Update INSTALL.md

* Update README.md

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-10-08 17:47:43 +01:00
Kegsay
bf7e85848b
Rename serverkeyapi to signingkeyserver (#1492)
* Rename serverkeyapi to signingkeyserver

We use "api" for public facing stuff and "server" for internal stuff.
As the server key API is internal only, we call it 'signing key server',
which also clarifies the type of key (as opposed to TLS keys, E2E keys, etc)

* Convert docker/scripts to use signing-key-server

* Rename missed bits
2020-10-07 16:23:18 +01:00
S7evinK
3bd66ff196
Route several paths to sync_api (#1473) 2020-10-02 22:35:54 +01:00
Neil Alexander
43b3c4a2fc
Fix bugs in nginx sample configs 2020-10-01 14:41:08 +01:00
Neil Alexander
cb4b93b16c
Add nginx sample reverse proxy configs (#1458)
* Add nginx sample reverse proxy configs

* Add line breaks at end of files
2020-10-01 14:33:48 +01:00
bn4t
4ff7ac7b65
Fix a small typo (#1448)
Signed-off-by: Benjamin Nater <me@bn4t.me>
2020-09-29 10:07:23 +01:00
Matthew Hodgson
39507bacc3
Peeking via MSC2753 (#1370)
Initial implementation of MSC2753, as tested by https://github.com/matrix-org/sytest/pull/944.
Doesn't yet handle unpeeks, peeked EDUs, or history viz changing during a peek - these will follow.
https://github.com/matrix-org/dendrite/pull/1370 has full details.
2020-09-10 14:39:18 +01:00
Kegsay
c992f4f1f4
Remove current state server (#1405)
* Remove current state server

Closes #1365 #1272 #1357

* Remove current state server from scripts/docs
2020-09-07 14:47:59 +01:00
Kegsay
421b6b2313
Add OpenTracing documentation (#1345)
* Add OpenTracing documentation

Closes #274

* Reshuffle

* Update setup.md
2020-08-27 10:24:56 +01:00
Kegsay
3b14119aff
Update README/CONTRIBUTING/sytest for AWSY issues (#1333)
* Update README/CONTRIBUTING/sytest for AWSY issues

* More docs
2020-08-25 08:11:41 +01:00
oliverpool
466d795375
Fix scripts path in documentation (#1294)
Broken by 24d8df664c

Signed-off-by: Olivier Charvin <git@olivier.pfad.fr>
2020-08-24 18:13:43 +01:00