Dendrite is a second-generation Matrix homeserver written in Go!
Go to file
Kegsay a97b8eafd4
Add peer-to-peer support into Dendrite via libp2p and fetch (#880)
* Use a fork of pq which supports userCurrent on wasm

* Use sqlite3_js driver when running in JS

* Add cmd/dendritejs to pull in sqlite3_js driver for wasm only

* Update to latest go-sqlite-js version

* Replace prometheus with a stub. sigh

* Hard-code a config and don't use opentracing

* Latest go-sqlite3-js version

* Generate a key for now

* Listen for fetch traffic rather than HTTP

* Latest hacks for js

* libp2p support

* More libp2p

* Fork gjson to allow us to enforce auth checks as before

Previously, all events would come down redacted because the hash
checks would fail. They would fail because sjson.DeleteBytes didn't
remove keys not used for hashing. This didn't work because of a build
tag which included a file which no-oped the index returned.

See https://github.com/tidwall/gjson/issues/157

When it's resolved, let's go back to mainline.

* Use gjson@1.6.0 as it fixes https://github.com/tidwall/gjson/issues/157

* Use latest gomatrixserverlib for sig checks

* Fix a bug which could cause exclude_from_sync to not be set

Caused when sending events over federation.

* Use query variadic to make lookups actually work!

* Latest gomatrixserverlib

* Add notes on getting p2p up and running

Partly so I don't forget myself!

* refactor: Move p2p specific stuff to cmd/dendritejs

This is important or else the normal build of dendrite will fail
because the p2p libraries depend on syscall/js which doesn't work
on normal builds.

Also, clean up main.go to read a bit better.

* Update ho-http-js-libp2p to return errors from RoundTrip

* Add an LRU cache around the key DB

We actually need this for P2P because otherwise we can *segfault*
with things like: "runtime: unexpected return pc for runtime.handleEvent"
where the event is a `syscall/js` event, caused by spamming sql.js
caused by "Checking event signatures for 14 events of room state" which
hammers the key DB repeatedly in quick succession.

Using a cache fixes this, though the underlying cause is probably a bug
in the version of Go I'm on (1.13.7)

* breaking: Add Tracing.Enabled to toggle whether we do opentracing

Defaults to false, which is why this is a breaking change. We need
this flag because WASM builds cannot do opentracing.

* Start adding conditional builds for wasm to handle lib/pq

The general idea here is to have the wasm build have a `NewXXXDatabase`
that doesn't import any postgres package and hence we never import
`lib/pq`, which doesn't work under WASM (undefined `userCurrent`).

* Remove lib/pq for wasm for syncapi

* Add conditional building to remaining storage APIs

* Update build script to set env vars correctly for dendritejs

* sqlite bug fixes

* Docs

* Add a no-op main for dendritejs when not building under wasm

* Use the real prometheus, even for WASM

Instead, the dendrite-sw.js must mock out `process.pid` and
`fs.stat` - which must invoke the callback with an error (e.g `EINVAL`)
in order for it to work:

```
    global.process = {
        pid: 1,
    };
    global.fs.stat = function(path, cb) {
        cb({
            code: "EINVAL",
        });
    }
```

* Linting
2020-03-06 10:23:55 +00:00
.github Add SyTest support to CI (#704) 2019-05-24 15:00:03 +01:00
appservice Add peer-to-peer support into Dendrite via libp2p and fetch (#880) 2020-03-06 10:23:55 +00:00
clientapi Add peer-to-peer support into Dendrite via libp2p and fetch (#880) 2020-03-06 10:23:55 +00:00
cmd Add peer-to-peer support into Dendrite via libp2p and fetch (#880) 2020-03-06 10:23:55 +00:00
common Add peer-to-peer support into Dendrite via libp2p and fetch (#880) 2020-03-06 10:23:55 +00:00
docker Fix dockerfile 2020-02-20 13:54:50 +00:00
docs Adding sslmode: disable to sytest server config (#813) 2020-02-11 13:53:00 +00:00
federationapi Remove httputil.LogThenError so that the line numbers are reported properly - make error reporting slightly more useful (#879) 2020-03-02 16:20:44 +00:00
federationsender Add peer-to-peer support into Dendrite via libp2p and fetch (#880) 2020-03-06 10:23:55 +00:00
hooks run commit hook on what we're committing (#356) 2017-11-27 12:19:04 +00:00
mediaapi Add peer-to-peer support into Dendrite via libp2p and fetch (#880) 2020-03-06 10:23:55 +00:00
publicroomsapi Add peer-to-peer support into Dendrite via libp2p and fetch (#880) 2020-03-06 10:23:55 +00:00
roomserver Add peer-to-peer support into Dendrite via libp2p and fetch (#880) 2020-03-06 10:23:55 +00:00
scripts Move files back if linting fails (#810) 2019-12-25 18:14:58 +00:00
syncapi Add peer-to-peer support into Dendrite via libp2p and fetch (#880) 2020-03-06 10:23:55 +00:00
typingserver Lots of small typo fixes (#737) 2019-07-12 16:43:01 +01:00
.editorconfig Add .editorconfig (#179) 2017-08-05 02:25:40 +01:00
.gitignore use go module for dependencies (#594) 2019-05-21 21:56:55 +01:00
.golangci.yml Fix pipeline, emoji and syntax (#713) 2019-06-19 14:05:03 +01:00
build.sh Add peer-to-peer support into Dendrite via libp2p and fetch (#880) 2020-03-06 10:23:55 +00:00
CODE_STYLE.md replaced gometalinter description with golangci-lint (#837) 2019-12-25 18:28:10 +00:00
CONTRIBUTING.md Update documentation for Go 1.13 (#867) 2020-02-06 11:54:26 +00:00
dendrite-config.yaml Remove riot.im from the list of trusted ID servers in the default config (#688) 2019-01-31 10:06:11 +00:00
DESIGN.md Clean up README a bit (#289) 2017-10-10 09:33:41 +01:00
go.mod Add peer-to-peer support into Dendrite via libp2p and fetch (#880) 2020-03-06 10:23:55 +00:00
go.sum Add peer-to-peer support into Dendrite via libp2p and fetch (#880) 2020-03-06 10:23:55 +00:00
INSTALL.md Update documentation for Go 1.13 (#867) 2020-02-06 11:54:26 +00:00
LICENSE Add Apache Version 2.0 license and headers to all golang files 2017-04-21 00:40:52 +02:00
p2p.md Add peer-to-peer support into Dendrite via libp2p and fetch (#880) 2020-03-06 10:23:55 +00:00
README.md Kick CI 2020-02-28 14:46:03 +00:00
show-expected-fail-tests.sh Add whitelist/blacklist maintenance complainer to buildkite annotations (#870) 2020-02-14 13:07:14 +00:00
sytest-blacklist Implement missing device management features (#835) 2020-02-11 12:13:38 +00:00
sytest-whitelist Add whitelist/blacklist maintenance complainer to buildkite annotations (#870) 2020-02-14 13:07:14 +00:00
WIRING.md Document internal APIs and Application Services component (#499) 2018-07-06 02:33:53 -07:00

Dendrite Build Status Dendrite Dev on Matrix Dendrite on Matrix

Dendrite will be a second-generation Matrix homeserver written in Go.

It's still very much a work in progress, but installation instructions can be found in INSTALL.md. It is not recommended to use Dendrite as a production homeserver at this time.

An overview of the design can be found in DESIGN.md.

Contributing

Everyone is welcome to help out and contribute! See CONTRIBUTING.md to get started!

Please note that, as of February 2020, Dendrite now only targets Go 1.13 or later. Please ensure that you are using at least Go 1.13 when developing for Dendrite.

Discussion

For questions about Dendrite we have a dedicated room on Matrix #dendrite:matrix.org. Development discussion should happen in #dendrite-dev:matrix.org.

Progress

There's plenty still to do to make Dendrite usable! We're tracking progress in a project board.