mirror of
https://github.com/1f349/dendrite.git
synced 2024-11-08 18:16:59 +00:00
parent
72b3160776
commit
37db60f4d4
68
README.md
68
README.md
@ -1,22 +1,68 @@
|
||||
# Dendrite [![Build Status](https://badge.buildkite.com/4be40938ab19f2bbc4a6c6724517353ee3ec1422e279faf374.svg?branch=master)](https://buildkite.com/matrix-dot-org/dendrite) [![Dendrite Dev on Matrix](https://img.shields.io/matrix/dendrite-dev:matrix.org.svg?label=%23dendrite-dev%3Amatrix.org&logo=matrix&server_fqdn=matrix.org)](https://matrix.to/#/#dendrite-dev:matrix.org) [![Dendrite on Matrix](https://img.shields.io/matrix/dendrite:matrix.org.svg?label=%23dendrite%3Amatrix.org&logo=matrix&server_fqdn=matrix.org)](https://matrix.to/#/#dendrite:matrix.org)
|
||||
|
||||
Dendrite will be a second-generation Matrix homeserver written in Go.
|
||||
Dendrite is a second-generation Matrix homeserver written in Go. It is not recommended to use Dendrite as
|
||||
a production homeserver at this time as there is no stable release. An overview of the design can be found
|
||||
in [DESIGN.md](docs/DESIGN.md).
|
||||
|
||||
It's still very much a work in progress, but installation instructions can be
|
||||
found in [INSTALL.md](docs/INSTALL.md). It is not recommended to use Dendrite as a
|
||||
production homeserver at this time.
|
||||
# Quick start
|
||||
|
||||
Requires Go 1.13+ and SQLite3 (Postgres is also supported):
|
||||
|
||||
```bash
|
||||
$ git clone https://github.com/matrix-org/dendrite
|
||||
$ cd dendrite
|
||||
|
||||
# generate self-signed certificate and an event signing key for federation
|
||||
$ go build ./cmd/generate-keys
|
||||
$ ./generate-keys --private-key matrix_key.pem --tls-cert server.crt --tls-key server.key
|
||||
|
||||
# Copy and modify the config file:
|
||||
# you'll need to set a server name and paths to the keys at the very least, along with setting
|
||||
# up the database filenames
|
||||
$ cp dendrite-config.yaml dendrite.yaml
|
||||
|
||||
# build and run the server
|
||||
$ go build ./cmd/dendrite-monolith-server
|
||||
$ ./dendrite-monolith-server --tls-cert server.crt --tls-key server.key --config dendrite.yaml
|
||||
```
|
||||
|
||||
Then point your favourite Matrix client at `http://localhost:8008`. For full installation information, see
|
||||
[INSTALL.md](docs/INSTALL.md). For running in Docker, see [build/docker](build/docker).
|
||||
|
||||
# Progress
|
||||
|
||||
We use a script called Are We Synapse Yet which checks Sytest compliance rates. Sytest is a black-box homeserver
|
||||
test rig with around 900 tests. The script works out how many of these tests are passing on Dendrite and it
|
||||
updates with CI. As of July 2020 we're at around 48% CS API coverage and 50% Federation coverage, though check
|
||||
CI for the latest numbers. In practice, this means you can communicate locally and via federation with Synapse
|
||||
servers such as matrix.org reasonably well. There's a long list of features that are not implemented, notably:
|
||||
- Receipts
|
||||
- Push
|
||||
- Search and Context
|
||||
- User Directory
|
||||
- Presence
|
||||
- Guests
|
||||
- E2E keys and device lists
|
||||
|
||||
We are prioritising features that will benefit single-user homeservers first (e.g Receipts, E2E) rather
|
||||
than features that massive deployments may be interested in (User Directory, OpenID, Guests, Admin APIs, AS API).
|
||||
This means Dendrite supports amongst others:
|
||||
- Core room functionality (creating rooms, invites, auth rules)
|
||||
- Federation in rooms v1-v6
|
||||
- Backfilling locally and via federation
|
||||
- Accounts, Profiles and Devices
|
||||
- Published room lists
|
||||
- Typing
|
||||
- Media APIs
|
||||
- Redaction
|
||||
- Tagging
|
||||
|
||||
An overview of the design can be found in [DESIGN.md](docs/DESIGN.md).
|
||||
|
||||
# Contributing
|
||||
|
||||
Everyone is welcome to help out and contribute! See
|
||||
[CONTRIBUTING.md](docs/CONTRIBUTING.md) to get started!
|
||||
|
||||
Please note that, as of February 2020, Dendrite now only targets Go 1.13 or
|
||||
later. Please ensure that you are using at least Go 1.13 when developing for
|
||||
Dendrite.
|
||||
|
||||
# Discussion
|
||||
|
||||
For questions about Dendrite we have a dedicated room on Matrix
|
||||
@ -24,8 +70,4 @@ For questions about Dendrite we have a dedicated room on Matrix
|
||||
discussion should happen in
|
||||
[#dendrite-dev:matrix.org](https://matrix.to/#/#dendrite-dev:matrix.org).
|
||||
|
||||
# Progress
|
||||
|
||||
There's plenty still to do to make Dendrite usable! We're tracking progress in a
|
||||
[project board](https://github.com/matrix-org/dendrite/projects/2).
|
||||
|
||||
|
@ -1,12 +1,7 @@
|
||||
# Code Style
|
||||
|
||||
We follow the standard Go style using goimports, but with a few extra
|
||||
considerations.
|
||||
|
||||
## Linters
|
||||
|
||||
We use `golangci-lint` to run a number of linters, the exact list can be found
|
||||
under linters in [.golangci.yml](.golangci.yml).
|
||||
In addition to standard Go code style (`gofmt`, `goimports`), we use `golangci-lint`
|
||||
to run a number of linters, the exact list can be found under linters in [.golangci.yml](.golangci.yml).
|
||||
[Installation](https://github.com/golangci/golangci-lint#install) and [Editor
|
||||
Integration](https://github.com/golangci/golangci-lint#editor-integration) for
|
||||
it can be found in the readme of golangci-lint.
|
||||
@ -22,37 +17,10 @@ The linters can be run using [scripts/find-lint.sh](scripts/find-lint.sh)
|
||||
[scripts/build-test-lint.sh](scripts/build-test-lint.sh).
|
||||
|
||||
|
||||
## HTTP Error Handling
|
||||
|
||||
Unfortunately, converting errors into HTTP responses with the correct status
|
||||
code and message can be done in a number of ways in golang:
|
||||
|
||||
1. Having functions return `JSONResponse` directly, which can then either set
|
||||
it to an error response or a `200 OK`.
|
||||
2. Have the HTTP handler try and cast error values to types that are handled
|
||||
differently.
|
||||
3. Have the HTTP handler call functions whose errors can only be interpreted
|
||||
one way, for example if a `validate(...)` call returns an error then handler
|
||||
knows to respond with a `400 Bad Request`.
|
||||
|
||||
We attempt to always use option #3, as it more naturally fits with the way that
|
||||
golang generally does error handling. In particular, option #1 effectively
|
||||
requires reinventing a new error handling scheme just for HTTP handlers.
|
||||
|
||||
|
||||
## Line length
|
||||
|
||||
We strive for a line length of roughly 80 characters, though less than 100 is
|
||||
acceptable if necessary. Longer lines are fine if there is nothing of interest
|
||||
after the first 80-100 characters (e.g. long string literals).
|
||||
|
||||
|
||||
## TODOs and FIXMEs
|
||||
|
||||
The majority of TODOs and FIXMEs should have an associated tracking issue on
|
||||
github. These can be added just before merging of the PR to master, and the
|
||||
issue number should be added to the comment, e.g. `// TODO(#324): ...`
|
||||
## Labels
|
||||
|
||||
In addition to `TODO` and `FIXME` we also use `NOTSPEC` to identify deviations
|
||||
from the Matrix specification.
|
||||
|
||||
## Logging
|
||||
|
||||
|
151
docs/INSTALL.md
151
docs/INSTALL.md
@ -3,20 +3,21 @@
|
||||
Dendrite can be run in one of two configurations:
|
||||
|
||||
* **Polylith mode**: A cluster of individual components, dealing with different
|
||||
aspects of the Matrix protocol (see [WIRING.md](WIRING.md)). Components communicate with each other using internal HTTP APIs and [Apache Kafka](https://kafka.apache.org). This will almost certainly be the preferred model
|
||||
for large-scale deployments.
|
||||
aspects of the Matrix protocol (see [WIRING.md](WIRING-Current.md)). Components communicate
|
||||
with each other using internal HTTP APIs and [Apache Kafka](https://kafka.apache.org).
|
||||
This will almost certainly be the preferred model for large-scale deployments.
|
||||
|
||||
* **Monolith mode**: All components run in the same process. In this mode,
|
||||
Kafka is completely optional and can instead be replaced with an in-process
|
||||
lightweight implementation called [Naffka](https://github.com/matrix-org/naffka). This will usually be the preferred model for low-volume, low-user
|
||||
or experimental deployments.
|
||||
lightweight implementation called [Naffka](https://github.com/matrix-org/naffka). This
|
||||
will usually be the preferred model for low-volume, low-user or experimental deployments.
|
||||
|
||||
Regardless of whether you are running in polylith or monolith mode, each Dendrite component that requires storage has its own database. Both Postgres
|
||||
and SQLite are supported and can be mixed-and-matched across components as
|
||||
needed in the configuration file.
|
||||
Regardless of whether you are running in polylith or monolith mode, each Dendrite component that
|
||||
requires storage has its own database. Both Postgres and SQLite are supported and can be
|
||||
mixed-and-matched across components as needed in the configuration file.
|
||||
|
||||
Be advised that Dendrite is still developmental and it's not recommended for
|
||||
use in production environments yet!
|
||||
Be advised that Dendrite is still in development and it's not recommended for
|
||||
use in production environments just yet!
|
||||
|
||||
## Requirements
|
||||
|
||||
@ -119,16 +120,10 @@ Assuming that Postgres 9.5 (or later) is installed:
|
||||
|
||||
Each Dendrite server requires unique server keys.
|
||||
|
||||
Generate the self-signed SSL certificate for federation:
|
||||
Generate the self-signed SSL certificate for federation and the server signing key:
|
||||
|
||||
```bash
|
||||
test -f server.key || openssl req -x509 -newkey rsa:4096 -keyout server.key -out server.crt -days 3650 -nodes -subj /CN=localhost
|
||||
```
|
||||
|
||||
Generate the server signing key:
|
||||
|
||||
```
|
||||
test -f matrix_key.pem || ./bin/generate-keys -private-key matrix_key.pem
|
||||
./bin/generate-keys --private-key matrix_key.pem --tls-cert server.crt --tls-key server.key
|
||||
```
|
||||
|
||||
### Configuration file
|
||||
@ -152,7 +147,8 @@ public keys for dead homeservers from somewhere else.
|
||||
## Starting a monolith server
|
||||
|
||||
It is possible to use Naffka as an in-process replacement to Kafka when using
|
||||
the monolith server. To do this, set `use_naffka: true` in your `dendrite.yaml` configuration and uncomment the relevant Naffka line in the `database` section.
|
||||
the monolith server. To do this, set `use_naffka: true` in your `dendrite.yaml`
|
||||
configuration and uncomment the relevant Naffka line in the `database` section.
|
||||
Be sure to update the database username and password if needed.
|
||||
|
||||
The monolith server can be started as shown below. By default it listens for
|
||||
@ -166,60 +162,7 @@ as shown below, it will also listen for HTTPS connections on port 8448.
|
||||
|
||||
## Starting a polylith deployment
|
||||
|
||||
The following contains scripts which will run all the required processes in order to point a Matrix client at Dendrite. Conceptually, you are wiring together to form the following diagram:
|
||||
|
||||
```
|
||||
|
||||
/media +---------------------------+
|
||||
+----------->+------------->| dendrite-media-api-server |
|
||||
^ ^ +---------------------------+
|
||||
| | :7774
|
||||
| |
|
||||
| |
|
||||
| |
|
||||
| |
|
||||
| |
|
||||
| |
|
||||
| |
|
||||
| |
|
||||
| | /sync +--------------------------+
|
||||
| | +--------->| dendrite-sync-api-server |<================++
|
||||
| | | +--------------------------+ ||
|
||||
| | | :7773 | ^^ ||
|
||||
Matrix +------------------+ | | | || client_data ||
|
||||
Clients --->| client-api-proxy |-------+ +<-----------+ ++=============++ ||
|
||||
+------------------+ | | | || ||
|
||||
:8008 | | CS API +----------------------------+ || ||
|
||||
| +--------->| dendrite-client-api-server |==++ ||
|
||||
| | +----------------------------+ ||
|
||||
| | :7771 | ||
|
||||
| | | ||
|
||||
| +<-----------+ ||
|
||||
| | ||
|
||||
| | ||
|
||||
| | +----------------------+ room_event ||
|
||||
| +---------->| dendrite-room-server |===============++
|
||||
| | +----------------------+ ||
|
||||
| | :7770 ||
|
||||
| | ++==========================++
|
||||
| +<------------+ ||
|
||||
| | | VV
|
||||
| | +-----------------------------------+ Matrix
|
||||
| | | dendrite-federation-sender-server |------------> Servers
|
||||
| | +-----------------------------------+
|
||||
| | :7776
|
||||
| |
|
||||
+---------->+ +<-----------+
|
||||
| |
|
||||
Matrix +----------------------+ SS API +--------------------------------+
|
||||
Servers --->| federation-api-proxy |--------->| dendrite-federation-api-server |
|
||||
+----------------------+ +--------------------------------+
|
||||
:8448 :7772
|
||||
|
||||
|
||||
A --> B = HTTP requests (A = client, B = server)
|
||||
A ==> B = Kafka (A = producer, B = consumer)
|
||||
```
|
||||
The following contains scripts which will run all the required processes in order to point a Matrix client at Dendrite.
|
||||
|
||||
### Client proxy
|
||||
|
||||
@ -248,21 +191,13 @@ to support federation.
|
||||
|
||||
### Client API server
|
||||
|
||||
This is what implements message sending. Clients talk to this via the proxy in
|
||||
order to send messages.
|
||||
This is what implements CS API endpoints. Clients talk to this via the proxy in
|
||||
order to send messages, create and join rooms, etc.
|
||||
|
||||
```bash
|
||||
./bin/dendrite-client-api-server --config=dendrite.yaml
|
||||
```
|
||||
|
||||
### Room server
|
||||
|
||||
This is what implements the room DAG. Clients do not talk to this.
|
||||
|
||||
```bash
|
||||
./bin/dendrite-room-server --config=dendrite.yaml
|
||||
```
|
||||
|
||||
### Sync server
|
||||
|
||||
This is what implements `/sync` requests. Clients talk to this via the proxy
|
||||
@ -283,7 +218,7 @@ order to upload and retrieve media.
|
||||
|
||||
### Federation API server
|
||||
|
||||
This implements federation requests. Servers talk to this via the proxy in
|
||||
This implements the federation API. Servers talk to this via the proxy in
|
||||
order to send transactions. This is only required if you want to support
|
||||
federation.
|
||||
|
||||
@ -291,7 +226,30 @@ federation.
|
||||
./bin/dendrite-federation-api-server --config dendrite.yaml
|
||||
```
|
||||
|
||||
### Federation sender
|
||||
### Internal components
|
||||
|
||||
This refers to components that are not directly spoken to by clients. They are only
|
||||
contacted by other components. This includes the following components.
|
||||
|
||||
#### Room server
|
||||
|
||||
This is what implements the room DAG. Clients do not talk to this.
|
||||
|
||||
```bash
|
||||
./bin/dendrite-room-server --config=dendrite.yaml
|
||||
```
|
||||
|
||||
#### Current state server
|
||||
|
||||
This tracks the current state of rooms which various components need to know. For example,
|
||||
`/publicRooms` implemented by client API asks this server for the room names, joined member
|
||||
counts, etc.
|
||||
|
||||
```bash
|
||||
./bin/dendrite-current-state-server --config=dendrite.yaml
|
||||
```
|
||||
|
||||
#### Federation sender
|
||||
|
||||
This sends events from our users to other servers. This is only required if
|
||||
you want to support federation.
|
||||
@ -300,7 +258,7 @@ you want to support federation.
|
||||
./bin/dendrite-federation-sender-server --config dendrite.yaml
|
||||
```
|
||||
|
||||
### Appservice server
|
||||
#### Appservice server
|
||||
|
||||
This sends events from the network to [application
|
||||
services](https://matrix.org/docs/spec/application_service/unstable.html)
|
||||
@ -311,16 +269,31 @@ application services on your homeserver.
|
||||
./bin/dendrite-appservice-server --config dendrite.yaml
|
||||
```
|
||||
|
||||
### Key server
|
||||
#### Key server
|
||||
|
||||
This manages end-to-end encryption keys (or rather, it will do when it's
|
||||
finished).
|
||||
This manages end-to-end encryption keys for users.
|
||||
|
||||
```bash
|
||||
./bin/dendrite-key-server --config dendrite.yaml
|
||||
```
|
||||
|
||||
### User server
|
||||
#### Server Key server
|
||||
|
||||
This manages signing keys for servers.
|
||||
|
||||
```bash
|
||||
./bin/dendrite-server-key-api-server --config dendrite.yaml
|
||||
```
|
||||
|
||||
#### EDU server
|
||||
|
||||
This manages processing EDUs such as typing, send-to-device events and presence. Clients do not talk to
|
||||
|
||||
```bash
|
||||
./bin/dendrite-edu-server --config dendrite.yaml
|
||||
```
|
||||
|
||||
#### User server
|
||||
|
||||
This manages user accounts, device access tokens and user account data,
|
||||
amongst other things.
|
||||
|
Loading…
Reference in New Issue
Block a user