mirror of
https://github.com/1f349/dendrite.git
synced 2024-11-25 13:11:37 +00:00
Update INSTALL.md, move docs (#1034)
* Update INSTALL.md * Move some things * Clean up * Move some more things * Don't build all the things for the monolith * Update INSTALL.md * Nuke hooks
This commit is contained in:
parent
1b34130a5b
commit
3cb04e8004
@ -1,18 +0,0 @@
|
|||||||
root = true
|
|
||||||
|
|
||||||
[*]
|
|
||||||
charset = utf-8
|
|
||||||
|
|
||||||
end_of_line = lf
|
|
||||||
insert_final_newline = true
|
|
||||||
trim_trailing_whitespace = true
|
|
||||||
|
|
||||||
[*.go]
|
|
||||||
indent_style = tab
|
|
||||||
indent_size = 4
|
|
||||||
|
|
||||||
[*.md]
|
|
||||||
trim_trailing_whitespace = false
|
|
||||||
|
|
||||||
[*.{yml,yaml}]
|
|
||||||
indent_style = space
|
|
@ -3,15 +3,15 @@
|
|||||||
Dendrite will be a second-generation Matrix homeserver written in Go.
|
Dendrite will be a second-generation Matrix homeserver written in Go.
|
||||||
|
|
||||||
It's still very much a work in progress, but installation instructions can be
|
It's still very much a work in progress, but installation instructions can be
|
||||||
found in [INSTALL.md](INSTALL.md). It is not recommended to use Dendrite as a
|
found in [INSTALL.md](docs/INSTALL.md). It is not recommended to use Dendrite as a
|
||||||
production homeserver at this time.
|
production homeserver at this time.
|
||||||
|
|
||||||
An overview of the design can be found in [DESIGN.md](DESIGN.md).
|
An overview of the design can be found in [DESIGN.md](docs/DESIGN.md).
|
||||||
|
|
||||||
# Contributing
|
# Contributing
|
||||||
|
|
||||||
Everyone is welcome to help out and contribute! See
|
Everyone is welcome to help out and contribute! See
|
||||||
[CONTRIBUTING.md](CONTRIBUTING.md) to get started!
|
[CONTRIBUTING.md](docs/CONTRIBUTING.md) to get started!
|
||||||
|
|
||||||
Please note that, as of February 2020, Dendrite now only targets Go 1.13 or
|
Please note that, as of February 2020, Dendrite now only targets Go 1.13 or
|
||||||
later. Please ensure that you are using at least Go 1.13 when developing for
|
later. Please ensure that you are using at least Go 1.13 when developing for
|
||||||
|
@ -2,38 +2,66 @@
|
|||||||
|
|
||||||
Dendrite can be run in one of two configurations:
|
Dendrite can be run in one of two configurations:
|
||||||
|
|
||||||
* A cluster of individual components, dealing with different aspects of the
|
* **Polylith mode**: A cluster of individual components, dealing with different
|
||||||
Matrix protocol (see [WIRING.md](./WIRING.md)). Components communicate with
|
aspects of the Matrix protocol (see [WIRING.md](WIRING.md)). Components communicate with each other using internal HTTP APIs and [Apache Kafka](https://kafka.apache.org). This will almost certainly be the preferred model
|
||||||
one another via [Apache Kafka](https://kafka.apache.org).
|
for large-scale deployments.
|
||||||
|
|
||||||
* A monolith server, in which all components run in the same process. In this
|
* **Monolith mode**: All components run in the same process. In this mode,
|
||||||
configuration, Kafka can be replaced with an in-process implementation
|
Kafka is completely optional and can instead be replaced with an in-process
|
||||||
called [naffka](https://github.com/matrix-org/naffka).
|
lightweight implementation called [Naffka](https://github.com/matrix-org/naffka). This will usually be the preferred model for low-volume, low-user
|
||||||
|
or experimental deployments.
|
||||||
|
|
||||||
|
Regardless of whether you are running in polylith or monolith mode, each Dendrite component that requires storage has its own database. Both Postgres
|
||||||
|
and SQLite are supported and can be mixed-and-matched across components as
|
||||||
|
needed in the configuration file.
|
||||||
|
|
||||||
|
Be advised that Dendrite is still developmental and it's not recommended for
|
||||||
|
use in production environments yet!
|
||||||
|
|
||||||
## Requirements
|
## Requirements
|
||||||
|
|
||||||
- Go 1.13+
|
Dendrite requires:
|
||||||
- Postgres 9.5+
|
|
||||||
- For Kafka (optional if using the monolith server):
|
|
||||||
- Unix-based system (https://kafka.apache.org/documentation/#os)
|
|
||||||
- JDK 1.8+ / OpenJDK 1.8+
|
|
||||||
- Apache Kafka 0.10.2+ (see [scripts/install-local-kafka.sh](scripts/install-local-kafka.sh) for up-to-date version numbers)
|
|
||||||
|
|
||||||
|
* Go 1.13 or higher
|
||||||
|
* Postgres 9.5 or higher (if using Postgres databases, not needed for SQLite)
|
||||||
|
|
||||||
## Setting up a development environment
|
If you want to run a polylith deployment, you also need:
|
||||||
|
|
||||||
Assumes Go 1.13+ and JDK 1.8+ are already installed and are on PATH.
|
* Apache Kafka 0.10.2+
|
||||||
|
|
||||||
|
## Building up a monolith deploment
|
||||||
|
|
||||||
|
Start by cloning the code:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Get the code
|
|
||||||
git clone https://github.com/matrix-org/dendrite
|
git clone https://github.com/matrix-org/dendrite
|
||||||
cd dendrite
|
cd dendrite
|
||||||
|
```
|
||||||
|
|
||||||
# Build it
|
Then build it:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go build -o bin/dendrite-monolith-server ./cmd/dendrite-monolith-server
|
||||||
|
go build -o bin/generate-keys ./cmd/generate-keys
|
||||||
|
```
|
||||||
|
|
||||||
|
## Building up a polylith deployment
|
||||||
|
|
||||||
|
Start by cloning the code:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/matrix-org/dendrite
|
||||||
|
cd dendrite
|
||||||
|
```
|
||||||
|
|
||||||
|
Then build it:
|
||||||
|
|
||||||
|
```bash
|
||||||
./build.sh
|
./build.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
If using Kafka, install and start it (c.f. [scripts/install-local-kafka.sh](scripts/install-local-kafka.sh)):
|
Install and start Kafka (c.f. [scripts/install-local-kafka.sh](scripts/install-local-kafka.sh)):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
KAFKA_URL=http://archive.apache.org/dist/kafka/2.1.0/kafka_2.11-2.1.0.tgz
|
KAFKA_URL=http://archive.apache.org/dist/kafka/2.1.0/kafka_2.11-2.1.0.tgz
|
||||||
|
|
||||||
@ -51,7 +79,7 @@ kafka/bin/zookeeper-server-start.sh -daemon kafka/config/zookeeper.properties
|
|||||||
kafka/bin/kafka-server-start.sh -daemon kafka/config/server.properties
|
kafka/bin/kafka-server-start.sh -daemon kafka/config/server.properties
|
||||||
```
|
```
|
||||||
|
|
||||||
On MacOS, you can use [homebrew](https://brew.sh/) for easier setup of kafka
|
On macOS, you can use [Homebrew](https://brew.sh/) for easier setup of Kafka:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
brew install kafka
|
brew install kafka
|
||||||
@ -61,15 +89,24 @@ brew services start kafka
|
|||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
|
### SQLite database setup
|
||||||
|
|
||||||
|
Dendrite can use the built-in SQLite database engine for small setups.
|
||||||
|
The SQLite databases do not need to be preconfigured - Dendrite will
|
||||||
|
create them automatically at startup.
|
||||||
|
|
||||||
### Postgres database setup
|
### Postgres database setup
|
||||||
|
|
||||||
Dendrite requires a postgres database engine, version 9.5 or later.
|
Assuming that Postgres 9.5 (or later) is installed:
|
||||||
|
|
||||||
|
* Create role, choosing a new password when prompted:
|
||||||
|
|
||||||
* Create role:
|
|
||||||
```bash
|
```bash
|
||||||
sudo -u postgres createuser -P dendrite # prompts for password
|
sudo -u postgres createuser -P dendrite
|
||||||
```
|
```
|
||||||
* Create databases:
|
|
||||||
|
* Create the component databases:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
for i in account device mediaapi syncapi roomserver serverkey federationsender publicroomsapi appservice naffka; do
|
for i in account device mediaapi syncapi roomserver serverkey federationsender publicroomsapi appservice naffka; do
|
||||||
sudo -u postgres createdb -O dendrite dendrite_$i
|
sudo -u postgres createdb -O dendrite dendrite_$i
|
||||||
@ -78,42 +115,56 @@ Dendrite requires a postgres database engine, version 9.5 or later.
|
|||||||
|
|
||||||
(On macOS, omit `sudo -u postgres` from the above commands.)
|
(On macOS, omit `sudo -u postgres` from the above commands.)
|
||||||
|
|
||||||
### Crypto key generation
|
### Server key generation
|
||||||
|
|
||||||
Generate the keys:
|
Each Dendrite server requires unique server keys.
|
||||||
|
|
||||||
|
Generate the self-signed SSL certificate for federation:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Generate a self-signed SSL cert for federation:
|
|
||||||
test -f server.key || openssl req -x509 -newkey rsa:4096 -keyout server.key -out server.crt -days 3650 -nodes -subj /CN=localhost
|
test -f server.key || openssl req -x509 -newkey rsa:4096 -keyout server.key -out server.crt -days 3650 -nodes -subj /CN=localhost
|
||||||
|
```
|
||||||
|
|
||||||
# generate ed25519 signing key
|
Generate the server signing key:
|
||||||
|
|
||||||
|
```
|
||||||
test -f matrix_key.pem || ./bin/generate-keys -private-key matrix_key.pem
|
test -f matrix_key.pem || ./bin/generate-keys -private-key matrix_key.pem
|
||||||
```
|
```
|
||||||
|
|
||||||
### Configuration
|
### Configuration file
|
||||||
|
|
||||||
Create config file, based on `dendrite-config.yaml`. Call it `dendrite.yaml`. Things that will need editing include *at least*:
|
Create config file, based on `dendrite-config.yaml`. Call it `dendrite.yaml`. Things that will need editing include *at least*:
|
||||||
* `server_name`
|
|
||||||
* `database/*` (All lines in the database section must have the username and password of the user created with the `createuser` command above. eg:`dendrite:password@localhost`)
|
|
||||||
|
|
||||||
|
* The `server_name` entry to reflect the hostname of your Dendrite server
|
||||||
|
* The `database` lines with an updated connection string based on your
|
||||||
|
desired setup, e.g. replacing `component` with the name of the component:
|
||||||
|
* For Postgres: `postgres://dendrite:password@localhost/component`
|
||||||
|
* For SQLite on disk: `file:component.db` or `file:///path/to/component.db`
|
||||||
|
* Postgres and SQLite can be mixed and matched.
|
||||||
|
* The `use_naffka` option if using Naffka in a monolith deployment
|
||||||
|
|
||||||
|
There are other options which may be useful so review them all. In particular,
|
||||||
|
if you are trying to federate from your Dendrite instance into public rooms
|
||||||
|
then configuring `key_perspectives` (like `matrix.org` in the sample) can
|
||||||
|
help to improve reliability considerably by allowing your homeserver to fetch
|
||||||
|
public keys for dead homeservers from somewhere else.
|
||||||
|
|
||||||
## Starting a monolith server
|
## Starting a monolith server
|
||||||
|
|
||||||
It is possible to use 'naffka' as an in-process replacement to Kafka when using
|
It is possible to use Naffka as an in-process replacement to Kafka when using
|
||||||
the monolith server. To do this, set `use_naffka: true` in `dendrite.yaml` and uncomment
|
the monolith server. To do this, set `use_naffka: true` in your `dendrite.yaml` configuration and uncomment the relevant Naffka line in the `database` section.
|
||||||
the necessary line related to naffka in the `database` section. Be sure to update the
|
Be sure to update the database username and password if needed.
|
||||||
database username and password if needed.
|
|
||||||
|
|
||||||
The monolith server can be started as shown below. By default it listens for
|
The monolith server can be started as shown below. By default it listens for
|
||||||
HTTP connections on port 8008, so point your client at
|
HTTP connections on port 8008, so you can configure your Matrix client to use
|
||||||
`http://localhost:8008`. If you set `--tls-cert` and `--tls-key` as shown
|
`http://localhost:8008` as the server. If you set `--tls-cert` and `--tls-key`
|
||||||
below, it will also listen for HTTPS connections on port 8448.
|
as shown below, it will also listen for HTTPS connections on port 8448.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./bin/dendrite-monolith-server --tls-cert=server.crt --tls-key=server.key
|
./bin/dendrite-monolith-server --tls-cert=server.crt --tls-key=server.key
|
||||||
```
|
```
|
||||||
|
|
||||||
## Starting a multiprocess server
|
## Starting a polylith deployment
|
||||||
|
|
||||||
The following contains scripts which will run all the required processes in order to point a Matrix client at Dendrite. Conceptually, you are wiring together to form the following diagram:
|
The following contains scripts which will run all the required processes in order to point a Matrix client at Dendrite. Conceptually, you are wiring together to form the following diagram:
|
||||||
|
|
||||||
@ -170,9 +221,10 @@ Servers --->| federation-api-proxy |--------->| dendrite-federation-api-server |
|
|||||||
A ==> B = Kafka (A = producer, B = consumer)
|
A ==> B = Kafka (A = producer, B = consumer)
|
||||||
```
|
```
|
||||||
|
|
||||||
### Run a client api proxy
|
### Client proxy
|
||||||
|
|
||||||
This is what Matrix clients will talk to. If you use the script below, point your client at `http://localhost:8008`.
|
This is what Matrix clients will talk to. If you use the script below, point
|
||||||
|
your client at `http://localhost:8008`.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./bin/client-api-proxy \
|
./bin/client-api-proxy \
|
||||||
@ -183,51 +235,10 @@ This is what Matrix clients will talk to. If you use the script below, point you
|
|||||||
--public-rooms-api-server-url "http://localhost:7775" \
|
--public-rooms-api-server-url "http://localhost:7775" \
|
||||||
```
|
```
|
||||||
|
|
||||||
### Run a client api
|
### Federation proxy
|
||||||
|
|
||||||
This is what implements message sending. Clients talk to this via the proxy in order to send messages.
|
This is what Matrix servers will talk to. This is only required if you want
|
||||||
|
to support federation.
|
||||||
```bash
|
|
||||||
./bin/dendrite-client-api-server --config=dendrite.yaml
|
|
||||||
```
|
|
||||||
|
|
||||||
(If this fails with `pq: syntax error at or near "ON"`, check you are using at least postgres 9.5.)
|
|
||||||
|
|
||||||
### Run a room server
|
|
||||||
|
|
||||||
This is what implements the room DAG. Clients do not talk to this.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./bin/dendrite-room-server --config=dendrite.yaml
|
|
||||||
```
|
|
||||||
|
|
||||||
### Run a sync server
|
|
||||||
|
|
||||||
This is what implements `/sync` requests. Clients talk to this via the proxy in order to receive messages.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./bin/dendrite-sync-api-server --config dendrite.yaml
|
|
||||||
```
|
|
||||||
|
|
||||||
### Run a media server
|
|
||||||
|
|
||||||
This implements `/media` requests. Clients talk to this via the proxy in order to upload and retrieve media.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./bin/dendrite-media-api-server --config dendrite.yaml
|
|
||||||
```
|
|
||||||
|
|
||||||
### Run public room server
|
|
||||||
|
|
||||||
This implements `/directory` requests. Clients talk to this via the proxy in order to retrieve room directory listings.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./bin/dendrite-public-rooms-api-server --config dendrite.yaml
|
|
||||||
```
|
|
||||||
|
|
||||||
### Run a federation api proxy
|
|
||||||
|
|
||||||
This is what Matrix servers will talk to. This is only required if you want to support federation.
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./bin/federation-api-proxy \
|
./bin/federation-api-proxy \
|
||||||
@ -236,7 +247,51 @@ This is what Matrix servers will talk to. This is only required if you want to s
|
|||||||
--media-api-server-url "http://localhost:7774" \
|
--media-api-server-url "http://localhost:7774" \
|
||||||
```
|
```
|
||||||
|
|
||||||
### Run a federation api server
|
### Client API server
|
||||||
|
|
||||||
|
This is what implements message sending. Clients talk to this via the proxy in
|
||||||
|
order to send messages.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/dendrite-client-api-server --config=dendrite.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
### Room server
|
||||||
|
|
||||||
|
This is what implements the room DAG. Clients do not talk to this.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/dendrite-room-server --config=dendrite.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
### Sync server
|
||||||
|
|
||||||
|
This is what implements `/sync` requests. Clients talk to this via the proxy
|
||||||
|
in order to receive messages.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/dendrite-sync-api-server --config dendrite.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
### Media server
|
||||||
|
|
||||||
|
This implements `/media` requests. Clients talk to this via the proxy in
|
||||||
|
order to upload and retrieve media.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/dendrite-media-api-server --config dendrite.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
### Public room server
|
||||||
|
|
||||||
|
This implements `/directory` requests. Clients talk to this via the proxy
|
||||||
|
in order to retrieve room directory listings.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/dendrite-public-rooms-api-server --config dendrite.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
### Federation API server
|
||||||
|
|
||||||
This implements federation requests. Servers talk to this via the proxy in
|
This implements federation requests. Servers talk to this via the proxy in
|
||||||
order to send transactions. This is only required if you want to support
|
order to send transactions. This is only required if you want to support
|
||||||
@ -246,7 +301,7 @@ federation.
|
|||||||
./bin/dendrite-federation-api-server --config dendrite.yaml
|
./bin/dendrite-federation-api-server --config dendrite.yaml
|
||||||
```
|
```
|
||||||
|
|
||||||
### Run a federation sender server
|
### Federation sender
|
||||||
|
|
||||||
This sends events from our users to other servers. This is only required if
|
This sends events from our users to other servers. This is only required if
|
||||||
you want to support federation.
|
you want to support federation.
|
||||||
@ -255,7 +310,7 @@ you want to support federation.
|
|||||||
./bin/dendrite-federation-sender-server --config dendrite.yaml
|
./bin/dendrite-federation-sender-server --config dendrite.yaml
|
||||||
```
|
```
|
||||||
|
|
||||||
### Run an appservice server
|
### Appservice server
|
||||||
|
|
||||||
This sends events from the network to [application
|
This sends events from the network to [application
|
||||||
services](https://matrix.org/docs/spec/application_service/unstable.html)
|
services](https://matrix.org/docs/spec/application_service/unstable.html)
|
||||||
@ -265,3 +320,12 @@ application services on your homeserver.
|
|||||||
```bash
|
```bash
|
||||||
./bin/dendrite-appservice-server --config dendrite.yaml
|
./bin/dendrite-appservice-server --config dendrite.yaml
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Key server
|
||||||
|
|
||||||
|
This manages end-to-end encryption keys (or rather, it will do when it's
|
||||||
|
finished).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/dendrite-key-server --config dendrite.yaml
|
||||||
|
```
|
@ -1,5 +0,0 @@
|
|||||||
#! /bin/bash
|
|
||||||
|
|
||||||
DOT_GIT="$(dirname $0)/../.git"
|
|
||||||
|
|
||||||
ln -s "../../hooks/pre-commit" "$DOT_GIT/hooks/pre-commit"
|
|
@ -1,22 +0,0 @@
|
|||||||
#! /bin/bash
|
|
||||||
|
|
||||||
set -eu
|
|
||||||
|
|
||||||
# make the GIT_DIR and GIT_INDEX_FILE absolute, before we change dir
|
|
||||||
export GIT_DIR=$(readlink -f `git rev-parse --git-dir`)
|
|
||||||
if [ -n "${GIT_INDEX_FILE:+x}" ]; then
|
|
||||||
export GIT_INDEX_FILE=$(readlink -f "$GIT_INDEX_FILE")
|
|
||||||
fi
|
|
||||||
|
|
||||||
# create a temp dir. The `trap` incantation will ensure that it is removed
|
|
||||||
# again when this script completes.
|
|
||||||
tmpdir=`mktemp -d`
|
|
||||||
trap 'rm -rf "$tmpdir"' EXIT
|
|
||||||
cd "$tmpdir"
|
|
||||||
|
|
||||||
# get a clean copy of the index (ie, what has been `git add`ed), so that we can
|
|
||||||
# run the checks against what we are about to commit, rather than what is in
|
|
||||||
# the working copy.
|
|
||||||
git checkout-index -a
|
|
||||||
|
|
||||||
./scripts/find-lint.sh fast
|
|
Loading…
Reference in New Issue
Block a user