Commit Graph

2787 Commits

Author SHA1 Message Date
genofire
8223e1f2e1
fix(helm): improve documentation and grafana dashboard (#2992)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

---

* **docs**: enabling of metrics in there config is needed
* **dashboard**: since the imported dashboard was created:
  * many metrics was dropped by dendrite (mainly #2967)
  * grafana has new version of diagrams ...

---------

Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>

[skip ci]
2023-04-03 09:04:59 +02:00
Rhea Danzey
01dd02dad2
chart - Add configuration for extra volumes / volume mounts (#3042)
Adds configuration for additional volumes / volumeMounts to the Dendrite
pod to inject configuration / secrets outside of the chart's templates

### Pull Request Checklist

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests - Helm chart changes
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: Rhea Danzey <rdanzey@element.io>

---------

Signed-off-by: Rhea Danzey <rdanzey@element.io>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>

[skip ci]
2023-04-03 09:00:32 +02:00
George Antoniadis
44ed0a3279
add deployment strategy option to helm chart (#3021)
@S7evinK minor update to the helm chart on top of you existing fixes to
allow setting the update strategy as the default `RollingUpdate` one is
a bit annoying if using `ReadWriteOnce` volumes for media. Hope this
makes sense.

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] ~~I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests~~ Haven't touched any go files.
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `George Antoniadis <george@noodles.gr>`
[skip ci]
2023-04-03 08:24:47 +02:00
Till
2854ffeb7d
Add CS API device tests (#3029)
Adds tests for
- `/devices`
- `/delete_devices` (also adds UIA)
2023-03-31 10:15:01 +02:00
Rhea Danzey
28d3e296a8
Rdanzey/helm-fixes-existing-db-secrets (#3033)
Fixes some Helm templating issues when setting up a deployment with an
existing database / signing keys.

- Allows for `.Values.postgresql.enabled: false` as long as
`.Values.global.dendrite_config.database.connection_string` is defined
- Allows for '.Values.signing_key.create: false' if
`.Values.signing_key.existingSecret` is set

Also fixes an error in the template resulting in profiling port not
being set correctly:

```
Error: template: dendrite-meta/charts/dendrite/templates/deployment.yaml:60:35: executing "dendrite-meta/charts/dendrite/templates/deployment.yaml" at <$.Values.global.profiling.port>: nil pointer evaluating interface {}.port
```

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
  - Helm template fixes, no golang changes
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: Rhea Danzey <rdanzey@element.io>

---------

Signed-off-by: Rhea Danzey <rdanzey@element.io>
Co-authored-by: Till Faelligen <2353100+S7evinK@users.noreply.github.com>
2023-03-28 08:30:19 +02:00
Devon Hudson
f4104b4b5d
Pinecone-demo: Wait on dendrite before shutting down 2023-03-27 17:19:53 -06:00
Devon Hudson
69e3bd82a9
Add dendrite-demo-pinecone cypress tests 2023-03-27 07:57:30 -06:00
Till
fa7710315a
Add tests for the Dendrite admin APIs (#3028)
Contains a breaking change, since the endpoints `/_dendrite/admin/evacuateRoom/{roomID}` and `/_dendrite/admin/evacuateUser/{userID}` are now using `POST` instead of `GET`
2023-03-27 15:39:33 +02:00
Till
e8b2162a01
Add /search tests (#3025) 2023-03-27 11:26:52 +02:00
Till
aa1bda4c58
Add AS invite test, fix issue with invitations being processed twice (#3020)
The AS roomserver consumer would receive the events twice, one time as
type `OutputTypeNewInviteEvent` and the other time as
`OutputTypeNewRoomEvent`. 

[skip ci]
2023-03-27 11:26:26 +02:00
Till Faelligen
e2d2482ca6
Get the logs for dendrite when installing the chart 2023-03-27 11:07:30 +02:00
Alex Kirk
05f72fc4be
Update docs and sample config for the relay_api (#3011)
This adds an empty `relay_api` section to the sample configuration. For
SQLite environments, or others where a `database.connection_string` is
needed for each section, there should be an entry in the configuration
sample as a basis.

This PR also changes the "Configuring Dendrite" documentation in that
respect.

The requirement was introduced in #2917. When upgrading dendrite, it
will complain about `relay_api.database.connection_string` not being
configured.

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Alex Kirk <akirk@users.noreply.github.com>`
2023-03-27 10:55:36 +02:00
Till
234ed603e6
Move every db.Prepare to sqlutil.Statementlist, remove trace driver (#3026)
Doesn't buy us much, but makes everything a bit more consistent.

Also removes the SQL trace driver, as it is unused and the output is
hard to read anyway.
2023-03-23 13:52:53 +01:00
Till Faelligen
cb18ba0230
Upload covdatafiles for each server 2023-03-22 17:36:33 +01:00
Till Faelligen
14085d30ac
Update workflow to not use commas when joining names 2023-03-22 16:01:12 +01:00
Till Faelligen
a4400bdd76
Sytest coverage file 2023-03-22 14:58:36 +01:00
Till Faelligen
b741d38e10
Update Workflow 2023-03-22 14:51:18 +01:00
Till Faelligen
6948d16527
Update Go, use go tool covdata for coverage files? 2023-03-22 14:50:21 +01:00
Till
5e85a00cb3
Remove BaseDendrite (#3023)
Removes `BaseDendrite` to, hopefully, make testing and composing of
components easier in the future.
2023-03-22 09:21:32 +01:00
Till Faelligen
ec6879e5ae
Update GMSL to fix #3013 2023-03-21 16:04:51 +01:00
Till Faelligen
0459d2b9e5
Make "m.upload.size" optional 2023-03-20 09:24:00 +01:00
Till
5579121c6f
Preparations for removing BaseDendrite (#3016)
Preparations to actually remove/replace `BaseDendrite`.
Quite a few changes:
- SyncAPI accepts an `fulltext.Indexer` interface (fulltext is removed
from `BaseDendrite`)
- Caches are removed from `BaseDendrite`
- Introduces a `Router` struct (likely to change)
  - also fixes #2903
- Introduces a `sqlutil.ConnectionManager`, which should remove
`base.DatabaseConnection` later on
- probably more
2023-03-17 11:09:45 +00:00
Boris Rybalkin
d88f71ab71
simplify unix socket permission format (#3014)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Boris Rybalkin <ribalkin@gmail.com>`
2023-03-16 08:51:21 +01:00
Till Faelligen
2c58bab6a8
Fix UTs on x86 2023-03-15 08:21:00 +01:00
Till
74dc54684b
Version 0.12.0 (#3009) 2023-03-13 20:02:57 +01:00
Till
232aef016c
Add basic runtime tracing (#2996)
This allows us in almost all places to use regions to further trace down
long running tasks.
Also removes an unused function.
2023-03-13 16:45:14 +01:00
Till
689b5ee72f
Change default stats reporting endpoint (#3007)
It's the same instance we report to, only using the subdomain.
2023-03-10 12:27:08 +01:00
Till
c7303cbf76
Update dependencies (#3006)
In preparation for a new release, let's also update a few dependencies.
2023-03-10 10:32:50 +01:00
Till
70322699ab
Unset RoomServerEvent, since we can't be sure that Set actually updates the cached entry (#3002)
This should deflake UTs and be more correct in terms of getting
`Events`.
`Events` tries to fetch the event from the cache first and may get an
unredacted event from it, while it should already be redacted.
2023-03-09 09:52:13 +01:00
Till Faelligen
baef523cb0
Fix invalid roomNID returned 2023-03-07 15:35:08 +01:00
Till Faelligen
11a3fcc6cb
RoomServerEvents are mutable, given they can be redacted 2023-03-06 17:58:08 +01:00
Till Faelligen
a684b850b9
Actually ignore the error if we were able to backfill events 2023-03-06 17:45:21 +01:00
Till
7d83f8b633
Add tests for UpdateRelations (#2999)
This also fixes an issue regarding updates to relations for invalid
events, which could result in us retrying said event over and over
again, if we fail to unmarshal the event to
`gomatrixserverlib.RelationContent`, this was discovered by
`@sleroq:virto.community`
2023-03-06 12:43:59 +01:00
Tim McCormack
7fc839f751
Update admin-promotion instructions; clarify ID for evacuation (#2997)
Table name has changed since instructions were written.

There's probably a better way to describe how to get the internal room
ID than I've attempted here, so feel free to adjust as needed. (It may
even be good to show an example of what an internal room ID looks like,
e.g. `!nc93825:example.com`)

### Pull Request Checklist

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
  * Doc-only change
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)

Signed-off-by: `Tim McCormack <cortex@brainonfire.net>`

---------

Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
2023-03-05 18:42:38 +01:00
Till Faelligen
56b28b01db
Update the cache with the redacted event 2023-03-03 14:49:41 +01:00
Till
9bcd0a2105
Make redaction check easier to read (#2995)
We need to check the redaction PL in Dendrite, if we do it in GMSL, we
end up not sending the event to the output stream because it will be
rejected.

---------

Co-authored-by: kegsay <kegan@matrix.org>
2023-03-03 14:03:17 +01:00
Robin Westerik
7cde99a7a7
Updated instructions and references to monolith to their new names (#2994)
Currently, the documentation makes use of the old names for the binary
and configuration files. This updates the documentation so that users
can follow the guide without issues again.
These changes don't require any go unit tests because it does not modify
any golang code.

Signed-off-by: `Robin Westerik <gh@westerik.me>`
2023-03-03 10:20:53 +01:00
Boris Rybalkin
6b1c9eafa9
unix socket support (#2974)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Boris Rybalkin <ribalkin@gmail.com>`

I need this for Syncloud project (https://github.com/syncloud/platform)
where I run multiple apps behind an nginx on the same RPi like device so
unix socket is very convenient to not have port conflicts between apps.
Also someone opened this Issue:
https://github.com/matrix-org/dendrite/issues/2924

---------

Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
2023-03-01 22:57:30 +01:00
Till
6c20f8f742
Refactor StoreEvent, add MaybeRedactEvent, create an EventDatabase (#2989)
This PR changes the following:
- `StoreEvent` now only stores an event (and possibly prev event),
instead of also doing redactions
- Adds a `MaybeRedactEvent` (pulled out from `StoreEvent`), which should
be called after storing events
- a few other things
2023-03-01 17:06:47 +01:00
Till Faelligen
1aa70b0f56
Fix UTs 2023-03-01 15:09:10 +01:00
Till Faelligen
f1ccfcf150
Only run CI if there are changes to go files or the workflow [skip ci] 2023-02-28 15:35:53 +01:00
Till Faelligen
086e205eba
Deploy on gh-pages push 2023-02-28 15:15:19 +01:00
genofire
7fff7cd2ac
feat(helm): add prometheus resources to monitor (#2958)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

---

I do not know, how you run helm-docs ....


otherwise i would like to add somewhere:
````markdown
* Works well with [Prometheus Operator](https://prometheus-operator.dev/) ([Helmchart](https://artifacthub.io/packages/helm/prometheus-community/kube-prometheus-stack)) and there setup of [Grafana](https://grafana.com/grafana/), by enabling following values:
```yaml
prometheus:
  servicemonitor:
    enabled: true
    labels:
      release: "kube-prometheus-stack"
  rules:
    enabled: true # will deploy alert rules
    additionalLabels:
      release: "kube-prometheus-stack"
grafana:
  dashboards:
    enabled: true # will deploy default dashboards 
```
PS: The labels `release=kube-prometheus-stack` is setup with the helmchart of the Prometheus Operator. For Grafana Dashboards it maybe need scan enable to scan in correct namespaces (or ALL), enabled by `sidecar.dashboards.searchNamespace` in [Helmchart of grafana](https://artifacthub.io/packages/helm/grafana/grafana) (which is part of PrometheusOperator, so `grafana.sidecar.dashboards.searchNamespace`) 
````

Maybe also put somewhere the Screenshot of that Grafana Dashboard:
https://grafana.com/grafana/dashboards/13916-dendrite/

---


@S7evinK do you take a look?

Signed-off-by: genofire <geno+dev@fireorbit.de>
2023-02-28 14:18:26 +01:00
Devon Hudson
eddf31f915
Fix lint error 2023-02-24 15:49:51 -07:00
Devon Hudson
b28406c7d0
Tweaks to pinecone demo to shutdown more cleanly 2023-02-24 15:41:47 -07:00
Till Faelligen
3d31b131fc
Cache all the things 2023-02-24 11:45:01 +01:00
Till
ad07b169b8
Refactor StoreEvent and create a new RoomDatabase interface (#2985)
This PR changes a few things:
- It pulls out the creation of several NIDs from the `StoreEvent`
function to make the functions more reusable
- Uses more caching when using those NIDs to avoid DB round trips
2023-02-24 09:40:20 +01:00
David Schneider
e6aa0955ff
Unify logging by using logrus for jetstream logs (#2976)
I guess tests for the logging is rather unusual so I omitted tests for
this change.

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `David Schneider <dsbrng25b@gmail.com>`

---------

Signed-off-by: David Schneider <dsbrng25b@gmail.com>
2023-02-24 08:56:53 +01:00
Yoann N
d34277a6c0
Update README sample config link (#2987)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately
2023-02-23 09:07:20 +01:00
Till Faelligen
c8ca23acdb
Fix building Element web in CI 2023-02-20 15:28:45 +01:00