Commit Graph

17 Commits

Author SHA1 Message Date
S7evinK
9de7efa0b0
Remove sarama/saramajetstream dependencies (#2138)
* Remove dependency on saramajetstream & sarama

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Remove internal.ContinualConsumer from federationapi

* Remove internal.ContinualConsumer from syncapi

* Remove internal.ContinualConsumer from keyserver

* Move to new Prepare function

* Remove saramajetstream & sarama dependency

* Delete unneeded file

* Remove duplicate import

* Log error instead of silently irgnoring it

* Move `OffsetNewest` and `OffsetOldest` into keyserver types, change them to be more sane values

* Fix comments

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-02-04 13:08:13 +00:00
kegsay
2c581377a5
Remodel how device list change IDs are created (#2098)
* Remodel how device list change IDs are created

Previously we made them using the offset Kafka supplied.
We don't run Kafka anymore, so now we make the SQL table assign
the change ID via an AUTOINCREMENTing ID. Redesign the
`keyserver_key_changes` table to have `UNIQUE(user_id)` so we
don't accumulate key changes forevermore, we now have at most 1
row per user which contains the highest change ID.

This needs a SQL migration.

* Ensure we bump the change ID on sqlite

* Actually read the DeviceChangeID not the Offset in synapi

* Add SQL migrations

* Prepare after migration; fixup dendrite-upgrade-test logging

* Use higher version numbers; fix sqlite query to increment better

* Default 0 on postgres

* fixup postgres migration on fresh dendrite instances
2022-01-21 09:56:06 +00:00
Neil Alexander
614e67280d
Delete device keys/signatures from key server when deleting devices (#1979)
* Delete device keys/signatures from key server when deleting device from user API

* Move loop to within database transaction

* Don't fall over deleting no rows
2021-08-18 12:07:09 +01:00
Neil Alexander
e95b1fd238
Cross-signing validation for self-sigs, expose signatures over /user/keys/query and /user/devices/{userId} (#1962)
* Enable unstable feature again

* Try to verify when a device signs a key

* Try to verify when a key signs a device

* It's the self-signing key, not the master key

* Fix error

* Try to verify master key uploads

* Actually we can't guarantee we can do that so nevermind

* Add signatures into /devices/list request

* Fix nil pointer

* Reprioritise map creation

* Don't skip devices that don't have signatures

* Add some debug logging

* Fix logic error in QuerySignatures

* Fix bugs

* Expose master and self-signing keys on /devices/list hopefully

* maps are tedious

* Expose signatures via /keys/query

* Upload signatures when uploading keys

* Fixes

* Disable the feature again
2021-08-06 10:13:35 +01:00
Neil Alexander
7a9a2547b3
Cross-signing storage code (#1959) 2021-08-04 17:31:18 +01:00
Kegsay
820c56c165
Fix more E2E sytests (#1265)
* WIP: Eagerly sync device lists on /user/keys/query requests

Also notify servers when a user's device display name changes. Few
caveats:
 - sytest `Device deletion propagates over federation` fails
 - `populateResponseWithDeviceKeysFromDatabase` is called from multiple
   goroutines and hence is unsafe.

* Handle deleted devices correctly over federation
2020-08-12 22:43:02 +01:00
Kegsay
32a4565b55
Add device list updater which manages updating remote device lists (#1242)
* Add device list updater which manages updating remote device lists

- Doesn't persist stale lists to the database yet
- Doesn't have tests yet

* Mark device lists as fresh when we persist
2020-08-06 17:48:10 +01:00
Kegsay
642f9cb964
Process inbound device list updates from federation (#1240)
* Add InputDeviceListUpdate

* Unbreak unit tests

* Process inbound device list updates from federation

- Persist the keys in the keyserver and produce key changes
- Does not currently fetch keys from the remote server if the prev IDs are missing

* Linting
2020-08-05 13:41:16 +01:00
Kegsay
fb56bbf0b7
Generate stream IDs for locally uploaded device keys (#1236)
* Breaking: add stream_id to keyserver_device_keys table

* Add tests for stream ID generation

* Fix whitelist
2020-08-03 17:07:06 +01:00
Kegsay
ffcb6d2ea1
Produce OTK counts in /sync response (#1235)
* Add QueryOneTimeKeys for /sync extensions

* Unbreak tests

* Produce OTK counts in /sync response

* Linting
2020-08-03 12:29:58 +01:00
Kegsay
a7e67e65a8
Notify clients when devices are deleted (#1233)
* Recheck device lists when join/leave events come in

* Add PerformDeviceDeletion

* Notify clients when devices are deleted

* Unbreak things

* Remove debug logging
2020-07-30 18:00:56 +01:00
Kegsay
a2174d3294
Implement /keys/changes (#1232)
* Implement /keys/changes

And refactor QueryKeyChanges to accept a `to` offset.

* Unbreak tests

* Sort keys when serialising log tokens
2020-07-30 14:52:21 +01:00
Kegsay
adf7b59294
Persist partition|offset|user_id in the keyserver (#1226)
* Persist partition|offset|user_id in the keyserver

Required for a query API which will be used by the syncapi which
will be called when a `/sync` request comes in which will return
a list of user IDs of people who have changed their device keys
between two tokens.

* Add tests and fix maxOffset bug

* s/offset/log_offset/g because 'offset' is a reserved word in postgres
2020-07-28 17:38:30 +01:00
Kegsay
1d72ce8b7a
Implement claiming one-time keys locally (#1210)
* Add API shape for claiming keys

* Implement claiming one-time keys locally

Fairly boring, nothing too special going on.
2020-07-21 14:47:53 +01:00
Kegsay
f5e7e7513c
Implement /keys/query locally (#1204)
* Implement /keys/query locally

* Fix sqlite tests and close rows
2020-07-15 18:40:41 +01:00
Kegsay
9dd2ed7f65
Implement key uploads (#1202)
* Add storage layer for postgres/sqlite

* Return OTK counts when inserting new keys

* Hook up the key DB and make a test pass

* Convert postgres queries to be sqlite queries

* Blacklist test due to requiring rejected events

* Unbreak tests

* Update blacklist
2020-07-15 12:02:34 +01:00
Kegsay
7daa3bf098
Implement logic for key uploads (#1197)
* begin work on storing keys

* Finish rough impl of the internal key API

* Linting
2020-07-14 12:59:07 +01:00