* Ensure worker has work before starting goroutine
* Revert "Remove processEventWithMissingStateMutex"
This reverts commit 7f02eab47d.
* Use request context when processing transactions
* Keep goroutine count down by not starting work for things where the caller gave up
* Remove mutex, start workers at correct time
* Try to process rooms concurrently in FS /send
* Clean up
* Use request context so that dead things don't linger for so long
* Remove mutex
* Free up pdus slice so only references remaining are in channel
* Revert "Remove mutex"
This reverts commit 8558075e8c9bab3c1d8b2252b4ab40c7eaf774e8.
* Process EDUs in parallel
* Try refactoring /send concurrency
* Fix waitgroup
* Release on waitgroup
* Respond to transaction
* Reduce CPU usage, fix unit tests
* Tweaks
* Move into one file
* More aggressive event caching
* Deduplicate /state results
* Deduplicate more
* Ensure we use the correct list of events when excluding repeated state
* Fixes
* Ensure we track all events we already knew about properly
* db migration: fix#1844 and add additional assertions
- Migration scripts will now check to see if there are any unconverted
snapshot IDs and fail the migration if there are any. This should
prevent people from getting a corrupt database in the event the root
cause is still unknown.
- Add an ORDER BY clause when doing batch queries in the postgres
migration. LIMIT and OFFSET without ORDER BY are undefined and must
not be relied upon to produce a deterministic ordering (e.g row order).
See https://www.postgresql.org/docs/current/queries-limit.html
* Linting
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Squashed commit of the following:
commit 7fad77c10e3c1c78feddb37351812b209d9c0f25
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Mon Jun 28 15:06:52 2021 +0100
Fix processEventWithMissingStateMutexes
commit 138cddcac7b8373a8e1816a232f84a7bda6adcdf
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Mon Jun 28 13:59:44 2021 +0100
Use internal.MutexByRoom
commit 6e6f026cfad31da391ad261cfec16d41dff1b15b
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Mon Jun 28 13:50:18 2021 +0100
Try to slow things down per room
commit b97d406dff2e11769a9202fbf58b138a541ca449
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Mon Jun 28 13:41:27 2021 +0100
Try to slow things down
commit 8866120ebf880b4fd8a456937f69903e233c19a2
Merge: 9f2de8a2 4a37b19a
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Mon Jun 28 13:40:33 2021 +0100
Merge branch 'neilalexander/rsinputfifo' into neilalexander/rsinputfifo2
commit 4a37b19a8f6fe8af02e979827253d22a0ccdedb8
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Mon Jun 28 13:34:54 2021 +0100
Add comments
commit f9ab3f4b8157a42d657735101bc2c768c663e814
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Mon Jun 28 13:31:21 2021 +0100
Tweaks
commit 9f2de8a29cadec4c785d9c2e4e74c1138305f759
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Mon Jun 28 13:15:59 2021 +0100
Ask origin only for missing things for now
commit 8fd878c75a4066abb21597d524a4eb4670a392d4
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Mon Jun 28 11:18:11 2021 +0100
Make sure someone wakes up
commit b63f699f1b74948d180885449398f999fafb18c8
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Mon Jun 28 11:12:58 2021 +0100
Use a FIFO queue instead of a channel to reduce backpressure
* Set MaxFileSizeBytes < 0 to "unlimited"
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
* int64 overflows later in mediaapi/routing/upload.go[doUpload]
* Prevent int overflow when uploading
This fixes the issue found in #1759 which broke due to go changes. The new command allows you to generate keys with docker, and drop them in the current working directory.
* Add parameters to specify password
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
* Fix typo
* Add testdata
* Use go1.13 compatible way to read files
Without this entry, setups where users have the homeserver on the URL
matrix.myurl.com but want the servername to be myurl.com don't work by default
since clients like element.io can't connect to the homeserver
* Try to optimize SelectOneTimeKeys
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
* Use pg.Array when using ANY...
Co-authored-by: Kegsay <kegan@matrix.org>
* Use LimitReader to prevent DoS risk
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
* Check if bytesWritten is equal to the maxFileSize
Add tests
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
* Use oldschool defer to cleanup after the tests
* Let LimitReader read MaxFileSizeBytes + 1
Co-authored-by: Kegsay <kegan@matrix.org>
* Avoid crash on non-compiled room regex
Roughly fixes#1845 (actual compiling still needed)
Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>
* Compile regexes for all namespaces
Deadheres the regex compiling from building larger regexes for possibly
exclusive namespaces only. A complete fix for #1845, so regexes for
rooms namespaces and other non-whitelisted namespaces can be used
more safely.
Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>
* Appservice config: handle regexp parsing errors
Signed-off-by: diamondburned <datutbrus@gmail.com>
Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>
Co-authored-by: Kegsay <kegan@matrix.org>
The update part wasn't executed actually for SQLite, so it is moved to
a separate statement.
Fixes#1852.
Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>
