* Rate limiting changes
This makes the following changes:
* For logged in users, the rate limiting now applies to the device session rather than the remote IP address;
* For non-logged in users, the rate limiting continues to apply to remote address as it does today;
* It is now possible to add user IDs to the `exempt_user_ids` option under `rate_limiting` to exclude bots from rate limiting;
* Admin and appservice users are now exempt from rate limiting by default.
* Fix build with media API
* Specify interfaces used by appservice, do half of clientapi
* convert more deps of clientapi to finer-grained interfaces
* Convert mediaapi and rest of clientapi
* Somehow this got missed
* Simplify federation API `AddPublicRoutes`
* Simplify client API `AddPublicRoutes`
* Simplify media API `AddPublicRoutes`
* Simplify sync API `AddPublicRoutes`
* Simplify `AddAllPublicRoutes`
* Set MaxFileSizeBytes < 0 to "unlimited"
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
* int64 overflows later in mediaapi/routing/upload.go[doUpload]
* Prevent int overflow when uploading
* Use LimitReader to prevent DoS risk
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
* Check if bytesWritten is equal to the maxFileSize
Add tests
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
* Use oldschool defer to cleanup after the tests
* Let LimitReader read MaxFileSizeBytes + 1
Co-authored-by: Kegsay <kegan@matrix.org>
* Remove the Content-Length requirement
* Make sure that the file size does not exceed max content size
* Address review comment - universally check if temp file size exceeds max file size
* Get downloaded file size regardless of the Content-Size header
Signed-off-by: Florian Le Minoux <darkgallium+git@deuxfleurs.fr>
* fixing lint issues
* Don't exhaust memory for large files, don't limit more than necessary
* Don't use errors.Wrap in download.go
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Offset updates take place using TransactionWriter
* Refactor TransactionWriter in current state server
* Refactor TransactionWriter in federation sender
* Refactor TransactionWriter in key server
* Refactor TransactionWriter in media API
* Refactor TransactionWriter in server key API
* Refactor TransactionWriter in sync API
* Refactor TransactionWriter in user API
* Fix deadlocking Sync API tests
* Un-deadlock device database
* Fix appservice API
* Rename TransactionWriters to Writers
* Move writers up a layer in sync API
* Document sqlutil.Writer interface
* Add note to Writer documentation
* Updated TransactionWriters, moved locks in roomserver, various other tweaks
* Fix redaction deadlocks
* Fix lint issue
* Rename SQLiteTransactionWriter to ExclusiveTransactionWriter
* Fix us not sending transactions through in latest events updater
* Initial pass at refactoring config (not finished)
* Don't forget current state and EDU servers
* More shifting around
* Update server key API tests
* Fix roomserver test
* Fix more tests
* Further tweaks
* Fix current state server test (sort of)
* Maybe fix appservices
* Fix client API test
* Include database connection string in database options
* Fix sync API build
* Update config test
* Fix unit tests
* Fix federation sender build
* Fix gobind build
* Set Listen address for all services in HTTP monolith mode
* Validate config, reinstate appservice derived in directory, tweaks
* Tweak federation API test
* Set MaxOpenConnections/MaxIdleConnections to previous values
* Update generate-config
* Use TransactionWriter on other component SQLites
* Fix sync API tests
* Fix panic in media API
* Fix a couple of transactions
* Fix wrong query, add some logging output
* Add debug logging into StoreEvent
* Adjust InsertRoomNID
* Update logging
* Derive content ID from hash+filename but preserve dedupe, improve Content-Disposition handling and ASCII handling
* Linter fix
* Some more comments
* Update sytest-whitelist
* Make userapi responsible for checking access tokens
There's still plenty of dependencies on account/device DBs, but this
is a start. This is a breaking change as it adds a required config
value `listen.user_api`.
* Cleanup
* Review comments and test fix
* Split out adding HTTP routes from making internal APIs for clarity
* Split out more components
* Split out more things
* Finish converting
* internal mux for internal routes
* Separate muxes for public and internal APIs
* Update client-api-proxy and federation-api-proxy so they don't add /api to the path
* Tidy up
* Consistent HTTP setup
* Set up prefixes properly
Error messages indicate that:
> mediaId must be a non-empty string using only characters in `mediaIDCharacters`
However the regex used only required that some characters in the filename match
the restriction, not that the entire filename does. This commit ensures that
the filename must entirely fullfill the `mediaIDCharacters` restriction
Signed-off-by: Sid Karunaratne <sid@karunaratne.net>
Co-authored-by: Kegsay <kegan@matrix.org>
