Fixes#16876
Before we'd just fetch everything from all storages we'd have access to.
Then we'd sort. And filter in php. Now this of course is tricky if a
user shared just a file with you and then has a ton of activity.
Now we try to contruct the prefix path. So that the filtering can happen
right away in the databae.
Now this will make the DB more busy. But it should help overall as in
most cases less queries are needed then etc.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
* introduces a new IRootMountProvider to register mount points inside the root storage
* adds a AppdataPreviewObjectStoreStorage to handle the split between preview folders and bucket number
Ref #22033
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
having the "cache rename" after the "storage move" caused the target
to get the fileid from the source file, without taking care that the object
is stored under the original file id.
By doing the "cache rename" first, we trigger the "update existing file"
logic while moving the file to the object store and the object gets stored for the
correct file id
Signed-off-by: Robin Appelman <robin@icewind.nl>
"Exception: substr() expects parameter 3 to be int, bool given" can occur on Line 378 $mimePart = substr($icon, 0, strpos($icon, '-'));
This happens, when '-' is not found and strpos returns false instead of an int.
When this occurs, e.g., Activity hangs.
Signed-off-by: lui87kw <lukas.ifflaender@uni-wuerzburg.de>
the php ftp streamwrapper doesn't handle hashes correctly and will break when it tries to enter a path containing a hash.
By filtering out paths containing a hash we can at least stop the external storage from breaking completely
Signed-off-by: Robin Appelman <robin@icewind.nl>
while some code paths do wrap the "raw" locking exception into one with a proper path, not all of them do
by adding the proper path to the original exception we ensure that we always have the usefull information in out logs
Signed-off-by: Robin Appelman <robin@icewind.nl>
The S3 client enables this by default and then tries to read
`.aws/config`. This causes `open_basedir` restriction related error for
some setups. So this patch disables the CSM because it's most likely
unused anyway.
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
while this scan *should* never be triggered, it's good to have some failsafe to ensure
that the users home contents don't end up getting scanned in the root storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
Some S3 providers need a custom upload part size (500 MB static value in Nextcloud).
Here is a commit to change this value via S3 configuration, instead of using S3_UPLOAD_PART_SIZE constant.
A new parameter is added for an S3 connection : uploadPartSize
Signed-off-by: Florent <florent@coppint.com>
Right now if you want to get events via the Node API you have to have a
real instance of the Root. Which in turns sets up the whole FS.
We should make sure this is done lazy. Else enabling the preview
generator for example makes you setup the whole FS on each and every
authenticated call.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
Else if a lot of writes happen. It might happen that an old stat result
is used. Resulting in a wrong file size for the file. For example the
text app when a lot of people edit at the same time.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
Until now, you wouldn't be able to create
objects larger that 5GB.
It's somewhat related with pull #18883
Signed-off-by: Adrian Brzezinski <adrian.brzezinski@eo.pl>
Improves efficiency when downloading files from Swift storage.
Before, files were downloaded and then pushed back to user.
That behaevior causes all kinds of performance problems.
Now, files are streamed directly to user.
Signed-off-by: Adrian Brzezinski <adrian.brzezinski@eo.pl>
Currently you need to use `opendir` and then call `getMetadata` for
every file, which adds overhead because most storage backends already
get the metadata when doing the `opendir`.
While storagebackends can (and do) use caching to relief this problem,
this adds cache invalidation dificulties and only a limited number of
items are generally cached (to prevent memory usage exploding when
scanning large storages)
With this new methods storage backends can use the child metadata they
got from listing the folder to return metadata without having to keep
seperate caches.
Signed-off-by: Robin Appelman <robin@icewind.nl>
hashes are set in "X-Hash-MD5", "X-Hash-SHA1" and "X-Hash-SHA256" headers.
these headers are set for file uploads and the MOVE request at the end of a multipart upload.
Signed-off-by: Robin Appelman <robin@icewind.nl>
Roeland Jago Douma
Morris Jobke
Robin Appelman
lui87kw
bladewing
J0WI
Christoph Wurst
Roeland Jago Douma
Daniel Kesselberg
Florent
Adrian Brzezinski
Scott Dutton
Christoph Wurst
Julien Lutran