streams get closed automatically when dropped, and in some cases the stream seems to be already closed by the S3 library, in which case trying to close it again will raise an error
Signed-off-by: Robin Appelman <robin@icewind.nl>
Since we try to do range requests this will fail hard.
However since empty files are not that interesting to read anyways we
just read from an emptry memory stream.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
If the object store errors we should not always delete the filecache
entry. As this might lead to people losing access to their files.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
The S3 client enables this by default and then tries to read
`.aws/config`. This causes `open_basedir` restriction related error for
some setups. So this patch disables the CSM because it's most likely
unused anyway.
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
Some S3 providers need a custom upload part size (500 MB static value in Nextcloud).
Here is a commit to change this value via S3 configuration, instead of using S3_UPLOAD_PART_SIZE constant.
A new parameter is added for an S3 connection : uploadPartSize
Signed-off-by: Florent <florent@coppint.com>
Until now, you wouldn't be able to create
objects larger that 5GB.
It's somewhat related with pull #18883
Signed-off-by: Adrian Brzezinski <adrian.brzezinski@eo.pl>
Improves efficiency when downloading files from Swift storage.
Before, files were downloaded and then pushed back to user.
That behaevior causes all kinds of performance problems.
Now, files are streamed directly to user.
Signed-off-by: Adrian Brzezinski <adrian.brzezinski@eo.pl>
Currently you need to use `opendir` and then call `getMetadata` for
every file, which adds overhead because most storage backends already
get the metadata when doing the `opendir`.
While storagebackends can (and do) use caching to relief this problem,
this adds cache invalidation dificulties and only a limited number of
items are generally cached (to prevent memory usage exploding when
scanning large storages)
With this new methods storage backends can use the child metadata they
got from listing the folder to return metadata without having to keep
seperate caches.
Signed-off-by: Robin Appelman <robin@icewind.nl>
To continue this formatting madness, here's a tiny patch that adds
unified formatting for control structures like if and loops as well as
classes, their methods and anonymous functions. This basically forces
the constructs to start on the same line. This is not exactly what PSR2
wants, but I think we can have a few exceptions with "our" style. The
starting of braces on the same line is pracrically standard for our
code.
This also removes and empty lines from method/function bodies at the
beginning and end.
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
* Order the imports
* No leading slash on imports
* Empty line before namespace
* One line per import
* Empty after imports
* Emmpty line at bottom of file
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
Some of the READs otherwise use HTTP/1.0 which is not always supported
by all backends. HTTP/1.1 is there since 1999 way longer than S3 so safe
to assume it is always there IMO.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
Apparently the if statement doesn't work in all cases (even if I could
not reproduce it). So for the time being we will just not directly
stream to swift.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
Morris Jobke
Robin Appelman
Roeland Jago Douma
Christoph Wurst
Daniel Kesselberg
Florent
Roeland Jago Douma
Adrian Brzezinski
Julien Lutran
Lukas Stabe
Marcel Klehr