Commit Graph

25 Commits

Author SHA1 Message Date
Robin Appelman 0532f64aa8 remove explicit fclose from S3->writeStream
streams get closed automatically when dropped, and in some cases the stream seems to be already closed by the S3 library, in which case trying to close it again will raise an error

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-17 09:36:02 +00:00
Robin Appelman 9d4848e863
use in objectstore copy
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-11-24 15:16:58 +01:00
Christoph Wurst 2a054e6c04
Update the license headers for Nextcloud 20
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-08-24 14:54:25 +02:00
Florent 3594ba6971 Upload part size as S3 parameter instead of constant value
Some S3 providers need a custom upload part size (500 MB static value in Nextcloud).
Here is a commit to change this value via S3 configuration, instead of using S3_UPLOAD_PART_SIZE constant.
A new parameter is added for an S3 connection : uploadPartSize

Signed-off-by: Florent <florent@coppint.com>
2020-06-09 09:18:42 +02:00
Christoph Wurst cb057829f7
Update license headers for 19
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-29 11:57:22 +02:00
Christoph Wurst 1584c9ae9c
Add visibility to all methods and position of static keyword
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:51:06 +02:00
Robin Appelman 7b07e7251c
make seekable s3 stream generic
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-01 15:21:05 +02:00
Lukas Stabe 14401efb0f
Enable fseek for files in S3 storage
Signed-off-by: Lukas Stabe <lukas@stabe.de>
2020-04-01 15:21:01 +02:00
Christoph Wurst 5bf3d1bb38
Update license headers
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2019-12-05 15:38:45 +01:00
Morris Jobke e45fb5fa3e
Fix typo in comment
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2019-07-29 14:55:43 +02:00
Roeland Jago Douma e953205908
Use HTTP1.1 to read S3 objects
Some of the READs otherwise use HTTP/1.0 which is not always supported
by all backends. HTTP/1.1 is there since 1999 way longer than S3 so safe
to assume it is always there IMO.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-07-10 11:42:22 +02:00
Roeland Jago Douma 61d66d7954
Fix empty file uploads to S3
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-02-15 08:41:39 +01:00
Robin Appelman 1d322d3b37
always use multipart uploader for s3 uploads
the multipart uploader handles non seekable streams while `upload` does not

Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-01-28 14:51:19 +01:00
Robin Appelman 6c9f2644cf
Add objectExists to objectstore interface
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-12-12 14:17:19 +01:00
Morris Jobke d3d045dd5c
Remove unused import statements
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-02-14 16:55:43 +01:00
Robin Appelman 3499cbdb76
Use S3Client::upload instead of splitting single/multipart upload ourselves
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-01-24 17:22:05 +01:00
Robin Appelman 70a4860ccb
Fix contructing headers for s3 download
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-11-08 17:50:46 +01:00
Morris Jobke 0eebff152a
Update license headers
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2017-11-06 16:56:19 +01:00
Robin Appelman f9a7294807
use fopen directly when reading objects from s3
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-10-27 12:58:19 +02:00
Robin Appelman 91b3536f45
propagate multipart upload exception when aborting upload
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-10-17 15:10:58 +02:00
Robin Appelman e393b3553e
set s3 part size to 500mb
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-10-11 15:59:53 +02:00
Robin Appelman 385d6f098c
Add tests for multipart upload
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-09-21 14:47:34 +02:00
Robin Appelman 4ae46d8876
only do multipart upload for large files
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-09-21 14:06:59 +02:00
Robin Appelman e4e5e735db
multipart upload for s3 object storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-09-20 17:51:58 +02:00
Robin Appelman d70607104e
reuse object read/write/delete logic in s3 implementations
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-09-18 15:16:27 +02:00