Commit Graph

22 Commits

Author SHA1 Message Date
Florent 6280d09765 Upload part size as S3 parameter instead of constant value
Some S3 providers need a custom upload part size (500 MB static value in Nextcloud).
Here is a commit to change this value via S3 configuration, instead of using S3_UPLOAD_PART_SIZE constant.
A new parameter is added for an S3 connection : uploadPartSize

Signed-off-by: Florent <florent@coppint.com>
2020-06-15 08:40:02 +00:00
Christoph Wurst cb057829f7
Update license headers for 19
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-29 11:57:22 +02:00
Christoph Wurst 1584c9ae9c
Add visibility to all methods and position of static keyword
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:51:06 +02:00
Robin Appelman 7b07e7251c
make seekable s3 stream generic
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-01 15:21:05 +02:00
Lukas Stabe 14401efb0f
Enable fseek for files in S3 storage
Signed-off-by: Lukas Stabe <lukas@stabe.de>
2020-04-01 15:21:01 +02:00
Christoph Wurst 5bf3d1bb38
Update license headers
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2019-12-05 15:38:45 +01:00
Morris Jobke e45fb5fa3e
Fix typo in comment
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2019-07-29 14:55:43 +02:00
Roeland Jago Douma e953205908
Use HTTP1.1 to read S3 objects
Some of the READs otherwise use HTTP/1.0 which is not always supported
by all backends. HTTP/1.1 is there since 1999 way longer than S3 so safe
to assume it is always there IMO.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-07-10 11:42:22 +02:00
Roeland Jago Douma 61d66d7954
Fix empty file uploads to S3
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-02-15 08:41:39 +01:00
Robin Appelman 1d322d3b37
always use multipart uploader for s3 uploads
the multipart uploader handles non seekable streams while `upload` does not

Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-01-28 14:51:19 +01:00
Robin Appelman 6c9f2644cf
Add objectExists to objectstore interface
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-12-12 14:17:19 +01:00
Morris Jobke d3d045dd5c
Remove unused import statements
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-02-14 16:55:43 +01:00
Robin Appelman 3499cbdb76
Use S3Client::upload instead of splitting single/multipart upload ourselves
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-01-24 17:22:05 +01:00
Robin Appelman 70a4860ccb
Fix contructing headers for s3 download
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-11-08 17:50:46 +01:00
Morris Jobke 0eebff152a
Update license headers
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2017-11-06 16:56:19 +01:00
Robin Appelman f9a7294807
use fopen directly when reading objects from s3
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-10-27 12:58:19 +02:00
Robin Appelman 91b3536f45
propagate multipart upload exception when aborting upload
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-10-17 15:10:58 +02:00
Robin Appelman e393b3553e
set s3 part size to 500mb
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-10-11 15:59:53 +02:00
Robin Appelman 385d6f098c
Add tests for multipart upload
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-09-21 14:47:34 +02:00
Robin Appelman 4ae46d8876
only do multipart upload for large files
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-09-21 14:06:59 +02:00
Robin Appelman e4e5e735db
multipart upload for s3 object storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-09-20 17:51:58 +02:00
Robin Appelman d70607104e
reuse object read/write/delete logic in s3 implementations
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-09-18 15:16:27 +02:00