Commit Graph

719 Commits

Author SHA1 Message Date
Robin Appelman b8d64d67fe
only require user to be set in a query that handles tags
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-23 18:02:15 +01:00
Robin Appelman 549db744dc
unify handling of Folder::search methods into "new" query objects
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-23 18:02:12 +01:00
Morris Jobke ceed64154f
Merge pull request #26148 from nextcloud/backport/26013/stable19
[stable19] Return the fileid from `copyFromCache` and use it instead of doing an extra query
2021-03-19 15:29:14 +01:00
Morris Jobke f302616544
Merge pull request #26092 from nextcloud/backport/25722/stable19
[stable19] apply object store copy optimization when 'cross storage' copy is wit…
2021-03-17 20:09:11 +01:00
Robin Appelman 68fb72b028 remove explicit fclose from S3->writeStream
streams get closed automatically when dropped, and in some cases the stream seems to be already closed by the S3 library, in which case trying to close it again will raise an error

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-17 09:36:15 +00:00
Robin Appelman 300c4dd6b8 verify that cache entry is valid
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-16 16:18:13 +00:00
Robin Appelman 53dcc7b1ca also implement for FailedCache and NullCache
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-16 16:18:11 +00:00
Robin Appelman cc1a5d5ced use returned file id for object store copy
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-16 16:18:09 +00:00
Robin Appelman 603ec9be0c add ICopyFromCache trait to expose existing implementation
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-16 16:18:06 +00:00
Robin Appelman 1d1d4de991
adjust tests
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-16 15:27:00 +01:00
Robin Appelman b9b2c6d2fa
fix search in nested jails
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-16 15:27:00 +01:00
Robin Appelman 75a48fbc2a
fix cachjail searching for root
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-16 15:26:59 +01:00
Robin Appelman 564390e303
do cachejail search filtering in sql
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-16 15:26:59 +01:00
Robin Appelman b39f9fb2c9 apply object store copy optimization when 'cross storage' copy is within the same object store
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-12 16:16:38 +00:00
Robin Appelman dedff0facb use in objectstore copy
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-12-07 16:07:25 +00:00
aler9 349f754833 Fix file size computation on 32bit platforms
Signed-off-by: aler9 <46489434+aler9@users.noreply.github.com>
2020-12-02 15:27:22 +00:00
lui87kw aecf4eff27 Extra white space to follow other ifs
Signed-off-by: lui87kw <lukas.ifflaender@uni-wuerzburg.de>
2020-11-25 11:55:16 +00:00
bladewing 43e401e3a2 Avoid substr() error when strpos returns false
"Exception: substr() expects parameter 3 to be int, bool given" can occur on Line 378 $mimePart = substr($icon, 0, strpos($icon, '-'));
This happens, when '-' is not found and strpos returns false instead of an int.
When this occurs, e.g., Activity hangs.

Signed-off-by: lui87kw <lukas.ifflaender@uni-wuerzburg.de>
2020-11-25 11:55:15 +00:00
Ashod Nakashian da7f445179 Unlock when promoting to exclusive lock fails
In certain cases changeLock to EXCLUSIVE fails
and throws LockedException. This leaves the
file locked as SHARED in file_put_contents,
which prevents retrying (because on second
call file_put_contents takes another SHARED
lock on the same file, and changeLock doesn't
allow more than a single SHARED lock to promote
to EXCLUSIVE).

To avoid this case, we catch the LockedException
and unlock before re-throwing.

Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2020-11-12 19:07:35 +00:00
John Molakvoæ (skjnldsv) 40a2cf96be Make sure root storage is valid before checking its size
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2020-11-09 10:08:17 +00:00
Julius Härtl 14c3f1ebd8
Transfer shares of the transferred root node
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-09-17 19:03:07 +02:00
korelstar bd24021efc fix: file quota was not applied in all cases 2020-09-17 07:13:18 +00:00
Roeland Jago Douma 24b69b2402 Fix reading empty files from objectstorage
Since we try to do range requests this will fail hard.
However since empty files are not that interesting to read anyways we
just read from an emptry memory stream.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-09-09 19:47:23 +00:00
Julius Härtl 554fab45aa Properly add both mimetypes to secure mimetype mapping
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-09-09 12:17:26 +00:00
Julius Härtl 568f68856f
Merge pull request #22519 from nextcloud/backport/22517/stable19
[stable19] set the mount id before calling storage wrapper
2020-08-31 21:51:59 +02:00
Roeland Jago Douma 952ec3370e Only update the filecache entry once the file has been written to S3
If we already update before we have no way to revert if the upload
fails.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-08-31 17:08:27 +00:00
Roeland Jago Douma 704f515175 Don't lose filecache entry on s3 overwrite error
If the object store errors we should not always delete the filecache
entry. As this might lead to people losing access to their files.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-08-31 17:08:27 +00:00
Robin Appelman fc8c5965fa set the mount id before calling storage wrapper
this allows the storage wrapper to use the mount id for it's own logic

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-08-31 15:23:13 +00:00
Roeland Jago Douma d8c958907f
Make legacy cipher opt in
* Systems that upgrade have this enabled by default
* New systems disable it
* We'll have to add some wargning in the setup checks if this is enabled

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-08-24 11:50:04 +02:00
Morris Jobke d7aa1c00ed
Merge pull request #21983 from nextcloud/backport/21628/stable19
[stable19] fix moving files from external storage to object store trashbin
2020-08-19 21:29:01 +02:00
J0WI be73fd90fc Only clear the realpath cache
Signed-off-by: J0WI <J0WI@users.noreply.github.com>
2020-08-12 01:26:33 +00:00
J0WI 6f72871ca5 clearstatcache() on rmdir
Signed-off-by: J0WI <J0WI@users.noreply.github.com>
2020-08-12 01:26:32 +00:00
Robin Appelman 4e6719b28c
dont update mimetype when moving to trash
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-31 17:21:02 +02:00
Robin Appelman 142b6eb08f fix renameFromStorage messing with folder mimetype
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-23 21:01:23 +00:00
Robin Appelman 7ac4d57fd5 use exceptions for error signaling in writeStream
this remove the ambiguity when writing zero length files

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-23 21:01:23 +00:00
Robin Appelman c567b1d6b2 fix moving files from external storage to object store trashbin
having the "cache rename" after the "storage move" caused the target
to get the fileid from the source file, without taking care that the object
is stored under the original file id.

By doing the "cache rename" first, we trigger the "update existing file"
logic while moving the file to the object store and the object gets stored for the
correct file id

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-23 21:01:22 +00:00
Robin Appelman 157f619812 ensure home storage is initialized on first setup
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-09 12:33:20 +00:00
Christoph Wurst 3201e7674f Fix static method call for s3 bucket compat check
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-07-03 08:26:00 +00:00
Morris Jobke f70b2dba18
Merge pull request #21573 from nextcloud/backport/21558/stable19
[stable19] relax permissions mask check for detecting part file rename
2020-07-02 23:51:03 +02:00
Daniel Kesselberg 4397e57201 Update presign method to match with interface again.
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2020-06-30 18:39:08 +00:00
Robin Appelman 654f5a2109 relax permissions mask check for detecting part file rename
with files drop uploads the original file name isn't always used for the '.ocTransferId' source path

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-06-24 19:11:18 +00:00
Christoph Wurst 60a3bf8572 Disable Client-Side Monitoring on AWS storage
The S3 client enables this by default and then tries to read
`.aws/config`. This causes `open_basedir` restriction related error for
some setups. So this patch disables the CSM because it's most likely
unused anyway.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-06-17 09:00:28 +00:00
Florent 6280d09765 Upload part size as S3 parameter instead of constant value
Some S3 providers need a custom upload part size (500 MB static value in Nextcloud).
Here is a commit to change this value via S3 configuration, instead of using S3_UPLOAD_PART_SIZE constant.
A new parameter is added for an S3 connection : uploadPartSize

Signed-off-by: Florent <florent@coppint.com>
2020-06-15 08:40:02 +00:00
Roeland Jago Douma 9d2d3c4809 Clear the statscache before fetching the metadata
Else if a lot of writes happen. It might happen that an old stat result
is used. Resulting in a wrong file size for the file. For example the
text app when a lot of people edit at the same time.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-05-28 10:30:35 +00:00
Roeland Jago Douma d5850eb28f
Merge pull request #18955 from adrb/swift_upload_large_objects
Large Object support for OpenStack Swift
2020-05-06 13:03:24 +02:00
Robin Appelman 4e6d3d1880
Fix getDirectoryContent implementation for Jail wrapper
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-30 15:27:09 +02:00
Roeland Jago Douma f6b435d0d9
Run the cs fixer for green ci
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-04-30 10:28:44 +02:00
Roeland Jago Douma a1c1b354fc
Merge pull request #18883 from adrb/optimize_swift_14116
Optimize Openstack Swift files download
2020-04-30 09:06:19 +02:00
Christoph Wurst cb057829f7
Update license headers for 19
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-29 11:57:22 +02:00
Adrian Brzezinski e107519295 Large Object support for OpenStack Swift.
Until now, you wouldn't be able to create
objects larger that 5GB.

It's somewhat related with pull #18883

Signed-off-by: Adrian Brzezinski <adrian.brzezinski@eo.pl>
2020-04-27 14:23:59 +02:00