Commit Graph

699 Commits

Author SHA1 Message Date
Robin Appelman 8c3b22859c
dont update mimetype when moving to trash
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-30 16:11:45 +02:00
Robin Appelman c8cf2e8a5b
fix renameFromStorage messing with folder mimetype
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-23 20:31:35 +02:00
Robin Appelman ad7798f9c9
use exceptions for error signaling in writeStream
this remove the ambiguity when writing zero length files

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-23 15:24:52 +02:00
Robin Appelman 5949849322
fix moving files from external storage to object store trashbin
having the "cache rename" after the "storage move" caused the target
to get the fileid from the source file, without taking care that the object
is stored under the original file id.

By doing the "cache rename" first, we trigger the "update existing file"
logic while moving the file to the object store and the object gets stored for the
correct file id

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-23 15:24:44 +02:00
lui87kw 3fe3d1fca3 Extra white space to follow other ifs
Signed-off-by: lui87kw <lukas.ifflaender@uni-wuerzburg.de>
2020-07-15 10:18:47 +02:00
bladewing 7dafd5831f Avoid substr() error when strpos returns false
"Exception: substr() expects parameter 3 to be int, bool given" can occur on Line 378 $mimePart = substr($icon, 0, strpos($icon, '-'));
This happens, when '-' is not found and strpos returns false instead of an int.
When this occurs, e.g., Activity hangs.

Signed-off-by: lui87kw <lukas.ifflaender@uni-wuerzburg.de>
2020-07-15 10:18:47 +02:00
Robin Appelman e357d46863
filter files containing a hash in the path for ftp storages
the php ftp streamwrapper doesn't handle hashes correctly and will break when it tries to enter a path containing a hash.

By filtering out paths containing a hash we can at least stop the external storage from breaking completely

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-10 14:18:40 +02:00
Robin Appelman 70906a359f
ensure home storage is initialized on first setup
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-09 11:14:45 +02:00
Morris Jobke e900f424ce
Merge pull request #21636 from nextcloud/lock-exception-readable-path
add proper paths to locking exceptions
2020-07-05 22:37:12 +02:00
Christoph Wurst d3a4bc91f7
Fix static method call for s3 bucket compat check
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-07-03 09:02:58 +02:00
Roeland Jago Douma d53e005458
Merge pull request #21360 from nextcloud/root-storage-prevent-scanning
prevent the root storage from accidentally scanning user folders
2020-07-02 21:07:16 +02:00
Roeland Jago Douma 873be16c8d
Merge pull request #21588 from nextcloud/bug/21214/method-signature
Update presign method to match with interface again.
2020-06-30 20:36:30 +02:00
Robin Appelman da2d425044
add proper paths to locking exceptions
while some code paths do wrap the "raw" locking exception into one with a proper path, not all of them do
by adding the proper path to the original exception we ensure that we always have the usefull information in out logs

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-06-30 18:10:42 +02:00
Daniel Kesselberg e984357cfc
Set retry = false for forbidden exception.
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2020-06-25 23:34:41 +02:00
Daniel Kesselberg 8aa99aa8ff
Update presign method to match with interface again.
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2020-06-25 22:56:28 +02:00
Robin Appelman 35d75dbb38
relax permissions mask check for detecting part file rename
with files drop uploads the original file name isn't always used for the '.ocTransferId' source path

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-06-24 13:06:14 +02:00
Christoph Wurst 9d392891be
Disable Client-Side Monitoring on AWS storage
The S3 client enables this by default and then tries to read
`.aws/config`. This causes `open_basedir` restriction related error for
some setups. So this patch disables the CSM because it's most likely
unused anyway.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-06-15 09:13:08 +02:00
Robin Appelman e9b5514186
prevent the root storage from accidentally scanning user folders
while this scan *should* never be triggered, it's good to have some failsafe to ensure
that the users home contents don't end up getting scanned in the root storage

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-06-11 15:41:33 +02:00
Roeland Jago Douma 8bee8e1317
Merge pull request #21317 from FlorentCoppint/s3_upload_part_size
Upload part size as S3 parameter instead of constant value
2020-06-09 19:28:58 +02:00
Florent 3594ba6971 Upload part size as S3 parameter instead of constant value
Some S3 providers need a custom upload part size (500 MB static value in Nextcloud).
Here is a commit to change this value via S3 configuration, instead of using S3_UPLOAD_PART_SIZE constant.
A new parameter is added for an S3 connection : uploadPartSize

Signed-off-by: Florent <florent@coppint.com>
2020-06-09 09:18:42 +02:00
Robin Appelman e75797ad67
increase log level for storage self-test
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-06-08 18:12:49 +02:00
Roeland Jago Douma 6aa6ab3e02
Add lazy events for the Node API
Right now if you want to get events via the Node API you have to have a
real instance of the Root. Which in turns sets up the whole FS.

We should make sure this is done lazy. Else enabling the preview
generator for example makes you setup the whole FS on each and every
authenticated call.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-05-28 12:35:45 +02:00
Roeland Jago Douma 6b26744787
Clear the statscache before fetching the metadata
Else if a lot of writes happen. It might happen that an old stat result
is used. Resulting in a wrong file size for the file. For example the
text app when a lot of people edit at the same time.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-05-22 09:21:53 +02:00
Robin Appelman f7153a3d0c
generalize the "lazy folder" concept
makes it easy to use lazy intialized folder in other places

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-05-11 15:36:13 +02:00
Roeland Jago Douma d5850eb28f
Merge pull request #18955 from adrb/swift_upload_large_objects
Large Object support for OpenStack Swift
2020-05-06 13:03:24 +02:00
Robin Appelman 4e6d3d1880
Fix getDirectoryContent implementation for Jail wrapper
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-30 15:27:09 +02:00
Roeland Jago Douma f6b435d0d9
Run the cs fixer for green ci
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-04-30 10:28:44 +02:00
Roeland Jago Douma a1c1b354fc
Merge pull request #18883 from adrb/optimize_swift_14116
Optimize Openstack Swift files download
2020-04-30 09:06:19 +02:00
Christoph Wurst cb057829f7
Update license headers for 19
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-29 11:57:22 +02:00
Adrian Brzezinski e107519295 Large Object support for OpenStack Swift.
Until now, you wouldn't be able to create
objects larger that 5GB.

It's somewhat related with pull #18883

Signed-off-by: Adrian Brzezinski <adrian.brzezinski@eo.pl>
2020-04-27 14:23:59 +02:00
Adrian Brzezinski 354c0996d8 It's a fix for issue #14116,
Improves efficiency when downloading files from Swift storage.
Before, files were downloaded and then pushed back to user.
That behaevior causes all kinds of performance problems.

Now, files are streamed directly to user.

Signed-off-by: Adrian Brzezinski <adrian.brzezinski@eo.pl>
2020-04-27 14:16:50 +02:00
Robin Appelman 436908ee3f
filter out blacklisted files in getDirectoryContent
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-23 13:34:55 +02:00
Robin Appelman 9735b5d61f
Add method to storage backends to get directory content with metadata
Currently you need to use `opendir` and then call `getMetadata` for
every file, which adds overhead because most storage backends already
get the metadata when doing the `opendir`.

While storagebackends can (and do) use caching to relief this problem,
this adds cache invalidation dificulties and only a limited number of
items are generally cached (to prevent memory usage exploding when
scanning large storages)

With this new methods storage backends can use the child metadata they
got from listing the folder to return metadata without having to keep
seperate caches.

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-20 15:45:04 +02:00
Roeland Jago Douma 79cf13e349
Merge pull request #16946 from exussum12/patch-1
Increase max read
2020-04-16 10:37:10 +02:00
Scott Dutton 65e134c13a
Update View.php 2020-04-15 19:10:06 +01:00
Morris Jobke 7fe8477830
Run cs:fix and fix HashWrapper code
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-04-15 14:08:56 +02:00
Morris Jobke d8557d0d8a
Merge pull request #19540 from jlutran/fixKeystoneV3Auth
Fix issues with Keystone auth v3 in files_external app
2020-04-15 14:05:32 +02:00
Christoph Wurst 301327135c
Merge pull request #20361 from nextcloud/write-stream-close-on-exception
Close the streams in `writeStream` even when there is an exception
2020-04-15 10:48:24 +02:00
Julien Lutran 3e849daf11
Fix issues with Keystone auth v3 in files_external app
Signed-off-by: Julien Lutran <julien.lutran@corp.ovh.com>
2020-04-15 10:33:08 +02:00
Robin Appelman c2677682c4
Return hashes of uploaded content for dav uploads
hashes are set in "X-Hash-MD5", "X-Hash-SHA1" and "X-Hash-SHA256" headers.

these headers are set for file uploads and the MOVE request at the end of a multipart upload.

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-13 12:57:27 +02:00
Scott Dutton 75b99e85ff
Increase max read
8kb is very low, especially given this will be local files

Signed-off-by: Scott Dutton <scott@exussum.co.uk>
2020-04-12 13:01:29 +01:00
Christoph Wurst 734c62bee0
Format code according to PSR2
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:56:50 +02:00
Christoph Wurst 28f8eb5dba
Add visibility to all constants
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:54:27 +02:00
Christoph Wurst 1584c9ae9c
Add visibility to all methods and position of static keyword
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:51:06 +02:00
Christoph Wurst a7c8d26d31
Add visibility to all properties and move static keyword
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:48:31 +02:00
Christoph Wurst caff1023ea
Format control structures, classes, methods and function
To continue this formatting madness, here's a tiny patch that adds
unified formatting for control structures like if and loops as well as
classes, their methods and anonymous functions. This basically forces
the constructs to start on the same line. This is not exactly what PSR2
wants, but I think we can have a few exceptions with "our" style. The
starting of braces on the same line is pracrically standard for our
code.

This also removes and empty lines from method/function bodies at the
beginning and end.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 14:19:56 +02:00
Christoph Wurst 14c996d982
Use elseif instead of else if
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 10:35:09 +02:00
Roeland Jago Douma fb64fda4d8
Merge pull request #20385 from nextcloud/skeeable-http-harden
harden seekable http stream a bit against failures
2020-04-10 08:34:52 +02:00
Christoph Wurst 44577e4345
Remove trailing and in between spaces
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 16:07:47 +02:00
Christoph Wurst 42625a46be
Remove spaces after method or function call
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 16:05:56 +02:00