Commit Graph

441 Commits

Author SHA1 Message Date
Morris Jobke 63bc633d89
Merge pull request #8857 from nextcloud/swift-verify-cached-token
verify cached swift token
2018-03-19 10:10:04 +01:00
Robin Appelman 41954d2903
verify cached swift token
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-03-16 15:20:16 +01:00
Roeland Jago Douma fa886ef4eb
Merge pull request #8830 from nextcloud/node_emit_read
Emit read for nodes
2018-03-15 14:04:14 +01:00
Roeland Jago Douma d6b326650b
Emit read for nodes
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-03-15 11:18:39 +01:00
Roeland Jago Douma d3e7996e80
Fix bucket and container usage
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-03-14 13:57:14 +01:00
Roeland Jago Douma 4e8d5504d2
Properly cache swift v3 token
The V3 token has an export function. Without this the token is
effectively never cached.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-03-12 20:53:50 +01:00
Roeland Jago Douma d05469959e
Merge pull request #8715 from nextcloud/objectstore-no-part-files
disable part files for object stores
2018-03-08 20:52:16 +01:00
Robin Appelman fc35352122
disable part files for object stores
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-03-07 13:33:35 +01:00
Robin Appelman 9764e70494
don't read existing file when overwriting using object store
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-03-07 12:05:57 +01:00
Joas Schilling 158aac00a0
Merge pull request #8289 from nextcloud/appdata_hardening
AppData hardening
2018-03-02 11:26:36 +01:00
Roeland Jago Douma 7f736923d9
Merge pull request #8284 from nextcloud/lib_log-storage-test-failed
External storage test: log exceptions
2018-03-01 20:32:09 +01:00
Roeland Jago Douma 8b5997483c
Hardening of SimpleFile getContent
if file_get_contents fails remove the file. And traverse up the tree
checking if the other folders are there.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-02-27 13:35:09 +01:00
Roeland Jago Douma 812ea354d0
User proper username + pass
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-02-26 14:54:18 +01:00
Robin Appelman 31dd6d6aae
add support for v3 swift auth
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-02-26 14:54:08 +01:00
Robin Appelman 817f2f8f49
adjust swift external storage to new openstack sdk
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-02-19 19:41:16 +01:00
Robin Appelman 3192efc415
adjust swift object storage to new openstack sdk
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-02-19 19:41:16 +01:00
Morris Jobke 24f96513fd
Merge pull request #8259 from nextcloud/guzzle6
update guzzlehttp/guzzle to 6.3.0
2018-02-14 22:26:19 +01:00
Morris Jobke 236086c457
Merge pull request #8335 from nextcloud/remove-unused-import
Remove unused import statements
2018-02-14 22:23:07 +01:00
Morris Jobke d3d045dd5c
Remove unused import statements
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-02-14 16:55:43 +01:00
Morris Jobke e2974f1133
Simplify return statement
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-02-13 21:55:24 +01:00
Roland Tapken dc99442bc1 External storage test: log exceptions
If an external storage test fails the exception's message is now written
into the logfile (level INFO). Additionally, the stack trace is printed
as DEBUG. This helps to resolve the reason for a failing connection.

Signed-off-by: Roland Tapken <roland@bitarbeiter.net>
2018-02-11 16:45:32 +01:00
Robin Appelman 97b44605f4
Adjust dav client to new guzzle
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-02-09 17:13:24 +01:00
Roeland Jago Douma 3e0ef7e3f2
Try to silence phan
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-02-08 15:18:26 +01:00
Robin Appelman e34c20af7d
don't swallow exception from touch
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-02-08 15:18:26 +01:00
Robin Appelman 85d23dd6fc
better error message on catalog error
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-02-08 15:18:26 +01:00
Morris Jobke 26e0c14c6d
Merge pull request #8033 from nextcloud/s3-uploader
Use S3Client::upload instead of splitting single/multipart upload ourselves
2018-02-08 14:30:02 +01:00
Morris Jobke e5c08621af
Merge pull request #8023 from nextcloud/webdavsearch_limit_order_fixing
Respect limit and order in webdav search
2018-02-08 09:11:39 +01:00
Robin Appelman fd09c06eb4
actually return stream from swift
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-02-07 17:10:59 +01:00
Roeland Jago Douma c1ff12e234
CacheJail should apply limit and offset after searching
Else the results might not be correct.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-02-07 13:41:15 +01:00
Roeland Jago Douma 53a780e31c
Do not try to get the jailed path if we can't find the id
Fixes #8047

If we can't find the file by id there we should just return null instead
of trying to get the jailed path of null.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-02-03 14:45:28 +01:00
Morris Jobke eb51f06a3b
Use ::class statement instead of string
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-01-29 12:03:47 +01:00
Morris Jobke a661f043e1
Remove unneeded semicolon and parentheses
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-01-26 23:46:40 +01:00
Morris Jobke 97c216ea06
Merge pull request #8071 from nextcloud/remove-silly-code
Cleanup unused code
2018-01-26 15:47:06 +01:00
Morris Jobke 4c38d1ed01
Merge pull request #8054 from nextcloud/substr-use-index
Use index based string access for substr with length of 1
2018-01-26 15:40:44 +01:00
Morris Jobke c005fc6755
Cleanup unused code
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-01-26 15:25:19 +01:00
Morris Jobke 9ff51aafc5
Use index based string access for substr with length of 1
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-01-26 13:38:34 +01:00
Morris Jobke 6bbea33133
Simplify ternary operator statements
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-01-26 12:36:25 +01:00
Joas Schilling c2b1bd92d6
Merge pull request #8063 from nextcloud/inline-value
Return value immediately instead of assigning to a one-time variable
2018-01-26 10:15:39 +01:00
Roeland Jago Douma 5a7a5e5352
Merge pull request #8058 from nextcloud/useless-return
Remove useless return statements
2018-01-26 09:51:26 +01:00
Morris Jobke 0a56d2185e
Return value immediately instead of assigning to a one-time variable
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-01-26 00:02:03 +01:00
Morris Jobke 16a558871c
Use proper code flow instead of not needed else branch
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-01-25 23:01:03 +01:00
Morris Jobke 4dbee99b7b
Remove useless return statements
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-01-25 22:50:40 +01:00
Robin Appelman 3499cbdb76
Use S3Client::upload instead of splitting single/multipart upload ourselves
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-01-24 17:22:05 +01:00
Roeland Jago Douma 4537faa5da
Merge pull request #7918 from nextcloud/properly-log-exceptions
Properly log the full exception instead of only the message
2018-01-23 11:56:02 +01:00
Morris Jobke 2a38605545
Properly log the full exception instead of only the message
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-01-23 10:57:21 +01:00
Morris Jobke 5c360a41e9
Merge pull request #7858 from nextcloud/fix-local-storage
Use fallback path if data dir is not available for Storage/Local.php
2018-01-23 10:52:41 +01:00
Morris Jobke bcba1a97ad
Use fallback path if data dir is not available for Storage/Local.php
Found while testing strict types for PHP7+.

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-01-18 16:38:01 +01:00
Robin Appelman 326d55db81
Support arbitrary number of arguments for d:or and d:and in search queries
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-01-16 13:22:28 +01:00
Roeland Jago Douma 8a41d05761
Remove deprecated \OCP\Config
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-01-13 14:25:04 +01:00
Morris Jobke 01040daf02
Fix call to clearstatcache to actually use the file path
* found while adding the strict_typing for PHP 7+ migration
* first argument is a boolean - second one is the path
* see http://php.net/manual/en/function.clearstatcache.php

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-01-12 13:59:58 +01:00
Robin Appelman 34ced4dd97
add option to use legacy v2 auth with s3
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-12-19 11:54:55 +01:00
Morris Jobke d2d73f1ce8
Also replace all other occurences
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2017-12-18 20:57:11 +01:00
Robin Appelman 4f0d03e172
additional debug logging on smb copy/rename failures
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-12-18 13:59:58 +01:00
Robin Appelman 05d2f61a59
handle not found mounts when cleaning avatars
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-12-11 16:56:10 +01:00
Roeland Jago Douma e47137c7d4
Merge pull request #6682 from nextcloud/dav-mimetype-fallback
Fallback to filename based detection if the remote dav server doesn't…
2017-12-11 15:28:05 +01:00
Morris Jobke c32cb6b4b7
Merge pull request #7385 from nextcloud/shared-delay-root
delay calculating the shared cache root until it's used
2017-12-06 15:14:48 +01:00
Robin Appelman 046eb0c635
add retry wrapper when reading files from swift
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-12-04 16:34:53 +01:00
Robin Appelman c87d689253
delay calculating the shared cache root until it's used
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-12-04 15:16:39 +01:00
Morris Jobke be4e921306
Use basename() instead of self implementation
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2017-11-23 12:35:47 +01:00
Morris Jobke 106d932e8f
Merge pull request #7137 from nextcloud/shared-jailed-source-root
Use the correct root for shared jail when the source storage is also a jail
2017-11-23 12:03:36 +01:00
Bjoern Schiessle 1ade6b0817
only create the file cache entry after the empty file was created successfully, otherwise file_exists() call on the initial file_put_content() will indicate that the file already exists
Signed-off-by: Bjoern Schiessle <bjoern@schiessle.org>
2017-11-17 12:45:46 +01:00
Björn Schießle c225bbd021
Merge pull request #7113 from nextcloud/s3-download-stream-fix
Fix s3 download and touch
2017-11-14 20:38:50 +01:00
Robin Appelman 56fd462718
Use the correct root for shared jail when the source storage is also a jail
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-11-10 17:03:27 +01:00
Morris Jobke 5d8421135e
Merge pull request #7014 from nextcloud/rename-locks
Unlock files even if an exception occurs while renaming
2017-11-09 09:47:36 +01:00
Lukas Reschke 989a8a3668
Merge pull request #6220 from nextcloud/cached-mount-info-filepath
Allow getting the filepath when getting cached mounts by fileid
2017-11-09 00:04:25 +01:00
Robin Appelman c566b2854b
Use non empty files for object store touch
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-11-08 17:51:02 +01:00
Robin Appelman 70a4860ccb
Fix contructing headers for s3 download
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-11-08 17:50:46 +01:00
Morris Jobke 19069f2b9c
Merge pull request #6988 from starypatyk/stable12-mount-cache-performance
Improve performance of UserMountCache with external storage folders
2017-11-07 17:32:06 +01:00
Morris Jobke 4866314ce0
Run updated license header updater
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2017-11-07 13:47:42 +01:00
Morris Jobke 31c5c2a592
Change @georgehrke's email
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2017-11-06 20:38:59 +01:00
Morris Jobke da6c2c9da1
Merge pull request #7083 from nextcloud/update-license-headers
Update license headers
2017-11-06 18:48:04 +01:00
Morris Jobke 0eebff152a
Update license headers
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2017-11-06 16:56:19 +01:00
Morris Jobke 8b763ea0f7
Better error message for invalid bucket names
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2017-11-06 09:48:41 +01:00
korelstar c7482402b8 Unlock files even if an exception occurs
Signed-off-by: Kristof Hamann <korelstar@users.noreply.github.com>
2017-11-03 18:29:04 +01:00
Roeland Jago Douma 687c92bfd3
Merge pull request #6974 from nextcloud/fix-fseek-on-objectstorage
Fix seeking on object storage
2017-11-02 14:56:51 +01:00
Dariusz Olszewski 07016a1e5b Improve performance of UserMountCache with external storage folders 2017-11-01 21:11:39 +01:00
Joas Schilling 3e0789aa94
Merge pull request #6958 from nextcloud/improved-mimetype-detection-for-object-storage
Improve mimetype detection for object storages
2017-11-01 11:04:15 +01:00
Joas Schilling defc4bf6a8
Merge pull request #6975 from nextcloud/s3-read-object-fopen
use fopen directly when reading objects from s3
2017-11-01 10:57:56 +01:00
Joas Schilling 94f3fc63bc
Use ::class and fix missing doc
Signed-off-by: Joas Schilling <coding@schilljs.com>
2017-10-28 11:49:36 +02:00
Robin Appelman 3f0cb13042
Allow getting the filepath when getting cached mounts by fileid
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-10-27 14:35:34 +02:00
Robin Appelman f9a7294807
use fopen directly when reading objects from s3
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-10-27 12:58:19 +02:00
Morris Jobke de912385e0
Fix seeking on object storage
Seeking is not needed if the $from is 0, because then the pointer is already at the correct position. Additionally another fallback is added, that if the fseek fails it just uses an fread to skip the beginning of the file until it is at the correct position. This skipping is done with a chunked fread.

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2017-10-27 12:16:28 +02:00
Morris Jobke 405bbc1c61
Improve mimetype detection for object storages
Object storage instances always fall back to the content based mimetype detection, because the file name for object storage was always random due to the fact that it was temporarily storage in a generated temp file. This patch adds a check before that to make sure to use the original file name for this purpose and also remove possible other extensions like the versioning or part file extension.

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2017-10-26 16:31:59 +02:00
Julius Härtl 4cfa1c66b8
Doc: Fix phpDoc issues
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2017-10-23 23:23:56 +02:00
Robin Appelman 91b3536f45
propagate multipart upload exception when aborting upload
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-10-17 15:10:58 +02:00
Roeland Jago Douma a5912cbe33 Merge pull request #6812 from nextcloud/upload-cache-path
also use configured 'cache_path' for new chunking
2017-10-15 21:42:41 +02:00
Robin Appelman e393b3553e
set s3 part size to 500mb
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-10-11 15:59:53 +02:00
Robin Appelman d27b0b07bc
also use configured 'cache_path' for new chunking
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-10-11 14:18:45 +02:00
Robin Appelman b36dd8b71f
Fallback to filename based detection if the remote dav server doesn't know the mimetype
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-09-28 14:22:42 +02:00
Robin Appelman 385d6f098c
Add tests for multipart upload
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-09-21 14:47:34 +02:00
Robin Appelman 4ae46d8876
only do multipart upload for large files
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-09-21 14:06:59 +02:00
Robin Appelman e4e5e735db
multipart upload for s3 object storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-09-20 17:51:58 +02:00
Robin Appelman d526969a68
fix path style
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-09-18 15:16:28 +02:00
Robin Appelman d70607104e
reuse object read/write/delete logic in s3 implementations
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-09-18 15:16:27 +02:00
Robin Appelman dad18baec8
update aws sdk and move it to 3rdparty
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-09-18 15:16:27 +02:00
William Pain 9a63ded43b
Fix uninitialized variable $this->params
Signed-off-by: William Pain <pain.william@gmail.com>
2017-09-11 10:01:12 +02:00
Morris Jobke c04a494ea7 Merge pull request #6064 from nextcloud/fix-5219-absolute-path-must-be-relative-to-files-on-theming-update
Still throw a locked exception when the path is not relative to $user/files/
2017-09-04 14:05:34 +02:00
Robin Appelman 804d97d6ff
unlock file when an exception occurs during scanning
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-08-26 18:58:06 +02:00
Robin Appelman 18908af87b
Dont swallow exception when inserting mimetypes if we're inside a transaction
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-08-26 18:58:06 +02:00
Joas Schilling b9b88aa666
Don't lock in the appdata_ directory
Signed-off-by: Joas Schilling <coding@schilljs.com>
2017-08-10 12:35:33 +02:00