0fb070b5b4
Allow chunked uploads even if your quota is not sufficient
...
Fixes #11485
This allows uploads to shared folders.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-10-24 20:29:00 +02:00
6c5ea0ceca
pass `Cache::getMoveInfo` along cache wrappers
...
fixes cross storage move in some cases
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-10-17 14:57:33 +02:00
565d830c11
expose storagefactory in server container
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-10-17 14:57:30 +02:00
ea42474b31
use name in fileinfo if set
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-10-17 14:56:51 +02:00
51e96dc3f6
Normalize getUnjailedPath
...
Fixes #11637
If we do not normalize the unjailed path we might end up with a path
like files/user/folder/. which can break on objectstores
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-10-11 14:43:48 +02:00
d5bf2c4523
Move normalizePath to regexes instead of looping
...
This is IMO a bit more readable and it seems to make the code faster.
Tested it on the company instance where there are over 3k calls to this
function. It shaves off around 10ms.
The advantage here is that the pattern gets optimized by php itsel and
cached.
Also looking for all patterns at the same time and especially no longer
looping for /./ patterns should save time.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-10-10 13:23:59 +02:00
4c0cbdff36
tokens can't be valid local user names
...
this saves searching for shares on non-public link dav requests
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-16 21:27:57 +02:00
c300516e51
more efficient way to detect added and removed mounts
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-16 19:56:49 +02:00
f7ae235372
cache OC\Files\Mount\Manager::findIn results
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-16 19:02:00 +02:00
f8116ad4cf
use more efficient method to find mountpoint for path
...
this changes the complexity from the number of mounts to the depth of
the path
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-16 18:55:52 +02:00
95981810c0
remove double loop for detecting changed mounts
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-16 18:24:56 +02:00
5e9d6b1585
use insertIfNotExists to store new mimetypes.
...
Also throw an error if we can't find the mimetype after insert
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-09 15:20:55 +02:00
11e99859ef
Merge pull request #10397 from nextcloud/encryption-s3-fix
...
make file cache updates more robust
2018-08-08 18:27:18 +02:00
01bbd28530
Do not log to error loglevel when converting the error
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-08-06 16:31:30 +02:00
a9e22c5f1c
make file cache updates more robust
...
only update the encrypted version after the write operation is finished and the stream is closed
Signed-off-by: Bjoern Schiessle <bjoern@schiessle.org>
2018-07-31 12:24:38 +02:00
5e940366c2
Do scan the root storage in background scan
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-07-24 14:15:27 +02:00
19c49c8473
Use the path_hash instead of the path to query the filecache
...
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-07-16 16:10:34 +02:00
cc9073c251
Merge pull request #10179 from nextcloud/mount-filter
...
Add the option to filter mounts for a user
2018-07-11 16:29:09 +02:00
6da2b7c4f6
Separate fopen into read and write methods
...
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2018-07-11 12:08:37 +02:00
f1469e34aa
Add fopen method to ISimpleFile
...
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2018-07-11 10:29:03 +02:00
3ad6084891
Add the option to filter mounts for a user
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-07-10 16:49:26 +02:00
5d0fa12e12
allow to disable encryption
...
Signed-off-by: Bjoern Schiessle <bjoern@schiessle.org>
2018-07-05 17:24:44 +02:00
48b7499fa4
Merge pull request #10079 from nextcloud/search-double-slash
...
fix double slash in search result path when the result is in a submount
2018-07-02 15:49:27 +02:00
922c851494
fix double slash in search result path when the result is in a submount
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-07-02 14:17:38 +02:00
b9e685236f
Add option to only scan the home storage
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-06-28 16:09:25 +02:00
73cf71a20f
Add resource type phpDoc for putContent methods
...
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2018-06-26 13:58:58 +02:00
587d7cc7d2
Merge pull request #9526 from Blaok/files-scan-shallow
...
allow shallow (non-recursive) scan when scanning file storage
2018-06-25 13:01:22 +02:00
54c4d95a11
Request a valid property for DAV opendir
...
Apperently Sabre and Onedrive are not friends when requesting a single
404 property. I need to dig deeper on why this is. Anyways requesting a
valid property makes it work like a charm.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-06-21 19:01:27 +02:00
47fd3499a3
Fix listing s3 objects when using google cloud aws compatibility
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-06-11 17:10:57 +02:00
49145a78e9
Make sure initial files can be properly created
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-06-07 12:07:37 +02:00
ac26175a17
add azure unit tests with azurite
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-06-04 16:18:03 +02:00
60b7cbc0f0
add azure object storage backend
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-06-04 10:57:52 +02:00
6b6d522969
Merge pull request #9395 from nextcloud/scanner-error-lock-cleanup
...
cleanup locks in scanner on error
2018-05-23 20:05:01 +02:00
21b9ad3978
feat: allow files:scan --shallow
...
Signed-off-by: Yuze Chi <i@blaok.me>
2018-05-18 14:50:32 -07:00
575dc29c27
Make Appdata static
...
* Add fileid for simpleroot folders (only internal)
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-05-14 13:25:54 +02:00
7639f680a2
Merge pull request #9397 from nextcloud/log-fileinfo-not-found
...
Debug log file not found when getting fileinfo
2018-05-07 16:43:38 +02:00
7f5567e758
Merge pull request #9388 from nextcloud/log-lock-state-on-conflict
...
Log lock state on conflict
2018-05-07 16:42:26 +02:00
5484260569
Merge pull request #9372 from nextcloud/bugfix/4577
...
Do not allow folder creation with quota of 0
2018-05-07 10:11:44 +02:00
0da6f6b356
Merge pull request #9371 from nextcloud/bugfix/7444
...
Cache is not part of the quota
2018-05-07 10:11:04 +02:00
e3de7ec867
Debug log file not found when getting fileinfo
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-05-04 18:48:12 +02:00
63caf331e5
cleanup locks in scanner on error
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-05-04 18:05:46 +02:00
fb296342b9
Merge pull request #9387 from nextcloud/getcachentry-lock
...
remove unneeded locks in getCacheEntry
2018-05-04 14:57:09 +02:00
94c15efad0
always log locking errors
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-05-04 13:30:43 +02:00
734aec1522
remove unneeded locks in getCacheEntry
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-05-04 12:51:41 +02:00
2780bb0b2b
Merge pull request #9364 from nextcloud/log-invalid-storage
...
Log mount and storage issues in getFileInfo
2018-05-04 12:01:33 +02:00
0ff83f7230
Do not allow folder creation with quota of 0
...
Fixes #4577
Users with a quota of 0 are a special case. Since they can't (ever)
create files on their own storage. Therefor it makes no real that they
can create folders (and possible share those etc).
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-05-03 11:31:23 +02:00
f2d05120b4
Cache is not part of the quota
...
Fixes #7444
Since the quota is calculated on the files. We should allow apps etc to
store temp stuff in the <user>/cache
Else users can't upload avatars for example.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-05-02 21:46:28 +02:00
d71783fb93
Log mount and storage issues in getFileInfo
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-05-02 15:11:59 +02:00
57ba18f47f
Use better way to determine stream type in encryption stream wrapper
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-05-01 15:06:21 +02:00
7306d30708
Merge pull request #9321 from nextcloud/bugfix/noid/return_root_folder_when_at_root
...
Actually return the root folder when traversing up the tree
2018-04-27 13:59:29 +02:00
a0e038f912
generateTokenFromCache is only valid for V3
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-04-26 20:51:05 +02:00
7318882357
json_decode an associative array for the swift cached token
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-04-26 20:47:47 +02:00
235e3480e6
Actually return the root folder when traversing up the tree
...
If you now keep calling $node->getParent() you will at some point get
the RootFolder back. This is a nice termination check and will prevent
endless loops if an exit condition is slightly off.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-04-26 20:04:19 +02:00
38a90130ce
move log constants to ILogger
...
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2018-04-26 10:45:52 +02:00
1ceb081c9b
Merge pull request #9280 from nextcloud/bugfix/noid/swift-v3-scope
...
Swift v3 requires setting a scope
2018-04-24 09:49:27 +02:00
ba71918ad2
Swift v3 requires setting a scope
...
Unscoped auth is not guaranteed to return a catalog. So require the
scope to be set properly.
See https://developer.openstack.org/api-ref/identity/v3/index.html#password-authentication-with-unscoped-authorization
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-04-23 19:34:09 +02:00
431ccb6386
Fix undefined variable - found by phan
...
Added in https://github.com/nextcloud/server/pull/8314/files#diff-a55c10804586cd797b7f3f16556c1457R157
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-04-23 11:34:41 +02:00
67dcff7a1f
Fix undefined variable - found by Phan
...
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-04-23 11:30:22 +02:00
2f068bc9d1
Add more logging for the object storage during creation of the buckets
...
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-04-05 17:26:09 +02:00
73e6eea57e
Fix tests
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-04-04 12:50:54 +02:00
37233471b6
Add pathcache
...
* If we find the mountpoint for a path cache it
* If we modify the mount points empty the pathCache
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-04-04 12:44:56 +02:00
6868da9958
Make normalized cache path larger
...
On larger instances with a large number of shares this can make a real
impact as the default 512 entries are easily filled.
Making this contain max 2048 entries has basically no effect on smaller
installations (as they probably never hit the 512 now). But makes sure
we don't evict entries in the larger case.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-04-04 12:44:52 +02:00
dc222aa3a5
Comparing stringlength is cheaper than strpos
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-04-04 12:44:28 +02:00
e48d4c4aad
Cache the $foundMountPointLength
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-04-04 12:44:28 +02:00
003c624596
Inspections
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-04-04 12:44:28 +02:00
7f1141f7e9
Make the MountManager strict
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-04-04 12:44:28 +02:00
c3900c9d0d
Merge pull request #9018 from nextcloud/fix-copy-encrypted-files
...
reset encryptionVersion to '1' if a file was stream copied
2018-04-03 18:24:27 +02:00
a0923d9ffb
reset encryptionVersion to '1' if a file was stream copied, because this means that we basically write the file from scratch
...
Signed-off-by: Bjoern Schiessle <bjoern@schiessle.org>
2018-04-03 18:01:23 +02:00
53447ae2b2
Throw an exception if file_put_contents fails
...
* This will help with AppData hardening etc
* Introduced a GenericFileException for if nothing else is appropiate
- Maybe the other File exceptions should base on this?
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-03-27 16:23:06 +02:00
4e1161d254
Fix proper permissions for multiple file access
...
Fixes #8890
In case you have access to a file via multiple ways, for example:
1. the file is shared with you with permission read only
2. the folder containing the file is shared with your read/write
Requesting the getById function on the userFolder would give back two
entries but both with the same permissions. Depending on the node you
picked this is not right.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-03-22 12:06:41 +01:00
47a3008141
Remove \OCP\Files::tmpFile
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-03-19 10:52:35 +01:00
63bc633d89
Merge pull request #8857 from nextcloud/swift-verify-cached-token
...
verify cached swift token
2018-03-19 10:10:04 +01:00
41954d2903
verify cached swift token
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-03-16 15:20:16 +01:00
fa886ef4eb
Merge pull request #8830 from nextcloud/node_emit_read
...
Emit read for nodes
2018-03-15 14:04:14 +01:00
d6b326650b
Emit read for nodes
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-03-15 11:18:39 +01:00
d3e7996e80
Fix bucket and container usage
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-03-14 13:57:14 +01:00
4e8d5504d2
Properly cache swift v3 token
...
The V3 token has an export function. Without this the token is
effectively never cached.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-03-12 20:53:50 +01:00
d05469959e
Merge pull request #8715 from nextcloud/objectstore-no-part-files
...
disable part files for object stores
2018-03-08 20:52:16 +01:00
fc35352122
disable part files for object stores
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-03-07 13:33:35 +01:00
9764e70494
don't read existing file when overwriting using object store
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-03-07 12:05:57 +01:00
158aac00a0
Merge pull request #8289 from nextcloud/appdata_hardening
...
AppData hardening
2018-03-02 11:26:36 +01:00
7f736923d9
Merge pull request #8284 from nextcloud/lib_log-storage-test-failed
...
External storage test: log exceptions
2018-03-01 20:32:09 +01:00
8b5997483c
Hardening of SimpleFile getContent
...
if file_get_contents fails remove the file. And traverse up the tree
checking if the other folders are there.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-02-27 13:35:09 +01:00
812ea354d0
User proper username + pass
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-02-26 14:54:18 +01:00
31dd6d6aae
add support for v3 swift auth
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-02-26 14:54:08 +01:00
817f2f8f49
adjust swift external storage to new openstack sdk
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-02-19 19:41:16 +01:00
3192efc415
adjust swift object storage to new openstack sdk
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-02-19 19:41:16 +01:00
24f96513fd
Merge pull request #8259 from nextcloud/guzzle6
...
update guzzlehttp/guzzle to 6.3.0
2018-02-14 22:26:19 +01:00
236086c457
Merge pull request #8335 from nextcloud/remove-unused-import
...
Remove unused import statements
2018-02-14 22:23:07 +01:00
d3d045dd5c
Remove unused import statements
...
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-02-14 16:55:43 +01:00
e2974f1133
Simplify return statement
...
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-02-13 21:55:24 +01:00
dc99442bc1
External storage test: log exceptions
...
If an external storage test fails the exception's message is now written
into the logfile (level INFO). Additionally, the stack trace is printed
as DEBUG. This helps to resolve the reason for a failing connection.
Signed-off-by: Roland Tapken <roland@bitarbeiter.net>
2018-02-11 16:45:32 +01:00
97b44605f4
Adjust dav client to new guzzle
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-02-09 17:13:24 +01:00
3e0ef7e3f2
Try to silence phan
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-02-08 15:18:26 +01:00
e34c20af7d
don't swallow exception from touch
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-02-08 15:18:26 +01:00
85d23dd6fc
better error message on catalog error
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-02-08 15:18:26 +01:00
26e0c14c6d
Merge pull request #8033 from nextcloud/s3-uploader
...
Use S3Client::upload instead of splitting single/multipart upload ourselves
2018-02-08 14:30:02 +01:00
e5c08621af
Merge pull request #8023 from nextcloud/webdavsearch_limit_order_fixing
...
Respect limit and order in webdav search
2018-02-08 09:11:39 +01:00
fd09c06eb4
actually return stream from swift
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-02-07 17:10:59 +01:00
c1ff12e234
CacheJail should apply limit and offset after searching
...
Else the results might not be correct.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-02-07 13:41:15 +01:00
Roeland Jago Douma
Robin Appelman
Björn Schießle
Morris Jobke
Julius Härtl
Blaok
Arthur Schiwon
Roeland Jago Douma
Joas Schilling
Roland Tapken