48653d1a27
Merge pull request #16440 from marcelklehr/fix/objectstorage-put-contents
...
Fix File#putContents(string) on ObjectStorage
2019-07-17 22:38:41 +02:00
d46744e2f1
Fix File#putContents(string) on ObjectStorage
...
Signed-off-by: Marcel Klehr <mklehr@gmx.net>
2019-07-17 14:58:56 +02:00
8ca2b31804
Do not keep searching for recent
...
If userA has a lot of recent files. But only shares 1 file with userB
(that has no files at all). We could keep searching until we run out of
recent files for userA.
Now assume the inactive userB has 20 incomming shares like that from
different users. getRecent then basically keeps consuming huge amounts
of resources and with each iteration the load on the DB increases
(because of the offset).
This makes sure we do not get more than 3 times the limit we search for
or more than 5 queries.
This means we might miss some recent entries but we should fix that
separatly. This is just to make sure the load on the DB stays sane.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-07-16 19:14:55 +02:00
e953205908
Use HTTP1.1 to read S3 objects
...
Some of the READs otherwise use HTTP/1.0 which is not always supported
by all backends. HTTP/1.1 is there since 1999 way longer than S3 so safe
to assume it is always there IMO.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-07-10 11:42:22 +02:00
5a03189ce7
Mimetype list integrity check should not fail if it's changed ( #15810 )
...
Mimetype list integrity check should not fail if it's changed
2019-07-07 20:01:58 +02:00
5312a07f55
Merge pull request #16107 from nextcloud/local-check-path
...
verify that paths are valid for recursive local move
2019-06-27 16:36:30 +02:00
f39605dabc
verify that paths are valid for recursive local move
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-06-27 11:10:08 +02:00
2f5810f26e
Merge pull request #15832 from nextcloud/bugfix/noid/fulltext-search-groupfolders
...
Fix full text search for groupfolders
2019-06-25 21:43:37 +02:00
9dac9710e4
Fix full text search for groupfolders
...
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2019-06-17 12:52:22 -01:00
9cff71f5b0
Update Encryption.php
...
Signed-off-by: martink-p <47943787+martink-p@users.noreply.github.com>
2019-06-17 12:13:29 +02:00
8b22ede5f3
Update Encryption.php
...
Signed-off-by: martink-p <47943787+martink-p@users.noreply.github.com>
2019-06-17 12:13:29 +02:00
39a4fe40a3
Apply suggestions from code review
...
Co-Authored-By: Daniel Kesselberg <mail@danielkesselberg.de>
Signed-off-by: martink-p <47943787+martink-p@users.noreply.github.com>
2019-06-17 12:13:29 +02:00
4b9675df6c
Update Encryption.php
...
Signed-off-by: martink-p <47943787+martink-p@users.noreply.github.com>
2019-06-17 12:13:29 +02:00
b4e2d8f8ad
handle storage exceptions when trying to set mtime
...
not all storage backends can handle setting the mtime and they might
not always handle that error correctly.
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-06-06 16:09:27 +02:00
d227f4d34c
Do not fail integrity check if mimetype list is changed
...
Signed-off-by: Xheni Myrtaj <myrtajxheni@gmail.com>
2019-05-29 22:43:23 +02:00
58fe0b8032
Merge pull request #15776 from nextcloud/get-by-id-shares-last
...
search files by id in shared storages last
2019-05-29 21:56:15 +02:00
98dd526010
search files by id in shared storages last
...
this prevents unneeded recursion when the file is not in the share
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-05-28 20:03:45 +02:00
67dd4b018a
Check for free space on touch
...
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2019-05-28 13:06:59 +02:00
96e892770d
Merge pull request #15715 from nextcloud/s3-bucket-name
...
Allow other code to use the s3 client from object store
2019-05-24 15:03:22 +02:00
834db3af2c
add method to get bucket name from s3 storage
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-05-24 14:02:03 +02:00
06a119242e
make ObjectStoreStorage::getURN public
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-05-22 12:17:33 +02:00
439b341e89
add method to get the used object store from the storage
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-05-21 17:18:00 +02:00
6a30c65a32
remove extra param in cache update call
...
param was leftover from earlier iterations of the "update event"
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-05-07 14:53:02 +02:00
009c2dfbd5
take return of mkdir into account, throw exception on failure
...
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-04-29 16:45:59 +02:00
dedb84acb0
Always use a temp file for swift writes
...
Apparently the if statement doesn't work in all cases (even if I could
not reproduce it). So for the time being we will just not directly
stream to swift.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-04-18 21:04:27 +02:00
5d360bd16f
Harden appdata putcontent
...
If for whatever reason appdata got into a strange state this will at
least propegate up and not make it do boom the next run.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-04-08 13:22:00 +02:00
1c3abda280
Store user object later in the code
...
Signed-off-by: Vlastimil Pecinka <pecinka@email.cz>
2019-03-21 21:57:47 +01:00
cc29f5d719
always allow moving mountpoints inside the same mountpoint
...
even if that mountpoint isn't normally a valid target for moving mounts into
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-03-19 15:56:18 +01:00
6f88f28a15
Merge pull request #14689 from nextcloud/rename-moveable-mount-readonly-parent
...
allow renaming the root of a movable mount even if the parent folder …
2019-03-15 16:48:34 +01:00
17d5e47937
Remove spaming debug logs
...
Signed-off-by: Joas Schilling <coding@schilljs.com>
2019-03-15 12:13:44 +01:00
3ca1e90044
allow renaming the root of a movable mount even if the parent folder is readonly
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-03-14 15:27:25 +01:00
631ae17dce
handle long etags from dav external storage
...
we can only store etags up to 40 characters long in the database, so when we get an etag that's longer we simply hash it to bring down the length
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-03-14 14:46:39 +01:00
772303309d
Merge pull request #14425 from cowai/fix-multiple-incomplete-folders
...
Do not calculate folder size for parent that also needs proper scan, fixes #3524
2019-03-08 09:12:33 +01:00
1b27e9578d
Merge pull request #14424 from nextcloud/fix/13554/swift_to_tmp
...
Use a tmp file for swift writes
2019-03-06 15:33:15 +01:00
d16cfb519e
Avoid calculating folder size for parent that needs scan.
...
Signed-off-by: Ari Selseng <ari@selseng.net>
2019-03-06 15:23:37 +01:00
debd32b461
Merge pull request #14175 from skydiablo/master
...
Fix ObjectStorage expired token (v3)
2019-03-05 18:59:45 +01:00
1153123b90
Use a tmp file for swift writes
...
Else this leads to a seekable stream error with chunked uploads
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-03-04 13:11:59 +01:00
c778032276
Properly handle null as ownerId if file system access is denied by access token
...
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2019-03-01 20:57:01 +01:00
01b4db62fb
Add dispatcher events to User and Group objects
...
Signed-off-by: Joas Schilling <coding@schilljs.com>
2019-03-01 20:56:59 +01:00
df55781814
Merge pull request #14254 from sparrowjack63/master
...
Dont check Bucket Name in Nextcloud
2019-03-01 13:46:04 +01:00
e3142a91b2
Bucket name - Alert in debug mode
...
Alert in debug mode only when trying to mount non compatible DNS bucket name (in order to not flood the logs)
Signed-off-by: Sébastien Cat <sebastien.cat@inra.fr>
2019-03-01 11:28:54 +01:00
381ae44d38
Correct code indent
...
Just correct code indent.
Signed-off-by: Sébastien Cat <sebastien.cat@inra.fr>
2019-03-01 11:28:54 +01:00
77473b0228
Warning on S3 not compatible DNS bucket name
...
The new created bucket should respect the DNS compatibility, nevertheless, Nextcloud should accept to mount "old created" buckets that does not respect DNS compatibility (Backward compatibility, or compatibility with CEPH).
Signed-off-by: Sébastien Cat <sebastien.cat@inra.fr>
2019-03-01 11:28:54 +01:00
59c3d8d85f
Dont check Bucket Name in Nextcloud
...
Signed-off-by: Sebastien Cat <sparrow.jack63@gmail.com>
Signed-off-by: Sébastien Cat <sebastien.cat@inra.fr>
2019-03-01 11:28:54 +01:00
8fc47c6f00
add option to get raw size (without submounts) from fileinfo
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-02-27 15:35:44 +01:00
50d7f87eab
Merge pull request #14269 from nextcloud/cache-update-event
...
add event for cache update
2019-02-19 11:07:59 +01:00
092f78d7a4
add event for cache update
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-02-18 16:50:41 +01:00
f6f002e2ad
Merge pull request #14210 from nextcloud/fix/14192/fix_empty_uploads
...
Fix empty file uploads to S3 (and other streaming storages)
2019-02-18 15:39:17 +01:00
104616c1da
Use CountWrapper instead of almost identical CountReadStream
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-02-15 13:24:58 +01:00
61d66d7954
Fix empty file uploads to S3
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-02-15 08:41:39 +01:00
62ddb124a1
Filter out versions and trashbin
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-02-13 21:27:25 +01:00
cb67c6aa00
Loop for the recent search
...
Else it might happen that less entries than requested are returned
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-02-13 21:22:16 +01:00
4c7424d38b
We need to initialize the storage else the jailed path is always null
...
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-02-13 21:01:17 +01:00
d06c25b9c8
Update SwiftFactory.php
...
Auth-Service (in explizit v3) will recheck the cached-token and will end in an "token expired exception".
2019-02-13 12:26:27 +01:00
baa6a2d52c
Merge pull request #13748 from nextcloud/cache-insert-event
...
add event for inserting cache entries
2019-02-07 16:35:02 +01:00
748bcd407b
add event for inserting cache entries
...
this provides a reliable way for apps to listen to new files
without the need to of cache wrappers to hook into inserts themselves
(something which isn't 100% reliable)
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-02-07 15:51:30 +01:00
c26bdd4731
Fix: Check if `$this->params['user']` is an array
...
Signed-off-by: William Pain <pain.william@gmail.com>
2019-02-05 09:56:42 +01:00
1d322d3b37
always use multipart uploader for s3 uploads
...
the multipart uploader handles non seekable streams while `upload` does not
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-01-28 14:51:19 +01:00
90e985fe4f
Merge pull request #13739 from nextcloud/cache-cleanup-change
...
cleanup shared lock if changing to exclusive lock failed
2019-01-22 16:11:46 +01:00
f69c2d1e84
cleanup shared lock if changing to exclusive lock failed
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-01-22 15:32:48 +01:00
c3b22f05bb
Merge pull request #13468 from rummatee/issue13276
...
don't close input stream when writing in encrypted file
2019-01-14 16:21:08 +01:00
5a27e54f4b
Merge pull request #13032 from nextcloud/objectstore-write-exists
...
upload new files in objectstore to a .part path first
2019-01-14 11:26:29 +01:00
615fb8cd77
Cache tokens when using swift's v2 authentication
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-01-13 19:59:48 +01:00
c2f8df0573
don't close input stream when writing in encrypted file
...
Signed-off-by: Florian Schunk <florian.schunk@rwth-aachen.de>
2019-01-10 00:25:07 +01:00
d0956c9a42
Followup 12833, gracefully handle the getting of /
...
Else this breaks the app page
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-12-21 11:08:19 +01:00
6788e6e75c
Merge pull request #12883 from nextcloud/appdata-performance
...
try to grab the appdata folder directly without going trough the whole tree
2018-12-17 15:11:28 +01:00
d6bf5d4384
upload new files in objectstore to a .part path first
...
This prevent the object store and cache from getting out of sync
when an objectstore silently fails or the php process get's killed
during the upload without giving us the chance to cleanup
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-12-12 15:24:40 +01:00
6c9f2644cf
Add objectExists to objectstore interface
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-12-12 14:17:19 +01:00
740ea0d30e
try to grab the appdata folder directly without going trough the whole tree
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-12-06 15:10:08 +01:00
60681decdd
Merge pull request #12865 from nextcloud/fix/do_not_propogate_in_appdata
...
No need to propogate changes in appdata
2018-12-06 08:50:22 +01:00
ecc8068e66
Merge pull request #12857 from nextcloud/fix/emit_write_update_once_on_touch
...
Emit write update only once on touch
2018-12-05 22:54:47 +01:00
72f3882d4b
No need to propogate changes in appdata
...
Right now we propogate a lof of changes in appdata. So for example we
propogate each and every preview that is added to the system. This has
no real added value as far as I can tell.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-12-05 22:06:19 +01:00
2c6d36105c
Emit write update only once on touch
...
When a touch is done (i.e. when creating a new file in the node API)
The hooks contain 'write' and 'touch' (and the operation is touch as
well). This would cause a double write update. Which doesn't make sense.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-12-05 21:29:48 +01:00
f7a1ed8656
Only execute query in propagateChange once
...
The second execute statement should be inside the if block. Else it gets
executed twice which makes no sense.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-12-05 20:32:02 +01:00
9d9f6903c9
Fix folder path containing leading slash when getting mount root by id
...
This fixes collabora on public link shared groupfolders
Fixes https://github.com/nextcloud/groupfolders/issues/225
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-11-28 15:18:29 +01:00
cc6c30e769
Merge pull request #12494 from nextcloud/cache-notfound-id
...
return the correct value when trying to get a non existing item from cache by id
2018-11-20 09:51:18 +01:00
cef8eadf2a
Add PHPDoc for thrown exception
...
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-11-19 15:34:07 +01:00
35251928d5
forward object not found error in switch as dav 404
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-11-19 11:34:38 +01:00
5d5cfefd35
return the correct value when trying to get a non existing item from cache by id
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-11-16 17:26:42 +01:00
1b85ef4bf2
Fix string doc type casing
...
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2018-11-15 11:57:05 +01:00
fd8eeccb15
Merge pull request #11942 from nextcloud/techdebt/noid/use-count-function
...
Use the defined func()->count() instead of manual counting
2018-11-12 17:43:56 +01:00
93c62d78db
Fix UniqueConstraintViolationException while insert into oc_filecache
...
* fixes #6160 by not being prone to the race condition in insertIfNotExists
* fixes #12228 by not using a query that can result in a deadlock
* replaces the insertIfNotExists call with an insert which is wrapped into a try-catch block
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-11-12 14:43:28 +01:00
bb352fb667
Use the defined func()->count() instead of manual counting
...
Signed-off-by: Joas Schilling <coding@schilljs.com>
2018-11-08 15:44:45 +01:00
0277cd7158
Properly search the root of a shared external storage
...
Fixes #1020
When an external storage is shared with you in full the root is ''
(empty). This adds an extra check for an empty jail basically.
Because if the jail is on the empty string. It matches all paths.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-11-05 16:27:43 +01:00
9b3cc72f7c
fix writeStream for jail wrapper
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-10-31 21:10:57 +01:00
c6a48110bf
allways fall back to fopen for encryption wrapper
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-10-31 21:10:51 +01:00
93de63777e
extend storage api to allow directly writing a stream to storage
...
this removes the need for temporary storages with some external storage backends.
The new method is added to a separate interface to maintain compatibility with
storage backends implementing the storage interface directly (without inheriting common)
Currently the interface is implemented for objectstorage based storages and local storage
and used by webdav uploads
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-10-31 21:10:44 +01:00
e5c3e4b76f
Add getExtension() to FileInfo
...
this is a fairly common operation so it makes sense to prevent having
to repeatedly implement it.
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-10-29 15:24:02 +01:00
0fb070b5b4
Allow chunked uploads even if your quota is not sufficient
...
Fixes #11485
This allows uploads to shared folders.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-10-24 20:29:00 +02:00
6c5ea0ceca
pass `Cache::getMoveInfo` along cache wrappers
...
fixes cross storage move in some cases
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-10-17 14:57:33 +02:00
565d830c11
expose storagefactory in server container
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-10-17 14:57:30 +02:00
ea42474b31
use name in fileinfo if set
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-10-17 14:56:51 +02:00
51e96dc3f6
Normalize getUnjailedPath
...
Fixes #11637
If we do not normalize the unjailed path we might end up with a path
like files/user/folder/. which can break on objectstores
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-10-11 14:43:48 +02:00
d5bf2c4523
Move normalizePath to regexes instead of looping
...
This is IMO a bit more readable and it seems to make the code faster.
Tested it on the company instance where there are over 3k calls to this
function. It shaves off around 10ms.
The advantage here is that the pattern gets optimized by php itsel and
cached.
Also looking for all patterns at the same time and especially no longer
looping for /./ patterns should save time.
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-10-10 13:23:59 +02:00
4c0cbdff36
tokens can't be valid local user names
...
this saves searching for shares on non-public link dav requests
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-16 21:27:57 +02:00
c300516e51
more efficient way to detect added and removed mounts
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-16 19:56:49 +02:00
f7ae235372
cache OC\Files\Mount\Manager::findIn results
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-16 19:02:00 +02:00
f8116ad4cf
use more efficient method to find mountpoint for path
...
this changes the complexity from the number of mounts to the depth of
the path
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-16 18:55:52 +02:00
95981810c0
remove double loop for detecting changed mounts
...
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-16 18:24:56 +02:00
5e9d6b1585
use insertIfNotExists to store new mimetypes.
...
Also throw an error if we can't find the mimetype after insert
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-09 15:20:55 +02:00
Morris Jobke
Marcel Klehr
Roeland Jago Douma
John Molakvoæ
Roeland Jago Douma
Robin Appelman
martink-p
Xheni Myrtaj
Julius Härtl
Arthur Schiwon
Vlastimil Pecinka
Joas Schilling
Ari Selseng
S. Cat
Sebastien Cat
Volker
William Pain
Florian Schunk
Joas Schilling
Christoph Wurst