Commit Graph

763 Commits

Author SHA1 Message Date
Robin Appelman 9d4848e863
use in objectstore copy
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-11-24 15:16:58 +01:00
Joas Schilling cffad62771
Empty string is returned as null, but empty string in file cache is the root and exists
Signed-off-by: Joas Schilling <coding@schilljs.com>
2020-11-10 15:35:43 +01:00
Joas Schilling 8027dcbc6f
Don't leave cursors open when tests fail
Signed-off-by: Joas Schilling <coding@schilljs.com>
2020-11-09 12:28:17 +01:00
John Molakvoæ 72545ffd07
Merge pull request #23937 from nextcloud/bugfix/noid/files-ocdb
Use query builder instead of OC_DB in OC\Files\*
2020-11-09 11:58:36 +01:00
Morris Jobke 28464f71a2
Merge pull request #23982 from nextcloud/fix/quota-storage-getsize
Make sure root storage is valid before checking its size
2020-11-09 11:03:25 +01:00
Morris Jobke 005a132938
Merge pull request #23741 from Ashod/lock-leak
Unlock when promoting to exclusive lock fails
2020-11-09 11:00:46 +01:00
Joas Schilling c07720de77
Use query builder instead of OC_DB in OC\Files\*
Signed-off-by: Joas Schilling <coding@schilljs.com>
2020-11-09 09:05:15 +01:00
John Molakvoæ (skjnldsv) 6a453ac184
Make sure root storage is valid before checking its size
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2020-11-09 08:49:30 +01:00
Roeland Jago Douma a5fa9604ba
Merge pull request #23923 from nextcloud/bugfix/oci/fix-greatest-order-for-oracle
Fix greatest/least order for oracle
2020-11-07 16:55:13 +01:00
Ashod Nakashian a66fb45c0b Unlock when promoting to exclusive lock fails
In certain cases changeLock to EXCLUSIVE fails
and throws LockedException. This leaves the
file locked as SHARED in file_put_contents,
which prevents retrying (because on second
call file_put_contents takes another SHARED
lock on the same file, and changeLock doesn't
allow more than a single SHARED lock to promote
to EXCLUSIVE).

To avoid this case, we catch the LockedException
and unlock before re-throwing.

Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
2020-11-07 10:31:26 -05:00
Joas Schilling 4b125d4402
Use Query builder function
Signed-off-by: Joas Schilling <coding@schilljs.com>
2020-11-06 12:13:22 +01:00
Joas Schilling 3c027bd0cf
Fix order of GREATEST for Oracle
As per https://docs.oracle.com/cd/B19306_01/server.102/b14200/functions060.htm
Oracle uses the first value to cast the rest or the values.
So when the first value is a plain int, instead of doing the math,
it will cast the expression to int and continue with a potential 0.

Signed-off-by: Joas Schilling <coding@schilljs.com>
2020-11-06 12:13:22 +01:00
Roeland Jago Douma 65141d4864
Allow config to specify the bucket exists
In the 99% case the bucket is just always there. And if it is not the
read/write will fail hard anyways. Esp on big instances the Objectstore
is not always fast and this can save a few hundered ms of each request
that acess the objectstore.

In short it is adding

'verify_bucket_exists' => false

To the S3 config part

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-11-06 11:08:38 +01:00
Morris Jobke dc479aae2d
Improve CertificateManager to not be user context dependent
* removes the ability for users to import their own certificates (for external storage)
* reliably returns the same certificate bundles system wide (and not depending on the user context and available sessions)

The user specific certificates were broken in some cases anyways, as they are only loaded if the specific user is logged in and thus causing unexpected behavior for background jobs and other non-user triggered code paths.

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-11-03 00:13:01 +01:00
Christoph Wurst 0da89c0fa1
The file cache entry is array accessible
… so let's make this part of our API. We use it as array already.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-10-13 10:47:00 +02:00
Christoph Wurst d9015a8c94
Format code to a single space around binary operators
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-10-05 20:25:24 +02:00
Morris Jobke d144a84df1
Merge pull request #22903 from nextcloud/techdebt/16696/no-need-to-get-full-path-again
Do not fetch the normalized full path again if it is already available
2020-09-17 11:54:47 +02:00
Morris Jobke 063ac9d353
Do not fetch the normalized full path again if it is already available
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-09-17 09:56:45 +02:00
Roeland Jago Douma b0c04a3e0c
Merge pull request #22867 from nextcloud/shared-storage-init-less
Shared storage optimizations
2020-09-17 08:55:35 +02:00
Morris Jobke 99c9423766
Remove @suppress SqlInjectionChecker
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-09-16 15:53:56 +02:00
Robin Appelman cfde74442c
optimize View::getPath if we already know the storage id
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-09-16 13:49:07 +02:00
Robin Appelman 244fd6ae49
disable update checking for shared storages
update checking is already handled by the share source

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-09-15 16:42:40 +02:00
Roeland Jago Douma 6895d97cf0
Merge pull request #22520 from nextcloud/normalize-path-invalid-utf8
dont use `false` as cache key for non utf8 path in normalizePath
2020-09-09 21:52:36 +02:00
Roeland Jago Douma bb06b6cce4
Fix reading empty files from objectstorage
Since we try to do range requests this will fail hard.
However since empty files are not that interesting to read anyways we
just read from an emptry memory stream.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-09-09 20:45:13 +02:00
Morris Jobke cd563023db
Merge pull request #22657 from nextcloud/bugfix/noid/quota-trash-creation
Check if quota should be applied to path when creating directories
2020-09-09 17:37:28 +02:00
Morris Jobke d8dea5070b
Merge pull request #22729 from nextcloud/fix/object_storage_permissions_shortcut
Run stat less often for objectstorages
2020-09-09 10:43:37 +02:00
Roeland Jago Douma 9fdeed8cc0
Run stat less often for objectstorages
When we want to get the permissions we now do stat at least 5 times for
each entry. Which is a bit much. Especially since the permssions are all
just in the database already.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-09-08 14:38:36 +02:00
Roeland Jago Douma 919a8d473b
Merge pull request #22646 from nextcloud/bugfix/mimetype-mapping
Fix detecting text/x-php mimetype and secure mimetype mapping
2020-09-08 10:41:09 +02:00
Julius Härtl 87e5fd0d2c
Check if quota should be applied to path when creating directories
This fixes an issue where the files_trashbin hierarchy of a user could
not been created as the mkdir operations were blocked by the quota
storage wrapper. Even with 0 quota, users should be able to have a
trashbin for external storages.

Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-09-08 07:33:57 +02:00
Julius Härtl 5e4303b1fe
Properly add both mimetypes to secure mimetype mapping
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-09-07 16:42:11 +02:00
Christoph Wurst 1f7f93a695
Update license headers for Nextcloud 20 (again)
There are still lots of outdated headers, so time for another round of
updates.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-09-07 14:37:44 +02:00
Roeland Jago Douma d0a7f833cc
Merge pull request #22116 from nextcloud/bugfix/noid/transfer-ownership-share-root
Fix share transfer of single files and on the transfered node
2020-09-01 08:50:22 +02:00
Roeland Jago Douma e0d767d3e1
Merge pull request #16632 from nextcloud/bugfix/external-reshare
Set proper root path for single file shares originating from other storages
2020-09-01 08:48:31 +02:00
Roeland Jago Douma 8dd249937f
Merge pull request #22514 from nextcloud/fix/s3/only_delete_new_failed
Fix S3 error handling
2020-08-31 19:03:24 +02:00
Robin Appelman a792a51dca
dont use `false` as cache key for non utf8 path in normalizePath
since `json_encode` returns `false` if it's input isn't utf8, all non utf8 paths passed to normalizePath will currently return the same cached result.

Fixing this makes working with non utf8 storages a *little* bit more possible for apps

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-08-31 17:29:36 +02:00
Robin Appelman 81c5aa9819
set the mount id before calling storage wrapper
this allows the storage wrapper to use the mount id for it's own logic

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-08-31 16:38:47 +02:00
Julius Härtl ac2999a26a
Transfer shares of the transferred root node
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-08-31 14:23:34 +02:00
Roeland Jago Douma 789b33aba4
Only update the filecache entry once the file has been written to S3
If we already update before we have no way to revert if the upload
fails.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-08-31 12:28:04 +02:00
Roeland Jago Douma 6ffd7173f9
Don't lose filecache entry on s3 overwrite error
If the object store errors we should not always delete the filecache
entry. As this might lead to people losing access to their files.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-08-31 12:25:20 +02:00
Robin Appelman 2879472f81
trim slashes
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-08-28 11:14:50 +02:00
Julius Härtl 7512dc2b34
Set proper root path for single file shares originating from other storages
Co-authored-by: Christoph Wurst <christoph@winzerhof-wurst.at>
Signed-off-by: Julius Härtl <jus@bitgrid.net>
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-08-28 10:50:15 +02:00
Roeland Jago Douma 7e6f04490e
Merge pull request #22393 from nextcloud/fix/license-headers-20
Update the license headers for Nextcloud 20
2020-08-24 22:03:00 +02:00
Joas Schilling df857604d5
Merge pull request #21894 from nextcloud/fix-file-quota
fix: file quota was not applied in all cases
2020-08-24 15:48:48 +02:00
Christoph Wurst 2a054e6c04
Update the license headers for Nextcloud 20
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-08-24 14:54:25 +02:00
Roeland Jago Douma 6e4b089265
Merge pull request #20891 from cuppett/cuppett/issue#19790
Resolves #19790, Provides Support for IAM Credentials
2020-08-20 20:28:05 +02:00
korelstar d8a80cd45c
fix: file quota was not applied in all cases 2020-08-20 16:11:50 +02:00
Stephen Cuppett 5ef0f86ce7
Resolves #19790, Provides Support for IAM Credentials
Includes support for either leveraging environment variables
passed to the PHP runtime or IAM instance profile present
on the host being used. The default and first choice is
still the parameter file as documented.

See also: https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/guide_credentials_provider.html#chaining-providers

Signed-off-by: Stephen Cuppett <steve@cuppett.com>
2020-08-20 15:54:33 +02:00
Robin Appelman 5af7d921a9
Make Cache::removeChildren non recursive
Currently the "add new files during scanning" call stack is smaller than
the "remove deleted files during scanning" call stack. This can lead to
the scanner adding folders in the folder tree that are to deep to be
removed.

This changes the `removeChildren` logic to be non recursive so there is
no limit to the depth of the folder tree during removal

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-08-20 15:37:02 +02:00
Roeland Jago Douma af3a59fab5
Merge pull request #22237 from nextcloud/bugfix/noid/allow_putContent_empty_string
Allow writing empty content to new file
2020-08-20 09:03:06 +02:00
Morris Jobke d8bdb439a4
Merge pull request #22289 from nextcloud/techdebt/noid/fix-encryption-stream-invalid-scalar-arguments
Cast float to int to avoid invalid scalar argument warning
2020-08-19 22:08:15 +02:00