Commit Graph

1038 Commits

Author SHA1 Message Date
Robin Appelman 9735b5d61f
Add method to storage backends to get directory content with metadata
Currently you need to use `opendir` and then call `getMetadata` for
every file, which adds overhead because most storage backends already
get the metadata when doing the `opendir`.

While storagebackends can (and do) use caching to relief this problem,
this adds cache invalidation dificulties and only a limited number of
items are generally cached (to prevent memory usage exploding when
scanning large storages)

With this new methods storage backends can use the child metadata they
got from listing the folder to return metadata without having to keep
seperate caches.

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-20 15:45:04 +02:00
Julien Lutran 3e849daf11
Fix issues with Keystone auth v3 in files_external app
Signed-off-by: Julien Lutran <julien.lutran@corp.ovh.com>
2020-04-15 10:33:08 +02:00
Christoph Wurst 28f8eb5dba
Add visibility to all constants
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:54:27 +02:00
Christoph Wurst 1584c9ae9c
Add visibility to all methods and position of static keyword
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:51:06 +02:00
Christoph Wurst caff1023ea
Format control structures, classes, methods and function
To continue this formatting madness, here's a tiny patch that adds
unified formatting for control structures like if and loops as well as
classes, their methods and anonymous functions. This basically forces
the constructs to start on the same line. This is not exactly what PSR2
wants, but I think we can have a few exceptions with "our" style. The
starting of braces on the same line is pracrically standard for our
code.

This also removes and empty lines from method/function bodies at the
beginning and end.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 14:19:56 +02:00
Christoph Wurst 14c996d982
Use elseif instead of else if
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 10:35:09 +02:00
Christoph Wurst 008e6d7e84
Merge pull request #20391 from nextcloud/refactor/spaces-cleanup
Remove all extra whitespace PSR2 does not like
2020-04-09 20:39:37 +02:00
Christoph Wurst 64510932b8
Merge pull request #20384 from nextcloud/techdebt/lowercase-keywords
Use php keywords in lowercase
2020-04-09 16:25:14 +02:00
Christoph Wurst bcd13296de
Make sure every file ends with an empty newline
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 16:11:50 +02:00
Christoph Wurst 44577e4345
Remove trailing and in between spaces
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 16:07:47 +02:00
Christoph Wurst 36b3bc8148
Use php keywords in lowercase
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 14:04:56 +02:00
Christoph Wurst afbd9c4e6e
Unify function spacing to PSR2 recommendation
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 13:54:22 +02:00
Christoph Wurst 41b5e5923a
Use exactly one empty line after the namespace declaration
For PSR2

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 11:48:10 +02:00
Christoph Wurst 2fbad1ed72
Fix (array) indent style to always use one tab
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 10:16:08 +02:00
Christoph Wurst 85e369cddb
Fix multiline comments
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-08 22:24:54 +02:00
Robin Appelman 857957b45e
allow setting tooltips for external storage config options
allows explaining non-obvious config options a bit more

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-03 15:44:53 +02:00
Christoph Wurst 1a9330cd69
Update the license headers for Nextcloud 19
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-31 14:52:54 +02:00
John Molakvoæ fa4162e86f
Merge pull request #20208 from nextcloud/smb-folder-contents-normalize-path
normalize smb path while listing folder contents
2020-03-30 10:06:39 +02:00
Christoph Wurst 463b388589
Merge pull request #20170 from nextcloud/techdebt/remove-unused-imports
Remove unused imports
2020-03-27 17:14:08 +01:00
Robin Appelman ef7bb5225a
normalize smb path while listing folder contents
currently when listing `'/'`, it will include a double slash in the paths used for the listing logic
this causes unneeeded cache misses in the statCache

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-03-27 17:04:53 +01:00
Christoph Wurst b80ebc9674
Use the short array syntax, everywhere
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-26 16:34:56 +01:00
Christoph Wurst 74936c49ea
Remove unused imports
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-25 22:08:08 +01:00
Christoph Wurst c5cb3bb665
Do not use the deprecated service name
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-19 15:04:23 +01:00
Robin Appelman 59082661ca
remove the requirement that everything that looks like a placeholder is substituted
While maybe not common, `$` is a valid option in config fields

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-03-13 15:29:14 +01:00
Joas Schilling 17bc35e4f1
Merge pull request #19915 from nextcloud/external-storage-password-placeholders
Use placeholder values for password fields in external storage webui
2020-03-13 10:54:40 +01:00
Robin Appelman 0d112d7901
Use placeholder values for password fields in external storage webui
This prevents the password from being sent to the webui.

While an admin will always be able to retrieve the passwords (as they
can do arbitrairy code execution by design) this prevents casual
password snooping

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-03-12 19:45:23 +01:00
Robin Appelman 223ee42a53
faster implementation of SFTP write stream
using mostly the same techniques as the read stream

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-03-10 16:18:40 +01:00
Robin Appelman fe4e80093e
Faster implementation of sftp read stream
- Keeps a single file handle open instead of creating a new one for
every stream_read
- Read from sftp in large chunks instead of the 8k internal php size
- Request a read for the next chunk while we're still processing the
current one

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-03-06 16:57:18 +01:00
Arthur Schiwon 7e1d14d9a5
do not overwrite global user auth credentials with empty values
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2020-02-06 00:14:29 +01:00
Christoph Wurst 1b46621cd3
Update license headers for 18
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2019-12-20 09:23:25 +01:00
Roeland Jago Douma 0360ab408e
Merge pull request #18256 from nextcloud/files-aditional-scripts-new-events
Use non-depricated events for loading additional scripts in files app
2019-12-09 21:24:55 +01:00
Robin Appelman 1c585d2c50
use OCP\EventDispatcher\GenericEvent in more places
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-12-09 14:24:57 +01:00
Arthur Schiwon 43efa40864
rejig right to left join for sqlite's satisfaction
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-12-06 11:37:31 +01:00
Christoph Wurst 5bf3d1bb38
Update license headers
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2019-12-05 15:38:45 +01:00
Arthur Schiwon 4baab51f4c
extend with group deletion handling
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-11-28 17:03:33 +01:00
Arthur Schiwon be85d00a38
when a user was delete remove them from applicable list, unless
it is the only user, then remove the mount

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-11-28 17:03:33 +01:00
Roeland Jago Douma 68748d4f85
Some php-cs fixes
* Order the imports
* No leading slash on imports
* Empty line before namespace
* One line per import
* Empty after imports
* Emmpty line at bottom of file

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-11-22 20:52:10 +01:00
Daniel Kesselberg 2bd591f227
Make timeout a optional parameter
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-11-09 15:18:52 +01:00
Daniel Kesselberg 230f685987
Remove superfluous fclose
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-10-21 11:31:07 +02:00
Roeland Jago Douma 88b6dc5eaf
Merge pull request #17258 from nextcloud/smb-timeout-config
add (hidden) option to configure smb timeout
2019-09-27 12:29:56 +02:00
Robin Appelman bc160992d3
Update apps/files_external/lib/Lib/Backend/SMB.php
Co-Authored-By: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-25 14:59:44 +02:00
Robin Appelman 194edfc64e
add (hidden) option to configure smb timeout
hidden from ui to prevent clutter

```occ files_external:config <mount> timeout 30```

Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-09-25 13:30:40 +02:00
Daniel Kesselberg 72d22a4828
Use local variable for directory name
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:41 +02:00
Daniel Kesselberg 75b8944061
Don't strip path from directory prefix
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:41 +02:00
Daniel Kesselberg 80340a8d46
Fill directory cache from opendir
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:41 +02:00
Daniel Kesselberg 12863e0d24
Change visibility to private (from protected)
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:40 +02:00
Daniel Kesselberg 963d35c78c
Rename $location to $path
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:40 +02:00
Daniel Kesselberg 010473ca1f
Add method to check if directory exists
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:40 +02:00
Greta Doci 0a874c51af
Disable app token creation for impersonated people, ref #15539
Signed-off-by: Greta Doci <gretadoci@gmail.com>
2019-09-15 12:04:27 +02:00
Roeland Jago Douma 15e16d4a85
Merge pull request #16790 from nextcloud/admin-use-user-global-auth
Allow admin configured mounts to use user configured global credentials
2019-09-10 16:01:14 +02:00
Robin Appelman 603655341e
allow setting user provided global credentials trough credentials popup
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-09-09 17:15:30 +02:00
Robin Appelman 1d51b297a9
Allow admin configured mounts to use user configured global credentials
Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-09-09 16:51:24 +02:00
Arthur Schiwon 43bc31bacb
set a storage availability delay on auth issues to avoid lock out
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-08-22 12:40:15 +02:00
Arthur Schiwon cd62b38cc8
catch auth issues also when php-smbclient is in use
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-08-22 11:12:43 +02:00
Arthur Schiwon 792903e6b3
delay sending event from app init to when they are needed
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-08-02 14:26:43 +02:00
Arthur Schiwon 02d6870735
fixes terminology and allows to request an IUser instance
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-08-02 08:39:39 +02:00
Arthur Schiwon d577076457
do not send null value when string is expected
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-08-02 08:39:39 +02:00
Julius Härtl 30c44e1d24
Provide proper user context from initMountPoints
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2019-08-02 08:39:24 +02:00
Julius Härtl 72aaf2e5fb
files_external: Make sure the correct user context is used in substitution of variables
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2019-08-02 08:39:24 +02:00
Daniel Kesselberg b1bd3f1fe5
Fix wrong indentation
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-07-22 13:00:03 +02:00
Daniel Kesselberg 98c269be53
Fix invalid recursion
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-07-21 22:55:26 +02:00
Daniel Kesselberg 1ae68047ea
Allow hidden smb shares
A hidden smb share ends with $. This patch changes the placeholder
detection to allow shares with $ at the end.

Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-07-16 00:02:01 +02:00
Morris Jobke adc2ab4bb2
Ignore forbidden exception when fetching share mtime on SMB for symlinks
See https://github.com/nextcloud/server/pull/7556#issuecomment-499028026

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2019-07-08 21:43:24 +02:00
Robin Appelman 6d5579863e
add option to show hidden files in SMB shares
Note hidden files can mean different things in smb and the option the the files web ui,
the webui only counts files starting with '.' as hidden, while smb files
can be marked as hidden regardless, any files that are marked as hidden
on smb will thus be shown in the webui regardless of the setting in the files app.

Fixes #15644

Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-05-23 21:23:56 +02:00
Arthur Schiwon 2b84bbf372
do not attempt to replace anything in the password
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-04-02 23:08:34 +02:00
Ari Selseng 11b3fbf612 Use two queries to mark parent as outdated
Signed-off-by: Ari Selseng <ari@selseng.net>
2019-03-18 18:24:57 +01:00
Ari Selseng 1ac9a9a1bb Correct log attribute
Signed-off-by: Ari Selseng <ari@selseng.net>
2019-03-04 15:02:15 +01:00
Ari Selseng 3218e0f70d Added logging to Notify command.
Signed-off-by: Ari Selseng <ari@selseng.net>
2019-03-04 14:23:13 +01:00
Ari Selseng 71a0cdceaa Reconnect to DB after timeout for Notify command. Fixes #14479
Signed-off-by: Ari Selseng <ari@selseng.net>
2019-03-03 01:36:27 +01:00
Roeland Jago Douma ba155a98db
Merge pull request #14448 from mzamot/master
Convert value of timestamp metadata key in swift files_external
2019-03-01 21:26:56 +01:00
Michael Zamot dc35a8af83 Convert timestamp key value of metadata in Swift files_external
modified:   apps/files_external/lib/Lib/Storage/Swift.php

Signed-off-by: Michael Zamot <michael@zamot.io>

Convert value of timestamp metadata key in swift files_external to string
2019-03-01 12:10:37 -07:00
Ari Selseng f156079a8b Fix a bug with smb notify having leading slash when it should not
Signed-off-by: Ari Selseng <ari@selseng.net>
2019-03-01 13:55:52 +01:00
Morris Jobke 7fc68fe93e
Merge pull request #14211 from nextcloud/bugfix/prefix-path-to-filecache-s3
Prefix $path for filename for internal file cache
2019-02-19 12:01:08 +01:00
Arthur Schiwon 173836b95a
typos in doc, no code changes
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-02-15 22:41:29 +01:00
Daniel Kesselberg 0d667d18bb
Call headObject if file not in filecache 🙈
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-02-14 18:34:04 +01:00
Daniel Kesselberg 659ed3ecda
Prefix $path for filename for internal file cache
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-02-14 17:06:14 +01:00
Arthur Schiwon f8a133d39e
reject mounts with unsubstituted placeholders as incompletely configured
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-02-14 15:22:22 +01:00
Arthur Schiwon 792bcb82ae
add LDAP ConfigHandler for external storages and "$home" var
* handler registered upon OCA\\Files_External::loadAdditionalBackends
  event as user_ldap is loaded before files_external
* new configuration field "ldapExtStorageHomeAttribute" (not in GUI yet)

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-02-14 15:22:22 +01:00
Arthur Schiwon a26bcd8e8f
files_external: allow to register config handlers for flexible placeholders
* BackendService (directly accessable via \OC_Server) offers registerConfigHandler
* SimpleSubstitutionTrait brings reusable logic for simple string replacments
* internal $user replacement mechanism was migrated

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-02-14 15:20:48 +01:00
Morris Jobke 331f3b2652
Merge pull request #12691 from nextcloud/storage-no-encryption-interface
Add interface to allow storages from opting out of encryption
2018-11-29 22:02:23 +01:00
Robin Appelman 2c1f6331a4
Add interface to allow storages from opting out of encryption
As opposed to hard-coding a list of excluded storages

Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-11-29 16:31:34 +01:00
Robin Appelman 56c777f715
Fix cannot set 0 as value on files_external through OCC command
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-11-27 15:20:23 +01:00
Morris Jobke 6bace99aa1
Fix UniqueConstraintViolationException while insert into files_external config tables
* followup to #12371

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-11-14 15:24:03 +01:00
Joas Schilling ea21aa3f7a
Use numeric placeholders if there are multiple, so that RTL languages can operate better
Signed-off-by: Joas Schilling <coding@schilljs.com>
2018-10-09 14:32:14 +02:00
Daniel Kesselberg 4cbea5f7e1
Remove value from cache on delete/clear
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2018-10-08 16:40:39 +02:00
Daniel Kesselberg 5cf38254bb
Use helper method for content length and last modified
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2018-10-01 20:18:36 +02:00
Daniel Kesselberg 67d5380b91
Reuse information from ListObjects for stat / filetype
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2018-09-30 21:05:53 +02:00
Morris Jobke 8563ab94aa
Merge pull request #10589 from nextcloud/smb-stat-retry
retry smb stat on timeout
2018-08-24 07:00:40 +02:00
Robin Appelman bdc6d0b5a1
log entries that are hidden during file listing
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-16 01:20:05 +02:00
Robin Appelman 24aaa3bc97
retry stat on timeout
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-08-08 16:07:41 +02:00
Morris Jobke e35451fd4e
Merge pull request #10025 from nextcloud/smb-rename-invalidargument-retry
also retry rename operation on InvalidArgumentException
2018-07-11 16:25:22 +02:00
Robin Appelman 27089422e1
Improved logging of smb connection errors
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-07-03 14:56:37 +02:00
Robin Appelman 431710a5aa
also retry rename operation on InvalidArgumentException
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-06-27 15:33:51 +02:00
Robin Appelman 90be8f336c
Add support for using kerberos ticket to authenticate to smb servers
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-06-22 14:25:36 +02:00
Robin Appelman 4a424f86db
update icewind/smb to 3.0.0
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-06-21 16:02:51 +02:00
Morris Jobke bbfc028169
Merge pull request #9786 from nextcloud/feature/noid/external_sftp_password_and_key
Add SFTP auth mechanism to use a password and private key for SFTP
2018-06-08 09:23:47 +02:00
Robin Appelman 48fb126505
allow setting notify credentials in environment
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-06-07 15:33:58 +02:00
Roeland Jago Douma 2b19add6b0
Add SFTP auth mechanism to use a password and public key for SFTP
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-06-07 13:35:43 +02:00
Morris Jobke b348431bce
Merge pull request #8842 from lanxenet/patch-1
fix(AmazonS3):  fix loop $result['Contents'] error
2018-05-07 18:41:39 +02:00
Morris Jobke a56ec1062a
Merge pull request #9346 from nextcloud/techdep/noid/needsPartFile_to_storage
Use an actual function of the storage to determine needsPartFile
2018-05-02 16:03:31 +02:00