Commit Graph

161 Commits

Author SHA1 Message Date
Julius Härtl 0a24fcfc34 Continue iterating over diretory contents and just hide file if denied by acl
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-04-21 07:24:25 +00:00
Vincent Petry cea228ef50
Update psalm baseline
And backport error suppression from stable21 in SFTP class.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-04-20 17:13:31 +02:00
Robin Appelman 728488fcf5 add (hidden) option to always show smb root as writable
some smb servers are very insistent in reporting that the root of the share is readonly, even if it isn't.

This works around the problem by adding a hidden option to overwrite the permissions of the root of the share.

This can be enabled using

```bash
occ files_external:config <mount id> root_force_writable true
```

where you can find your mount id using

```bash
occ files_external:list
```

Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-19 15:47:23 +00:00
Robin Appelman 326f7ac857 catch notfound and forbidden exception in smb::getmetadata
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-03-05 07:54:09 +00:00
Julius Härtl fcfa39183c Properly handle SMB ACL blocking scanning a directory
This makes sure that a possible ForbiddenException is properly passed
through the storage as a ForbiddenException and can be catched when
trying to fetch the quota info of a parent folder

Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-02-01 11:13:44 +00:00
Roeland Jago Douma 42d56c9655 Check if array elements exist before using them
It seems that in some recent upgrade here. Not always the entries we
expect are returned. So we should first check if they exist. As to not
spam the log.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-10-27 10:57:51 +00:00
Robin Appelman eba4723428 improve handling of out of space errors for smb
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-09-17 18:47:33 +00:00
Julius Härtl 2004ae4603
Set delimiter so that the CommonPrefixes response can properly be determined
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-09-10 08:11:55 +02:00
Robin Appelman fc96fa0cc7
fix s3 doesDirectoryExist check for empty directories
if a directory is empty, only the 'marker' object `/` exists.

since not all s3 implementations return just the prefix when listing objects by prefix,
when listing objects by the folder prefix, nothing will be returned if the directory is empty.
by not including the trailing slash in the prefix, the folder marked will always be returned if it exists

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-09-10 08:11:55 +02:00
Roeland Jago Douma bb06b6cce4
Fix reading empty files from objectstorage
Since we try to do range requests this will fail hard.
However since empty files are not that interesting to read anyways we
just read from an emptry memory stream.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-09-09 20:45:13 +02:00
Christoph Wurst 2a054e6c04
Update the license headers for Nextcloud 20
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-08-24 14:54:25 +02:00
Morris Jobke ac73982395
Merge pull request #21734 from nextcloud/external-s3-no-touch
disable touch to update mtime for external s3 storage
2020-08-20 16:07:59 +02:00
Morris Jobke fedf9c69d9
Use matching parameter names form interfaces and implementations
Found by Psalm 3.14.1

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-08-19 18:16:35 +02:00
Daniel Kesselberg ba3a855ad8
Remove unused method uploadFile
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2020-08-13 08:02:28 +02:00
Morris Jobke 638e9438a9
Merge pull request #21728 from nextcloud/bugfix/noid/s3-external-mtime-cache
Reuse cache for directory mtime/size if filesystem changes can be ignored
2020-07-15 18:26:48 +02:00
Robin Appelman e357d46863
filter files containing a hash in the path for ftp storages
the php ftp streamwrapper doesn't handle hashes correctly and will break when it tries to enter a path containing a hash.

By filtering out paths containing a hash we can at least stop the external storage from breaking completely

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-10 14:18:40 +02:00
Robin Appelman 0d6910ac13
disable touch to update mtime for external s3 storage
since updating the mtime requires doing a copy this can have a significant overhead and it messes with versioning

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-07-07 15:55:38 +02:00
Julius Härtl f5cc34c9c2
Reuse cache for directory mtime/size if filesystem changes can be ignored
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-07-07 10:22:22 +02:00
Julius Härtl 69f4c50032
Remove rescanDelay from directory mtime
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-06-22 14:15:57 +02:00
Robin Appelman 6f2e920a5b
normalize sftp path in read and write stream
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-05-25 15:26:04 +02:00
Christoph Wurst cb057829f7
Update license headers for 19
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-29 11:57:22 +02:00
Robin Appelman 23e2baeb95
Add option to check share ACL's when listing directories
If a file or folder in a directory doesn't have read permissions they
will not be shown

Note that enabling this option incurs a performance penalty additional
requests need to be made to get all the acl.

Additionally the acl resolving logic is fairly primitive at the moment
and might not work correctly in all setups (it should error to showing
the entry)

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-21 16:26:12 +02:00
Robin Appelman 9735b5d61f
Add method to storage backends to get directory content with metadata
Currently you need to use `opendir` and then call `getMetadata` for
every file, which adds overhead because most storage backends already
get the metadata when doing the `opendir`.

While storagebackends can (and do) use caching to relief this problem,
this adds cache invalidation dificulties and only a limited number of
items are generally cached (to prevent memory usage exploding when
scanning large storages)

With this new methods storage backends can use the child metadata they
got from listing the folder to return metadata without having to keep
seperate caches.

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-20 15:45:04 +02:00
Christoph Wurst 28f8eb5dba
Add visibility to all constants
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:54:27 +02:00
Christoph Wurst 1584c9ae9c
Add visibility to all methods and position of static keyword
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 16:51:06 +02:00
Christoph Wurst caff1023ea
Format control structures, classes, methods and function
To continue this formatting madness, here's a tiny patch that adds
unified formatting for control structures like if and loops as well as
classes, their methods and anonymous functions. This basically forces
the constructs to start on the same line. This is not exactly what PSR2
wants, but I think we can have a few exceptions with "our" style. The
starting of braces on the same line is pracrically standard for our
code.

This also removes and empty lines from method/function bodies at the
beginning and end.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 14:19:56 +02:00
Christoph Wurst 14c996d982
Use elseif instead of else if
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 10:35:09 +02:00
Christoph Wurst bcd13296de
Make sure every file ends with an empty newline
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 16:11:50 +02:00
Christoph Wurst 44577e4345
Remove trailing and in between spaces
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 16:07:47 +02:00
Christoph Wurst afbd9c4e6e
Unify function spacing to PSR2 recommendation
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 13:54:22 +02:00
Christoph Wurst 41b5e5923a
Use exactly one empty line after the namespace declaration
For PSR2

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 11:48:10 +02:00
Christoph Wurst 85e369cddb
Fix multiline comments
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-08 22:24:54 +02:00
Christoph Wurst 1a9330cd69
Update the license headers for Nextcloud 19
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-31 14:52:54 +02:00
John Molakvoæ fa4162e86f
Merge pull request #20208 from nextcloud/smb-folder-contents-normalize-path
normalize smb path while listing folder contents
2020-03-30 10:06:39 +02:00
Robin Appelman ef7bb5225a
normalize smb path while listing folder contents
currently when listing `'/'`, it will include a double slash in the paths used for the listing logic
this causes unneeeded cache misses in the statCache

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-03-27 17:04:53 +01:00
Christoph Wurst b80ebc9674
Use the short array syntax, everywhere
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-26 16:34:56 +01:00
Robin Appelman 223ee42a53
faster implementation of SFTP write stream
using mostly the same techniques as the read stream

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-03-10 16:18:40 +01:00
Robin Appelman fe4e80093e
Faster implementation of sftp read stream
- Keeps a single file handle open instead of creating a new one for
every stream_read
- Read from sftp in large chunks instead of the 8k internal php size
- Request a read for the next chunk while we're still processing the
current one

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-03-06 16:57:18 +01:00
Christoph Wurst 5bf3d1bb38
Update license headers
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2019-12-05 15:38:45 +01:00
Roeland Jago Douma 68748d4f85
Some php-cs fixes
* Order the imports
* No leading slash on imports
* Empty line before namespace
* One line per import
* Empty after imports
* Emmpty line at bottom of file

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2019-11-22 20:52:10 +01:00
Daniel Kesselberg 230f685987
Remove superfluous fclose
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-10-21 11:31:07 +02:00
Roeland Jago Douma 88b6dc5eaf
Merge pull request #17258 from nextcloud/smb-timeout-config
add (hidden) option to configure smb timeout
2019-09-27 12:29:56 +02:00
Robin Appelman 194edfc64e
add (hidden) option to configure smb timeout
hidden from ui to prevent clutter

```occ files_external:config <mount> timeout 30```

Signed-off-by: Robin Appelman <robin@icewind.nl>
2019-09-25 13:30:40 +02:00
Daniel Kesselberg 72d22a4828
Use local variable for directory name
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:41 +02:00
Daniel Kesselberg 75b8944061
Don't strip path from directory prefix
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:41 +02:00
Daniel Kesselberg 80340a8d46
Fill directory cache from opendir
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:41 +02:00
Daniel Kesselberg 12863e0d24
Change visibility to private (from protected)
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:40 +02:00
Daniel Kesselberg 963d35c78c
Rename $location to $path
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:40 +02:00
Daniel Kesselberg 010473ca1f
Add method to check if directory exists
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2019-09-23 14:03:40 +02:00
Arthur Schiwon 43bc31bacb
set a storage availability delay on auth issues to avoid lock out
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2019-08-22 12:40:15 +02:00