Commit Graph

39 Commits

Author SHA1 Message Date
Robin Appelman 9735b5d61f
Add method to storage backends to get directory content with metadata
Currently you need to use `opendir` and then call `getMetadata` for
every file, which adds overhead because most storage backends already
get the metadata when doing the `opendir`.

While storagebackends can (and do) use caching to relief this problem,
this adds cache invalidation dificulties and only a limited number of
items are generally cached (to prevent memory usage exploding when
scanning large storages)

With this new methods storage backends can use the child metadata they
got from listing the folder to return metadata without having to keep
seperate caches.

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-20 15:45:04 +02:00
Christoph Wurst caff1023ea
Format control structures, classes, methods and function
To continue this formatting madness, here's a tiny patch that adds
unified formatting for control structures like if and loops as well as
classes, their methods and anonymous functions. This basically forces
the constructs to start on the same line. This is not exactly what PSR2
wants, but I think we can have a few exceptions with "our" style. The
starting of braces on the same line is pracrically standard for our
code.

This also removes and empty lines from method/function bodies at the
beginning and end.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 14:19:56 +02:00
Christoph Wurst 14c996d982
Use elseif instead of else if
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 10:35:09 +02:00
Robin Appelman 5199f80196
call cache->insert directly instead of going trough cache->put first when scanning
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-01 14:17:28 +02:00
Robin Appelman 9c7da26394
dont try to get existing cache data during scanning if we already know its a new file
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-01 14:17:25 +02:00
Robin Appelman 3fb32f6f3d
reuse known parentId when inserting cache entries in scanner
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-01 14:14:52 +02:00
Christoph Wurst 1a9330cd69
Update the license headers for Nextcloud 19
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-31 14:52:54 +02:00
Christoph Wurst b80ebc9674
Use the short array syntax, everywhere
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-26 16:34:56 +01:00
Christoph Wurst 5bf3d1bb38
Update license headers
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2019-12-05 15:38:45 +01:00
Ari Selseng d16cfb519e Avoid calculating folder size for parent that needs scan.
Signed-off-by: Ari Selseng <ari@selseng.net>
2019-03-06 15:23:37 +01:00
Robin Appelman 63caf331e5
cleanup locks in scanner on error
Signed-off-by: Robin Appelman <robin@icewind.nl>
2018-05-04 18:05:46 +02:00
Arthur Schiwon 38a90130ce
move log constants to ILogger
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2018-04-26 10:45:52 +02:00
Morris Jobke eb51f06a3b
Use ::class statement instead of string
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-01-29 12:03:47 +01:00
Morris Jobke a661f043e1
Remove unneeded semicolon and parentheses
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-01-26 23:46:40 +01:00
Morris Jobke 2a38605545
Properly log the full exception instead of only the message
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2018-01-23 10:57:21 +01:00
Roeland Jago Douma 8a41d05761
Remove deprecated \OCP\Config
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2018-01-13 14:25:04 +01:00
Morris Jobke 0eebff152a
Update license headers
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2017-11-06 16:56:19 +01:00
Robin Appelman 804d97d6ff
unlock file when an exception occurs during scanning
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-08-26 18:58:06 +02:00
Morris Jobke c27498db71 Use IConfig instead of static OCP\Config
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2017-07-27 13:43:18 +02:00
Robin Appelman 1f1e1b0d00
use unmasked permissions during scanning
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-05-18 16:38:54 +02:00
Morris Jobke c54a59d51e
Remove unused use statements
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2017-04-22 19:23:31 -05:00
Roeland Jago Douma 17c07e5caf Merge pull request #3447 from nextcloud/getmetadata-permissions-mask
also apply permissions mask in getMetaData
2017-02-15 20:22:08 +01:00
Robin Appelman f27113a6ee
also apply permissions mask in getMetaData
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-02-10 22:36:12 +01:00
Robin Appelman 2520a4831b
Make sure we use the correct storage id when scanning a jailed storage
Signed-off-by: Robin Appelman <robin@icewind.nl>
2017-01-25 17:03:45 +01:00
Joas Schilling 558f169671
Move the validation into one place only
Signed-off-by: Joas Schilling <coding@schilljs.com>
2016-11-21 09:23:37 +01:00
Joas Schilling 4652d203e3
Make sure we don't scan files that can not be accessed
Signed-off-by: Joas Schilling <coding@schilljs.com>
2016-11-21 09:23:32 +01:00
Joas Schilling 0215b004da
Update with robin 2016-07-21 18:13:58 +02:00
Joas Schilling ba87db3fcc
Fix others 2016-07-21 18:13:57 +02:00
Robin Appelman 73547f29be Ignore forbidden files while scanning 2016-06-07 14:01:55 +02:00
Lukas Reschke aba539703c
Update license headers 2016-05-26 19:57:24 +02:00
Vincent Petry aa56d42fa8 Merge pull request #24777 from owncloud/scanner-dont-propagate
dont needlessly triger the propgator in the scanner
2016-05-23 19:05:20 +02:00
Robin Appelman eca57be336 Only recurse into incomplete folders during background scans 2016-05-23 14:40:35 +02:00
Robin Appelman f3979e5459 dont needlessly triger the propgator in the scanner 2016-05-23 13:53:05 +02:00
Vincent Petry bd87f67473 Merge pull request #24349 from owncloud/nfd-storagewrapper
Add wrapper for NFD encoding workaround
2016-05-23 13:45:33 +02:00
Vincent Petry f8b2b95408
Scanner must normalize new children names for cache diff
Since new children from the storage might contain NFD entries, these
must be normalized to NFC to be properly diff'ed with the cache
contents which is always NFC.

This fixes an issue where NFD entries would disappear from the cache
after rescannng for children.
2016-05-20 09:33:59 +02:00
Robin Appelman a3623745d5 Free up folder content from memory before recursing in the file scanner 2016-04-29 15:16:41 +02:00
Thomas Müller f31d0ce87c
Merge branch 'master' of https://github.com/d--j/core into d--j-master 2016-04-25 14:52:38 +02:00
Robin Appelman 4ac16001e0
add locks in the scanner to prevent multiple scanners running on the same files 2016-04-25 11:09:08 +02:00
Roeland Jago Douma dedf392751
Move \OC\Files to PSR-4 2016-04-24 21:37:35 +02:00