When removing external storages, either system-wide or user-wide,
automatically remove the matching oc_storages and oc_filecache entries.
This can only work if the backend configuration doesn't contain any
substitution variable $user in which case the storage id cannot be
computed, so this case is ignored for now.
StoragesService::getStorages() will check the visibility of the backend
and auth mechanism for the storage, and if either are not visible to the
user (aka disabled by admin) then the storage will be filtered out. The
original method StoragesService::getAllStorages() still exists in case
such storages need to be detected, but its use is discouraged.
VisibilityTrait -> PermissionsTrait
PermissionsTrait stores two sets of data, $permissions and
$allowedPermissions (analogous to $visibility and $allowedVisibility of
VisibilityTrait). Each set is a map of user type ('admin' or 'personal')
to permissions (mounting permission, create permission).
The result is that a backend can now be restricted for creation, while
still allowing it to be mounted. This is useful for deprecating backends
or auth mechanisms, preventing new storages being created, while still
allowing existing storages to be mounted.
The following functions have been removed:
- addMountPoint()
- removeMountPoint()
- movePersonalMountPoint()
registerBackend() has been rewritten as a shim around BackendService,
allowing legacy code to interact with the new API seamlessly
addMountPoint() was already disconnected from all production code, so
this commit completes the job and removes the function itself, along
with disconnecting and removing related functions. Unit tests have
likewise been removed.
getAbsoluteMountPoints(), getSystemMountPoints() and
getPersonalMountPoints() have been rewritten to use the StoragesServices
Prior to this, the storage class name was stored in mount.json under the
"class" parameter, and the auth mechanism class name under the
"authMechanism" parameter. This decouples the class name from the
identifier used to retrieve the backend or auth mechanism.
Now, backends/auth mechanisms have a unique identifier, which is saved in
the "backend" or "authMechanism" parameter in mount.json respectively.
An identifier is considered unique for the object it references, but the
underlying class may change (e.g. files_external gets pulled into core
and namespaces are modified).
A backend can now specify generic authentication schemes that it
supports, instead of specifying the parameters for its authentication
method directly. This allows multiple authentication mechanisms to be
implemented for a single scheme, providing altered functionality.
This commit introduces the backend framework for this feature, and so at
this point the UI will be broken as the frontend does not specify the
required information.
Terminology:
- authentication scheme
Parameter interface for the authentication method. A backend
supporting the 'password' scheme accepts two parameters, 'user' and
'password'.
- authentication mechanism
Specific mechanism implementing a scheme. Basic mechanisms may
forward configuration options directly to the backend, more advanced
ones may lookup parameters or retrieve them from the session
New dropdown selector for external storage configurations to select the
authentication mechanism to be used.
Authentication mechanisms can have visibilities, just like backends.
The API was extended too to make it easier to add/remove visibilities.
In addition, the concept of 'allowed visibility' has been introduced, so
a backend/auth mechanism can force a maximum visibility level (e.g.
Local storage type) that cannot be overridden by configuration in the
web UI.
An authentication mechanism is a fully instantiated implementation. This
allows an implementation to have dependencies injected into it, e.g. an
\OCP\IDB for database operations.
When a StorageConfig is being prepared for mounting, the authentication
mechanism implementation has manipulateStorage() called,
which inserts the relevant authentication method options into the
storage ready for mounting.
Backends are registered to the BackendService through new data
structures:
Backends are concrete classes, deriving from
\OCA\Files_External\Lib\Backend\Backend. During construction, the
various configuration parameters of the Backend can be set, in a design
similar to Symfony Console.
DefinitionParameter stores a parameter configuration for an external
storage: name of parameter, human-readable name, type of parameter
(text, password, hidden, checkbox), flags (optional or not).
Storages in the StoragesController now get their parameters validated
server-side (fixes a TODO).
str_replace for $user substitution was converting the data type of
mountOptions to string. This fix prevents this to happen by making sure
only strings are processed by substitution.
Also added a int conversion when reading the watcher policy
Added option to disable autoscan of external storages
Mount option file system scan is now an int
Move priority field to avoid undefined field in storage options
All input elements inside the storage options block get parsed into
storage options.
Moving the priority field outside prevents it to appear in the storage
config, as expected. It is still parsed by special code.
When reading in old mount.json files, they do not contain config ids.
Since these are needed to be able to use the UI and the new service
classes, these will be generated automatically.
The config grouping is based on a config hash.
The mount options are now passed to the UI and stored in a hidden field.
The ext storage controllers and services also know how to load/save them
from the legacy config.
- Added StorageConfig class to replace ugly arrays
- Implemented StorageService and StorageController for Global and User
storages
- Async status checking for storages (from Xenopathic)
- Auto-generate id for external storage configs (not the same as
storage_id)
- Refactor JS classes for external storage settings, this mostly
moves/encapsulated existing global event handlers into the
MountConfigListView class.
- Added some JS unit tests for the external storage UI
We were using the SMB and SFTP backends, which since Icewind's new SMB library
results in an actual attempted connection just by constructing the storage.
This commit replaces all references to real storages (except when necessary)
with a dummy storage class.
Because setStorageId now instantiates storages, they might throw
exceptions if arguments are missing.
This fixes the unit test to not throw exceptions and have their default
config arguments set.
Add support for external files accessed via SFTP using public key exchange authentication.
Keys are generated automatically when the configuration is added, or can be regenerated on demand if a key is compromised.
Creation of a new configuration row now triggers focus on that row. This is used to trigger auto-configuration for SFTP keys.
Generated public keys are saved in user's data directory for easy retrieval by an external application.
Add controller for SFTP key generation AJAX
SFTP class initialisation no longer produces a warning if the password field is missing.
Add unit tests for SFTP with key authentication backend
document docker parameter and use random host port
fix typo
copy autotest.sh to autotest-external.sh
adds ability to add start* and stop* scripts in env in external tests
run files_external WebDAV tests against ownCloud instance
introduce executor number to be able to shut down the correct docker container
fetch docker images in advance - this also fetches latest versions of the docker images
add second argument to autotest-external.sh which can specify a single test to run
print out the explicit test run
change naming schema of files_external setup scripts
Allow specifying a protocol in the host field when mounting another
ownCloud instance. Note that this was already possible with the WebDAV
config but this bug made it inconsistent.
Whenever file actions are registered later, now the file lists are
automatically notified.
Added FileActions.addUpdateListener() to be able to receive such
notifications.
This removes the need for apps to manually call FileActions.display()
after registering new actions.
This fixes issues with race conditions when file actions are
registered after the file list was already rendered.
To make sure that the dynamic mount config and config classes from
external storages are tested, these are now added into the autotest.sh
test run by enabling the app.
Merging of configurations is whenever the same config is available for
multiple users/groups, in which case the config is considered as a
single one by the UI, and shows multiple users/groups selected.
Fixed merging logic to make sure that class, mount point and options are
the same before merging them.
Fixed merging to work correctly when the same mount point path is used
for separate users and configs. These are now correctly shows in the UI
as separate entries.
The old password field is now preserved in the JSON structure to make
sure that the order is preserved.
This is a quick fix until the UI is fixed to not rely on the PHP array
key order.
- test config was written correctly (global and local)
- test that personal mount points can be added for allowed backends
- test that personal mount points cannot be added for disallowed backends
- added $skipTest flag to make it possible add mount points without
doing an actual check/connection (note: this is necessary because the
static class OC_Mount_Config cannot be mocked)
- Some WebDAV servers like lighttpd need paths in URLs to be properly
encoded
- Added error log output when curl connection failed
- Added check for 'resourcetype' in case the WebDAV server doesn't
support/return it
- Fixed touch() to return false if the server doesn't implement
PROPPATCH
- Added optional delay in WebDAV unit tests to use when testing against
lighttpd's WebDAV
- fixed touch function to return true on success
- fixed local metadata cache to remove deleted files/directories
- fixed getMetaData() to ignore files reported as deleted by
Dropbox
- fixed "file not found" case to not log as exception
- fixed "overwrite on rename" case
- fixed unit tests to exclude unsupported cases
- added unit test for touch return value
- unit tests for Dropbox should all run correctly now
To make it possible to use the short ownCloud URL (without specifying
webdav.php or remote.php), a new backend is available for ownCloud.
The user must specify the host + context path in the "Url" field (which
is mapped to the "host" parameter) and the subdir to mount in the "Root"
field.
This is to prevent confusion because some users forget to append
webdav.php or remote.php to the WebDAV URL.
Fixes#4923
Robin McCorkell
Morris Jobke
Vincent Petry
Olivier Paroz
Thomas Müller
Viktor Szépe
Robin McCorkell
Robin Appelman
Jenkins for ownCloud
Ross Nicoll
Robin Appelman
Joas Schilling
Jörn Friedrich Dreyer
Bjoern Schiessle
Lukas Reschke
Morris Jobke
Christian Berendt
kondou
Bart Visscher
Michael Gapczynski