The builtin mechanism is used exclusively for legacy external storages,
which when upgraded to the new registration need to fallback to the
defined legacy authentication mechanism instead.
Introduces the basic password authentication mechanism, along with a
mechanism based on ownCloud credentials stored in the user session.
Change to lib/private is an extension of PermissionsMask, as
isSharable() override was missing.
Session credentials auth mechanism now disables sharing on applied
storages, as credentials will not be available.
Loading custom JS on a per-backend basis added needless complexity and
made dealing with async required. Now all backends/auth mechanisms load
custom JS in PHP
Failure to prepare the storage during backend or auth mechanism
manipulation will throw an InsufficientDataForMeaningfulAnswerException,
which is propagated to StorageNotAvailableException in the filesystem
layer via the FailedStorage helper class.
When a storage is unavailable not due to failure, but due to
insufficient data being available, a special 'indeterminate' status is
returned to the configuration UI.
The following functions have been removed:
- addMountPoint()
- removeMountPoint()
- movePersonalMountPoint()
registerBackend() has been rewritten as a shim around BackendService,
allowing legacy code to interact with the new API seamlessly
addMountPoint() was already disconnected from all production code, so
this commit completes the job and removes the function itself, along
with disconnecting and removing related functions. Unit tests have
likewise been removed.
getAbsoluteMountPoints(), getSystemMountPoints() and
getPersonalMountPoints() have been rewritten to use the StoragesServices
Prior to this, the storage class name was stored in mount.json under the
"class" parameter, and the auth mechanism class name under the
"authMechanism" parameter. This decouples the class name from the
identifier used to retrieve the backend or auth mechanism.
Now, backends/auth mechanisms have a unique identifier, which is saved in
the "backend" or "authMechanism" parameter in mount.json respectively.
An identifier is considered unique for the object it references, but the
underlying class may change (e.g. files_external gets pulled into core
and namespaces are modified).
A backend can now specify generic authentication schemes that it
supports, instead of specifying the parameters for its authentication
method directly. This allows multiple authentication mechanisms to be
implemented for a single scheme, providing altered functionality.
This commit introduces the backend framework for this feature, and so at
this point the UI will be broken as the frontend does not specify the
required information.
Terminology:
- authentication scheme
Parameter interface for the authentication method. A backend
supporting the 'password' scheme accepts two parameters, 'user' and
'password'.
- authentication mechanism
Specific mechanism implementing a scheme. Basic mechanisms may
forward configuration options directly to the backend, more advanced
ones may lookup parameters or retrieve them from the session
New dropdown selector for external storage configurations to select the
authentication mechanism to be used.
Authentication mechanisms can have visibilities, just like backends.
The API was extended too to make it easier to add/remove visibilities.
In addition, the concept of 'allowed visibility' has been introduced, so
a backend/auth mechanism can force a maximum visibility level (e.g.
Local storage type) that cannot be overridden by configuration in the
web UI.
An authentication mechanism is a fully instantiated implementation. This
allows an implementation to have dependencies injected into it, e.g. an
\OCP\IDB for database operations.
When a StorageConfig is being prepared for mounting, the authentication
mechanism implementation has manipulateStorage() called,
which inserts the relevant authentication method options into the
storage ready for mounting.
UserGlobalStoragesService reads the global storage configuration,
cherry-picking storages applicable to a user. Writing storages through
this service is forbidden, on punishment of throwing an exception.
Storage IDs may also be config hashes when retrieved from this service,
as it is unable to update the storages with real IDs.
As UserGlobalStoragesService and UserStoragesService share a bit of code
relating to users, that has been split into UserTrait. UserTrait also
allows for the user set to be overridden, rather than using the user
from IUserSession.
Config\ConfigAdapter has been reworked to use UserStoragesService and
UserGlobalStoragesService instead of
OC_Mount_Config::getAbsoluteMountPoints(), further reducing dependance
on that horrible static class.
Backends are registered to the BackendService through new data
structures:
Backends are concrete classes, deriving from
\OCA\Files_External\Lib\Backend\Backend. During construction, the
various configuration parameters of the Backend can be set, in a design
similar to Symfony Console.
DefinitionParameter stores a parameter configuration for an external
storage: name of parameter, human-readable name, type of parameter
(text, password, hidden, checkbox), flags (optional or not).
Storages in the StoragesController now get their parameters validated
server-side (fixes a TODO).
Storage status is saved in the database. Failed storages are rechecked every
10 minutes, while working storages are rechecked every request.
Using the files_external app will recheck all external storages when the
settings page is viewed, or whenever an external storage is saved.
Explicitly clear the stat cache after deleting an empty folder to make
sure it is properly detected as deleted in subsequent requests.
This works around a problem with phpseclib where the folder is properly
deleted remotely but the stat cache was not updated.
The code was missing the "MetadataDirective".
Once added, some other parts of the code failed because the format of mtime was wrong.
So this PR uses the RFC format that the S3 library already uses.
Additionally, the code path where mtime is null was missing. Now defaulting to
the current time.
str_replace for $user substitution was converting the data type of
mountOptions to string. This fix prevents this to happen by making sure
only strings are processed by substitution.
Also added a int conversion when reading the watcher policy
Added option to disable autoscan of external storages
Mount option file system scan is now an int
Move priority field to avoid undefined field in storage options
All input elements inside the storage options block get parsed into
storage options.
Moving the priority field outside prevents it to appear in the storage
config, as expected. It is still parsed by special code.
When reading in old mount.json files, they do not contain config ids.
Since these are needed to be able to use the UI and the new service
classes, these will be generated automatically.
The config grouping is based on a config hash.
The mount options are now passed to the UI and stored in a hidden field.
The ext storage controllers and services also know how to load/save them
from the legacy config.
- Added StorageConfig class to replace ugly arrays
- Implemented StorageService and StorageController for Global and User
storages
- Async status checking for storages (from Xenopathic)
- Auto-generate id for external storage configs (not the same as
storage_id)
- Refactor JS classes for external storage settings, this mostly
moves/encapsulated existing global event handlers into the
MountConfigListView class.
- Added some JS unit tests for the external storage UI
We were using the SMB and SFTP backends, which since Icewind's new SMB library
results in an actual attempted connection just by constructing the storage.
This commit replaces all references to real storages (except when necessary)
with a dummy storage class.
This enforces proper types on POST and GET arguments where I considered it sensible. I didn't update some as I don't know what kind of values they would support 🙈
Fixes https://github.com/owncloud/core/issues/14196 for core
Because setStorageId now instantiates storages, they might throw
exceptions if arguments are missing.
This fixes the unit test to not throw exceptions and have their default
config arguments set.
Modified CSS for files_external application to render labels around checkboxes at the same
width as other input elements, aligning them with other elements on the same page.
Add support for external files accessed via SFTP using public key exchange authentication.
Keys are generated automatically when the configuration is added, or can be regenerated on demand if a key is compromised.
Creation of a new configuration row now triggers focus on that row. This is used to trigger auto-configuration for SFTP keys.
Generated public keys are saved in user's data directory for easy retrieval by an external application.
Add controller for SFTP key generation AJAX
SFTP class initialisation no longer produces a warning if the password field is missing.
Add unit tests for SFTP with key authentication backend
document docker parameter and use random host port
fix typo
copy autotest.sh to autotest-external.sh
adds ability to add start* and stop* scripts in env in external tests
run files_external WebDAV tests against ownCloud instance
introduce executor number to be able to shut down the correct docker container
fetch docker images in advance - this also fetches latest versions of the docker images
add second argument to autotest-external.sh which can specify a single test to run
print out the explicit test run
change naming schema of files_external setup scripts
In GDrive, filenames aren't unique, and directories are just
special files - so you can have multiple files with the same
name, multiple directories with the same name, and even files
with the same names as directories.
OC doesn't handle this at all, though, and just wants to act
as if file and directory names *are* unique. So when renaming,
we must check if there's an existing object with the same
file or directory name before we commit the rename, and
explicitly delete it if the rename is successful. (Other
providers like dropbox do the same for files, but intentionally
don't do it for directories; we really need to do it for
directories too.)
A good way to observe this is to run the storage unit tests
and look at the state of the Drive afterwards. Without this
commit, there will be several copies of all the test files
and directories. After this commit, there's just one of each.
We can't just say "hey, Drive lets us do this, what's the
problem?" because we don't handle multiple-objects, same-name
cases - getDriveFile() just bails and prints an error if it
searches for the file or directory with a given name and gets
multiple results.
Sometimes there are bugs that cause setupFS() to be called for
non-existing users. Instead of failing hard and breaking the instance,
this fix simply logs a warning.
opendir() trims the path passed then calls getDriveFile() -
which immediately does the same trim operation. This breaks
opendir() on the root directory, which causes the failure of
the testStat() test when it checks the ctime of the root dir.
ownCloud passes us a Unix time integer, but the GDrive API wants
an RFC3339-formatted date. Actually it wants a single particular
RFC3339 format, not just anything that complies will do - it
requires the fractions to be specified, though RFC3339 doesn't.
This resolves issue #11267 (and was also noted by PVince81 in
reviewing PR #6989).
This is a slightly hacky workaround for
https://github.com/google/google-api-php-client/issues/59 .
There's a bug in the Google library which makes it go nuts on
file uploads and transfer *way* too much data if compression is
enabled and it's using its own IO handler (not curl). Upstream
'fixed' this (by disabling compression) for one upload
mechanism, but not for the one we use. The bug doesn't seem to
happen if the google lib detects that curl is available and
decides to use it instead of its own handler. So, let's disable
compression, but only if it looks like the Google lib's check
for curl is going to fail.
Latest version with various bugfixes, also implements support
for using curl instead of its own io class when available; this
avoids the bug that causes severe excess bandwidth use due to
some kind of zlib issue.
This is the upstream commit that merged my query separator fix. It's slightly
after the 1.0.3-beta tag. I eyeballed the other post 1.0.3-beta changes and
none of them looks like any kind of problem, so we may as well just use this
upstream state.
We need to do this in order to be able to refresh the access token without
prompting the user for their credentials every hour. This was the default
in 0.6 of the Google library, but needs to be explicitly specified in 1.0.
Submitted upstream as https://github.com/google/google-api-php-client/issues/76
Google's php lib has a function to generate a URL for OAuth2 authentication.
It uses http_build_query() to generate the query part of the URL, and in PHP
5.3 or later, this uses an encoded ampersand - & - as the query separator,
not a raw one. However, Google's OAuth server apparently can't handle encoded
ampersands as separators and so it fails.
This patch explicitly sets a raw ampersand as the separator. If Google decides
to fix their OAuth server instead of merging this patch into google-api-php-
client, we can drop this patch as soon as that happens.
Added addTranslations and fixed de.js file
Fixed de.js to use OC.L10N.register() and use to correct expected
format.
Added JS unit tests for OC.L10N class
Include translations JS script for all apps
Robin McCorkell
Robin Appelman
Roeland Jago Douma
Vincent Petry
Lukas Reschke
Andreas Fischer
Jenkins for ownCloud
Viktor Szépe
Robin McCorkell
Thomas Müller
Morris Jobke
Martin
Jan-Christoph Borchardt
Joas Schilling
Frank Karlitschek
SA
Ross Nicoll
Joas Schilling
Volkan Gezer
Christopher Bunn
Jörn Friedrich Dreyer
Robin Appelman
Volkan Gezer
Adam Williamson
Johan Björk