The code path called when using external storage with WebDAV is using `\OC\Files\Storage\Wrapper\Encryption::getMetaData` which did not contain the actual encrypted version inside the cache entry version. This lead to the following:
1. User uploaded a file
2. File is created and `\OC\Files\Storage\Wrapper\Encryption::getMetaData` is called. It has an empty `encryptedVersion` but sets `encrypted` to either `true` or `false`.
3. The call when updating the file cache will use the old version.
Fixes#23536
The new sharing code is much stricter in checking permissions. However
for non moveable mounts the permissions UPDATE+DELETE are not reported
on the mount point.
This is just a quick fix.
* Updated unit tests
In 9.0 we converted the old shares to the new shares. So for 9.1 we can
savely remove the fallback code.
This code was required when there was no initiator set.
* Fixed unit tests
The old behaviour was that only languages could be used for an app
that are already present in the apps/$app/l10n folder. If there is
a themed l10n that is not present in the apps default l10n folder
the language could not be used and the texts are not translated.
With this change this is possible and also the l10n files are
loaded even if the default l10n doesn't contain the l10n file.
In case of a move operation from an unencrypted to an encrypted
storage the old encrypted version would stay with "0" while the
correct value would be "1". Thus we manually set the value to "1"
for those cases.
See also https://github.com/owncloud/core/issues/23078
A PreconditionNotMetException must only be thrown if explicit
preconditions are specified for setValues(), not if the value is merely
the same as was already in the DB.
The setup uses `\OCP\IRequest::getInsecureServerHost` which in some cases can also include a port. This makes the trusted domain check fail thus.
I've decided to add this here that way because adjusting the setup would require parsing the host properly. This is not something that can be done very good in PHP. Check the following example for why `parse_url` is not our friend: https://3v4l.org/k501Z
When calling `\OC\Files\View::copy` we should also keep the version to ensure that the file will always have the correct version attached and can be successfully decrypted.
To test this the following steps are necessary (from https://github.com/owncloud/core/issues/22781#issuecomment-191328982):
1. setup a new ownCloud 9.0 beta2
2. enable encryption
2. upload a docx (5.7MB large)
3. upload the same file again and overwrite the existing file
4. I can download the original file and the first version
5. I restore the first version
6. restored version can no longer be downloaded with the error described above
The manual cache operation in `\OCA\Files_Versions\Storage` is unfortunately necessary since `\OCA\Files_Versions\Storage::copyFileContents` is not using `\OCP\Files\Storage::moveFromStorage` in the case when an object storage is used. Due to the workaround added in 54cea05271 the stream is directly copied and thus bypassing the FS.
Sharing of the users root folder should not be allowed as it is very
weird UX. Also many of our clients have no proper way of displaying
this.
Added unit test
Also added intergration tests to make sure we won't allow it in the
future.
Incomming federated shares are a special kind. We mount them as normal
webdav shares but we do supply owner info with the federated cloud id of
the share owner.
Since we do not yet have the new resharing behaviour on federated shares
we need to set the correct owner. Which will allow sharing and proper
mounting for other users.
fixes#22500
Fixes#22119
Just try to get the folder of the user. If it is not there a
NotFoundException will be thrown. Which will be handled by the avatar
endpoint.
Our issue template states that users should post the output of `/index.php/settings/integrity/failed`, at the moment it displays that all passes have been passed if the integrity checker has been disabled.
This is however a wrong approach considering that some distributions are gonna package Frankenstein releases and makes it harder for us to detect such issues. Thus if the integrity code checker is disabled (using the config switch) it displays now: `Appcode checker has been disabled. Integrity cannot be verified.`
This is not displayed anywhere else in the UI except these URL used for us for debugging purposes.
Fixes#19685
The default expiration date should only be set when we create a new
share. So if a share is created and the expiration date is unset. And
after that the password is updated the expiration date should remain
unset.
This allows all clients to quickly get the share info for a given path.
Instead of returning everything and filtering it then manually on the
client side.
For group shares we can have children. Those are custom shares when a
user has moved or deleted a group share. Those also have to be deleted
if the group share is removed.
Allows to inject something into the default content policy. This is for
example useful when you're injecting Javascript code into a view belonging
to another controller and cannot modify its Content-Security-Policy itself.
Note that the adjustment is only applied to applications that use AppFramework
controllers.
To use this from your `app.php` use `\OC::$server->getContentSecurityPolicyManager()->addDefaultPolicy($policy)`,
$policy has to be of type `\OCP\AppFramework\Http\ContentSecurityPolicy`.
To test this add something like the following into an `app.php` of any enabled app:
```
$manager = \OC::$server->getContentSecurityPolicyManager();
$policy = new \OCP\AppFramework\Http\ContentSecurityPolicy(false);
$policy->addAllowedFrameDomain('asdf');
$policy->addAllowedScriptDomain('yolo.com');
$policy->allowInlineScript(false);
$manager->addDefaultPolicy($policy);
$policy = new \OCP\AppFramework\Http\ContentSecurityPolicy(false);
$policy->addAllowedFontDomain('yolo.com');
$manager->addDefaultPolicy($policy);
$policy = new \OCP\AppFramework\Http\ContentSecurityPolicy(false);
$policy->addAllowedFrameDomain('banana.com');
$manager->addDefaultPolicy($policy);
```
If you now open the files app the policy should be:
```
Content-Security-Policy:default-src 'none';script-src yolo.com 'self' 'unsafe-eval';style-src 'self' 'unsafe-inline';img-src 'self' data: blob:;font-src yolo.com 'self';connect-src 'self';media-src 'self';frame-src asdf banana.com 'self'
```
ownCloud might not yet be setup. This causes an issue as the user config requires a setup ownCloud. Thus this needs a block whether ownCloud is installed or not.
Fixes https://github.com/owncloud/core/issues/21955
If a user deletes a group share we create a special share entry. To the
API this is just a normal group share for that user with permissions 0.
But we should not return this.
This adds a new CSRF manager for unit testing purposes, it's interface is based upon https://github.com/symfony/security-csrf. Due to some of our required custom changes it is however not possible to use the Symfony component directly.
This allows recipient to delete a share. For user shares this is the
same as deleting (at least for now).
But for group shares this means creating a new share with type 2. With
permissions set to 0.
After the initial installation ownCloud will write some content into the .htaccess file such as the 404 or 403 directives. This adds a magic marker into the .htaccess file and only the content above this marker will be compared in the integrity checker.
This change requires the usage of a path instead of the App ID when signing code. This has the advantage that developers can also sign code under a different location to make it easier. (e.g. remove `.git`, …)
Also it adds an example command usage as well as a link to the documentation
Added config.php option to replace the default implementation of system
tag manager and system tag object mapper.
Also adjusted the comments manager factory to inject the server container
* lostpassword.css is unneeded since #11696 is merged - 1b50d4f7ce
* js is already in core/js
* css is moved to core/css/lostpassword
* template is moved to core/templates/lostpassword
CredentialsManager performs a simple role, of storing and retrieving
encrypted credentials from the database. Credentials are stored by user
ID (which may be null) and credentials identifier. Credentials
themselves may be of any type that can be JSON encoded.
The rationale behind this is to avoid further (mis)use of
oc_preferences, which was being used for all manner of data not related
to user preferences.
Integers, doubles, booleans and even arrays can now be set, with the
--type=... option. Array setting can be specified by passing multiple
name arguments, e.g. `./occ config:system:set redis port --value=123
--type=integer`
Now that we support multiple managers we communicate shares to the
outside as 'providerId:shareId'. This makes sures that id's are unique
when references from the OCS API.
However, since we do not want to break the OCS API v1 we need to
somewhat hack around this.
When we switch to OCS API v2 (which we should when we support more
custom providers). We will change the id to always be the fullShareId.
This adds a hidden config flag that allows somebody to disable the code integrity check. If `integrity.check.disabled` is set to `true` in the config file:
1. The integrity check functions will return always an empty result
2. The integrity check is not performed when installing apps
3. The integrity check is not performed when updating apps
4. The integrity check is not performed when updating the core
Furthermore this adds support for a list of channels that the code checker will run on. At the moment this is only stable because I didn't want to break any build scripts that we have. Once we have a proper CA setup and updated the build process to sign the releases we can add the RC, alpha, beta as well as daily releases. So everything except "git" basically.
Fixes#21533
Before we just assumed that the passed path was a file. This does not
have to be the case. Thus check if it actually is a file before doing
any more tests.
There were code paths that nowadays call ISession::login directly thus bypassing the desired regeneration of the session ID. This moves the session regeneration deeper into the session handling and thus ensures that it is always called. Furthermore, I also added the session regeneration to the remember me cookie plus added some test case expectations for this.
In case of failure, PHPUnit seems to skip `tearDown`, so any useful
assertion messages cannot be shown because `tearDownAfterClass` is
throwing an error because of database usage.
This commit makes sure we also restore the database in
`tearDownAfterClass` to prevent the data root restoration to fail
`addWarning` needs to be defined as well for it:
```
➜ master git:(master) ✗ bash autotest.sh sqlite
Using PHP executable /usr/local/opt/php56/bin/php
Parsing all files in lib/public for the presence of @since or @deprecated on each method...
Using database oc_autotest
Setup environment for sqlite testing on local storage ...
Installing ....
ownCloud is not installed - only a limited number of commands are available
Mac OS X is not supported and ownCloud will not work properly on this platform. Use it at your own risk!
-> For the best results, please consider using a GNU/Linux server instead.
creating sqlite db
ownCloud was successfully installed
Testing with sqlite ...
No coverage
/usr/local/bin/phpunit --configuration phpunit-autotest.xml --log-junit autotest-results-sqlite.xml
PHP Notice: Constant PHPUNIT_RUN already defined in /Users/lukasreschke/Documents/Programming/master/apps/firstrunwizard/tests/bootstrap.php on line 6
PHP Stack trace:
PHP 1. {main}() /usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar:0
PHP 2. PHPUnit_TextUI_Command::main() /usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar:514
PHP 3. PHPUnit_TextUI_Command->run() phar:///usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar/phpunit/TextUI/Command.php:106
PHP 4. PHPUnit_TextUI_Command->handleArguments() phar:///usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar/phpunit/TextUI/Command.php:117
PHP 5. PHPUnit_Util_Configuration->getTestSuiteConfiguration() phar:///usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar/phpunit/TextUI/Command.php:663
PHP 6. PHPUnit_Util_Configuration->getTestSuite() phar:///usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar/phpunit/Util/Configuration.php:796
PHP 7. PHPUnit_Framework_TestSuite->addTestFile() phar:///usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar/phpunit/Util/Configuration.php:926
PHP 8. PHPUnit_Util_Fileloader::checkAndLoad() phar:///usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar/phpunit/Framework/TestSuite.php:335
PHP 9. PHPUnit_Util_Fileloader::load() phar:///usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar/phpunit/Util/Fileloader.php:38
PHP 10. include_once() phar:///usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar/phpunit/Util/Fileloader.php:56
PHP 11. loadDirectory() /Users/lukasreschke/Documents/Programming/master/tests/apps.php:42
PHP 12. require_once() /Users/lukasreschke/Documents/Programming/master/tests/apps.php:20
PHP 13. define() /Users/lukasreschke/Documents/Programming/master/apps/firstrunwizard/tests/bootstrap.php:6
PHP Fatal error: Class StartSessionListener contains 1 abstract method and must therefore be declared abstract or implement the remaining methods (PHPUnit_Framework_TestListener::addWarning) in /Users/lukasreschke/Documents/Programming/master/tests/startsessionlistener.php on line 47
PHP Stack trace:
PHP 1. {main}() /usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar:0
PHP 2. PHPUnit_TextUI_Command::main() /usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar:514
PHP 3. PHPUnit_TextUI_Command->run() phar:///usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar/phpunit/TextUI/Command.php:106
PHP 4. PHPUnit_TextUI_TestRunner->doRun() phar:///usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar/phpunit/TextUI/Command.php:155
PHP 5. PHPUnit_TextUI_TestRunner->handleConfiguration() phar:///usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar/phpunit/TextUI/TestRunner.php:153
PHP 6. require_once() phar:///usr/local/Cellar/phpunit/5.1.0/libexec/phpunit-5.1.0.phar/phpunit/TextUI/TestRunner.php:805
```
we do not listen to deletion hooks anymore, because there is no guarantee that they
will be heard - requires that something fetches the CommentsManager first.
Instead, in the user deletion routine the clean up method will be called directly. Same way
as it happens for files, group memberships, config values.
register CommentsManager service, allow override, document in config.sample.php
don't insert autoincrement ids in tests, because of dislikes from oracle and pgsql
specify timezone in null date
only accepts strings for ID parameter that can be converted to int
replace forgotten hardcoded IDs in tests
react on deleted users
react on file deletion
Postgresql compatibility
lastInsertId needs *PREFIX* with the table name
do not listen for file deletion, because it is not reliable (trashbin, external storages)
add runtime cache for comments
This changeset allows ownCloud to run with pretty URLs, they will be used if mod_rewrite and mod_env are available. This means basically that the `index.php` in the URL is not shown to the user anymore.
Also the not deprecated functions to generate URLs have been modified to support this behaviour, old functions such as `filePath` will still behave as before for compatibility reasons.
Examples:
http://localhost/owncloud/index.php/s/AIDyKbxiRZWAAjP => http://localhost/owncloud/s/AIDyKbxiRZWAAjPhttp://localhost/owncloud/index.php/apps/files/ => http://localhost/owncloud/apps/files/
Due to the way our CSS and JS is structured the .htaccess uses some hacks for the final result but could be worse... And I was just annoyed by all that users crying for the removal of `index.php` ;-)
This PR implements the base foundation of the code signing and integrity check. In this PR implemented is the signing and verification logic, as well as commands to sign single apps or the core repository.
Furthermore, there is a basic implementation to display problems with the code integrity on the update screen.
Code signing basically happens the following way:
- There is a ownCloud Root Certificate authority stored `resources/codesigning/root.crt` (in this PR I also ship the private key which we obviously need to change before a release 😉). This certificate is not intended to be used for signing directly and only is used to sign new certificates.
- Using the `integrity:sign-core` and `integrity:sign-app` commands developers can sign either the core release or a single app. The core release needs to be signed with a certificate that has a CN of `core`, apps need to be signed with a certificate that either has a CN of `core` (shipped apps!) or the AppID.
- The command generates a signature.json file of the following format:
```json
{
"hashes": {
"/filename.php": "2401fed2eea6f2c1027c482a633e8e25cd46701f811e2d2c10dc213fd95fa60e350bccbbebdccc73a042b1a2799f673fbabadc783284cc288e4f1a1eacb74e3d",
"/lib/base.php": "55548cc16b457cd74241990cc9d3b72b6335f2e5f45eee95171da024087d114fcbc2effc3d5818a6d5d55f2ae960ab39fd0414d0c542b72a3b9e08eb21206dd9"
},
"certificate": "-----BEGIN CERTIFICATE-----MIIBvTCCASagAwIBAgIUPvawyqJwCwYazcv7iz16TWxfeUMwDQYJKoZIhvcNAQEF\nBQAwIzEhMB8GA1UECgwYb3duQ2xvdWQgQ29kZSBTaWduaW5nIENBMB4XDTE1MTAx\nNDEzMTcxMFoXDTE2MTAxNDEzMTcxMFowEzERMA8GA1UEAwwIY29udGFjdHMwgZ8w\nDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANoQesGdCW0L2L+a2xITYipixkScrIpB\nkX5Snu3fs45MscDb61xByjBSlFgR4QI6McoCipPw4SUr28EaExVvgPSvqUjYLGps\nfiv0Cvgquzbx/X3mUcdk9LcFo1uWGtrTfkuXSKX41PnJGTr6RQWGIBd1V52q1qbC\nJKkfzyeMeuQfAgMBAAEwDQYJKoZIhvcNAQEFBQADgYEAvF/KIhRMQ3tYTmgHWsiM\nwDMgIDb7iaHF0fS+/Nvo4PzoTO/trev6tMyjLbJ7hgdCpz/1sNzE11Cibf6V6dsz\njCE9invP368Xv0bTRObRqeSNsGogGl5ceAvR0c9BG+NRIKHcly3At3gLkS2791bC\niG+UxI/MNcWV0uJg9S63LF8=\n-----END CERTIFICATE-----",
"signature": "U29tZVNpZ25lZERhdGFFeGFtcGxl"
}
```
`hashes` is an array of all files in the folder with their corresponding SHA512 hashes (this is actually quite cheap to calculate), the `certificate` is the certificate used for signing. It has to be issued by the ownCloud Root Authority and it's CN needs to be permitted to perform the required action. The `signature` is then a signature of the `hashes` which can be verified using the `certificate`.
Steps to do in other PRs, this is already a quite huge one:
- Add nag screen in case the code check fails to ensure that administrators are aware of this.
- Add code verification also to OCC upgrade and unify display code more.
- Add enforced code verification to apps shipped from the appstore with a level of "official"
- Add enfocrced code verification to apps shipped from the appstore that were already signed in a previous release
- Add some developer documentation on how devs can request their own certificate
- Check when installing ownCloud
- Add support for CRLs to allow revoking certificates
**Note:** The upgrade checks are only run when the instance has a defined release channel of `stable` (defined in `version.php`). If you want to test this, you need to change the channel thus and then generate the core signature:
```
➜ master git:(add-integrity-checker) ✗ ./occ integrity:sign-core --privateKey=resources/codesigning/core.key --certificate=resources/codesigning/core.crt
Successfully signed "core"
```
Then increase the version and you should see something like the following:
As you can see a failed code check will not prevent the further update. It will instead just be a notice to the admin. In a next step we will add some nag screen.
For packaging stable releases this requires the following additional steps as a last action before zipping:
1. Run `./occ integrity:sign-core` once
2. Run `./occ integrity:sign-app` _for each_ app. However, this can be simply automated using a simple foreach on the apps folder.
Adding group Db to federation tests and ldap tests
Add group DB to Test_UrlGenerator
Adding group DB to trashbin and versions tests
Adding group DB to Test_Util_CheckServer for pg
This is necessary to categorize unit test and avoid duplicate test case execution - it also allows us to closely review unit test implementations to identify unnecessary db calls.
Lukas Reschke
Roeland Jago Douma
Thomas Müller
Robin Appelman
Joas Schilling
Morris Jobke
Bjoern Schiessle
Robin McCorkell
C. Montero Luque
v1r0x
Björn Schießle
Arthur Schiwon
Vincent Petry
Faruk Uzun
Vincent Chan
Robin McCorkell
Bernhard Posselt
Scrutinizer Auto-Fixer
Jesus Macias Portela
Robin Appelman
