Commit Graph

10537 Commits

Author SHA1 Message Date
Lukas Reschke 4971015544 Add code integrity check
This PR implements the base foundation of the code signing and integrity check. In this PR implemented is the signing and verification logic, as well as commands to sign single apps or the core repository.

Furthermore, there is a basic implementation to display problems with the code integrity on the update screen.

Code signing basically happens the following way:

- There is a ownCloud Root Certificate authority stored `resources/codesigning/root.crt` (in this PR I also ship the private key which we obviously need to change before a release 😉). This certificate is not intended to be used for signing directly and only is used to sign new certificates.
- Using the `integrity:sign-core` and `integrity:sign-app` commands developers can sign either the core release or a single app. The core release needs to be signed with a certificate that has a CN of `core`,  apps need to be signed with a certificate that either has a CN of `core` (shipped apps!)  or the AppID.
- The command generates a signature.json file of the following format:
```json
{
    "hashes": {
        "/filename.php": "2401fed2eea6f2c1027c482a633e8e25cd46701f811e2d2c10dc213fd95fa60e350bccbbebdccc73a042b1a2799f673fbabadc783284cc288e4f1a1eacb74e3d",
        "/lib/base.php": "55548cc16b457cd74241990cc9d3b72b6335f2e5f45eee95171da024087d114fcbc2effc3d5818a6d5d55f2ae960ab39fd0414d0c542b72a3b9e08eb21206dd9"
    },
    "certificate": "-----BEGIN CERTIFICATE-----MIIBvTCCASagAwIBAgIUPvawyqJwCwYazcv7iz16TWxfeUMwDQYJKoZIhvcNAQEF\nBQAwIzEhMB8GA1UECgwYb3duQ2xvdWQgQ29kZSBTaWduaW5nIENBMB4XDTE1MTAx\nNDEzMTcxMFoXDTE2MTAxNDEzMTcxMFowEzERMA8GA1UEAwwIY29udGFjdHMwgZ8w\nDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANoQesGdCW0L2L+a2xITYipixkScrIpB\nkX5Snu3fs45MscDb61xByjBSlFgR4QI6McoCipPw4SUr28EaExVvgPSvqUjYLGps\nfiv0Cvgquzbx/X3mUcdk9LcFo1uWGtrTfkuXSKX41PnJGTr6RQWGIBd1V52q1qbC\nJKkfzyeMeuQfAgMBAAEwDQYJKoZIhvcNAQEFBQADgYEAvF/KIhRMQ3tYTmgHWsiM\nwDMgIDb7iaHF0fS+/Nvo4PzoTO/trev6tMyjLbJ7hgdCpz/1sNzE11Cibf6V6dsz\njCE9invP368Xv0bTRObRqeSNsGogGl5ceAvR0c9BG+NRIKHcly3At3gLkS2791bC\niG+UxI/MNcWV0uJg9S63LF8=\n-----END CERTIFICATE-----",
    "signature": "U29tZVNpZ25lZERhdGFFeGFtcGxl"
}
```
`hashes` is an array of all files in the folder with their corresponding SHA512 hashes (this is actually quite cheap to calculate), the `certificate` is the  certificate used for signing. It has to be issued by the ownCloud Root Authority and it's CN needs to be permitted to perform the required action. The `signature` is then a signature of the `hashes` which can be verified using the `certificate`.

Steps to do in other PRs, this is already a quite huge one:
- Add nag screen in case the code check fails to ensure that administrators are aware of this.
- Add code verification also to OCC upgrade and unify display code more.
- Add enforced code verification to apps shipped from the appstore with a level of "official"
- Add enfocrced code verification to apps shipped from the appstore that were already signed in a previous release
- Add some developer documentation on how devs can request their own certificate
- Check when installing ownCloud
- Add support for CRLs to allow revoking certificates

**Note:** The upgrade checks are only run when the instance has a defined release channel of `stable` (defined in `version.php`). If you want to test this, you need to change the channel thus and then generate the core signature:

```
➜  master git:(add-integrity-checker) ✗ ./occ integrity:sign-core --privateKey=resources/codesigning/core.key --certificate=resources/codesigning/core.crt
Successfully signed "core"
```

Then increase the version and you should see something like the following:

![2015-11-04_12-02-57](https://cloud.githubusercontent.com/assets/878997/10936336/6adb1d14-82ec-11e5-8f06-9a74801c9abf.png)

As you can see a failed code check will not prevent the further update. It will instead just be a notice to the admin. In a next step we will add some nag screen.

For packaging stable releases this requires the following additional steps as a last action before zipping:
1. Run `./occ integrity:sign-core` once
2. Run `./occ integrity:sign-app` _for each_ app. However, this can be simply automated using a simple foreach on the apps folder.
2015-12-01 11:55:20 +01:00
Joas Schilling 3c5a6b829e Allow DI the system tag stuff without Application class 2015-11-30 17:08:29 +01:00
Thomas Müller 08248f66ba Merge pull request #20650 from owncloud/systemtags-core
Implement systemtag managers and mapper
2015-11-30 15:38:43 +01:00
Thomas Müller 995ceeb8d9 Merge pull request #20815 from owncloud/avatar-cache-size
cache resized avatars
2015-11-30 14:27:05 +01:00
Thomas Müller fc21aa6c41 Merge pull request #20248 from owncloud/use-phpunit-groups-to-run-database-code-in-isolation
Use phpunit groups to run database code in isolation
2015-11-30 14:25:35 +01:00
Lukas Reschke f4eb15d340 Show error template
Otherwise this leads to an endless redirection in case of a CSRF exception. Also sets user expectation right.
2015-11-30 11:25:52 +01:00
Thomas Müller a94819f4a4 Merge pull request #20809 from owncloud/dont-trust-update-server-message
Don't trust update server
2015-11-30 11:20:30 +01:00
Thomas Müller f9fc254672 Add DB group to some files_external tests
Adding group Db to federation tests and ldap tests

Add group DB to Test_UrlGenerator

Adding group DB to trashbin and versions tests

Adding group DB to Test_Util_CheckServer for pg
2015-11-30 10:55:10 +01:00
Joas Schilling c4bebf66d4 Do not count the entries when we only need to know if it is at least one 2015-11-30 10:20:00 +01:00
Joas Schilling aa1a0a15ea Fix use statements 2015-11-30 10:12:15 +01:00
Thomas Müller 9c1dbaf0ad Merge pull request #20788 from owncloud/catch-missing-route
Dont die when we're missing a route
2015-11-30 10:11:12 +01:00
Joas Schilling 6816b36cdb Add backticks inside function to escape the column 2015-11-30 10:07:00 +01:00
Vincent Petry 17ce120736 Revert "Dont bother with stream_resolve_include_path if the path is already absolute" 2015-11-29 10:54:29 +01:00
Robin Appelman 6b7e748245 cache resized avatars 2015-11-28 17:33:16 +01:00
Lukas Reschke 391bc49dab Move files/ajax/scan.php to background job
The background job will now be executed in chunks of 500 users all 10 minutes.
2015-11-28 13:06:53 +01:00
Lukas Reschke f3e9106864 Don't trust update server
In case the update server may deliver malicious content this would allow an adversary to inject arbitrary HTML into the response. So very bad stuff.

While signing the response would be better and something we can also do in the future (considering the code signing work), this is already a good first start.
2015-11-28 12:21:53 +01:00
Robin Appelman 4c4331982c Merge pull request #20789 from owncloud/scanner-skip-not-available
Skip unavailable storages in scanner
2015-11-27 22:57:47 +01:00
Vincent Petry b666367a79 Added system tags data structure and PHP side managers
Added SystemTagManager and SystemTagObjectMapper
2015-11-27 17:54:29 +01:00
Thomas Müller 2b316eaa83 Merge pull request #20792 from owncloud/autoloader-absolute-paths
Dont bother with stream_resolve_include_path if the path is already absolute
2015-11-27 16:03:39 +01:00
Robin Appelman b05c8faba8 Dont die when we're missing a route 2015-11-27 14:29:06 +01:00
Robin Appelman 816cd66b4b also log exception 2015-11-27 14:28:15 +01:00
Thomas Müller bdbefe17d6 Merge pull request #20782 from mitar/better-https
Also allow empty value for no-HTTPS
2015-11-27 14:24:23 +01:00
Robin Appelman 50474c6041 Dont bother with stream_resolve_include_path if the path is already absolute 2015-11-27 14:12:14 +01:00
Robin Appelman ae71f80f4a Skip unavailable storages in scanner 2015-11-27 14:02:50 +01:00
Mitar 59511d97ee Also allow empty value for no-HTTPS.
This makes it work better with old version of Nginx.
2015-11-27 01:01:56 -08:00
Morris Jobke 7aed592957 Add full interface of server container as alias 2015-11-26 18:20:25 +01:00
Thomas Müller 8fe878afe9 Merge pull request #20602 from owncloud/fix-installed-apps
Always installed apps includes the hardcoded ones from shipped.json
2015-11-26 16:50:05 +01:00
Thomas Müller 80c43ffc6c Merge pull request #20702 from owncloud/move-user-principal-into-subfolder
Users are available under it's own principal resource named 'principa…
2015-11-26 16:49:49 +01:00
Thomas Müller 19d5059109 Merge pull request #20393 from owncloud/querybuilder-select-with-alias
Add a method to select a field or value with alias
2015-11-26 16:19:20 +01:00
Thomas Müller 97ebc6d80c Merge pull request #20746 from owncloud/untangle-linkToDocs
Untangle the linkToDocs method in OC_Helper
2015-11-26 15:08:28 +01:00
Thomas Müller 4d22c75d49 Merge pull request #20562 from owncloud/comments-interfaces
public interfaces for Comments
2015-11-26 15:06:35 +01:00
Morris Jobke 675417a75c Untangle the linkToDocs method in OC_Helper
* now uses the proper URLGenerator interface
* add comment about DI problems
2015-11-26 13:58:43 +01:00
Thomas Müller 3882cc8ef3 Merge pull request #20744 from owncloud/oc_helper-getMimeType-cleanup
Remove last occurences of OC_Helper::getMimeType()
2015-11-26 12:37:25 +01:00
Arthur Schiwon d660c6162f proper description for IllegalIDChangeException 2015-11-26 12:15:00 +01:00
Thomas Müller 5a33390a69 Merge pull request #20731 from owncloud/per-storage-updater
Make Cache\Updater per storage
2015-11-26 12:01:44 +01:00
Morris Jobke 0b8296a756 remove unused method 2015-11-26 10:58:30 +01:00
Morris Jobke 4fcab98694 Remove unused internal methods
* removes OC_Helper::mb_substr_replace and OC_Helper::mb_str_replace
* keeps public interface wrapper working as expected
2015-11-26 10:25:43 +01:00
Morris Jobke 9318606faf Remove last occurences of OC_Helper::getMimeType()
* ref #4774
2015-11-26 10:18:32 +01:00
Thomas Müller b799e42b4e Introduce \OCP\IUser::getEMailAddress() 2015-11-25 22:23:34 +01:00
Thomas Müller 9ec2f8886e Merge pull request #20691 from owncloud/share2.0_di_fixes
[Sharing 2.0] di fixes
2015-11-25 15:25:50 +01:00
Thomas Müller afe76840f8 Merge pull request #20705 from owncloud/fix_20648
Fix overriding function from 3rdparty warning
2015-11-25 15:24:20 +01:00
Robin Appelman b025f07fb7 Make Cache\Updater per storage 2015-11-25 14:16:00 +01:00
Thomas Müller 50f6817ce9 Merge pull request #20439 from owncloud/etag-propagate-in-storage
Take submount etag into account for folder etags
2015-11-25 12:49:54 +01:00
Thomas Müller 89b9f5518b Merge pull request #20636 from owncloud/savre-3.0
sabre/dav 3.0 and related
2015-11-24 16:20:48 +01:00
Thomas Müller 4bb346a1e6 Merge pull request #20692 from owncloud/federation_auto_add_servers
auto-add ownClouds to the list of trusted servers
2015-11-24 16:19:39 +01:00
Thomas Müller ae36c01b95 Adjust sabre changes in core 2015-11-24 15:11:54 +01:00
Roeland Jago Douma cfdf2b9976 Fix overriding function from 3rdparty warning
Fixes #20648
2015-11-24 13:25:50 +01:00
Björn Schießle 964fa1fce3 use hooks to auto add server to the list of trusted servers once a federated share was created 2015-11-24 11:34:38 +01:00
Thomas Müller b7b6770d35 Merge pull request #20693 from owncloud/phpdoc_node
Consistent interface declaration OCP\Files\Node
2015-11-24 10:56:53 +01:00
Thomas Müller d785883eb2 Merge pull request #20682 from owncloud/objectid-to-string
Make sure that object id can be a string
2015-11-24 10:56:34 +01:00
Roeland Jago Douma 85976b7293 [Sharing 2.0] Fix phpdoc etc 2015-11-24 10:26:36 +01:00
Roeland Jago Douma a2b8483779 [Sharing 2.0] Default share provider only generic DI
No injection of userfolders etc. Only generic DI components
(IRootFolder) etc should be used to make sure we can also run this from
the cli
2015-11-24 10:26:36 +01:00
Roeland Jago Douma aeae73b364 [Sharing 2.0] Removed unused DI stuff
The share manager etc should not care about filtering stuff. They should
return what is asked for them.
2015-11-24 10:26:36 +01:00
Jenkins for ownCloud cb69e6c201 [tx-robot] updated from transifex 2015-11-24 01:56:32 -05:00
Arthur Schiwon ab8937ba6a missing setters for setChildrenCount and setLatestChildDateTime (formerly …Timestamp) 2015-11-23 23:58:22 +01:00
Roeland Jago Douma 5873821b8d Consistent interface declaration 2015-11-23 22:03:19 +01:00
Arthur Schiwon 4005c0337b give creation datetime setter and getter a more meaningful and less misleading name 2015-11-23 17:32:22 +01:00
Arthur Schiwon d43abd0b8f public interfaces for Comments 2015-11-23 17:21:11 +01:00
Joas Schilling 018bd3ee24 Make sure that object id can be a string 2015-11-23 14:18:15 +01:00
Thomas Müller 021137d7ae Merge pull request #20679 from owncloud/fixphpdoc
p() supports string as argument
2015-11-23 13:32:46 +01:00
Thomas Müller df906f475b Merge pull request #20617 from owncloud/fix-usage-of-lastInsertId
lastInsertId() is properly working with Oracle if the table name is p…
2015-11-23 13:29:08 +01:00
Thomas Müller 240cc1c4ea Merge pull request #20543 from owncloud/share2.0_fix_hooks
Move hook and delete children logic to share manager
2015-11-23 13:28:49 +01:00
Joas Schilling 1c7d7288c4 Add a method to select a field or value with alias 2015-11-23 13:28:31 +01:00
Roeland Jago Douma 1360e22d7c [Sharing 2.0] Move hook and delete children logic to share manager
To make sure hooks are always fired and child entries are always
cleaned. This logic is moved to the share manager.

* Updated unit tests
2015-11-23 12:28:54 +01:00
Morris Jobke 7138659080 p() supports string as argument 2015-11-23 12:11:23 +01:00
Thomas Müller aa660ec232 Throw an exception in case no table name is passed into lastInsertId of the Oracle adapter 2015-11-23 11:44:08 +01:00
Thomas Müller 9f69021691 lastInsertId() is properly working with Oracle if the table name is properly passed in 2015-11-23 11:44:08 +01:00
Joas Schilling a370c29068 Use a DateTime object instead of a timestamp 2015-11-23 11:41:48 +01:00
Individual IT Services 1835462ec4 reuse code 2015-11-23 11:02:46 +01:00
Thomas Müller 79bbda994b Merge pull request #16902 from owncloud/jsocclient
Web UI uses Webdav instead of ajax/* calls
2015-11-23 09:38:01 +01:00
Thomas Müller 2f89eef334 Merge pull request #20524 from owncloud/pgsql-version-check-error
assume pgsql >=9 if checking the version fails
2015-11-23 09:05:13 +01:00
Jenkins for ownCloud 56e05a90af [tx-robot] updated from transifex 2015-11-23 01:54:53 -05:00
Robin Appelman 2d7c9f0ba9 also match ie11 with Request::USER_AGENT_IE 2015-11-22 16:05:52 +01:00
Vincent Petry a05e40932c Now using IE8 workaround of davclient.js for all IE versions 2015-11-22 16:05:52 +01:00
Thomas Müller 358858c9e3 Fix undefined HTTP_USER_AGENT 2015-11-22 16:05:50 +01:00
Vincent Petry fb3d5c7856 Add evert's davclient.js + es6-promise + IE8 workaround
- Add davclient.js lib
- Add es6-promise required by that lib
- Wrote IE8 workaround lib/shim for davclient.js
2015-11-22 16:05:49 +01:00
Jenkins for ownCloud 2321cc4854 [tx-robot] updated from transifex 2015-11-22 01:54:53 -05:00
Thomas Müller 427d107b9f Merge pull request #20614 from owncloud/use-mocks-when-testing-isSharingDisabledForUser
Use mocks when testing isSharingDisabledForUser
2015-11-20 15:22:52 +01:00
Thomas Müller c86483f3ed Adjust PHPDoc as suggested 2015-11-20 14:38:29 +01:00
Thomas Müller 479cee66f4 Merge pull request #20034 from owncloud/icewind_streams_0.3.0
Bump 3rdparty for icewind/streams-0.3.0
2015-11-19 17:27:30 +01:00
Thomas Müller c565a7b042 Fix failing unit test 💀 2015-11-19 16:44:43 +01:00
Joas Schilling ea2bf49be0 Adjust the method name to reflect it's behaviour 2015-11-19 16:08:21 +01:00
Thomas Müller 30dd610a2a Merge pull request #20563 from owncloud/systemtags-interface
Core interfaces for system tags and its manager
2015-11-19 15:47:47 +01:00
Thomas Müller 9ec2850c78 Use mocks when testing isSharingDisabledForUser 2015-11-19 15:36:16 +01:00
Roeland Jago Douma ca6bd5cacd Follow the interface again 2015-11-19 14:34:59 +01:00
Robin Appelman 888df3933d take the etag of child mounts into account for the folder etag
this replaces shared etag propagation
2015-11-19 13:32:00 +01:00
Robin Appelman d006a7c723 Fix scanning of incomplete folders 2015-11-19 13:30:50 +01:00
Robin Appelman 02f847bc66 use relative path 2015-11-19 13:30:50 +01:00
Robin Appelman d5061b8139 fix fileinfo for non existing files 2015-11-19 13:30:50 +01:00
Robin Appelman 1736c70075 Some more cleanup for getFIleInfo/getDirectoryContent 2015-11-19 13:30:50 +01:00
Robin Appelman 02be8a3a12 Split getting cache entry and checking update to a seperate method 2015-11-19 13:29:12 +01:00
Vincent Petry ea6341df29 Core interfaces for system tags and its manager 2015-11-19 11:31:25 +01:00
Morris Jobke 1189528e1f Always installed apps includes the hardcoded ones from shipped.json
* fixes #20568
2015-11-19 09:11:14 +01:00
Jenkins for ownCloud c10be07c1b [tx-robot] updated from transifex 2015-11-19 01:55:13 -05:00
Robin Appelman 71b86c0ed4 Handle non existing users as owner in the view 2015-11-18 17:08:18 +01:00
Thomas Müller 8d74e28af6 Merge pull request #20438 from owncloud/memcache-key-length-fix
Handle errors on memcached level
2015-11-18 14:20:57 +01:00
Thomas Müller 6efa72867a Merge pull request #20540 from owncloud/fix-format-call-in-logger
Try to fix logging errors we did observe on ci ....
2015-11-18 09:18:54 +01:00
Thomas Müller aba119951e Merge pull request #20494 from owncloud/storage-forbidden-exception
Allow storage wrappers to through a forbidden exception with retry information for clients
2015-11-18 09:13:45 +01:00
Thomas Müller 8db9c49cda Merge pull request #20551 from owncloud/notification-fix-tests
Notification - Add tests for the "return $this" and fix it on the new method
2015-11-18 09:12:27 +01:00
Thomas Müller 20c251a575 Merge pull request #20188 from owncloud/webdav-exposeshareowner
Expose share owner id and display name via files webdav
2015-11-17 13:40:25 +01:00