Compare commits

...

2 Commits

Author SHA1 Message Date
Robin Appelman f29d9f1a1f
add high level guide for using the filesystem api
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-05-14 16:57:36 +02:00
Robin Appelman 4dac11336d
Add basic overview of filesystem architecture
Signed-off-by: Robin Appelman <robin@icewind.nl>
2021-05-14 16:23:34 +02:00
2 changed files with 210 additions and 0 deletions

View File

@ -0,0 +1,131 @@
# Nextcloud filesystem API
## High level overview
The Nextcloud filesystem is roughly based on the unix filesystem, consisting of multiple storages
mounted at various locations.
```
┌──────────────────────────────────┐
│Code wanting to use the filesystem│
└─────────┬─────────────────────┬──┘
│ │
│ │
┌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┐
╎Filesystem │ │ ╎
╎layer │new │legacy ╎
╎ │ │ ╎
╎ ▼ ▼ ╎
╎ ┌────────┐ Partly build on ┌─┴──────┐ ╎
╎ │Node API├─────────────────►│View API│ ╎
╎ └───────┬┘ └─┬──────┘ ╎
╎ │ │ ╎
└╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┘
│ │
┌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┐
╎Storage layer │ │ ╎
╎ ├─────────────────────┤ ╎
╎ │ │ ╎
╎ ▼ ▼ ╎
╎ ┌───────┐ ┌───────┐ ┌──────┐ ╎
╎ │Storage│═══>│Scanner│═══>│Cache │ ╎
╎ └───────┘ └───────┘ └──────┘ ╎
╎ ╎
╎ ╎
└╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┘
```
### Filesystem layer
Any code that wants to use the filesystem has two API options to use, the new `Node` api and the old `View` api.
New code should preferably use the `Node` api as it allows building systems with less overhead than the old api.
Besides the filesystem apis, this layer also manages the available mounts, containing the logic to allow apps
to setup their mounts and translating filesystem paths into a mountpoint + "internal" path.
### Storage layer
The storage implementation handles the details of communicating with the filesystem or remote storage api
and provide a uniform api for Nextcloud to use the storage.
For each storage a metadata cache/index is maintained to allow reading metadata of the storage without having
to talk to the (potentially) slow storage backend. The scanner is responsible for updating the cache with
information from the storage backend.
## Storage/Cache wrappers
To allow apps to customize the behaviour of a storage without requiring the app to implement this for every
possible storage backend, a `Wrapper` system is used.
A `Wrapper` encapsulates an inner storage and allows overwriting any method to customize its behavior, with
all other methods being passed through to the inner storage.
Generally search storage wrapper has an equivalent cache wrapper encapsulating the cache of the inner storage
to provide the same behavior modifications when reading metadata from the cache.
Wrappers can be layered to stack the behavior of the wrappers, for example the `groupfolders` app works by
stacking a wrapper to provide access to a single folder on the root storage with a wrapper to limit the permissions
of the storage.
```
┌───────────────┐ ┌────────────────────┐
│PermissionsMask├─────►│CachePermissionsMask│ PermissionsMask applies a mask to the permissions of a storage
└───────┬───────┘ └─────────┬──────────┘ to provide less-privilaged access to a storage
│ │
▼ ▼
┌───────────────┐ ┌────────────────────┐
│Jail ├─────►│CacheJail │ Jail restricts access to a file or folder of a storage providing
└───────┬───────┘ └─────────┬──────────┘ a limited view into the storage (think unix chroot or bind mount)
│ │
▼ ▼
┌───────────────┐ ┌────────────────────┐
│Base Storage ├─────►│Base Cache │
└───────────────┘ └────────────────────┘
```
## Code Map
Approximate overview of the significant filesystem code
#### AppData
High level api for accessing "appdata" folders, based on the `Node` API
#### Cache
- `Cache` implementation
- Cache wrappers
- Scanner and cache update logic
- Search infrastructure
#### Mount
Mountpoint management and setup
#### Node
`Node` filesystem api implementation
#### ObjectStorage
Implementation of the various supported object store storage backends
#### SimpleFS
Simplified version of the Node api, for providing a more limited api for some filesystem bits
#### Storage
Implementation of various storage backends and wrappers
#### Streams
Various low-level php stream wrapper used in storage implementations
#### Type
Mimetype management and detection
#### View.php
Legacy View api

View File

@ -0,0 +1,79 @@
# Nextcloud filesystem API
High level guide to using the Nextcloud filesystem API
## Node API
The "Node API" is the primary api for apps to access the Nextcloud filesystem, each item in the filesystem is
represented as either a File or Folder node with each node providing access to the relevant filesystem information
and actions for the node.
### Getting access
Access to the filesystem is provided by the `IRootFolder` which can be injected into your class.
From the root folder you can either access a user's home folder or access a file or folder by its absolute path.
```php
use OCP\Files\IRootFolder;
use OCP\IUserSession;
class FileCreator {
/** @var IUserSession */
private $userSession;
/** @var IRootFolder */
private $rootFolder;
public function __constructor(IUserSession $userSession, IRootFolder $rootFolder) {
$this->userSession = $userSession;
$this->rootFolder = $rootFolder;
}
/**
* Create a new file with specified content in the home folder of the current user
* returning the size of the resulting file.
*/
public function createUserFile(string $path, string $content): int {
$user = $this->userSession->getUser();
if ($user !== null) {
// the "user folder" corresponds to the root of the user visible files
$userFolder = $this->rootFolder->getUserFolder($user->getUID());
// paths passed to a folder method are relative to that folder
$file = $userFolder->newFile($path, $content);
return $file->getSize();
} else {
return 0;
}
}
}
```
For details on the specific methods provided by file and folder nodes see the method documentation from the `OCP\Files\File` and `OCP\Files\Folder` interfaces.
## Direct storage access
While it should be generally avoided in favor of the higher level apis,
sometimes an app needs to talk directly to the storage implementation of it's metadata cache.
You can get access to the underlying storage of a file or folder by calling `getStorage` on the node or first getting
the mountpoint by calling `getMountPoint` and getting the storage from there.
Once you have the storage instance you can use the storage api from `OCP\Files\Storage\IStorage`, note however that
all paths used in the storage api are internal to the storage, the `IMountPoint` returned from `getMountPoint` provides
methods for translating between absolute filesystem paths and internal storage paths.
If you need to query the cached metadata directory you can get the `OCP\Files\Cache\ICache` from the storage by calling `getCache`.
## Implementing a storage
The recommended way for implementing a storage backend is by sub-classing `OC\Files\Storage\Common` which provides
fallback implementations for various methods, reducing the amount of work required to implement the full storage api.
Note however that various of these fallback implementations are likely to be significantly less efficient than an
implementation of the method optimized for the abilities of the storage backend.
## Adding mounts to the filesystem
The recommended way of adding your own mounts to the filesystem from an app is implementing `OCP\Files\Config\IMountProvider`
and registering the provider using `OCP\Files\Config\IMountProviderCollection::registerProvider`.
Once registered, your provider will be called every time the filesystem is being setup for a user and your mount provider
can return a list of mounts to add for that user.