Add helper context to isolate the test server with Docker containers
Scenarios in acceptance tests must be independent one of each other. That is, the execution of one scenario can not affect the execution of another scenario, nor it can depend on the result of the execution of a different scenario. Each scenario must be isolated and self-contained. As the acceptance tests are run against a Nextcloud server the server must be in a known and predefined initial state each time a scenario begins. The NextcloudTestServerContext is introduced to automatically set up the Nextcloud test server for each scenario. This can be achieved using Docker containers. Before an scenario begins a new Docker container with a Nextcloud server is run; the scenario is then run against the server provided by the container. When the scenario ends the container is destroyed. As long as the Nextcloud server uses local data storage each scenario is thus isolated from the rest. The NextcloudTestServerContext also notifies its sibling RawMinkContexts about the base URL of the Nextcloud test server being used in each scenario. Although it uses the Behat context system, NextcloudTestServerContext is not really part of the acceptance tests, but a provider of core features needed by them; it can be seen as part of a Nextcloud acceptance test library. Therefore, those classes are stored in the "core" directory instead of the "bootstrap" directory. Besides its own (quite limited) autoload configuration, Behat also uses the Composer autoloader, so the "core" directory has to be added there for its classes to be found by Behat. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
This commit is contained in:
Daniel Calviño Sánchez
parent
b5fba56499
commit
4c620f1fcb
|
|
@ -5,5 +5,10 @@
|
|||
"behat/mink-extension": "*",
|
||||
"behat/mink-selenium2-driver": "*",
|
||||
"phpunit/phpunit": "~4.6"
|
||||
},
|
||||
"autoload": {
|
||||
"psr-4": {
|
||||
"": "features/core"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -6,6 +6,8 @@ default:
|
|||
paths:
|
||||
- %paths.base%/../features
|
||||
contexts:
|
||||
- NextcloudTestServerContext
|
||||
|
||||
- FeatureContext
|
||||
extensions:
|
||||
Behat\MinkExtension:
|
||||
|
|
|
|||
|
|
@ -0,0 +1,156 @@
|
|||
<?php
|
||||
|
||||
/**
|
||||
*
|
||||
* @copyright Copyright (c) 2017, Daniel Calviño Sánchez (danxuliu@gmail.com)
|
||||
*
|
||||
* @license GNU AGPL version 3 or any later version
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as
|
||||
* published by the Free Software Foundation, either version 3 of the
|
||||
* License, or (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
*
|
||||
*/
|
||||
|
||||
use Behat\Behat\Context\Context;
|
||||
use Behat\Behat\Hook\Scope\BeforeScenarioScope;
|
||||
|
||||
/**
|
||||
* Behat context to run each scenario against a clean Nextcloud server.
|
||||
*
|
||||
* Before each scenario is run, this context sets up a fresh Nextcloud server
|
||||
* with predefined data and configuration. Thanks to this every scenario is
|
||||
* independent from the others and they all know the initial state of the
|
||||
* server.
|
||||
*
|
||||
* This context is expected to be used along with RawMinkContext contexts (or
|
||||
* subclasses). As the server address can be different for each scenario, this
|
||||
* context automatically sets the "base_url" parameter of all its sibling
|
||||
* RawMinkContexts; just add NextcloudTestServerContext to the context list of a
|
||||
* suite in "behat.yml".
|
||||
*
|
||||
* The Nextcloud server is set up by running a new Docker container; the Docker
|
||||
* image used by the container must provide a Nextcloud server ready to be used
|
||||
* by the tests. By default, the image "nextcloud-local-test-acceptance" is
|
||||
* used, although that can be customized using the "dockerImageName" parameter
|
||||
* in "behat.yml". In the same way, the range of ports in which the Nextcloud
|
||||
* server will be published in the local host (by default, "15000-16000") can be
|
||||
* customized using the "hostPortRangeForContainer" parameter.
|
||||
*
|
||||
* Note that using Docker containers as a regular user requires giving access to
|
||||
* the Docker daemon to that user. Unfortunately, that makes possible for that
|
||||
* user to get root privileges for the system. Please see the
|
||||
* NextcloudTestServerDockerHelper documentation for further information on this
|
||||
* issue.
|
||||
*/
|
||||
class NextcloudTestServerContext implements Context {
|
||||
|
||||
/**
|
||||
* @var NextcloudTestServerDockerHelper
|
||||
*/
|
||||
private $dockerHelper;
|
||||
|
||||
/**
|
||||
* Creates a new NextcloudTestServerContext.
|
||||
*
|
||||
* @param string $dockerImageName the name of the Docker image that provides
|
||||
* the Nextcloud test server.
|
||||
* @param string $hostPortRangeForContainer the range of local ports in the
|
||||
* host in which the port 80 of the container can be published.
|
||||
*/
|
||||
public function __construct($dockerImageName = "nextcloud-local-test-acceptance", $hostPortRangeForContainer = "15000-16000") {
|
||||
$this->dockerHelper = new NextcloudTestServerDockerHelper($dockerImageName, $hostPortRangeForContainer);
|
||||
}
|
||||
|
||||
/**
|
||||
* @BeforeScenario
|
||||
*
|
||||
* Sets up the Nextcloud test server before each scenario.
|
||||
*
|
||||
* It starts the Docker container and, once ready, it sets the "base_url"
|
||||
* parameter of the sibling RawMinkContexts to "http://" followed by the IP
|
||||
* address and port of the container; if the Docker container can not be
|
||||
* started after some time an exception is thrown (as it is just a warning
|
||||
* for the test runner and nothing to be explicitly catched a plain base
|
||||
* Exception is used).
|
||||
*
|
||||
* @param \Behat\Behat\Hook\Scope\BeforeScenarioScope $scope the
|
||||
* BeforeScenario hook scope.
|
||||
* @throws \Exception if the Docker container can not be started.
|
||||
*/
|
||||
public function startNextcloudTestServer(BeforeScenarioScope $scope) {
|
||||
$this->dockerHelper->createAndStartContainer();
|
||||
|
||||
$serverAddress = $this->dockerHelper->getNextcloudTestServerAddress();
|
||||
|
||||
$isServerReadyCallback = function() use ($serverAddress) {
|
||||
return $this->isServerReady($serverAddress);
|
||||
};
|
||||
$timeout = 10;
|
||||
$timeoutStep = 0.5;
|
||||
if (!Utils::waitFor($isServerReadyCallback, $timeout, $timeoutStep)) {
|
||||
throw new Exception("Docker container for Nextcloud could not be started");
|
||||
}
|
||||
|
||||
$this->setBaseUrlInSiblingRawMinkContexts($scope, "http://" . $serverAddress . "/index.php");
|
||||
}
|
||||
|
||||
/**
|
||||
* @AfterScenario
|
||||
*
|
||||
* Cleans up the Nextcloud test server after each scenario.
|
||||
*
|
||||
* It stops and removes the Docker container; if the Docker container can
|
||||
* not be removed after some time an exception is thrown (as it is just a
|
||||
* warning for the test runner and nothing to be explicitly catched a plain
|
||||
* base Exception is used).
|
||||
*
|
||||
* @throws \Exception if the Docker container can not be removed.
|
||||
*/
|
||||
public function stopNextcloudTestServer() {
|
||||
$this->dockerHelper->stopAndRemoveContainer();
|
||||
|
||||
$wasContainerRemovedCallback = function() {
|
||||
return !$this->dockerHelper->isContainerRegistered();
|
||||
};
|
||||
$timeout = 10;
|
||||
$timeoutStep = 0.5;
|
||||
if (!Utils::waitFor($wasContainerRemovedCallback, $timeout, $timeoutStep)) {
|
||||
throw new Exception("Docker container for Nextcloud (" . $this->dockerHelper->getContainerName() . ") could not be removed");
|
||||
}
|
||||
}
|
||||
|
||||
private function isServerReady($serverAddress) {
|
||||
$curlHandle = curl_init("http://" . $serverAddress);
|
||||
|
||||
// Returning the transfer as the result of curl_exec prevents the
|
||||
// transfer from being written to the output.
|
||||
curl_setopt($curlHandle, CURLOPT_RETURNTRANSFER, true);
|
||||
|
||||
$transfer = curl_exec($curlHandle);
|
||||
|
||||
curl_close($curlHandle);
|
||||
|
||||
return $transfer !== false;
|
||||
}
|
||||
|
||||
private function setBaseUrlInSiblingRawMinkContexts(BeforeScenarioScope $scope, $baseUrl) {
|
||||
$environment = $scope->getEnvironment();
|
||||
|
||||
foreach ($environment->getContexts() as $context) {
|
||||
if ($context instanceof Behat\MinkExtension\Context\RawMinkContext) {
|
||||
$context->setMinkParameter("base_url", $baseUrl);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
|
@ -0,0 +1,206 @@
|
|||
<?php
|
||||
|
||||
/**
|
||||
*
|
||||
* @copyright Copyright (c) 2017, Daniel Calviño Sánchez (danxuliu@gmail.com)
|
||||
*
|
||||
* @license GNU AGPL version 3 or any later version
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as
|
||||
* published by the Free Software Foundation, either version 3 of the
|
||||
* License, or (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
*
|
||||
*/
|
||||
|
||||
/**
|
||||
* Helper to manage the Docker container for the Nextcloud test server.
|
||||
*
|
||||
* The NextcloudTestServerDockerHelper abstracts the calls to the Docker Command
|
||||
* Line Interface (the "docker" command) to run, get information from, and
|
||||
* destroy containers. It is not a generic abstraction, but one tailored
|
||||
* specifically to the Nextcloud test server; a Docker image that provides an
|
||||
* installed and ready to run Nextcloud server with the configuration and data
|
||||
* expected by the acceptance tests must be available in the system. The
|
||||
* Nextcloud server must use a local storage so all the changes it makes are
|
||||
* confined to its running container.
|
||||
*
|
||||
* Also, the Nextcloud server installed in the Docker image is expected to see
|
||||
* "127.0.0.1" as a trusted domain (which would be the case if it was installed
|
||||
* by running "occ maintenance:install"). Therefore, the Nextcloud server is
|
||||
* accessed through a local port in the host system mapped to the port 80 of the
|
||||
* Docker container; if the Nextcloud server was instead accessed directly
|
||||
* through its IP address it would complain that it was being accessed from an
|
||||
* untrusted domain and refuse to work until the admin whitelisted it. The IP
|
||||
* address and port to access the Nextcloud server can be got from
|
||||
* "getNextcloudTestServerAddress".
|
||||
*
|
||||
* For better compatibility, Docker CLI commands used internally follow the
|
||||
* pre-1.13 syntax (also available in 1.13 and newer). For example,
|
||||
* "docker start" instead of "docker container start".
|
||||
*
|
||||
* In any case, the "docker" command requires special permissions to talk to the
|
||||
* Docker daemon, and those permissions are typically available only to the root
|
||||
* user. However, you should NOT run the acceptance tests as root, but as a
|
||||
* regular user instead. Please see the Docker documentation to find out how to
|
||||
* give access to a regular user to the Docker daemon:
|
||||
* https://docs.docker.com/engine/installation/linux/linux-postinstall/
|
||||
*
|
||||
* Note, however, that being able to communicate with the Docker daemon is the
|
||||
* same as being able to get root privileges for the system. Therefore, you must
|
||||
* give access to the Docker daemon (and thus run the acceptance tests as) ONLY
|
||||
* to trusted and secure users:
|
||||
* https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface
|
||||
*
|
||||
* All the public methods that use the 'docker' command throw an exception if
|
||||
* the command can not be executed or if it does not have enough permissions to
|
||||
* connect to the Docker daemon; as, due to the current use of this class, it is
|
||||
* just a warning for the test runner and nothing to be explicitly catched a
|
||||
* plain base Exception is used.
|
||||
*/
|
||||
class NextcloudTestServerDockerHelper {
|
||||
|
||||
/**
|
||||
* @var string
|
||||
*/
|
||||
private $imageName;
|
||||
|
||||
/**
|
||||
* @var string
|
||||
*/
|
||||
private $hostPortRangeForContainer;
|
||||
|
||||
/**
|
||||
* @var string
|
||||
*/
|
||||
private $containerName;
|
||||
|
||||
/**
|
||||
* Creates a new NextcloudTestServerDockerHelper.
|
||||
*
|
||||
* @param string $imageName the name of the Docker image that provides the
|
||||
* Nextcloud test server.
|
||||
* @param string $hostPortRangeForContainer the range of local ports in the
|
||||
* host in which the port 80 of the container can be published.
|
||||
*/
|
||||
public function __construct($imageName = "nextcloud-local-test-acceptance", $hostPortRangeForContainer = "15000-16000") {
|
||||
$this->imageName = $imageName;
|
||||
$this->hostPortRangeForContainer = $hostPortRangeForContainer;
|
||||
$this->containerName = null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates and starts the container.
|
||||
*
|
||||
* Note that, even if the container has started, the server it contains may
|
||||
* not have started yet when this method returns.
|
||||
*
|
||||
* @throws \Exception if the Docker command failed to execute.
|
||||
*/
|
||||
public function createAndStartContainer() {
|
||||
$moreEntropy = true;
|
||||
$this->containerName = uniqid($this->imageName . "-", $moreEntropy);
|
||||
|
||||
// There is no need to start the web server as root, so it is started
|
||||
// directly as www-data instead.
|
||||
// The port 80 of the container is mapped to a free port from a range in
|
||||
// the host system; due to this it can be accessed from the host using
|
||||
// the "127.0.0.1" IP address, which prevents Nextcloud from complaining
|
||||
// that it is being accessed from an untrusted domain.
|
||||
$this->executeDockerCommand("run --detach --user=www-data --publish 127.0.0.1:" . $this->hostPortRangeForContainer . ":80 --name=" . $this->containerName . " " . $this->imageName);
|
||||
}
|
||||
|
||||
/**
|
||||
* Stops and removes the container.
|
||||
*
|
||||
* @throws \Exception if the Docker command failed to execute.
|
||||
*/
|
||||
public function stopAndRemoveContainer() {
|
||||
// Although the Nextcloud image does not define a volume "--volumes" is
|
||||
// used anyway just in case any of its ancestor images does.
|
||||
$this->executeDockerCommand("rm --volumes --force " . $this->containerName);
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the container name.
|
||||
*
|
||||
* If the container has not been created yet the container name will be
|
||||
* null.
|
||||
*
|
||||
* @return string the container name.
|
||||
*/
|
||||
public function getContainerName() {
|
||||
return $this->containerName;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the IP address and port of the Nextcloud test server (which is
|
||||
* mapped to a local port in the host).
|
||||
*
|
||||
* @return string the IP address and port as "$ipAddress:$port".
|
||||
* @throws \Exception if the Docker command failed to execute or the
|
||||
* container is not running.
|
||||
*/
|
||||
public function getNextcloudTestServerAddress() {
|
||||
return $this->executeDockerCommand("port " . $this->containerName . " 80");
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns whether the container is running or not.
|
||||
*
|
||||
* @return boolean true if the container is running, false otherwise.
|
||||
* @throws \Exception if the Docker command failed to execute.
|
||||
*/
|
||||
public function isContainerRunning() {
|
||||
// By default, "docker ps" only shows running containers, and the
|
||||
// "--quiet" option only shows the ID of the matching containers,
|
||||
// without table headers. Therefore, if the container is not running the
|
||||
// output will be empty (not even a new line, as the last line of output
|
||||
// returned by "executeDockerCommand" does not include a trailing new
|
||||
// line character).
|
||||
return $this->executeDockerCommand("ps --quiet --filter 'name=" . $this->containerName . "'") !== "";
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns whether the container exists (no matter its state) or not.
|
||||
*
|
||||
* @return boolean true if the container exists, false otherwise.
|
||||
* @throws \Exception if the Docker command failed to execute.
|
||||
*/
|
||||
public function isContainerRegistered() {
|
||||
// With the "--quiet" option "docker ps" only shows the ID of the
|
||||
// matching containers, without table headers. Therefore, if the
|
||||
// container does not exist the output will be empty (not even a new
|
||||
// line, as the last line of output returned by "executeDockerCommand"
|
||||
// does not include a trailing new line character).
|
||||
return $this->executeDockerCommand("ps --all --quiet --filter 'name=" . $this->containerName . "'") !== "";
|
||||
}
|
||||
|
||||
/**
|
||||
* Executes the given Docker command.
|
||||
*
|
||||
* @return string the last line of output, without trailing new line
|
||||
* character.
|
||||
* @throws \Exception if the Docker command failed to execute.
|
||||
*/
|
||||
private function executeDockerCommand($dockerCommand) {
|
||||
$output = array();
|
||||
$returnValue = 0;
|
||||
$lastLine = exec("docker " . $dockerCommand . " 2>&1", $output, $returnValue);
|
||||
|
||||
if ($returnValue !== 0) {
|
||||
throw new Exception("Failed to execute 'docker " . $dockerCommand . "': " . implode("\n", $output));
|
||||
}
|
||||
|
||||
return $lastLine;
|
||||
}
|
||||
|
||||
}
|
||||
|
|
@ -0,0 +1,59 @@
|
|||
<?php
|
||||
|
||||
/**
|
||||
*
|
||||
* @copyright Copyright (c) 2017, Daniel Calviño Sánchez (danxuliu@gmail.com)
|
||||
*
|
||||
* @license GNU AGPL version 3 or any later version
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as
|
||||
* published by the Free Software Foundation, either version 3 of the
|
||||
* License, or (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
*
|
||||
*/
|
||||
|
||||
class Utils {
|
||||
|
||||
/**
|
||||
* Waits at most $timeout seconds for the given condition to be true,
|
||||
* checking it again every $timeoutStep seconds.
|
||||
*
|
||||
* Note that the timeout is no longer taken into account when a condition is
|
||||
* met; that is, true will be returned if the condition is met before the
|
||||
* timeout expires, but also if it is met exactly when the timeout expires.
|
||||
* For example, even if the timeout is set to 0, the condition will be
|
||||
* checked at least once, and true will be returned in that case if the
|
||||
* condition was met.
|
||||
*
|
||||
* @param \Closure $conditionCallback the condition to wait for, as a
|
||||
* function that returns a boolean.
|
||||
* @param float $timeout the number of seconds (decimals allowed) to wait at
|
||||
* most for the condition to be true.
|
||||
* @param float $timeoutStep the number of seconds (decimals allowed) to
|
||||
* wait before checking the condition again.
|
||||
* @return boolean true if the condition is met before (or exactly when) the
|
||||
* timeout expires, false otherwise.
|
||||
*/
|
||||
public static function waitFor($conditionCallback, $timeout, $timeoutStep) {
|
||||
$elapsedTime = 0;
|
||||
$conditionMet = false;
|
||||
|
||||
while (!($conditionMet = $conditionCallback()) && $elapsedTime < $timeout) {
|
||||
usleep($timeoutStep * 1000000);
|
||||
|
||||
$elapsedTime += $timeoutStep;
|
||||
}
|
||||
|
||||
return $conditionMet;
|
||||
}
|
||||
|
||||
}
|
||||
Loading…
Reference in New Issue