When run through "run.sh" the acceptance tests were executed in the same
system in which the script was called and they started and stopped the
Nextcloud server using Docker containers that provided real web servers.
For consistency now they use the same approach used when run through
Drone: the acceptance tests are run in a Docker container and they start
and stop the Nextcloud server directly using the PHP built-in web server.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Instead of running an additional Drone service with the Nextcloud server
now the Nextcloud server is run in the same Drone step as the acceptance
tests themselves using the PHP built-in web server.
Thanks to this, the Nextcloud server control is no longer needed, as the
acceptance tests can now directly reset, start and stop the Nextcloud
server. Also, the "nextcloudci/php7.0:php7.0-7" image provides
everything needed to run and manage the Nextcloud server (including the
Git command used to restore the directory to a saved state), so the
custom image is no longer needed either.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Instead of downloading the Selenium server and requiring a specific
Firefox version to be installed in the system now the Selenium server is
run using one of the official Selenium Docker images, which provides
both the Selenium server and the appropriate version of Firefox.
Moreover, as it is run inside the Docker container, the web browser is
now run in headless mode; however, if needed, it can still be viewed
through VNC.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Each acceptance test feature is run in its own Drone step. The container
of the step runs the acceptance tests themselves, but they require two
additional Drone services. One service provides the Selenium server that
performs the web browser actions specified by the tests, and the other
service provides the Nextcloud server that the tests will be run
against (due to security concerns the acceptance tests themselves can
not create Docker containers for the Nextcloud server as done when
running them in a local system, as if Drone containers had access to
Docker a malicious pull request could be used to take over the Drone
server).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Although the timeouts specified in the acceptance tests are enough in
most cases they may not be when running them in a slow system or
environment. For those situations a general multiplier for find
timeouts is added. It can be set in the "behat.yml" configuration file
to increase the timeout used in every find call (except those that used
a timeout of 0, as in those cases the element had to be already present
when finding it and whether the system is slow or not does not change
that).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Due to security concerns, the public Nextcloud server repository is not
set as "trusted" in Drone (otherwise a malicious pull request could be
used to take over the server), so it is not possible to create Docker
containers from the containers started by Drone. Therefore, the
Nextcloud server must be started as a service by Drone itself.
The NextcloudTestServerDroneHelper is added to manage from the
acceptance tests a Nextcloud test server running in a Drone service; to
be able to control the remote Nextcloud server the Drone service must
provide the Nextcloud server control server.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The installation and configuration of the Nextcloud server as expected
by the acceptance tests is extracted to its own script so it can be used
from any element that launches the acceptance tests.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Instead of depending on a Nextcloud test server created through Docker,
NextcloudTestServerContext now uses the NextcloudTestServerHelper
interface. This makes possible to provide other implementations of the
interface for those cases in which using a Docker container is not a
valid approach, like in the continuous integration system of the public
repository due to security concerns.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The NextcloudTestServerHelper interface provides the needed methods to
manage the Nextcloud server used in acceptance tests.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
For consistency with the rest of private methods in the class,
"isContainerRegistered" is moved below the only public method in which
it is used ("cleanUp").
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The acceptance tests require several elements to be set up in order to
be run. Besides those PHP packages that it depends on, like Behat or
Mink, it requires a running Selenium server and a Docker image with the
Nextcloud server to be tested available in the system. The "run.sh"
script takes care of preparing all the needed elements and then run the
acceptance tests; once finished, either normally or due to an error, it
also cleans up the temporal elements created/started by the script and
the acceptance tests.
The Docker image with the Nextcloud server to be tested is created from
the Nextcloud code in the greatparent directory each time "run.sh" is
executed; the code is copied inside the image, so once the acceptance
tests are started the code in the greatparent directory can be modified
without affecting them. As it is based on the current code at the time
of the launch that image is created and destroyed each time the
acceptance tests are run. However, the image that it is based on, which
is created using "docker/nextcloud-local-parent/Dockerfile", does not
change between runs, so it is kept built in the system to speed up the
launch of acceptance tests.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Mink elements (including the document element) provide a
"find(selector, locator)" method to look for child elements in their web
browser session. The Locator class is added to be able to store the
selector and locator in a single object; it also provides a fluent API
to ease the definition of Mink locators, specially those using the
"named" selector.
The method "find(locator, timeout, timeoutStep)" is added to Actor
objects; it is simply a wrapper over Mink's "find(selector, locator)"
method, although it throws an exception if the element can not be found
instead of returning null, and it also makes possible to automatically
retry to find the element for certain amount of time.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
An actor plays the role of an end-user in the test scenario. As such,
each actor has its own web browser session used to perform the actions
specified by the steps of the scenario. Only one actor is active at a
time in a test scenario, and the current actor can be set through the "I
act as XXX" step; from then on, all the steps are performed by that
actor, until a different actor is set by calling "I act as XXX" again.
If no actor was explicitly set in a scenario then the default actor,
unsurprisingly named "default", is the one used.
The ActorContext class is added to provide automatic support for all
that. To use the ActorContext, besides adding it to the context list in
"behat.yml", a Mink session for each actor used in the features must be
specified in "behat.yml". Once done other Contexts just need to
implement the ActorAwareInterface (which can be done simply by using the
ActorAware trait) to have access to the current Actor object of the test
scenario; as the Actor object provides its own session other Contexts do
not need to extend from RawMinkContext. The ActorContext is itself a
RawMinkContext, so it automatically receives the base URL of the
Nextcloud test server run by NextcloudTestServerContext and propagates
that base URL to all the actors.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Scenarios in acceptance tests must be independent one of each other.
That is, the execution of one scenario can not affect the execution of
another scenario, nor it can depend on the result of the execution of a
different scenario. Each scenario must be isolated and self-contained.
As the acceptance tests are run against a Nextcloud server the server
must be in a known and predefined initial state each time a scenario
begins.
The NextcloudTestServerContext is introduced to automatically set up the
Nextcloud test server for each scenario.
This can be achieved using Docker containers. Before an scenario begins
a new Docker container with a Nextcloud server is run; the scenario is
then run against the server provided by the container. When the scenario
ends the container is destroyed. As long as the Nextcloud server uses
local data storage each scenario is thus isolated from the rest.
The NextcloudTestServerContext also notifies its sibling RawMinkContexts
about the base URL of the Nextcloud test server being used in each
scenario.
Although it uses the Behat context system, NextcloudTestServerContext is
not really part of the acceptance tests, but a provider of core features
needed by them; it can be seen as part of a Nextcloud acceptance test
library. Therefore, those classes are stored in the "core" directory
instead of the "bootstrap" directory. Besides its own (quite limited)
autoload configuration, Behat also uses the Composer autoloader, so the
"core" directory has to be added there for its classes to be found by
Behat.
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
The acceptance tests verify that a Nextcloud server works as expected
from the point of view of an end-user. They are specified as user
stories using Behat paired with Mink, which provides web browser
automation.
Mink supports several browser emulators, but the system is set up to use
Selenium, as it is FOSS and the one that better reflects the use of a
web browser by an end-user (as, in fact, it controls real web browsers).
Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
Otherwise your mail program shows "foo@mail.com" instead of "Nextcloud" or whatever your instance name is.
Signed-off-by: Lukas Reschke <lukas@statuscode.ch>
While the risk is actually quite low because one would already have the user session and could potentially do other havoc it makes sense to throttle here in case of invalid previous password attempts.
Signed-off-by: Lukas Reschke <lukas@statuscode.ch>
Daniel Calviño Sánchez
Morris Jobke
Nextcloud bot
Jan-Christoph Borchardt
Joas Schilling
Lukas Reschke
Christoph Wurst