mbk-lab
/
nextcloud
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
nextcloud/tests/acceptance/config/behat.yml

78 lines
2.0 KiB
YAML
Raw Normal View History

Add basic files for the automated acceptance test system The acceptance tests verify that a Nextcloud server works as expected from the point of view of an end-user. They are specified as user stories using Behat paired with Mink, which provides web browser automation. Mink supports several browser emulators, but the system is set up to use Selenium, as it is FOSS and the one that better reflects the use of a web browser by an end-user (as, in fact, it controls real web browsers). Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2017-04-04 17:11:20 +03:00
default:
suites:
default:
paths:
- %paths.base%/../features
contexts:
Add actors for test scenarios An actor plays the role of an end-user in the test scenario. As such, each actor has its own web browser session used to perform the actions specified by the steps of the scenario. Only one actor is active at a time in a test scenario, and the current actor can be set through the "I act as XXX" step; from then on, all the steps are performed by that actor, until a different actor is set by calling "I act as XXX" again. If no actor was explicitly set in a scenario then the default actor, unsurprisingly named "default", is the one used. The ActorContext class is added to provide automatic support for all that. To use the ActorContext, besides adding it to the context list in "behat.yml", a Mink session for each actor used in the features must be specified in "behat.yml". Once done other Contexts just need to implement the ActorAwareInterface (which can be done simply by using the ActorAware trait) to have access to the current Actor object of the test scenario; as the Actor object provides its own session other Contexts do not need to extend from RawMinkContext. The ActorContext is itself a RawMinkContext, so it automatically receives the base URL of the Nextcloud test server run by NextcloudTestServerContext and propagates that base URL to all the actors. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2017-04-04 17:18:05 +03:00
- ActorContext
Add helper context to isolate the test server with Docker containers Scenarios in acceptance tests must be independent one of each other. That is, the execution of one scenario can not affect the execution of another scenario, nor it can depend on the result of the execution of a different scenario. Each scenario must be isolated and self-contained. As the acceptance tests are run against a Nextcloud server the server must be in a known and predefined initial state each time a scenario begins. The NextcloudTestServerContext is introduced to automatically set up the Nextcloud test server for each scenario. This can be achieved using Docker containers. Before an scenario begins a new Docker container with a Nextcloud server is run; the scenario is then run against the server provided by the container. When the scenario ends the container is destroyed. As long as the Nextcloud server uses local data storage each scenario is thus isolated from the rest. The NextcloudTestServerContext also notifies its sibling RawMinkContexts about the base URL of the Nextcloud test server being used in each scenario. Although it uses the Behat context system, NextcloudTestServerContext is not really part of the acceptance tests, but a provider of core features needed by them; it can be seen as part of a Nextcloud acceptance test library. Therefore, those classes are stored in the "core" directory instead of the "bootstrap" directory. Besides its own (quite limited) autoload configuration, Behat also uses the Composer autoloader, so the "core" directory has to be added there for its classes to be found by Behat. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2017-04-04 17:11:34 +03:00
- NextcloudTestServerContext
Extract app navigation elements to its own context The app navigation is not exclusive to the Files app but a generic component used by other apps too, so its locators and steps should be in its own context. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2017-07-08 16:36:01 +03:00
- AppNavigationContext
Users list acceptance tests Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2018-05-09 20:53:08 +03:00
- AppSettingsContext
Add basic acceptance tests for apps management Signed-off-by: Julius Härtl <jus@bitgrid.net>
2018-06-09 12:37:41 +03:00
- AppsManagementContext
add acceptance tests Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2017-10-29 00:57:35 +03:00
- CommentsAppContext
Header acceptance features Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2018-03-08 18:33:33 +03:00
- ContactsMenuContext
Users list acceptance tests Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2018-05-09 20:53:08 +03:00
- DialogContext
Add basic files for the automated acceptance test system The acceptance tests verify that a Nextcloud server works as expected from the point of view of an end-user. They are specified as user stories using Behat paired with Mink, which provides web browser automation. Mink supports several browser emulators, but the system is set up to use Selenium, as it is FOSS and the one that better reflects the use of a web browser by an end-user (as, in fact, it controls real web browsers). Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2017-04-04 17:11:20 +03:00
- FeatureContext
Extract file list locators and steps to its own class Besides the extraction some minor adjustments (sorting locators for file action menu entries to reflect the order of the menu entries in the UI, moving parametrized locators like "createMenuItemFor" above the locators that use them and placing "descendantOf" calls always in a new line) were made too. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-02-19 01:52:44 +03:00
- FileListContext
Add acceptance tests related to login Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2017-04-04 18:20:19 +03:00
- FilesAppContext
Extract locators and steps for sharing in Files app to their own context Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-11-23 15:22:08 +03:00
- FilesAppSharingContext
Add acceptance tests related to login Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2017-04-04 18:20:19 +03:00
- LoginPageContext
- NotificationContext
Rename "FilesSharingAppContext" to "PublicShareContext" Having both "FilesAppSharingContext" and "FilesSharingAppContext" was confusing, so "FilesSharingAppContext" was renamed to a more descriptive name. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-11-23 15:32:27 +03:00
- PublicShareContext
Add acceptance tests for unselecting items in dropdown for tags Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2017-07-08 16:46:16 +03:00
- SettingsContext
Add acceptance tests related to login Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2017-04-04 18:20:19 +03:00
- SettingsMenuContext
Add acceptance tests for setting the color in the Theming app Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2017-08-04 14:20:22 +03:00
- ThemingAppContext
Add acceptance tests related to login Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2017-04-04 18:20:19 +03:00
- UsersSettingsContext
Move acceptance tests that crash the PHP built-in server to Apache The PHP built-in server can crash when certain actions are performed in Nextcloud (but although the crash is triggered by Nextcloud it does not seem to be a Nextcloud bug), which can lead to failures in the acceptance tests that would have otherwise passed. A crash of the PHP built-in server during an acceptance test can be identified by the message "sh: 1: kill: No such process" in the acceptance tests output; as the PHP built-in server crashed its process does no longer exist when it is tried to be killed when the scenario ends. Although the crash has been observed in other tests too it is more prevalent in the tests for tags and the theming app. In order to reduce the false positives those tests are now run on Apache instead of on the PHP built-in sever. However, the rest of tests are still run on the PHP built-in server due to its lower resource consumption. In order to run a feature or just a scenario using Apache it has to be tagged with "@apache"; features or scenarios without that tag (the default) will run on the PHP built-in server instead. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-10-28 21:51:13 +03:00
filters:
tags: "~@apache"
apache:
paths:
- %paths.base%/../features
contexts:
- ActorContext
- NextcloudTestServerContext:
nextcloudTestServerHelper: NextcloudTestServerLocalApacheHelper
- AppNavigationContext
- AppSettingsContext
- AppsManagementContext
- CommentsAppContext
- ContactsMenuContext
- DialogContext
- FeatureContext
- FileListContext
- FilesAppContext
Extract locators and steps for sharing in Files app to their own context Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-11-23 15:22:08 +03:00
- FilesAppSharingContext
Move acceptance tests that crash the PHP built-in server to Apache The PHP built-in server can crash when certain actions are performed in Nextcloud (but although the crash is triggered by Nextcloud it does not seem to be a Nextcloud bug), which can lead to failures in the acceptance tests that would have otherwise passed. A crash of the PHP built-in server during an acceptance test can be identified by the message "sh: 1: kill: No such process" in the acceptance tests output; as the PHP built-in server crashed its process does no longer exist when it is tried to be killed when the scenario ends. Although the crash has been observed in other tests too it is more prevalent in the tests for tags and the theming app. In order to reduce the false positives those tests are now run on Apache instead of on the PHP built-in sever. However, the rest of tests are still run on the PHP built-in server due to its lower resource consumption. In order to run a feature or just a scenario using Apache it has to be tagged with "@apache"; features or scenarios without that tag (the default) will run on the PHP built-in server instead. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-10-28 21:51:13 +03:00
- LoginPageContext
- NotificationContext
Rename "FilesSharingAppContext" to "PublicShareContext" Having both "FilesAppSharingContext" and "FilesSharingAppContext" was confusing, so "FilesSharingAppContext" was renamed to a more descriptive name. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-11-23 15:32:27 +03:00
- PublicShareContext
Move acceptance tests that crash the PHP built-in server to Apache The PHP built-in server can crash when certain actions are performed in Nextcloud (but although the crash is triggered by Nextcloud it does not seem to be a Nextcloud bug), which can lead to failures in the acceptance tests that would have otherwise passed. A crash of the PHP built-in server during an acceptance test can be identified by the message "sh: 1: kill: No such process" in the acceptance tests output; as the PHP built-in server crashed its process does no longer exist when it is tried to be killed when the scenario ends. Although the crash has been observed in other tests too it is more prevalent in the tests for tags and the theming app. In order to reduce the false positives those tests are now run on Apache instead of on the PHP built-in sever. However, the rest of tests are still run on the PHP built-in server due to its lower resource consumption. In order to run a feature or just a scenario using Apache it has to be tagged with "@apache"; features or scenarios without that tag (the default) will run on the PHP built-in server instead. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-10-28 21:51:13 +03:00
- SettingsContext
- SettingsMenuContext
- ThemingAppContext
- UsersSettingsContext
filters:
tags: "@apache"
Add basic files for the automated acceptance test system The acceptance tests verify that a Nextcloud server works as expected from the point of view of an end-user. They are specified as user stories using Behat paired with Mink, which provides web browser automation. Mink supports several browser emulators, but the system is set up to use Selenium, as it is FOSS and the one that better reflects the use of a web browser by an end-user (as, in fact, it controls real web browsers). Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2017-04-04 17:11:20 +03:00
extensions:
Behat\MinkExtension:
sessions:
default:
Set "wd_host" directly in "behat.yml" instead of through "BEHAT_PARAMS" The "wd_host" parameter of Selenium2 sessions specify the URL used by the Selenium driver to connect with the Selenium server. Thus, when the Selenium server is at a different host or port than the default one (for example, when run on Drone) the "wd_host" parameter must be set for each of the Selenium2 sessions defined in "behat.yml". The "BEHAT_PARAMS" environment variable, which extends the "behat.yml" configuration file, was used for that. However, this required adding to the "BEHAT_PARAMS" in "run-local.sh" each new session added to "behat.yml", including those added in the acceptance tests of apps. To address that limitation, this commit introduces a simulated variable, "selenium.server"; just before the acceptance tests are run the "selenium.server" variable in the "wd_host" parameter is replaced in the "behat.yml" file used by the acceptance tests. Note that the file that is modified is the one inside the Docker container used to run the acceptance tests, so the original file is not touched. Note that a simulated variable is needed because Behat does not support overridding nor setting configuration parameters with environment variables. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-06-22 19:29:08 +03:00
selenium2:
wd_host: %selenium.server%
Add acceptance tests related to login Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2017-04-04 18:20:19 +03:00
John:
Set "wd_host" directly in "behat.yml" instead of through "BEHAT_PARAMS" The "wd_host" parameter of Selenium2 sessions specify the URL used by the Selenium driver to connect with the Selenium server. Thus, when the Selenium server is at a different host or port than the default one (for example, when run on Drone) the "wd_host" parameter must be set for each of the Selenium2 sessions defined in "behat.yml". The "BEHAT_PARAMS" environment variable, which extends the "behat.yml" configuration file, was used for that. However, this required adding to the "BEHAT_PARAMS" in "run-local.sh" each new session added to "behat.yml", including those added in the acceptance tests of apps. To address that limitation, this commit introduces a simulated variable, "selenium.server"; just before the acceptance tests are run the "selenium.server" variable in the "wd_host" parameter is replaced in the "behat.yml" file used by the acceptance tests. Note that the file that is modified is the one inside the Docker container used to run the acceptance tests, so the original file is not touched. Note that a simulated variable is needed because Behat does not support overridding nor setting configuration parameters with environment variables. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-06-22 19:29:08 +03:00
selenium2:
wd_host: %selenium.server%
Add acceptance tests related to login Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2017-04-04 18:20:19 +03:00
Jane:
Set "wd_host" directly in "behat.yml" instead of through "BEHAT_PARAMS" The "wd_host" parameter of Selenium2 sessions specify the URL used by the Selenium driver to connect with the Selenium server. Thus, when the Selenium server is at a different host or port than the default one (for example, when run on Drone) the "wd_host" parameter must be set for each of the Selenium2 sessions defined in "behat.yml". The "BEHAT_PARAMS" environment variable, which extends the "behat.yml" configuration file, was used for that. However, this required adding to the "BEHAT_PARAMS" in "run-local.sh" each new session added to "behat.yml", including those added in the acceptance tests of apps. To address that limitation, this commit introduces a simulated variable, "selenium.server"; just before the acceptance tests are run the "selenium.server" variable in the "wd_host" parameter is replaced in the "behat.yml" file used by the acceptance tests. Note that the file that is modified is the one inside the Docker container used to run the acceptance tests, so the original file is not touched. Note that a simulated variable is needed because Behat does not support overridding nor setting configuration parameters with environment variables. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-06-22 19:29:08 +03:00
selenium2:
wd_host: %selenium.server%
Add acceptance test for resharing a file with another user To reshare a file there must be at least three enabled users in the system; although it would be possible to run the steps to create a third user in the scenarios that need it for convenience a third enabled user besides "admin" and "user0" was added to the default setup. In a similar way, a new step was added too to login as a given user name, similar to the steps to log in as "user0" and as "admin". Finally, another actor, "Jim", was introduced for those scenarios which should be played by three standard actors (that is, without a special configuration like "Rubeus"). Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-11-23 14:29:30 +03:00
Jim:
selenium2:
wd_host: %selenium.server%
Add Firefox profile with support for CSS grid to acceptance tests The acceptance tests are currently run on Firefox 47; in that version the CSS grid support was not enabled by default, but it could be enabled through a setting in the Firefox profile. By default Selenium uses a clean Firefox profile when a new session is started, but it also allows the customization of the profile through a zipped "user.js" file. The contents of that file have to be provided in the "firefox_profile" capability when the Firefox session is created. In the Mink extension for Behat several Mink sessions can be defined in the "behat.yml" file. Each Mink session uses a different browser session in Selenium, and each of those browser sessions is initialized with the capabilities provided in the "behat.yml" file. From the point of view of the acceptance tests each Mink session is an actor, so different actors can use different browsers with different capabilities. Due to all this a new actor was introduced, "Rubeus", who uses a Firefox browser that has CSS grid support; this actor is meant to be used only in those acceptance tests that require proper support for CSS grids. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-05-18 14:34:41 +03:00
Rubeus:
# Rubeus uses a browser that has CSS grid support.
selenium2:
Set "wd_host" directly in "behat.yml" instead of through "BEHAT_PARAMS" The "wd_host" parameter of Selenium2 sessions specify the URL used by the Selenium driver to connect with the Selenium server. Thus, when the Selenium server is at a different host or port than the default one (for example, when run on Drone) the "wd_host" parameter must be set for each of the Selenium2 sessions defined in "behat.yml". The "BEHAT_PARAMS" environment variable, which extends the "behat.yml" configuration file, was used for that. However, this required adding to the "BEHAT_PARAMS" in "run-local.sh" each new session added to "behat.yml", including those added in the acceptance tests of apps. To address that limitation, this commit introduces a simulated variable, "selenium.server"; just before the acceptance tests are run the "selenium.server" variable in the "wd_host" parameter is replaced in the "behat.yml" file used by the acceptance tests. Note that the file that is modified is the one inside the Docker container used to run the acceptance tests, so the original file is not touched. Note that a simulated variable is needed because Behat does not support overridding nor setting configuration parameters with environment variables. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-06-22 19:29:08 +03:00
wd_host: %selenium.server%
Add Firefox profile with support for CSS grid to acceptance tests The acceptance tests are currently run on Firefox 47; in that version the CSS grid support was not enabled by default, but it could be enabled through a setting in the Firefox profile. By default Selenium uses a clean Firefox profile when a new session is started, but it also allows the customization of the profile through a zipped "user.js" file. The contents of that file have to be provided in the "firefox_profile" capability when the Firefox session is created. In the Mink extension for Behat several Mink sessions can be defined in the "behat.yml" file. Each Mink session uses a different browser session in Selenium, and each of those browser sessions is initialized with the capabilities provided in the "behat.yml" file. From the point of view of the acceptance tests each Mink session is an actor, so different actors can use different browsers with different capabilities. Due to all this a new actor was introduced, "Rubeus", who uses a Firefox browser that has CSS grid support; this actor is meant to be used only in those acceptance tests that require proper support for CSS grids. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-05-18 14:34:41 +03:00
capabilities:
firefox:
Use "paths.base" parameter for the path to the custom Firefox profile Although in the case of the acceptance tests for the server it is not strictly needed it was modified for consistency with the configuration used for the acceptance tests in apps. Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
2018-05-16 13:28:23 +03:00
profile: %paths.base%/firefox-profiles/css-grid-enabled.zip