Test Frameworks

General test framework support

The Citric plugin provides Citrus specific test framework implementations on top of the existing JUnit and TestNG frameworks. This helps e.g. in the generation of Citrus specific test methods and test classes.

NOTES:

• Citrus JUnit frameworks require the JUnit plugin, while the Citrus TestNG framework requires the TestNG plugin to be enabled. (They are both bundled in IntelliJ.)
• Methods that invoke status(TestCase.DISABLED) are not (yet) recognized as ignored/disabled test methods.
• Test method code templates use the Citrus 3.x specific fully qualified name of @CitrusTest. If you already use Citrus 4.x, please update the FQN.

JUnit 5

A test class is recognized as Citrus JUnit5 when the class is annotated with one of the following:

• @org.citrusframework.junit.jupiter.CitrusSupport
• @CitrusSpringSupport
• @ExtendWith(CitrusExtension.class)
• @ExtendWith(CitrusSpringExtension.class)

A test method is recognized as such when it is annotated both with @Test and @CitrusTest/@CitrusTestSource. Accordingly, there is a code template called Citrus JUnit5 Test Method that is used for generating these test methods.

JUnit 4

A test class is recognized as Citrus JUnit4 when it subclasses one of the following:

• JUnit4CitrusSupport
• JUnit4CitrusSpringSupport

A test method is recognized as such when it is annotated both with @Test and @CitrusTest/@CitrusTestSource. Accordingly, there is a code template called Citrus JUnit4 Test Method that is used for generating these test methods.

TestNG

A test class is recognized as Citrus TestNG when it subclasses one of the following:

• TestNGCitrusSupport
• TestNGCitrusSpringSupport

A test method is recognized as such when it is annotated both with @CitrusTest/@CitrusTestSource, and either the test method or the test class is annotated with @Test. Accordingly, there is a code template called Citrus TestNG Test Method that is used for generating these test methods.

Quarkus JUnit

A test class is recognized as Citrus Quarkus when the class is annotated with the following:

• @io.quarkus.test.junit.QuarkusTest.QuarkusTest
• @org.citrusframework.quarkus.CitrusSupport

Common

@CitrusTest annotated method in non-Citrus test class

Citrus test classes must either subclass a Citrus specific support class, or be annotated with a specific annotation, or in case of a @Nested class inside a class with the aforementioned criteria.

However, when none of these criteria is met, but the test class still contains at least one @CitrusTest, @CitrusTestSource or @CitruesTestFactory annotated method, those test methods wouldn’t be processed properly.

Thus, this inspection reports test classes that contain at least one Citrus specific test method, but are not recognized as any kind of Citrus test class.

Citrus test class without Citrus test

This inspection reports Citrus JUnit/TestNG test classes that don’t contain any @CitrusTest, @CitrusTestSource or @CitrusTestFactory annotated method.

Citrus test source file and methods

Generate @CitrusTestSource test method

To complement the insertion of regular @CitrusTest test methods, this action generates a @CitrusTestSource test method based on a user-customizable code template called Citrus Test Source Method.

The action is available in the Alt+Insert (or similar) Generate menu, right under the Test Method item, in Citrus JUnit or Citrus TestNG test classes.

Settings

You can enable/disable this action in Settings > Languages & Frameworks > Citric (Citrus) , so that it can be hidden in case the project doesn’t use any non-Java DSL test.

The type annotation attribute part of the template provides code completion of the available test types/DSLs, and upon selection, inserts a reference to the corresponding constant in the TestLoader Citrus class.

The test DSLs that appear in the code completion list can be customized in the plugin settings, so that only the ones are completed that you actually use in your project. If only one DSL is selected, then upon invoking the action, the template doesn’t stop at the type part to provide a smoother and quicker insertion.

Please note that these settings are project-level ones meaning that you can configure them differently in each of your projects.

Line marker for @CitrusTestSource methods for quick navigation to test files

In the @CitrusTestSource annotation one can define test source files in various ways. To make the discovery and navigation to those files and packages simpler, this line marker provides popup menu actions to navigate to them.

The displayed options are the following:

• Open '[filename]': for the combination of the name and packageName attributes and their fallback values, the test method’s name and the package of the current test class.
• Navigate to Package '[package]': for each package in the packageScan attribute that can be resolved.
• Open '': for each classpath/file/jar resource (other types are not supported yet) in the sources attribute that can be resolved.
• No Matching Test Source File or Package: for the case when no file or package can be resolved by the annotation configuration.

@CitrusTestSource annotation configuration checks

This inspection helps discover issues and highlight improvement options on the @CitrusTestSource annotation. It provides the following checks:

• Invalid test types in the type attribute.

• The same name specified in the name attribute as the method name. This is reported when there is no packageScan and no sources attribute. It is redundant, and removing it makes the test fall back to use the test method’s name as the test source name.

• The same package specified in the packageName attribute as the test class’s package. It is redundant, and removing it makes the test fall back to use the test class’s package.

• Having specified the file extension of the test type in the values in the name attribute. They must be removed because the extension is added automatically during test execution.

• Classpath resource paths in the sources attribute that doesn’t have the file extension of the current test type are reported.

• Classpath source files that match in the name and sources attributes can be deduplicated.

Test specific icon for test source files

Citrus, apart from its Java DSL, utilizes XML, YAML and Groovy test DSLs too. To be able to distinguish those files easier,
their icons are added a test marker in line with how e.g. Java test classes are marked by IntelliJ.

Whether a file is an actual Citrus test source file is determined by the following logic:

• the file must be under a test resources root,
• if the module the file resides in has a citrus-application.properties, and that properties file has the corresponding citrus.*.file.name.pattern specified, the file has to match at least one of the patterns in that value,
• otherwise, the file name must match one of the default file name patterns coming from Citrus.

Execute test source files

Citrus tests, for the most part, work with executing JUnit and TestNG test methods. But, when Citrus tests are implemented in non-Java DSLs, it can be easier to simply run the tests from the Citrus YAML, XML or Groovy test files, instead of navigating each time to the appropriate Java test methods and executing them.

The Citric plugin provides Run and Debug actions as line markers in YAML and Groovy files, as well as in the Project View context menu, for the following combinations of DSLs and build management tools:

Currently, if the project is not a Gradle one, the gutter icon is not displayed, and the actions are not available in the Project View context menu either.

The workflow of the current Run and Debug actions are as follows:

• They look up the JUnit/TestNG test methods in the module of the test source file, that match and can execute that test source file.
• If
  • there is a single matching Java test method, it is executed automatically.
  • there are multiple ones, it displays them and lets the user choose which one to execute.

• If there is no matching test method, it looks up all @CitrusTestSource methods in the module whose test type matches the test source file’s type.
  • it presents to the user the list of test classes that contain such @CitrusTestSource methods (if there is more than 1)

• it presents to the user the list of such test method in the selected class (if there is more than 1)

• Then, if there are multiple Gradle test tasks configured, it also lets the user choose which one(s) to execute, just like when executing JUnit/TestNG methods directly.

• If already exists a Gradle Run Configuration that matches the test file, no chooser popup is displayed, instead the first found run configuration is executed.
• If there is no matching @CitrusTestSourcemethod at all for the particular test type in the module, a notification popup is displayed with that information.

NOTES:

• It is important to note that while these actions look for test methods that match the Citrus test source file, depending on the @CitrusTestSource annotation configuration on the found/selected test method, it might execute other Citrus test source files too, not just the current one.
• When looking up matching JUnit/TestNG test methods, currently only classpath, file and jar resources are recognized in the sources attribute of the @CitrusTestSource annotation.
• When the run/debug action collects matching @CitrusTestSource test methods, a progress bar dialog may be displayed. It is not cancellable at the moment because it is not straightforward how to ensure that cached values are updated/re-evaluated properly when users cancel that progress.

Navigate to test method from test source file

Currently supported Citrus test types are: YAML, Groovy, XML and Spring XML.

These actions are bound to the same shortcut key (Ctrl+Shift+T on Windows) that the ‘Java class to Java test class’ navigation action is bound to.

If there is

• no matching test method, a message is displayed about that

• if there is a single one, the corresponding file is opened in a separate editor tab, and the test method is put into focus
• if there are multiple test methods, a popup list is displayed, so that users can choose which one they want to navigate to

TestNG

@CitrusParameters annotation checks

Citrus has the @CitrusParameters annotation with which test variables can be set for the test method based on data coming from a TestNG data provider.

This inspection bundles a handful of checks for the TestNG specific @CitrusParameters annotation:

• Not used in a Citrus TestNG class.
• Not used on @CitrusTest or @CitrusTestSource annotated test method.

  @Test
  @CitrusParameters("parameter")
  public void testMethod(String param) {
  }

• Used with an empty or not specified data provider. In this case there is no data to inject into the Citrus test parameter.

  @Test
  @CitrusParameters("parameter")
  @CitrusTest
  public void testMethod(String param) {
  }

• Used along with TestNG @Parameters annotation. In this case @CitrusParameters takes precedence. A quick fix is also available to remove the @Parameters annotation.

  @Test(dataProvider = "provider")
  @CitrusParameters("parameter")
  @Parameters("parameter")
  @CitrusTest
  public void testMethod(String param) {
  }

• Used with an empty test method parameter list.

  @Test(dataProvider = "provider")
  @CitrusParameters("parameter")
  @CitrusTest
  public void testMethod() {
  }

• All parameter names in @CitrusParameters match the names in the test method parameter list. In that case @CitrusParameters can be omitted, so that the method parameter names are used as Citrus parameter names.

  @Test(dataProvider = "provider")
  @CitrusParameters({ "parameter1", "parameter2" })
  @CitrusTest
  public void testMethod(String parameter1, int parameter2) {
  }

• The number of parameters doesn’t match in the method parameter list and in the @CitrusParameters annotation.

  @Test(dataProvider = "someProvider")
  @CitrusTest
  @CitrusParameters({ "parameter1", "parameter2" })
  void testMethod(String parameter1) {
  }

Generate @CitrusParameters annotation

When one wants to use different test variable names for the TestNG test method parameters within Citrus tests, they can make use of the @CitrusParameters annotation.

To make the creation of this annotation and the parameter names within easier, this intention action on Citrus TestNG test methods generates the intention as follows:

//From:
@Test(dataProvider = "provider")
@CitrusTest
void testMethod(String strParam, int intParam) {
}
//To:
@CitrusParameters({ "strParam", "intParam" })
@Test(dataProvider = "provider")
@CitrusTest
void testMethod(String strParam, int intParam) {
}

It is available as Generate @CitrusParameters:

• in Citrus TestNG test classes,
• on Citrus TestNG test methods with at least one parameter,
• when the @CitrusParameters annotation is not specified.

The generated parameter names are the same as the method parameter names, so it might be easier to update them, rather than to specify them from scratch.

Highlight parameter names in @CitrusParameter

To provide a visual clue about what Citrus parameter a TestNG test method corresponds to, when the caret is placed on a test method parameter, the same-index Citrus parameter name is highlighted.

Highlighting happens only when the parameter count is the same in the annotation and in the method parameter list.

JUnit 5

Use of @ExtendWith() annotations with various *Extension classes

This inspection reports the following usages of the JUnit 5 @ExtendWith() annotation:

Combined use of CitrusSpringExtension and SpringExtension

The Citrus documentation says that:

@CitrusSpringSupport annotation which combines the @ExtendWith(CitrusSpringExtension.class) and @ExtendWith(SpringExtension.class) annotation

Therefore, it reports the combined use of SpringExtension and CitrusSpringExtension (in this order) in the @ExtendWith annotation, and provides a quick fix to replace them with @CitrusSpringSupport.

The following combinations are reported

@ExtendWith(SpringExtension.class)
@ExtendWith(CitrusSpringExtension.class)
class SomeTest {
}
@ExtendWith({ SpringExtension.class, CitrusSpringExtension.class })
class SomeTest {
}

Usage of @ExtendWith(CitrusExtension.class)

According to the Citrus documentation, ExtendWith(CitrusExtension.class) can be replaced with the simple @CitrusSupport annotation. A quick fix is also available to perform the actual replacement.

The following is reported:

@ExtendWith(CitrusExtension.class)
class SomeTest {
}

@CitrusTestFactory and @CitrusSpringXmlTestFactory method checks

According to the JUnit 5 documentation:

a @TestFactory method must return a single DynamicNode or a Stream, Collection, Iterable, Iterator, or array of DynamicNode instances.
Instantiable subclasses of DynamicNode are DynamicContainer and DynamicTest.

and

@TestFactory methods must not be private or static

Since, at least for now, Citrus doesn’t (seem to) add any extra functionality over @TestFactory in @CitrusTestFactory
and @CitrusSpringXmlTestFactory, the aforementioned return types and visibility modifiers are validated against @CitrusTestFactory and @CitrusSpringXmlTestFactory annotated test methods by this inspection.

Additionally, if such a method is annotated with any other JUnit or Citrus test runner related annotation (@Test, @TestFactory, @CitrusTest or @CitrusTestSource), it reports those extra annotations as well. They are not required for the test method to run.

CitrusTestFactorySupport checks

Citric provides a handful of check for Citrus’ @CitrusSpringXmlTestFactory annotation that is intended to be used for Spring XML tests:

• If the corresponding test method returns tests via CitrusTestFactorySupport that are not of type Spring XML, it may be misleading to users regarding the contents of the test method.

• If CitrusTestFactorySupport.factory(...) is passed in an invalid test type:

• If CitrusTestFactorySupport.factory(...) is called with an argument the evaluates to either “xml”, “spring” or “groovy”, a quick fix becomes available to replace the call with .xml(), .springXml() and groovy, respectively.
• If CitrusTestFactorySupport.dynamicTests(String, String...) or CitrusTestFactorySupport.dynamicTests(Class, String...) doesn’t have any test name specified.
• If CitrusTestFactorySupport.packageScan(String...) doesn’t have any package specified.
• If the test names in .dynamicTest() and .dynamicTests() end with the .xml file extension for XML and Spring XML tests.

Line marker for CitrusTestFactorySupport methods for quick navigation to test files

Using the @CitrusTestFactorySupport class one can define JUnit 5 dynamic tests for certain test source files and packages contianing them. To make the discovery and navigation to those files and packages simpler, this line marker provides popup menu actions to navigate to them.

The displayed options are the following:

• Open '[filename]': for the dynamicTest() and dynamicTests() calls.
• Navigate to Package '[package]': for each package in the packageScan() call that can be resolved.
• No Matching Test Source File or Package: for the case when no file or package can be resolved from the call arguments.

Generate @CitrusTestFactory test method

To complement the insertion of regular @CitrusTest test methods, this action generates a @CitrusTestFactory test method based on a user-customizable code template called Citrus Test Factory Method.

The action is available in the Alt+Insert (or similar) Generate menu, right under the Test Method item, in Citrus JUnit 5 test classes.

Quarkus

Citrus Quarkus test class checks

Citrus provides a Quarkus specific package to run Quarkus test in a Citrus environment. It provides the org.citrusframework.quarkus.CitrusSupport annotation (not to be confused with the JUnit5 specific org.citrusframework.junit.jupiter.CitrusSupport) that can be added on @QuarkusTest annotated test classes.

The checks in this inspection make sure that various misconfiguration of such test classes are reported.

• Test class is annotated with @CitrusSupport but not with QuarkusTest.

• Test class is annotated with the JUnit5 specific @CitrusSupport annotation and not with the Quarkus specific one.

• Test class not annotated with @CitrusSupport contains fields or methods annotated with Citrus specific annotations.