Test Frameworks
General test framework support
The Citric plugin provides Citrus specific test framework implementations on top of the existing JUnit and TestNG frameworks. This helps e.g. in the generation of Citrus specific test methods and test classes.
NOTES:
- Citrus JUnit frameworks require the JUnit plugin, while the Citrus TestNG framework requires the TestNG plugin to be enabled. (They are both bundled in IntelliJ.)
- Methods that invoke
status(TestCase.DISABLED)
are not (yet) recognized as ignored/disabled test methods. - Test method code templates use the Citrus 3.x specific fully qualified name of
@CitrusTest
. If you already use Citrus 4.x, please update the FQN.
JUnit 5
A test class is recognized as Citrus JUnit5 when the class is annotated with one of the following:
@org.citrusframework.junit.jupiter.CitrusSupport
@CitrusSpringSupport
@ExtendWith(CitrusExtension.class)
@ExtendWith(CitrusSpringExtension.class)
A test method is recognized as such when it is annotated both with @Test
and @CitrusTest
/@CitrusTestSource
. Accordingly, there is a code template called Citrus JUnit5 Test Method
that is used for generating these test methods.
JUnit 4
A test class is recognized as Citrus JUnit4 when it subclasses one of the following:
JUnit4CitrusSupport
JUnit4CitrusSpringSupport
A test method is recognized as such when it is annotated both with @Test
and @CitrusTest
/@CitrusTestSource
. Accordingly, there is a code template called Citrus JUnit4 Test Method
that is used for generating these test methods.
TestNG
A test class is recognized as Citrus TestNG when it subclasses one of the following:
TestNGCitrusSupport
TestNGCitrusSpringSupport
A test method is recognized as such when it is annotated both with @CitrusTest
/@CitrusTestSource
, and either the test method or the test class is annotated with @Test
. Accordingly, there is a code template called Citrus TestNG Test Method
that is used for generating these test methods.
Quarkus JUnit
A test class is recognized as Citrus Quarkus when the class is annotated with the following:
@io.quarkus.test.junit.QuarkusTest.QuarkusTest
@org.citrusframework.quarkus.CitrusSupport
Common
@CitrusTest annotated method in non-Citrus test class
Citrus test classes must either subclass a Citrus specific support class, or be annotated with a specific annotation, or in case of a @Nested
class inside a class with the aforementioned criteria.
However, when none of these criteria is met, but the test class still contains at least one @CitrusTest
, @CitrusTestSource
or @CitruesTestFactory
annotated method, those test methods wouldn’t be processed properly.
Thus, this inspection reports test classes that contain at least one Citrus specific test method, but are not recognized as any kind of Citrus test class.
Citrus test class without Citrus test
This inspection reports Citrus JUnit/TestNG test classes that don’t contain any @CitrusTest
, @CitrusTestSource
or @CitrusTestFactory
annotated method.
Citrus test source file and methods
Generate @CitrusTestSource test method
To complement the insertion of regular @CitrusTest
test methods, this action generates a @CitrusTestSource
test method based on a user-customizable code template called Citrus Test Source Method.
The action is available in the Alt+Insert (or similar) Generate menu, right under the Test Method item, in Citrus JUnit or Citrus TestNG test classes.
Settings
You can enable/disable this action in Settings > Languages & Frameworks > Citric (Citrus)
, so that it can be hidden in case the project doesn’t use any non-Java DSL test.
The type
annotation attribute part of the template provides code completion of the available test types/DSLs, and upon selection, inserts a reference to the corresponding constant in the TestLoader
Citrus class.
The test DSLs that appear in the code completion list can be customized in the plugin settings, so that only the ones are completed that you actually use in your project. If only one DSL is selected, then upon invoking the action, the template doesn’t stop at the type part to provide a smoother and quicker insertion.
Please note that these settings are project-level ones meaning that you can configure them differently in each of your projects.
Line marker for @CitrusTestSource methods for quick navigation to test files
In the @CitrusTestSource
annotation one can define test source files in various ways. To make the discovery and navigation to those files and packages simpler, this line marker provides popup menu actions to navigate to them.
The displayed options are the following:
Open '[filename]'
: for the combination of thename
andpackageName
attributes and their fallback values, the test method’s name and the package of the current test class.Navigate to Package '[package]'
: for each package in thepackageScan
attribute that can be resolved.Open ''
: for each classpath/file/jar resource (other types are not supported yet) in thesources
attribute that can be resolved.No Matching Test Source File or Package
: for the case when no file or package can be resolved by the annotation configuration.
@CitrusTestSource annotation configuration checks
This inspection helps discover issues and highlight improvement options on the @CitrusTestSource
annotation. It provides the following checks:
- Invalid test types in the
type
attribute.
- The same name specified in the
name
attribute as the method name. This is reported when there is nopackageScan
and nosources
attribute. It is redundant, and removing it makes the test fall back to use the test method’s name as the test source name.
- The same package specified in the
packageName
attribute as the test class’s package. It is redundant, and removing it makes the test fall back to use the test class’s package.
- Having specified the file extension of the test type in the values in the
name
attribute. They must be removed because the extension is added automatically during test execution.
- Classpath resource paths in the
sources
attribute that doesn’t have the file extension of the current test type are reported.
- Classpath source files that match in the
name
andsources
attributes can be deduplicated.
Test specific icon for test source files
Citrus, apart from its Java DSL, utilizes XML, YAML and Groovy test DSLs too. To be able to distinguish those files easier,
their icons are added a test marker in line with how e.g. Java test classes are marked by IntelliJ.
Whether a file is an actual Citrus test source file is determined by the following logic:
- the file must be under a test resources root,
- if the module the file resides in has a
citrus-application.properties
, and that properties file has the correspondingcitrus.*.file.name.pattern
specified, the file has to match at least one of the patterns in that value, - otherwise, the file name must match one of the default file name patterns coming from Citrus.
Execute test source files
Citrus tests, for the most part, work with executing JUnit and TestNG test methods. But, when Citrus tests are implemented in non-Java DSLs, it can be easier to simply run the tests from the Citrus YAML, XML or Groovy test files, instead of navigating each time to the appropriate Java test methods and executing them.
The Citric plugin provides Run and Debug actions as line markers in YAML and Groovy files, as well as in the Project View context menu, for the following combinations of DSLs and build management tools:
DSL | Gradle | Maven | Comments | Context menu comments |
---|---|---|---|---|
YAML | + | Displayed for root-level actions properties. | Since TestNG tests may be executed from YAML configuration files, Run/Debug actions provided by the TestNG plugin itself may also be displayed besides the ones Citric provides. | |
XML | + | Displayed for root-level <test> tags with the http://citrusframework.org/schema/xml/testcase namespace. | ||
Spring XML | + | Displayed for the first <testcase> tag within a file with the http://citrusframework.org/schema/testcase namespace whose name attribute equals the containing file name without extension. | ||
Groovy | + | Displayed for the first root-level occurrence of any method from TestCaseScript or ActionsBuilder (except doFinally() ). | Run/Debug actions provided by the JUnit/TestNG plugins may also be displayed besides the ones Citric provides. |
Currently, if the project is not a Gradle one, the gutter icon is not displayed, and the actions are not available in the Project View context menu either.
The workflow of the current Run and Debug actions are as follows:
- They look up the JUnit/TestNG test methods in the module of the test source file, that match and can execute that test source file.
- If
- there is a single matching Java test method, it is executed automatically.
- there are multiple ones, it displays them and lets the user choose which one to execute.
- If there is no matching test method, it looks up all
@CitrusTestSource
methods in the module whose test type matches the test source file’s type.- it presents to the user the list of test classes that contain such
@CitrusTestSource
methods (if there is more than 1)
- it presents to the user the list of test classes that contain such
- it presents to the user the list of such test method in the selected class (if there is more than 1)
- Then, if there are multiple Gradle test tasks configured, it also lets the user choose which one(s) to execute, just like when executing JUnit/TestNG methods directly.
- If already exists a Gradle Run Configuration that matches the test file, no chooser popup is displayed, instead the first found run configuration is executed.
- If there is no matching
@CitrusTestSource
method at all for the particular test type in the module, a notification popup is displayed with that information.
NOTES:
- It is important to note that while these actions look for test methods that match the Citrus test source file, depending on the
@CitrusTestSource
annotation configuration on the found/selected test method, it might execute other Citrus test source files too, not just the current one. - When looking up matching JUnit/TestNG test methods, currently only classpath, file and jar resources are recognized in the
sources
attribute of the@CitrusTestSource
annotation. - When the run/debug action collects matching
@CitrusTestSource
test methods, a progress bar dialog may be displayed. It is not cancellable at the moment because it is not straightforward how to ensure that cached values are updated/re-evaluated properly when users cancel that progress.
Navigate to test method from test source file
These action complement the line marker based navigation from Java test methods to Citrus test source files, and provide a way to navigate to a matching Java test method from a Citrus test source file.Currently supported Citrus test types are: YAML, Groovy, XML and Spring XML.
These actions are bound to the same shortcut key (Ctrl+Shift+T on Windows) that the ‘Java class to Java test class’ navigation action is bound to.
If there is
- no matching test method, a message is displayed about that
- if there is a single one, the corresponding file is opened in a separate editor tab, and the test method is put into focus
- if there are multiple test methods, a popup list is displayed, so that users can choose which one they want to navigate to
TestNG
@CitrusParameters annotation checks
Citrus has the @CitrusParameters
annotation with which test variables can be set for the test method based on data coming from a TestNG data provider.
This inspection bundles a handful of checks for the TestNG specific @CitrusParameters
annotation:
- Not used in a Citrus TestNG class.
- Not used on
@CitrusTest
or@CitrusTestSource
annotated test method.
@Test
@CitrusParameters("parameter")
public void testMethod(String param) {
}
- Used with an empty or not specified data provider. In this case there is no data to inject into the Citrus test parameter.
@Test
@CitrusParameters("parameter")
@CitrusTest
public void testMethod(String param) {
}
- Used along with TestNG
@Parameters
annotation. In this case@CitrusParameters
takes precedence. A quick fix is also available to remove the@Parameters
annotation.
@Test(dataProvider = "provider")
@CitrusParameters("parameter")
@Parameters("parameter")
@CitrusTest
public void testMethod(String param) {
}
- Used with an empty test method parameter list.
@Test(dataProvider = "provider")
@CitrusParameters("parameter")
@CitrusTest
public void testMethod() {
}
- All parameter names in
@CitrusParameters
match the names in the test method parameter list. In that case@CitrusParameters
can be omitted, so that the method parameter names are used as Citrus parameter names.
@Test(dataProvider = "provider")
@CitrusParameters({ "parameter1", "parameter2" })
@CitrusTest
public void testMethod(String parameter1, int parameter2) {
}
- The number of parameters doesn’t match in the method parameter list and in the
@CitrusParameters
annotation.
@Test(dataProvider = "someProvider")
@CitrusTest
@CitrusParameters({ "parameter1", "parameter2" })
void testMethod(String parameter1) {
}
Generate @CitrusParameters annotation
When one wants to use different test variable names for the TestNG test method parameters within Citrus tests, they can make use of the @CitrusParameters
annotation.
To make the creation of this annotation and the parameter names within easier, this intention action on Citrus TestNG test methods generates the intention as follows:
//From:
@Test(dataProvider = "provider")
@CitrusTest
void testMethod(String strParam, int intParam) {
}
//To:
@CitrusParameters({ "strParam", "intParam" })
@Test(dataProvider = "provider")
@CitrusTest
void testMethod(String strParam, int intParam) {
}
It is available as Generate @CitrusParameters:
- in Citrus TestNG test classes,
- on Citrus TestNG test methods with at least one parameter,
- when the
@CitrusParameters
annotation is not specified.
The generated parameter names are the same as the method parameter names, so it might be easier to update them, rather than to specify them from scratch.
Highlight parameter names in @CitrusParameter
To provide a visual clue about what Citrus parameter a TestNG test method corresponds to, when the caret is placed on a test method parameter, the same-index Citrus parameter name is highlighted.
Highlighting happens only when the parameter count is the same in the annotation and in the method parameter list.
JUnit 5
Use of @ExtendWith() annotations with various *Extension classes
This inspection reports the following usages of the JUnit 5 @ExtendWith()
annotation:
Combined use of CitrusSpringExtension and SpringExtension
The Citrus documentation says that:
@CitrusSpringSupport annotation which combines the @ExtendWith(CitrusSpringExtension.class) and @ExtendWith(SpringExtension.class) annotation
Therefore, it reports the combined use of SpringExtension
and CitrusSpringExtension
(in this order) in the @ExtendWith
annotation, and provides a quick fix to replace them with @CitrusSpringSupport
.
The following combinations are reported
@ExtendWith(SpringExtension.class)
@ExtendWith(CitrusSpringExtension.class)
class SomeTest {
}
@ExtendWith({ SpringExtension.class, CitrusSpringExtension.class })
class SomeTest {
}
Usage of @ExtendWith(CitrusExtension.class)
According to the Citrus documentation, ExtendWith(CitrusExtension.class)
can be replaced with the simple @CitrusSupport
annotation. A quick fix is also available to perform the actual replacement.
The following is reported:
@ExtendWith(CitrusExtension.class)
class SomeTest {
}
@CitrusTestFactory and @CitrusSpringXmlTestFactory method checks
According to the JUnit 5 documentation:
a @TestFactory method must return a single DynamicNode or a Stream, Collection, Iterable, Iterator, or array of DynamicNode instances.
Instantiable subclasses of DynamicNode are DynamicContainer and DynamicTest.
and
@TestFactory methods must not be private or static
Since, at least for now, Citrus doesn’t (seem to) add any extra functionality over @TestFactory
in @CitrusTestFactory
and @CitrusSpringXmlTestFactory
, the aforementioned return types and visibility modifiers are validated against @CitrusTestFactory
and @CitrusSpringXmlTestFactory
annotated test methods by this inspection.
Additionally, if such a method is annotated with any other JUnit or Citrus test runner related annotation (@Test
, @TestFactory
, @CitrusTest
or @CitrusTestSource
), it reports those extra annotations as well. They are not required for the test method to run.
CitrusTestFactorySupport checks
Citric provides a handful of check for Citrus’ @CitrusSpringXmlTestFactory
annotation that is intended to be used for Spring XML tests:
- If the corresponding test method returns tests via
CitrusTestFactorySupport
that are not of type Spring XML, it may be misleading to users regarding the contents of the test method.
- If
CitrusTestFactorySupport.factory(...)
is passed in an invalid test type:
- If
CitrusTestFactorySupport.factory(...)
is called with an argument the evaluates to either “xml”, “spring” or “groovy”, a quick fix becomes available to replace the call with.xml()
,.springXml()
andgroovy
, respectively. - If
CitrusTestFactorySupport.dynamicTests(String, String...)
orCitrusTestFactorySupport.dynamicTests(Class, String...)
doesn’t have any test name specified. - If
CitrusTestFactorySupport.packageScan(String...)
doesn’t have any package specified. - If the test names in
.dynamicTest()
and.dynamicTests()
end with the.xml
file extension for XML and Spring XML tests.
Line marker for CitrusTestFactorySupport methods for quick navigation to test files
Using the @CitrusTestFactorySupport
class one can define JUnit 5 dynamic tests for certain test source files and packages contianing them. To make the discovery and navigation to those files and packages simpler, this line marker provides popup menu actions to navigate to them.
The displayed options are the following:
Open '[filename]'
: for thedynamicTest()
anddynamicTests()
calls.Navigate to Package '[package]'
: for each package in thepackageScan()
call that can be resolved.No Matching Test Source File or Package
: for the case when no file or package can be resolved from the call arguments.
Generate @CitrusTestFactory test method
To complement the insertion of regular @CitrusTest
test methods, this action generates a @CitrusTestFactory
test method based on a user-customizable code template called Citrus Test Factory Method.
The action is available in the Alt+Insert (or similar) Generate menu, right under the Test Method item, in Citrus JUnit 5 test classes.
Quarkus
Citrus Quarkus test class checks
Citrus provides a Quarkus specific package to run Quarkus test in a Citrus environment. It provides the org.citrusframework.quarkus.CitrusSupport
annotation (not to be confused with the JUnit5 specific org.citrusframework.junit.jupiter.CitrusSupport
) that can be added on @QuarkusTest
annotated test classes.
The checks in this inspection make sure that various misconfiguration of such test classes are reported.
- Test class is annotated with
@CitrusSupport
but not withQuarkusTest
.
- Test class is annotated with the JUnit5 specific
@CitrusSupport
annotation and not with the Quarkus specific one.
- Test class not annotated with
@CitrusSupport
contains fields or methods annotated with Citrus specific annotations.