Running categorized Tests using JUnit, Maven and Annotated-Test Suites

December 6th, 2012 by Micha Kops

Sometimes we need to classify the tests in a project and a possible solution to achieve this goal is to assign different categories to the tests.

Often we’re doing this to separate the execution of fast-running and long-running tests or to run a specific set of tests that is only applicable in special situations.

To run a specific set of categorized tests there are different options and in the following tutorial we’ll be covering two of them: by configuring the Maven Surefire Plug-in or by using a JUnit Test Suite and the JUnit annotations.

Dependencies
JUnit Test Categories
Test with Categories assigned
Test Profiles in Maven
1. Running with the default Profile
2. Running with a specific Profile
Using a Test Suite
Tutorial Sources
Resources

Dependencies

As you might have imagined, we’re in need of a testing framework here namely JUnit .. adding this one dependency to your pom.xml should be sufficient to run the following examples

<dependencies>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.11</version>
        <scope>test</scope>
    </dependency>
</dependencies>

JUnit Test Categories

Now we should add some test categories .. a test category is a simple marker interface – that means an empy interface. We’re adding two test categories here:

The firs one is used for slow running tests:

package com.hascode.tutorial.test.group;
 
/**
 * Marker interface for fast-running tests.
 */
public interface FastRunningTests {}

The following one is for slow running tests:

package com.hascode.tutorial.test.group;
 
/**
 * Marker interface for slow-running tests.
 */
public interface SlowRunningTests {}

Test with Categories assigned

Now that we’ve got test categories we should create some tests and assign the categories to them – either on class level or method level …

So this is our first test class and we’re assigning the category for fast running tests on class level

package com.hascode.tutorial.test;
 
import static org.junit.Assert.assertTrue;
 
import org.junit.Test;
import org.junit.experimental.categories.Category;
 
import com.hascode.tutorial.test.group.FastRunningTests;
 
@Category({ FastRunningTests.class })
public class FirstTest {
 @Test
 public void testSth() throws Exception {
    System.out.println("FirstTest.testSth run");
    assertTrue(true);
 }
 
 @Test
 public void testAnotherThing() throws Exception {
    System.out.println("FirstTest.testAnotherThing run");
    assertTrue(true);
 }
}

And another test class with a mixed set of slow and fast categorized test methods assigned at method level

package com.hascode.tutorial.test;
 
import static org.junit.Assert.assertTrue;
 
import org.junit.Test;
import org.junit.experimental.categories.Category;
 
import com.hascode.tutorial.test.group.FastRunningTests;
import com.hascode.tutorial.test.group.SlowRunningTests;
 
public class SecondTest {
 @Test
 @Category({ FastRunningTests.class, SlowRunningTests.class })
  public void testSth() throws Exception {
    System.out.println("SecondTest.testSth run");
    assertTrue(true);
 }
 
 @Test
 @Category({ SlowRunningTests.class })
 public void testAnotherThing() throws Exception {
    System.out.println("SecondTest.testAnotherThing run");
    assertTrue(true);
 }
}

Test Profiles in Maven

If you want to handle different test runs with different categories then one solution is to use maven profiles here.

We’re defining one maven profile for the fast running tests – this one is enabled by default – and another for the slow running tests.

<profiles>
    <profile>
        <id>fastTests</id>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                        <groups>com.hascode.tutorial.test.group.FastRunningTests</groups>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>
    <profile>
        <id>slowTests</id>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                        <groups>com.hascode.tutorial.test.group.SlowRunningTests</groups>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

Running with the default Profile

These profiles allow you to select which test categories you’d like to run … in the first example we’re running with the default profile “fastTest“.

$ mvn test
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Concurrency config is parallel='none', perCoreThreadCount=true, threadCount=2, useUnlimitedThreads=false
Running com.hascode.tutorial.test.SecondTest
SecondTest.testSth run
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
Running com.hascode.tutorial.test.FirstTest
FirstTest.testSth run
FirstTest.testAnotherThing run
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
 
Results :
 
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0

Running with a specific Profile

If you’d like to run a specific profile and the tests with the assigned category you may specify the profile using -PprofileName

$ mvn -Dtest=SlowRunningTestSuite -o test
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Concurrency config is parallel='none', perCoreThreadCount=true, threadCount=2, useUnlimitedThreads=false
Running com.hascode.tutorial.test.SecondTest
SecondTest.testSth run
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec
 
Results :
 
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

Using a Test Suite

Another viable option is to create a classic test suite and assign the test categories affected in there using annotations.

JUnit Test Suite for fast-running tests

package com.hascode.tutorial.test.suite;
 
import org.junit.experimental.categories.Categories;
import org.junit.experimental.categories.Categories.IncludeCategory;
import org.junit.runner.RunWith;
import org.junit.runners.Suite.SuiteClasses;
 
import com.hascode.tutorial.test.FirstTest;
import com.hascode.tutorial.test.SecondTest;
import com.hascode.tutorial.test.group.FastRunningTests;
 
@RunWith(Categories.class)
@IncludeCategory(FastRunningTests.class)
@SuiteClasses({ FirstTest.class, SecondTest.class })
public class FastRunningTestSuite {}

JUnit Test Suite for slow-running tests

package com.hascode.tutorial.test.suite;
 
import org.junit.experimental.categories.Categories;
import org.junit.experimental.categories.Categories.IncludeCategory;
import org.junit.runner.RunWith;
import org.junit.runners.Suite.SuiteClasses;
 
import com.hascode.tutorial.test.FirstTest;
import com.hascode.tutorial.test.SecondTest;
import com.hascode.tutorial.test.group.SlowRunningTests;
 
@RunWith(Categories.class)
@IncludeCategory(SlowRunningTests.class)
@SuiteClasses({ FirstTest.class, SecondTest.class })
public class SlowRunningTestSuite {}

Tutorial Sources

Please feel free to download the tutorial sources from my Bitbucket repository, fork it there or clone it using Git:

git clone https://bitbucket.org/hascode/junit-category-selection.git

Resources

Tags: categories, junit, maven, suite, surefire, tdd, testing

This entry was posted on Thursday, December 6th, 2012 at 10:02 pm and is filed under Java, testing. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.

3 Responses to “Running categorized Tests using JUnit, Maven and Annotated-Test Suites”

Laura Chen Says:
April 14th, 2014 at 3:00 am
Hi,
I was trying JUnit categories annotation with group parameter in maven surefire plugin to run Unit Tests or Integration Tests. It looks like the JUnit categories annotation doesnt work for method level tests, but only works for class level.
Is there anything wrong with my configuration? See details below.
Please help out ASAP.

Setups in pom.xml

junit
junit
4.8.2
test

……

org.apache.maven.plugins
maven-surefire-plugin
2.17

testSuite.FastTests

The JUnit Categories interface marker
package testSuite;
public interface FastTests {
}

Here’s the method level JUnit Categories annotation
public class BackendServiceTest extends TestCase {
@Test
@Category(FastTests.class)
public void testInitialize(){
EagleEyeBackendService.start();
Assert.assertNotNull(CassiniService.getInstance());
……
}

Thanks
Laura
Kamaraju Says:
July 7th, 2014 at 8:12 am
I followed the above mentioned steps but i see the below error when running the tests

java.lang.NullPointerException
at org.junit.runner.Description.getAnnotation(Description.java:253)
at org.junit.experimental.categories.Categories.assertNoDescendantsHaveCategoryAnnotations(Categories.java:188)
at org.junit.experimental.categories.Categories.assertNoDescendantsHaveCategoryAnnotations(Categories.java:191)
at org.junit.experimental.categories.Categories.assertNoDescendantsHaveCategoryAnnotations(Categories.java:191)
at org.junit.experimental.categories.Categories.assertNoCategorizedDescendentsOfUncategorizeableParents(Categories.java:179)
at org.junit.experimental.categories.Categories.assertNoCategorizedDescendentsOfUncategorizeableParents(Categories.java:182)
at org.junit.experimental.categories.Categories.(Categories.java:164)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBuilder.java:33)
at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(AnnotatedBuilder.java:21)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:26)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:41)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
micha kops Says:
July 7th, 2014 at 6:19 pm
Hi Kamaraju,

from the stacktrace I can see that you’re using IntelliJ Idea .. I’m not able to reproduce your problem running the test with version 13.1 – perhaps you’re using an older version with an older internal version of jUnit installed (Settings > Plugins > jUnit)?