Generating JUnit Tests with Java, EvoSuite and Maven

February 28th, 2016 by

Generating test suites for existing code allows us to verify the behaviour of an application before we’re making changes to its code base or for regression testing.

In the following short tutorial I’d like to demonstrate how to derive test suites from an existing Java application using EvoSuite and the EvoSuite Maven plug-in.

EvoSuite offers some nice features like running in a sandbox to avoid dangerous operations, virtual file-system and network and optimizing of different coverage criteria.

Running generated JUnit test in Eclipse IDE.

Running generated JUnit test in Eclipse IDE.

 

Application Code

This is our application under test. To verify the ability to create test cases for the different branches of execution flow, we’ve chosen to implement a method with a cyclomatic complexity > 1.

package com.hascode.tutorial;
 
public class MyApp {
	public String getId(Long prefix, String suffix) {
		if (prefix == null) {
			throw new NullPointerException("id must not be null");
		}
 
		if (suffix == null) {
			return prefix + "--ID--(-)";
		}
 
		return prefix + "--ID--(" + suffix + ")";
	}
}

Maven Setup

For a seamless integration, we need to tune our pom.xml by means of..

  • adding dependencies for evosuite-standalone-runtime and of course junit
  • adding the evosuite-maven-plugin
  • adding the EvoSuite Maven Repositories to locate the dependencies above
  • adding a hook to EvoSuite in the Surefire plug-in configuration

That’s what the excerpt finally looks like:

<properties>
	<evosuiteVersion>1.0.1</evosuiteVersion>
</properties>
 
<dependencies>
	<dependency>
		<groupId>org.evosuite</groupId>
		<artifactId>evosuite-standalone-runtime</artifactId>
		<version>${evosuiteVersion}</version>
		<scope>test</scope>
	</dependency>
	<dependency>
		<groupId>junit</groupId>
		<artifactId>junit</artifactId>
		<version>4.12</version>
		<scope>test</scope>
	</dependency>
</dependencies>
 
<build>
	<plugins>
		<plugin>
			<groupId>org.evosuite.plugins</groupId>
			<artifactId>evosuite-maven-plugin</artifactId>
			<version>${evosuiteVersion}</version>
			<executions>
				<execution>
					<goals>
						<goal>prepare</goal>
					</goals>
					<phase> process-test-classes </phase>
				</execution>
			</executions>
		</plugin>
		<plugin>
			<groupId>org.apache.maven.plugins</groupId>
			<artifactId>maven-surefire-plugin</artifactId>
			<version>2.17</version>
			<configuration>
				<properties>
					<property>
						<name>listener</name>
						<value>org.evosuite.runtime.InitializingListener</value>
					</property>
				</properties>
			</configuration>
		</plugin>
	</plugins>
</build>
 
<pluginRepositories>
	<pluginRepository>
		<id>EvoSuite</id>
		<name>EvoSuite Repository</name>
		<url>http://www.evosuite.org/m2</url>
	</pluginRepository>
</pluginRepositories>
<repositories>
	<repository>
		<id>EvoSuite</id>
		<name>EvoSuite Repository</name>
		<url>http://www.evosuite.org/m2</url>
	</repository>
</repositories>

Generating Tests

We’re now ready to generate test suites using maven.

Different parameters allow us to fine-tune this process here, more detailed information can be found in the project’s documentation here:

  • memoryInMB: maximal memory allocation for this process
  • cores: number of CPU cores used
  • timeInMinutesPerClass: maximal amount of time (in minutes) that may be spent analyzing/processing each class
$ mvn -DmemoryInMB=2000 -Dcores=4 evosuite:generate evosuite:export
[INFO] Scanning for projects...
[INFO]
[INFO] Using the builder org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder with a thread count of 1
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building evosuite-testing 1.0.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- evosuite-maven-plugin:1.0.1:generate (default-cli) @ evosuite-testing ---
[INFO] Going to generate tests with EvoSuite
[INFO] Total memory: 2000mb
[INFO] Time per class: 2 minutes
[INFO] Number of used cores: 4
[INFO] Target: /data/project/evosuite-testing/target/classes
[INFO] Basedir: /data/project/evosuite-testing
[INFO] SLF4J: Class path contains multiple SLF4J bindings.
[INFO] SLF4J: Found binding in [jar:file:/data/m2-repository/org/evosuite/evosuite-master/1.0.1/evosuite-master-1.0.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
[INFO] SLF4J: Found binding in [jar:file:/data/m2-repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
[INFO] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
[INFO] SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
[INFO] * EvoSuite 1.0.1
[INFO] Going to execute 1 jobs
[INFO] Going to start job for: com.hascode.tutorial.MyApp
[INFO] Removed test suites: 0
[INFO] === CTG run results ===
[INFO] New test suites: 1
[INFO]
[INFO] --- evosuite-maven-plugin:1.0.1:export (default-cli) @ evosuite-testing ---
[INFO] Exporting tests
[INFO] Exported tests to /data/project/evosuite-testing/src/test/java
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 36.734 s
[INFO] Finished at: 2016-02-22T20:20:19+01:00
[INFO] Final Memory: 12M/205M
[INFO] ------------------------------------------------------------------------

This is how our generated test looks like. We can see that three single tests have been added covering each branch in the control flow.

/*
 * This file was automatically generated by EvoSuite
 * Mon Feb 22 19:20:18 GMT 2016
 */
 
package com.hascode.tutorial;
 
import org.junit.Test;
import static org.junit.Assert.*;
import static org.evosuite.runtime.EvoAssertions.*;
import com.hascode.tutorial.MyApp;
import org.evosuite.runtime.EvoRunner;
import org.evosuite.runtime.EvoRunnerParameters;
import org.junit.runner.RunWith;
 
@RunWith(EvoRunner.class) @EvoRunnerParameters(mockJVMNonDeterminism = true, useVFS = true, useVNET = true, resetStaticState = true)
public class MyApp_ESTest extends MyApp_ESTest_scaffolding {
 
  @Test
  public void test0()  throws Throwable  {
      MyApp myApp0 = new MyApp();
      String string0 = myApp0.getId((Long) (-1625L), "-1625--ID--(-)");
      assertEquals("-1625--ID--(-1625--ID--(-))", string0);
  }
 
  @Test
  public void test1()  throws Throwable  {
      MyApp myApp0 = new MyApp();
      // Undeclared exception!
      try {
        myApp0.getId((Long) null, "-1625--ID--(-1625--ID--(-))");
        fail("Expecting exception: NullPointerException");
 
      } catch(NullPointerException e) {
         //
         // id must not be null
         //
         assertThrownBy("com.hascode.tutorial.MyApp", e);
      }
  }
 
  @Test
  public void test2()  throws Throwable  {
      MyApp myApp0 = new MyApp();
      String string0 = myApp0.getId((Long) (-1625L), (String) null);
      assertEquals("-1625--ID--(-)", string0);
  }
}

In addition to this tests, a scaffolding class for the test is generated (MyApp_ESTest_scaffolding), we may fine-tune the testing environment here, setting system properties, global timeouts etc..

Running Tests

Now we may run our tests in the IDE-of-choice or using Maven with mvn test:

Running generated JUnit test in Eclipse IDE.

Running generated JUnit test in Eclipse IDE.

$ mvn test
[..]
[INFO] ------------------------------------------------------------------------
[INFO] Building evosuite-testing 1.0.0
[INFO] ------------------------------------------------------------------------
[..]
 
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Executing org.evosuite.runtime.InitializingListener
Running com.hascode.tutorial.MyApp_ESTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.251 sec - in com.hascode.tutorial.MyApp_ESTest
 
Results :
 
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

Tutorial Sources

Please feel free to download the tutorial sources from my Bitbucket repository, fork it there or clone it using Git:

git clone https://bitbucket.org/hascode/evosuite-junit-tutorial.git

Resources

Other Testing Tutorials of mine

Tags: , , , , , , ,

4 Responses to “Generating JUnit Tests with Java, EvoSuite and Maven”

  1. Julien Says:

    Hey, what about when you have private or protected methods ? Does it deal with it ?

  2. wesley Says:

    i managed to get to the mvn evosuite:generate step. however, after processing, it did mentioned how many jobs left to complete and so on. however when it finally ended, it just prompt the message saying unable to generate test suites. zero generated. zero coverage. i am wondering how do u manage?

  3. Micha Kops Says:

    Hi Wesley,
    did you encounter this situation when running EvoSuite on your own project or when applying in to this tutorial’s sources? Which parameters did you pass for memory, cores and time-per-class? The output directory is writable?

    Cheers,
    Micha

  4. pramya Says:

    evosuite generate fails with error, could you please help
    [MASTER] 23:24:45.924 [main] ERROR ClientServices – Failed to register client services
    java.rmi.ConnectException: Connection refused to host: 10.221.0.150; nested exception is:
    java.net.ConnectException: Connection timed out: connect
    at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619) ~[na:1.8.0_171]
    at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216) ~[na:1.8.0_171]
    at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) ~[na:1.8.0_171]
    at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:338) ~[na:1.8.0_171]
    at sun.rmi.registry.RegistryImpl_Stub.rebind(RegistryImpl_Stub.java:147) ~[na:1.8.0_171]
    at org.evosuite.rmi.ClientServices.registerServices(ClientServices.java:67) ~[evosuite-master-1.0.6.jar:1.0.6]
    at org.evosuite.ClientProcess.run(ClientProcess.java:82) [evosuite-master-1.0.6.jar:1.0.6]
    at org.evosuite.ClientProcess.main(ClientProcess.java:203) [evosuite-master-1.0.6.jar:1.0.6]
    Caused by: java.net.ConnectException: Connection timed out: connect
    at java.net.DualStackPlainSocketImpl.connect0(Native Method) ~[na:1.8.0_171]
    at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79) ~[na:1.8.0_171]
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_171]
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_171]
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_171]
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[na:1.8.0_171]
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_171]
    at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_171]
    at java.net.Socket.connect(Socket.java:538) ~[na:1.8.0_171]
    at java.net.Socket.(Socket.java:434) ~[na:1.8.0_171]
    at java.net.Socket.(Socket.java:211) ~[na:1.8.0_171]
    at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40) ~[na:1.8.0_171]
    at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:148) ~[na:1.8.0_171]
    at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613) ~[na:1.8.0_171]
    … 7 common frames omitted
    [MASTER] 23:24:45.984 [main] ERROR ClientProcess – Error when generating tests for: tutorial.Evolution with seed 1586282064721. Configuration id : default
    java.lang.RuntimeException: Could not connect to master process on port 11965
    at org.evosuite.ClientProcess.run(ClientProcess.java:87) ~[evosuite-master-1.0.6.jar:1.0.6]
    at org.evosuite.ClientProcess.main(ClientProcess.java:203) ~[evosuite-master-1.0.6.jar:1.0.6]

Search
Tags
Categories