Generating JUnit Tests with Java, EvoSuite and Maven
February 28th, 2016 by Micha KopsGenerating test suites for existing code allows us to verify the behaviour of an application before we’re making changes to its code base or for regression testing.
In the following short tutorial I’d like to demonstrate how to derive test suites from an existing Java application using EvoSuite and the EvoSuite Maven plug-in.
EvoSuite offers some nice features like running in a sandbox to avoid dangerous operations, virtual file-system and network and optimizing of different coverage criteria.
Contents
Application Code
This is our application under test. To verify the ability to create test cases for the different branches of execution flow, we’ve chosen to implement a method with a cyclomatic complexity > 1.
package com.hascode.tutorial; public class MyApp { public String getId(Long prefix, String suffix) { if (prefix == null) { throw new NullPointerException("id must not be null"); } if (suffix == null) { return prefix + "--ID--(-)"; } return prefix + "--ID--(" + suffix + ")"; } }
Maven Setup
For a seamless integration, we need to tune our pom.xml by means of..
- adding dependencies for evosuite-standalone-runtime and of course junit
- adding the evosuite-maven-plugin
- adding the EvoSuite Maven Repositories to locate the dependencies above
- adding a hook to EvoSuite in the Surefire plug-in configuration
That’s what the excerpt finally looks like:
<properties> <evosuiteVersion>1.0.1</evosuiteVersion> </properties> <dependencies> <dependency> <groupId>org.evosuite</groupId> <artifactId>evosuite-standalone-runtime</artifactId> <version>${evosuiteVersion}</version> <scope>test</scope> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.12</version> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.evosuite.plugins</groupId> <artifactId>evosuite-maven-plugin</artifactId> <version>${evosuiteVersion}</version> <executions> <execution> <goals> <goal>prepare</goal> </goals> <phase> process-test-classes </phase> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.17</version> <configuration> <properties> <property> <name>listener</name> <value>org.evosuite.runtime.InitializingListener</value> </property> </properties> </configuration> </plugin> </plugins> </build> <pluginRepositories> <pluginRepository> <id>EvoSuite</id> <name>EvoSuite Repository</name> <url>http://www.evosuite.org/m2</url> </pluginRepository> </pluginRepositories> <repositories> <repository> <id>EvoSuite</id> <name>EvoSuite Repository</name> <url>http://www.evosuite.org/m2</url> </repository> </repositories>
Generating Tests
We’re now ready to generate test suites using maven.
Different parameters allow us to fine-tune this process here, more detailed information can be found in the project’s documentation here:
- memoryInMB: maximal memory allocation for this process
- cores: number of CPU cores used
- timeInMinutesPerClass: maximal amount of time (in minutes) that may be spent analyzing/processing each class
$ mvn -DmemoryInMB=2000 -Dcores=4 evosuite:generate evosuite:export [INFO] Scanning for projects... [INFO] [INFO] Using the builder org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder with a thread count of 1 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building evosuite-testing 1.0.0 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- evosuite-maven-plugin:1.0.1:generate (default-cli) @ evosuite-testing --- [INFO] Going to generate tests with EvoSuite [INFO] Total memory: 2000mb [INFO] Time per class: 2 minutes [INFO] Number of used cores: 4 [INFO] Target: /data/project/evosuite-testing/target/classes [INFO] Basedir: /data/project/evosuite-testing [INFO] SLF4J: Class path contains multiple SLF4J bindings. [INFO] SLF4J: Found binding in [jar:file:/data/m2-repository/org/evosuite/evosuite-master/1.0.1/evosuite-master-1.0.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] [INFO] SLF4J: Found binding in [jar:file:/data/m2-repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class] [INFO] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. [INFO] SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder] [INFO] * EvoSuite 1.0.1 [INFO] Going to execute 1 jobs [INFO] Going to start job for: com.hascode.tutorial.MyApp [INFO] Removed test suites: 0 [INFO] === CTG run results === [INFO] New test suites: 1 [INFO] [INFO] --- evosuite-maven-plugin:1.0.1:export (default-cli) @ evosuite-testing --- [INFO] Exporting tests [INFO] Exported tests to /data/project/evosuite-testing/src/test/java [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 36.734 s [INFO] Finished at: 2016-02-22T20:20:19+01:00 [INFO] Final Memory: 12M/205M [INFO] ------------------------------------------------------------------------
This is how our generated test looks like. We can see that three single tests have been added covering each branch in the control flow.
/* * This file was automatically generated by EvoSuite * Mon Feb 22 19:20:18 GMT 2016 */ package com.hascode.tutorial; import org.junit.Test; import static org.junit.Assert.*; import static org.evosuite.runtime.EvoAssertions.*; import com.hascode.tutorial.MyApp; import org.evosuite.runtime.EvoRunner; import org.evosuite.runtime.EvoRunnerParameters; import org.junit.runner.RunWith; @RunWith(EvoRunner.class) @EvoRunnerParameters(mockJVMNonDeterminism = true, useVFS = true, useVNET = true, resetStaticState = true) public class MyApp_ESTest extends MyApp_ESTest_scaffolding { @Test public void test0() throws Throwable { MyApp myApp0 = new MyApp(); String string0 = myApp0.getId((Long) (-1625L), "-1625--ID--(-)"); assertEquals("-1625--ID--(-1625--ID--(-))", string0); } @Test public void test1() throws Throwable { MyApp myApp0 = new MyApp(); // Undeclared exception! try { myApp0.getId((Long) null, "-1625--ID--(-1625--ID--(-))"); fail("Expecting exception: NullPointerException"); } catch(NullPointerException e) { // // id must not be null // assertThrownBy("com.hascode.tutorial.MyApp", e); } } @Test public void test2() throws Throwable { MyApp myApp0 = new MyApp(); String string0 = myApp0.getId((Long) (-1625L), (String) null); assertEquals("-1625--ID--(-)", string0); } }
In addition to this tests, a scaffolding class for the test is generated (MyApp_ESTest_scaffolding), we may fine-tune the testing environment here, setting system properties, global timeouts etc..
Running Tests
Now we may run our tests in the IDE-of-choice or using Maven with mvn test:
$ mvn test [..] [INFO] ------------------------------------------------------------------------ [INFO] Building evosuite-testing 1.0.0 [INFO] ------------------------------------------------------------------------ [..] ------------------------------------------------------- T E S T S ------------------------------------------------------- Executing org.evosuite.runtime.InitializingListener Running com.hascode.tutorial.MyApp_ESTest Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.251 sec - in com.hascode.tutorial.MyApp_ESTest Results : Tests run: 3, Failures: 0, Errors: 0, Skipped: 0 [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------
Tutorial Sources
Please feel free to download the tutorial sources from my Bitbucket repository, fork it there or clone it using Git:
git clone https://bitbucket.org/hascode/evosuite-junit-tutorial.git
Resources
Other Testing Tutorials of mine
- Testing Asynchronous Applications with Java and Awaitility
- Mutation Testing with Pitest and Maven
- Mocking HTTP Interaction with Java, JUnit and MockServer
- Running categorized Tests using JUnit, Maven and Annotated-Test Suites
- Ordering your JUnit Rules using a RuleChain
- Mocking, Stubbing and Test Spying using the Mockito Framework and PowerMock
- Make your Tests more readable with custom Hamcrest Matchers
- Selenium WebDriver, Selenium Server and PageObjects by Example
- Performance Testing a Multiuser Web Application with JMeter and Maven
- Marrying Java EE and BDD with Cucumber, Arquillian and Cukespace
- BDD Testing with Cucumber, Java and JUnit
- Java EE: Setting up and Testing Form-Based JDBC Authentication with Arquillian and Maven
- Running JavaScript Tests with Maven, Jasmine and PhantomJS
- Writing Java Integration Tests for MongoDB
- Arquillian Transaction Extension: Transaction Rollback for your Java EE Integration Tests
- A short Introduction to ScalaTest
- and more ..
Tags: evosuite, generator, junit, maven, regression, tdd, test, testing
October 27th, 2016 at 5:15 am
Hey, what about when you have private or protected methods ? Does it deal with it ?
April 16th, 2018 at 6:03 am
i managed to get to the mvn evosuite:generate step. however, after processing, it did mentioned how many jobs left to complete and so on. however when it finally ended, it just prompt the message saying unable to generate test suites. zero generated. zero coverage. i am wondering how do u manage?
April 16th, 2018 at 8:16 am
Hi Wesley,
did you encounter this situation when running EvoSuite on your own project or when applying in to this tutorial’s sources? Which parameters did you pass for memory, cores and time-per-class? The output directory is writable?
Cheers,
Micha
April 7th, 2020 at 7:56 pm
evosuite generate fails with error, could you please help
[MASTER] 23:24:45.924 [main] ERROR ClientServices – Failed to register client services
java.rmi.ConnectException: Connection refused to host: 10.221.0.150; nested exception is:
java.net.ConnectException: Connection timed out: connect
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619) ~[na:1.8.0_171]
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216) ~[na:1.8.0_171]
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) ~[na:1.8.0_171]
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:338) ~[na:1.8.0_171]
at sun.rmi.registry.RegistryImpl_Stub.rebind(RegistryImpl_Stub.java:147) ~[na:1.8.0_171]
at org.evosuite.rmi.ClientServices.registerServices(ClientServices.java:67) ~[evosuite-master-1.0.6.jar:1.0.6]
at org.evosuite.ClientProcess.run(ClientProcess.java:82) [evosuite-master-1.0.6.jar:1.0.6]
at org.evosuite.ClientProcess.main(ClientProcess.java:203) [evosuite-master-1.0.6.jar:1.0.6]
Caused by: java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method) ~[na:1.8.0_171]
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79) ~[na:1.8.0_171]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_171]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_171]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_171]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[na:1.8.0_171]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_171]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_171]
at java.net.Socket.connect(Socket.java:538) ~[na:1.8.0_171]
at java.net.Socket.(Socket.java:434) ~[na:1.8.0_171]
at java.net.Socket.(Socket.java:211) ~[na:1.8.0_171]
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40) ~[na:1.8.0_171]
at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:148) ~[na:1.8.0_171]
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613) ~[na:1.8.0_171]
… 7 common frames omitted
[MASTER] 23:24:45.984 [main] ERROR ClientProcess – Error when generating tests for: tutorial.Evolution with seed 1586282064721. Configuration id : default
java.lang.RuntimeException: Could not connect to master process on port 11965
at org.evosuite.ClientProcess.run(ClientProcess.java:87) ~[evosuite-master-1.0.6.jar:1.0.6]
at org.evosuite.ClientProcess.main(ClientProcess.java:203) ~[evosuite-master-1.0.6.jar:1.0.6]