Resilient Architecture in Practice – Circuit Breakers for Java: Failsafe, Javaslang, Hystrix and Vert.x

February 14th, 2017 by

When dealing with remote services or APIs there is always the risk of latency issues, failures or connection losses. The worst thing to happen is when the remote service is down and our application hangs until the underlying protocol’s (e.g. TCP) connection timeout is reached and we’re receiving an exception. Until this moment is reached, our application might hang, memory is allocated for threads or bound objects and at last, our continuous requests might prevent the remote system from recovering.

We might use timeouts here but circuit-breakers take this approach one step further: A potential failing, critical or dangerous operation is encapsulated with a circuit breaker tracking failures and when a specified threshold is reached it tripping the breaker.  Now all calls to the API fail immediately and the  system does not even try to communicate with the failing remote system or the isolated API.
This avoids  flooding a  dead remote system with requests and allocating memory and system resources for waiting threads etc. and is also a good mechanism and  integration point to track business-critical errors.

In the following tutorial I’m going to demonstrate four different circuit-breaker implementations for Java forced to interact with a failing API.

Circuit Breaker - State Diagram

Circuit Breaker - State Diagram

 

Circuit Breaker Pattern – Short Introduction

As mentioned in the introduction, circuit-breakers prevent calls to dysfunctional services or APIs. After a specified timeout is reached, the circuit-breaker tries to make a successful call to the remote-service and if successful might reset its state.

In addition I highly recommend reading Michael T Nygard: Release it! Design and deploy production ready software, an excellent book covering different stability patterns as well as anti-patterns.

Circuit-breaker-pattern is best described using a state-diagram and the following three states. The default state is often the closed-state but some of the libraries demonstrated here allow configurations to start in the open-state as well.

  • Closed: The circuit-breaker executes operations as usual. If a failure occurs, the circuit-breaker writes it down and if a specified error threshold (number of failures or frequency of failures) is reached, it trips and opens the circuit (transitions to the open-state).
  • Open: Calls to the circuit-breaker in the open state fail immediately without even trying to call the underlying operation. After a specified timeout is reached, the circuit-breaker transitions to the half-open state.
  • Half-open: In this state, one call is allowed to call the underlying operation. If this call fails, the circuit-breaker transitions to the open-state again until another timeout is reached, if it succeeds, the circuit-breaker resets and transitions to the closed-state.
Circuit Breaker - State Diagram

Circuit Breaker - State Diagram

When implementing circuit-breakers in our applications we may fine-tune which exceptions are handled how, which timeouts and recovery strategies are used for which APIs and so on. As state-changes, especially closed-to-open indicate problems, we should always add monitoring to those integration points.

Failsafe Example

Out first Java library, Jonathan Halterman’s Failsafe is a lightweight library with zero dependencies, covering different aspects of application error handling and offering the following features:

  • Circuit breakers
  • Retries
  • Fallbacks
  • Execution context
  • Event listeners
  • Asynchronous API integration
  • CompletableFuture and functional interface integration
  • Execution tracking

Souce: GitHub Project

Dependencies

We need to add just one dependency to our project’s pom.xml (using Maven).

<dependency>
    <groupId>net.jodah</groupId>
    <artifactId>failsafe</artifactId>
    <version>1.0.1</version>
</dependency>

Our unstable API

The following sample application is used as a test service that tends to be unstable. The service signals this behavior by throwing a SampleException for a given amount of API calls, having reached this limit (4), the service continuous to work without problems and returns a generated id (UUID).

package com.hascode.tutorial;
 
import java.time.ZonedDateTime;
import java.util.UUID;
import java.util.concurrent.atomic.AtomicInteger;
 
public class UnstableApplication {
    private final int MAX_FAILS = 4;
    private AtomicInteger failCount = new AtomicInteger(1);
 
    public String generateId() throws SampleException {
        if (failCount.getAndIncrement() < MAX_FAILS) {
            System.err.printf("UnstableApplication throws SampleException at '%s'\n", ZonedDateTime.now());
            throw new SampleException();
        }
 
        final String id = UUID.randomUUID().toString();
        System.out.printf("UnstableApplication: id '%s' generated at '%s'\n", id, ZonedDateTime.now());
 
        return id;
    }
 
}

Circuit Breaker Example

In our example, we’re setting up a new circuit-breaker with the following configuration:

  • Failure Threshold = 2: when 2 successive failures occur in the closed state, the circuit is opened.
  • Success Threshold = 5: when 5 successive successful calls in the half-opened state occur, the circuit is closed.
  • Delay of = 1 second: the delay to wait in the open state before transitioning to half-open.

In addition we’re setting up a retry policy with the following configuration:

  • Delay = 2 seconds: the delay between the retries.
  • Max-duration = 60 seconds: Max-duration for retries, else execution fails.
  • With back-off delay = 4, max-delay = 40 seconds: Sets the delay between retries, incrementing it exponentially backing to the max-delay of 40 seconds and multiplying successive delays by a factor of 2.

Now we’re wrapping call to the UnstableApplication and add some output including the circuit-breaker state so this is our final example application using Failsafe:

package com.hascode.tutorial.examples;
 
import java.time.ZonedDateTime;
import java.util.concurrent.TimeUnit;
 
import com.hascode.tutorial.UnstableApplication;
 
import net.jodah.failsafe.CircuitBreaker;
import net.jodah.failsafe.CircuitBreakerOpenException;
import net.jodah.failsafe.Failsafe;
import net.jodah.failsafe.RetryPolicy;
 
public class FailsafeExample {
 
    public static void main(String[] args) throws Exception {
        UnstableApplication app = new UnstableApplication();
 
        CircuitBreaker breaker = new CircuitBreaker().withFailureThreshold(2).withSuccessThreshold(5).withDelay(1,
                TimeUnit.SECONDS);
        RetryPolicy retryPolicy = new RetryPolicy().withDelay(2, TimeUnit.SECONDS).withMaxDuration(60, TimeUnit.SECONDS)
                .withBackoff(4, 40, TimeUnit.SECONDS);
 
        System.out.printf("circuit-breaker state is: %s\n", breaker.getState());
        for (int i = 0; i < 10; i++) {
            Thread.sleep(1000);
            try {
                String id = Failsafe.with(breaker).with(retryPolicy)
                        .onFailedAttempt((a, b) -> System.err.printf(
                                "failed with exception: '%s' at '%s', circuit-breaker state is: '%s'\n", b,
                                ZonedDateTime.now(), breaker.getState()))
                        .onSuccess((a, b) -> System.out.printf("call succeeded, circuit-breaker state is: '%s'\n",
                                breaker.getState()))
                        .get(app::generateId);
                System.out.printf("FailsafeExample: id '%s' received at '%s'\n", id, ZonedDateTime.now());
            } catch (CircuitBreakerOpenException e) {
                System.out.printf("circuit-breaker is open (state %s), time is '%s'\n", breaker.getState(),
                        ZonedDateTime.now());
            }
        }
    }
 
}

Running the Example

We’re now ready to run our Failsafe example using our IDE of choice, pure Java or using Maven in the command line:

$ mvn exec:java -Dexec.mainClass=com.hascode.tutorial.examples.FailsafeExample
circuit-breaker state is: CLOSED
UnstableApplication throws SampleException at '2017-02-05T20:56:57.854+01:00[Europe/Berlin]'
failed with exception: 'com.hascode.tutorial.SampleException' at '2017-02-05T20:56:57.856+01:00[Europe/Berlin]', circuit-breaker state is: 'CLOSED'
UnstableApplication throws SampleException at '2017-02-05T20:57:05.857+01:00[Europe/Berlin]'
failed with exception: 'com.hascode.tutorial.SampleException' at '2017-02-05T20:57:05.859+01:00[Europe/Berlin]', circuit-breaker state is: 'OPEN'
UnstableApplication throws SampleException at '2017-02-05T20:57:21.861+01:00[Europe/Berlin]'
failed with exception: 'com.hascode.tutorial.SampleException' at '2017-02-05T20:57:21.861+01:00[Europe/Berlin]', circuit-breaker state is: 'OPEN'
UnstableApplication: id '393e3491-68b6-4ae7-a39e-52ab87afa242' generated at '2017-02-05T20:57:53.865+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'HALF_OPEN'
FailsafeExample: id '393e3491-68b6-4ae7-a39e-52ab87afa242' received at '2017-02-05T20:57:53.866+01:00[Europe/Berlin]'
UnstableApplication: id 'b40b4f0b-de67-43fc-b46a-6445c6cf07f9' generated at '2017-02-05T20:57:54.867+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'HALF_OPEN'
FailsafeExample: id 'b40b4f0b-de67-43fc-b46a-6445c6cf07f9' received at '2017-02-05T20:57:54.867+01:00[Europe/Berlin]'
UnstableApplication: id '1de22246-d6f0-41c4-ae40-ae091e16ec54' generated at '2017-02-05T20:57:55.868+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'HALF_OPEN'
FailsafeExample: id '1de22246-d6f0-41c4-ae40-ae091e16ec54' received at '2017-02-05T20:57:55.868+01:00[Europe/Berlin]'
UnstableApplication: id '906d3783-7f5e-4be7-8abf-9244560c8da6' generated at '2017-02-05T20:57:56.869+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'HALF_OPEN'
FailsafeExample: id '906d3783-7f5e-4be7-8abf-9244560c8da6' received at '2017-02-05T20:57:56.870+01:00[Europe/Berlin]'
UnstableApplication: id '6c27e9b9-511c-464e-875b-16a273d90ba2' generated at '2017-02-05T20:57:57.870+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'CLOSED'
FailsafeExample: id '6c27e9b9-511c-464e-875b-16a273d90ba2' received at '2017-02-05T20:57:57.871+01:00[Europe/Berlin]'
UnstableApplication: id '5af68933-7b47-423a-88cf-79f9262f9cc4' generated at '2017-02-05T20:57:58.872+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'CLOSED'
FailsafeExample: id '5af68933-7b47-423a-88cf-79f9262f9cc4' received at '2017-02-05T20:57:58.873+01:00[Europe/Berlin]'
UnstableApplication: id '391f5108-5f07-4c99-bf6d-7cc5164d31d0' generated at '2017-02-05T20:57:59.873+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'CLOSED'
FailsafeExample: id '391f5108-5f07-4c99-bf6d-7cc5164d31d0' received at '2017-02-05T20:57:59.874+01:00[Europe/Berlin]'
UnstableApplication: id '7633df20-fd0f-41fe-b4be-453e30164dfe' generated at '2017-02-05T20:58:00.875+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'CLOSED'
FailsafeExample: id '7633df20-fd0f-41fe-b4be-453e30164dfe' received at '2017-02-05T20:58:00.876+01:00[Europe/Berlin]'
UnstableApplication: id '1c95c875-0679-436b-af09-0e349cded736' generated at '2017-02-05T20:58:01.876+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'CLOSED'
FailsafeExample: id '1c95c875-0679-436b-af09-0e349cded736' received at '2017-02-05T20:58:01.877+01:00[Europe/Berlin]'
UnstableApplication: id 'f28f1453-4798-4a48-8ba7-95e4bac92b9a' generated at '2017-02-05T20:58:02.878+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'CLOSED'
FailsafeExample: id 'f28f1453-4798-4a48-8ba7-95e4bac92b9a' received at '2017-02-05T20:58:02.879+01:00[Europe/Berlin]'
Failsafe Circuit Breaker running in Eclipse IDE

Failsafe Circuit Breaker running in Eclipse IDE

Javaslang Circuitbreaker Example

Javaslang-circuitbreaker is a slim library, inspired by Netflix’ Hystrix but designed for Java 8 and functional programming (Javaslang is a library for Java 8 offering persistent data types and functional programming control structures). The only dependencies this library needs, are RxJava, SLF4J and – of course Javaslang. A summary of the USPs of using this library in contrast to using Hystrix is documented on the project’s GitHub site, please feel free to read them in detail there if interested.

Dependencies

Again we only need to one dependency for Javaslang-Circuitbreaker to our “mavenized” project’s pom.xml:

<dependency>
        <groupId>io.github.robwin</groupId>
        <artifactId>javaslang-circuitbreaker</artifactId>
        <version>0.8.1</version>
    </dependency>

Circuit Breaker Example

In our Javaslang example, we’re first setting up a circuit-breaker configuration with the following settings:

  • Ring-buffer size in closed state = 2: Where Hystrix stores executions results in 10 1-second window buckets, Javaslang uses a ring-bit-buffer. Settings this to a value of 2 means, that when in closed state, at least 2 calls must be evaluated before the failure rate gets calculated.
  • Ring-buffer size in open state = 2: Same as above but configured for the open-state.
  • Failure-rate Threshold = 50: The failure rate threshold in percent above which the circuit-breaker trips open, in our case it’s 50 which is also the default.
  • Wait-duration in open-state = 100: Duration how long the circuit-breaker resides in the open-state before switching to half-open-state.

We’re initializing a named circuit-breaker in the next line using the configuration above. The Try-Supplier-structures in the next lines might seem unfamiliar for some of us, they’re Javaslang’s implementation of Scala’s Try control. We’re decorating the call to our UnstableApplication with the circuit-breaker and we’re ready to go – the rest of the code triggers the API calls and prints the breaker’s current state to STDOUT.

package com.hascode.tutorial.examples;
 
import java.time.Duration;
import java.time.ZonedDateTime;
 
import com.hascode.tutorial.UnstableApplication;
 
import io.github.robwin.circuitbreaker.CircuitBreaker;
import io.github.robwin.circuitbreaker.CircuitBreakerConfig;
import io.github.robwin.decorators.Decorators;
import javaslang.control.Try;
 
public class JavaslangExample {
 
    public static void main(String[] args) throws Exception {
        UnstableApplication app = new UnstableApplication();
 
        CircuitBreakerConfig breakerConfig = CircuitBreakerConfig.custom().ringBufferSizeInClosedState(2)
                .ringBufferSizeInHalfOpenState(2).failureRateThreshold(50)
                .waitDurationInOpenState(Duration.ofMillis(1000)).build();
        CircuitBreaker breaker = CircuitBreaker.of("unstableAppBreaker", breakerConfig);
 
        Try.CheckedSupplier<String> decoratedSupplier = Decorators.ofCheckedSupplier(app::generateId)
                .withCircuitBreaker(breaker).decorate();
 
        System.out.printf("circuit-breaker state is: %s\n", breaker.getState());
        for (int i = 0; i < 10; i++) {
            Thread.sleep(1000);
            Try<String> result = Try.of(decoratedSupplier)
                    .onSuccess((a) -> System.out.printf("call succeeded, circuit-breaker state is: '%s'\n",
                            breaker.getState()))
                    .onFailure(e -> System.err.printf(
                            "failed with exception: '%s' at '%s', circuit-breaker state is: '%s'\n", e,
                            ZonedDateTime.now(), breaker.getState()));
            if (!result.isEmpty()) {
                System.out.printf("JavaslangExample: id '%s' received at '%s'\n", result.get(), ZonedDateTime.now());
            }
        }
    }
 
}

Running the Example

We may now run our Javaslang example application using our IDE of choice or Java/Maven in the command-line:

$ mvn exec:java -Dexec.mainClass=com.hascode.tutorial.examples.JavaslangExample
circuit-breaker state is: CLOSED
UnstableApplication throws SampleException at '2017-02-05T20:54:53.301+01:00[Europe/Berlin]'
failed with exception: 'com.hascode.tutorial.SampleException' at '2017-02-05T20:54:53.307+01:00[Europe/Berlin]', circuit-breaker state is: 'CLOSED'
UnstableApplication throws SampleException at '2017-02-05T20:54:54.307+01:00[Europe/Berlin]'
failed with exception: 'com.hascode.tutorial.SampleException' at '2017-02-05T20:54:54.312+01:00[Europe/Berlin]', circuit-breaker state is: 'OPEN'
UnstableApplication throws SampleException at '2017-02-05T20:54:55.314+01:00[Europe/Berlin]'
failed with exception: 'com.hascode.tutorial.SampleException' at '2017-02-05T20:54:55.315+01:00[Europe/Berlin]', circuit-breaker state is: 'HALF_OPEN'
UnstableApplication: id '56d5d045-d7fc-41c5-b16d-f894f53e8140' generated at '2017-02-05T20:54:56.319+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'OPEN'
JavaslangExample: id '56d5d045-d7fc-41c5-b16d-f894f53e8140' received at '2017-02-05T20:54:56.321+01:00[Europe/Berlin]'
UnstableApplication: id 'e68e6b5b-e905-4fb6-9e9e-fe5402092057' generated at '2017-02-05T20:54:57.321+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'HALF_OPEN'
JavaslangExample: id 'e68e6b5b-e905-4fb6-9e9e-fe5402092057' received at '2017-02-05T20:54:57.322+01:00[Europe/Berlin]'
UnstableApplication: id '89918204-f85b-4142-94ff-24fa0035e69c' generated at '2017-02-05T20:54:58.323+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'CLOSED'
JavaslangExample: id '89918204-f85b-4142-94ff-24fa0035e69c' received at '2017-02-05T20:54:58.325+01:00[Europe/Berlin]'
UnstableApplication: id '00bb2104-421a-4333-a122-d13ea30babc6' generated at '2017-02-05T20:54:59.326+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'CLOSED'
JavaslangExample: id '00bb2104-421a-4333-a122-d13ea30babc6' received at '2017-02-05T20:54:59.326+01:00[Europe/Berlin]'
UnstableApplication: id '126d47d3-8eeb-494b-afd6-9a39626e7edf' generated at '2017-02-05T20:55:00.327+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'CLOSED'
JavaslangExample: id '126d47d3-8eeb-494b-afd6-9a39626e7edf' received at '2017-02-05T20:55:00.328+01:00[Europe/Berlin]'
UnstableApplication: id '36459b3e-644c-4aa9-9617-ae7259d09df3' generated at '2017-02-05T20:55:01.328+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'CLOSED'
JavaslangExample: id '36459b3e-644c-4aa9-9617-ae7259d09df3' received at '2017-02-05T20:55:01.329+01:00[Europe/Berlin]'
UnstableApplication: id '5dd2220e-1618-462e-bc56-2f19c6a95133' generated at '2017-02-05T20:55:02.330+01:00[Europe/Berlin]'
call succeeded, circuit-breaker state is: 'CLOSED'
JavaslangExample: id '5dd2220e-1618-462e-bc56-2f19c6a95133' received at '2017-02-05T20:55:02.330+01:00[Europe/Berlin]'
Javaslang Circuit Breaker in Eclipse IDE

Javaslang Circuit Breaker in Eclipse IDE

Hystrix Example

Hystrix is a library from the Netflix tool stack to provide possibilities to control the interaction between distributed services in regards of latency or error tolerance logic, so some main features are:

  • Circuit-breakers
  • Fallbacks
  • Timeouts
  • Monitoring / Metrics
  • Plugin-API

More detailed information is available in Hystrix’ wiki: “What is Hystrix?”.

Dependencies

We need to add only one dependency to our project’s pom.xml:

<dependency>
    <groupId>com.netflix.hystrix</groupId>
    <artifactId>hystrix-core</artifactId>
    <version>1.5.9</version>
</dependency>

Circuit Breaker Example

In the first step, we’re wrapping our logic into a Hystrix command for isolation and controlling its behavior. We’re doing this by sub-classing from HystrixObservableCommand (as we’re going to create an observable instead of a simple command, see more information). In the next step, we’re creating a configuration for this command (using a HystrixObservableCommand.Setter) including the following circuit-breaker configuration:

  • Error threshold in percentage = 50: The percentage at or above the circuit-breaker trips open.
  • Sleep window in milliseconds = 1000: The amount of time in milliseconds, after tripping the circuit, to reject requests before allowing attempts again to determine if the circuit should again be closed.
  • Request volume threshold = 1: The minimum number of requests in a rolling window to trip the circuit – in our case, it’s one..

A complete list of all configurable values is listed in Hystrix’ wiki here. Then  we’re subscribing to the observable that wraps our call to our beloved UnstableApplication, handling success and error cases in the lambda expression. Finally we’re gathering some basic statistics using Hystrix Metrics / HealthCounts and we’re printing details like the error-rate and the circuit-breaker’s state to STDOUT.

package com.hascode.tutorial.examples;
 
import java.time.ZonedDateTime;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
 
import com.hascode.tutorial.SampleException;
import com.hascode.tutorial.UnstableApplication;
import com.netflix.hystrix.HystrixCommandGroupKey;
import com.netflix.hystrix.HystrixCommandMetrics.HealthCounts;
import com.netflix.hystrix.HystrixCommandProperties;
import com.netflix.hystrix.HystrixObservableCommand;
 
import rx.Observable;
 
public class HystrixExample {
 
    static class IdGeneratingCommand extends HystrixObservableCommand<String> {
        private final UnstableApplication app;
 
        public IdGeneratingCommand(HystrixObservableCommand.Setter setter, UnstableApplication app) {
            super(setter);
            this.app = app;
        }
 
        @Override
        protected Observable<String> construct() {
            return Observable.create(observer -> {
                try {
                    if (!observer.isUnsubscribed()) {
                        observer.onNext(app.generateId());
                        observer.onCompleted();
                    }
                } catch (SampleException e) {
                    observer.onError(e);
                }
            });
        }
    };
 
    public static void main(String[] args) throws Exception {
        UnstableApplication app = new UnstableApplication();
 
        HystrixObservableCommand.Setter setter = HystrixObservableCommand.Setter
                .withGroupKey(HystrixCommandGroupKey.Factory.asKey("unstableAppCmdGroup"))
                .andCommandPropertiesDefaults(HystrixCommandProperties.Setter().withCircuitBreakerEnabled(true)
                        .withCircuitBreakerErrorThresholdPercentage(50)
                        .withCircuitBreakerSleepWindowInMilliseconds(1000).withCircuitBreakerRequestVolumeThreshold(1));
 
        for (int i = 0; i < 10; i++) {
            CountDownLatch l = new CountDownLatch(1);
            IdGeneratingCommand cmd = new IdGeneratingCommand(setter, app);
            final HealthCounts healthCounts = cmd.getMetrics().getHealthCounts();
            System.out.printf("circuit-breaker state is open: %s, %d errors of %d requests\n",
                    cmd.isCircuitBreakerOpen(), healthCounts.getErrorCount(), healthCounts.getTotalRequests());
            Observable<String> observable = cmd.observe();
            observable.subscribe(s -> {
                System.out.printf("HystrixExample: id '%s' received at '%s'\n", s, ZonedDateTime.now());
            } , t -> {
                System.err.printf("HystrixExample: error %s, circuit-breaker state is open: %s\n", t,
                        cmd.isCircuitBreakerOpen());
            } , () -> {
                l.countDown();
            });
            l.await(4, TimeUnit.SECONDS);
        }
    }
 
}

Running the Example

Our Hystrix-empowered application is now ready to run using our IDE or the command-line like this:

$ mvn exec:java -Dexec.mainClass=com.hascode.tutorial.examples.HystrixExample
circuit-breaker state is open: false, 0 errors of 0 requests
UnstableApplication throws SampleException at '2017-02-05T20:49:55.860+01:00[Europe/Berlin]'
HystrixExample: error com.netflix.hystrix.exception.HystrixRuntimeException: IdGeneratingCommand failed and no fallback available., circuit-breaker state is open: false
circuit-breaker state is open: true, 1 errors of 1 requests
HystrixExample: error com.netflix.hystrix.exception.HystrixRuntimeException: IdGeneratingCommand short-circuited and no fallback available., circuit-breaker state is open: true
circuit-breaker state is open: true, 1 errors of 1 requests
UnstableApplication throws SampleException at '2017-02-05T20:50:03.872+01:00[Europe/Berlin]'
HystrixExample: error com.netflix.hystrix.exception.HystrixRuntimeException: IdGeneratingCommand failed and no fallback available., circuit-breaker state is open: true
circuit-breaker state is open: true, 1 errors of 1 requests
UnstableApplication throws SampleException at '2017-02-05T20:50:07.875+01:00[Europe/Berlin]'
HystrixExample: error com.netflix.hystrix.exception.HystrixRuntimeException: IdGeneratingCommand failed and no fallback available., circuit-breaker state is open: true
circuit-breaker state is open: true, 2 errors of 2 requests
UnstableApplication: id '754658c6-b1f8-4276-a17f-75d8372d8511' generated at '2017-02-05T20:50:11.881+01:00[Europe/Berlin]'
HystrixExample: id '754658c6-b1f8-4276-a17f-75d8372d8511' received at '2017-02-05T20:50:11.892+01:00[Europe/Berlin]'
circuit-breaker state is open: false, 0 errors of 1 requests
UnstableApplication: id '536abd15-fff4-4dab-aea4-777f5c8eb84f' generated at '2017-02-05T20:50:11.902+01:00[Europe/Berlin]'
HystrixExample: id '536abd15-fff4-4dab-aea4-777f5c8eb84f' received at '2017-02-05T20:50:11.903+01:00[Europe/Berlin]'
circuit-breaker state is open: false, 0 errors of 1 requests
UnstableApplication: id 'ff25793f-0840-431f-a508-1968791506fa' generated at '2017-02-05T20:50:11.905+01:00[Europe/Berlin]'
HystrixExample: id 'ff25793f-0840-431f-a508-1968791506fa' received at '2017-02-05T20:50:11.906+01:00[Europe/Berlin]'
circuit-breaker state is open: false, 0 errors of 1 requests
UnstableApplication: id '8f1bae88-18ef-4a21-9a7a-eecca07ba0c5' generated at '2017-02-05T20:50:11.908+01:00[Europe/Berlin]'
HystrixExample: id '8f1bae88-18ef-4a21-9a7a-eecca07ba0c5' received at '2017-02-05T20:50:11.908+01:00[Europe/Berlin]'
circuit-breaker state is open: false, 0 errors of 1 requests
UnstableApplication: id '7b75d9b5-65dd-46d5-8266-857f503b37b4' generated at '2017-02-05T20:50:11.910+01:00[Europe/Berlin]'
HystrixExample: id '7b75d9b5-65dd-46d5-8266-857f503b37b4' received at '2017-02-05T20:50:11.911+01:00[Europe/Berlin]'
circuit-breaker state is open: false, 0 errors of 1 requests
UnstableApplication: id '2bf13dac-c445-4930-97ca-8086a4e4fde6' generated at '2017-02-05T20:50:11.913+01:00[Europe/Berlin]'
HystrixExample: id '2bf13dac-c445-4930-97ca-8086a4e4fde6' received at '2017-02-05T20:50:11.914+01:00[Europe/Berlin]'
Hystrix Circuit Breaker running in Eclipse IDE

Hystrix Circuit Breaker running in Eclipse IDE

Vert.x Example

Vert.x is a tool-kit for implementing reactive applications for the Java virtual machine offering an event-driven, non-blocking execution environment, polyglot programming API to write applications inJava, Groovy, Ruby, Python or even JavaScript and a lot of other fun stuff. I have written another article about Vert.x in my blog, please feel free to read if interested: “Creating a Websocket Chat Application with Vert.x and Java“. The Vert.x Circuit Breaker offers the classical circuit-breaker state machine and the possibility to specify fallback-behaviour.

Dependencies

We just need to add one dependency for vertx-circuit-breaker to our project’s pom.xml:

<dependency>
    <groupId>io.vertx</groupId>
    <artifactId>vertx-circuit-breaker</artifactId>
    <version>3.3.3</version>
</dependency>

Circuit Breaker Example

We’re starting by creating a new circuit-breaker instance using the build-in factory using the following configuration:

  • Max-failures = 2: After 2 failures, the circuit trips to the open-state.
  • Timeout = 2000: If the operation has not finished after 2 seconds, it counts as a failure.
  • Fallback-on-failure = true: We’re calling the fallback on failure.
  • Reset-timeout = 2000: We’re waiting for 2 seconds in the open-state before an retry.

In addition, we’re adding handlers for all three states to log state-changes and print them to STDOUT or STDERR using the circuit-breaker’s openHandler, closeHandler and halfOpenHandler methods. Finally we’re wrapping our call to the UnstableApplication and we’re running it using the configured circuit-breaker.

package com.hascode.tutorial.examples;
 
import java.time.ZonedDateTime;
 
import com.hascode.tutorial.SampleException;
import com.hascode.tutorial.UnstableApplication;
 
import io.vertx.circuitbreaker.CircuitBreaker;
import io.vertx.circuitbreaker.CircuitBreakerOptions;
import io.vertx.core.Vertx;
 
public class VertxExample {
 
    public static void main(String[] args) throws Exception {
        UnstableApplication app = new UnstableApplication();
 
        Vertx vertx = Vertx.vertx();
        CircuitBreaker breaker = CircuitBreaker
                .create("unstableAppBreaker", vertx,
                        new CircuitBreakerOptions().setMaxFailures(2).setTimeout(2000).setFallbackOnFailure(true)
                                .setResetTimeout(2000))
                .openHandler(h -> System.err.println("circuit-breaker opened"))
                .closeHandler(h -> System.out.println("circuit-breaker closed"))
                .halfOpenHandler(h -> System.err.println("circuit-breaker half-opened"));
 
        System.out.printf("circuit-breaker state is: %s\n", breaker.state());
        for (int i = 0; i < 10; i++) {
            Thread.sleep(1000);
            breaker.<String> execute(future -> {
                try {
                    final String id = app.generateId();
                    future.complete(id);
                } catch (SampleException e) {
                    future.fail(e);
                }
                if (future.failed()) {
                    System.err.printf("failed with exception: '%s' at '%s', circuit-breaker state is: '%s'\n",
                            future.cause(), ZonedDateTime.now(), breaker.state());
                }
            }).setHandler(id -> {
                if (id.succeeded())
                    System.out.printf("VertxExample: id '%s' received at '%s'\n", id, ZonedDateTime.now());
            });
        }
 
        vertx.close();
    }
}

Running the Example

We may run our Vert.x application now using our IDE or the command-line like this:

$ mvn exec:java -Dexec.mainClass=com.hascode.tutorial.examples.VertxExample
circuit-breaker state is: CLOSED
UnstableApplication throws SampleException at '2017-02-05T20:52:46.466+01:00[Europe/Berlin]'
failed with exception: 'com.hascode.tutorial.SampleException' at '2017-02-05T20:52:46.468+01:00[Europe/Berlin]', circuit-breaker state is: 'CLOSED'
UnstableApplication throws SampleException at '2017-02-05T20:52:47.469+01:00[Europe/Berlin]'
circuit-breaker opened
failed with exception: 'com.hascode.tutorial.SampleException' at '2017-02-05T20:52:47.471+01:00[Europe/Berlin]', circuit-breaker state is: 'OPEN'
circuit-breaker half-opened
UnstableApplication throws SampleException at '2017-02-05T20:52:49.473+01:00[Europe/Berlin]'
circuit-breaker opened
failed with exception: 'com.hascode.tutorial.SampleException' at '2017-02-05T20:52:49.474+01:00[Europe/Berlin]', circuit-breaker state is: 'OPEN'
circuit-breaker half-opened
UnstableApplication: id '30f446c5-0703-43b5-8af3-20fe28db52ec' generated at '2017-02-05T20:52:52.476+01:00[Europe/Berlin]'
circuit-breaker closed
VertxExample: id 'io.vertx.core.impl.FutureImpl@1450ed79' received at '2017-02-05T20:52:52.476+01:00[Europe/Berlin]'
UnstableApplication: id '5f8cfd22-980d-43dd-b84a-fb8820db2beb' generated at '2017-02-05T20:52:53.477+01:00[Europe/Berlin]'
VertxExample: id 'io.vertx.core.impl.FutureImpl@6c427ae3' received at '2017-02-05T20:52:53.478+01:00[Europe/Berlin]'
UnstableApplication: id '88c5a9f8-c465-4d69-aba9-de847fe23ff0' generated at '2017-02-05T20:52:54.479+01:00[Europe/Berlin]'
VertxExample: id 'io.vertx.core.impl.FutureImpl@6e6d85c6' received at '2017-02-05T20:52:54.479+01:00[Europe/Berlin]'
UnstableApplication: id '8dbe20a8-af4d-46b7-a648-3e7f278c7a3a' generated at '2017-02-05T20:52:55.480+01:00[Europe/Berlin]'
VertxExample: id 'io.vertx.core.impl.FutureImpl@f902e7a' received at '2017-02-05T20:52:55.481+01:00[Europe/Berlin]'
Vert.x Circuit Breaker running in Eclipse IDE

Vert.x Circuit Breaker running in Eclipse IDE

Using Hystrix in Vert.x

We may also use Hystrix instead or in combination with the circuit-breaker described above. The Vert.x website offers good documentation how to achieve this: “Vert.x: Using Netflix Hystrix“.

Tutorial Sources

Please feel free to download the tutorial sources from my Bitbucket repository, fork it there or clone it using Git:

git clone https://bitbucket.org/hascode/circuit-breaker-comparison.git

Resources

Article Updates

  • 2017-02-22: Wrong typed dependency for Javaslang-Circuitbreaker added, thanks @Robert Winkler for pointing out.

Tags: , , , , , , , , , ,

Search
Categories