Messaging with NATS and Java

Figure 1. NATS Architecture Component Diagram NATS is a high-performance messaging system that offers simplicity, speed, and scalability. It is particularly suited for building distributed systems and microservices. This article demonstrates how to integrate NATS with Java, showcasing the essential steps to set up, connect, and publish/subscribe to messages. Prerequisites Before diving in, we should ensure to have the following installed: Java Development Kit (JDK) 11 or later. Maven or Gradle for dependency management. Docker (optional). ...

November 26, 2024 · 13 min · 2646 words · Micha Kops

Object Audit with Java and Javers

Just a quick snippet Maven Integration pom.xml <dependency> <groupId>org.javers</groupId> <artifactId>javers-core</artifactId> <version>${javers.version}</version> </dependency> Calculate Changes in Object Graph package io.hascode; import org.javers.core.Changes; import org.javers.core.Javers; import org.javers.core.JaversBuilder; import org.javers.core.diff.Diff; public <T> Changes diff(T snapshot, T latest) { Javers javers = JaversBuilder.javers().build(); Diff diff = javers.compare(snapshot, latest); return diff.getChanges(); } Resources Javers Website

September 18, 2023 · 1 min · 50 words · Micha Kops

Forwardings Requests to static content in Spring Boot Webflux

The following WebFilter redirects incoming requests for / to a static HTML file, index.html. ToIndexPageRedirection.java package com.hascode.tutorial; import org.springframework.stereotype.Component; import org.springframework.web.server.ServerWebExchange; import org.springframework.web.server.WebFilter; import org.springframework.web.server.WebFilterChain; import reactor.core.publisher.Mono; @Component public class ToIndexPageRedirection implements WebFilter { @Override public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) { if (exchange.getRequest().getURI().getPath().equals("/")) { return chain.filter( exchange.mutate().request( exchange.getRequest().mutate().path("/index.html").build() ).build() ); } return chain.filter(exchange); } } Ressources: JavaDocs WebFilter

August 16, 2022 · 1 min · 59 words · Micha Kops

Single Class Java HTTP Client with Proxy and SSL/TLS Keystore Settings

Sometimes this is useful for the diagnosis of configuration and network problems of ones Java application. This is our single-class HTTP client example without the need for external dependencies: HttpTest.java import javax.net.ssl.KeyManagerFactory; import javax.net.ssl.SSLContext; import javax.net.ssl.TrustManagerFactory; import java.io.FileInputStream; import java.io.IOException; import java.net.InetSocketAddress; import java.net.ProxySelector; import java.net.URI; import java.net.URISyntaxException; import java.net.http.HttpClient; import java.net.http.HttpRequest; import java.net.http.HttpRequest.BodyPublishers; import java.net.http.HttpResponse; import java.net.http.HttpResponse.BodyHandlers; import java.security.KeyManagementException; import java.security.KeyStore; import java.security.KeyStoreException; import java.security.NoSuchAlgorithmException; import java.security.UnrecoverableKeyException; import java.security.cert.CertificateException; import java.time.Duration; import java.time.temporal.ChronoUnit; public class HttpTest { public static void main(String[] args) throws URISyntaxException, NoSuchAlgorithmException, KeyManagementException, KeyStoreException, UnrecoverableKeyException, IOException, CertificateException { String bodyPayload = """ ourpayload, json, xml, ... """; String keyStorePath = "/opt/keystore.jks"; String keyStorePassword = "ABCDEFG"; String proxyHost = "ourproxy.proxy"; String uriString = "https://some-service/api"; int proxyPort = 8080; int timeoutInSeconds = 60; KeyStore keyStore = KeyStore.getInstance("JKS"); keyStore.load(new FileInputStream(keyStorePath), keyStorePassword.toCharArray()); KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance("PKIX"); keyManagerFactory.init(keyStore, null); TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance("PKIX"); // using same keystore for both trustManagerFactory.init(keyStore); SSLContext sslContext = SSLContext.getInstance("TLS"); sslContext.init(keyManagerFactory.getKeyManagers(), trustManagerFactory.getTrustManagers(), null); HttpClient client = HttpClient.newBuilder() .proxy( ProxySelector.of(new InetSocketAddress(proxyHost, proxyPort))) .sslContext(sslContext) .build(); URI uri = new URI(uriString); HttpRequest request = HttpRequest.newBuilder(uri) .POST(BodyPublishers.ofString(bodyPayload)) .timeout(Duration.of(timeoutInSeconds, ChronoUnit.SECONDS)) .build(); System.out.printf("Sending POST request to %s, timeout: %ds%n", uri, timeoutInSeconds); try { HttpResponse response = client.send(request, BodyHandlers.ofString()); System.out.println("Response received..."); System.out.printf("\tResponse-Status: %d%n", response.statusCode()); System.out.printf("\tResponse-Body: %s%n", response.body()); System.out.println("-------------------------------------%n"); } catch (IOException e) { System.err.printf("IOException caught: %s%n", e.getMessage()); e.printStackTrace(System.err); } catch (InterruptedException e) { System.err.printf("IOException caught: %s%n", e.getMessage()); e.printStackTrace(System.err); } } } ...

August 2, 2022 · 2 min · 253 words · Micha Kops

Spring Boot Kafka Increase Message Size Limit

Let’s say we would like to increase the limit t 10MB …​ Broker Configuration Apply the new limit either by modifying the server.properties like this…​ max.message.bytes=10485760 or apply it to a specific topic using kafka-configs.sh --bootstrap-server localhost:9092 \ --entity-type topics \ --entity-name thetopic \ --alter \ --add-config max.message.bytes=10485760 Producer Configuration for Spring Boot We simply need to add the following line to our application.properties: spring.kafka.producer.properties.max.request.size=spring.kafka.producer.properties.max.request.size The following proof-of-concept demonstrates that without the property, sending a large message fails, with the property it succeeds: ...

June 9, 2022 · 2 min · 270 words · Micha Kops

Embedded Kafka for Spring Boot Testing without using Docker

Sometimes it is nice to set up an embedded Kafka broker for testing without the need to have Docker installed (e.g. for using testcontainers-lib). The following snippet shows, how to set up an embedded Kafka instance for testing for a Spring Boot project. Setup Using Maven, this is our Spring Boot project with dependencies needed: pom.xml <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId>(1) <version>2.6.4</version> <relativePath/> <!-- lookup parent from repository --> </parent> <groupId>com.hascode.tutorial</groupId> <artifactId>kafka-testing</artifactId> <version>1.0.0-SNAPSHOT</version> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <java.version>17</java.version> <maven.compiler.source>${java.version}</maven.compiler.source> <maven.compiler.target>${java.version}</maven.compiler.target> <kafka.version>3.1.0</kafka.version>(2) </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter</artifactId> </dependency> <dependency> <groupId>org.springframework.kafka</groupId> <artifactId>spring-kafka</artifactId>(3) </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.springframework.kafka</groupId> <artifactId>spring-kafka-test</artifactId>(4) <scope>test</scope> </dependency> <dependency>(5) <groupId>org.junit.jupiter</groupId> <artifactId>junit-jupiter-api</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.junit.jupiter</groupId> <artifactId>junit-jupiter-engine</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.junit.platform</groupId> <artifactId>junit-platform-launcher</artifactId> <scope>test</scope> </dependency> [..] </project> ...

May 25, 2022 · 4 min · 839 words · Micha Kops

Java Bean Mapping with MapStruct

MapStruct is a nice tool to generate mappers for converting one Java bean into another e.g. for projections, data-transfer-objects and so on …​ As long as fields in source and target beans do match, the mapper is able to generate the data setting automatically .. else we may specify which source fields to map into which target fields or to register custom converters with ease. Using Maven, we need to add dependencies and plugin integration to our pom.xml: ...

March 31, 2022 · 3 min · 459 words · Micha Kops

Configuring Spring Boot WebserviceTemplate to sign WSS SOAP Requests

Sometimes when accessing SOAP APIs, our SOAP client needs to sign the request. How this can be achieved using Spring Boot’s WebserviceTemplate within a few steps is the scope of this short article. This snippet only deals with the client side not with the security configuration on the server side. Also it assumes, that you have already set up your keystore/truststore and that you’re loading these with your Spring Boot application’s startup without errors. ...

March 30, 2022 · 2 min · 329 words · Micha Kops

Converting XML Schema (XSD) to Protocol Buffers (Protobuf)

Sometimes one needs to derive a Google Protocol Buffers schema from an XML schema .. e.g. from an Enterprise Architect Export. Tool used here: schema2proto (GitLab project) Steps Download schema2proto-lib from the global Maven repository: https://search.maven.org/search?q=schema2proto Run Schema2Proto against a give XSD schema file and with a given output directory: java -jar schema2proto-lib-1.53.jar Schema2Proto --outputDirectory=src/main/protobuf input.xsd A yaml config file may be given instead of cli parameters: java -jar schema2proto-lib-1.53.jar Schema2Proto ----configFile=config.yaml input.xsd ...

February 2, 2022 · 1 min · 184 words · Micha Kops

Kafka Java Quickstart with Docker

Goals Setup Kafka and Zookeeper with Docker and docker-compose Create a message consumer and producer in Java Kafka Setup We’re using docker-compose to set up our message broker, zookeper and other stuff using confluent-platform. This is our docker-compose.yaml config file from Confluent’s following GitHub repository. docker-compose.yaml --- version: '2' services: zookeeper: image: confluentinc/cp-zookeeper:7.0.1 hostname: zookeeper container_name: zookeeper ports: - "2181:2181" environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 broker: image: confluentinc/cp-kafka:7.0.1 hostname: broker container_name: broker depends_on: - zookeeper ports: - "29092:29092" - "9092:9092" - "9101:9101" environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181' KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092 KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0 KAFKA_JMX_PORT: 9101 KAFKA_JMX_HOSTNAME: localhost schema-registry: image: confluentinc/cp-schema-registry:7.0.1 hostname: schema-registry container_name: schema-registry depends_on: - broker ports: - "8081:8081" environment: SCHEMA_REGISTRY_HOST_NAME: schema-registry SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: 'broker:29092' SCHEMA_REGISTRY_LISTENERS: http://0.0.0.0:8081 connect: image: cnfldemos/kafka-connect-datagen:0.5.0-6.2.0 hostname: connect container_name: connect depends_on: - broker - schema-registry ports: - "8083:8083" environment: CONNECT_BOOTSTRAP_SERVERS: 'broker:29092' CONNECT_REST_ADVERTISED_HOST_NAME: connect CONNECT_GROUP_ID: compose-connect-group CONNECT_CONFIG_STORAGE_TOPIC: docker-connect-configs CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 1 CONNECT_OFFSET_FLUSH_INTERVAL_MS: 10000 CONNECT_OFFSET_STORAGE_TOPIC: docker-connect-offsets CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 1 CONNECT_STATUS_STORAGE_TOPIC: docker-connect-status CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 1 CONNECT_KEY_CONVERTER: org.apache.kafka.connect.storage.StringConverter CONNECT_VALUE_CONVERTER: io.confluent.connect.avro.AvroConverter CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: http://schema-registry:8081 CONNECT_PLUGIN_PATH: "/usr/share/java,/usr/share/confluent-hub-components" CONNECT_LOG4J_LOGGERS: org.apache.zookeeper=ERROR,org.I0Itec.zkclient=ERROR,org.reflections=ERROR ksqldb-server: image: confluentinc/cp-ksqldb-server:7.0.1 hostname: ksqldb-server container_name: ksqldb-server depends_on: - broker - connect ports: - "8088:8088" environment: KSQL_CONFIG_DIR: "/etc/ksql" KSQL_BOOTSTRAP_SERVERS: "broker:29092" KSQL_HOST_NAME: ksqldb-server KSQL_LISTENERS: "http://0.0.0.0:8088" KSQL_CACHE_MAX_BYTES_BUFFERING: 0 KSQL_KSQL_SCHEMA_REGISTRY_URL: "http://schema-registry:8081" KSQL_PRODUCER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor" KSQL_CONSUMER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringConsumerInterceptor" KSQL_KSQL_CONNECT_URL: "http://connect:8083" KSQL_KSQL_LOGGING_PROCESSING_TOPIC_REPLICATION_FACTOR: 1 KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: 'true' KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: 'true' ksqldb-cli: image: confluentinc/cp-ksqldb-cli:7.0.1 container_name: ksqldb-cli depends_on: - broker - connect - ksqldb-server entrypoint: /bin/sh tty: true ksql-datagen: image: confluentinc/ksqldb-examples:7.0.1 hostname: ksql-datagen container_name: ksql-datagen depends_on: - ksqldb-server - broker - schema-registry - connect command: "bash -c 'echo Waiting for Kafka to be ready... && \ cub kafka-ready -b broker:29092 1 40 && \ echo Waiting for Confluent Schema Registry to be ready... && \ cub sr-ready schema-registry 8081 40 && \ echo Waiting a few seconds for topic creation to finish... && \ sleep 11 && \ tail -f /dev/null'" environment: KSQL_CONFIG_DIR: "/etc/ksql" STREAMS_BOOTSTRAP_SERVERS: broker:29092 STREAMS_SCHEMA_REGISTRY_HOST: schema-registry STREAMS_SCHEMA_REGISTRY_PORT: 8081 rest-proxy: image: confluentinc/cp-kafka-rest:7.0.1 depends_on: - broker - schema-registry ports: - 8082:8082 hostname: rest-proxy container_name: rest-proxy environment: KAFKA_REST_HOST_NAME: rest-proxy KAFKA_REST_BOOTSTRAP_SERVERS: 'broker:29092' KAFKA_REST_LISTENERS: "http://0.0.0.0:8082" KAFKA_REST_SCHEMA_REGISTRY_URL: 'http://schema-registry:8081' ...

January 29, 2022 · 8 min · 1500 words · Micha Kops