Json on Micha Kops' Tech Notes

Avro Schema Compatibility and the Confluent Schema Registry

Sun, 08 Jun 2025 00:00:00 +0200

What is Schema Compatibility?

Schema compatibility ensures that data serialized by a producer using one schema can be deserialized by a consumer using another. The Confluent Schema Registry provides several compatibility modes to enforce these rules when updating schemas.

There are two key directions of compatibility:

Backward Compatibility (Upward Compatibility): Old consumers can read data produced with a newer schema.
Forward Compatibility (Downward Compatibility): New consumers can read data produced with an older schema.

Prettier Code Formatter Configuration

Fri, 14 May 2021 00:00:00 +0200

Goals

Setup Prettier for different environments / IDEs
Run Prettier via git hooks

Install Prettier

Install via npm

npm install --save-dev --save-exact prettier

Add empty configuration file

echo {}> .prettierrc.json

Create Prettier ignore file .prettierignore:

# Ignore artifacts:
build
coverage

Format all project files with Prettier:

npx prettier --write .

Git Hooks

Adding the following lines to the project’s package-json makes ESLint and Prettier run before each commit:

{
  "husky": {
    "hooks": {
      "pre-commit": "lint-staged"
    }
  },
  "lint-staged": {
    "**/*": "prettier --write --ignore-unknown"
  }
}

Testing OpenAPI Swagger Schema Compliance with Java, JUnit and assertj-swagger

Fri, 31 Aug 2018 00:00:00 +0200

The OpenAPI and Swagger API description format are becoming important standards to specify API contracts for RESTful web services and the Microservices trend pushes the need for such contracts even further.

Therefore arises the need for software architects, testers and developers to write tests to verify if an exposed API follows such a specified contract.

In the following tutorial I will demonstrate a setup with Java, Maven, JUnit and the designated contract-testing-library, assertj-swagger that verifies the validity of such a contract exposed by a Spring Boot application against a local stored definition.

Writing BDD-Style Webservice Tests with Karate and Java

Thu, 06 Apr 2017 00:00:00 +0200

There is a new testing framework out there called Karate that is build on top of the popular Cucumber framework.

Karate makes it easy to script interactions with out web-services under test and verify the results. In addition it offers us a lot of useful features like parallelization, script/feature re-use, data-tables, JsonPath and XPath support, gherkin syntax, switchable staging-configurations and many others.

In the following tutorial we’ll be writing different scenarios and features for a real-world RESTful web-service to demonstrate some of its features.

Transforming JSON Structures with Java and JOLT

Sun, 29 Jan 2017 00:00:00 +0100

When it comes to web-services (especially RESTful web-services), configuration files or other data-descriptors, the JavaScript Object Notation, short: JSON is often used as format of choice.

As the task arises to transform these JSON structures, I have seen a variety of different approaches from converting JSON into XML using JAX-B, applying XSLT, transforming into JSON again on the one hand to loading JSON structures into Hash-maps of Hash-maps (..) and manually removing single elements from these collections on the other hand.

Creating REST Clients for JAX-RS based Webservices with Netflix Feign

Thu, 22 Oct 2015 00:00:00 +0200

For us developers there plenty of libraries exist helping us in deriving and generating clients for existing RESTful web-services and I have already covered some of the in this blog (e.g. the JAX-RS Client API). Nevertheless, I’d like to share my experience with another interesting lightweight library here: Netflix Feign.

Feign offers a nice fluent-builder API, a rich integration for common libraries and APIs like JAX-RS, Jackson, GSON, SAX, JAX-B, OkHttp, Ribbon, Hystrix, SLF4J and more and last bot not least, it is setup easy and the service contracts are specified using interfaces and annotations.

Integrating Swagger into a Spring Boot RESTful Webservice with Springfox

Wed, 01 Jul 2015 00:00:00 +0200

Spring Boot allows us to create RESTful web-services with ease, Swagger specifies a format to describe the capabilities and operations of these services and with Swagger UI it is possible to explore our REST API with a nice graphical user interface in our browser.

Springfox is a project that aims at creating automated JSON API documentation for API’s built with Spring and is used in the following tutorial to integrate Swagger into a sample application.

Documenting RESTful Webservices in Swagger, AsciiDoc and Plain Text with Maven and the JAX-RS Analyzer

Tue, 16 Jun 2015 00:00:00 +0200

A variety of different tools exists to help us analyze RESTful web-services and create documentations for their APIs in different formats.

In the following tutorial I’d like to demonstrate how to document an existing JAX-RS web-service in multiple formats like Swagger, AsciiDoc or Plain Text using Maven, the JAX-RS Analyzer and the JAX-RS Analyzer Maven Plugin.

The JAX-RS Analyzer gathers its information not only by reflection like most other tools but also by bytecode analysis and therefore does not require us to add special annotations for documentation to our code.

Creating a hybrid mobile Application with Ionic, Cordova and AngularJS

Sun, 03 May 2015 00:00:00 +0200

Nowadays in the realm of hybrid mobile application development there is a variety of available frameworks worth having a look at.

In the following tutorial I’d like to demonstrate the development life-cycle for a complete mobile application using Ionic, Cordova and AngularJS (and others) covering every step from the initial project setup, creating Angular Controllers, Directives, adding Cordova Plug-Ins, running and testing the application in the browser and emulator up to finally running the application on an Android device and assembling files for a release.

Immutables 2.0 for sexy Immutable Object Creation and more

Sun, 26 Apr 2015 00:00:00 +0200

Using immutable objects in Java (and other programming languages as well) is a good thing because immutable objects may be shared safely, are thread-safe and reduce the risk of side effects in your applications.

Nowadays multiple frameworks exist to reduce the need of writing boilerplate code here but there is one special framework whose features I’d like to demonstrate in the following short tutorial.

It hooks into your application using annotation processing and generates type-safe builders, toString, hashCode, equals methods for you, supports lazy attributes, singleton instances, serialization into data-formats like JSON and a lot of other features,too.

Creating different Websocket Chat Clients in Java

Sun, 09 Nov 2014 00:00:00 +0100

Having written two articles about different websocket based chat server implementations in Java, I was recently asked how an implementation of the client side would look like in Java.

That’s why I added this article to demonstrate how to create a websocket chat client applications within a few steps with the Java API for Websocket.

In the following tutorial, we’re going to write a text-based chat client for the console first and afterwards we’re going to program a chat client with a graphical user interface, implemented in JavaFX.

JAX-RS Server API Snippets

Sun, 28 Sep 2014 00:00:00 +0200

Because a lot of my current projects are using JAX-RS in different versions I’d like to write down and share some frequently used snippets for implementing RESTful web-services with the JAX-RS specification here.

Using RegEx in Path Expressions

Sometimes we need to extract multiple parameters from a path expression e.g. in the following example where year, month and day are fragments if the path.

@GET
@Path("/orders/{year:\\d{4}}-{month:\\d{2}}-{day:\\d{2}}")
@Produces(MediaType.TEXT_PLAIN)
public Response getOrders(@PathParam("year") final int year, @PathParam("month") final int month, @PathParam("day") final int day) {
	return Response.ok("Year: " + year + ", month: " + month + ", day: " + day).build();
}

Java EE 7 JMX Reports with Yammer Metrics

Tue, 26 Aug 2014 00:00:00 +0200

There are several ways to aggregate and report application performance indicators in a Java application. One common way here is to use Java Management Extensions (JMX) and MBeans.

The Yammer Metrics Library eases this task for us and simplifies the aggregation of different reports.

In the following tutorial, we’re going to set up a full Java EE 7 web application by the help of Maven archetypes and we’re running the application on WildFly application server that is downloaded and configured completely by the WildFly Maven Plugin.

Java EE 7 Database Migrations with Liquibase and WildFly

Thu, 31 Jul 2014 00:00:00 +0200

I have written about other database migration frameworks before but in this article I’d like to cover the Liquibase framework in combination with WildFly as Java EE 7 compatible application server.

In the following tutorial, we’re going to write a full Java EE 7 book store application with a few steps and with Liquibase on board to create the database structure and insert example data into the database.

Thanks to the WildFly Maven Plug-in we even do not need to download and configure the application server but let Maven and the plug-in do the work for us.

Writing Java Integration Tests for MongoDB

Wed, 16 Oct 2013 00:00:00 +0200

MongoDB is matured, document-oriented, cross-platform NoSQL database system with drivers available for a bunch of different programming languages.

In the following short examples I’m going to write some integration tests for MongoDB using the MongoDB Java driver and the Flapdoodle library to create an embedded MongoDB instance for testing.

We’re going to write tests for a simple persist-and-query scenarion and for a map-reduce function and in addition I’m going to show how to bind the start and stop of a MongoDB instance to a Maven goal using the embedmongo-maven-plugin.

Creating a Chat Application using Java EE 7, Websockets and GlassFish 4

Tue, 13 Aug 2013 00:00:00 +0200

Java EE 7 is out now and so I was curious to play around with the new specifications and APIs from in this technology stack.

That’s why I didn’t hesitate to add yet another websocket-chat tutorial to the existing ones on the internet in favour of gathering some experience with this technology and a possible integration using a GlassFish 4 server, the new Java API for JSON Processing for data serialization combined with custom websocket encoders/decoders and finally adding some Bootstrap and jQuery on the client side.

Java Snippets

Mon, 01 Mar 2010 00:00:00 +0100

Remote Debug a Pod’s Java Process

Simple steps for remote debugging a Java process running on a k8 pod:

Edit deployment and add the following parameters to the Java start line: -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=127.0.0.1:5005

Also add the following port mapping at the section container → ports in the deployment:
```
- containerPort: 5005
  protocol: TCP
```
Safe, wait for the new pods and then add a port forward for port 5005 for this pod:
```
kubectl port-forward podname 5005
```

jq Snippets

Mon, 01 Mar 2010 00:00:00 +0100

Sample JSON File

Sample JSON File containing well known programmers that we use for the following examples

coders.json

[
  {
    "name": "Bjarne Stroustrup",
    "languages": [
      {
        "name": "C++",
        "year_created": 1983
      }
    ],
    "details": {
      "nationality": "Danish",
      "awards": ["IEEE Computer Society Computer Pioneer Award", "Charles Stark Draper Prize"]
    }
  },
  {
    "name": "Guido van Rossum",
    "languages": [
      {
        "name": "Python",
        "year_created": 1991
      }
    ],
    "details": {
      "nationality": "Dutch",
      "awards": ["Free Software Foundation Award for the Advancement of Free Software", "NLUUG Award"]
    }
  },
  {
    "name": "James Gosling",
    "languages": [
      {
        "name": "Java",
        "year_created": 1995
      }
    ],
    "details": {
      "nationality": "Canadian",
      "awards": ["Order of Canada", "The Economist Innovation Award"]
    }
  },
  {
    "name": "Dennis Ritchie",
    "languages": [
      {
        "name": "C",
        "year_created": 1972
      },
      {
        "name": "Unix",
        "year_created": 1969
      }
    ],
    "details": {
      "nationality": "American",
      "awards": ["Turing Award", "National Medal of Technology"]
    }
  },
  {
    "name": "Brendan Eich",
    "languages": [
      {
        "name": "JavaScript",
        "year_created": 1995
      }
    ],
    "details": {
      "nationality": "American",
      "awards": ["Webby Award"]
    }
  }
]

Linux Snippets

Mon, 01 Mar 2010 00:00:00 +0100

These are not only linux snippets but also bash snippets and snippets using tools that run under Linux, *nix or sometimes even MacOSX, I should reorder this article someday ;)

Bash Scripting

Settings for more reliable bash scripts

set -euo pipefail

this gives us …

-e: exit script if a single command fails
-u: exit script if an unset variable is used
-o pipefail: return value of a pipeline is the status of the last command to exit with a non-zero status, or zero if no command exited with a non-zero status

Postgres Snippets

Mon, 01 Mar 2010 00:00:00 +0100

Administration & Configuration

Connections

Show Max Connections (value and source)

SELECT setting, source, sourcefile, sourceline
FROM pg_settings
WHERE name = 'max_connections';

Set Max Connections

ALTER system SET max_connections = 250;

Kill Connections for a Database

SELECT pg_terminate_backend(pid)
FROM pg_catalog.pg_stat_activity
-- we don't want to kill our own connection
WHERE pid != pg_backend_pid()
-- we don't want to kill connections to other databases
AND datname = 'MYDATABASE'

Show Statement Timeout Settings for All Users

SELECT r.rolname,
       (SELECT unnest(s.setconfig) FROM pg_db_role_setting s WHERE s.setrole = r.oid AND s.setconfig::text LIKE '%statement_timeout%' LIMIT 1) AS statement_timeout
FROM pg_roles r
WHERE r.rolcanlogin
ORDER BY r.rolname;

Vim Snippets

Mon, 01 Mar 2010 00:00:00 +0100

Format JSON with jq

:%!jq .

Vim Plugin Manager

Installation

Download it from its GitHub repository or the risky but easy way using curl:

curl -fLo ~/.vim/autoload/plug.vim --create-dirs https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim

Adding Plugins

Add a plugin section to the .vimrc:

.vimrc

call plug#begin()
" The default plugin directory will be as follows:
"   - Vim (Linux/macOS): '~/.vim/plugged'
"   - Vim (Windows): '~/vimfiles/plugged'
"   - Neovim (Linux/macOS/Windows): stdpath('data') . '/plugged'
" You can specify a custom plugin directory by passing it as the argument
"   - e.g. `call plug#begin('~/.vim/plugged')`
"   - Avoid using standard Vim directory names like 'plugin'

" Make sure you use single quotes

" Gruvbox Theme
Plug 'sainnhe/gruvbox-material'

" Initialize plugin system
" - Automatically executes `filetype plugin indent on` and `syntax enable`.
call plug#end()
" You can revert the settings after the call like so:
"   filetype indent off   " Disable file-type-specific indentation
"   syntax off            " Disable syntax highlighting