Bash on Micha Kops' Tech Notes

Shell Script Testing with Bats

Tue, 13 Feb 2024 00:00:00 +0100

We’ve all been there: a small shell script that starts as a quick helper suddenly becomes a critical part of your workflow.

Then one day, it breaks - maybe because of a missing argument, a typo, or an unexpected input.

Debugging at 2 AM isn’t fun.

That’s why adding tests to your shell scripts is a lifesaver. With Bats (Bash Automated Testing System), you can catch these issues early and make your scripts as reliable as any other piece of software.

Single Class Java HTTP Client with Proxy and SSL/TLS Keystore Settings

Tue, 02 Aug 2022 00:00:00 +0200

Sometimes this is useful for the diagnosis of configuration and network problems of ones Java application.

This is our single-class HTTP client example without the need for external dependencies:

HttpTest.java

import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManagerFactory;
import java.io.FileInputStream;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.ProxySelector;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpRequest.BodyPublishers;
import java.net.http.HttpResponse;
import java.net.http.HttpResponse.BodyHandlers;
import java.security.KeyManagementException;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.UnrecoverableKeyException;
import java.security.cert.CertificateException;
import java.time.Duration;
import java.time.temporal.ChronoUnit;

public class HttpTest {
    public static void main(String[] args)
        throws URISyntaxException, NoSuchAlgorithmException, KeyManagementException, KeyStoreException, UnrecoverableKeyException, IOException, CertificateException {
        String bodyPayload = """
            ourpayload, json, xml, ...
            """;

        String keyStorePath = "/opt/keystore.jks";
        String keyStorePassword = "ABCDEFG";

        String proxyHost = "ourproxy.proxy";
        String uriString = "https://some-service/api";
        int proxyPort = 8080;
        int timeoutInSeconds = 60;

        KeyStore keyStore = KeyStore.getInstance("JKS");
        keyStore.load(new FileInputStream(keyStorePath), keyStorePassword.toCharArray());

        KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance("PKIX");
        keyManagerFactory.init(keyStore, null);

        TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance("PKIX"); // using same keystore for both
        trustManagerFactory.init(keyStore);

        SSLContext sslContext = SSLContext.getInstance("TLS");
        sslContext.init(keyManagerFactory.getKeyManagers(), trustManagerFactory.getTrustManagers(), null);

        HttpClient client = HttpClient.newBuilder()
                                      .proxy(
                                          ProxySelector.of(new InetSocketAddress(proxyHost, proxyPort)))
                                      .sslContext(sslContext)
                                      .build();
        URI uri = new URI(uriString);
        HttpRequest request = HttpRequest.newBuilder(uri)
                                         .POST(BodyPublishers.ofString(bodyPayload))
                                         .timeout(Duration.of(timeoutInSeconds, ChronoUnit.SECONDS))
                                         .build();
        System.out.printf("Sending POST request to %s, timeout: %ds%n", uri, timeoutInSeconds);
        try {
            HttpResponse response = client.send(request, BodyHandlers.ofString());
            System.out.println("Response received...");
            System.out.printf("\tResponse-Status: %d%n", response.statusCode());
            System.out.printf("\tResponse-Body: %s%n", response.body());
            System.out.println("-------------------------------------%n");

        } catch (IOException e) {
            System.err.printf("IOException caught: %s%n", e.getMessage());
            e.printStackTrace(System.err);
        } catch (InterruptedException e) {
            System.err.printf("IOException caught: %s%n", e.getMessage());
            e.printStackTrace(System.err);
        }
    }
}

Creating Pre-Commit-Hooks in Git and Mercurial: Prefix Commit Messages for Feature/Story Branches

Sun, 16 Dec 2012 00:00:00 +0100

Managing my projects’ source code I am using Git also as Mercurial. Therefore I often encounter the situation where I am creating a special branch to implement a specific user story or feature request.

Now when working on such a story branch I often enter the issue-key or a short title as a prefix for each commit message. Doing this by manually is a waste of time and error-prone and luckily for us, each of both DVCS offers us an easy API to add custom hooks to the different life-cycle events.

Docker Snippets

Mon, 01 Mar 2010 00:00:00 +0100

Restrict Network

Can be useful when using a third-party image that we do not trust

Run with no network

docker run --network none <image>

Run with private isolated network

At least containers attached to this network can talk with another

docker network create --internal my_isolated_network
docker run --network my_isolated_network <image>

Block using firewall

e.g. using iptables or ipfw

# Get container's IP
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' <container_name>

# Block all outbound connections from that IP
sudo iptables -I DOCKER-USER -s <container_ip> -j DROP

Git Snippets

Mon, 01 Mar 2010 00:00:00 +0100

Show commits from another branch not contained in current branch

git cherry -v otherbranch
+ f7d6a569bb6912aac97fce9ac92c4302863fb0d9 thecommit

Create empty commit

sometimes necessary for build/deploy pipelines …

git commit --allow-empty -m "Empty-Commit"

Cherry pick without commit

git cherry-pick -n HASH

Using vimdiff for diff

set it via git config

git config --global diff.tool vimdiff
git config --global merge.tool vimdiff

set it via ~/.gitconfig

[diff]
    tool = vimdiff
[merge]
    tool = vimdiff

Using vscode as diff and mergetool

You need to have the shell integration installed (code binary in PATH)

Linux Snippets

Mon, 01 Mar 2010 00:00:00 +0100

These are not only linux snippets but also bash snippets and snippets using tools that run under Linux, *nix or sometimes even MacOSX, I should reorder this article someday ;)

Settings for more reliable bash scripts

set -euo pipefail

this gives us …

-e: exit script if a single command fails
-u: exit script if an unset variable is used
-o pipefail: return value of a pipeline is the status of the last command to exit with a non-zero status, or zero if no command exited with a non-zero status

XML Snippets

Mon, 01 Mar 2010 00:00:00 +0100

Ignore Namespaces in XPath Query

e.g. Query for all xxx nodes ignoring their namespace:

xmllint --xpath '//*[local-name()="xxx"]' input.xml

An example parsing URLs from a sitemap XML.

The URLs are located in //url/loc where all nodes are bound to the namespace http://www.sitemaps.org/schemas/sitemap/0.9.

The following query ignores the namespace

xmllint --xpath '//*[local-name()="url"]/*[local-name()="loc"]' sitemap.xml

Pretty Print XML in the Console using xmllint

echo '<blogs><blog url="https://www.hascode.com/">hasCode.com</blog></blogs>' | xmllint --format -
<?xml version="1.0"?>
<blogs>
    <blog url="https://www.hascode.com/">hasCode.com</blog>
</blogs>