Lucene on Micha Kops' Tech Notes

Elasticsearch Integration Testing with Java

Tue, 23 Aug 2016 00:00:00 +0200

When building up search engines, indexing tons of data into a schema-less, distributed data store, Elasticsearch has always been a favourite tool of mine.

In addition to its core features, it also offers tools and documentation for us developers when we need to write integration tests for our Elasticsearch powered Java applications.

In the following tutorial I’d like to demonstrate how to implement a small sample application using Elasticsearch under the hood and how to write integration-tests with these tools for this application afterwards.

Lucene by Example: Specifying Analyzers on a per-field-basis and writing a custom Analyzer/Tokenizer

Sun, 06 Jul 2014 00:00:00 +0200

Lucene is my favourite search engine library and the more often I use it in my projects the more features or functionality I find that were unknown to me.

Two of those features I’d like to share in the following tutorial is one the one hand the possibility to specify different analyzers on a per-field basis and on the other hand the API to create a simple character based tokenizer and analyzer within a few steps.

Creating elegant, typesafe Queries for JPA, mongoDB Morphia and Lucene using Querydsl

Thu, 13 Feb 2014 00:00:00 +0100

Querydsl is a framework that allows us to create elegant, type-safe queries for a variety of different data-sources like Java Persistence API (JPA) entities, Java Data Objects (JDO), mongoDB with Morphia, SQL, Hibernate Search up to Lucene.

In the following tutorial we’re implementing example queries for different environments – Java Persistence API compared with a JPQL and a criteria API query, mongoDB with Morphia and last but not least for Lucene.

Content Detection, Metadata and Content Extraction with Apache Tika

Sun, 02 Dec 2012 00:00:00 +0100

Encountering the situation that you want to extract meta-data or content from a file – might it be an office document, a spreadsheet or even a mp3 or an image – or you’d like to detect the content type for a given file then Apache Tika might be a helpful tool for you.

Apache Tika supports a variety of document formats and has a nice, extendable parser and detection API with a lot of built-in parsers available.

Lucene Snippets: Index Stats

Sat, 08 Sep 2012 00:00:00 +0200

In Lucene 4.x there is an API to fetch index statistics for specific document’s fields.

The following examples shows how to create an index with some random documents and fetch some statistics for a field afterwards ..

Lucene Dependencies

Just one dependency needed here .. lucene-core. I’ve added the declarations needed for Maven and SBT here .. if you’re using Gradle or Buildr you should’t have a problem to create your build file either..

Lucene Snippets: Faceting Search

Tue, 28 Aug 2012 00:00:00 +0200

The latest snippet from my Lucene examples demonstrates how to achieve a facet search using the Lucene 4.0 API and how easy it is to define multiple category paths to aggregate search results for different possible facets.

In the following example we’re indexing some books as a classical example and create multiple category paths for author, publication date and category afterwards ..

Lucene Dependencies

We simply need two dependencies here .. lucene-core of course and in addition the lucene-facet library .. I’ve added the declarations needed for Maven and SBT here .. if you’re using Gradle or Buildr you should’t have a problem to transfer the information needed ;)

Hibernate Search Faceting: Discrete and Range Faceting by Example

Mon, 26 Mar 2012 00:00:00 +0200

In today’s tutorial we’re exploring the world of faceted searches like the one we’re used to see when we’re searching for an item on Amazon.com or other websites. We’re using Hibernate Search here that offers an API to perform discrete as well as range faceted searches on our persisted data.

Maven Dependencies Needed

For simplicity’s sake am I going to use an HSQL database for persistence, in addition the dependencies for hibernate-entitymanager and hibernate-search (of course) should be added to your pom.xml

JPA Persistence and Lucene Indexing combined in Hibernate Search

Sun, 05 Feb 2012 00:00:00 +0100

Often we’re writing an application that has to handle entities that – on the one side need to be persisted in a relational database using standards like the Java Persistence API (JPA) and using frameworks like Hibernate ORM or EclipseLink.

On the other side those entities and their fields are often stored in a highspeed indexer like Lucene. From this situation arises a bunch of common problems .. to synchronize both data sources, to handle special data mapped in an entity like an office document and so on..

Neo4j Graph Database Tutorial: How to build a Route Planner and other Examples

Fri, 20 Jan 2012 00:00:00 +0100

Often in the life of developer’s life there is a scenario where using a relational database tends to get complicated or sometimes even slow – especially when there are fragments with multiple relationships or multiple connections present. This often leads to complex database queries or desperate software engineers trying to handle those problems with their ORM framework.

A possible solution might be to switch from a relational database to a graph database – and – neo4j is our tool of choice here. In the following tutorial we’re going to implement several examples to demonstrate the strengths of a graph database .. from a route planner to a social graph.

Extending the Confluence Search Index

Sun, 23 May 2010 00:00:00 +0200

Developing plugins for the Confluence Wiki a developer sometimes needs to save additional metadata to a page object using Bandana or the ContentPropertyManager. Wouldn’t it be nice if this metadata was available in the built-in Lucene index?

That is were the Confluence Extractor Module comes into play..

Overview

An extractor allows the developer to add new fields to the lucene search index. Creating a new extractor is quite simple – just implement the interface com.atlassian.bonnie.search.Extractor or bucket.search.lucene.extractor.BaseAttachmentContentExtractor if you want to build a new file extractor.

How to build a quick Lucene Search

Thu, 25 Mar 2010 00:00:00 +0100

Helo – today I wanted to post a small tutorial for a small index and search operation using the Lucene indexer and Maven for the project setup.

Setup

Create an empty Maven sample project using the Eclipse Maven Plugin or use the following console command:
```
mvn archetype:create -DgroupId=com.hascode.demo.search -DartifactId=lucene-sample
```

Here is my pom.xml there are some dependencies for Lucene defined:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.hascode.demo.search</groupId>
  <artifactId>lucene-sample</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <name>My Lucene Search Sample</name>
  <description>Lucene Search Sample</description>
  <dependencies>
    <dependency>
      <groupId>org.apache.lucene</groupId>
      <artifactId>lucene-core</artifactId>
      <version>2.4.1</version>
    </dependency>
    <dependency>
      <groupId>lucene</groupId>
      <artifactId>lucene</artifactId>
      <version>1.4.3</version>
    </dependency>
  </dependencies>
</project>