Stratio’s Lucene-based index for Cassandra is now a plugin

Thanks to the changes proposed at CASSANDRA-8717, CASSANDRA-7575 and CASSANDRA-6480, Stratio is glad to present its Lucene-based implementation of Cassandra secondary indexes as a plugin that can be attached to the Apache distribution. Before the above changes, Lucene index was distributed inside a fork of Apache Cassandra, with all the difficulties it implied, i.e. maintaining a fork. As of now, the fork is discontinued and new users should use the recently created plugin, which maintains all the features of Stratio Cassandra.

Stratio’s Lucene index extends Cassandra’s functionality to provide near real-time distributed search engine capabilities such as with ElasticSearch or Solr, including full text search capabilities, free multivariable search,relevance queries and field-based sorting. Each node indexes its own data, so high availability and scalability is guaranteed.

With the new distribution as a plugin you can easily patch an existing installation of Cassandra with the following command:

 mvn clean package -Ppatch -Dcassandra_home=<CASSANDRA_INSTALL_DIR>

Now, given an existing table with the following schema

CREATE TABLE tweets (
   id        bigint,
   createdAt timestamp,
   message   text,
   userid    bigint,
   username  text,
   PRIMARY KEY (userid, createdAt, id) );

you can create a Lucene index with this query:

ALTER TABLE tweets ADD lucene TEXT;
CREATE CUSTOM INDEX tweets_idx ON  twitter.tweets (lucene) 
USING 'com.stratio.cassandra.lucene.Index' 
WITH OPTIONS = {
   'refresh_seconds'    : '60',
   'schema' : '{ default_analyzer : "English",
     fields : {
           createdat : {type : "date", pattern : "yyyy-MM-dd"}, 
           message   : {type : "text", analyzer : "English"}, 
           userid    : {type : "string"}, 
           username  : {type : "string"}
       }}     
'};

Once the index has been created, we can start issuing queries. For instance, retrieve 10 tweets containing the word Cassandra in the message body:

SELECT * FROM tweets WHERE lucene = ‘{
   filter : {type  : "match", 
             field : "message",
             value : "cassandra"}
}’ LIMIT 10;

We can also find the 5 best matching tweets containing the word Cassandra in the message body for the previous query, involving Lucene relevance features:

SELECT * FROM tweets WHERE lucene = ‘{
   query: {type: "match", 
           field : "message",
           value : "cassandra"}
}’ LIMIT 5;

Queries, filters and boolean predicates can be combined to do queries as complex as requesting the tweets inside a certain time range starting with an ‘A’ or containing a ‘B’, or with the word ‘FAST’, sorting them by ascending time, and then by descending alphabetic user name:

SELECT * FROM tweets WHERE lucene = '{
   filter : 
   {
    type : "boolean", must : 
    [
       {type : "range", field : "createdat", lower : "2014-04-25"},
       {type : "boolean", should : 
        [
           {type : "prefix", field : "userid", value : "a"} ,
           {type : "wildcard", field : "userid", value : "*b*"} ,
           {type : "match", field : "userid", value : "fast"} 
        ]
       }
    ]
   },
   sort : 
   {
     fields: [ {field :"createdat", reverse : true},
               {field : "userid", reverse : false} ] 
   }
}' LIMIT 10000;

You can learn more about Stratio’s Lucene index for Cassandra reading the comprehensive documentation or viewing the video of its presentation at Cassandra Summit Europe 2014.

We will be back very soon with new exciting features such as geospatial search and bitemporal indexing.