Are you looking for an alternative to Elasticsearch? Manticore Search might be the solution you’ve been searching for. Manticore Search is an easy-to-use, open-source search engine that offers a fast and efficient database for full-text search capabilities. It serves as a powerful drop-in replacement for Elasticsearch, providing high-performance indexing and retrieval of data.

What makes Manticore Search different from other solutions is:

  • It is fast and efficient as compared to other alternatives in the sense that:
    • 182x faster than MySQL for small data (reproducible)
    • 15x faster than Elasticsearch for small datasets (reproducible)
    • 29x faster than Elasticsearch for log analytics (reproducible)
    • 5x faster than Elasticsearch for medium-size data (reproducible)
    • up to 2x faster max throughput than Elasticsearch for data ingestion on a single server (reproducible)
    • 4x faster than Elasticsearch for big data (reproducible)
  • It offers modern multithreading architecture with great query parallelization capabilities. It utilizes all the CPU to provide the quickest response times possible.
  • Performant secondary indexes are automatically created, saving you time and effort.
  • Manticore is SQL-first, this is to mean that it utilises SQL as its native syntax. It also offers compatibility with the MySQL protocol, allowing you to use your preferred MySQL client.
  • It is easy to use and integrate with other applications. It has clients available in PHP, Python, JavaScript, Java, Elixir, and Go
  • Newly added documents are immediately accessible using the real-time inserts
  • For added reliability, it offers built-in replication and load balancing
  • Data can also easily be synced with sources such as MySQL, PostgreSQL, ODBC, XML, and CSV.
  • It provides built-in tools and other SQL commands that make it easy to perform data backup and recovery.
  • It is not fully ACID-compliant but still supports transactions and binlog to ensure safe writes

This guide provides an end-to-end demonstration of how to install and use Manticore Search – Alternative to Elasticsearch in ELK.

System Requirements

To be able to install and run Manticore Search, you need a system that meets the below specifications:

  • Architecture: arm64 or x86_64
  • Operating System: Debian-based (e.g. Debian, Ubuntu, Mint), RHEL-based (e.g. RHEL, CentOS, Alma, Oracle Linux, Amazon Linux), Windows, or MacOS.
  • Manticore Columnar Library, which provides columnar storage and secondary indexes, requires a CPU with SSE >= 4.2.
  • No specific disk space or RAM requirements are needed. An empty Manticore Search instance only uses around 40MB of RSS RAM.

1. Install Manticore Search – Alternative to Elasticsearch in ELK

There are several ways to get Manticore Search installed on your system. In this guide, we will cover the two major ways. These are:

  • Using package managers
  • Using Docker

Follow any of the below options to install Manticore Search on your system.

Option 1: Install Manticore Search using Package Managers

Manticore Search can be installed on your Linux system using package managers. To achieve that, you need to add the Manticore Search repositories as shown:

  • On Debian/Ubuntu/Mint Linux
sudo apt update && sudo apt install gnupg2 wget -y
wget https://repo.manticoresearch.com/manticore-repo.noarch.deb
sudo dpkg -i manticore-repo.noarch.deb
sudo apt update
  • On RedHat/CentOS/Rocky Linux/Alma Linux/Amazon Linux 2/Oracle Linux
sudo yum install https://repo.manticoresearch.com/manticore-repo.noarch.rpm

Once the repositories have been added, proceed and install Manticore Search using the commands:

##On Debian/Ubuntu/Mint Linux
sudo apt install manticore manticore-extra

##On RedHat/CentOS/Rocky Linux/Alma Linux/Amazon Linux 2/Oracle Linux
sudo yum install manticore manticore-extra

##On MacOS
brew install manticoresoftware/tap/manticoresearch manticoresoftware/tap/manticore-extra

Now you are ready to use Manticore Search. Start the Manticore Search service:

##On Linux
sudo systemctl start manticore

##On MacOS
brew services start manticoresearch

Verify if all is okay:

$ curl -X GET localhost:9308
{"cluster_name":"docker-cluster","cluster_uuid":"Z7igA6xDRDKCVwnMuyXCOQ","name":"4e9d933ebde2","tagline":"You Know, for Search","version":{"build_date":"2019-10-22T17:16:35.176724Z","build_flavor":"default","build_hash":"fc0eeb6e2c25915d63d871d344e3d0b45ea0ea1e","build_snapshot":false,"build_type":"docker","lucene_version":"8.2.0","minimum_index_compatibility_version":"6.0.

There is a configuration file at /etc/manticoresearch/manticore.conf that can be modified as desired.

Option 2: Run Manticore Search in Docker Containers

This is another option for those who want to run Manticore Search in a container. This is an easy method as it eliminates the dependency issues required. Before that, you need to have Docker and Docker Compose installed:

Ensure that your system user is added to the Docker group:

sudo usermod -aG docker $USER
newgrp docker

Once Docker and Docker Compose have been installed, create a Docker compose file as shown:

vim docker-compose.yml

In the file, add these lines:

version: '2.2'

services:
  manticore:
    container_name: manticore
    image: manticoresearch/manticore
    environment:
      - EXTRA=1
    restart: always
    ports:
      - 127.0.0.1:9306:9306
      - 127.0.0.1:9308:9308
    ulimits:
      nproc: 65535
      nofile:
         soft: 65535
         hard: 65535
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./data:/var/lib/manticore
#      - ./manticore.conf:/etc/manticoresearch/manticore.conf # uncomment if you use a custom config

In the above file, we have exposed two ports:

  • 9306 for connections from a MySQL client
  • 9308 for connections via HTTP

Beside form using the MySQL port, you can log in to the shell using the command docker-compose exec manticore mysql

Now start the container:

docker-compose up -d

View if the container is running:

$ docker ps
CONTAINER ID   IMAGE                       COMMAND                  CREATED         STATUS         PORTS                                                          NAMES
ff5e44ac2380   manticoresearch/manticore   "docker-entrypoint.s…"   5 seconds ago   Up 4 seconds   127.0.0.1:9306->9306/tcp, 127.0.0.1:9308->9308/tcp, 9312/tcp   manticore

2. Getting Started with Manticore Search

There are several activities you can perform for Manticore Search. Below are some of the basic activities you can perform. First, connect the Manticore on the following ports:

  • 9306 for MySQL clients
  • 9308 for HTTP/HTTPS connections
  • 9312 for connections from other Manticore nodes and clients based on Manticore binary API

In this guide, we will use the MySQL clients:

$ mysql -h0 -P9306
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 20
Server version: 6.2.0 45680f95d@230804 (columnar 2.2.0 dc33868@230804) (secondary 2.2.0 dc33868@230804) git branch manticore-6.2.0...origin/manticore-6.2.0

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [(none)]>
  • Creating a table

Data types can be divided into two, full-text fields and attributes. Below is a simple example to create a table with attributes:

create table products(title text, price float) morphology='stem_en';

Add data into the table:

insert into products(title,price) values ('Crossbody Bag with Tassel', 19.85), ('microfiber sheet set', 19.99), ('Pet Hair Remover Glove', 7.99);

You can then run a full-text search query to filter by  remove hair

MySQL [(none)]> select id, highlight(), price from products where match('remove hair');
+---------------------+-------------------------------+----------+
| id                  | highlight()                   | price    |
+---------------------+-------------------------------+----------+
| 7928595846310920195 | Pet <b>Hair Remover</b> Glove | 7.990000 |
+---------------------+-------------------------------+----------+
1 row in set (0.001 sec)

MySQL [(none)]>
  • Swicthing between Row-wise and columnar storages

The supported storage types are:

  • row-wise – traditional storage available in Manticore Search out of the box
  • columnar – provided by Manticore Columnar Library

You can switch between the storages by specifying the storage. For example:

create table tbl(title text, type int, price float engine='rowwise') engine='columnar';
##OR
create table tbl(title text, type int, price float engine='columnar') engine='rowwise';
  • Listing tables:

Manticore makes it easy to view tables:

MySQL [(none)]> SHOW TABLES;
+----------+------+
| Index    | Type |
+----------+------+
| forum    | rt   |
| products | rt   |
| tbl      | rt   |
+----------+------+
3 rows in set (0.000 sec)

MySQL [(none)]>

You can also filter the tables:

MySQL [(none)]> SHOW TABLES LIKE 'pro%';
+----------+------+
| Index    | Type |
+----------+------+
| products | rt   |
+----------+------+
1 row in set (0.000 sec)

MySQL [(none)]>

You can also use select * from <table_name>.table

select * from tbl.table where type='text';
  • Deleting a table:

Deleting a table can be done using the below commands:

DROP TABLE products;
##OR
DROP TABLE IF EXISTS products;

3. Integrating Manticore Search with ELK Components

It is possible to integrate Manticore Search with the components of the ELK stack(Logstash, Kibana and Beats) but the work is in progress and in a beta stage. To test the interpretation, you can follow the below steps.

First, download and install the Manticore beta version that supports Kibana integration on the Manticore Beta release page. There are releases for several other OS versions there.

##On Ubuntu 22.04
wget https://repo.manticoresearch.com/repository/kibana_beta/ubuntu/jammy.zip

##On Debian 11
wget https://repo.manticoresearch.com/repository/kibana_beta/debian/bullseye.zip

##On CentOS 8/Rhel 8/Rocky Linux 8/ALma Linux 8
wget https://repo.manticoresearch.com/repository/kibana_beta/centos/rhel8.zip

Once downloaded. extract the archive:

unzip *.zip

Now run the installation:

##On Debian/ubuntu
sudo dpkg -i build/*

##On RedHat/CentOS/Rocky Linux/Alma Linux/Amazon Linux 2/Oracle Linux
sudo rpm -i build/*

Once installed, Manticore will be started. The next thing is to enable Kibana support using the command:

mysql -P9306 -h0 -e "set global log_management = 0; set global log_management = 1;"

The next thing is to run Kibana and point it to Manticore on port 9308. Here, we are running Kibana using Docker-compose:

$ vim docker-compose.yml
version: '2.2'

services:
  kibana:
    image: docker.elastic.co/kibana/kibana:7.4.2
    environment:
      ELASTICSEARCH_HOSTS: '["http://192.168.200.85:9308"]'
    ports:
      - 5601:5601

Remember to provide the correct IP address of your Manticore search server. You can modify the config file in /etc/manticoresearch/manticore.conf to allow the service to be accessed externally. Also, allow the port through the firewall.

Run the container using:

docker-compose up -d

Now, we can put sample data into Manticore. For example on Debian/Ubuntu

sudo apt install php composer php-mysql
wget https://gist.githubusercontent.com/sanikolaev/13bf61bbe6c39350bded7c577216435f/raw/8d8029c0d99998c901973fd9ac66a6fb920deda7/load_manticore_sharded.php
composer require fakerphp/faker
php load_manticore_sharded.php 10000 16 1000000 16 1

Once imported. you should see the data on your Kibana interface on port 5601 as shown:

Manticore Search Alternative to Elasticsearch in ELK

Verdict

That marks the end of this detailed guide on how to install and use Manticore Search – Alternative to Elasticsearch in ELK. You are now free to explore the additional features on your own.

See more:

LEAVE A REPLY

Please enter your comment!
Please enter your name here