|
|
@@ -1,12 +1,14 @@
|
|
|
# Manticore
|
|
|
|
|
|
-Manticore Search is a multi-storage database specifically designed for search, with robust full-text search capabilities.
|
|
|
+Manticore Search is a high-performance, multi-storage database purpose-built for search and analytics, offering lightning-fast full-text search, real-time indexing, and advanced features like vector search and columnar storage for efficient data analysis. Designed to handle both small and large datasets, it delivers seamless scalability and powerful insights for modern applications.
|
|
|
|
|
|
-As an open-source database (available on GitHub), Manticore Search was created in 2017 as a continuation of Sphinx Search engine. Our development team took all the best features of Sphinx and significantly improved its functionality, fixing hundreds of bugs along the way (as detailed in our Changelog). With nearly complete code rewrites, Manticore Search is now a modern, fast, and light-weight database with full features and exceptional full-text search capabilities.
|
|
|
+As an open-source database (available on GitHub), Manticore Search was created in 2017 as a continuation of Sphinx Search engine. Our development team took all the best features of Sphinx and significantly improved its functionality, fixing hundreds of bugs along the way (as detailed in our Changelog). Manticore Search is a modern, fast, and lightweight database with exceptional full-text search capabilities, built on a nearly complete rewrite of its predecessor.
|
|
|
|
|
|
## Manticore's key features are:
|
|
|
#### Powerful and fast full-text searching that works well for small and large datasets
|
|
|
|
|
|
+ * Query autocomplete
|
|
|
+ * Fuzzy search
|
|
|
* Over 20 full-text operators and over 20 ranking factors
|
|
|
* Custom ranking
|
|
|
* Stemming
|
|
|
@@ -19,9 +21,11 @@ As an open-source database (available on GitHub), Manticore Search was created i
|
|
|
* Text highlighting
|
|
|
|
|
|
#### Vector search capabilities
|
|
|
-
|
|
|
Manticore Search supports the ability to add embeddings generated by your Machine Learning models to each document, and then doing a nearest-neighbor search on them. This lets you build features like similarity search, recommendations, semantic search, and relevance ranking based on NLP algorithms, among others, including image, video, and sound searches.
|
|
|
|
|
|
+#### JOIN
|
|
|
+Manticore Search supports JOIN queries via SQL and JSON, allowing you to combine data from multiple tables.
|
|
|
+
|
|
|
#### Multithreading
|
|
|
Manticore Search utilizes a smart query parallelization to lower response time and fully utilize all CPU cores when needed.
|
|
|
|
|
|
@@ -32,7 +36,7 @@ The cost-based query optimizer uses statistical data about the indexed data to e
|
|
|
Manticore offers both row-wise and column-oriented storage options to accommodate datasets of various sizes. The traditional and default row-wise storage option is available for datasets of all sizes - small, medium, and large, while the columnar storage option is provided through the Manticore Columnar Library for even larger datasets. The key difference between these storage options is that row-wise storage requires all attributes (excluding full-text fields) to be kept in RAM for optimal performance, while columnar storage does not, thus offering lower RAM consumption, but with a potential for slightly slower performance (as demonstrated by the statistics on https://db-benchmarks.com/).
|
|
|
|
|
|
#### Automatic secondary indexes
|
|
|
-Manticore Columnar Library uses Piecewise Geometric Model index, which exploits a learned mapping between the indexed keys and their location in memory. The succinctness of this mapping, coupled with a peculiar recursive construction algorithm, makes the PGM-index a data structure that dominates traditional indexes by orders of magnitude in space while still offering the best query and update time performance. Secondary indexes are ON by default for all numeric fields.
|
|
|
+Manticore Columnar Library uses Piecewise Geometric Model index, which exploits a learned mapping between the indexed keys and their location in memory. The succinctness of this mapping, coupled with a peculiar recursive construction algorithm, makes the PGM-index a data structure that dominates traditional indexes by orders of magnitude in space while still offering the best query and update time performance. Secondary indexes are ON by default for all numeric and string fields and can be enabled for json attributes.
|
|
|
|
|
|
#### SQL-first
|
|
|
Manticore's native syntax is SQL and it supports SQL over HTTP and MySQL protocol, allowing for connection through popular mysql clients in any programming language.
|
|
|
@@ -47,7 +51,7 @@ You can execute Elasticsearch-compatible insert and replace JSON queries which e
|
|
|
Easily create, update, and delete tables online or through a configuration file.
|
|
|
|
|
|
#### The benefits of C++ and the convenience of PHP
|
|
|
-The Manticore Search daemon is developed in C++, offering fast start times and efficient memory utilization. The utilization of low-level optimizations further boosts performance. Another crucial component, called Manticore Buddy, is written in PHP and is utilized for high-level functionality that does not require lightning-fast response times or extremely high processing power. Although contributing to the C++ code may pose a challenge, adding a new SQL/JSON command using Manticore Buddy should be a straightforward process.
|
|
|
+The Manticore Search daemon is developed in C++, offering fast start times and efficient memory utilization. Low-level optimizations further enhance performance. Another crucial component, called Manticore Buddy, is written in PHP and is utilized for high-level functionality that does not require lightning-fast response times or extremely high processing power. Although contributing to the C++ code may pose a challenge, adding a new SQL/JSON command using Manticore Buddy should be a straightforward process.
|
|
|
|
|
|
#### Real-Time inserts
|
|
|
Newly added or updated documents can be immediately read.
|
|
|
@@ -70,23 +74,39 @@ The `indexer` tool and comprehensive configuration syntax of Manticore make it e
|
|
|
#### Integration options
|
|
|
You can integrate Manticore Search with a MySQL/MariaDB server using the FEDERATED engine or via ProxySQL.
|
|
|
|
|
|
-You can use Apache Superset and Grafana to visualize data stored in Manticore. Various MySQL tools can be used to develop Manticore queries interactively, such as HeidiSQL and DBForge.
|
|
|
+You can use Apache Superset, Kibana and Grafana to visualize data stored in Manticore. Various MySQL tools can be used to develop Manticore queries interactively, such as HeidiSQL and DBForge.
|
|
|
|
|
|
#### Stream filtering made easy
|
|
|
Manticore offers a special table type, the "percolate" table, which allows you to search queries instead of data, making it an efficient tool for filtering full-text data streams. Simply store your queries in the table, process your data stream by sending each batch of documents to Manticore Search, and receive only the results that match your stored queries.
|
|
|
|
|
|
-# Possible applications:
|
|
|
-Manticore has a variety of use cases, including:
|
|
|
-
|
|
|
- * Full-text search
|
|
|
- * With small data volumes, enjoy the benefits of powerful full-text search syntax and low memory consumption (as low as 7-8 MB).
|
|
|
- * With large data, benefit from Manticore's high availability and ability to handle massive tables.
|
|
|
- * OLAP: Use Manticore Search and the Manticore Columnar Library to analyze terabytes of data on a single or multiple servers.
|
|
|
- * Faceted search
|
|
|
- * Geo-spatial search
|
|
|
- * Spell correction
|
|
|
- * Autocomplete
|
|
|
- * Data stream filtering
|
|
|
+#### Possible Applications
|
|
|
+Manticore Search is versatile and can be applied in various scenarios, including:
|
|
|
+
|
|
|
+- **Full-Text Search**:
|
|
|
+ - Ideal for e-commerce platforms, enabling fast and accurate product searches with features like autocomplete and fuzzy search.
|
|
|
+ - Perfect for content-heavy websites, allowing users to quickly find relevant articles or documents.
|
|
|
+
|
|
|
+- **Data Analytics**:
|
|
|
+ - Ingest data into Manticore Search using Beats/Logstash, Vector.dev, Fluentbit.
|
|
|
+ - Analyze large datasets efficiently using Manticore's columnar storage and OLAP capabilities.
|
|
|
+ - Perform complex queries on terabytes of data with minimal latency.
|
|
|
+ - Visualize data using Kibana, Grafana, or Apache Superset.
|
|
|
+
|
|
|
+- **Faceted Search**:
|
|
|
+ - Enable users to filter search results by categories, such as price, brand, or date, for a more refined search experience.
|
|
|
+
|
|
|
+- **Geo-Spatial Search**:
|
|
|
+ - Implement location-based searches, such as finding nearby restaurants or stores, with Manticore's geo-spatial capabilities.
|
|
|
+
|
|
|
+- **Spell Correction**:
|
|
|
+ - Automatically correct user typos in search queries to improve search accuracy and user experience.
|
|
|
+
|
|
|
+- **Autocomplete**:
|
|
|
+ - Provide real-time suggestions as users type, enhancing search usability and speed.
|
|
|
+
|
|
|
+- **Data Stream Filtering**:
|
|
|
+ - Use percolate tables to filter and process real-time data streams, such as social media feeds or log data, efficiently.
|
|
|
+
|
|
|
|
|
|
# Requirements
|
|
|
|
Sergey Nikolaev