manticoresearch/doc/indextool.1

'\" t
.\"     Title: indexer
.\"    Author: [see the "Author" section]
.\"    Manual: Manticore Search
.\"    Source: Manticore Search
.\"  Language: English
.\"
.TH "INDEXTOOL" "1" "05/02/2026" "Manticore Search" ""
.SH "NAME"
indextool
.SH "SYNOPSIS"
indextool <command> [options]
.SH DESCRIPTION
.PP
\fB\fCindextool\fR is a helpful utility that extracts various information about a physical table, excluding \fB\fCtemplate\fR or \fB\fCdistributed\fR tables. Here's the general syntax for utilizing \fB\fCindextool\fR:
.PP
.RS
.nf
indextool <command> [options]
.fi
.RE
.SH OPTIONS
.PP
These options are applicable to all commands:
.RS
.IP \(bu 2
\fB\fC\-\-config <file>\fR (\fB\fC\-c <file>\fR for short) lets you override the default configuration file names.
.IP \(bu 2
\fB\fC\-\-quiet\fR (\fB\fC\-q\fR for short) suppresses the output of banners and such by \fB\fCindextool\fR\&.
.IP \(bu 2
\fB\fC\-\-help\fR (\fB\fC\-h\fR for short) displays all parameters available in your specific build of \fB\fCindextool\fR\&.
.IP \(bu 2
\fB\fC\-v\fR displays the version information of your specific \fB\fCindextool\fR build.
.RE
.SS Commands
.PP
Here are the available commands:
.RS
.IP \(bu 2
\fB\fC\-\-checkconfig\fR loads and verifies the config file, checking its validity and for any syntax errors.
.IP \(bu 2
\fB\fC\-\-buildidf DICTFILE1 [DICTFILE2 ...] \-\-out IDFILE\fR constructs an IDF file from one or more dictionary dumps (refer to \fB\fC\-\-dumpdict\fR). The additional parameter \fB\fC\-\-skip\-uniq\fR will omit unique words (df=1).
.IP \(bu 2
\fB\fC\-\-build\-infixes TABLENAME\fR generates infixes for a pre\-existing dict=keywords table (updates .sph, .spi in place). Use this option for legacy table files already employing dict=keywords, but now requiring infix search support; updating the table files with indextool may be simpler or quicker than recreating them from scratch with indexer.
.IP \(bu 2
\fB\fC\-\-dumpheader FILENAME.sph\fR promptly dumps the given table header file without disturbing any other table files or even the config file. The report offers a detailed view of all the table settings, especially the complete attribute and field list.
.IP \(bu 2
\fB\fC\-\-dumpconfig FILENAME.sph\fR extracts the table definition from the specified table header file in an (almost) manticore.conf file\-compliant format.
.IP \(bu 2
\fB\fC\-\-dumpheader TABLENAME\fR dumps table header by table name while searching for the header path in the config file.
.IP \(bu 2
\fB\fC\-\-dumpdict TABLENAME\fR dumps the dictionary. An extra \fB\fC\-stats\fR switch will add the total document count to the dictionary dump. This is necessary for dictionary files used in IDF file creation.
.IP \(bu 2
\fB\fC\-\-dumpdocids TABLENAME\fR dumps document IDs by table name.
.IP \(bu 2
\fB\fC\-\-dumphitlist TABLENAME KEYWORD\fR dumps all instances (occurrences) of a specified keyword in a given table, with the keyword defined as text.
.IP \(bu 2
\fB\fC\-\-dumphitlist TABLENAME \-\-wordid ID\fR dumps all instances (occurrences) of a specific keyword in a given table, with the keyword represented as an internal numeric ID.
.IP \(bu 2
\fB\fC\-\-dumpkilllist TABLENAME\fR dumps the kill list contents of the specified table, including targets and flags. It can also be used as \fB\fC\-\-dumpkilllist FILENAME.spk\fR\&.
.IP \(bu 2
\fB\fC\-\-docextract TBL DOCID\fR executes a standard table check pass of the entire dictionary/docs/hits, and gathers all the words and hits associated with the requested document. Subsequently, all the words are arranged according to their fields and positions, and the result is printed, grouped by field.
.IP \(bu 2
\fB\fC\-\-fold TABLENAME OPTFILE\fR This option helps understand how the tokenizer processes input. You can supply the indextool with text from a file, if specified, or from stdin otherwise. The output will replace separators with spaces (based on your \fB\fCcharset_table\fR settings) and convert letters in words to lowercase.
.IP \(bu 2
\fB\fC\-\-htmlstrip TABLENAME\fR applies the HTML stripper settings for a specified table to filter stdin, and sends the filtering results to stdout. Be aware that the settings will be fetched from manticore.conf, and not from the table header.
.IP \(bu 2
\fB\fC\-\-mergeidf NODE1.idf [NODE2.idf ...] \-\-out GLOBAL.idf\fR combines multiple .idf files into a single one. The extra parameter \fB\fC\-\-skip\-uniq\fR will ignore unique words (df=1).
.IP \(bu 2
\fB\fC\-\-morph TABLENAME\fR applies morphology to the given stdin and directs the result to stdout.
.IP \(bu 2
\fB\fC\-\-check TABLENAME\fR evaluates the table data files for consistency errors that could be caused by bugs in \fB\fCindexer\fR or hardware faults. \fB\fC\-\-check\fR is also functional on RT tables, RAM, and disk chunks. Additional options:
.RS
.IP \(bu 2
\fB\fC\-\-check\-id\-dups\fR checks for duplicate document IDs in the plain table and all disk chunks of an RT table
.IP \(bu 2
\fB\fC\-\-check\-disk\-chunk CHUNK_NAME\fR checks only a specific disk chunk of an RT table. The argument is the numeric extension of the RT table's disk chunk to be checked.
.RE
.IP \(bu 2
\fB\fC\-\-strip\-path\fR removes the path names from all file names referred to from the table (stopwords, wordforms, exceptions, etc). This is helpful when verifying tables built on a different machine with possibly varying path layouts.
.IP \(bu 2
\fB\fC\-\-rotate\fR is only compatible with \fB\fC\-\-check\fR and determines whether to check the table waiting for rotation, i.e., with a .new extension. This is useful when you wish to validate your table before actually putting it into use.
.IP \(bu 2
\fB\fC\-\-apply\-killlists\fR loads and applies kill\-lists for all tables listed in the config file. Changes are saved in .SPM files. Kill\-list files (.SPK) are removed. This can be handy if you want to shift the application of tables from server startup to indexing stage.
.RE
.SS Important Consideration for RT Table Checking
.PP
\fB\fCindextool\fR cannot fully check an RT table that is currently served by the daemon. When attempting to check an active RT table, you may encounter the following warning:
.PP
.RS
.nf
WARNING: failed to load RAM chunks, checking only N disk chunks
.fi
.RE
.PP
To avoid these warning and ensure a proper check of an RT table, consider the following approaches:
.RS
.IP \(bu 2
stop the daemon before running \fB\fCindextool \-\-check\fR\&.
.IP \(bu 2
ensure that the RT table is not actively served by the daemon.
.IP \(bu 2
check a separate copy of the RT table instead of the live one.
.RE
.PP
If stopping the daemon is not an option, you can prevent unintended modifications to the RT table by executing the following MySQL statement before running \fB\fCindextool \-\-check\fR:
.PP
.RS
.nf
SET GLOBAL AUTO_OPTIMIZE=0;
.fi
.RE
.PP
This command prevents the daemon from performing auto\-optimization, ensuring that RT table files remain unchanged. After executing this statement, wait until the optimization thread has completely stopped before proceeding with \fB\fCindextool \-\-check\fR\&. This ensures that no disk chunks are unintentionally modified or removed during the check process.
If auto\-optimize was previously enabled, you should manually re\-enable it after the check is complete by running:
.PP
.RS
.nf
SET GLOBAL AUTO_OPTIMIZE=1;
.fi
.RE
.SH "AUTHOR"
.PP
Manticore Software LTD (https://manticoresearch\&.com)
.SH "COPYRIGHT"
.PP
Copyright 2017\-2026 Manticore Software LTD (https://manticoresearch\&.com), 2008\-2016 Sphinx Technologies Inc (http://sphinxsearch\&.com), 2001\-2016 Andrew Aksyonoff
.PP
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU General Public License, Version 3 or any later version published by the Free Software Foundation\&.
.SH "SEE ALSO"
.PP
\fBsearchd\fR(1),
\fBindexer\fR(1)
.PP
Manticore Search and its related programs are thoroughly documented
in the \fIManticore Search reference manual\fR, which is accessible
at https://manual.manticoresearch.com/