From cfb8dd7a40023aa8cfb7a61c9e190461c7b41ea7 Mon Sep 17 00:00:00 2001 From: Ingo Schwarze Date: Tue, 15 Apr 2014 20:18:26 +0000 Subject: Document the database format and SQL code generation; suggested by kristaps@ and espie@. --- mansearch.3 | 227 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 227 insertions(+) create mode 100644 mansearch.3 (limited to 'mansearch.3') diff --git a/mansearch.3 b/mansearch.3 new file mode 100644 index 00000000..38051528 --- /dev/null +++ b/mansearch.3 @@ -0,0 +1,227 @@ +.\" $Id: mansearch.3,v 1.1 2014/04/15 20:18:26 schwarze Exp $ +.\" +.\" Copyright (c) 2014 Ingo Schwarze +.\" +.\" Permission to use, copy, modify, and distribute this software for any +.\" purpose with or without fee is hereby granted, provided that the above +.\" copyright notice and this permission notice appear in all copies. +.\" +.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +.\" +.Dd $Mdocdate: April 15 2014 $ +.Dt MANSEARCH 3 +.Os +.Sh NAME +.Nm mansearch , +.Nm mansearch_setup +.Nd search manual page databases +.Sh SYNOPSIS +.In manpath.h +.In mansearch.h +.Ft int +.Fo mansearch_setup +.Fa "int start" +.Fc +.Ft int +.Fo mansearch +.Fa "const struct mansearch *search" +.Fa "const struct manpaths *paths" +.Fa "int argc" +.Fa "char *argv[]" +.Fa "const char *outkey" +.Fa "struct manpage **res" +.Fa "size_t *sz" +.Fc +.Sh DESCRIPTION +The +.Fn mansearch +function returns information about manuals matching a search query from a +.Xr mandoc.db 5 +SQLite3 database. +.Pp +The query arguments are as follows: +.Bl -tag -width Ds +.It Fa "const struct mansearch *search" +Search options, defined in +.In mansearch.h . +.It Fa "const struct manpaths *paths" +Directories to be searched, defined in +.In manpath.h . +.It Fa "int argc" , "char *argv[]" +Search criteria, usually taken from the command line. +.El +.Pp +The +.Fa "const char *outkey" +selects which data to return in the +.Va output +field of the +.Fa res +structures. +It takes any of the macro keys defined in +.Pa mansearch_const.c +and described in +.Xr apropos 1 . +.Pp +The output arguments are as follows: +.Bl -tag -width Ds +.It Fa "struct manpage **res" +Returns a pointer to an array of result structures defined in +.In mansearch.h . +The user is expected to call +.Xr free 3 +on the +.Va file , +.Va names , +and +.Va output +fields of all structures, as well as the +.Fa res +array itself. +.It Fa "size_t *sz" +Returns the number of result structures contained in +.Fa res . +.El +.Pp +To speed up searches, the +.Fn mansearch_setup +function can optionally be called with a +.Fa start +argument of 1 before +.Fn mansearch +to set up an SQLite3 pagecache. +If it was called, it has to be called again with a +.Fa start +argument of 0 after the last call to +.Fn mansearch +to release the memory used for the pagecache. +.Sh IMPLEMENTATION NOTES +For each manual page tree, the search is done in two steps. +In the first step, a list of pages matching the search criteria is built. +In the second step, the requested information about these pages is +retrieved from the database and assembled into the +.Fa res +array. +.Pp +All function mentioned here are defined in the file +.Pa mansearch.c . +No functions except +.Fn mansearch +and +.Fn sql_statement +build any SQL code, and no functions except +.Fn mansearch , +.Fn buildnames , +and +.Fn buildoutput +execute it. +.Ss Finding matches +The query is built using the following grammar: +.Bd -literal -offset indent + ::= "SELECT * FROM mpages WHERE" + ::= "(" ")" | + "OR" | + "AND" | + "desc" "?" | + "id IN (SELECT pageid FROM" ")" + ::= "names WHERE name" "?" | + "keys WHERE key" "? AND bits & ?" + ::= "MATCH" | "REGEXP" +.Ed +.Pp +The MATCH and REGEXP operators are implemented by the functions +.Fn sql_match +and +.Fn sql_regexp , +respectively. +This is required because SQLite3 natively neither supports +case-insensitive substring matching nor regular expression matching, +but only string identity, shell globbing, and the weird home-brewed +LIKE operator. +.Pp +Command line parsing is done by the function +.Fn exprcomp +building a singly linked list of +.Vt expr +structures, using the helper functions +.Fn exprterm +and +.Fn exprspec . +The resulting SQL statement is assembled by the function +.Fn sql_statement +and evaluated in the main loop of the +.Fn mansearch +function. +.Ss Assembling the results +The names, sections, and architectures of the manuals found +are assembled into the +.Va names +field of the result structure by the function +.Fn buildnames , +using the following query: +.Pp +.Dl "SELECT * FROM mlinks WHERE pageid=? ORDER BY sec, arch, name" +.Pp +If the +.Fa outkey +differs from +.Qq Nd , +the requested output data is assembled into the +.Va output +field of the result structure by the function +.Fn buildoutput , +using the following query: +.Pp +.Dl "SELECT * FROM keys WHERE pageid=? AND bits & ?" +.Sh FILES +.Bl -tag -width mandoc.db -compact +.It Pa mandoc.db +The manual page database. +.El +.Sh EXAMPLES +The simplest invocation +.Pp +.Dl apropos keyword +.Pp +results in the following SQL query: +.Bd -literal +SELECT * FROM mpages WHERE ( + id IN (SELECT pageid FROM names WHERE name MATCH 'keyword') OR + desc MATCH 'keyword' +); +.Ed +.Pp +A more complicated request like +.Pp +.Dl apropos -s 2 Nm,Xr=getuid +.Pp +results in: +.Bd -literal +SELECT * FROM mpages WHERE ( + id IN (SELECT pageid FROM names WHERE name MATCH 'getuid') OR + id IN (SELECT pageid FROM keys WHERE key MATCH 'getuid' AND bits & 4) +) AND id IN (SELECT pageid FROM keys WHERE key REGEXP '^2$' AND bits & 2); +.Ed +.Sh SEE ALSO +.Xr apropos 1 , +.Xr mandoc.db 5 , +.Xr makewhatis 8 +.Sh HISTORY +The +.Fn mansearch +subsystem first appeared in +.Ox 5.6 . +.Sh AUTHORS +.An -nosplit +A module to search manual page databases was first written by +.An Kristaps Dzonsons Aq Mt kristaps@bsd.lv +in 2011, at first using the Berkeley DB; +he rewrote it for SQLite3 in 2012. +The current version received major changes from +.An Ingo Schwarze Aq Mt schwarze@openbsd.org . -- cgit v1.2.3-56-ge451