mansearch.3

   1 .\"     $Id: mansearch.3,v 1.1 2014/04/15 20:18:26 schwarze Exp $
   2 .\"
   3 .\" Copyright (c) 2014 Ingo Schwarze <schwarze@openbsd.org>
   4 .\"
   5 .\" Permission to use, copy, modify, and distribute this software for any
   6 .\" purpose with or without fee is hereby granted, provided that the above
   7 .\" copyright notice and this permission notice appear in all copies.
   8 .\"
   9 .\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
  10 .\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
  11 .\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
  12 .\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
  13 .\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
  14 .\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
  15 .\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
  16 .\"
  17 .Dd $Mdocdate: April 15 2014 $
  18 .Dt MANSEARCH 3
  19 .Os
  20 .Sh NAME
  21 .Nm mansearch ,
  22 .Nm mansearch_setup
  23 .Nd search manual page databases
  24 .Sh SYNOPSIS
  25 .In manpath.h
  26 .In mansearch.h
  27 .Ft int
  28 .Fo mansearch_setup
  29 .Fa "int start"
  30 .Fc
  31 .Ft int
  32 .Fo mansearch
  33 .Fa "const struct mansearch *search"
  34 .Fa "const struct manpaths *paths"
  35 .Fa "int argc"
  36 .Fa "char *argv[]"
  37 .Fa "const char *outkey"
  38 .Fa "struct manpage **res"
  39 .Fa "size_t *sz"
  40 .Fc
  41 .Sh DESCRIPTION
  42 The
  43 .Fn mansearch
  44 function returns information about manuals matching a search query from a
  45 .Xr mandoc.db 5
  46 SQLite3 database.
  47 .Pp
  48 The query arguments are as follows:
  49 .Bl -tag -width Ds
  50 .It Fa "const struct mansearch *search"
  51 Search options, defined in
  52 .In mansearch.h .
  53 .It Fa "const struct manpaths *paths"
  54 Directories to be searched, defined in
  55 .In manpath.h .
  56 .It Fa "int argc" , "char *argv[]"
  57 Search criteria, usually taken from the command line.
  58 .El
  59 .Pp
  60 The
  61 .Fa "const char *outkey"
  62 selects which data to return in the
  63 .Va output
  64 field of the
  65 .Fa res
  66 structures.
  67 It takes any of the macro keys defined in
  68 .Pa mansearch_const.c
  69 and described in
  70 .Xr apropos 1 .
  71 .Pp
  72 The output arguments are as follows:
  73 .Bl -tag -width Ds
  74 .It Fa "struct manpage **res"
  75 Returns a pointer to an array of result structures defined in
  76 .In mansearch.h .
  77 The user is expected to call
  78 .Xr free 3
  79 on the
  80 .Va file ,
  81 .Va names ,
  82 and
  83 .Va output
  84 fields of all structures, as well as the
  85 .Fa res
  86 array itself.
  87 .It Fa "size_t *sz"
  88 Returns the number of result structures contained in
  89 .Fa res .
  90 .El
  91 .Pp
  92 To speed up searches, the
  93 .Fn mansearch_setup
  94 function can optionally be called with a
  95 .Fa start
  96 argument of 1 before
  97 .Fn mansearch
  98 to set up an SQLite3 pagecache.
  99 If it was called, it has to be called again with a
 100 .Fa start
 101 argument of 0 after the last call to
 102 .Fn mansearch
 103 to release the memory used for the pagecache.
 104 .Sh IMPLEMENTATION NOTES
 105 For each manual page tree, the search is done in two steps.
 106 In the first step, a list of pages matching the search criteria is built.
 107 In the second step, the requested information about these pages is
 108 retrieved from the database and assembled into the
 109 .Fa res
 110 array.
 111 .Pp
 112 All function mentioned here are defined in the file
 113 .Pa mansearch.c .
 114 No functions except
 115 .Fn mansearch
 116 and
 117 .Fn sql_statement
 118 build any SQL code, and no functions except
 119 .Fn mansearch ,
 120 .Fn buildnames ,
 121 and
 122 .Fn buildoutput
 123 execute it.
 124 .Ss Finding matches
 125 The query is built using the following grammar:
 126 .Bd -literal -offset indent
 127 <query>     ::= "SELECT * FROM mpages WHERE" <condition>
 128 <condition> ::= "(" <condition> ")" |
 129                 <condition> "OR" <condition> |
 130                 <condition> "AND" <condition> |
 131                 "desc" <operator> "?" |
 132                 "id IN (SELECT pageid FROM" <subquery> ")"
 133 <subquery>  ::= "names WHERE name" <operator> "?" |
 134                 "keys WHERE key" <operator> "? AND bits & ?"
 135 <operator>  ::= "MATCH" | "REGEXP"
 136 .Ed
 137 .Pp
 138 The MATCH and REGEXP operators are implemented by the functions
 139 .Fn sql_match
 140 and
 141 .Fn sql_regexp ,
 142 respectively.
 143 This is required because SQLite3 natively neither supports
 144 case-insensitive substring matching nor regular expression matching,
 145 but only string identity, shell globbing, and the weird home-brewed
 146 LIKE operator.
 147 .Pp
 148 Command line parsing is done by the function
 149 .Fn exprcomp
 150 building a singly linked list of
 151 .Vt expr
 152 structures, using the helper functions
 153 .Fn exprterm
 154 and
 155 .Fn exprspec .
 156 The resulting SQL statement is assembled by the function
 157 .Fn sql_statement
 158 and evaluated in the main loop of the
 159 .Fn mansearch
 160 function.
 161 .Ss Assembling the results
 162 The names, sections, and architectures of the manuals found
 163 are assembled into the
 164 .Va names
 165 field of the result structure by the function
 166 .Fn buildnames ,
 167 using the following query:
 168 .Pp
 169 .Dl "SELECT * FROM mlinks WHERE pageid=? ORDER BY sec, arch, name"
 170 .Pp
 171 If the
 172 .Fa outkey
 173 differs from
 174 .Qq Nd ,
 175 the requested output data is assembled into the
 176 .Va output
 177 field of the result structure by the function
 178 .Fn buildoutput ,
 179 using the following query:
 180 .Pp
 181 .Dl "SELECT * FROM keys WHERE pageid=? AND bits & ?"
 182 .Sh FILES
 183 .Bl -tag -width mandoc.db -compact
 184 .It Pa mandoc.db
 185 The manual page database.
 186 .El
 187 .Sh EXAMPLES
 188 The simplest invocation
 189 .Pp
 190 .Dl apropos keyword
 191 .Pp
 192 results in the following SQL query:
 193 .Bd -literal
 194 SELECT * FROM mpages WHERE (
 195   id IN (SELECT pageid FROM names WHERE name MATCH 'keyword') OR
 196   desc MATCH 'keyword'
 197 );
 198 .Ed
 199 .Pp
 200 A more complicated request like
 201 .Pp
 202 .Dl apropos -s 2 Nm,Xr=getuid
 203 .Pp
 204 results in:
 205 .Bd -literal
 206 SELECT * FROM mpages WHERE (
 207   id IN (SELECT pageid FROM names WHERE name MATCH 'getuid') OR
 208   id IN (SELECT pageid FROM keys WHERE key MATCH 'getuid' AND bits & 4)
 209 ) AND id IN (SELECT pageid FROM keys WHERE key REGEXP '^2$' AND bits & 2);
 210 .Ed
 211 .Sh SEE ALSO
 212 .Xr apropos 1 ,
 213 .Xr mandoc.db 5 ,
 214 .Xr makewhatis 8
 215 .Sh HISTORY
 216 The
 217 .Fn mansearch
 218 subsystem first appeared in
 219 .Ox 5.6 .
 220 .Sh AUTHORS
 221 .An -nosplit
 222 A module to search manual page databases was first written by
 223 .An Kristaps Dzonsons Aq Mt kristaps@bsd.lv
 224 in 2011, at first using the Berkeley DB;
 225 he rewrote it for SQLite3 in 2012.
 226 The current version received major changes from
 227 .An Ingo Schwarze Aq Mt schwarze@openbsd.org .