A missing initialization could randomly cause regular expression

[mandoc.git] / mandoc.db.5

diff --git a/mandoc.db.5 b/mandoc.db.5
index 8d5649578f54d0a13675e319c5f13dba2bd74000..26907d139f4174b0f7ba598d8cc0c9b118092d90 100644 (file)
--- a/mandoc.db.5
+++ b/mandoc.db.5
@@ -1,6 +1,6 @@
-.\"    $Id: mandoc.db.5,v 1.1 2014/04/15 20:18:26 schwarze Exp $
+.\"    $Id: mandoc.db.5,v 1.5 2016/08/01 12:27:15 schwarze Exp $

 .\"
 .\"

-.\" Copyright (c) 2014 Ingo Schwarze <schwarze@openbsd.org>
+.\" Copyright (c) 2014, 2016 Ingo Schwarze <schwarze@openbsd.org>

 .\"
 .\" Permission to use, copy, modify, and distribute this software for any
 .\" purpose with or without fee is hereby granted, provided that the above
 .\"
 .\" Permission to use, copy, modify, and distribute this software for any
 .\" purpose with or without fee is hereby granted, provided that the above


@@ -14,7 +14,7 @@
 .\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
 .\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 .\"
 .\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
 .\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 .\"

-.Dd $Mdocdate: April 15 2014 $
+.Dd $Mdocdate: August 1 2016 $

 .Dt MANDOC.DB 5
 .Os
 .Sh NAME
 .Dt MANDOC.DB 5
 .Os
 .Sh NAME


@@ -23,7 +23,7 @@
 .Sh DESCRIPTION
 The
 .Nm
 .Sh DESCRIPTION
 The
 .Nm

-SQLite3 file format is used to store information about installed manual
+file format is used to store information about installed manual

 pages to facilitate semantic searching for manuals.
 Each manual page tree contains its own
 .Nm
 pages to facilitate semantic searching for manuals.
 Each manual page tree contains its own
 .Nm


@@ -34,93 +34,180 @@ for examples.
 Such database files are generated by
 .Xr makewhatis 8
 and used by
 Such database files are generated by
 .Xr makewhatis 8
 and used by

+.Xr man 1 ,

 .Xr apropos 1
 and
 .Xr whatis 1 .
 .Pp
 .Xr apropos 1
 and
 .Xr whatis 1 .
 .Pp

-One line in the following tables describes:
-.Bl -tag -width Ds
-.It Sy mpages
-One physical manual page file, no matter how many times and under which
-names it may appear in the file system.
-.It Sy mlinks
-One entry in the file system, no matter which content it points to.
-.It Sy names
-One manual page name, no matter whether it appears in a page header,
-in a NAME or SYNOPSIS section, or as a file name.
-.It Sy keys
-One chunk of text from some macro invocation.
+The file format uses three datatypes:
+.Pp
+.Bl -dash -compact -offset 2n -width 1n
+.It
+32-bit signed integer numbers in big endian (network) byte ordering
+.It
+NUL-terminated strings
+.It
+lists of NUL-terminated strings, terminated by a second NUL character

 .El
 .Pp
 .El
 .Pp

-Each record in the latter three tables uses its
-.Va pageid
-column to point to a record in the
-.Sy mpages
-table.
+Numbers are aligned to four-byte boundaries; where they follow
+strings or lists of strings, padding with additional NUL characters
+occurs.
+Some, but not all, numbers point to positions in the file.
+These pointers are measured in bytes, and the first byte of the
+file is considered to be byte 0.

 .Pp
 .Pp

-The other columns are as follows; unless stated otherwise, they are
-of type
-.Vt TEXT .
-.Bl -tag -width mpages.desc
-.It Sy mpages.desc
-The description line
-.Pq Sq \&Nd
-of the page.
-.It Sy mpages.form
-The
-.Vt INTEGER
-1 if the page is unformatted, i.e. in
+Each file consists of:
+.Pp
+.Bl -dash -compact -offset 2n -width 1n
+.It
+One magic number, 0x3a7d0cdb.
+.It
+One version number, currently 1.
+.It
+One pointer to the macros table.
+.It
+One pointer to the final magic number.
+.It
+The pages table (variable length).
+.It
+The macros table (variable length).
+.It
+The magic number once again, 0x3a7d0cdb.
+.El
+.Pp
+The pages table contains one entry for each physical manual page
+file, no matter how many hard and soft links it may have in the
+file system.
+The pages table consists of:
+.Pp
+.Bl -dash -compact -offset 2n -width 1n
+.It
+The number of pages in the database.
+.It
+For each page:
+.Bl -dash -compact -offset 2n -width 1n
+.It
+One pointer to the list of names.
+.It
+One pointer to the list of sections.
+.It
+One pointer to the list of architectures
+or 0 if the page is machine-independent.
+.It
+One pointer to the one-line description string.
+.It
+One pointer to the list of filenames.
+.El
+.It
+For each page, the list of names.
+Each name is preceded by a single byte indicating the sources of the name.
+The meaning of the bits is:
+.Bl -dash -compact -offset 2n -width 1n
+.It
+0x10: The name appears in a filename.
+.It
+0x08: The name appears in a header line, i.e. in a .Dt or .TH macro.
+.It
+0x04: The name is the first one in the title line, i.e. it appears
+in the first .Nm macro in the NAME section.
+.It
+0x02: The name appears in any .Nm macro in the NAME section.
+.It
+0x01: The name appears in an .Nm block in the SYNOPSIS section.
+.El
+.It
+For each page, the list of sections.
+Each section is given as a string, not as a number.
+.It
+For each architecture-dependent page, the list of architectures.
+.It
+For each page, the one-line description string taken from the .Nd macro.
+.It
+For each page, the list of filenames relative to the root of the
+respective manpath.
+This list includes hard links, soft links, and links simulated
+with .so
+.Xr roff 7
+requests.
+The first filename is preceded by a single byte
+having the following significance:
+.Bl -dash -compact -offset 2n -width 1n
+.It
+.Dv FORM_SRC No = 0x01 :
+The file format is

 .Xr mdoc 7
 or
 .Xr mdoc 7
 or

-.Xr man 7
-format, and 2 if it is formatted, i.e. a
-.Sq cat
-page.
-.It Sy mlinks.sec
-The manual section as found in the subdirectory name.
-.It Sy mlinks.arch
-The manual architecture as found in the subdirectory name, or
-.Qq any .
-.It Sy mlinks.name
-The manual name as found in the file name.
-.It Sy names.bits
-An
-.Vt INTEGER
-bit mask telling whether the name came from a header line, from the
-NAME or SYNOPSIS section, or from a file name.
-Bits are defined in
-.In mansearch.h .
-.It Sy names.name
-The name itself.
-.It Sy keys.bits
-An
-.Vt INTEGER
-bit mask telling which semantic contexts the key was found in;
-defined in
-.In mansearch.h ,
-documented in
+.Xr man 7 .
+.It
+.Dv FORM_CAT No = 0x02 :
+The manual page is preformatted.
+.El
+.It
+Zero to three NUL bytes for padding.
+.El
+.Pp
+The macros table consists of:
+.Pp
+.Bl -dash -compact -offset 2n -width 1n
+.It
+The number of different macro keys, currently 36.
+The ordering of macros is defined in
+.In mansearch.h
+and the significance of the macro keys is documented in

 .Xr apropos 1 .
 .Xr apropos 1 .

-.It Sy keys.key
-The string found in those contexts.
+.It
+For each macro key, one pointer to the respective macro table.
+.It
+For each macro key, the macro table (variable length).
+.El
+.Pp
+Each macro table consists of:
+.Pp
+.Bl -dash -compact -offset 2n -width 1n
+.It
+The number of entries in the table.
+.It
+For each entry:
+.Bl -dash -compact -offset 2n -width 1n
+.It
+One pointer to the value of the macro key.
+Each value is a string of text taken from some macro invocation.
+.It
+One pointer to the list of pages.
+.El
+.It
+For each entry, the value of the macro key.
+.It
+Zero to three NUL bytes for padding.
+.It
+For each entry, one or more pointers to pages in the pages table,
+pointing to the pointer to the list of names,
+followed by the number 0.

 .El
 .Sh FILES
 .El
 .Sh FILES

-.Bl -tag -width /usr/share/mandoc.db -compact
-.It Pa /usr/share/mandoc.db
+.Bl -tag -width /usr/share/man/mandoc.db -compact
+.It Pa /usr/share/man/mandoc.db

 The manual page database for the base system.
 The manual page database for the base system.

-.It Pa /usr/X11R6/mandoc.db
+.It Pa /usr/X11R6/man/mandoc.db

 The same for the
 .Xr X 7
 Window System.
 The same for the
 .Xr X 7
 Window System.

-.It Pa /usr/local/mandoc.db
+.It Pa /usr/local/man/mandoc.db

 The same for
 .Xr packages 7 .
 .El
 The same for
 .Xr packages 7 .
 .El

+.Pp
+A program to dump
+.Nm
+files in a human-readable format suitable for
+.Xr diff 1
+is provided in the directory
+.Pa /usr/src/regress/usr.bin/mandoc/db/dbm_dump/ .

 .Sh SEE ALSO
 .Xr apropos 1 ,
 .Xr man 1 ,
 .Sh SEE ALSO
 .Xr apropos 1 ,
 .Xr man 1 ,

-.Xr sqlite3 1 ,

 .Xr whatis 1 ,
 .Xr whatis 1 ,

-.Xr mansearch 3 ,

 .Xr makewhatis 8
 .Sh HISTORY
 A manual page database
 .Xr makewhatis 8
 .Sh HISTORY
 A manual page database


@@ -128,7 +215,7 @@ A manual page database
 first appeared in
 .Bx 2 .
 The present format first appeared in
 first appeared in
 .Bx 2 .
 The present format first appeared in

-.Ox 5.6 .
+.Ox 6.1 .

 .Sh AUTHORS
 .An -nosplit
 The original version of
 .Sh AUTHORS
 .An -nosplit
 The original version of


@@ -136,9 +223,6 @@ The original version of
 was written by
 .An Bill Joy
 in 1979.
 was written by
 .An Bill Joy
 in 1979.

-An SQLite3 version was first implemented by
-.An Kristaps Dzonsons Aq Mt kristaps@bsd.lv
-in 2012.

 The present database format was designed by
 .An Ingo Schwarze Aq Mt schwarze@openbsd.org
 The present database format was designed by
 .An Ingo Schwarze Aq Mt schwarze@openbsd.org

-in 2014.
+in 2016.