Finally port the OpenBSD regression suite.

Both kristaps@ and wiz@ repeated asked for this,
literally for years.

1 files changed, 189 insertions, 0 deletions

diff --git a/regress/regress.pl.1 b/regress/regress.pl.1
new file mode 100644
index 00000000..6feebd16
--- /dev/null
+++ b/regress/regress.pl.1
@@ -0,0 +1,189 @@
+.\"	$Id: regress.pl.1,v 1.1 2017/02/08 03:02:13 schwarze Exp $
+.\"
+.\" Copyright (c) 2017 Ingo Schwarze <schwarze@openbsd.org>
+.\"
+.\" Permission to use, copy, modify, and distribute this software for any
+.\" purpose with or without fee is hereby granted, provided that the above
+.\" copyright notice and this permission notice appear in all copies.
+.\"
+.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+.\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+.\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+.\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+.\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+.\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+.\"
+.Dd $Mdocdate: February 8 2017 $
+.Dt REGRESS.PL 1
+.Os
+.Sh NAME
+.Nm regress.pl
+.Nd portable steering script for mandoc regression tests
+.Sh SYNOPSIS
+.Nm ./regress.pl
+.Oo
+.Ar directory Ns Op Pf : Ar test
+.Op Ar modifier ...
+.Oc
+.Sh DESCRIPTION
+The
+.Nm
+steering script allows running the
+.Xr mandoc 1
+regression suite on arbitrary operating systems,
+even though the suite was designed for OpenBSD only.
+.Pp
+When run without an argument,
+.Nm
+runs the complete regression suite.
+.Pp
+When run with one argument, that argument can be:
+.Bl -enum
+.It
+A single dot to run the complete suite.
+.It
+One of the top level directories, for example
+.Pa mdoc ,
+to run the test suite for a complete language or feature group.
+.It
+A subdirectory, for example
+.Pa man/IP ,
+to run the tests for a specific macro or an individual feature.
+.It
+A subdirectory with a test name appended with a colon, for example
+.Pa char/unicode : Ns Pa named ,
+to run the tests for one particular input file.
+.El
+.Pp
+Any additional arguments modify the way the tests are run.
+The default is
+.Cm all .
+The following modifiers are available:
+.Bl -tag -width verbose
+.It Cm all
+Run all kinds of subtests.
+This implies all other modifiers except
+.Cm verbose
+and
+.Cm clean .
+.It Cm ascii
+Run subtests for
+.Fl T Cm ascii
+output mode.
+.It Cm clean
+Remove all output files created by running the tests.
+.It Cm html
+Run subtests for
+.Fl T Cm html
+output mode.
+.It Cm lint
+Run subtests for
+.Fl T Cm lint
+warning and error output.
+.It Cm man
+Run subtests for
+.Fl T Cm man
+output mode.
+.It Cm utf8
+Run subtests for
+.Fl T Cm utf8
+output mode.
+.It Cm verbose
+Display approximate indications of what is being done.
+.El
+.Pp
+The amount of summary lines shown can be modified by giving an
+argument consisting of a single digit:
+.Bl -tag -width verbose
+.It Cm 3
+Show all summary lines for all directories entered.
+Even without
+.Cm verbose ,
+this generates more than hundred lines of output when running the
+complete regression suite.
+.It Cm 2
+This is the default.
+It shows the summary lines for the
+.Ar directory
+given on the command line and its immediate children.
+Except for
+.Pa mdoc ,
+the output usually fits on one screen.
+.It Cm 1
+Only show a single summary line for the whole run.
+.It Cm 0
+Do not show any summary lines.
+No output means success.
+Success or failure can also be seen from the exit status.
+.El
+.Pp
+All failed tests are always reported, even when the
+.Cm 0
+modifier is given.
+.Sh EXIT STATUS
+.Ex -std
+.Sh EXAMPLES
+The recommended invocation for casual users:
+.Pp
+.Dl ./regress.pl
+.Pp
+Maximum output:
+.Pp
+.Dl ./regress.pl \&. verbose
+.Pp
+Complete check, but keep the tree clean:
+.Pp
+.Dl ./regress.pl \&. all clean
+.Pp
+Test all of
+.Pa mdoc ,
+but don't print the usual 65 lines of output:
+.Pp
+.Dl ./regress.pl mdoc 1
+.Pp
+Investigate a specific failure:
+.Pp
+.Dl ./regress.pl mdoc/Bd:broken man verbose
+.Sh HISTORY
+The
+.Nm
+script appeared in release 1.14.1 of the portable
+.Sy mandoc
+distribution.
+.Sh AUTHORS
+.An Ingo Schwarze Aq Mt schwarze@openbsd.org
+.Sh CAVEATS
+This script is not optimized for elegance.
+Regression suites for other software should not copy the design.
+.Pp
+The problem it solves is that the
+.Sy mandoc
+regression suite is tightly integrated into the regression
+testing system of the OpenBSD base system, which requires
+both OpenBSD
+.Xr make 1 ,
+working neither with POSIX make nor with GNU make, and which
+also requires the OpenBSD-specific Makefile fragments in
+.Pa /usr/share/mk .
+The workaround of parsing the Makefiles by hand and constructing
+the required command lines by hand is unavoidably messy; it's
+the classic no-no of parsing a language with an ad-hoc incomplete
+parser.
+But the problem of providing this regression suite for other
+operating systems stood unsolved for many years, and no cleaner
+solution was found that could be implemented with reasonable effort.
+So maybe this is better than nothing.
+.Pp
+The top-level Makefiles for running this regression suite on
+OpenBSD are not included in the portable distribution.
+They are too OpenBSD-specific to be useful elsewhere,
+and on OpenBSD itself, the suite ought be run natively from
+.Pa /usr/src/regress/usr.bin/mandoc
+and not from the portable distribution.
+.Pp
+The
+.Pa db
+subdirectory of the regression suite is not included.
+It uses a Makefile structure that differs vastly from the
+rest of the suite.