regress/regress.pl.1

   1 .\"     $Id: regress.pl.1,v 1.3 2017/07/18 18:47:06 schwarze Exp $
   2 .\"
   3 .\" Copyright (c) 2017 Ingo Schwarze <schwarze@openbsd.org>
   4 .\"
   5 .\" Permission to use, copy, modify, and distribute this software for any
   6 .\" purpose with or without fee is hereby granted, provided that the above
   7 .\" copyright notice and this permission notice appear in all copies.
   8 .\"
   9 .\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
  10 .\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
  11 .\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
  12 .\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
  13 .\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
  14 .\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
  15 .\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
  16 .\"
  17 .Dd $Mdocdate: July 18 2017 $
  18 .Dt REGRESS.PL 1
  19 .Os
  20 .Sh NAME
  21 .Nm regress.pl
  22 .Nd portable steering script for mandoc regression tests
  23 .Sh SYNOPSIS
  24 .Nm ./regress.pl
  25 .Oo
  26 .Ar directory Ns Op Pf / Ar test
  27 .Op Ar modifier ...
  28 .Oc
  29 .Sh DESCRIPTION
  30 The
  31 .Nm
  32 steering script allows running the
  33 .Xr mandoc 1
  34 regression suite on arbitrary operating systems,
  35 even though the suite was originally designed for OpenBSD only.
  36 .Pp
  37 When run without an argument,
  38 .Nm
  39 runs the complete regression suite.
  40 .Pp
  41 The first argument is a Perl regular expression to match test names,
  42 automatically anchored at the beginning of the names.
  43 Test names are names of test input files without the file name extension
  44 .Pa .in ,
  45 for example
  46 .Pa char/unicode/named .
  47 .Pp
  48 Any additional arguments modify the way the tests are run.
  49 The default is
  50 .Cm all .
  51 The following modifiers are available:
  52 .Bl -tag -width markdown
  53 .It Cm all
  54 Run all kinds of subtests.
  55 This implies all other modifiers except
  56 .Cm verbose
  57 and
  58 .Cm clean .
  59 .It Cm ascii
  60 Run subtests for
  61 .Fl T Cm ascii
  62 output mode.
  63 .It Cm clean
  64 Remove all output files created by running the tests.
  65 .It Cm html
  66 Run subtests for
  67 .Fl T Cm html
  68 output mode.
  69 .It Cm lint
  70 Run subtests for
  71 .Fl T Cm lint
  72 warning and error output.
  73 .It Cm man
  74 Run subtests for
  75 .Fl T Cm man
  76 output mode.
  77 .It Cm markdown
  78 Run subtests for
  79 .Fl T Cm markdown
  80 output mode.
  81 .It Cm utf8
  82 Run subtests for
  83 .Fl T Cm utf8
  84 output mode.
  85 .It Cm verbose
  86 Display approximate indications of what is being done.
  87 .El
  88 .Sh EXIT STATUS
  89 .Ex -std
  90 .Sh EXAMPLES
  91 The recommended invocation for casual users:
  92 .Pp
  93 .Dl ./regress.pl
  94 .Pp
  95 Maximum output:
  96 .Pp
  97 .Dl ./regress.pl \&. verbose
  98 .Pp
  99 Complete check, but keep the tree clean:
 100 .Pp
 101 .Dl ./regress.pl \&. all clean
 102 .Pp
 103 Investigate a specific failure:
 104 .Pp
 105 .Dl ./regress.pl mdoc/Bd/broken man verbose
 106 .Sh HISTORY
 107 The
 108 .Nm
 109 script appeared in release 1.14.1 of the portable
 110 .Sy mandoc
 111 distribution.
 112 .Sh AUTHORS
 113 .An Ingo Schwarze Aq Mt schwarze@openbsd.org
 114 .Sh CAVEATS
 115 This script is not optimized for elegance.
 116 Regression suites for other software should not copy the design.
 117 .Pp
 118 The problem it solves is that the
 119 .Sy mandoc
 120 regression suite is tightly integrated into the regression
 121 testing system of the OpenBSD base system, which requires
 122 both OpenBSD
 123 .Xr make 1 ,
 124 working neither with POSIX make nor with GNU make, and which
 125 also requires the OpenBSD-specific Makefile fragments in
 126 .Pa /usr/share/mk .
 127 The workaround of parsing the Makefiles by hand and constructing
 128 the required command lines by hand is unavoidably messy; it's
 129 the classic no-no of parsing a language with an ad-hoc incomplete
 130 parser.
 131 But the problem of providing this regression suite for other
 132 operating systems stood unsolved for many years, and no cleaner
 133 solution was found that could be implemented with reasonable effort.
 134 So maybe this is better than nothing.
 135 .Pp
 136 The top-level Makefiles for running this regression suite on
 137 OpenBSD are not included in the portable distribution.
 138 They are too OpenBSD-specific to be useful elsewhere,
 139 and on OpenBSD itself, the suite ought be run natively from
 140 .Pa /usr/src/regress/usr.bin/mandoc
 141 and not from the portable distribution.
 142 .Pp
 143 The
 144 .Pa db
 145 subdirectory of the regression suite is not included.
 146 It uses a Makefile structure that differs vastly from the
 147 rest of the suite.