regress/regress.pl.1

   1 .\"     $Id: regress.pl.1,v 1.5 2020/03/13 15:32:31 schwarze Exp $
   2 .\"
   3 .\" Copyright (c) 2017, 2019, 2020 Ingo Schwarze <schwarze@openbsd.org>
   4 .\"
   5 .\" Permission to use, copy, modify, and distribute this software for any
   6 .\" purpose with or without fee is hereby granted, provided that the above
   7 .\" copyright notice and this permission notice appear in all copies.
   8 .\"
   9 .\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
  10 .\" WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
  11 .\" MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
  12 .\" ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
  13 .\" WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
  14 .\" ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
  15 .\" OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
  16 .\"
  17 .Dd $Mdocdate: March 13 2020 $
  18 .Dt REGRESS.PL 1
  19 .Os
  20 .Sh NAME
  21 .Nm regress.pl
  22 .Nd portable steering script for mandoc regression tests
  23 .Sh SYNOPSIS
  24 .Nm ./regress.pl
  25 .Oo
  26 .Ar directory Ns Op Pf / Ar test
  27 .Op Ar modifier ...
  28 .Oc
  29 .Sh DESCRIPTION
  30 The
  31 .Nm
  32 steering script allows running the
  33 .Xr mandoc 1
  34 regression suite on arbitrary operating systems,
  35 even though the suite was originally designed for OpenBSD only.
  36 .Pp
  37 When run without an argument,
  38 .Nm
  39 runs the complete regression suite.
  40 .Pp
  41 The first argument is a Perl regular expression to match test names,
  42 automatically anchored at the beginning of the names.
  43 Test names are names of test input files without the file name extension
  44 .Pa .in ,
  45 for example
  46 .Pa char/unicode/named .
  47 .Pp
  48 Any additional arguments modify the way the tests are run.
  49 The default is
  50 .Cm all .
  51 The following modifiers are available:
  52 .Bl -tag -width markdown
  53 .It Cm all
  54 Run all kinds of subtests.
  55 This implies all other modifiers except
  56 .Cm verbose
  57 and
  58 .Cm clean .
  59 .It Cm ascii
  60 Run subtests for
  61 .Fl T Cm ascii
  62 output mode.
  63 .It Cm clean
  64 Remove all output files created by running the tests.
  65 .It Cm html
  66 Run subtests for
  67 .Fl T Cm html
  68 output mode.
  69 .It Cm lint
  70 Run subtests for
  71 .Fl T Cm lint
  72 warning and error output.
  73 .It Cm man
  74 Run subtests for
  75 .Fl T Cm man
  76 output mode.
  77 .It Cm markdown
  78 Run subtests for
  79 .Fl T Cm markdown
  80 output mode.
  81 .It Cm tag
  82 Run subtests for automatic and manual tagging.
  83 .It Cm utf8
  84 Run subtests for
  85 .Fl T Cm utf8
  86 output mode.
  87 .It Cm verbose
  88 Display approximate indications of what is being done.
  89 .El
  90 .Sh EXIT STATUS
  91 .Ex -std
  92 .Sh EXAMPLES
  93 The recommended invocation for casual users:
  94 .Pp
  95 .Dl ./regress.pl
  96 .Pp
  97 Maximum output:
  98 .Pp
  99 .Dl ./regress.pl \&. verbose
 100 .Pp
 101 Complete check, but keep the tree clean:
 102 .Pp
 103 .Dl ./regress.pl \&. all clean
 104 .Pp
 105 Investigate a specific failure:
 106 .Pp
 107 .Dl ./regress.pl mdoc/Bd/broken man verbose
 108 .Sh HISTORY
 109 The
 110 .Nm
 111 script appeared in release 1.14.1 of the portable
 112 .Sy mandoc
 113 distribution.
 114 .Sh AUTHORS
 115 .An Ingo Schwarze Aq Mt schwarze@openbsd.org
 116 .Sh CAVEATS
 117 This script is not optimized for elegance.
 118 Regression suites for other software should not copy the design.
 119 .Pp
 120 The problem it solves is that the
 121 .Sy mandoc
 122 regression suite is tightly integrated into the regression
 123 testing system of the OpenBSD base system, which requires
 124 both OpenBSD
 125 .Xr make 1 ,
 126 working neither with POSIX make nor with GNU make, and which
 127 also requires the OpenBSD-specific Makefile fragments in
 128 .Pa /usr/share/mk .
 129 The workaround of parsing the Makefiles by hand and constructing
 130 the required command lines by hand is unavoidably messy; it's
 131 the classic no-no of parsing a language with an ad-hoc incomplete
 132 parser.
 133 But the problem of providing this regression suite for other
 134 operating systems stood unsolved for many years, and no cleaner
 135 solution was found that could be implemented with reasonable effort.
 136 So maybe this is better than nothing.
 137 .Pp
 138 The top-level Makefiles for running this regression suite on
 139 OpenBSD are not included in the portable distribution.
 140 They are too OpenBSD-specific to be useful elsewhere,
 141 and on OpenBSD itself, the suite ought be run natively from
 142 .Pa /usr/src/regress/usr.bin/mandoc
 143 and not from the portable distribution.
 144 .Pp
 145 The
 146 .Pa db
 147 subdirectory of the regression suite is not included.
 148 It uses a Makefile structure that differs vastly from the
 149 rest of the suite.
 150 .Sh BUGS
 151 The C library function
 152 .Xr wcwidth 3
 153 is known to be buggy on Solaris, which may cause failures in the
 154 regression suite, usually because output lines containing affected
 155 Unicode characters wrap too early.
 156 .Pp
 157 The regression suite does not work at all on Solaris 10 or earlier
 158 because the Perl interpreter provided with those systems is too old.