Team Ai
Datasetpublic

codekingpro/portable-devtools

sourceHugging Faceupdated 5mo agoView on Hugging Face
1likes14kdownloads
regress-run.html264 linesDownload Raw Back to html
1<?xml version="1.0" encoding="UTF-8" standalone="no"?>2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>33.1. Running the Tests</title><link rel="stylesheet" type="text/css" href="stylesheet.css" /><link rev="made" href="pgsql-docs@lists.postgresql.org" /><meta name="generator" content="DocBook XSL Stylesheets Vsnapshot" /><link rel="prev" href="regress.html" title="Chapter 33. Regression Tests" /><link rel="next" href="regress-evaluation.html" title="33.2. Test Evaluation" /></head><body id="docContent" class="container-fluid col-10"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="5" align="center">33.1. Running the Tests</th></tr><tr><td width="10%" align="left"><a accesskey="p" href="regress.html" title="Chapter 33. Regression Tests">Prev</a> </td><td width="10%" align="left"><a accesskey="u" href="regress.html" title="Chapter 33. Regression Tests">Up</a></td><th width="60%" align="center">Chapter 33. Regression Tests</th><td width="10%" align="right"><a accesskey="h" href="index.html" title="PostgreSQL 16.3 Documentation">Home</a></td><td width="10%" align="right"> <a accesskey="n" href="regress-evaluation.html" title="33.2. Test Evaluation">Next</a></td></tr></table><hr /></div><div class="sect1" id="REGRESS-RUN"><div class="titlepage"><div><div><h2 class="title" style="clear: both">33.1. Running the Tests <a href="#REGRESS-RUN" class="id_link">#</a></h2></div></div></div><div class="toc"><dl class="toc"><dt><span class="sect2"><a href="regress-run.html#REGRESS-RUN-TEMP-INST">33.1.1. Running the Tests Against a Temporary Installation</a></span></dt><dt><span class="sect2"><a href="regress-run.html#REGRESS-RUN-EXISTING-INST">33.1.2. Running the Tests Against an Existing Installation</a></span></dt><dt><span class="sect2"><a href="regress-run.html#REGRESS-ADDITIONAL">33.1.3. Additional Test Suites</a></span></dt><dt><span class="sect2"><a href="regress-run.html#REGRESS-RUN-LOCALE">33.1.4. Locale and Encoding</a></span></dt><dt><span class="sect2"><a href="regress-run.html#REGRESS-RUN-CUSTOM-SETTINGS">33.1.5. Custom Server Settings</a></span></dt><dt><span class="sect2"><a href="regress-run.html#REGRESS-RUN-EXTRA-TESTS">33.1.6. Extra Tests</a></span></dt></dl></div><p>3   The regression tests can be run against an already installed and4   running server, or using a temporary installation within the build5   tree.  Furthermore, there is a <span class="quote">“<span class="quote">parallel</span>”</span> and a6   <span class="quote">“<span class="quote">sequential</span>”</span> mode for running the tests.  The7   sequential method runs each test script alone, while the8   parallel method starts up multiple server processes to run groups9   of tests in parallel.  Parallel testing adds confidence that10   interprocess communication and locking are working correctly.11   Some tests may run sequentially even in the <span class="quote">“<span class="quote">parallel</span>”</span>12   mode in case this is required by the test.13  </p><div class="sect2" id="REGRESS-RUN-TEMP-INST"><div class="titlepage"><div><div><h3 class="title">33.1.1. Running the Tests Against a Temporary Installation <a href="#REGRESS-RUN-TEMP-INST" class="id_link">#</a></h3></div></div></div><p>14   To run the parallel regression tests after building but before installation,15   type:16</p><pre class="screen">17make check18</pre><p>19   in the top-level directory.  (Or you can change to20   <code class="filename">src/test/regress</code> and run the command there.)21   Tests which are run in parallel are prefixed with <span class="quote">“<span class="quote">+</span>”</span>, and22   tests which run sequentially are prefixed with <span class="quote">“<span class="quote">-</span>”</span>.23   At the end you should see something like:24</p><pre class="screen">25<code class="computeroutput">26# All 213 tests passed.27</code>28</pre><p>29   or otherwise a note about which tests failed.  See <a class="xref" href="regress-evaluation.html" title="33.2. Test Evaluation">Section 33.2</a> below before assuming that a30   <span class="quote">“<span class="quote">failure</span>”</span> represents a serious problem.31  </p><p>32    Because this test method runs a temporary server, it will not work33    if you did the build as the root user, since the server will not start as34    root.  Recommended procedure is not to do the build as root, or else to35    perform testing after completing the installation.36   </p><p>37    If you have configured <span class="productname">PostgreSQL</span> to install38    into a location where an older <span class="productname">PostgreSQL</span>39    installation already exists, and you perform <code class="literal">make check</code>40    before installing the new version, you might find that the tests fail41    because the new programs try to use the already-installed shared42    libraries.  (Typical symptoms are complaints about undefined symbols.)43    If you wish to run the tests before overwriting the old installation,44    you'll need to build with <code class="literal">configure --disable-rpath</code>.45    It is not recommended that you use this option for the final installation,46    however.47   </p><p>48    The parallel regression test starts quite a few processes under your49    user ID.  Presently, the maximum concurrency is twenty parallel test50    scripts, which means forty processes: there's a server process and a51    <span class="application">psql</span> process for each test script.52    So if your system enforces a per-user limit on the number of processes,53    make sure this limit is at least fifty or so, else you might get54    random-seeming failures in the parallel test.  If you are not in55    a position to raise the limit, you can cut down the degree of parallelism56    by setting the <code class="literal">MAX_CONNECTIONS</code> parameter.  For example:57</p><pre class="screen">58make MAX_CONNECTIONS=10 check59</pre><p>60    runs no more than ten tests concurrently.61   </p></div><div class="sect2" id="REGRESS-RUN-EXISTING-INST"><div class="titlepage"><div><div><h3 class="title">33.1.2. Running the Tests Against an Existing Installation <a href="#REGRESS-RUN-EXISTING-INST" class="id_link">#</a></h3></div></div></div><p>62   To run the tests after installation (see <a class="xref" href="installation.html" title="Chapter 17. Installation from Source Code">Chapter 17</a>),63   initialize a data directory and start the64   server as explained in <a class="xref" href="runtime.html" title="Chapter 19. Server Setup and Operation">Chapter 19</a>, then type:65</p><pre class="screen">66make installcheck67</pre><p>68or for a parallel test:69</p><pre class="screen">70make installcheck-parallel71</pre><p>72   The tests will expect to contact the server at the local host and the73   default port number, unless directed otherwise by <code class="envar">PGHOST</code> and74   <code class="envar">PGPORT</code> environment variables.  The tests will be run in a75   database named <code class="literal">regression</code>; any existing database by this name76   will be dropped.77  </p><p>78   The tests will also transiently create some cluster-wide objects, such as79   roles, tablespaces, and subscriptions.  These objects will have names80   beginning with <code class="literal">regress_</code>.  Beware of81   using <code class="literal">installcheck</code> mode with an installation that has82   any actual global objects named that way.83  </p></div><div class="sect2" id="REGRESS-ADDITIONAL"><div class="titlepage"><div><div><h3 class="title">33.1.3. Additional Test Suites <a href="#REGRESS-ADDITIONAL" class="id_link">#</a></h3></div></div></div><p>84   The <code class="literal">make check</code> and <code class="literal">make installcheck</code> commands85   run only the <span class="quote">“<span class="quote">core</span>”</span> regression tests, which test built-in86   functionality of the <span class="productname">PostgreSQL</span> server.  The source87   distribution contains many additional test suites, most of them having88   to do with add-on functionality such as optional procedural languages.89  </p><p>90   To run all test suites applicable to the modules that have been selected91   to be built, including the core tests, type one of these commands at the92   top of the build tree:93</p><pre class="screen">94make check-world95make installcheck-world96</pre><p>97   These commands run the tests using temporary servers or an98   already-installed server, respectively, just as previously explained99   for <code class="literal">make check</code> and <code class="literal">make installcheck</code>.  Other100   considerations are the same as previously explained for each method.101   Note that <code class="literal">make check-world</code> builds a separate instance102   (temporary data directory) for each tested module, so it requires more103   time and disk space than <code class="literal">make installcheck-world</code>.104  </p><p>105   On a modern machine with multiple CPU cores and no tight operating-system106   limits, you can make things go substantially faster with parallelism.107   The recipe that most PostgreSQL developers actually use for running all108   tests is something like109</p><pre class="screen">110make check-world -j8 &gt;/dev/null111</pre><p>112   with a <code class="option">-j</code> limit near to or a bit more than the number113   of available cores.  Discarding <span class="systemitem">stdout</span>114   eliminates chatter that's not interesting when you just want to verify115   success.  (In case of failure, the <span class="systemitem">stderr</span>116   messages are usually enough to determine where to look closer.)117  </p><p>118   Alternatively, you can run individual test suites by typing119   <code class="literal">make check</code> or <code class="literal">make installcheck</code> in the appropriate120   subdirectory of the build tree.  Keep in mind that <code class="literal">make121   installcheck</code> assumes you've installed the relevant module(s), not122   only the core server.123  </p><p>124   The additional tests that can be invoked this way include:125  </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>126     Regression tests for optional procedural languages.127     These are located under <code class="filename">src/pl</code>.128    </p></li><li class="listitem"><p>129     Regression tests for <code class="filename">contrib</code> modules,130     located under <code class="filename">contrib</code>.131     Not all <code class="filename">contrib</code> modules have tests.132    </p></li><li class="listitem"><p>133     Regression tests for the interface libraries,134     located in <code class="filename">src/interfaces/libpq/test</code> and135     <code class="filename">src/interfaces/ecpg/test</code>.136    </p></li><li class="listitem"><p>137     Tests for core-supported authentication methods,138     located in <code class="filename">src/test/authentication</code>.139     (See below for additional authentication-related tests.)140    </p></li><li class="listitem"><p>141     Tests stressing behavior of concurrent sessions,142     located in <code class="filename">src/test/isolation</code>.143    </p></li><li class="listitem"><p>144     Tests for crash recovery and physical replication,145     located in <code class="filename">src/test/recovery</code>.146    </p></li><li class="listitem"><p>147     Tests for logical replication,148     located in <code class="filename">src/test/subscription</code>.149    </p></li><li class="listitem"><p>150     Tests of client programs, located under <code class="filename">src/bin</code>.151    </p></li></ul></div><p>152   When using <code class="literal">installcheck</code> mode, these tests will create153   and destroy test databases whose names154   include <code class="literal">regression</code>, for155   example <code class="literal">pl_regression</code>156   or <code class="literal">contrib_regression</code>.  Beware of157   using <code class="literal">installcheck</code> mode with an installation that has158   any non-test databases named that way.159  </p><p>160   Some of these auxiliary test suites use the TAP infrastructure explained161   in <a class="xref" href="regress-tap.html" title="33.4. TAP Tests">Section 33.4</a>.162   The TAP-based tests are run only when PostgreSQL was configured with the163   option <code class="option">--enable-tap-tests</code>.  This is recommended for164   development, but can be omitted if there is no suitable Perl installation.165  </p><p>166   Some test suites are not run by default, either because they are not secure167   to run on a multiuser system, because they require special software or168   because they are resource intensive.  You can decide which test suites to169   run additionally by setting the <code class="command">make</code> or environment170   variable <code class="varname">PG_TEST_EXTRA</code> to a whitespace-separated list,171   for example:172</p><pre class="programlisting">173make check-world PG_TEST_EXTRA='kerberos ldap ssl load_balance'174</pre><p>175   The following values are currently supported:176   </p><div class="variablelist"><dl class="variablelist"><dt><span class="term"><code class="literal">kerberos</code></span></dt><dd><p>177       Runs the test suite under <code class="filename">src/test/kerberos</code>.  This178       requires an MIT Kerberos installation and opens TCP/IP listen sockets.179      </p></dd><dt><span class="term"><code class="literal">ldap</code></span></dt><dd><p>180       Runs the test suite under <code class="filename">src/test/ldap</code>.  This181       requires an <span class="productname">OpenLDAP</span> installation and opens182       TCP/IP listen sockets.183      </p></dd><dt><span class="term"><code class="literal">ssl</code></span></dt><dd><p>184       Runs the test suite under <code class="filename">src/test/ssl</code>.  This opens TCP/IP listen sockets.185      </p></dd><dt><span class="term"><code class="literal">load_balance</code></span></dt><dd><p>186       Runs the test <code class="filename">src/interfaces/libpq/t/004_load_balance_dns.pl</code>.187       This requires editing the system <code class="filename">hosts</code> file and188       opens TCP/IP listen sockets.189      </p></dd><dt><span class="term"><code class="literal">wal_consistency_checking</code></span></dt><dd><p>190       Uses <code class="literal">wal_consistency_checking=all</code> while running191       certain tests under <code class="filename">src/test/recovery</code>.  Not192       enabled by default because it is resource intensive.193      </p></dd></dl></div><p>194 195   Tests for features that are not supported by the current build196   configuration are not run even if they are mentioned in197   <code class="varname">PG_TEST_EXTRA</code>.198  </p><p>199   In addition, there are tests in <code class="filename">src/test/modules</code>200   which will be run by <code class="literal">make check-world</code> but not201   by <code class="literal">make installcheck-world</code>.  This is because they202   install non-production extensions or have other side-effects that are203   considered undesirable for a production installation.  You can204   use <code class="literal">make install</code> and <code class="literal">make205   installcheck</code> in one of those subdirectories if you wish,206   but it's not recommended to do so with a non-test server.207  </p></div><div class="sect2" id="REGRESS-RUN-LOCALE"><div class="titlepage"><div><div><h3 class="title">33.1.4. Locale and Encoding <a href="#REGRESS-RUN-LOCALE" class="id_link">#</a></h3></div></div></div><p>208    By default, tests using a temporary installation use the209    locale defined in the current environment and the corresponding210    database encoding as determined by <code class="command">initdb</code>.  It211    can be useful to test different locales by setting the appropriate212    environment variables, for example:213</p><pre class="screen">214make check LANG=C215make check LC_COLLATE=en_US.utf8 LC_CTYPE=fr_CA.utf8216</pre><p>217    For implementation reasons, setting <code class="envar">LC_ALL</code> does not218    work for this purpose; all the other locale-related environment219    variables do work.220   </p><p>221    When testing against an existing installation, the locale is222    determined by the existing database cluster and cannot be set223    separately for the test run.224   </p><p>225    You can also choose the database encoding explicitly by setting226    the variable <code class="envar">ENCODING</code>, for example:227</p><pre class="screen">228make check LANG=C ENCODING=EUC_JP229</pre><p>230    Setting the database encoding this way typically only makes sense231    if the locale is C; otherwise the encoding is chosen automatically232    from the locale, and specifying an encoding that does not match233    the locale will result in an error.234   </p><p>235    The database encoding can be set for tests against either a temporary or236    an existing installation, though in the latter case it must be237    compatible with the installation's locale.238   </p></div><div class="sect2" id="REGRESS-RUN-CUSTOM-SETTINGS"><div class="titlepage"><div><div><h3 class="title">33.1.5. Custom Server Settings <a href="#REGRESS-RUN-CUSTOM-SETTINGS" class="id_link">#</a></h3></div></div></div><p>239    Custom server settings to use when running a regression test suite can be240    set in the <code class="varname">PGOPTIONS</code> environment variable (for settings241    that allow this):242</p><pre class="screen">243make check PGOPTIONS="-c debug_parallel_query=regress -c work_mem=50MB"244</pre><p>245    When running against a temporary installation, custom settings can also be246    set by supplying a pre-written <code class="filename">postgresql.conf</code>:247</p><pre class="screen">248echo 'log_checkpoints = on' &gt; test_postgresql.conf249echo 'work_mem = 50MB' &gt;&gt; test_postgresql.conf250make check EXTRA_REGRESS_OPTS="--temp-config=test_postgresql.conf"251</pre><p>252   </p><p>253    This can be useful to enable additional logging, adjust resource limits,254    or enable extra run-time checks such as <a class="xref" href="runtime-config-developer.html#GUC-DEBUG-DISCARD-CACHES">debug_discard_caches</a>.255   </p></div><div class="sect2" id="REGRESS-RUN-EXTRA-TESTS"><div class="titlepage"><div><div><h3 class="title">33.1.6. Extra Tests <a href="#REGRESS-RUN-EXTRA-TESTS" class="id_link">#</a></h3></div></div></div><p>256    The core regression test suite contains a few test files that are not257    run by default, because they might be platform-dependent or take a258    very long time to run.  You can run these or other extra test259    files by setting the variable <code class="envar">EXTRA_TESTS</code>.  For260    example, to run the <code class="literal">numeric_big</code> test:261</p><pre class="screen">262make check EXTRA_TESTS=numeric_big263</pre><p>264   </p></div></div><div class="navfooter"><hr /><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="regress.html" title="Chapter 33. Regression Tests">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="regress.html" title="Chapter 33. Regression Tests">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="regress-evaluation.html" title="33.2. Test Evaluation">Next</a></td></tr><tr><td width="40%" align="left" valign="top">Chapter 33. Regression Tests </td><td width="20%" align="center"><a accesskey="h" href="index.html" title="PostgreSQL 16.3 Documentation">Home</a></td><td width="40%" align="right" valign="top"> 33.2. Test Evaluation</td></tr></table></div></body></html>
codekingpro/portable-devtools · Team Ai