codekingpro/portable-devtools
114k
1<?xml version="1.0" encoding="UTF-8" standalone="no"?>2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>33.1. Running the Tests</title><link rel="stylesheet" type="text/css" href="stylesheet.css" /><link rev="made" href="pgsql-docs@lists.postgresql.org" /><meta name="generator" content="DocBook XSL Stylesheets Vsnapshot" /><link rel="prev" href="regress.html" title="Chapter 33. Regression Tests" /><link rel="next" href="regress-evaluation.html" title="33.2. Test Evaluation" /></head><body id="docContent" class="container-fluid col-10"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="5" align="center">33.1. Running the Tests</th></tr><tr><td width="10%" align="left"><a accesskey="p" href="regress.html" title="Chapter 33. Regression Tests">Prev</a> </td><td width="10%" align="left"><a accesskey="u" href="regress.html" title="Chapter 33. Regression Tests">Up</a></td><th width="60%" align="center">Chapter 33. Regression Tests</th><td width="10%" align="right"><a accesskey="h" href="index.html" title="PostgreSQL 16.3 Documentation">Home</a></td><td width="10%" align="right"> <a accesskey="n" href="regress-evaluation.html" title="33.2. Test Evaluation">Next</a></td></tr></table><hr /></div><div class="sect1" id="REGRESS-RUN"><div class="titlepage"><div><div><h2 class="title" style="clear: both">33.1. Running the Tests <a href="#REGRESS-RUN" class="id_link">#</a></h2></div></div></div><div class="toc"><dl class="toc"><dt><span class="sect2"><a href="regress-run.html#REGRESS-RUN-TEMP-INST">33.1.1. Running the Tests Against a Temporary Installation</a></span></dt><dt><span class="sect2"><a href="regress-run.html#REGRESS-RUN-EXISTING-INST">33.1.2. Running the Tests Against an Existing Installation</a></span></dt><dt><span class="sect2"><a href="regress-run.html#REGRESS-ADDITIONAL">33.1.3. Additional Test Suites</a></span></dt><dt><span class="sect2"><a href="regress-run.html#REGRESS-RUN-LOCALE">33.1.4. Locale and Encoding</a></span></dt><dt><span class="sect2"><a href="regress-run.html#REGRESS-RUN-CUSTOM-SETTINGS">33.1.5. Custom Server Settings</a></span></dt><dt><span class="sect2"><a href="regress-run.html#REGRESS-RUN-EXTRA-TESTS">33.1.6. Extra Tests</a></span></dt></dl></div><p>3 The regression tests can be run against an already installed and4 running server, or using a temporary installation within the build5 tree. Furthermore, there is a <span class="quote">“<span class="quote">parallel</span>”</span> and a6 <span class="quote">“<span class="quote">sequential</span>”</span> mode for running the tests. The7 sequential method runs each test script alone, while the8 parallel method starts up multiple server processes to run groups9 of tests in parallel. Parallel testing adds confidence that10 interprocess communication and locking are working correctly.11 Some tests may run sequentially even in the <span class="quote">“<span class="quote">parallel</span>”</span>12 mode in case this is required by the test.13 </p><div class="sect2" id="REGRESS-RUN-TEMP-INST"><div class="titlepage"><div><div><h3 class="title">33.1.1. Running the Tests Against a Temporary Installation <a href="#REGRESS-RUN-TEMP-INST" class="id_link">#</a></h3></div></div></div><p>14 To run the parallel regression tests after building but before installation,15 type:16</p><pre class="screen">17make check18</pre><p>19 in the top-level directory. (Or you can change to20 <code class="filename">src/test/regress</code> and run the command there.)21 Tests which are run in parallel are prefixed with <span class="quote">“<span class="quote">+</span>”</span>, and22 tests which run sequentially are prefixed with <span class="quote">“<span class="quote">-</span>”</span>.23 At the end you should see something like:24</p><pre class="screen">25<code class="computeroutput">26# All 213 tests passed.27</code>28</pre><p>29 or otherwise a note about which tests failed. See <a class="xref" href="regress-evaluation.html" title="33.2. Test Evaluation">Section 33.2</a> below before assuming that a30 <span class="quote">“<span class="quote">failure</span>”</span> represents a serious problem.31 </p><p>32 Because this test method runs a temporary server, it will not work33 if you did the build as the root user, since the server will not start as34 root. Recommended procedure is not to do the build as root, or else to35 perform testing after completing the installation.36 </p><p>37 If you have configured <span class="productname">PostgreSQL</span> to install38 into a location where an older <span class="productname">PostgreSQL</span>39 installation already exists, and you perform <code class="literal">make check</code>40 before installing the new version, you might find that the tests fail41 because the new programs try to use the already-installed shared42 libraries. (Typical symptoms are complaints about undefined symbols.)43 If you wish to run the tests before overwriting the old installation,44 you'll need to build with <code class="literal">configure --disable-rpath</code>.45 It is not recommended that you use this option for the final installation,46 however.47 </p><p>48 The parallel regression test starts quite a few processes under your49 user ID. Presently, the maximum concurrency is twenty parallel test50 scripts, which means forty processes: there's a server process and a51 <span class="application">psql</span> process for each test script.52 So if your system enforces a per-user limit on the number of processes,53 make sure this limit is at least fifty or so, else you might get54 random-seeming failures in the parallel test. If you are not in55 a position to raise the limit, you can cut down the degree of parallelism56 by setting the <code class="literal">MAX_CONNECTIONS</code> parameter. For example:57</p><pre class="screen">58make MAX_CONNECTIONS=10 check59</pre><p>60 runs no more than ten tests concurrently.61 </p></div><div class="sect2" id="REGRESS-RUN-EXISTING-INST"><div class="titlepage"><div><div><h3 class="title">33.1.2. Running the Tests Against an Existing Installation <a href="#REGRESS-RUN-EXISTING-INST" class="id_link">#</a></h3></div></div></div><p>62 To run the tests after installation (see <a class="xref" href="installation.html" title="Chapter 17. Installation from Source Code">Chapter 17</a>),63 initialize a data directory and start the64 server as explained in <a class="xref" href="runtime.html" title="Chapter 19. Server Setup and Operation">Chapter 19</a>, then type:65</p><pre class="screen">66make installcheck67</pre><p>68or for a parallel test:69</p><pre class="screen">70make installcheck-parallel71</pre><p>72 The tests will expect to contact the server at the local host and the73 default port number, unless directed otherwise by <code class="envar">PGHOST</code> and74 <code class="envar">PGPORT</code> environment variables. The tests will be run in a75 database named <code class="literal">regression</code>; any existing database by this name76 will be dropped.77 </p><p>78 The tests will also transiently create some cluster-wide objects, such as79 roles, tablespaces, and subscriptions. These objects will have names80 beginning with <code class="literal">regress_</code>. Beware of81 using <code class="literal">installcheck</code> mode with an installation that has82 any actual global objects named that way.83 </p></div><div class="sect2" id="REGRESS-ADDITIONAL"><div class="titlepage"><div><div><h3 class="title">33.1.3. Additional Test Suites <a href="#REGRESS-ADDITIONAL" class="id_link">#</a></h3></div></div></div><p>84 The <code class="literal">make check</code> and <code class="literal">make installcheck</code> commands85 run only the <span class="quote">“<span class="quote">core</span>”</span> regression tests, which test built-in86 functionality of the <span class="productname">PostgreSQL</span> server. The source87 distribution contains many additional test suites, most of them having88 to do with add-on functionality such as optional procedural languages.89 </p><p>90 To run all test suites applicable to the modules that have been selected91 to be built, including the core tests, type one of these commands at the92 top of the build tree:93</p><pre class="screen">94make check-world95make installcheck-world96</pre><p>97 These commands run the tests using temporary servers or an98 already-installed server, respectively, just as previously explained99 for <code class="literal">make check</code> and <code class="literal">make installcheck</code>. Other100 considerations are the same as previously explained for each method.101 Note that <code class="literal">make check-world</code> builds a separate instance102 (temporary data directory) for each tested module, so it requires more103 time and disk space than <code class="literal">make installcheck-world</code>.104 </p><p>105 On a modern machine with multiple CPU cores and no tight operating-system106 limits, you can make things go substantially faster with parallelism.107 The recipe that most PostgreSQL developers actually use for running all108 tests is something like109</p><pre class="screen">110make check-world -j8 >/dev/null111</pre><p>112 with a <code class="option">-j</code> limit near to or a bit more than the number113 of available cores. Discarding <span class="systemitem">stdout</span>114 eliminates chatter that's not interesting when you just want to verify115 success. (In case of failure, the <span class="systemitem">stderr</span>116 messages are usually enough to determine where to look closer.)117 </p><p>118 Alternatively, you can run individual test suites by typing119 <code class="literal">make check</code> or <code class="literal">make installcheck</code> in the appropriate120 subdirectory of the build tree. Keep in mind that <code class="literal">make121 installcheck</code> assumes you've installed the relevant module(s), not122 only the core server.123 </p><p>124 The additional tests that can be invoked this way include:125 </p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>126 Regression tests for optional procedural languages.127 These are located under <code class="filename">src/pl</code>.128 </p></li><li class="listitem"><p>129 Regression tests for <code class="filename">contrib</code> modules,130 located under <code class="filename">contrib</code>.131 Not all <code class="filename">contrib</code> modules have tests.132 </p></li><li class="listitem"><p>133 Regression tests for the interface libraries,134 located in <code class="filename">src/interfaces/libpq/test</code> and135 <code class="filename">src/interfaces/ecpg/test</code>.136 </p></li><li class="listitem"><p>137 Tests for core-supported authentication methods,138 located in <code class="filename">src/test/authentication</code>.139 (See below for additional authentication-related tests.)140 </p></li><li class="listitem"><p>141 Tests stressing behavior of concurrent sessions,142 located in <code class="filename">src/test/isolation</code>.143 </p></li><li class="listitem"><p>144 Tests for crash recovery and physical replication,145 located in <code class="filename">src/test/recovery</code>.146 </p></li><li class="listitem"><p>147 Tests for logical replication,148 located in <code class="filename">src/test/subscription</code>.149 </p></li><li class="listitem"><p>150 Tests of client programs, located under <code class="filename">src/bin</code>.151 </p></li></ul></div><p>152 When using <code class="literal">installcheck</code> mode, these tests will create153 and destroy test databases whose names154 include <code class="literal">regression</code>, for155 example <code class="literal">pl_regression</code>156 or <code class="literal">contrib_regression</code>. Beware of157 using <code class="literal">installcheck</code> mode with an installation that has158 any non-test databases named that way.159 </p><p>160 Some of these auxiliary test suites use the TAP infrastructure explained161 in <a class="xref" href="regress-tap.html" title="33.4. TAP Tests">Section 33.4</a>.162 The TAP-based tests are run only when PostgreSQL was configured with the163 option <code class="option">--enable-tap-tests</code>. This is recommended for164 development, but can be omitted if there is no suitable Perl installation.165 </p><p>166 Some test suites are not run by default, either because they are not secure167 to run on a multiuser system, because they require special software or168 because they are resource intensive. You can decide which test suites to169 run additionally by setting the <code class="command">make</code> or environment170 variable <code class="varname">PG_TEST_EXTRA</code> to a whitespace-separated list,171 for example:172</p><pre class="programlisting">173make check-world PG_TEST_EXTRA='kerberos ldap ssl load_balance'174</pre><p>175 The following values are currently supported:176 </p><div class="variablelist"><dl class="variablelist"><dt><span class="term"><code class="literal">kerberos</code></span></dt><dd><p>177 Runs the test suite under <code class="filename">src/test/kerberos</code>. This178 requires an MIT Kerberos installation and opens TCP/IP listen sockets.179 </p></dd><dt><span class="term"><code class="literal">ldap</code></span></dt><dd><p>180 Runs the test suite under <code class="filename">src/test/ldap</code>. This181 requires an <span class="productname">OpenLDAP</span> installation and opens182 TCP/IP listen sockets.183 </p></dd><dt><span class="term"><code class="literal">ssl</code></span></dt><dd><p>184 Runs the test suite under <code class="filename">src/test/ssl</code>. This opens TCP/IP listen sockets.185 </p></dd><dt><span class="term"><code class="literal">load_balance</code></span></dt><dd><p>186 Runs the test <code class="filename">src/interfaces/libpq/t/004_load_balance_dns.pl</code>.187 This requires editing the system <code class="filename">hosts</code> file and188 opens TCP/IP listen sockets.189 </p></dd><dt><span class="term"><code class="literal">wal_consistency_checking</code></span></dt><dd><p>190 Uses <code class="literal">wal_consistency_checking=all</code> while running191 certain tests under <code class="filename">src/test/recovery</code>. Not192 enabled by default because it is resource intensive.193 </p></dd></dl></div><p>194 195 Tests for features that are not supported by the current build196 configuration are not run even if they are mentioned in197 <code class="varname">PG_TEST_EXTRA</code>.198 </p><p>199 In addition, there are tests in <code class="filename">src/test/modules</code>200 which will be run by <code class="literal">make check-world</code> but not201 by <code class="literal">make installcheck-world</code>. This is because they202 install non-production extensions or have other side-effects that are203 considered undesirable for a production installation. You can204 use <code class="literal">make install</code> and <code class="literal">make205 installcheck</code> in one of those subdirectories if you wish,206 but it's not recommended to do so with a non-test server.207 </p></div><div class="sect2" id="REGRESS-RUN-LOCALE"><div class="titlepage"><div><div><h3 class="title">33.1.4. Locale and Encoding <a href="#REGRESS-RUN-LOCALE" class="id_link">#</a></h3></div></div></div><p>208 By default, tests using a temporary installation use the209 locale defined in the current environment and the corresponding210 database encoding as determined by <code class="command">initdb</code>. It211 can be useful to test different locales by setting the appropriate212 environment variables, for example:213</p><pre class="screen">214make check LANG=C215make check LC_COLLATE=en_US.utf8 LC_CTYPE=fr_CA.utf8216</pre><p>217 For implementation reasons, setting <code class="envar">LC_ALL</code> does not218 work for this purpose; all the other locale-related environment219 variables do work.220 </p><p>221 When testing against an existing installation, the locale is222 determined by the existing database cluster and cannot be set223 separately for the test run.224 </p><p>225 You can also choose the database encoding explicitly by setting226 the variable <code class="envar">ENCODING</code>, for example:227</p><pre class="screen">228make check LANG=C ENCODING=EUC_JP229</pre><p>230 Setting the database encoding this way typically only makes sense231 if the locale is C; otherwise the encoding is chosen automatically232 from the locale, and specifying an encoding that does not match233 the locale will result in an error.234 </p><p>235 The database encoding can be set for tests against either a temporary or236 an existing installation, though in the latter case it must be237 compatible with the installation's locale.238 </p></div><div class="sect2" id="REGRESS-RUN-CUSTOM-SETTINGS"><div class="titlepage"><div><div><h3 class="title">33.1.5. Custom Server Settings <a href="#REGRESS-RUN-CUSTOM-SETTINGS" class="id_link">#</a></h3></div></div></div><p>239 Custom server settings to use when running a regression test suite can be240 set in the <code class="varname">PGOPTIONS</code> environment variable (for settings241 that allow this):242</p><pre class="screen">243make check PGOPTIONS="-c debug_parallel_query=regress -c work_mem=50MB"244</pre><p>245 When running against a temporary installation, custom settings can also be246 set by supplying a pre-written <code class="filename">postgresql.conf</code>:247</p><pre class="screen">248echo 'log_checkpoints = on' > test_postgresql.conf249echo 'work_mem = 50MB' >> test_postgresql.conf250make check EXTRA_REGRESS_OPTS="--temp-config=test_postgresql.conf"251</pre><p>252 </p><p>253 This can be useful to enable additional logging, adjust resource limits,254 or enable extra run-time checks such as <a class="xref" href="runtime-config-developer.html#GUC-DEBUG-DISCARD-CACHES">debug_discard_caches</a>.255 </p></div><div class="sect2" id="REGRESS-RUN-EXTRA-TESTS"><div class="titlepage"><div><div><h3 class="title">33.1.6. Extra Tests <a href="#REGRESS-RUN-EXTRA-TESTS" class="id_link">#</a></h3></div></div></div><p>256 The core regression test suite contains a few test files that are not257 run by default, because they might be platform-dependent or take a258 very long time to run. You can run these or other extra test259 files by setting the variable <code class="envar">EXTRA_TESTS</code>. For260 example, to run the <code class="literal">numeric_big</code> test:261</p><pre class="screen">262make check EXTRA_TESTS=numeric_big263</pre><p>264 </p></div></div><div class="navfooter"><hr /><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="regress.html" title="Chapter 33. Regression Tests">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="regress.html" title="Chapter 33. Regression Tests">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="regress-evaluation.html" title="33.2. Test Evaluation">Next</a></td></tr><tr><td width="40%" align="left" valign="top">Chapter 33. Regression Tests </td><td width="20%" align="center"><a accesskey="h" href="index.html" title="PostgreSQL 16.3 Documentation">Home</a></td><td width="40%" align="right" valign="top"> 33.2. Test Evaluation</td></tr></table></div></body></html>