codekingpro/portable-devtools

1<?xml version="1.0" encoding="UTF-8" standalone="no"?>2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>pg_rewind</title><link rel="stylesheet" type="text/css" href="stylesheet.css" /><link rev="made" href="pgsql-docs@lists.postgresql.org" /><meta name="generator" content="DocBook XSL Stylesheets Vsnapshot" /><link rel="prev" href="app-pgresetwal.html" title="pg_resetwal" /><link rel="next" href="pgtestfsync.html" title="pg_test_fsync" /></head><body id="docContent" class="container-fluid col-10"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="5" align="center"><span class="application">pg_rewind</span></th></tr><tr><td width="10%" align="left"><a accesskey="p" href="app-pgresetwal.html" title="pg_resetwal">Prev</a> </td><td width="10%" align="left"><a accesskey="u" href="reference-server.html" title="PostgreSQL Server Applications">Up</a></td><th width="60%" align="center">PostgreSQL Server Applications</th><td width="10%" align="right"><a accesskey="h" href="index.html" title="PostgreSQL 16.3 Documentation">Home</a></td><td width="10%" align="right"> <a accesskey="n" href="pgtestfsync.html" title="pg_test_fsync">Next</a></td></tr></table><hr /></div><div class="refentry" id="APP-PGREWIND"><div class="titlepage"></div><a id="id-1.9.5.9.1" class="indexterm"></a><div class="refnamediv"><h2><span class="refentrytitle"><span class="application">pg_rewind</span></span></h2><p>pg_rewind — synchronize a <span class="productname">PostgreSQL</span> data directory with another data directory that was forked from it</p></div><div class="refsynopsisdiv"><h2>Synopsis</h2><div class="cmdsynopsis"><p id="id-1.9.5.9.4.1"><code class="command">pg_rewind</code> [<em class="replaceable"><code>option</code></em>...]  { <code class="option">-D</code>  |   <code class="option">--target-pgdata</code> }<em class="replaceable"><code> directory</code></em> { <code class="option">--source-pgdata=<em class="replaceable"><code>directory</code></em></code>  |   <code class="option">--source-server=<em class="replaceable"><code>connstr</code></em></code> } </p></div></div><div class="refsect1" id="id-1.9.5.9.5"><h2>Description</h2><p>3   <span class="application">pg_rewind</span> is a tool for synchronizing a PostgreSQL cluster4   with another copy of the same cluster, after the clusters' timelines have5   diverged. A typical scenario is to bring an old primary server back online6   after failover as a standby that follows the new primary.7  </p><p>8   After a successful rewind, the state of the target data directory is9   analogous to a base backup of the source data directory. Unlike taking10   a new base backup or using a tool like <span class="application">rsync</span>,11   <span class="application">pg_rewind</span> does not require comparing or copying12   unchanged relation blocks in the cluster. Only changed blocks from existing13   relation files are copied; all other files, including new relation files,14   configuration files, and WAL segments, are copied in full. As such the15   rewind operation is significantly faster than other approaches when the16   database is large and only a small fraction of blocks differ between the17   clusters.18  </p><p>19   <span class="application">pg_rewind</span> examines the timeline histories of the source20   and target clusters to determine the point where they diverged, and21   expects to find WAL in the target cluster's <code class="filename">pg_wal</code> directory22   reaching all the way back to the point of divergence. The point of divergence23   can be found either on the target timeline, the source timeline, or their common24   ancestor. In the typical failover scenario where the target cluster was25   shut down soon after the divergence, this is not a problem, but if the26   target cluster ran for a long time after the divergence, its old WAL27   files might no longer be present. In this case, you can manually copy them28   from the WAL archive to the <code class="filename">pg_wal</code> directory, or run29   <span class="application">pg_rewind</span> with the <code class="literal">-c</code> option to30   automatically retrieve them from the WAL archive. The use of31   <span class="application">pg_rewind</span> is not limited to failover, e.g.,  a standby32   server can be promoted, run some write transactions, and then rewound33   to become a standby again.34  </p><p>35   After running <span class="application">pg_rewind</span>, WAL replay needs to36   complete for the data directory to be in a consistent state. When the37   target server is started again it will enter archive recovery and replay38   all WAL generated in the source server from the last checkpoint before39   the point of divergence. If some of the WAL was no longer available in the40   source server when <span class="application">pg_rewind</span> was run, and41   therefore could not be copied by the <span class="application">pg_rewind</span>42   session, it must be made available when the target server is started.43   This can be done by creating a <code class="filename">recovery.signal</code> file44   in the target data directory and by configuring a suitable45   <a class="xref" href="runtime-config-wal.html#GUC-RESTORE-COMMAND">restore_command</a> in46   <code class="filename">postgresql.conf</code>.47  </p><p>48   <span class="application">pg_rewind</span> requires that the target server either has49   the <a class="xref" href="runtime-config-wal.html#GUC-WAL-LOG-HINTS">wal_log_hints</a> option enabled50   in <code class="filename">postgresql.conf</code> or data checksums enabled when51   the cluster was initialized with <span class="application">initdb</span>.  Neither of these52   are currently on by default.  <a class="xref" href="runtime-config-wal.html#GUC-FULL-PAGE-WRITES">full_page_writes</a>53   must also be set to <code class="literal">on</code>, but is enabled by default.54  </p><div class="warning"><h3 class="title">Warning: Failures While Rewinding</h3><p>55    If <span class="application">pg_rewind</span> fails while processing, then56    the data folder of the target is likely not in a state that can be57    recovered.  In such a case, taking a new fresh backup is recommended.58   </p><p>59    As <span class="application">pg_rewind</span> copies configuration files60    entirely from the source, it may be required to correct the configuration61    used for recovery before restarting the target server, especially if62    the target is reintroduced as a standby of the source. If you restart63    the server after the rewind operation has finished but without configuring64    recovery, the target may again diverge from the primary.65   </p><p>66    <span class="application">pg_rewind</span> will fail immediately if it finds67    files it cannot write directly to.  This can happen for example when68    the source and the target server use the same file mapping for read-only69    SSL keys and certificates.  If such files are present on the target server70    it is recommended to remove them before running71    <span class="application">pg_rewind</span>.  After doing the rewind, some of72    those files may have been copied from the source, in which case it may73    be necessary to remove the data copied and restore back the set of links74    used before the rewind.75   </p></div></div><div class="refsect1" id="id-1.9.5.9.6"><h2>Options</h2><p>76    <span class="application">pg_rewind</span> accepts the following command-line77    arguments:78 79    </p><div class="variablelist"><dl class="variablelist"><dt><span class="term"><code class="option">-D <em class="replaceable"><code>directory</code></em></code><br /></span><span class="term"><code class="option">--target-pgdata=<em class="replaceable"><code>directory</code></em></code></span></dt><dd><p>80        This option specifies the target data directory that is synchronized81        with the source. The target server must be shut down cleanly before82        running <span class="application">pg_rewind</span>83       </p></dd><dt><span class="term"><code class="option">--source-pgdata=<em class="replaceable"><code>directory</code></em></code></span></dt><dd><p>84        Specifies the file system path to the data directory of the source85        server to synchronize the target with. This option requires the86        source server to be cleanly shut down.87       </p></dd><dt><span class="term"><code class="option">--source-server=<em class="replaceable"><code>connstr</code></em></code></span></dt><dd><p>88        Specifies a libpq connection string to connect to the source89        <span class="productname">PostgreSQL</span> server to synchronize the target90        with. The connection must be a normal (non-replication) connection91        with a role having sufficient permissions to execute the functions92        used by <span class="application">pg_rewind</span> on the source server93        (see Notes section for details) or a superuser role.  This option94        requires the source server to be running and accepting connections.95       </p></dd><dt><span class="term"><code class="option">-R</code><br /></span><span class="term"><code class="option">--write-recovery-conf</code></span></dt><dd><p>96        Create <code class="filename">standby.signal</code> and append connection97        settings to <code class="filename">postgresql.auto.conf</code> in the output98        directory.  <code class="literal">--source-server</code> is mandatory with99        this option.100       </p></dd><dt><span class="term"><code class="option">-n</code><br /></span><span class="term"><code class="option">--dry-run</code></span></dt><dd><p>101        Do everything except actually modifying the target directory.102       </p></dd><dt><span class="term"><code class="option">-N</code><br /></span><span class="term"><code class="option">--no-sync</code></span></dt><dd><p>103        By default, <code class="command">pg_rewind</code> will wait for all files104        to be written safely to disk.  This option causes105        <code class="command">pg_rewind</code> to return without waiting, which is106        faster, but means that a subsequent operating system crash can leave107        the data directory corrupt.  Generally, this option is useful for108        testing but should not be used on a production109        installation.110       </p></dd><dt><span class="term"><code class="option">-P</code><br /></span><span class="term"><code class="option">--progress</code></span></dt><dd><p>111        Enables progress reporting. Turning this on will deliver an approximate112        progress report while copying data from the source cluster.113       </p></dd><dt><span class="term"><code class="option">-c</code><br /></span><span class="term"><code class="option">--restore-target-wal</code></span></dt><dd><p>114        Use <code class="varname">restore_command</code> defined in the target cluster115        configuration to retrieve WAL files from the WAL archive if these116        files are no longer available in the <code class="filename">pg_wal</code>117        directory.118       </p></dd><dt><span class="term"><code class="option">--config-file=<em class="replaceable"><code>filename</code></em></code></span></dt><dd><p>119        Use the specified main server configuration file for the target120        cluster. This affects <span class="application">pg_rewind</span> when121        it uses internally the <span class="application">postgres</span> command122        for the rewind operation on this cluster (when retrieving123        <code class="varname">restore_command</code> with the option124        <code class="option">-c/--restore-target-wal</code> and when forcing a125        completion of crash recovery).126       </p></dd><dt><span class="term"><code class="option">--debug</code></span></dt><dd><p>127        Print verbose debugging output that is mostly useful for developers128        debugging <span class="application">pg_rewind</span>.129       </p></dd><dt><span class="term"><code class="option">--no-ensure-shutdown</code></span></dt><dd><p>130        <span class="application">pg_rewind</span> requires that the target server131        is cleanly shut down before rewinding. By default, if the target server132        is not shut down cleanly, <span class="application">pg_rewind</span> starts133        the target server in single-user mode to complete crash recovery first,134        and stops it.135        By passing this option, <span class="application">pg_rewind</span> skips136        this and errors out immediately if the server is not cleanly shut137        down. Users are expected to handle the situation themselves in that138        case.139       </p></dd><dt><span class="term"><code class="option">-V</code><br /></span><span class="term"><code class="option">--version</code></span></dt><dd><p>Display version information, then exit.</p></dd><dt><span class="term"><code class="option">-?</code><br /></span><span class="term"><code class="option">--help</code></span></dt><dd><p>Show help, then exit.</p></dd></dl></div><p>140   </p></div><div class="refsect1" id="id-1.9.5.9.7"><h2>Environment</h2><p>141   When <code class="option">--source-server</code> option is used,142   <span class="application">pg_rewind</span> also uses the environment variables143   supported by <span class="application">libpq</span> (see <a class="xref" href="libpq-envars.html" title="34.15. Environment Variables">Section 34.15</a>).144  </p><p>145   The environment variable <code class="envar">PG_COLOR</code> specifies whether to use146   color in diagnostic messages. Possible values are147   <code class="literal">always</code>, <code class="literal">auto</code> and148   <code class="literal">never</code>.149  </p></div><div class="refsect1" id="id-1.9.5.9.8"><h2>Notes</h2><p>150   When executing <span class="application">pg_rewind</span> using an online151   cluster as source, a role having sufficient permissions to execute the152   functions used by <span class="application">pg_rewind</span> on the source153   cluster can be used instead of a superuser.  Here is how to create such154   a role, named <code class="literal">rewind_user</code> here:155</p><pre class="programlisting">156CREATE USER rewind_user LOGIN;157GRANT EXECUTE ON function pg_catalog.pg_ls_dir(text, boolean, boolean) TO rewind_user;158GRANT EXECUTE ON function pg_catalog.pg_stat_file(text, boolean) TO rewind_user;159GRANT EXECUTE ON function pg_catalog.pg_read_binary_file(text) TO rewind_user;160GRANT EXECUTE ON function pg_catalog.pg_read_binary_file(text, bigint, bigint, boolean) TO rewind_user;161</pre><p>162  </p><div class="refsect2" id="id-1.9.5.9.8.3"><h3>How It Works</h3><p>163    The basic idea is to copy all file system-level changes from the source164    cluster to the target cluster:165   </p><div class="procedure"><ol class="procedure" type="1"><li class="step"><p>166      Scan the WAL log of the target cluster, starting from the last167      checkpoint before the point where the source cluster's timeline168      history forked off from the target cluster. For each WAL record,169      record each data block that was touched. This yields a list of all170      the data blocks that were changed in the target cluster, after the171      source cluster forked off. If some of the WAL files are no longer172      available, try re-running <span class="application">pg_rewind</span> with173      the <code class="option">-c</code> option to search for the missing files in174      the WAL archive.175     </p></li><li class="step"><p>176      Copy all those changed blocks from the source cluster to177      the target cluster, either using direct file system access178      (<code class="option">--source-pgdata</code>) or SQL (<code class="option">--source-server</code>).179      Relation files are now in a state equivalent to the moment of the last180      completed checkpoint prior to the point at which the WAL timelines of the181      source and target diverged plus the current state on the source of any182      blocks changed on the target after that divergence.183     </p></li><li class="step"><p>184      Copy all other files, including new relation files, WAL segments,185      <code class="filename">pg_xact</code>, and configuration files from the source186      cluster to the target cluster. Similarly to base backups, the contents187      of the directories <code class="filename">pg_dynshmem/</code>,188      <code class="filename">pg_notify/</code>, <code class="filename">pg_replslot/</code>,189      <code class="filename">pg_serial/</code>, <code class="filename">pg_snapshots/</code>,190      <code class="filename">pg_stat_tmp/</code>, and <code class="filename">pg_subtrans/</code>191      are omitted from the data copied from the source cluster. The files192      <code class="filename">backup_label</code>,193      <code class="filename">tablespace_map</code>,194      <code class="filename">pg_internal.init</code>,195      <code class="filename">postmaster.opts</code>,196      <code class="filename">postmaster.pid</code> and197      <code class="filename">.DS_Store</code> as well as any file or directory198      beginning with <code class="filename">pgsql_tmp</code>, are omitted.199     </p></li><li class="step"><p>200      Create a <code class="filename">backup_label</code> file to begin WAL replay at201      the checkpoint created at failover and configure the202      <code class="filename">pg_control</code> file with a minimum consistency LSN203      defined as the result of <code class="literal">pg_current_wal_insert_lsn()</code>204      when rewinding from a live source or the last checkpoint LSN when205      rewinding from a stopped source.206     </p></li><li class="step"><p>207      When starting the target, <span class="productname">PostgreSQL</span> replays208      all the required WAL, resulting in a data directory in a consistent209      state.210     </p></li></ol></div></div></div></div><div class="navfooter"><hr /><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="app-pgresetwal.html" title="pg_resetwal">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="reference-server.html" title="PostgreSQL Server Applications">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="pgtestfsync.html" title="pg_test_fsync">Next</a></td></tr><tr><td width="40%" align="left" valign="top"><span class="application">pg_resetwal</span> </td><td width="20%" align="center"><a accesskey="h" href="index.html" title="PostgreSQL 16.3 Documentation">Home</a></td><td width="40%" align="right" valign="top"> <span class="application">pg_test_fsync</span></td></tr></table></div></body></html>