codekingpro/portable-devtools

1<?xml version="1.0" encoding="UTF-8" standalone="no"?>2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>49.2. Logical Decoding Concepts</title><link rel="stylesheet" type="text/css" href="stylesheet.css" /><link rev="made" href="pgsql-docs@lists.postgresql.org" /><meta name="generator" content="DocBook XSL Stylesheets Vsnapshot" /><link rel="prev" href="logicaldecoding-example.html" title="49.1. Logical Decoding Examples" /><link rel="next" href="logicaldecoding-walsender.html" title="49.3. Streaming Replication Protocol Interface" /></head><body id="docContent" class="container-fluid col-10"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="5" align="center">49.2. Logical Decoding Concepts</th></tr><tr><td width="10%" align="left"><a accesskey="p" href="logicaldecoding-example.html" title="49.1. Logical Decoding Examples">Prev</a> </td><td width="10%" align="left"><a accesskey="u" href="logicaldecoding.html" title="Chapter 49. Logical Decoding">Up</a></td><th width="60%" align="center">Chapter 49. Logical Decoding</th><td width="10%" align="right"><a accesskey="h" href="index.html" title="PostgreSQL 16.3 Documentation">Home</a></td><td width="10%" align="right"> <a accesskey="n" href="logicaldecoding-walsender.html" title="49.3. Streaming Replication Protocol Interface">Next</a></td></tr></table><hr /></div><div class="sect1" id="LOGICALDECODING-EXPLANATION"><div class="titlepage"><div><div><h2 class="title" style="clear: both">49.2. Logical Decoding Concepts <a href="#LOGICALDECODING-EXPLANATION" class="id_link">#</a></h2></div></div></div><div class="toc"><dl class="toc"><dt><span class="sect2"><a href="logicaldecoding-explanation.html#LOGICALDECODING-EXPLANATION-LOG-DEC">49.2.1. Logical Decoding</a></span></dt><dt><span class="sect2"><a href="logicaldecoding-explanation.html#LOGICALDECODING-REPLICATION-SLOTS">49.2.2. Replication Slots</a></span></dt><dt><span class="sect2"><a href="logicaldecoding-explanation.html#LOGICALDECODING-EXPLANATION-OUTPUT-PLUGINS">49.2.3. Output Plugins</a></span></dt><dt><span class="sect2"><a href="logicaldecoding-explanation.html#LOGICALDECODING-EXPLANATION-EXPORTED-SNAPSHOTS">49.2.4. Exported Snapshots</a></span></dt></dl></div><div class="sect2" id="LOGICALDECODING-EXPLANATION-LOG-DEC"><div class="titlepage"><div><div><h3 class="title">49.2.1. Logical Decoding <a href="#LOGICALDECODING-EXPLANATION-LOG-DEC" class="id_link">#</a></h3></div></div></div><a id="id-1.8.14.8.2.2" class="indexterm"></a><p>3     Logical decoding is the process of extracting all persistent changes4     to a database's tables into a coherent, easy to understand format which5     can be interpreted without detailed knowledge of the database's internal6     state.7    </p><p>8     In <span class="productname">PostgreSQL</span>, logical decoding is implemented9     by decoding the contents of the <a class="link" href="wal.html" title="Chapter 30. Reliability and the Write-Ahead Log">write-ahead10     log</a>, which describe changes on a storage level, into an11     application-specific form such as a stream of tuples or SQL statements.12    </p></div><div class="sect2" id="LOGICALDECODING-REPLICATION-SLOTS"><div class="titlepage"><div><div><h3 class="title">49.2.2. Replication Slots <a href="#LOGICALDECODING-REPLICATION-SLOTS" class="id_link">#</a></h3></div></div></div><a id="id-1.8.14.8.3.2" class="indexterm"></a><p>13     In the context of logical replication, a slot represents a stream of14     changes that can be replayed to a client in the order they were made on15     the origin server. Each slot streams a sequence of changes from a single16     database.17    </p><div class="note"><h3 class="title">Note</h3><p><span class="productname">PostgreSQL</span> also has streaming replication slots18     (see <a class="xref" href="warm-standby.html#STREAMING-REPLICATION" title="27.2.5. Streaming Replication">Section 27.2.5</a>), but they are used somewhat19     differently there.20     </p></div><p>21     A replication slot has an identifier that is unique across all databases22     in a <span class="productname">PostgreSQL</span> cluster. Slots persist23     independently of the connection using them and are crash-safe.24    </p><p>25     A logical slot will emit each change just once in normal operation.26     The current position of each slot is persisted only at checkpoint, so in27     the case of a crash the slot may return to an earlier LSN, which will28     then cause recent changes to be sent again when the server restarts.29     Logical decoding clients are responsible for avoiding ill effects from30     handling the same message more than once.  Clients may wish to record31     the last LSN they saw when decoding and skip over any repeated data or32     (when using the replication protocol) request that decoding start from33     that LSN rather than letting the server determine the start point.34     The Replication Progress Tracking feature is designed for this purpose,35     refer to <a class="link" href="replication-origins.html" title="Chapter 50. Replication Progress Tracking">replication origins</a>.36    </p><p>37     Multiple independent slots may exist for a single database. Each slot has38     its own state, allowing different consumers to receive changes from39     different points in the database change stream. For most applications, a40     separate slot will be required for each consumer.41    </p><p>42     A logical replication slot knows nothing about the state of the43     receiver(s).  It's even possible to have multiple different receivers using44     the same slot at different times; they'll just get the changes following45     on from when the last receiver stopped consuming them. Only one receiver46     may consume changes from a slot at any given time.47    </p><p>48     A logical replication slot can also be created on a hot standby. To prevent49     <code class="command">VACUUM</code> from removing required rows from the system50     catalogs, <code class="varname">hot_standby_feedback</code> should be set on the51     standby. In spite of that, if any required rows get removed, the slot gets52     invalidated. It's highly recommended to use a physical slot between the53     primary and the standby. Otherwise, <code class="varname">hot_standby_feedback</code>54     will work but only while the connection is alive (for example a node55     restart would break it). Then, the primary may delete system catalog rows56     that could be needed by the logical decoding on the standby (as it does57     not know about the catalog_xmin on the standby). Existing logical slots58     on standby also get invalidated if <code class="varname">wal_level</code> on the59     primary is reduced to less than <code class="literal">logical</code>.60     This is done as soon as the standby detects such a change in the WAL stream.61     It means that, for walsenders which are lagging (if any), some WAL records up62     to the <code class="varname">wal_level</code> parameter change on the primary won't be63     decoded.64    </p><p>65     Creation of a logical slot requires information about all the currently66     running transactions. On the primary, this information is available67     directly, but on a standby, this information has to be obtained from68     primary. Thus, slot creation may need to wait for some activity to happen69     on the primary. If the primary is idle, creating a logical slot on70     standby may take noticeable time. This can be sped up by calling the71     <code class="function">pg_log_standby_snapshot</code> function on the primary.72    </p><div class="caution"><h3 class="title">Caution</h3><p>73      Replication slots persist across crashes and know nothing about the state74      of their consumer(s). They will prevent removal of required resources75      even when there is no connection using them. This consumes storage76      because neither required WAL nor required rows from the system catalogs77      can be removed by <code class="command">VACUUM</code> as long as they are required by a replication78      slot.  In extreme cases this could cause the database to shut down to prevent79      transaction ID wraparound (see <a class="xref" href="routine-vacuuming.html#VACUUM-FOR-WRAPAROUND" title="25.1.5. Preventing Transaction ID Wraparound Failures">Section 25.1.5</a>).80      So if a slot is no longer required it should be dropped.81     </p></div></div><div class="sect2" id="LOGICALDECODING-EXPLANATION-OUTPUT-PLUGINS"><div class="titlepage"><div><div><h3 class="title">49.2.3. Output Plugins <a href="#LOGICALDECODING-EXPLANATION-OUTPUT-PLUGINS" class="id_link">#</a></h3></div></div></div><p>82     Output plugins transform the data from the write-ahead log's internal83     representation into the format the consumer of a replication slot desires.84    </p></div><div class="sect2" id="LOGICALDECODING-EXPLANATION-EXPORTED-SNAPSHOTS"><div class="titlepage"><div><div><h3 class="title">49.2.4. Exported Snapshots <a href="#LOGICALDECODING-EXPLANATION-EXPORTED-SNAPSHOTS" class="id_link">#</a></h3></div></div></div><p>85     When a new replication slot is created using the streaming replication86     interface (see <a class="xref" href="protocol-replication.html#PROTOCOL-REPLICATION-CREATE-REPLICATION-SLOT">CREATE_REPLICATION_SLOT</a>), a87     snapshot is exported88     (see <a class="xref" href="functions-admin.html#FUNCTIONS-SNAPSHOT-SYNCHRONIZATION" title="9.27.5. Snapshot Synchronization Functions">Section 9.27.5</a>), which will show89     exactly the state of the database after which all changes will be90     included in the change stream. This can be used to create a new replica by91     using <a class="link" href="sql-set-transaction.html" title="SET TRANSACTION"><code class="literal">SET TRANSACTION92     SNAPSHOT</code></a> to read the state of the database at the moment93     the slot was created. This transaction can then be used to dump the94     database's state at that point in time, which afterwards can be updated95     using the slot's contents without losing any changes.96    </p><p>97     Creation of a snapshot is not always possible.  In particular, it will98     fail when connected to a hot standby.  Applications that do not require99     snapshot export may suppress it with the <code class="literal">NOEXPORT_SNAPSHOT</code>100     option.101    </p></div></div><div class="navfooter"><hr /><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="logicaldecoding-example.html" title="49.1. Logical Decoding Examples">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="logicaldecoding.html" title="Chapter 49. Logical Decoding">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="logicaldecoding-walsender.html" title="49.3. Streaming Replication Protocol Interface">Next</a></td></tr><tr><td width="40%" align="left" valign="top">49.1. Logical Decoding Examples </td><td width="20%" align="center"><a accesskey="h" href="index.html" title="PostgreSQL 16.3 Documentation">Home</a></td><td width="40%" align="right" valign="top"> 49.3. Streaming Replication Protocol Interface</td></tr></table></div></body></html>