parthtamu/rag-code-assistant
0
1<!DOCTYPE html>2 3<html lang="en" data-content_root="../">4 <head>5 <meta charset="utf-8" />6 <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />7<meta property="og:title" content="pickle — Python object serialization" />8<meta property="og:type" content="website" />9<meta property="og:url" content="https://docs.python.org/3/library/pickle.html" />10<meta property="og:site_name" content="Python documentation" />11<meta property="og:description" content="Source code: Lib/pickle.py The pickle module implements binary protocols for serializing and de-serializing a Python object structure. “Pickling” is the process whereby a Python object hierarchy is..." />12<meta property="og:image:width" content="1146" />13<meta property="og:image:height" content="600" />14<meta property="og:image" content="https://docs.python.org/3.15/_images/social_previews/summary_library_pickle_60b2042b.png" />15<meta property="og:image:alt" content="Source code: Lib/pickle.py The pickle module implements binary protocols for serializing and de-serializing a Python object structure. “Pickling” is the process whereby a Python object hierarchy is..." />16<meta name="description" content="Source code: Lib/pickle.py The pickle module implements binary protocols for serializing and de-serializing a Python object structure. “Pickling” is the process whereby a Python object hierarchy is..." />17<meta name="twitter:card" content="summary_large_image" />18<meta name="theme-color" content="#3776ab">19 20 <title>pickle — Python object serialization — Python 3.15.0a6 documentation</title><meta name="viewport" content="width=device-width, initial-scale=1.0">21 22 <link rel="stylesheet" type="text/css" href="../_static/pygments.css?v=b86133f3" />23 <link rel="stylesheet" type="text/css" href="../_static/classic.css?v=234b1a7c" />24 <link rel="stylesheet" type="text/css" href="../_static/pydoctheme.css?v=89a2f22a" />25 <link rel="stylesheet" type="text/css" href="../_static/profiling-sampling-visualization.css?v=0c2600ae" />26 <link id="pygments_dark_css" media="(prefers-color-scheme: dark)" rel="stylesheet" type="text/css" href="../_static/pygments_dark.css?v=5349f25f" />27 28 <script src="../_static/documentation_options.js?v=6b7c9ff5"></script>29 <script src="../_static/doctools.js?v=9bcbadda"></script>30 <script src="../_static/sphinx_highlight.js?v=dc90522c"></script>31 <script src="../_static/profiling-sampling-visualization.js?v=9811ed04"></script>32 33 <script src="../_static/sidebar.js"></script>34 35 <link rel="search" type="application/opensearchdescription+xml"36 title="Search within Python 3.15.0a6 documentation"37 href="../_static/opensearch.xml"/>38 <link rel="author" title="About these documents" href="../about.html" />39 <link rel="index" title="Index" href="../genindex.html" />40 <link rel="search" title="Search" href="../search.html" />41 <link rel="copyright" title="Copyright" href="../copyright.html" />42 <link rel="next" title="copyreg — Register pickle support functions" href="copyreg.html" />43 <link rel="prev" title="Data Persistence" href="persistence.html" />44 45 46 <script defer file-types="bz2,epub,zip" data-domain="docs.python.org" src="https://analytics.python.org/js/script.file-downloads.outbound-links.js"></script>47 48 <link rel="canonical" href="https://docs.python.org/3/library/pickle.html">49 50 51 52 53 <style>54 @media only screen {55 table.full-width-table {56 width: 100%;57 }58 }59 </style>60<link rel="stylesheet" href="../_static/pydoctheme_dark.css" media="(prefers-color-scheme: dark)" id="pydoctheme_dark_css">61 <link rel="shortcut icon" type="image/png" href="../_static/py.svg">62 <script type="text/javascript" src="../_static/copybutton.js"></script>63 <script type="text/javascript" src="../_static/menu.js"></script>64 <script type="text/javascript" src="../_static/search-focus.js"></script>65 <script type="text/javascript" src="../_static/themetoggle.js"></script> 66 <script type="text/javascript" src="../_static/rtd_switcher.js"></script>67 <meta name="readthedocs-addons-api-version" content="1">68 69 </head>70<body>71<div class="mobile-nav">72 <input type="checkbox" id="menuToggler" class="toggler__input" aria-controls="navigation"73 aria-pressed="false" aria-expanded="false" role="button" aria-label="Menu">74 <nav class="nav-content" role="navigation">75 <label for="menuToggler" class="toggler__label">76 <span></span>77 </label>78 <span class="nav-items-wrapper">79 <a href="https://www.python.org/" class="nav-logo">80 <img src="../_static/py.svg" alt="Python logo">81 </a>82 <span class="version_switcher_placeholder"></span>83 <form role="search" class="search" action="../search.html" method="get">84 <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" class="search-icon">85 <path fill-rule="nonzero" fill="currentColor" d="M15.5 14h-.79l-.28-.27a6.5 6.5 0 001.48-5.34c-.47-2.78-2.79-5-5.59-5.34a6.505 6.505 0 00-7.27 7.27c.34 2.8 2.56 5.12 5.34 5.59a6.5 6.5 0 005.34-1.48l.27.28v.79l4.25 4.25c.41.41 1.08.41 1.49 0 .41-.41.41-1.08 0-1.49L15.5 14zm-6 0C7.01 14 5 11.99 5 9.5S7.01 5 9.5 5 14 7.01 14 9.5 11.99 14 9.5 14z"></path>86 </svg>87 <input placeholder="Quick search" aria-label="Quick search" type="search" name="q">88 <input type="submit" value="Go">89 </form>90 </span>91 </nav>92 <div class="menu-wrapper">93 <nav class="menu" role="navigation" aria-label="main navigation">94 <div class="language_switcher_placeholder"></div>95 96<label class="theme-selector-label">97 Theme98 <select class="theme-selector" oninput="activateTheme(this.value)">99 <option value="auto" selected>Auto</option>100 <option value="light">Light</option>101 <option value="dark">Dark</option>102 </select>103</label>104 <div>105 <h3><a href="../contents.html">Table of Contents</a></h3>106 <ul>107<li><a class="reference internal" href="#"><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> — Python object serialization</a><ul>108<li><a class="reference internal" href="#relationship-to-other-python-modules">Relationship to other Python modules</a><ul>109<li><a class="reference internal" href="#comparison-with-marshal">Comparison with <code class="docutils literal notranslate"><span class="pre">marshal</span></code></a></li>110<li><a class="reference internal" href="#comparison-with-json">Comparison with <code class="docutils literal notranslate"><span class="pre">json</span></code></a></li>111</ul>112</li>113<li><a class="reference internal" href="#data-stream-format">Data stream format</a></li>114<li><a class="reference internal" href="#module-interface">Module Interface</a></li>115<li><a class="reference internal" href="#what-can-be-pickled-and-unpickled">What can be pickled and unpickled?</a></li>116<li><a class="reference internal" href="#pickling-class-instances">Pickling Class Instances</a><ul>117<li><a class="reference internal" href="#persistence-of-external-objects">Persistence of External Objects</a></li>118<li><a class="reference internal" href="#dispatch-tables">Dispatch Tables</a></li>119<li><a class="reference internal" href="#handling-stateful-objects">Handling Stateful Objects</a></li>120</ul>121</li>122<li><a class="reference internal" href="#custom-reduction-for-types-functions-and-other-objects">Custom Reduction for Types, Functions, and Other Objects</a></li>123<li><a class="reference internal" href="#out-of-band-buffers">Out-of-band Buffers</a><ul>124<li><a class="reference internal" href="#provider-api">Provider API</a></li>125<li><a class="reference internal" href="#consumer-api">Consumer API</a></li>126<li><a class="reference internal" href="#example">Example</a></li>127</ul>128</li>129<li><a class="reference internal" href="#restricting-globals">Restricting Globals</a></li>130<li><a class="reference internal" href="#performance">Performance</a></li>131<li><a class="reference internal" href="#examples">Examples</a></li>132<li><a class="reference internal" href="#command-line-interface">Command-line interface</a></li>133</ul>134</li>135</ul>136 137 </div>138 <div>139 <h4>Previous topic</h4>140 <p class="topless"><a href="persistence.html"141 title="previous chapter">Data Persistence</a></p>142 </div>143 <div>144 <h4>Next topic</h4>145 <p class="topless"><a href="copyreg.html"146 title="next chapter"><code class="xref py py-mod docutils literal notranslate"><span class="pre">copyreg</span></code> — Register <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> support functions</a></p>147 </div>148 <script>149 document.addEventListener('DOMContentLoaded', () => {150 const title = document.querySelector('meta[property="og:title"]').content;151 const elements = document.querySelectorAll('.improvepage');152 const pageurl = window.location.href.split('?')[0];153 elements.forEach(element => {154 const url = new URL(element.href.split('?')[0].replace("-nojs", ""));155 url.searchParams.set('pagetitle', title);156 url.searchParams.set('pageurl', pageurl);157 url.searchParams.set('pagesource', "library/pickle.rst");158 element.href = url.toString();159 });160 });161 </script>162 <div role="note" aria-label="source link">163 <h3>This page</h3>164 <ul class="this-page-menu">165 <li><a href="../bugs.html">Report a bug</a></li>166 <li><a class="improvepage" href="../improve-page-nojs.html">Improve this page</a></li>167 <li>168 <a href="https://github.com/python/cpython/blob/main/Doc/library/pickle.rst?plain=1"169 rel="nofollow">Show source170 </a>171 </li>172 173 </ul>174 </div>175 </nav>176 </div>177</div>178 179 180 <div class="related" role="navigation" aria-label="Related">181 <h3>Navigation</h3>182 <ul>183 <li class="right" style="margin-right: 10px">184 <a href="../genindex.html" title="General Index"185 accesskey="I">index</a></li>186 <li class="right" >187 <a href="../py-modindex.html" title="Python Module Index"188 >modules</a> |</li>189 <li class="right" >190 <a href="copyreg.html" title="copyreg — Register pickle support functions"191 accesskey="N">next</a> |</li>192 <li class="right" >193 <a href="persistence.html" title="Data Persistence"194 accesskey="P">previous</a> |</li>195 196 <li><img src="../_static/py.svg" alt="Python logo" style="vertical-align: middle; margin-top: -1px"></li>197 <li><a href="https://www.python.org/">Python</a> »</li>198 <li class="switchers">199 <div class="language_switcher_placeholder"></div>200 <div class="version_switcher_placeholder"></div>201 </li>202 <li>203 204 </li>205 <li id="cpython-language-and-version">206 <a href="../index.html">3.15.0a6 Documentation</a> »207 </li>208 209 <li class="nav-item nav-item-1"><a href="index.html" >The Python Standard Library</a> »</li>210 <li class="nav-item nav-item-2"><a href="persistence.html" accesskey="U">Data Persistence</a> »</li>211 <li class="nav-item nav-item-this"><a href=""><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> — Python object serialization</a></li>212 <li class="right">213 214 215 <div class="inline-search" role="search">216 <form class="inline-search" action="../search.html" method="get">217 <input placeholder="Quick search" aria-label="Quick search" type="search" name="q" id="search-box">218 <input type="submit" value="Go">219 </form>220 </div>221 |222 </li>223 <li class="right">224<label class="theme-selector-label">225 Theme226 <select class="theme-selector" oninput="activateTheme(this.value)">227 <option value="auto" selected>Auto</option>228 <option value="light">Light</option>229 <option value="dark">Dark</option>230 </select>231</label> |</li>232 233 </ul>234 </div> 235 236 <div class="document">237 <div class="documentwrapper">238 <div class="bodywrapper">239 <div class="body" role="main">240 241 <section id="module-pickle">242<span id="pickle-python-object-serialization"></span><h1><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> — Python object serialization<a class="headerlink" href="#module-pickle" title="Link to this heading">¶</a></h1>243<p><strong>Source code:</strong> <a class="extlink-source reference external" href="https://github.com/python/cpython/tree/main/Lib/pickle.py">Lib/pickle.py</a></p>244<hr class="docutils" id="index-0" />245<p>The <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> module implements binary protocols for serializing and246de-serializing a Python object structure. <em>“Pickling”</em> is the process247whereby a Python object hierarchy is converted into a byte stream, and248<em>“unpickling”</em> is the inverse operation, whereby a byte stream249(from a <a class="reference internal" href="../glossary.html#term-binary-file"><span class="xref std std-term">binary file</span></a> or <a class="reference internal" href="../glossary.html#term-bytes-like-object"><span class="xref std std-term">bytes-like object</span></a>) is converted250back into an object hierarchy. Pickling (and unpickling) is alternatively251known as “serialization”, “marshalling,” <a class="footnote-reference brackets" href="#id7" id="id1" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a> or “flattening”; however, to252avoid confusion, the terms used here are “pickling” and “unpickling”.</p>253<div class="admonition warning">254<p class="admonition-title">Warning</p>255<p>The <code class="docutils literal notranslate"><span class="pre">pickle</span></code> module <strong>is not secure</strong>. Only unpickle data you trust.</p>256<p>It is possible to construct malicious pickle data which will <strong>execute257arbitrary code during unpickling</strong>. Never unpickle data that could have come258from an untrusted source, or that could have been tampered with.</p>259<p>Consider signing data with <a class="reference internal" href="hmac.html#module-hmac" title="hmac: Keyed-Hashing for Message Authentication (HMAC) implementation"><code class="xref py py-mod docutils literal notranslate"><span class="pre">hmac</span></code></a> if you need to ensure that it has not260been tampered with.</p>261<p>Safer serialization formats such as <a class="reference internal" href="json.html#module-json" title="json: Encode and decode the JSON format."><code class="xref py py-mod docutils literal notranslate"><span class="pre">json</span></code></a> may be more appropriate if262you are processing untrusted data. See <a class="reference internal" href="#comparison-with-json"><span class="std std-ref">Comparison with json</span></a>.</p>263</div>264<section id="relationship-to-other-python-modules">265<h2>Relationship to other Python modules<a class="headerlink" href="#relationship-to-other-python-modules" title="Link to this heading">¶</a></h2>266<section id="comparison-with-marshal">267<h3>Comparison with <code class="docutils literal notranslate"><span class="pre">marshal</span></code><a class="headerlink" href="#comparison-with-marshal" title="Link to this heading">¶</a></h3>268<p>Python has a more primitive serialization module called <a class="reference internal" href="marshal.html#module-marshal" title="marshal: Convert Python objects to streams of bytes and back (with different constraints)."><code class="xref py py-mod docutils literal notranslate"><span class="pre">marshal</span></code></a>, but in269general <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> should always be the preferred way to serialize Python270objects. <code class="xref py py-mod docutils literal notranslate"><span class="pre">marshal</span></code> exists primarily to support Python’s <code class="file docutils literal notranslate"><span class="pre">.pyc</span></code>271files.</p>272<p>The <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> module differs from <a class="reference internal" href="marshal.html#module-marshal" title="marshal: Convert Python objects to streams of bytes and back (with different constraints)."><code class="xref py py-mod docutils literal notranslate"><span class="pre">marshal</span></code></a> in several significant ways:</p>273<ul class="simple">274<li><p><a class="reference internal" href="marshal.html#module-marshal" title="marshal: Convert Python objects to streams of bytes and back (with different constraints)."><code class="xref py py-mod docutils literal notranslate"><span class="pre">marshal</span></code></a> cannot be used to serialize user-defined classes and their275instances. <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> can save and restore class instances transparently,276however the class definition must be importable and live in the same module as277when the object was stored.</p></li>278<li><p>The <a class="reference internal" href="marshal.html#module-marshal" title="marshal: Convert Python objects to streams of bytes and back (with different constraints)."><code class="xref py py-mod docutils literal notranslate"><span class="pre">marshal</span></code></a> serialization format is not guaranteed to be portable279across Python versions. Because its primary job in life is to support280<code class="file docutils literal notranslate"><span class="pre">.pyc</span></code> files, the Python implementers reserve the right to change the281serialization format in non-backwards compatible ways should the need arise.282The <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> serialization format is guaranteed to be backwards compatible283across Python releases provided a compatible pickle protocol is chosen and284pickling and unpickling code deals with Python 2 to Python 3 type differences285if your data is crossing that unique breaking change language boundary.</p></li>286</ul>287</section>288<section id="comparison-with-json">289<span id="id2"></span><h3>Comparison with <code class="docutils literal notranslate"><span class="pre">json</span></code><a class="headerlink" href="#comparison-with-json" title="Link to this heading">¶</a></h3>290<p>There are fundamental differences between the pickle protocols and291<a class="reference external" href="https://json.org">JSON (JavaScript Object Notation)</a>:</p>292<ul class="simple">293<li><p>JSON is a text serialization format (it outputs unicode text, although294most of the time it is then encoded to <code class="docutils literal notranslate"><span class="pre">utf-8</span></code>), while pickle is295a binary serialization format;</p></li>296<li><p>JSON is human-readable, while pickle is not;</p></li>297<li><p>JSON is interoperable and widely used outside of the Python ecosystem,298while pickle is Python-specific;</p></li>299<li><p>JSON, by default, can only represent a subset of the Python built-in300types, and no custom classes; pickle can represent an extremely large301number of Python types (many of them automatically, by clever usage302of Python’s introspection facilities; complex cases can be tackled by303implementing <a class="reference internal" href="#pickle-inst"><span class="std std-ref">specific object APIs</span></a>);</p></li>304<li><p>Unlike pickle, deserializing untrusted JSON does not in itself create an305arbitrary code execution vulnerability.</p></li>306</ul>307<div class="admonition seealso">308<p class="admonition-title">See also</p>309<p>The <a class="reference internal" href="json.html#module-json" title="json: Encode and decode the JSON format."><code class="xref py py-mod docutils literal notranslate"><span class="pre">json</span></code></a> module: a standard library module allowing JSON310serialization and deserialization.</p>311</div>312</section>313</section>314<section id="data-stream-format">315<span id="pickle-protocols"></span><h2>Data stream format<a class="headerlink" href="#data-stream-format" title="Link to this heading">¶</a></h2>316<p id="index-1">The data format used by <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> is Python-specific. This has the317advantage that there are no restrictions imposed by external standards such as318JSON (which can’t represent pointer sharing); however it means that319non-Python programs may not be able to reconstruct pickled Python objects.</p>320<p>By default, the <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> data format uses a relatively compact binary321representation. If you need optimal size characteristics, you can efficiently322<a class="reference internal" href="archiving.html"><span class="doc">compress</span></a> pickled data.</p>323<p>The module <a class="reference internal" href="pickletools.html#module-pickletools" title="pickletools: Contains extensive comments about the pickle protocols and pickle-machine opcodes, as well as some useful functions."><code class="xref py py-mod docutils literal notranslate"><span class="pre">pickletools</span></code></a> contains tools for analyzing data streams324generated by <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code>. <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickletools</span></code> source code has extensive325comments about opcodes used by pickle protocols.</p>326<p>There are currently 6 different protocols which can be used for pickling.327The higher the protocol used, the more recent the version of Python needed328to read the pickle produced.</p>329<ul class="simple">330<li><p>Protocol version 0 is the original “human-readable” protocol and is331backwards compatible with earlier versions of Python.</p></li>332<li><p>Protocol version 1 is an old binary format which is also compatible with333earlier versions of Python.</p></li>334<li><p>Protocol version 2 was introduced in Python 2.3. It provides much more335efficient pickling of <a class="reference internal" href="../glossary.html#term-new-style-class"><span class="xref std std-term">new-style classes</span></a>. Refer to <span class="target" id="index-2"></span><a class="pep reference external" href="https://peps.python.org/pep-0307/"><strong>PEP 307</strong></a> for336information about improvements brought by protocol 2.</p></li>337<li><p>Protocol version 3 was added in Python 3.0. It has explicit support for338<a class="reference internal" href="stdtypes.html#bytes" title="bytes"><code class="xref py py-class docutils literal notranslate"><span class="pre">bytes</span></code></a> objects and cannot be unpickled by Python 2.x. This was339the default protocol in Python 3.0–3.7.</p></li>340<li><p>Protocol version 4 was added in Python 3.4. It adds support for very large341objects, pickling more kinds of objects, and some data format342optimizations. This was the default protocol in Python 3.8–3.13.343Refer to <span class="target" id="index-3"></span><a class="pep reference external" href="https://peps.python.org/pep-3154/"><strong>PEP 3154</strong></a> for information about improvements brought by344protocol 4.</p></li>345<li><p>Protocol version 5 was added in Python 3.8. It adds support for out-of-band346data and speedup for in-band data. It is the default protocol starting with347Python 3.14. Refer to <span class="target" id="index-4"></span><a class="pep reference external" href="https://peps.python.org/pep-0574/"><strong>PEP 574</strong></a> for information about improvements brought348by protocol 5.</p></li>349</ul>350<div class="admonition note">351<p class="admonition-title">Note</p>352<p>Serialization is a more primitive notion than persistence; although353<code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> reads and writes file objects, it does not handle the issue of354naming persistent objects, nor the (even more complicated) issue of concurrent355access to persistent objects. The <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> module can transform a complex356object into a byte stream and it can transform the byte stream into an object357with the same internal structure. Perhaps the most obvious thing to do with358these byte streams is to write them onto a file, but it is also conceivable to359send them across a network or store them in a database. The <a class="reference internal" href="shelve.html#module-shelve" title="shelve: Python object persistence."><code class="xref py py-mod docutils literal notranslate"><span class="pre">shelve</span></code></a>360module provides a simple interface to pickle and unpickle objects on361DBM-style database files.</p>362</div>363</section>364<section id="module-interface">365<h2>Module Interface<a class="headerlink" href="#module-interface" title="Link to this heading">¶</a></h2>366<p>To serialize an object hierarchy, you simply call the <a class="reference internal" href="#pickle.dumps" title="pickle.dumps"><code class="xref py py-func docutils literal notranslate"><span class="pre">dumps()</span></code></a> function.367Similarly, to de-serialize a data stream, you call the <a class="reference internal" href="#pickle.loads" title="pickle.loads"><code class="xref py py-func docutils literal notranslate"><span class="pre">loads()</span></code></a> function.368However, if you want more control over serialization and de-serialization,369you can create a <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a> or an <a class="reference internal" href="#pickle.Unpickler" title="pickle.Unpickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Unpickler</span></code></a> object, respectively.</p>370<p>The <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> module provides the following constants:</p>371<dl class="py data">372<dt class="sig sig-object py" id="pickle.HIGHEST_PROTOCOL">373<span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">HIGHEST_PROTOCOL</span></span><a class="headerlink" href="#pickle.HIGHEST_PROTOCOL" title="Link to this definition">¶</a></dt>374<dd><p>An integer, the highest <a class="reference internal" href="#pickle-protocols"><span class="std std-ref">protocol version</span></a>375available. This value can be passed as a <em>protocol</em> value to functions376<a class="reference internal" href="#pickle.dump" title="pickle.dump"><code class="xref py py-func docutils literal notranslate"><span class="pre">dump()</span></code></a> and <a class="reference internal" href="#pickle.dumps" title="pickle.dumps"><code class="xref py py-func docutils literal notranslate"><span class="pre">dumps()</span></code></a> as well as the <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a>377constructor.</p>378</dd></dl>379 380<dl class="py data">381<dt class="sig sig-object py" id="pickle.DEFAULT_PROTOCOL">382<span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">DEFAULT_PROTOCOL</span></span><a class="headerlink" href="#pickle.DEFAULT_PROTOCOL" title="Link to this definition">¶</a></dt>383<dd><p>An integer, the default <a class="reference internal" href="#pickle-protocols"><span class="std std-ref">protocol version</span></a> used384for pickling. May be less than <a class="reference internal" href="#pickle.HIGHEST_PROTOCOL" title="pickle.HIGHEST_PROTOCOL"><code class="xref py py-data docutils literal notranslate"><span class="pre">HIGHEST_PROTOCOL</span></code></a>. Currently the385default protocol is 5, introduced in Python 3.8 and incompatible386with previous versions. This version introduces support for out-of-band387buffers, where <span class="target" id="index-5"></span><a class="pep reference external" href="https://peps.python.org/pep-3118/"><strong>PEP 3118</strong></a>-compatible data can be transmitted separately388from the main pickle stream.</p>389<div class="versionchanged">390<p><span class="versionmodified changed">Changed in version 3.0: </span>The default protocol is 3.</p>391</div>392<div class="versionchanged">393<p><span class="versionmodified changed">Changed in version 3.8: </span>The default protocol is 4.</p>394</div>395<div class="versionchanged">396<p><span class="versionmodified changed">Changed in version 3.14: </span>The default protocol is 5.</p>397</div>398</dd></dl>399 400<p>The <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> module provides the following functions to make the pickling401process more convenient:</p>402<dl class="py function">403<dt class="sig sig-object py" id="pickle.dump">404<span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">dump</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">obj</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">file</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">protocol</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="keyword-only-separator o"><abbr title="Keyword-only parameters separator (PEP 3102)"><span class="pre">*</span></abbr></span></em>, <em class="sig-param"><span class="n"><span class="pre">fix_imports</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">buffer_callback</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.dump" title="Link to this definition">¶</a></dt>405<dd><p>Write the pickled representation of the object <em>obj</em> to the open406<a class="reference internal" href="../glossary.html#term-file-object"><span class="xref std std-term">file object</span></a> <em>file</em>. This is equivalent to407<code class="docutils literal notranslate"><span class="pre">Pickler(file,</span> <span class="pre">protocol).dump(obj)</span></code>.</p>408<p>Arguments <em>file</em>, <em>protocol</em>, <em>fix_imports</em> and <em>buffer_callback</em> have409the same meaning as in the <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a> constructor.</p>410<div class="versionchanged">411<p><span class="versionmodified changed">Changed in version 3.8: </span>The <em>buffer_callback</em> argument was added.</p>412</div>413</dd></dl>414 415<dl class="py function">416<dt class="sig sig-object py" id="pickle.dumps">417<span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">dumps</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">obj</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">protocol</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="keyword-only-separator o"><abbr title="Keyword-only parameters separator (PEP 3102)"><span class="pre">*</span></abbr></span></em>, <em class="sig-param"><span class="n"><span class="pre">fix_imports</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">buffer_callback</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.dumps" title="Link to this definition">¶</a></dt>418<dd><p>Return the pickled representation of the object <em>obj</em> as a <a class="reference internal" href="stdtypes.html#bytes" title="bytes"><code class="xref py py-class docutils literal notranslate"><span class="pre">bytes</span></code></a> object,419instead of writing it to a file.</p>420<p>Arguments <em>protocol</em>, <em>fix_imports</em> and <em>buffer_callback</em> have the same421meaning as in the <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a> constructor.</p>422<div class="versionchanged">423<p><span class="versionmodified changed">Changed in version 3.8: </span>The <em>buffer_callback</em> argument was added.</p>424</div>425</dd></dl>426 427<dl class="py function">428<dt class="sig sig-object py" id="pickle.load">429<span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">load</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">file</span></span></em>, <em class="sig-param"><span class="keyword-only-separator o"><abbr title="Keyword-only parameters separator (PEP 3102)"><span class="pre">*</span></abbr></span></em>, <em class="sig-param"><span class="n"><span class="pre">fix_imports</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">encoding</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'ASCII'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">errors</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'strict'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">buffers</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.load" title="Link to this definition">¶</a></dt>430<dd><p>Read the pickled representation of an object from the open <a class="reference internal" href="../glossary.html#term-file-object"><span class="xref std std-term">file object</span></a>431<em>file</em> and return the reconstituted object hierarchy specified therein.432This is equivalent to <code class="docutils literal notranslate"><span class="pre">Unpickler(file).load()</span></code>.</p>433<p>The protocol version of the pickle is detected automatically, so no434protocol argument is needed. Bytes past the pickled representation435of the object are ignored.</p>436<p>Arguments <em>file</em>, <em>fix_imports</em>, <em>encoding</em>, <em>errors</em>, <em>strict</em> and <em>buffers</em>437have the same meaning as in the <a class="reference internal" href="#pickle.Unpickler" title="pickle.Unpickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Unpickler</span></code></a> constructor.</p>438<div class="versionchanged">439<p><span class="versionmodified changed">Changed in version 3.8: </span>The <em>buffers</em> argument was added.</p>440</div>441</dd></dl>442 443<dl class="py function">444<dt class="sig sig-object py" id="pickle.loads">445<span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">loads</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">data</span></span></em>, <em class="sig-param"><span class="positional-only-separator o"><abbr title="Positional-only parameter separator (PEP 570)"><span class="pre">/</span></abbr></span></em>, <em class="sig-param"><span class="keyword-only-separator o"><abbr title="Keyword-only parameters separator (PEP 3102)"><span class="pre">*</span></abbr></span></em>, <em class="sig-param"><span class="n"><span class="pre">fix_imports</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">encoding</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'ASCII'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">errors</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'strict'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">buffers</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.loads" title="Link to this definition">¶</a></dt>446<dd><p>Return the reconstituted object hierarchy of the pickled representation447<em>data</em> of an object. <em>data</em> must be a <a class="reference internal" href="../glossary.html#term-bytes-like-object"><span class="xref std std-term">bytes-like object</span></a>.</p>448<p>The protocol version of the pickle is detected automatically, so no449protocol argument is needed. Bytes past the pickled representation450of the object are ignored.</p>451<p>Arguments <em>fix_imports</em>, <em>encoding</em>, <em>errors</em>, <em>strict</em> and <em>buffers</em>452have the same meaning as in the <a class="reference internal" href="#pickle.Unpickler" title="pickle.Unpickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Unpickler</span></code></a> constructor.</p>453<div class="versionchanged">454<p><span class="versionmodified changed">Changed in version 3.8: </span>The <em>buffers</em> argument was added.</p>455</div>456</dd></dl>457 458<p>The <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> module defines three exceptions:</p>459<dl class="py exception">460<dt class="sig sig-object py" id="pickle.PickleError">461<em class="property"><span class="k"><span class="pre">exception</span></span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">PickleError</span></span><a class="headerlink" href="#pickle.PickleError" title="Link to this definition">¶</a></dt>462<dd><p>Common base class for the other pickling exceptions. It inherits from463<a class="reference internal" href="exceptions.html#Exception" title="Exception"><code class="xref py py-exc docutils literal notranslate"><span class="pre">Exception</span></code></a>.</p>464</dd></dl>465 466<dl class="py exception">467<dt class="sig sig-object py" id="pickle.PicklingError">468<em class="property"><span class="k"><span class="pre">exception</span></span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">PicklingError</span></span><a class="headerlink" href="#pickle.PicklingError" title="Link to this definition">¶</a></dt>469<dd><p>Error raised when an unpicklable object is encountered by <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a>.470It inherits from <a class="reference internal" href="#pickle.PickleError" title="pickle.PickleError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">PickleError</span></code></a>.</p>471<p>Refer to <a class="reference internal" href="#pickle-picklable"><span class="std std-ref">What can be pickled and unpickled?</span></a> to learn what kinds of objects can be472pickled.</p>473</dd></dl>474 475<dl class="py exception">476<dt class="sig sig-object py" id="pickle.UnpicklingError">477<em class="property"><span class="k"><span class="pre">exception</span></span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">UnpicklingError</span></span><a class="headerlink" href="#pickle.UnpicklingError" title="Link to this definition">¶</a></dt>478<dd><p>Error raised when there is a problem unpickling an object, such as a data479corruption or a security violation. It inherits from <a class="reference internal" href="#pickle.PickleError" title="pickle.PickleError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">PickleError</span></code></a>.</p>480<p>Note that other exceptions may also be raised during unpickling, including481(but not necessarily limited to) AttributeError, EOFError, ImportError, and482IndexError.</p>483</dd></dl>484 485<p>The <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> module exports three classes, <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a>,486<a class="reference internal" href="#pickle.Unpickler" title="pickle.Unpickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Unpickler</span></code></a> and <a class="reference internal" href="#pickle.PickleBuffer" title="pickle.PickleBuffer"><code class="xref py py-class docutils literal notranslate"><span class="pre">PickleBuffer</span></code></a>:</p>487<dl class="py class">488<dt class="sig sig-object py" id="pickle.Pickler">489<em class="property"><span class="k"><span class="pre">class</span></span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">Pickler</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">file</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">protocol</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="keyword-only-separator o"><abbr title="Keyword-only parameters separator (PEP 3102)"><span class="pre">*</span></abbr></span></em>, <em class="sig-param"><span class="n"><span class="pre">fix_imports</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">buffer_callback</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Pickler" title="Link to this definition">¶</a></dt>490<dd><p>This takes a binary file for writing a pickle data stream.</p>491<p>The optional <em>protocol</em> argument, an integer, tells the pickler to use492the given protocol; supported protocols are 0 to <a class="reference internal" href="#pickle.HIGHEST_PROTOCOL" title="pickle.HIGHEST_PROTOCOL"><code class="xref py py-data docutils literal notranslate"><span class="pre">HIGHEST_PROTOCOL</span></code></a>.493If not specified, the default is <a class="reference internal" href="#pickle.DEFAULT_PROTOCOL" title="pickle.DEFAULT_PROTOCOL"><code class="xref py py-data docutils literal notranslate"><span class="pre">DEFAULT_PROTOCOL</span></code></a>. If a negative494number is specified, <code class="xref py py-data docutils literal notranslate"><span class="pre">HIGHEST_PROTOCOL</span></code> is selected.</p>495<p>The <em>file</em> argument must have a write() method that accepts a single bytes496argument. It can thus be an on-disk file opened for binary writing, an497<a class="reference internal" href="io.html#io.BytesIO" title="io.BytesIO"><code class="xref py py-class docutils literal notranslate"><span class="pre">io.BytesIO</span></code></a> instance, or any other custom object that meets this498interface.</p>499<p>If <em>fix_imports</em> is true and <em>protocol</em> is less than 3, pickle will try to500map the new Python 3 names to the old module names used in Python 2, so501that the pickle data stream is readable with Python 2.</p>502<p>If <em>buffer_callback</em> is <code class="docutils literal notranslate"><span class="pre">None</span></code> (the default), buffer views are503serialized into <em>file</em> as part of the pickle stream.</p>504<p>If <em>buffer_callback</em> is not <code class="docutils literal notranslate"><span class="pre">None</span></code>, then it can be called any number505of times with a buffer view. If the callback returns a false value506(such as <code class="docutils literal notranslate"><span class="pre">None</span></code>), the given buffer is <a class="reference internal" href="#pickle-oob"><span class="std std-ref">out-of-band</span></a>;507otherwise the buffer is serialized in-band, i.e. inside the pickle stream.</p>508<p>It is an error if <em>buffer_callback</em> is not <code class="docutils literal notranslate"><span class="pre">None</span></code> and <em>protocol</em> is509<code class="docutils literal notranslate"><span class="pre">None</span></code> or smaller than 5.</p>510<div class="versionchanged">511<p><span class="versionmodified changed">Changed in version 3.8: </span>The <em>buffer_callback</em> argument was added.</p>512</div>513<dl class="py method">514<dt class="sig sig-object py" id="pickle.Pickler.dump">515<span class="sig-name descname"><span class="pre">dump</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">obj</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Pickler.dump" title="Link to this definition">¶</a></dt>516<dd><p>Write the pickled representation of <em>obj</em> to the open file object given in517the constructor.</p>518</dd></dl>519 520<dl class="py method">521<dt class="sig sig-object py" id="pickle.Pickler.persistent_id">522<span class="sig-name descname"><span class="pre">persistent_id</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">obj</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Pickler.persistent_id" title="Link to this definition">¶</a></dt>523<dd><p>Do nothing by default. This exists so a subclass can override it.</p>524<p>If <code class="xref py py-meth docutils literal notranslate"><span class="pre">persistent_id()</span></code> returns <code class="docutils literal notranslate"><span class="pre">None</span></code>, <em>obj</em> is pickled as usual. Any525other value causes <code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code> to emit the returned value as a526persistent ID for <em>obj</em>. The meaning of this persistent ID should be527defined by <a class="reference internal" href="#pickle.Unpickler.persistent_load" title="pickle.Unpickler.persistent_load"><code class="xref py py-meth docutils literal notranslate"><span class="pre">Unpickler.persistent_load()</span></code></a>. Note that the value528returned by <code class="xref py py-meth docutils literal notranslate"><span class="pre">persistent_id()</span></code> cannot itself have a persistent ID.</p>529<p>See <a class="reference internal" href="#pickle-persistent"><span class="std std-ref">Persistence of External Objects</span></a> for details and examples of uses.</p>530<div class="versionchanged">531<p><span class="versionmodified changed">Changed in version 3.13: </span>Add the default implementation of this method in the C implementation532of <code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code>.</p>533</div>534</dd></dl>535 536<dl class="py attribute">537<dt class="sig sig-object py" id="pickle.Pickler.dispatch_table">538<span class="sig-name descname"><span class="pre">dispatch_table</span></span><a class="headerlink" href="#pickle.Pickler.dispatch_table" title="Link to this definition">¶</a></dt>539<dd><p>A pickler object’s dispatch table is a registry of <em>reduction540functions</em> of the kind which can be declared using541<a class="reference internal" href="copyreg.html#copyreg.pickle" title="copyreg.pickle"><code class="xref py py-func docutils literal notranslate"><span class="pre">copyreg.pickle()</span></code></a>. It is a mapping whose keys are classes542and whose values are reduction functions. A reduction function543takes a single argument of the associated class and should544conform to the same interface as a <a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a>545method.</p>546<p>By default, a pickler object will not have a547<a class="reference internal" href="#pickle.Pickler.dispatch_table" title="pickle.Pickler.dispatch_table"><code class="xref py py-attr docutils literal notranslate"><span class="pre">dispatch_table</span></code></a> attribute, and it will instead use the548global dispatch table managed by the <a class="reference internal" href="copyreg.html#module-copyreg" title="copyreg: Register pickle support functions."><code class="xref py py-mod docutils literal notranslate"><span class="pre">copyreg</span></code></a> module.549However, to customize the pickling for a specific pickler object550one can set the <code class="xref py py-attr docutils literal notranslate"><span class="pre">dispatch_table</span></code> attribute to a dict-like551object. Alternatively, if a subclass of <code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code> has a552<code class="xref py py-attr docutils literal notranslate"><span class="pre">dispatch_table</span></code> attribute then this will be used as the553default dispatch table for instances of that class.</p>554<p>See <a class="reference internal" href="#pickle-dispatch"><span class="std std-ref">Dispatch Tables</span></a> for usage examples.</p>555<div class="versionadded">556<p><span class="versionmodified added">Added in version 3.3.</span></p>557</div>558</dd></dl>559 560<dl class="py method">561<dt class="sig sig-object py" id="pickle.Pickler.reducer_override">562<span class="sig-name descname"><span class="pre">reducer_override</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">obj</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Pickler.reducer_override" title="Link to this definition">¶</a></dt>563<dd><p>Special reducer that can be defined in <code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code> subclasses. This564method has priority over any reducer in the <a class="reference internal" href="#pickle.Pickler.dispatch_table" title="pickle.Pickler.dispatch_table"><code class="xref py py-attr docutils literal notranslate"><span class="pre">dispatch_table</span></code></a>. It565should conform to the same interface as a <a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a> method, and566can optionally return <a class="reference internal" href="constants.html#NotImplemented" title="NotImplemented"><code class="xref py py-data docutils literal notranslate"><span class="pre">NotImplemented</span></code></a> to fallback on567<code class="xref py py-attr docutils literal notranslate"><span class="pre">dispatch_table</span></code>-registered reducers to pickle <code class="docutils literal notranslate"><span class="pre">obj</span></code>.</p>568<p>For a detailed example, see <a class="reference internal" href="#reducer-override"><span class="std std-ref">Custom Reduction for Types, Functions, and Other Objects</span></a>.</p>569<div class="versionadded">570<p><span class="versionmodified added">Added in version 3.8.</span></p>571</div>572</dd></dl>573 574<dl class="py attribute">575<dt class="sig sig-object py" id="pickle.Pickler.fast">576<span class="sig-name descname"><span class="pre">fast</span></span><a class="headerlink" href="#pickle.Pickler.fast" title="Link to this definition">¶</a></dt>577<dd><p>Deprecated. Enable fast mode if set to a true value. The fast mode578disables the usage of memo, therefore speeding the pickling process by not579generating superfluous PUT opcodes. It should not be used with580self-referential objects, doing otherwise will cause <code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code> to581recurse infinitely.</p>582<p>Use <a class="reference internal" href="pickletools.html#pickletools.optimize" title="pickletools.optimize"><code class="xref py py-func docutils literal notranslate"><span class="pre">pickletools.optimize()</span></code></a> if you need more compact pickles.</p>583</dd></dl>584 585<dl class="py method">586<dt class="sig sig-object py" id="pickle.Pickler.clear_memo">587<span class="sig-name descname"><span class="pre">clear_memo</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Pickler.clear_memo" title="Link to this definition">¶</a></dt>588<dd><p>Clears the pickler’s “memo”.</p>589<p>The memo is the data structure that remembers which objects the590pickler has already seen, so that shared or recursive objects591are pickled by reference and not by value. This method is592useful when re-using picklers.</p>593</dd></dl>594 595</dd></dl>596 597<dl class="py class">598<dt class="sig sig-object py" id="pickle.Unpickler">599<em class="property"><span class="k"><span class="pre">class</span></span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">Unpickler</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">file</span></span></em>, <em class="sig-param"><span class="keyword-only-separator o"><abbr title="Keyword-only parameters separator (PEP 3102)"><span class="pre">*</span></abbr></span></em>, <em class="sig-param"><span class="n"><span class="pre">fix_imports</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">True</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">encoding</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'ASCII'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">errors</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">'strict'</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">buffers</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Unpickler" title="Link to this definition">¶</a></dt>600<dd><p>This takes a binary file for reading a pickle data stream.</p>601<p>The protocol version of the pickle is detected automatically, so no602protocol argument is needed.</p>603<p>The argument <em>file</em> must have three methods, a read() method that takes an604integer argument, a readinto() method that takes a buffer argument605and a readline() method that requires no arguments, as in the606<a class="reference internal" href="io.html#io.BufferedIOBase" title="io.BufferedIOBase"><code class="xref py py-class docutils literal notranslate"><span class="pre">io.BufferedIOBase</span></code></a> interface. Thus <em>file</em> can be an on-disk file607opened for binary reading, an <a class="reference internal" href="io.html#io.BytesIO" title="io.BytesIO"><code class="xref py py-class docutils literal notranslate"><span class="pre">io.BytesIO</span></code></a> object, or any other608custom object that meets this interface.</p>609<p>The optional arguments <em>fix_imports</em>, <em>encoding</em> and <em>errors</em> are used610to control compatibility support for pickle stream generated by Python 2.611If <em>fix_imports</em> is true, pickle will try to map the old Python 2 names612to the new names used in Python 3. The <em>encoding</em> and <em>errors</em> tell613pickle how to decode 8-bit string instances pickled by Python 2;614these default to ‘ASCII’ and ‘strict’, respectively. The <em>encoding</em> can615be ‘bytes’ to read these 8-bit string instances as bytes objects.616Using <code class="docutils literal notranslate"><span class="pre">encoding='latin1'</span></code> is required for unpickling NumPy arrays and617instances of <a class="reference internal" href="datetime.html#datetime.datetime" title="datetime.datetime"><code class="xref py py-class docutils literal notranslate"><span class="pre">datetime</span></code></a>, <a class="reference internal" href="datetime.html#datetime.date" title="datetime.date"><code class="xref py py-class docutils literal notranslate"><span class="pre">date</span></code></a> and618<a class="reference internal" href="datetime.html#datetime.time" title="datetime.time"><code class="xref py py-class docutils literal notranslate"><span class="pre">time</span></code></a> pickled by Python 2.</p>619<p>If <em>buffers</em> is <code class="docutils literal notranslate"><span class="pre">None</span></code> (the default), then all data necessary for620deserialization must be contained in the pickle stream. This means621that the <em>buffer_callback</em> argument was <code class="docutils literal notranslate"><span class="pre">None</span></code> when a <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a>622was instantiated (or when <a class="reference internal" href="#pickle.dump" title="pickle.dump"><code class="xref py py-func docutils literal notranslate"><span class="pre">dump()</span></code></a> or <a class="reference internal" href="#pickle.dumps" title="pickle.dumps"><code class="xref py py-func docutils literal notranslate"><span class="pre">dumps()</span></code></a> was called).</p>623<p>If <em>buffers</em> is not <code class="docutils literal notranslate"><span class="pre">None</span></code>, it should be an iterable of buffer-enabled624objects that is consumed each time the pickle stream references625an <a class="reference internal" href="#pickle-oob"><span class="std std-ref">out-of-band</span></a> buffer view. Such buffers have been626given in order to the <em>buffer_callback</em> of a Pickler object.</p>627<div class="versionchanged">628<p><span class="versionmodified changed">Changed in version 3.8: </span>The <em>buffers</em> argument was added.</p>629</div>630<dl class="py method">631<dt class="sig sig-object py" id="pickle.Unpickler.load">632<span class="sig-name descname"><span class="pre">load</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Unpickler.load" title="Link to this definition">¶</a></dt>633<dd><p>Read the pickled representation of an object from the open file object634given in the constructor, and return the reconstituted object hierarchy635specified therein. Bytes past the pickled representation of the object636are ignored.</p>637</dd></dl>638 639<dl class="py method">640<dt class="sig sig-object py" id="pickle.Unpickler.persistent_load">641<span class="sig-name descname"><span class="pre">persistent_load</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">pid</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Unpickler.persistent_load" title="Link to this definition">¶</a></dt>642<dd><p>Raise an <a class="reference internal" href="#pickle.UnpicklingError" title="pickle.UnpicklingError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">UnpicklingError</span></code></a> by default.</p>643<p>If defined, <code class="xref py py-meth docutils literal notranslate"><span class="pre">persistent_load()</span></code> should return the object specified by644the persistent ID <em>pid</em>. If an invalid persistent ID is encountered, an645<a class="reference internal" href="#pickle.UnpicklingError" title="pickle.UnpicklingError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">UnpicklingError</span></code></a> should be raised.</p>646<p>See <a class="reference internal" href="#pickle-persistent"><span class="std std-ref">Persistence of External Objects</span></a> for details and examples of uses.</p>647<div class="versionchanged">648<p><span class="versionmodified changed">Changed in version 3.13: </span>Add the default implementation of this method in the C implementation649of <code class="xref py py-class docutils literal notranslate"><span class="pre">Unpickler</span></code>.</p>650</div>651</dd></dl>652 653<dl class="py method">654<dt class="sig sig-object py" id="pickle.Unpickler.find_class">655<span class="sig-name descname"><span class="pre">find_class</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">module</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">name</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.Unpickler.find_class" title="Link to this definition">¶</a></dt>656<dd><p>Import <em>module</em> if necessary and return the object called <em>name</em> from it,657where the <em>module</em> and <em>name</em> arguments are <a class="reference internal" href="stdtypes.html#str" title="str"><code class="xref py py-class docutils literal notranslate"><span class="pre">str</span></code></a> objects. Note,658unlike its name suggests, <code class="xref py py-meth docutils literal notranslate"><span class="pre">find_class()</span></code> is also used for finding659functions.</p>660<p>Subclasses may override this to gain control over what type of objects and661how they can be loaded, potentially reducing security risks. Refer to662<a class="reference internal" href="#pickle-restrict"><span class="std std-ref">Restricting Globals</span></a> for details.</p>663<p class="audit-hook">Raises an <a class="reference internal" href="sys.html#auditing"><span class="std std-ref">auditing event</span></a> <code class="docutils literal notranslate"><span class="pre">pickle.find_class</span></code> with arguments <code class="docutils literal notranslate"><span class="pre">module</span></code>, <code class="docutils literal notranslate"><span class="pre">name</span></code>.</p>664</dd></dl>665 666</dd></dl>667 668<dl class="py class">669<dt class="sig sig-object py" id="pickle.PickleBuffer">670<em class="property"><span class="k"><span class="pre">class</span></span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pickle.</span></span><span class="sig-name descname"><span class="pre">PickleBuffer</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">buffer</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pickle.PickleBuffer" title="Link to this definition">¶</a></dt>671<dd><p>A wrapper for a buffer representing picklable data. <em>buffer</em> must be a672<a class="reference internal" href="../c-api/buffer.html#bufferobjects"><span class="std std-ref">buffer-providing</span></a> object, such as a673<a class="reference internal" href="../glossary.html#term-bytes-like-object"><span class="xref std std-term">bytes-like object</span></a> or a N-dimensional array.</p>674<p><code class="xref py py-class docutils literal notranslate"><span class="pre">PickleBuffer</span></code> is itself a buffer provider, therefore it is675possible to pass it to other APIs expecting a buffer-providing object,676such as <a class="reference internal" href="stdtypes.html#memoryview" title="memoryview"><code class="xref py py-class docutils literal notranslate"><span class="pre">memoryview</span></code></a>.</p>677<p><code class="xref py py-class docutils literal notranslate"><span class="pre">PickleBuffer</span></code> objects can only be serialized using pickle678protocol 5 or higher. They are eligible for679<a class="reference internal" href="#pickle-oob"><span class="std std-ref">out-of-band serialization</span></a>.</p>680<div class="versionadded">681<p><span class="versionmodified added">Added in version 3.8.</span></p>682</div>683<dl class="py method">684<dt class="sig sig-object py" id="pickle.PickleBuffer.raw">685<span class="sig-name descname"><span class="pre">raw</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pickle.PickleBuffer.raw" title="Link to this definition">¶</a></dt>686<dd><p>Return a <a class="reference internal" href="stdtypes.html#memoryview" title="memoryview"><code class="xref py py-class docutils literal notranslate"><span class="pre">memoryview</span></code></a> of the memory area underlying this buffer.687The returned object is a one-dimensional, C-contiguous memoryview688with format <code class="docutils literal notranslate"><span class="pre">B</span></code> (unsigned bytes). <a class="reference internal" href="exceptions.html#BufferError" title="BufferError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">BufferError</span></code></a> is raised if689the buffer is neither C- nor Fortran-contiguous.</p>690</dd></dl>691 692<dl class="py method">693<dt class="sig sig-object py" id="pickle.PickleBuffer.release">694<span class="sig-name descname"><span class="pre">release</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#pickle.PickleBuffer.release" title="Link to this definition">¶</a></dt>695<dd><p>Release the underlying buffer exposed by the PickleBuffer object.</p>696</dd></dl>697 698</dd></dl>699 700</section>701<section id="what-can-be-pickled-and-unpickled">702<span id="pickle-picklable"></span><h2>What can be pickled and unpickled?<a class="headerlink" href="#what-can-be-pickled-and-unpickled" title="Link to this heading">¶</a></h2>703<p>The following types can be pickled:</p>704<ul class="simple">705<li><p>built-in constants (<code class="docutils literal notranslate"><span class="pre">None</span></code>, <code class="docutils literal notranslate"><span class="pre">True</span></code>, <code class="docutils literal notranslate"><span class="pre">False</span></code>, <code class="docutils literal notranslate"><span class="pre">Ellipsis</span></code>, and706<a class="reference internal" href="constants.html#NotImplemented" title="NotImplemented"><code class="xref py py-data docutils literal notranslate"><span class="pre">NotImplemented</span></code></a>);</p></li>707<li><p>integers, floating-point numbers, complex numbers;</p></li>708<li><p>strings, bytes, bytearrays;</p></li>709<li><p>tuples, lists, sets, and dictionaries containing only picklable objects;</p></li>710<li><p>functions (built-in and user-defined) accessible from the top level of a711module (using <a class="reference internal" href="../reference/compound_stmts.html#def"><code class="xref std std-keyword docutils literal notranslate"><span class="pre">def</span></code></a>, not <a class="reference internal" href="../reference/expressions.html#lambda"><code class="xref std std-keyword docutils literal notranslate"><span class="pre">lambda</span></code></a>);</p></li>712<li><p>classes accessible from the top level of a module;</p></li>713<li><p>instances of such classes for which the result of calling <a class="reference internal" href="#object.__getstate__" title="object.__getstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getstate__()</span></code></a>714is picklable (see section <a class="reference internal" href="#pickle-inst"><span class="std std-ref">Pickling Class Instances</span></a> for details).</p></li>715</ul>716<p>Attempts to pickle unpicklable objects will raise the <a class="reference internal" href="#pickle.PicklingError" title="pickle.PicklingError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">PicklingError</span></code></a>717exception; when this happens, an unspecified number of bytes may have already718been written to the underlying file. Trying to pickle a highly recursive data719structure may exceed the maximum recursion depth, a <a class="reference internal" href="exceptions.html#RecursionError" title="RecursionError"><code class="xref py py-exc docutils literal notranslate"><span class="pre">RecursionError</span></code></a> will be720raised in this case. You can carefully raise this limit with721<a class="reference internal" href="sys.html#sys.setrecursionlimit" title="sys.setrecursionlimit"><code class="xref py py-func docutils literal notranslate"><span class="pre">sys.setrecursionlimit()</span></code></a>.</p>722<p>Note that functions (built-in and user-defined) are pickled by fully723<a class="reference internal" href="../glossary.html#term-qualified-name"><span class="xref std std-term">qualified name</span></a>, not by value. <a class="footnote-reference brackets" href="#id8" id="id3" role="doc-noteref"><span class="fn-bracket">[</span>2<span class="fn-bracket">]</span></a> This means that only the function name is724pickled, along with the name of the containing module and classes. Neither725the function’s code, nor any of its function attributes are pickled. Thus the726defining module must be importable in the unpickling environment, and the module727must contain the named object, otherwise an exception will be raised. <a class="footnote-reference brackets" href="#id9" id="id4" role="doc-noteref"><span class="fn-bracket">[</span>3<span class="fn-bracket">]</span></a></p>728<p>Similarly, classes are pickled by fully qualified name, so the same restrictions in729the unpickling environment apply. Note that none of the class’s code or data is730pickled, so in the following example the class attribute <code class="docutils literal notranslate"><span class="pre">attr</span></code> is not731restored in the unpickling environment:</p>732<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">class</span><span class="w"> </span><span class="nc">Foo</span><span class="p">:</span>733 <span class="n">attr</span> <span class="o">=</span> <span class="s1">'A class attribute'</span>734 735<span class="n">picklestring</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">Foo</span><span class="p">)</span>736</pre></div>737</div>738<p>These restrictions are why picklable functions and classes must be defined at739the top level of a module.</p>740<p>Similarly, when class instances are pickled, their class’s code and data are not741pickled along with them. Only the instance data are pickled. This is done on742purpose, so you can fix bugs in a class or add methods to the class and still743load objects that were created with an earlier version of the class. If you744plan to have long-lived objects that will see many versions of a class, it may745be worthwhile to put a version number in the objects so that suitable746conversions can be made by the class’s <a class="reference internal" href="#object.__setstate__" title="object.__setstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code></a> method.</p>747</section>748<section id="pickling-class-instances">749<span id="pickle-inst"></span><h2>Pickling Class Instances<a class="headerlink" href="#pickling-class-instances" title="Link to this heading">¶</a></h2>750<p>In this section, we describe the general mechanisms available to you to define,751customize, and control how class instances are pickled and unpickled.</p>752<p>In most cases, no additional code is needed to make instances picklable. By753default, pickle will retrieve the class and the attributes of an instance via754introspection. When a class instance is unpickled, its <a class="reference internal" href="../reference/datamodel.html#object.__init__" title="object.__init__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__init__()</span></code></a> method755is usually <em>not</em> invoked. The default behaviour first creates an uninitialized756instance and then restores the saved attributes. The following code shows an757implementation of this behaviour:</p>758<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">save</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span>759 <span class="k">return</span> <span class="p">(</span><span class="n">obj</span><span class="o">.</span><span class="vm">__class__</span><span class="p">,</span> <span class="n">obj</span><span class="o">.</span><span class="vm">__dict__</span><span class="p">)</span>760 761<span class="k">def</span><span class="w"> </span><span class="nf">restore</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">attributes</span><span class="p">):</span>762 <span class="n">obj</span> <span class="o">=</span> <span class="bp">cls</span><span class="o">.</span><span class="fm">__new__</span><span class="p">(</span><span class="bp">cls</span><span class="p">)</span>763 <span class="n">obj</span><span class="o">.</span><span class="vm">__dict__</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">attributes</span><span class="p">)</span>764 <span class="k">return</span> <span class="n">obj</span>765</pre></div>766</div>767<p>Classes can alter the default behaviour by providing one or several special768methods:</p>769<dl class="py method">770<dt class="sig sig-object py" id="object.__getnewargs_ex__">771<span class="sig-prename descclassname"><span class="pre">object.</span></span><span class="sig-name descname"><span class="pre">__getnewargs_ex__</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#object.__getnewargs_ex__" title="Link to this definition">¶</a></dt>772<dd><p>In protocols 2 and newer, classes that implement the773<code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs_ex__()</span></code> method can dictate the values passed to the774<a class="reference internal" href="../reference/datamodel.html#object.__new__" title="object.__new__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__new__()</span></code></a> method upon unpickling. The method must return a pair775<code class="docutils literal notranslate"><span class="pre">(args,</span> <span class="pre">kwargs)</span></code> where <em>args</em> is a tuple of positional arguments776and <em>kwargs</em> a dictionary of named arguments for constructing the777object. Those will be passed to the <code class="xref py py-meth docutils literal notranslate"><span class="pre">__new__()</span></code> method upon778unpickling.</p>779<p>You should implement this method if the <a class="reference internal" href="../reference/datamodel.html#object.__new__" title="object.__new__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__new__()</span></code></a> method of your780class requires keyword-only arguments. Otherwise, it is recommended for781compatibility to implement <a class="reference internal" href="#object.__getnewargs__" title="object.__getnewargs__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs__()</span></code></a>.</p>782<div class="versionchanged">783<p><span class="versionmodified changed">Changed in version 3.6: </span><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs_ex__()</span></code> is now used in protocols 2 and 3.</p>784</div>785</dd></dl>786 787<dl class="py method">788<dt class="sig sig-object py" id="object.__getnewargs__">789<span class="sig-prename descclassname"><span class="pre">object.</span></span><span class="sig-name descname"><span class="pre">__getnewargs__</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#object.__getnewargs__" title="Link to this definition">¶</a></dt>790<dd><p>This method serves a similar purpose as <a class="reference internal" href="#object.__getnewargs_ex__" title="object.__getnewargs_ex__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs_ex__()</span></code></a>, but791supports only positional arguments. It must return a tuple of arguments792<code class="docutils literal notranslate"><span class="pre">args</span></code> which will be passed to the <a class="reference internal" href="../reference/datamodel.html#object.__new__" title="object.__new__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__new__()</span></code></a> method upon unpickling.</p>793<p><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs__()</span></code> will not be called if <a class="reference internal" href="#object.__getnewargs_ex__" title="object.__getnewargs_ex__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs_ex__()</span></code></a> is794defined.</p>795<div class="versionchanged">796<p><span class="versionmodified changed">Changed in version 3.6: </span>Before Python 3.6, <code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs__()</span></code> was called instead of797<a class="reference internal" href="#object.__getnewargs_ex__" title="object.__getnewargs_ex__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs_ex__()</span></code></a> in protocols 2 and 3.</p>798</div>799</dd></dl>800 801<dl class="py method">802<dt class="sig sig-object py" id="object.__getstate__">803<span class="sig-prename descclassname"><span class="pre">object.</span></span><span class="sig-name descname"><span class="pre">__getstate__</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#object.__getstate__" title="Link to this definition">¶</a></dt>804<dd><p>Classes can further influence how their instances are pickled by overriding805the method <code class="xref py py-meth docutils literal notranslate"><span class="pre">__getstate__()</span></code>. It is called and the returned object806is pickled as the contents for the instance, instead of a default state.807There are several cases:</p>808<ul class="simple">809<li><p>For a class that has no instance <a class="reference internal" href="../reference/datamodel.html#object.__dict__" title="object.__dict__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__dict__</span></code></a> and no810<a class="reference internal" href="../reference/datamodel.html#object.__slots__" title="object.__slots__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__slots__</span></code></a>, the default state is <code class="docutils literal notranslate"><span class="pre">None</span></code>.</p></li>811<li><p>For a class that has an instance <a class="reference internal" href="../reference/datamodel.html#object.__dict__" title="object.__dict__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__dict__</span></code></a> and no812<a class="reference internal" href="../reference/datamodel.html#object.__slots__" title="object.__slots__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__slots__</span></code></a>, the default state is <code class="docutils literal notranslate"><span class="pre">self.__dict__</span></code>.</p></li>813<li><p>For a class that has an instance <a class="reference internal" href="../reference/datamodel.html#object.__dict__" title="object.__dict__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__dict__</span></code></a> and814<a class="reference internal" href="../reference/datamodel.html#object.__slots__" title="object.__slots__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__slots__</span></code></a>, the default state is a tuple consisting of two815dictionaries: <code class="docutils literal notranslate"><span class="pre">self.__dict__</span></code>, and a dictionary mapping slot816names to slot values. Only slots that have a value are817included in the latter.</p></li>818<li><p>For a class that has <a class="reference internal" href="../reference/datamodel.html#object.__slots__" title="object.__slots__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__slots__</span></code></a> and no instance819<a class="reference internal" href="../reference/datamodel.html#object.__dict__" title="object.__dict__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__dict__</span></code></a>, the default state is a tuple whose first item820is <code class="docutils literal notranslate"><span class="pre">None</span></code> and whose second item is a dictionary mapping slot names821to slot values described in the previous bullet.</p></li>822</ul>823<div class="versionchanged">824<p><span class="versionmodified changed">Changed in version 3.11: </span>Added the default implementation of the <code class="docutils literal notranslate"><span class="pre">__getstate__()</span></code> method in the825<a class="reference internal" href="functions.html#object" title="object"><code class="xref py py-class docutils literal notranslate"><span class="pre">object</span></code></a> class.</p>826</div>827</dd></dl>828 829<dl class="py method">830<dt class="sig sig-object py" id="object.__setstate__">831<span class="sig-prename descclassname"><span class="pre">object.</span></span><span class="sig-name descname"><span class="pre">__setstate__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">state</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#object.__setstate__" title="Link to this definition">¶</a></dt>832<dd><p>Upon unpickling, if the class defines <code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code>, it is called with833the unpickled state. In that case, there is no requirement for the state834object to be a dictionary. Otherwise, the pickled state must be a dictionary835and its items are assigned to the new instance’s dictionary.</p>836<div class="admonition note">837<p class="admonition-title">Note</p>838<p>If <a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a> returns a state with value <code class="docutils literal notranslate"><span class="pre">None</span></code> at pickling,839the <code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code> method will not be called upon unpickling.</p>840</div>841</dd></dl>842 843<p>Refer to the section <a class="reference internal" href="#pickle-state"><span class="std std-ref">Handling Stateful Objects</span></a> for more information about how to use844the methods <a class="reference internal" href="#object.__getstate__" title="object.__getstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getstate__()</span></code></a> and <a class="reference internal" href="#object.__setstate__" title="object.__setstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code></a>.</p>845<div class="admonition note">846<p class="admonition-title">Note</p>847<p>At unpickling time, some methods like <a class="reference internal" href="../reference/datamodel.html#object.__getattr__" title="object.__getattr__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getattr__()</span></code></a>,848<a class="reference internal" href="../reference/datamodel.html#object.__getattribute__" title="object.__getattribute__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getattribute__()</span></code></a>, or <a class="reference internal" href="../reference/datamodel.html#object.__setattr__" title="object.__setattr__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setattr__()</span></code></a> may be called upon the849instance. In case those methods rely on some internal invariant being850true, the type should implement <a class="reference internal" href="../reference/datamodel.html#object.__new__" title="object.__new__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__new__()</span></code></a> to establish such an851invariant, as <a class="reference internal" href="../reference/datamodel.html#object.__init__" title="object.__init__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__init__()</span></code></a> is not called when unpickling an852instance.</p>853</div>854<p id="index-6">As we shall see, pickle does not use directly the methods described above. In855fact, these methods are part of the copy protocol which implements the856<a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a> special method. The copy protocol provides a unified857interface for retrieving the data necessary for pickling and copying858objects. <a class="footnote-reference brackets" href="#id10" id="id5" role="doc-noteref"><span class="fn-bracket">[</span>4<span class="fn-bracket">]</span></a></p>859<p>Although powerful, implementing <a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a> directly in your classes is860error prone. For this reason, class designers should use the high-level861interface (i.e., <a class="reference internal" href="#object.__getnewargs_ex__" title="object.__getnewargs_ex__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getnewargs_ex__()</span></code></a>, <a class="reference internal" href="#object.__getstate__" title="object.__getstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__getstate__()</span></code></a> and862<a class="reference internal" href="#object.__setstate__" title="object.__setstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code></a>) whenever possible. We will show, however, cases where863using <code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code> is the only option or leads to more efficient pickling864or both.</p>865<dl class="py method">866<dt class="sig sig-object py" id="object.__reduce__">867<span class="sig-prename descclassname"><span class="pre">object.</span></span><span class="sig-name descname"><span class="pre">__reduce__</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#object.__reduce__" title="Link to this definition">¶</a></dt>868<dd><p>The interface is currently defined as follows. The <code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code> method869takes no argument and shall return either a string or preferably a tuple (the870returned object is often referred to as the “reduce value”).</p>871<p>If a string is returned, the string should be interpreted as the name of a872global variable. It should be the object’s local name relative to its873module; the pickle module searches the module namespace to determine the874object’s module. This behaviour is typically useful for singletons.</p>875<p>When a tuple is returned, it must be between two and six items long.876Optional items can either be omitted, or <code class="docutils literal notranslate"><span class="pre">None</span></code> can be provided as their877value. The semantics of each item are in order:</p>878<ul>879<li><p>A callable object that will be called to create the initial version of the880object.</p></li>881<li><p>A tuple of arguments for the callable object. An empty tuple must be given882if the callable does not accept any argument.</p></li>883<li><p>Optionally, the object’s state, which will be passed to the object’s884<a class="reference internal" href="#object.__setstate__" title="object.__setstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code></a> method as previously described. If the object has no885such method then, the value must be a dictionary and it will be added to886the object’s <a class="reference internal" href="../reference/datamodel.html#object.__dict__" title="object.__dict__"><code class="xref py py-attr docutils literal notranslate"><span class="pre">__dict__</span></code></a> attribute.</p></li>887<li><p>Optionally, an iterator (and not a sequence) yielding successive items.888These items will be appended to the object either using889<code class="docutils literal notranslate"><span class="pre">obj.append(item)</span></code> or, in batch, using <code class="docutils literal notranslate"><span class="pre">obj.extend(list_of_items)</span></code>.890This is primarily used for list subclasses, but may be used by other891classes as long as they have <a class="reference internal" href="stdtypes.html#sequence.append" title="sequence.append"><code class="xref py py-meth docutils literal notranslate"><span class="pre">append()</span></code></a>892and <a class="reference internal" href="stdtypes.html#sequence.extend" title="sequence.extend"><code class="xref py py-meth docutils literal notranslate"><span class="pre">extend()</span></code></a> methods with893the appropriate signature. (Whether <code class="xref py py-meth docutils literal notranslate"><span class="pre">append()</span></code> or <code class="xref py py-meth docutils literal notranslate"><span class="pre">extend()</span></code> is894used depends on which pickle protocol version is used as well as the number895of items to append, so both must be supported.)</p></li>896<li><p>Optionally, an iterator (not a sequence) yielding successive key-value897pairs. These items will be stored to the object using <code class="docutils literal notranslate"><span class="pre">obj[key]</span> <span class="pre">=</span>898<span class="pre">value</span></code>. This is primarily used for dictionary subclasses, but may be used899by other classes as long as they implement <a class="reference internal" href="../reference/datamodel.html#object.__setitem__" title="object.__setitem__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setitem__()</span></code></a>.</p></li>900<li><p>Optionally, a callable with a <code class="docutils literal notranslate"><span class="pre">(obj,</span> <span class="pre">state)</span></code> signature. This901callable allows the user to programmatically control the state-updating902behavior of a specific object, instead of using <code class="docutils literal notranslate"><span class="pre">obj</span></code>’s static903<a class="reference internal" href="#object.__setstate__" title="object.__setstate__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code></a> method. If not <code class="docutils literal notranslate"><span class="pre">None</span></code>, this callable will have904priority over <code class="docutils literal notranslate"><span class="pre">obj</span></code>’s <code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code>.</p>905<div class="versionadded">906<p><span class="versionmodified added">Added in version 3.8: </span>The optional sixth tuple item, <code class="docutils literal notranslate"><span class="pre">(obj,</span> <span class="pre">state)</span></code>, was added.</p>907</div>908</li>909</ul>910</dd></dl>911 912<dl class="py method">913<dt class="sig sig-object py" id="object.__reduce_ex__">914<span class="sig-prename descclassname"><span class="pre">object.</span></span><span class="sig-name descname"><span class="pre">__reduce_ex__</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">protocol</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#object.__reduce_ex__" title="Link to this definition">¶</a></dt>915<dd><p>Alternatively, a <code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce_ex__()</span></code> method may be defined. The only916difference is this method should take a single integer argument, the protocol917version. When defined, pickle will prefer it over the <a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a>918method. In addition, <code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code> automatically becomes a synonym for919the extended version. The main use for this method is to provide920backwards-compatible reduce values for older Python releases.</p>921</dd></dl>922 923<section id="persistence-of-external-objects">924<span id="pickle-persistent"></span><h3>Persistence of External Objects<a class="headerlink" href="#persistence-of-external-objects" title="Link to this heading">¶</a></h3>925<p id="index-7">For the benefit of object persistence, the <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> module supports the926notion of a reference to an object outside the pickled data stream. Such927objects are referenced by a persistent ID, which should be either a string of928alphanumeric characters (for protocol 0) <a class="footnote-reference brackets" href="#id11" id="id6" role="doc-noteref"><span class="fn-bracket">[</span>5<span class="fn-bracket">]</span></a> or just an arbitrary object (for929any newer protocol).</p>930<p>The resolution of such persistent IDs is not defined by the <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code>931module; it will delegate this resolution to the user-defined methods on the932pickler and unpickler, <a class="reference internal" href="#pickle.Pickler.persistent_id" title="pickle.Pickler.persistent_id"><code class="xref py py-meth docutils literal notranslate"><span class="pre">persistent_id()</span></code></a> and933<a class="reference internal" href="#pickle.Unpickler.persistent_load" title="pickle.Unpickler.persistent_load"><code class="xref py py-meth docutils literal notranslate"><span class="pre">persistent_load()</span></code></a> respectively.</p>934<p>To pickle objects that have an external persistent ID, the pickler must have a935custom <a class="reference internal" href="#pickle.Pickler.persistent_id" title="pickle.Pickler.persistent_id"><code class="xref py py-meth docutils literal notranslate"><span class="pre">persistent_id()</span></code></a> method that takes an object as an936argument and returns either <code class="docutils literal notranslate"><span class="pre">None</span></code> or the persistent ID for that object.937When <code class="docutils literal notranslate"><span class="pre">None</span></code> is returned, the pickler simply pickles the object as normal.938When a persistent ID string is returned, the pickler will pickle that object,939along with a marker so that the unpickler will recognize it as a persistent ID.</p>940<p>To unpickle external objects, the unpickler must have a custom941<a class="reference internal" href="#pickle.Unpickler.persistent_load" title="pickle.Unpickler.persistent_load"><code class="xref py py-meth docutils literal notranslate"><span class="pre">persistent_load()</span></code></a> method that takes a persistent ID object and942returns the referenced object.</p>943<p>Here is a comprehensive example presenting how persistent ID can be used to944pickle external objects by reference.</p>945<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="c1"># Simple example presenting how persistent ID can be used to pickle</span>946<span class="c1"># external objects by reference.</span>947 948<span class="kn">import</span><span class="w"> </span><span class="nn">pickle</span>949<span class="kn">import</span><span class="w"> </span><span class="nn">sqlite3</span>950<span class="kn">from</span><span class="w"> </span><span class="nn">collections</span><span class="w"> </span><span class="kn">import</span> <span class="n">namedtuple</span>951 952<span class="c1"># Simple class representing a record in our database.</span>953<span class="n">MemoRecord</span> <span class="o">=</span> <span class="n">namedtuple</span><span class="p">(</span><span class="s2">"MemoRecord"</span><span class="p">,</span> <span class="s2">"key, task"</span><span class="p">)</span>954 955<span class="k">class</span><span class="w"> </span><span class="nc">DBPickler</span><span class="p">(</span><span class="n">pickle</span><span class="o">.</span><span class="n">Pickler</span><span class="p">):</span>956 957 <span class="k">def</span><span class="w"> </span><span class="nf">persistent_id</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">obj</span><span class="p">):</span>958 <span class="c1"># Instead of pickling MemoRecord as a regular class instance, we emit a</span>959 <span class="c1"># persistent ID.</span>960 <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="n">MemoRecord</span><span class="p">):</span>961 <span class="c1"># Here, our persistent ID is simply a tuple, containing a tag and a</span>962 <span class="c1"># key, which refers to a specific record in the database.</span>963 <span class="k">return</span> <span class="p">(</span><span class="s2">"MemoRecord"</span><span class="p">,</span> <span class="n">obj</span><span class="o">.</span><span class="n">key</span><span class="p">)</span>964 <span class="k">else</span><span class="p">:</span>965 <span class="c1"># If obj does not have a persistent ID, return None. This means obj</span>966 <span class="c1"># needs to be pickled as usual.</span>967 <span class="k">return</span> <span class="kc">None</span>968 969 970<span class="k">class</span><span class="w"> </span><span class="nc">DBUnpickler</span><span class="p">(</span><span class="n">pickle</span><span class="o">.</span><span class="n">Unpickler</span><span class="p">):</span>971 972 <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">file</span><span class="p">,</span> <span class="n">connection</span><span class="p">):</span>973 <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>974 <span class="bp">self</span><span class="o">.</span><span class="n">connection</span> <span class="o">=</span> <span class="n">connection</span>975 976 <span class="k">def</span><span class="w"> </span><span class="nf">persistent_load</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">pid</span><span class="p">):</span>977 <span class="c1"># This method is invoked whenever a persistent ID is encountered.</span>978 <span class="c1"># Here, pid is the tuple returned by DBPickler.</span>979 <span class="n">cursor</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">connection</span><span class="o">.</span><span class="n">cursor</span><span class="p">()</span>980 <span class="n">type_tag</span><span class="p">,</span> <span class="n">key_id</span> <span class="o">=</span> <span class="n">pid</span>981 <span class="k">if</span> <span class="n">type_tag</span> <span class="o">==</span> <span class="s2">"MemoRecord"</span><span class="p">:</span>982 <span class="c1"># Fetch the referenced record from the database and return it.</span>983 <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"SELECT * FROM memos WHERE key=?"</span><span class="p">,</span> <span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">key_id</span><span class="p">),))</span>984 <span class="n">key</span><span class="p">,</span> <span class="n">task</span> <span class="o">=</span> <span class="n">cursor</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()</span>985 <span class="k">return</span> <span class="n">MemoRecord</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">task</span><span class="p">)</span>986 <span class="k">else</span><span class="p">:</span>987 <span class="c1"># Always raises an error if you cannot return the correct object.</span>988 <span class="c1"># Otherwise, the unpickler will think None is the object referenced</span>989 <span class="c1"># by the persistent ID.</span>990 <span class="k">raise</span> <span class="n">pickle</span><span class="o">.</span><span class="n">UnpicklingError</span><span class="p">(</span><span class="s2">"unsupported persistent object"</span><span class="p">)</span>991 992 993<span class="k">def</span><span class="w"> </span><span class="nf">main</span><span class="p">():</span>994 <span class="kn">import</span><span class="w"> </span><span class="nn">io</span>995 <span class="kn">import</span><span class="w"> </span><span class="nn">pprint</span>996 997 <span class="c1"># Initialize and populate our database.</span>998 <span class="n">conn</span> <span class="o">=</span> <span class="n">sqlite3</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="s2">":memory:"</span><span class="p">)</span>999 <span class="n">cursor</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">cursor</span><span class="p">()</span>1000 <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"CREATE TABLE memos(key INTEGER PRIMARY KEY, task TEXT)"</span><span class="p">)</span>1001 <span class="n">tasks</span> <span class="o">=</span> <span class="p">(</span>1002 <span class="s1">'give food to fish'</span><span class="p">,</span>1003 <span class="s1">'prepare group meeting'</span><span class="p">,</span>1004 <span class="s1">'fight with a zebra'</span><span class="p">,</span>1005 <span class="p">)</span>1006 <span class="k">for</span> <span class="n">task</span> <span class="ow">in</span> <span class="n">tasks</span><span class="p">:</span>1007 <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"INSERT INTO memos VALUES(NULL, ?)"</span><span class="p">,</span> <span class="p">(</span><span class="n">task</span><span class="p">,))</span>1008 1009 <span class="c1"># Fetch the records to be pickled.</span>1010 <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"SELECT * FROM memos"</span><span class="p">)</span>1011 <span class="n">memos</span> <span class="o">=</span> <span class="p">[</span><span class="n">MemoRecord</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">task</span><span class="p">)</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">task</span> <span class="ow">in</span> <span class="n">cursor</span><span class="p">]</span>1012 <span class="c1"># Save the records using our custom DBPickler.</span>1013 <span class="n">file</span> <span class="o">=</span> <span class="n">io</span><span class="o">.</span><span class="n">BytesIO</span><span class="p">()</span>1014 <span class="n">DBPickler</span><span class="p">(</span><span class="n">file</span><span class="p">)</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">memos</span><span class="p">)</span>1015 1016 <span class="nb">print</span><span class="p">(</span><span class="s2">"Pickled records:"</span><span class="p">)</span>1017 <span class="n">pprint</span><span class="o">.</span><span class="n">pprint</span><span class="p">(</span><span class="n">memos</span><span class="p">)</span>1018 1019 <span class="c1"># Update a record, just for good measure.</span>1020 <span class="n">cursor</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">"UPDATE memos SET task='learn italian' WHERE key=1"</span><span class="p">)</span>1021 1022 <span class="c1"># Load the records from the pickle data stream.</span>1023 <span class="n">file</span><span class="o">.</span><span class="n">seek</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>1024 <span class="n">memos</span> <span class="o">=</span> <span class="n">DBUnpickler</span><span class="p">(</span><span class="n">file</span><span class="p">,</span> <span class="n">conn</span><span class="p">)</span><span class="o">.</span><span class="n">load</span><span class="p">()</span>1025 1026 <span class="nb">print</span><span class="p">(</span><span class="s2">"Unpickled records:"</span><span class="p">)</span>1027 <span class="n">pprint</span><span class="o">.</span><span class="n">pprint</span><span class="p">(</span><span class="n">memos</span><span class="p">)</span>1028 1029 1030<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">'__main__'</span><span class="p">:</span>1031 <span class="n">main</span><span class="p">()</span>1032</pre></div>1033</div>1034</section>1035<section id="dispatch-tables">1036<span id="pickle-dispatch"></span><h3>Dispatch Tables<a class="headerlink" href="#dispatch-tables" title="Link to this heading">¶</a></h3>1037<p>If one wants to customize pickling of some classes without disturbing1038any other code which depends on pickling, then one can create a1039pickler with a private dispatch table.</p>1040<p>The global dispatch table managed by the <a class="reference internal" href="copyreg.html#module-copyreg" title="copyreg: Register pickle support functions."><code class="xref py py-mod docutils literal notranslate"><span class="pre">copyreg</span></code></a> module is1041available as <code class="xref py py-data docutils literal notranslate"><span class="pre">copyreg.dispatch_table</span></code>. Therefore, one may1042choose to use a modified copy of <code class="xref py py-data docutils literal notranslate"><span class="pre">copyreg.dispatch_table</span></code> as a1043private dispatch table.</p>1044<p>For example</p>1045<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="n">f</span> <span class="o">=</span> <span class="n">io</span><span class="o">.</span><span class="n">BytesIO</span><span class="p">()</span>1046<span class="n">p</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">Pickler</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>1047<span class="n">p</span><span class="o">.</span><span class="n">dispatch_table</span> <span class="o">=</span> <span class="n">copyreg</span><span class="o">.</span><span class="n">dispatch_table</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>1048<span class="n">p</span><span class="o">.</span><span class="n">dispatch_table</span><span class="p">[</span><span class="n">SomeClass</span><span class="p">]</span> <span class="o">=</span> <span class="n">reduce_SomeClass</span>1049</pre></div>1050</div>1051<p>creates an instance of <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">pickle.Pickler</span></code></a> with a private dispatch1052table which handles the <code class="docutils literal notranslate"><span class="pre">SomeClass</span></code> class specially. Alternatively,1053the code</p>1054<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">class</span><span class="w"> </span><span class="nc">MyPickler</span><span class="p">(</span><span class="n">pickle</span><span class="o">.</span><span class="n">Pickler</span><span class="p">):</span>1055 <span class="n">dispatch_table</span> <span class="o">=</span> <span class="n">copyreg</span><span class="o">.</span><span class="n">dispatch_table</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>1056 <span class="n">dispatch_table</span><span class="p">[</span><span class="n">SomeClass</span><span class="p">]</span> <span class="o">=</span> <span class="n">reduce_SomeClass</span>1057<span class="n">f</span> <span class="o">=</span> <span class="n">io</span><span class="o">.</span><span class="n">BytesIO</span><span class="p">()</span>1058<span class="n">p</span> <span class="o">=</span> <span class="n">MyPickler</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>1059</pre></div>1060</div>1061<p>does the same but all instances of <code class="docutils literal notranslate"><span class="pre">MyPickler</span></code> will by default1062share the private dispatch table. On the other hand, the code</p>1063<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="n">copyreg</span><span class="o">.</span><span class="n">pickle</span><span class="p">(</span><span class="n">SomeClass</span><span class="p">,</span> <span class="n">reduce_SomeClass</span><span class="p">)</span>1064<span class="n">f</span> <span class="o">=</span> <span class="n">io</span><span class="o">.</span><span class="n">BytesIO</span><span class="p">()</span>1065<span class="n">p</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">Pickler</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>1066</pre></div>1067</div>1068<p>modifies the global dispatch table shared by all users of the <a class="reference internal" href="copyreg.html#module-copyreg" title="copyreg: Register pickle support functions."><code class="xref py py-mod docutils literal notranslate"><span class="pre">copyreg</span></code></a> module.</p>1069</section>1070<section id="handling-stateful-objects">1071<span id="pickle-state"></span><h3>Handling Stateful Objects<a class="headerlink" href="#handling-stateful-objects" title="Link to this heading">¶</a></h3>1072<p id="index-8">Here’s an example that shows how to modify pickling behavior for a class.1073The <code class="xref py py-class docutils literal notranslate"><span class="pre">TextReader</span></code> class below opens a text file, and returns the line number and1074line contents each time its <code class="xref py py-meth docutils literal notranslate"><span class="pre">readline()</span></code> method is called. If a1075<code class="xref py py-class docutils literal notranslate"><span class="pre">TextReader</span></code> instance is pickled, all attributes <em>except</em> the file object1076member are saved. When the instance is unpickled, the file is reopened, and1077reading resumes from the last location. The <code class="xref py py-meth docutils literal notranslate"><span class="pre">__setstate__()</span></code> and1078<code class="xref py py-meth docutils literal notranslate"><span class="pre">__getstate__()</span></code> methods are used to implement this behavior.</p>1079<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="k">class</span><span class="w"> </span><span class="nc">TextReader</span><span class="p">:</span>1080<span class="w"> </span><span class="sd">"""Print and number lines in a text file."""</span>1081 1082 <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">filename</span><span class="p">):</span>1083 <span class="bp">self</span><span class="o">.</span><span class="n">filename</span> <span class="o">=</span> <span class="n">filename</span>1084 <span class="bp">self</span><span class="o">.</span><span class="n">file</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">filename</span><span class="p">)</span>1085 <span class="bp">self</span><span class="o">.</span><span class="n">lineno</span> <span class="o">=</span> <span class="mi">0</span>1086 1087 <span class="k">def</span><span class="w"> </span><span class="nf">readline</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>1088 <span class="bp">self</span><span class="o">.</span><span class="n">lineno</span> <span class="o">+=</span> <span class="mi">1</span>1089 <span class="n">line</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">file</span><span class="o">.</span><span class="n">readline</span><span class="p">()</span>1090 <span class="k">if</span> <span class="ow">not</span> <span class="n">line</span><span class="p">:</span>1091 <span class="k">return</span> <span class="kc">None</span>1092 <span class="k">if</span> <span class="n">line</span><span class="o">.</span><span class="n">endswith</span><span class="p">(</span><span class="s1">'</span><span class="se">\n</span><span class="s1">'</span><span class="p">):</span>1093 <span class="n">line</span> <span class="o">=</span> <span class="n">line</span><span class="p">[:</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>1094 <span class="k">return</span> <span class="s2">"</span><span class="si">%i</span><span class="s2">: </span><span class="si">%s</span><span class="s2">"</span> <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">lineno</span><span class="p">,</span> <span class="n">line</span><span class="p">)</span>1095 1096 <span class="k">def</span><span class="w"> </span><span class="nf">__getstate__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>1097 <span class="c1"># Copy the object's state from self.__dict__ which contains</span>1098 <span class="c1"># all our instance attributes. Always use the dict.copy()</span>1099 <span class="c1"># method to avoid modifying the original state.</span>1100 <span class="n">state</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="vm">__dict__</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>1101 <span class="c1"># Remove the unpicklable entries.</span>1102 <span class="k">del</span> <span class="n">state</span><span class="p">[</span><span class="s1">'file'</span><span class="p">]</span>1103 <span class="k">return</span> <span class="n">state</span>1104 1105 <span class="k">def</span><span class="w"> </span><span class="nf">__setstate__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">state</span><span class="p">):</span>1106 <span class="c1"># Restore instance attributes (i.e., filename and lineno).</span>1107 <span class="bp">self</span><span class="o">.</span><span class="vm">__dict__</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">state</span><span class="p">)</span>1108 <span class="c1"># Restore the previously opened file's state. To do so, we need to</span>1109 <span class="c1"># reopen it and read from it until the line count is restored.</span>1110 <span class="n">file</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">filename</span><span class="p">)</span>1111 <span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">lineno</span><span class="p">):</span>1112 <span class="n">file</span><span class="o">.</span><span class="n">readline</span><span class="p">()</span>1113 <span class="c1"># Finally, save the file.</span>1114 <span class="bp">self</span><span class="o">.</span><span class="n">file</span> <span class="o">=</span> <span class="n">file</span>1115</pre></div>1116</div>1117<p>A sample usage might be something like this:</p>1118<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">reader</span> <span class="o">=</span> <span class="n">TextReader</span><span class="p">(</span><span class="s2">"hello.txt"</span><span class="p">)</span>1119<span class="gp">>>> </span><span class="n">reader</span><span class="o">.</span><span class="n">readline</span><span class="p">()</span>1120<span class="go">'1: Hello world!'</span>1121<span class="gp">>>> </span><span class="n">reader</span><span class="o">.</span><span class="n">readline</span><span class="p">()</span>1122<span class="go">'2: I am line number two.'</span>1123<span class="gp">>>> </span><span class="n">new_reader</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">pickle</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">reader</span><span class="p">))</span>1124<span class="gp">>>> </span><span class="n">new_reader</span><span class="o">.</span><span class="n">readline</span><span class="p">()</span>1125<span class="go">'3: Goodbye!'</span>1126</pre></div>1127</div>1128</section>1129</section>1130<section id="custom-reduction-for-types-functions-and-other-objects">1131<span id="reducer-override"></span><h2>Custom Reduction for Types, Functions, and Other Objects<a class="headerlink" href="#custom-reduction-for-types-functions-and-other-objects" title="Link to this heading">¶</a></h2>1132<div class="versionadded">1133<p><span class="versionmodified added">Added in version 3.8.</span></p>1134</div>1135<p>Sometimes, <a class="reference internal" href="#pickle.Pickler.dispatch_table" title="pickle.Pickler.dispatch_table"><code class="xref py py-attr docutils literal notranslate"><span class="pre">dispatch_table</span></code></a> may not be flexible enough.1136In particular we may want to customize pickling based on another criterion1137than the object’s type, or we may want to customize the pickling of1138functions and classes.</p>1139<p>For those cases, it is possible to subclass from the <a class="reference internal" href="#pickle.Pickler" title="pickle.Pickler"><code class="xref py py-class docutils literal notranslate"><span class="pre">Pickler</span></code></a> class and1140implement a <a class="reference internal" href="#pickle.Pickler.reducer_override" title="pickle.Pickler.reducer_override"><code class="xref py py-meth docutils literal notranslate"><span class="pre">reducer_override()</span></code></a> method. This method can return an1141arbitrary reduction tuple (see <a class="reference internal" href="#object.__reduce__" title="object.__reduce__"><code class="xref py py-meth docutils literal notranslate"><span class="pre">__reduce__()</span></code></a>). It can alternatively return1142<a class="reference internal" href="constants.html#NotImplemented" title="NotImplemented"><code class="xref py py-data docutils literal notranslate"><span class="pre">NotImplemented</span></code></a> to fallback to the traditional behavior.</p>1143<p>If both the <a class="reference internal" href="#pickle.Pickler.dispatch_table" title="pickle.Pickler.dispatch_table"><code class="xref py py-attr docutils literal notranslate"><span class="pre">dispatch_table</span></code></a> and1144<a class="reference internal" href="#pickle.Pickler.reducer_override" title="pickle.Pickler.reducer_override"><code class="xref py py-meth docutils literal notranslate"><span class="pre">reducer_override()</span></code></a> are defined, then1145<code class="xref py py-meth docutils literal notranslate"><span class="pre">reducer_override()</span></code> method takes priority.</p>1146<div class="admonition note">1147<p class="admonition-title">Note</p>1148<p>For performance reasons, <a class="reference internal" href="#pickle.Pickler.reducer_override" title="pickle.Pickler.reducer_override"><code class="xref py py-meth docutils literal notranslate"><span class="pre">reducer_override()</span></code></a> may not be1149called for the following objects: <code class="docutils literal notranslate"><span class="pre">None</span></code>, <code class="docutils literal notranslate"><span class="pre">True</span></code>, <code class="docutils literal notranslate"><span class="pre">False</span></code>, and1150exact instances of <a class="reference internal" href="functions.html#int" title="int"><code class="xref py py-class docutils literal notranslate"><span class="pre">int</span></code></a>, <a class="reference internal" href="functions.html#float" title="float"><code class="xref py py-class docutils literal notranslate"><span class="pre">float</span></code></a>, <a class="reference internal" href="stdtypes.html#bytes" title="bytes"><code class="xref py py-class docutils literal notranslate"><span class="pre">bytes</span></code></a>,1151<a class="reference internal" href="stdtypes.html#str" title="str"><code class="xref py py-class docutils literal notranslate"><span class="pre">str</span></code></a>, <a class="reference internal" href="stdtypes.html#dict" title="dict"><code class="xref py py-class docutils literal notranslate"><span class="pre">dict</span></code></a>, <a class="reference internal" href="stdtypes.html#set" title="set"><code class="xref py py-class docutils literal notranslate"><span class="pre">set</span></code></a>, <a class="reference internal" href="stdtypes.html#frozenset" title="frozenset"><code class="xref py py-class docutils literal notranslate"><span class="pre">frozenset</span></code></a>, <a class="reference internal" href="stdtypes.html#list" title="list"><code class="xref py py-class docutils literal notranslate"><span class="pre">list</span></code></a>1152and <a class="reference internal" href="stdtypes.html#tuple" title="tuple"><code class="xref py py-class docutils literal notranslate"><span class="pre">tuple</span></code></a>.</p>1153</div>1154<p>Here is a simple example where we allow pickling and reconstructing1155a given class:</p>1156<div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">io</span>1157<span class="kn">import</span><span class="w"> </span><span class="nn">pickle</span>1158 1159<span class="k">class</span><span class="w"> </span><span class="nc">MyClass</span><span class="p">:</span>1160 <span class="n">my_attribute</span> <span class="o">=</span> <span class="mi">1</span>1161 1162<span class="k">class</span><span class="w"> </span><span class="nc">MyPickler</span><span class="p">(</span><span class="n">pickle</span><span class="o">.</span><span class="n">Pickler</span><span class="p">):</span>1163 <span class="k">def</span><span class="w"> </span><span class="nf">reducer_override</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">obj</span><span class="p">):</span>1164<span class="w"> </span><span class="sd">"""Custom reducer for MyClass."""</span>1165 <span class="k">if</span> <span class="nb">getattr</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s2">"__name__"</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span> <span class="o">==</span> <span class="s2">"MyClass"</span><span class="p">:</span>1166 <span class="k">return</span> <span class="nb">type</span><span class="p">,</span> <span class="p">(</span><span class="n">obj</span><span class="o">.</span><span class="vm">__name__</span><span class="p">,</span> <span class="n">obj</span><span class="o">.</span><span class="vm">__bases__</span><span class="p">,</span>1167 <span class="p">{</span><span class="s1">'my_attribute'</span><span class="p">:</span> <span class="n">obj</span><span class="o">.</span><span class="n">my_attribute</span><span class="p">})</span>1168 <span class="k">else</span><span class="p">:</span>1169 <span class="c1"># For any other object, fallback to usual reduction</span>1170 <span class="k">return</span> <span class="bp">NotImplemented</span>1171 1172<span class="n">f</span> <span class="o">=</span> <span class="n">io</span><span class="o">.</span><span class="n">BytesIO</span><span class="p">()</span>1173<span class="n">p</span> <span class="o">=</span> <span class="n">MyPickler</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>1174<span class="n">p</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">MyClass</span><span class="p">)</span>1175 1176<span class="k">del</span> <span class="n">MyClass</span>1177 1178<span class="n">unpickled_class</span> <span class="o">=</span> <span class="n">pickle</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">getvalue</span><span class="p">())</span>1179 1180<span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">unpickled_class</span><span class="p">,</span> <span class="nb">type</span><span class="p">)</span>1181<span class="k">assert</span> <span class="n">unpickled_class</span><span class="o">.</span><span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"MyClass"</span>1182<span class="k">assert</span> <span class="n">unpickled_class</span><span class="o">.</span><span class="n">my_attribute</span> <span class="o">==</span> <span class="mi">1</span>1183</pre></div>1184</div>1185</section>1186<section id="out-of-band-buffers">1187<span id="pickle-oob"></span><h2>Out-of-band Buffers<a class="headerlink" href="#out-of-band-buffers" title="Link to this heading">¶</a></h2>1188<div class="versionadded">1189<p><span class="versionmodified added">Added in version 3.8.</span></p>1190</div>1191<p>In some contexts, the <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> module is used to transfer massive amounts1192of data. Therefore, it can be important to minimize the number of memory1193copies, to preserve performance and resource consumption. However, normal1194operation of the <code class="xref py py-mod docutils literal notranslate"><span class="pre">pickle</span></code> module, as it transforms a graph-like structure1195of objects into a sequential stream of bytes, intrinsically involves copying1196data to and from the pickle stream.</p>1197<p>This constraint can be eschewed if both the <em>provider</em> (the implementation1198of the object types to be transferred) and the <em>consumer</em> (the implementation1199of the communications system) support the out-of-band transfer facilities1200provided by pickle protocol 5 and higher.</p>