codekingpro/portable-devtools

1Metadata-Version: 2.12Name: dataclasses-json3Version: 0.6.74Summary: Easily serialize dataclasses to and from JSON.5Home-page: https://github.com/lidatong/dataclasses-json6License: MIT7Author: Charles Li8Author-email: charles.dt.li@gmail.com9Maintainer: Charles Li10Maintainer-email: charles.dt.li@gmail.com11Requires-Python: >=3.7,<4.012Classifier: License :: OSI Approved :: MIT License13Classifier: Programming Language :: Python :: 314Classifier: Programming Language :: Python :: 3.715Classifier: Programming Language :: Python :: 3.816Classifier: Programming Language :: Python :: 3.917Classifier: Programming Language :: Python :: 3.1018Classifier: Programming Language :: Python :: 3.1119Classifier: Programming Language :: Python :: 3.1220Requires-Dist: marshmallow (>=3.18.0,<4.0.0)21Requires-Dist: typing-inspect (>=0.4.0,<1)22Project-URL: Repository, https://github.com/lidatong/dataclasses-json23Project-URL: changelog, https://github.com/lidatong/dataclasses-json/releases24Project-URL: documentation, https://lidatong.github.io/dataclasses-json/25Project-URL: issues, https://github.com/lidatong/dataclasses-json/issues26Description-Content-Type: text/markdown27 28# Dataclasses JSON29 30![](https://github.com/lidatong/dataclasses-json/workflows/dataclasses-json/badge.svg)31 32This library provides a simple API for encoding and decoding [dataclasses](https://docs.python.org/3/library/dataclasses.html) to and from JSON.33 34It's very easy to get started.35 36[README / Documentation website](https://lidatong.github.io/dataclasses-json). Features a navigation bar and search functionality, and should mirror this README exactly -- take a look!37 38## Quickstart39 40`pip install dataclasses-json`41 42```python43from dataclasses import dataclass44from dataclasses_json import dataclass_json45 46 47@dataclass_json48@dataclass49class Person:50    name: str51 52 53person = Person(name='lidatong')54person.to_json()  # '{"name": "lidatong"}' <- this is a string55person.to_dict()  # {'name': 'lidatong'} <- this is a dict56Person.from_json('{"name": "lidatong"}')  # Person(1)57Person.from_dict({'name': 'lidatong'})  # Person(1)58 59# You can also apply _schema validation_ using an alternative API60# This can be useful for "typed" Python code61 62Person.from_json('{"name": 42}')  # This is ok. 42 is not a `str`, but63                                  # dataclass creation does not validate types64Person.schema().loads('{"name": 42}')  # Error! Raises `ValidationError`65```66 67**What if you want to work with camelCase JSON?**68 69```python70# same imports as above, with the additional `LetterCase` import71from dataclasses import dataclass72from dataclasses_json import dataclass_json, LetterCase73 74@dataclass_json(letter_case=LetterCase.CAMEL)  # now all fields are encoded/decoded from camelCase75@dataclass76class ConfiguredSimpleExample:77    int_field: int78 79ConfiguredSimpleExample(1).to_json()  # {"intField": 1}80ConfiguredSimpleExample.from_json('{"intField": 1}')  # ConfiguredSimpleExample(1)81```82 83## Supported types84 85It's recursive (see caveats below), so you can easily work with nested dataclasses.86In addition to the supported types in the 87[py to JSON table](https://docs.python.org/3/library/json.html#py-to-json-table), this library supports the following:88 89- any arbitrary [Collection](https://docs.python.org/3/library/collections.abc.html#collections.abc.Collection) type is supported.90[Mapping](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) types are encoded as JSON objects and `str` types as JSON strings. 91Any other Collection types are encoded into JSON arrays, but decoded into the original collection types.92 93- [datetime](https://docs.python.org/3/library/datetime.html#available-types) 94objects. `datetime` objects are encoded to `float` (JSON number) using 95[timestamp](https://docs.python.org/3/library/datetime.html#datetime.datetime.timestamp).96As specified in the `datetime` docs, if your `datetime` object is naive, it will 97assume your system local timezone when calling `.timestamp()`. JSON numbers 98corresponding to a `datetime` field in your dataclass are decoded 99into a datetime-aware object, with `tzinfo` set to your system local timezone.100Thus, if you encode a datetime-naive object, you will decode into a 101datetime-aware object. This is important, because encoding and decoding won't 102strictly be inverses. See [this section](#Overriding) if you want to override this default103behavior (for example, if you want to use ISO).104 105- [UUID](https://docs.python.org/3/library/uuid.html#uuid.UUID) objects. They 106are encoded as `str` (JSON string).107 108- [Decimal](https://docs.python.org/3/library/decimal.html) objects. They are109also encoded as `str`.110 111**The [latest release](https://github.com/lidatong/dataclasses-json/releases/latest) is compatible with both Python 3.7 and Python 3.6 (with the dataclasses backport).**112 113## Usage114 115#### Approach 1: Class decorator116 117```python118from dataclasses import dataclass119from dataclasses_json import dataclass_json120 121@dataclass_json122@dataclass123class Person:124    name: str125 126lidatong = Person('lidatong')127 128# Encoding to JSON129lidatong.to_json()  # '{"name": "lidatong"}'130 131# Decoding from JSON132Person.from_json('{"name": "lidatong"}')  # Person(name='lidatong')133```134 135Note that the `@dataclass_json` decorator must be stacked above the `@dataclass`136decorator (order matters!)137 138#### Approach 2: Inherit from a mixin139 140```python141from dataclasses import dataclass142from dataclasses_json import DataClassJsonMixin143 144@dataclass145class Person(DataClassJsonMixin):146    name: str147 148lidatong = Person('lidatong')149 150# A different example from Approach 1 above, but usage is the exact same151assert Person.from_json(lidatong.to_json()) == lidatong152```153 154Pick whichever approach suits your taste. Note that there is better support for155 the mixin approach when using _static analysis_ tools (e.g. linting, typing),156 but the differences in implementation will be invisible in _runtime_ usage.157 158## How do I...159 160 161 162### Use my dataclass with JSON arrays or objects?163 164```python165from dataclasses import dataclass166from dataclasses_json import dataclass_json167 168@dataclass_json169@dataclass170class Person:171    name: str172```173 174**Encode into a JSON array containing instances of my Data Class**175 176```python177people_json = [Person('lidatong')]178Person.schema().dumps(people_json, many=True)  # '[{"name": "lidatong"}]'179```180 181**Decode a JSON array containing instances of my Data Class**182 183```python184people_json = '[{"name": "lidatong"}]'185Person.schema().loads(people_json, many=True)  # [Person(name='lidatong')]186```187 188**Encode as part of a larger JSON object containing my Data Class (e.g. an HTTP 189request/response)**190 191```python192import json193 194response_dict = {195    'response': {196        'person': Person('lidatong').to_dict()197    }198}199 200response_json = json.dumps(response_dict)201```202 203In this case, we do two steps. First, we encode the dataclass into a 204**python dictionary** rather than a JSON string, using `.to_dict`. 205 206Second, we leverage the built-in `json.dumps` to serialize our `dataclass` into 207a JSON string.208 209**Decode as part of a larger JSON object containing my Data Class (e.g. an HTTP 210response)**211 212```python213import json214 215response_dict = json.loads('{"response": {"person": {"name": "lidatong"}}}')216 217person_dict = response_dict['response']218 219person = Person.from_dict(person_dict)220```221 222In a similar vein to encoding above, we leverage the built-in `json` module.223 224First, call `json.loads` to read the entire JSON object into a 225dictionary. We then access the key of the value containing the encoded dict of 226our `Person` that we want to decode (`response_dict['response']`).227 228Second, we load in the dictionary using `Person.from_dict`.229 230 231### Encode or decode into Python lists/dictionaries rather than JSON?232 233This can be by calling `.schema()` and then using the corresponding 234encoder/decoder methods, ie. `.load(...)`/`.dump(...)`.235 236**Encode into a single Python dictionary**237 238```python239person = Person('lidatong')240person.to_dict()  # {'name': 'lidatong'}241```242 243**Encode into a list of Python dictionaries**244 245```python246people = [Person('lidatong')]247Person.schema().dump(people, many=True)  # [{'name': 'lidatong'}]248```249 250**Decode a dictionary into a single dataclass instance**251 252```python253person_dict = {'name': 'lidatong'}254Person.from_dict(person_dict)  # Person(name='lidatong')255```256 257**Decode a list of dictionaries into a list of dataclass instances**258 259```python260people_dicts = [{"name": "lidatong"}]261Person.schema().load(people_dicts, many=True)  # [Person(name='lidatong')]262```263 264### Encode or decode from camelCase (or kebab-case)?265 266JSON letter case by convention is camelCase, in Python members are by convention snake_case.267 268You can configure it to encode/decode from other casing schemes at both the class level and the field level.269 270```python271from dataclasses import dataclass, field272 273from dataclasses_json import LetterCase, config, dataclass_json274 275 276# changing casing at the class level277@dataclass_json(letter_case=LetterCase.CAMEL)278@dataclass279class Person:280    given_name: str281    family_name: str282    283Person('Alice', 'Liddell').to_json()  # '{"givenName": "Alice"}'284Person.from_json('{"givenName": "Alice", "familyName": "Liddell"}')  # Person('Alice', 'Liddell')285 286# at the field level287@dataclass_json288@dataclass289class Person:290    given_name: str = field(metadata=config(letter_case=LetterCase.CAMEL))291    family_name: str292    293Person('Alice', 'Liddell').to_json()  # '{"givenName": "Alice"}'294# notice how the `family_name` field is still snake_case, because it wasn't configured above295Person.from_json('{"givenName": "Alice", "family_name": "Liddell"}')  # Person('Alice', 'Liddell')296```297 298**This library assumes your field follows the Python convention of snake_case naming.**299If your field is not `snake_case` to begin with and you attempt to parameterize `LetterCase`, 300the behavior of encoding/decoding is undefined (most likely it will result in subtle bugs).301 302### Encode or decode using a different name303 304```python305from dataclasses import dataclass, field306 307from dataclasses_json import config, dataclass_json308 309@dataclass_json310@dataclass311class Person:312    given_name: str = field(metadata=config(field_name="overriddenGivenName"))313 314Person(given_name="Alice")  # Person('Alice')315Person.from_json('{"overriddenGivenName": "Alice"}')  # Person('Alice')316Person('Alice').to_json()  # {"overriddenGivenName": "Alice"}317```318 319### Handle missing or optional field values when decoding?320 321By default, any fields in your dataclass that use `default` or 322`default_factory` will have the values filled with the provided default, if the323corresponding field is missing from the JSON you're decoding.324 325**Decode JSON with missing field**326 327```python328@dataclass_json329@dataclass330class Student:331    id: int332    name: str = 'student'333 334Student.from_json('{"id": 1}')  # Student(id=1, name='student')335```336 337Notice `from_json` filled the field `name` with the specified default 'student'338when it was missing from the JSON.339 340Sometimes you have fields that are typed as `Optional`, but you don't 341necessarily want to assign a default. In that case, you can use the 342`infer_missing` kwarg to make `from_json` infer the missing field value as `None`.343 344**Decode optional field without default**345 346```python347@dataclass_json348@dataclass349class Tutor:350    id: int351    student: Optional[Student] = None352 353Tutor.from_json('{"id": 1}')  # Tutor(id=1, student=None)354```355 356Personally I recommend you leverage dataclass defaults rather than using 357`infer_missing`, but if for some reason you need to decouple the behavior of 358JSON decoding from the field's default value, this will allow you to do so.359 360 361### Handle unknown / extraneous fields in JSON?362 363By default, it is up to the implementation what happens when a `json_dataclass` receives input parameters that are not defined.364(the `from_dict` method ignores them, when loading using `schema()` a ValidationError is raised.)365There are three ways to customize this behavior.366 367Assume you want to instantiate a dataclass with the following dictionary:368```python369dump_dict = {"endpoint": "some_api_endpoint", "data": {"foo": 1, "bar": "2"}, "undefined_field_name": [1, 2, 3]}370```371 3721. You can enforce to always raise an error by setting the `undefined` keyword to `Undefined.RAISE`373 (`'RAISE'` as a case-insensitive string works as well). Of course it works normally if you don't pass any undefined parameters.374    375```python376from dataclasses_json import Undefined377 378@dataclass_json(undefined=Undefined.RAISE)379@dataclass()380class ExactAPIDump:381    endpoint: str382    data: Dict[str, Any]383 384dump = ExactAPIDump.from_dict(dump_dict)  # raises UndefinedParameterError385```386 3872. You can simply ignore any undefined parameters by setting the `undefined` keyword to `Undefined.EXCLUDE`388 (`'EXCLUDE'` as a case-insensitive string works as well). Note that you will not be able to retrieve them using `to_dict`:389    390```python391from dataclasses_json import Undefined392 393@dataclass_json(undefined=Undefined.EXCLUDE)394@dataclass()395class DontCareAPIDump:396    endpoint: str397    data: Dict[str, Any]398 399dump = DontCareAPIDump.from_dict(dump_dict)  # DontCareAPIDump(endpoint='some_api_endpoint', data={'foo': 1, 'bar': '2'})400dump.to_dict()  # {"endpoint": "some_api_endpoint", "data": {"foo": 1, "bar": "2"}}401```402 4033. You can save them in a catch-all field and do whatever needs to be done later. Simply set the `undefined`404keyword to `Undefined.INCLUDE` (`'INCLUDE'` as a case-insensitive string works as well) and define a field405of type `CatchAll` where all unknown values will end up.406 This simply represents a dictionary that can hold anything. 407 If there are no undefined parameters, this will be an empty dictionary.408    409```python410from dataclasses_json import Undefined, CatchAll411 412@dataclass_json(undefined=Undefined.INCLUDE)413@dataclass()414class UnknownAPIDump:415    endpoint: str416    data: Dict[str, Any]417    unknown_things: CatchAll418 419dump = UnknownAPIDump.from_dict(dump_dict)  # UnknownAPIDump(endpoint='some_api_endpoint', data={'foo': 1, 'bar': '2'}, unknown_things={'undefined_field_name': [1, 2, 3]})420dump.to_dict()  # {'endpoint': 'some_api_endpoint', 'data': {'foo': 1, 'bar': '2'}, 'undefined_field_name': [1, 2, 3]}421```422 423Notes:424- When using `Undefined.INCLUDE`, an `UndefinedParameterError` will be raised if you don't specify425exactly one field of type `CatchAll`.426- Note that `LetterCase` does not affect values written into the `CatchAll` field, they will be as they are given.427- When specifying a default (or a default factory) for the the `CatchAll`-field, e.g. `unknown_things: CatchAll = None`, the default value will be used instead of an empty dict if there are no undefined parameters.428- Calling __init__ with non-keyword arguments resolves the arguments to the defined fields and writes everything else into the catch-all field.429 4304. All 3 options work as well using `schema().loads` and `schema().dumps`, as long as you don't overwrite it by specifying `schema(unknown=<a marshmallow value>)`.431marshmallow uses the same 3 keywords ['include', 'exclude', 'raise'](https://marshmallow.readthedocs.io/en/stable/quickstart.html#handling-unknown-fields).432 4335. All 3 operations work as well using `__init__`, e.g. `UnknownAPIDump(**dump_dict)` will **not** raise a `TypeError`, but write all unknown values to the field tagged as `CatchAll`.434   Classes tagged with `EXCLUDE` will also simply ignore unknown parameters. Note that classes tagged as `RAISE` still raise a `TypeError`, and **not** a `UndefinedParameterError` if supplied with unknown keywords.435 436 437### Override the default encode / decode / marshmallow field of a specific field?438 439See [Overriding](#Overriding)440 441### Handle recursive dataclasses?442Object hierarchies where fields are of the type that they are declared within require a small443type hinting trick to declare the forward reference.444```python445from typing import Optional446from dataclasses import dataclass447from dataclasses_json import dataclass_json448 449@dataclass_json450@dataclass451class Tree():452    value: str453    left: Optional['Tree']454    right: Optional['Tree']455```456 457Avoid using458```python459from __future__ import annotations460```461as it will cause problems with the way dataclasses_json accesses the type annotations.462 463### Use numpy or pandas types?464Data types specific to libraries commonly used in data analysis and machine learning like [numpy](https://github.com/numpy/numpy) and [pandas](https://github.com/pandas-dev/pandas) are not supported by default, but you can easily enable them by using custom decoders and encoders. Below are two examples for `numpy` and `pandas` types.465 466```python467from dataclasses import field, dataclass468from dataclasses_json import config, dataclass_json469import numpy as np470import pandas as pd471 472@dataclass_json473@dataclass474class DataWithNumpy:475    my_int: np.int64 = field(metadata=config(decoder=np.int64))476    my_float: np.float64 = field(metadata=config(decoder=np.float64))477    my_array: np.ndarray = field(metadata=config(decoder=np.asarray))478DataWithNumpy.from_json("{\"my_int\": 42, \"my_float\": 13.37, \"my_array\": [1,2,3]}")479 480@dataclass_json481@dataclass482class DataWithPandas:483    my_df: pd.DataFrame = field(metadata=config(decoder=pd.DataFrame.from_records, encoder=lambda x: x.to_dict(orient="records")))484data = DataWithPandas.from_dict({"my_df": [{"col1": 1, "col2": 2}, {"col1": 3, "col2": 4}]})485# my_df results in:486# col1  col2487# 1    2    488# 3    4489data.to_dict()490# {"my_df": [{"col1": 1, "col2": 2}, {"col1": 3, "col2": 4}]}491```492 493## Marshmallow interop494 495Using the `dataclass_json` decorator or mixing in `DataClassJsonMixin` will496provide you with an additional method `.schema()`.497 498`.schema()` generates a schema exactly equivalent to manually creating a499marshmallow schema for your dataclass. You can reference the [marshmallow API docs](https://marshmallow.readthedocs.io/en/3.0/api_reference.html#schema)500to learn other ways you can use the schema returned by `.schema()`.501 502You can pass in the exact same arguments to `.schema()` that you would when503constructing a `PersonSchema` instance, e.g. `.schema(many=True)`, and they will504get passed through to the marshmallow schema.505 506 507```python508from dataclasses import dataclass509from dataclasses_json import dataclass_json510 511@dataclass_json512@dataclass513class Person:514    name: str515 516# You don't need to do this - it's generated for you by `.schema()`!517from marshmallow import Schema, fields518 519class PersonSchema(Schema):520    name = fields.Str()521```522 523Briefly, on what's going on under the hood in the above examples: calling 524`.schema()` will have this library generate a525[marshmallow schema]('https://marshmallow.readthedocs.io/en/3.0/api_reference.html#schema)526for you. It also fills in the corresponding object hook, so that marshmallow527will create an instance of your Data Class on `load` (e.g.528`Person.schema().load` returns a `Person`) rather than a `dict`, which it does529by default in marshmallow.530 531**Performance note**532 533`.schema()` is not cached (it generates the schema on every call), so if you534have a nested Data Class you may want to save the result to a variable to 535avoid re-generation of the schema on every usage.536 537```python538person_schema = Person.schema()539person_schema.dump(people, many=True)540 541# later in the code...542 543person_schema.dump(person)544```545 546## Overriding / Extending547 548#### Overriding549 550For example, you might want to encode/decode `datetime` objects using ISO format551rather than the default `timestamp`.552 553```python554from dataclasses import dataclass, field555from dataclasses_json import dataclass_json, config556from datetime import datetime557from marshmallow import fields558 559@dataclass_json560@dataclass561class DataClassWithIsoDatetime:562    created_at: datetime = field(563        metadata=config(564            encoder=datetime.isoformat,565            decoder=datetime.fromisoformat,566            mm_field=fields.DateTime(format='iso')567        )568    )569```570 571#### Extending572 573Similarly, you might want to extend `dataclasses_json` to encode `date` objects.574 575```python576from dataclasses import dataclass, field577from dataclasses_json import dataclass_json, config578from datetime import date579from marshmallow import fields580 581dataclasses_json.cfg.global_config.encoders[date] = date.isoformat582dataclasses_json.cfg.global_config.decoders[date] = date.fromisoformat583 584@dataclass_json585@dataclass586class DataClassWithIsoDatetime:587    created_at: date588    modified_at: date589    accessed_at: date590```591 592As you can see, you can **override** or **extend** the default codecs by providing a "hook" via a 593callable:594- `encoder`: a callable, which will be invoked to convert the field value when encoding to JSON595- `decoder`: a callable, which will be invoked to convert the JSON value when decoding from JSON596- `mm_field`: a marshmallow field, which will affect the behavior of any operations involving `.schema()`597 598Note that these hooks will be invoked regardless if you're using 599`.to_json`/`dump`/`dumps`600and `.from_json`/`load`/`loads`. So apply overrides / extensions judiciously, making sure to 601carefully consider whether the interaction of the encode/decode/mm_field is consistent with what you expect!602 603 604#### What if I have other dataclass field extensions that rely on `metadata`605 606All the `dataclasses_json.config` does is return a mapping, namespaced under the key `'dataclasses_json'`.607 608Say there's another module, `other_dataclass_package` that uses metadata. Here's how you solve your problem:609 610```python611metadata = {'other_dataclass_package': 'some metadata...'}  # pre-existing metadata for another dataclass package612dataclass_json_config = config(613            encoder=datetime.isoformat,614            decoder=datetime.fromisoformat,615            mm_field=fields.DateTime(format='iso')616        )617metadata.update(dataclass_json_config)618 619@dataclass_json620@dataclass621class DataClassWithIsoDatetime:622    created_at: datetime = field(metadata=metadata)623```624 625You can also manually specify the dataclass_json configuration mapping.626 627```python628@dataclass_json629@dataclass630class DataClassWithIsoDatetime:631    created_at: date = field(632        metadata={'dataclasses_json': {633            'encoder': date.isoformat,634            'decoder': date.fromisoformat,635            'mm_field': fields.DateTime(format='iso')636        }}637    )638```639 640## A larger example641 642```python643from dataclasses import dataclass644from dataclasses_json import dataclass_json645 646from typing import List647 648@dataclass_json649@dataclass(frozen=True)650class Minion:651    name: str652 653 654@dataclass_json655@dataclass(frozen=True)656class Boss:657    minions: List[Minion]658 659boss = Boss([Minion('evil minion'), Minion('very evil minion')])660boss_json = """661{662    "minions": [663        {664            "name": "evil minion"665        },666        {667            "name": "very evil minion"668        }669    ]670}671""".strip()672 673assert boss.to_json(indent=4) == boss_json674assert Boss.from_json(boss_json) == boss675```676 677## Performance678 679Take a look at [this issue](https://github.com/lidatong/dataclasses-json/issues/228)680 681## Versioning682 683Note this library is still pre-1.0.0 (SEMVER).684 685The current convention is:686- **PATCH** version upgrades for bug fixes and minor feature additions.687- **MINOR** version upgrades for big API features and breaking changes.688 689Once this library is 1.0.0, it will follow standard SEMVER conventions.690 691### Python compatibility 692 693Any version that is not listed in the table below we do not test against, though you might still be able to install the library. For future Python versions, please open an issue and/or a pull request, adding them to the CI suite.694 695 696| Python version range | Compatible dataclasses-json version |697|----------------------|:-----------------------------------:|698| 3.7.x - 3.12.x       |            0.5.x - 0.6.x            |699| >= 3.13.x            |         No official support (yet)   |700 701 702## Roadmap703 704Currently the focus is on investigating and fixing bugs in this library, working705on performance, and finishing [this issue](https://github.com/lidatong/dataclasses-json/issues/31).706 707That said, if you think there's a feature missing / something new needed in the708library, please see the contributing section below.709 710 711## Contributing712 713First of all, thank you for being interested in contributing to this library.714I really appreciate you taking the time to work on this project.715 716- If you're just interested in getting into the code, a good place to start are 717issues tagged as bugs.718- If introducing a new feature, especially one that modifies the public API, 719consider submitting an issue for discussion before a PR. Please also take a look 720at existing issues / PRs to see what you're proposing has  already been covered 721before / exists.722- I like to follow the commit conventions documented [here](https://www.conventionalcommits.org/en/v1.0.0/#summary)723 724### Setting up your environment725 726This project uses [Poetry](https://python-poetry.org/) for dependency and venv management. It is quite simple to get ready for your first commit:727- [Install](https://python-poetry.org/docs/#installation) latest stable Poetry728- Navigate to where you cloned `dataclasses-json`729- Run `poetry install`730- Create a branch and start writing code!731 732