v0.0.26 – 40x performance boost; breaking changes

Migration to Pydantic v2 and libipld

All models have been migrated to Pydantic v2. Fields constraints have been added. Decoding of DAG-CBOR, CID and CAR files has been migrated to the brand-new library libipld. This library is powered by Rust and is much faster than the previous implementation. Pydantic v2 also uses Rust in the core. This leads to a significant performance boost.

Firehose catch up benchmark:

• The previous SDK version: 700 commits in 5 seconds.
• After migration to Pydantic v2: 2650 commits in 5 seconds.
• After migration to libipld: 20000 commits in 5 seconds.
• Using pydantic v2 and libipld with multiprocessing: 30000 commits in 5 seconds.

The new release gives a 40x performance boost! But the cost is a lot of breaking changes.

Example of firehose consumer with multiprocessing: process_commits.py

Test stand for benchmarks: MacBook Pro 2021, Apple M1 Pro, 32 GB RAM, 450mbps connection speed, Python 3.8

❗Breaking changes

• Python 3.7.0 has been dropped. The minimum supported version is now Python 3.7.1.
• Camel cased fields are gone. Use snake case instead. For example, createdAt is now created_at.
• Root namespace has been fixed from bsky to app. For example, Client().bsky.feed.get_likes is now Client().app.bsky.feed.get_likes.
• Using similar model instances as strong refs is not allowed anymore. Use models.create_strong_ref helper function to convert refs (example).
• Creating model instances using positional arguments is no longer supported. Use keyword arguments instead. For example, thant's not possible anymore models.ComAtprotoIdentityResolveHandle.Params('marshal.dev'). Use models.ComAtprotoIdentityResolveHandle.Params(handle='marshal.dev') instead.
• Fields that conflict with reserved Pydantic names has _ (underscore) suffix. For example, validation is now validation_.
• DotDict has been moved to models.dot_dict.
• Inheritance of base models has been changed. Please check new base classes.
• Inheritance of DotDict has been changed. Please check the new base class.
• BlobRef model doesn't contain to_dict() method anymore.
• CID class has been reimplemented using libipld lib. It supports much less API.
• _type field of models has been renamed to py_type. Now it's constant.
• leb128 module has been removed.
• Type hint of CID has been changed to CIDType.
• Type hint of DotDict has been changed to DotDictType.
• multiformats and dag-cbor dependencies have been removed.
• These reference classes have been removed:
  • ResponseRef from get_profile. Use models.AppBskyActorDefs.ProfileViewDetailed instead.
  • ResponseRef from get_moderation_action. Use models.ComAtprotoAdminDefs.ActionViewDetail instead.
  • ResponseRef from get_moderation_report. Use models.ComAtprotoAdminDefs.ReportViewDetail instead.
  • ResponseRef from get_record. Use models.ComAtprotoAdminDefs.RecordViewDetail instead.
  • ResponseRef from get_repo. Use models.ComAtprotoAdminDefs.RepoViewDetail instead.
  • ResponseRef from resolve_moderation_reports. Use models.ComAtprotoAdminDefs.ActionView instead.
  • ResponseRef from reverse_moderation_action. Use models.ComAtprotoAdminDefs.ActionView instead.
  • ResponseRef from take_moderation_action. Use models.ComAtprotoAdminDefs.ActionView instead.
  • ResponseRef from create_app_password. Use models.ComAtprotoServerCreateAppPassword.AppPassword instead.
• These exceptions have been removed:
  • UnexpectedFieldError. Use ModelError instead.
  • MissingValueError. Use ModelError instead.
  • ModelFieldError. Use ModelError instead.
  • WrongTypeError. Use ModelError instead.
  • CBORDecodingError. Use DAGCBORDecodingError instead.

New Features

• Unit tests for model serialization and deserialization.
• Nested dictionaries support in DotDict models.
• DotDict models now support __getitem__ and __setitem__ methods.
• create_strong_ref helper function to convert ref-like models to strong refs.
• Fields constraints for models. Now you can see the max items count for the image array, max string length, etc.
• Better documentation of models.

Minor Changes

• Fixed nesting of DotDict models.
• Fixed serialization of Union types.
• Fixed serialization of Literal types.
• Fixed sending proper datetime values to the server.
• Fixed printing tracebacks in the Firehose async client.
• Fixed chaining of firehose exceptions.
• Fixed locked and outdated typing-extensions dependency.
• Fixed passing of arguments to ClientBase.

Full Changelog: v0.0.25...v0.0.26

v0.0.25

What's Changed

🔥 Bsky made breaking changes in models of firehose. This release fixes it

• Update changelog for v0.0.24 by @MarshalX in #126
• Update lexicons fetched from ad1fcf1 committed 2023-08-30T00:07:21Z by @MarshalX in #131

Full Changelog: v0.0.24...v0.0.25

v0.0.24

What's Changed

• Add update profile example by @MarshalX in #120
• Add automatic link (facet) detection example by @Jxck-S in #122
• Update changelog for v0.0.23 by @MarshalX in #117
• Update changelog by @MarshalX in #118
• Update lexicons fetched from 244bf46 committed 2023-08-10T20:54:24Z by @MarshalX in #124
• Update packages; fix CVE-2023-37920 (certifi) by @MarshalX in #125
• Fix update profile example by @IamC8 in #121

New Contributors

• @IamC8 made their first contribution in #121
• @Jxck-S made their first contribution in #122

Full Changelog: v0.0.23...v0.0.24

v0.0.23

What's Changed

• ❗ Delete get_or_create_model method (backward incompatible) by @MarshalX in #111
• Add documentation for base models by @MarshalX in #109
• Add import aliases for "models.utils" by @MarshalX in #111
• Add str and repr for BlobRef by @MarshalX in #113
• Add Firehose process commits example by @MarshalX in #114
• Implement additional magic methods for DotDict by @MarshalX in #109
• Make languages constants immutable by @MarshalX in #110
• Fix get_model_as_dict for DotDict by @MarshalX in #111
• Fix is_record_type for DotDict models by @MarshalX in #112
• Fix access to unknown fields in DotDict by @MarshalX in #116
• Update changelog for v0.0.21 by @MarshalX in #108

Full Changelog: v0.0.21...v0.0.23

v0.0.22

release has been yanked

v0.0.21

What's Changed

• Add dot notation for dictionaries by @MarshalX in #106
• Fix unknown type that could be plain dictionary by @MarshalX in #105
• Fix parsing of custom (extended) records by @MarshalX in #106
• Fix a small typo in README.md by @ndrezn in #104
• Update lexicons fetched from b2ef386 committed 2023-07-20T16:00:51Z by @MarshalX in #107
• Update changelog for v0.0.20 by @MarshalX in #103

New Contributors

• @ndrezn made their first contribution in #104

Full Changelog: v0.0.20...v0.0.21

v0.0.20

What's Changed

• Update changelog for v0.0.19 by @MarshalX in #99
• Add pagination example using cursors by @ymdpharm in #93
• Migrate to websockets lib (fixed all known issues with lost firehouse frames, reconnections and crashes) by @MarshalX in #101

New Contributors

• @ymdpharm made their first contribution in #93

Full Changelog: v0.0.19...v0.0.20

v0.0.19

What's Changed

• Update changelog for v0.0.18 by @MarshalX in #92
• Add posts langs support by @MarshalX in #96
• Fix infinite loop of reconnections to Firehose by @MarshalX in #97
• Update lexicons fetched from 775944e committed 2023-07-17T23:06:44Z by @MarshalX in #98

Full Changelog: v0.0.18...v0.0.19

v0.0.18

What's Changed

• Update changelog for v0.0.17 by @MarshalX in #86
• Fix is_record_type for dict record types by @joelghill in #88
• Lower version of typing-extensions as possible; update ruff by @MarshalX in #90
• Update lexicons fetched from b9ca76f committed 2023-07-14T23:05:56Z by @MarshalX in #91

New Contributors

• @joelghill made their first contribution in #88

Full Changelog: v0.0.17...v0.0.18

v0.0.17

What's Changed

• Update changelog for v0.0.16 by @MarshalX in #82
• Move the websocket client into part of the atproto package by @MarshalX in #84
• Update lexicons fetched from e7a0d27 committed 2023-07-03T16:28:39Z by @MarshalX in #85

Full Changelog: v0.0.16...v0.0.17