Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

atproto/lexicon: package for working with lexicon schemas, and runtime data validation #420

Merged
merged 23 commits into from
Oct 4, 2024
Merged

atproto/lexicon: package for working with lexicon schemas, and runtime data validation #420

merged 23 commits into from
Oct 4, 2024

Conversation

bnewbold
Copy link
Collaborator

@bnewbold bnewbold commented Nov 7, 2023 •
edited
Loading

This is currently a branch on top of #407

  • parse lexicon schema JSON
  • load entire directories of schema JSON files from disk as a catalog
  • check lexicon schema semantics (eg, can't have min greater than max)
  • validate runtime data (map[string]any) against lexicons
  • whole bunch of corner-case tests
  • CLI tool for some live-network testing
  • add support for tid and record-key lex formats (not in specs yet)
  • configurable flexible to legacy blobs and lenient datetime parsing (?)
  • comments and example code

probably in a later iteration:

  • ensure empty body works (Handle empty bodies in lexicon validation atproto#2746)
  • validate rkey type against lexicon
  • CLI tool to validate prod firehose
  • CLI tool to validate CAR files
  • clarify specs around unions: only object and token types?
  • clarify specs around unknown: only object type?
  • validate other "primary" lexicon types: subscription, HTTP body, HTTP URL params, etc

@bnewbold bnewbold self-assigned this Dec 25, 2023
@bnewbold bnewbold force-pushed the bnewbold/sdk-lexicon branch from e4b3fe9 to 4cc401b Compare January 16, 2024 08:33
@bnewbold bnewbold force-pushed the bnewbold/sdk-lexicon branch from 22fe362 to 2a5ee98 Compare February 29, 2024 08:53
@bnewbold bnewbold force-pushed the bnewbold/sdk-lexicon branch from 93b3a64 to cfbbce3 Compare August 14, 2024 03:54
@bnewbold bnewbold marked this pull request as ready for review September 8, 2024 16:52
snarfed added a commit to snarfed/lexrpc that referenced this pull request Sep 22, 2024
@snarfed
add lexicon schema validation test data from indigo
12eea85 
specifically, from:
* `atproto/lexicon/testdata/` and `atproto/data/testdata/` at bluesky-social/indigo@5c2cb48
* `atproto/syntax/testdata/` at bluesky-social/indigo#420 / bluesky-social/indigo@f19a2a4

via the MIT and Apache 2.0 licenses (indigo is dual licensed)

this commit adds these files unchanged from the commits above.
@bnewbold bnewbold merged commit 5668e31 into main Oct 4, 2024
7 checks passed
@bnewbold bnewbold deleted the bnewbold/sdk-lexicon branch October 4, 2024 00:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Vincentroland Vincentroland Vincentroland approved these changes

Assignees

@bnewbold bnewbold

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bnewbold @Vincentroland