Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Unrecognized lexicon type" for every record in my BlueSky download #847

Open
bobg opened this issue Nov 24, 2024 · 2 comments
Open

"Unrecognized lexicon type" for every record in my BlueSky download #847

bobg opened this issue Nov 24, 2024 · 2 comments

Comments

@bobg
Copy link

bobg commented Nov 24, 2024

Hello! I am trying to parse the contents of the repo.car file that I downloaded from BlueSky just now, using the sample code at https://docs.bsky.app/blog/repo-export#parse-records-from-car-file-as-json. I'm getting a util.ErrUnrecognizedType from Repo.GetRecord for every record in the archive.

Here's my code. Am I missing a step? Thanks in advance.

	...
	r, err := repo.ReadRepoFromCar(ctx, os.Stdin)
	if err != nil {
		return errors.Wrap(err, "reading repo from stdin")
	}

	err = r.ForEach(ctx, "", func(k string, v cid.Cid) error {
		_, rec, err := r.GetRecord(ctx, k)
		if errors.Is(err, util.ErrUnrecognizedType) {
			log.Printf("Skipping record %s: %s", k, err)
			return nil
		}
		...
	}
	...
@bnewbold
Copy link
Collaborator

Yup! Many of the default methods in the indigo codebase assume that there are defined structs/types for the records being marshalled/unmarshalled. This "registration" happens when relevant packages (like the api/bsky package) are imported.

It is also possible to pull out data as bytes, and unmarshall to generic map[string]any objects, using the atproto/data package. Check out the goat code for examples, eg:
https://github.com/bluesky-social/indigo/blob/main/cmd/goat/repo.go#L239

definitely need docs around all this, and probably refactor things to default to "schema/type agnostic" with the registered types being more explicit?

@bobg
Copy link
Author

bobg commented Nov 24, 2024

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants