Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support running the evaluator component on simdjson #140

Open
7 tasks
jviotti opened this issue Nov 5, 2024 · 2 comments
Open
7 tasks

Support running the evaluator component on simdjson #140

jviotti opened this issue Nov 5, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@jviotti
Copy link
Member

jviotti commented Nov 5, 2024

This ticket aims to eventually be a master issue for tracking what's needed to support https://github.com/simdjson/simdjson. The idea is to support simdjson on the evaluator, while keeping the compiler JSON Toolkit only.

Phase 1: Abstract away JSON Toolkit

  • Create new header (evaluator_json.h?) that introduces a layer of indirection for type aliases and functions that we can #ifdef based on the chosen backend. All other parts of the evaluator should use this instead. We would even need to abstract how to loop over objects, etc which might get nasty?
  • Revisit number comparison instructions (like AssertionGreaterEqual) to NOT take JSON values as input

Phase 2: Bridge any interoperability requirements between simdjson and JSON Toolkit

  • Integrate simdjson into the build using vendorpull
  • Allow traversing a JSON Toolkit JSON Pointer on a simdjson JSON document. Probably needs to happen here in Blaze as a re-implementation of pointer traversal?
  • Implement an equality check helper between simdjson and JSON Toolkit

Phase 3: Enable simdjson

  • Add a new CMake option for selecting the desired JSON backend for the evaluator with options simdjson and jsontoolkit. Maybe BLAZE_EVALUATOR_BACKEND? Note Blaze would still always link to JSON Toolkit for using other of its stuff, like JSON Pointers
  • Implement a simdjson variant of the common abstraction interface

Resolved

  • Will we emit annotation values using simdjson or JSON Toolkit?
    • Resolution: Annotations are never stored, so we can keep using JSON Toolkit, otherwise we need to get into emitting simdjson from the compiler
@jviotti jviotti added the enhancement New feature or request label Nov 5, 2024
@jviotti jviotti changed the title Support simdjson Support running the evaluator component on simdjson Nov 5, 2024
@jviotti
Copy link
Member Author

jviotti commented Nov 5, 2024

cc @michaelmior @lemire

@jviotti
Copy link
Member Author

jviotti commented Nov 5, 2024

The clear first step would be to already start abstracting away the evaluator from JSON Toolkit, creating the layer of indirection mentioned above. Once all of the details about JSON Toolkit are in one place, then we can start thinking about implementing the same interface for another JSON implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant