Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

backwards and forward schema evolution #117

Open
sudeepdino008 opened this issue Apr 18, 2023 · 0 comments
Open

backwards and forward schema evolution #117

sudeepdino008 opened this issue Apr 18, 2023 · 0 comments

Comments

@sudeepdino008
Copy link

sudeepdino008 commented Apr 18, 2023

I was trying out erlavro for schema evolution. I'm not sure what the stance on evolution is here. Is it not supported? Or is there a way to support it that I'm not aware of. For example:

the schema (block-ethereum.avsc) has a lot of fields deleted (in contrast to the data provided which was encoded using an earlier version). No additional fields are added or modification done on any of the retained fields in this new schema:

{
   "type":"record",
   "namespace":"com.covalenthq.brp.avro",
   "name":"ReplicationSegment",
   "fields":[
      {
         "name":"startBlock",
         "type":"long"
      },
      {
         "name":"endBlock",
         "type":"long"
      },
      {
         "name":"elements",
         "type":"long"
      },
      {
         "name":"codecVersion",
         "type":"double",
         "default":0.33
      }
   ]
}

the encoded binary is here

when i try running the decoder:

iex> {:ok, scheman} = Avrora.Schema.Name.parse("block-ethereum")

iex> scheman1 = %Avrora.Schema{full_name: scheman.name}

iex> {:ok, schema} = Avrora.Resolver.resolve(scheman1.full_name)

iex> :avro_binary_decoder.decode(specimen, schema.full_name, schema.lookup_table, Avrora.AvroDecoderOptions.options())

** (MatchError) no match of right hand side value: {%{"codecVersion" => 4.3256427312130535e-37, "elements" => 24, "endBlock" => 66, "startBlock" => 1}, <<97, 100, 54, 99, 57, 57, 54, 51, 97, 56, 55, 52, 56, 48, 54, 97, 51, 102, 50, 51, 51, 49, 49, 50, 50, 99, 56, 50, 99, 54, 50, 55, 48, 49, 98, 55, 54, 48, 55, 50, 98, 48, 100, 55, 55, 50, 49, 56, ...>>}
    (erlavro 2.9.8) /Users/sudeep/repos/rudder/deps/erlavro/src/avro_binary_decoder.erl:66: :avro_binary_decoder.decode/4
    iex:6: (file)

How can i support backwards of forward evolution of schema here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant