Support for Schema Evolution

[caevv]

Does it have support for avro schema evolution? Providing reader and writer schema?

Example on different library:
actgardner/gogen-avro#93 (comment)

And overall topic related articles:
https://martin.kleppmann.com/2012/12/05/schema-evolution-in-avro-protocol-buffers-thrift.html
https://docs.confluent.io/5.2.2/schema-registry/index.html#schema-management

[nrwiersma]

Hi,

It does not yet. In its current state it is mostly aimed at stream processing where a registry is used to manage this.

I am currently working on the IPC/RPC part of the spec, and will need to support this for that, so it is coming.

[sebnyberg]

I think it's perfectly reasonable to not add schema evolution do to the sheer complexity of the feature.

Based on actgardners blog post on the topic, how it required the implementation of a virtual machine to get it to work, and how linkedin/goavro does not support it either, I can see how a decision to not implement it would well-grounded.

[nrwiersma]

Thanks for the opinion. You are correct that it is fairly complex feature, one that I have looked at with no solid answers as yet. I think it is do-able, but would need major changes to schema and decoder to allow for some form of composite schema.

I think it would be great to support IPC which would need this, but given the fact that gRPC is pretty much defacto, even with its failings, I am not sure anyone would actually use Avro IPC. Given that I am not sure the effort would ultimately be worth it, but perhaps I am wrong.

[redaLaanait]

I am currently experimenting with Avro in an event-sourced service, where events are expected to have a longer lifespan. In this scenario, it seems essential to have support for schema evolution.

I tried to work on it and made a POC: #329

@nrwiersma I’d like to receive your feedback and hopefully find a way to support this feature.

Meanwhile, I'll keep polishing my implementation and using my fork.

Thanks.