Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avro Deserialization Breaks on Java 17 #683

Open
strygul opened this issue Jul 22, 2022 · 0 comments
Open

Avro Deserialization Breaks on Java 17 #683

strygul opened this issue Jul 22, 2022 · 0 comments

Comments

@strygul
Copy link

strygul commented Jul 22, 2022

Describe the bug
Broken deserialization of Avro schemas in Java 17

To Reproduce


import com.twitter.chill.{KryoPool, ScalaKryoInstantiator}
import org.apache.avro.Schema

object Test extends App {

  private val pool: KryoPool = ScalaKryoInstantiator.defaultPool
  val bytes = pool.toBytesWithClass(Schema.parse("""{
                                       |"type" : "record",
                                       |"namespace" : "Foo",
                                       |"name" : "Employee",
                                       |"fields" : [
                                       |{ "name" : "Name" , "type" : "string" },
                                       |{ "name" : "Age" , "type" : "int" }
                                       |]
                                       |}""".stripMargin))

  val deserialized = pool.fromBytes(bytes)
}

Expected behavior
Successful deserialization

Screenshots
N/A

Environment
Java Amazon Corretto 17.0.4
org.apache.avro 1.8.2
org.twitter.chill 0.10.0

Additional context
The code works on Java 8. However, due to the stronger encapsulation rules in the later versions of Java, private inner class constructors (org.apache.avro.Schema$IntSchema in this case) could not be accessed anymore in Java 17 without explicitly making them accessible with the java reflection API:

Exception in thread "main" com.esotericsoftware.kryo.KryoException: Error constructing instance of class: org.apache.avro.Schema$IntSchema
Serialization trace:
schema (org.apache.avro.Schema$Field)
fieldMap (org.apache.avro.Schema$RecordSchema)
	at com.twitter.chill.Instantiators$$anon$1.newInstance(KryoBase.scala:142)
	at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1139)
	at com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldSerializer.java:562)
	at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:538)
	at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
	at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
	at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)
	at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813)
	at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:161)
	at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39)
	at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
	at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
	at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)
	at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813)
	at com.twitter.chill.SerDeState.readClassAndObject(SerDeState.java:61)
	at com.twitter.chill.KryoPool.fromBytes(KryoPool.java:94)
	at Test$.delayedEndpoint$Test$1(Test.scala:22)
	at Test$delayedInit$body.apply(Test.scala:6)
	at scala.Function0.apply$mcV$sp(Function0.scala:39)
	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
	at scala.App.$anonfun$main$1$adapted(App.scala:80)
	at scala.collection.immutable.List.foreach(List.scala:431)
	at scala.App.main(App.scala:80)
	at scala.App.main$(App.scala:78)
	at Test$.main(Test.scala:6)
	at Test.main(Test.scala)
Caused by: java.lang.IllegalAccessException: class com.twitter.chill.Instantiators$ cannot access a member of class org.apache.avro.Schema$IntSchema with modifiers "public"
	at java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:392)
	at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:674)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:489)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
	at com.twitter.chill.Instantiators$.$anonfun$normalJava$1(KryoBase.scala:174)
	at com.twitter.chill.Instantiators$$anon$1.newInstance(KryoBase.scala:139)
	... 26 more

It has to do with the call to com.esotericsoftware.reflectasm that utilizes the following code:

val access = ConstructorAccess.get(Class.forName("org.apache.avro.Schema$StringSchema"))
 access.newInstance

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant