NewDataFileReader returns all nil values on fields and ends with "Block read is unfinished"

[RoarkeRandall]

Here's my code (pretty much copy pasted from the example)...

                reader, err := avro.NewDataFileReader(fileName, avro.NewSpecificDatumReader())
		if err != nil {
			fmt.Println(err)
			return
		}
		for {
			obj := &PADirectJustListedItem{}
			ok, err := reader.Next(obj)
			if !ok {
				if err != nil {
					fmt.Println(err)
					return
				}
				break
			} else {
				fmt.Printf("%#v\n", obj)
			}
		}

ouput:
go run main.go
&main.PADirectJustListedItem{snapshotdate:(*int64)(nil), propertyid:(*int32)(nil), accountid:(*int32)(nil), bedrooms:(*int)(nil), bathrooms:(*string)(nil), finishedsqft:(*int)(nil), lotsizesqft:(*int)(nil), city:(*string)(nil), state:(*string)(nil), postalcode:(*string)(nil), propetyaddress:(*string)(nil), image1id:(*int64)(nil), image2id:(*int64)(nil), image3id:(*int64)(nil), manualimageid:(*int64)(nil), sellingpricedollarcnt:(*int32)(nil), realestatebrokerid:(*int32)(nil), daysonzillow:(*int32)(nil), multiplelistingservicecode:(*string)(nil), postingid:(*int32)(nil), postingdateinitial:(*int64)(nil), auditdatecreated:(*int64)(nil)}
&main.PADirectJustListedItem{snapshotdate:(*int64)(nil), propertyid:(*int32)(nil), accountid:(*int32)(nil), bedrooms:(*int)(nil), bathrooms:(*string)(nil), finishedsqft:(*int)(nil), lotsizesqft:(*int)(nil), city:(*string)(nil), state:(*string)(nil), postalcode:(*string)(nil), propetyaddress:(*string)(nil), image1id:(*int64)(nil), image2id:(*int64)(nil), image3id:(*int64)(nil), manualimageid:(*int64)(nil), sellingpricedollarcnt:(*int32)(nil), realestatebrokerid:(*int32)(nil), daysonzillow:(*int32)(nil), multiplelistingservicecode:(*string)(nil), postingid:(*int32)(nil), postingdateinitial:(*int64)(nil), auditdatecreated:(*int64)(nil)}
Block read is unfinished

I have a couple files to test with. All of them I'm able to use the avro tools to convert them to json and it works fine:

java -jar avro-tools-1.8.2.jar tojson part-m-00000.avro > 00001.json

[RoarkeRandall]

here's our schema... I think the problem relates to how null is being handled.

{
	"type":"record",
	"name":"QueryResult",
	"doc":"Sqoop import of QueryResult",
	"fields":[
		{"name":"snapshotdate","type":["null","long"],"default":null,"columnName":"snapshotdate","sqlType":"91"},
		{"name":"propertyid","type":["null","int"],"default":null,"columnName":"propertyid","sqlType":"4"},
		{"name":"accountid","type":["null","int"],"default":null,"columnName":"accountid","sqlType":"4"},
		{"name":"bedrooms","type":["null","int"],"default":null,"columnName":"bedrooms","sqlType":"5"},
		{"name":"bathrooms","type":["null","string"],"default":null,"columnName":"bathrooms","sqlType":"3"},
		{"name":"finishedsqft","type":["null","int"],"default":null,"columnName":"finishedsqft","sqlType":"4"},
		{"name":"lotsizesqft","type":["null","int"],"default":null,"columnName":"lotsizesqft","sqlType":"4"},
		{"name":"city","type":["null","string"],"default":null,"columnName":"city","sqlType":"12"},
		{"name":"state","type":["null","string"],"default":null,"columnName":"state","sqlType":"1"},
		{"name":"postalcode","type":["null","string"],"default":null,"columnName":"postalcode","sqlType":"1"},
		{"name":"propetyaddress","type":["null","string"],"default":null,"columnName":"propetyaddress","sqlType":"12"},
		{"name":"image1id","type":["null","long"],"default":null,"columnName":"image1id","sqlType":"-5"},
		{"name":"image2id","type":["null","long"],"default":null,"columnName":"image2id","sqlType":"-5"},
		{"name":"image3id","type":["null","long"],"default":null,"columnName":"image3id","sqlType":"-5"},
		{"name":"manualimageid","type":["null","long"],"default":null,"columnName":"manualimageid","sqlType":"-5"},
		{"name":"sellingpricedollarcnt","type":["null","int"],"default":null,"columnName":"sellingpricedollarcnt","sqlType":"4"},
		{"name":"realestatebrokerid","type":["null","int"],"default":null,"columnName":"realestatebrokerid","sqlType":"4"},
		{"name":"daysonzillow","type":["null","int"],"default":null,"columnName":"daysonzillow","sqlType":"4"},
		{"name":"multiplelistingservicecode","type":["null","string"],"default":null,"columnName":"multiplelistingservicecode","sqlType":"12"},
		{"name":"postingid","type":["null","int"],"default":null,"columnName":"postingid","sqlType":"4"},
		{"name":"postingdateinitial","type":["null","long"],"default":null,"columnName":"postingdateinitial","sqlType":"93"},
		{"name":"auditdatecreated","type":["null","long"],"default":null,"columnName":"auditdatecreated","sqlType":"93"}
	],
	"tableName":"QueryResult"
}

[serejja]

hi @RoarkeRandall, this repo is unfortunately abandoned as the maintainer has left and didn't leave push/merge rights to anyone. There's a maintained fork here - https://github.com/go-avro/avro

[crast]

@RoarkeRandall do you have them as public struct fields? From your output I suggest they don't, the go-avro reader cannot decode into private struct fields, it's a language/library limitation.

In either case, as mentioned above there is a maintained fork, and this branch might interest you:
go-avro/avro#7

[RoarkeRandall]

Sorry, I should have closed this awhle ago. Yes, that is the case