Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RCFileProtobufInputFormat not work in hive 0.13. I need help, thanks a lot~~~ #438

Open
bluezone2015 opened this issue Apr 15, 2015 · 0 comments

Comments

@bluezone2015
Copy link

--------------------------create table first:--------------------------
create external table test_rc
row format serde "com.twitter.elephantbird.hive.serde.ProtobufDeserializer"
with serdeproperties (
"serialization.class"="com.galaxy.example.seqfile.proto.AddressBookProtos$Person")
stored as
inputformat "com.twitter.elephantbird.mapred.input.RCFileProtobufInputFormat"
outputformat "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat";

--------------------------then I load data to hive table:--------------------------
load data local inpath '/root/galaxy-cdh-client/hadoop/bin/part-r-00000.rc' overwrite into table test_rc;

--------------------------exec the hql:--------------------------
select id,name from test_rc;

--------------------------then I encounter the exception:--------------------------
Diagnostic Messages for this Task:
Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable org.apache.hadoop.hive.serde2.columnar.BytesRefArrayWritable@141ed711
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:195)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing writable org.apache.hadoop.hive.serde2.columnar.BytesRefArrayWritable@141ed711
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:533)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177)
... 8 more
Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.columnar.BytesRefArrayWritable cannot be cast to org.apache.hadoop.io.BytesWritable
at com.twitter.elephantbird.hive.serde.ProtobufDeserializer.deserialize(ProtobufDeserializer.java:56)
at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.readRow(MapOperator.java:154)
at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.access$200(MapOperator.java:127)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:508)
... 9 more

@bluezone2015 bluezone2015 changed the title RCFProtobufInputFormat not work in hive 0.13,I need help. RCFileProtobufInputFormat not work in hive 0.13. I need help, thanks a lot~~~ Apr 15, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant