Skip to content
This repository has been archived by the owner on Sep 6, 2023. It is now read-only.

Use kuromoji from maven repository. #58

Merged
merged 1 commit into from
Nov 18, 2022
Merged

Use kuromoji from maven repository. #58

merged 1 commit into from
Nov 18, 2022

Conversation

zaimy
Copy link
Contributor

@zaimy zaimy commented Nov 16, 2022

Hello!

The following error occurs when running version 0.4.0. It seems that the kuromoji classes are not properly referenced using the pre-built jar file bundled with the repository.

Exception in thread "main" java.lang.NoClassDefFoundError: Lorg/atilika/kuromoji/Tokenizer;
        at java.lang.Class.getDeclaredFields0(Native Method)
        at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
        at java.lang.Class.getDeclaredFields(Class.java:1916)
        at org.apache.beam.sdk.util.common.ReflectHelpers.declaredMembersWithAnnotation(ReflectHelpers.java:313)
        at org.apache.beam.sdk.util.common.ReflectHelpers.declaredFieldsWithAnnotation(ReflectHelpers.java:289)
        at org.apache.beam.sdk.transforms.reflect.DoFnSignatures.analyzeStateDeclarations(DoFnSignatures.java:2274)
        at org.apache.beam.sdk.transforms.reflect.DoFnSignatures.parseSignature(DoFnSignatures.java:532)
        at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
        at org.apache.beam.sdk.transforms.reflect.DoFnSignatures.getSignature(DoFnSignatures.java:300)
        at org.apache.beam.sdk.transforms.ParDo.validate(ParDo.java:614)
        at org.apache.beam.sdk.transforms.ParDo.of(ParDo.java:403)
        at com.github.yuiskw.beam.Kuromoji4BigQuery.main(Kuromoji4BigQuery.java:133)
Caused by: java.lang.ClassNotFoundException: org.atilika.kuromoji.Tokenizer
        at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        ... 12 more

As of #51, kuromoji 0.7.7 was not available in the maven repository. However, I have discovered that it is now available again. And I have confirmed that morphological analysis can be performed on Dataflow using version 0.4.1, which reflects the changes in this pull request. :)

@zaimy zaimy requested a review from yu-iskw as a code owner November 16, 2022 02:46
Copy link
Owner

@yu-iskw yu-iskw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you for fixing this.

@yu-iskw yu-iskw merged commit a944542 into yu-iskw:master Nov 18, 2022
@zaimy
Copy link
Contributor Author

zaimy commented Nov 25, 2022

Thank you for your review!

@zaimy zaimy deleted the use-kuromoji-from-mvn-repository branch November 25, 2022 02:23
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants