-
Notifications
You must be signed in to change notification settings - Fork 424
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Libpostal parsing Indonesian addresses with poor accuracy #645
Comments
think it has trouble with the "Bar" part (maybe listed differently in OSM/GeoNames, etc.). This works fine: Jl. Arif Rahman Hakim No.5, Mataram, Nusa Tenggara, 83127, Indonesia
|
But it was just a sample text that I shared with you. Classification is poor in 90% of the cases for Indonesian localities |
Will probably need to check the training data (in the address_parser cli it’s possible to type .print_features and then some test addresses and it will print out a JSON representation of what the model is doing for every word in the input and can test different formulations). Could be that the municipality names from OSM are substantially different or there’s some convention being used that is different from how things are tagged in OSM/GeoNames, etc. In some cases there are things that can be in preprocessing like extracting out a regex. For instance if all the cities in the test set had something like “Bar” at the end, and in the training set they did not, it’s easy to write a regex to remove that before parsing and then let the model handle the rest and optionally add it back later. If most of the test addresses are comma-separated you can also walk backward through the string parsing and keep adding one comma-separated phrase to the parse until something becomes inconsistent (two non-adjacent phrases labeled as “road”). If so, try throwing out that component and reparsing. Generally for something like an admin there’s often a reference database/search index to look it up. |
@AsfarHorani have you tried the new Senzing parsing model? Check out the README instructions. It may help, though that isn't clear from the latest metrics: https://github.com/Senzing/libpostal-data/blob/main/files/stats/v1.1.0/Parsing_comparison_v1_1_0.md |
Hi!
I was checking out libpostal, and saw something that could be improved.
My country is
Pakistan but i was working on Indonesian data for a project
Here's how I'm using libpostal
Created a docket images
Here's what I did
Here's what I got
Result:
{
"road": "jl. arif rahman hakim no.5 mataram nusa tenggara bar",
"postcode": "83127",
"country": "indonesia"
}
Here's what I was expecting
Result:
Street: Jl. Arif Rahman Hakim No.5
City: Mataram
State/Province: Nusa Tenggara Bar
Postal Code: 83127
Country: Indonesia
For parsing issues, please answer "yes" or "no" to all that apply.
yes: https://www.openstreetmap.org/search?query=Jl.%20Arif%20Rahman%20Hakim%20Mataram#map=17/-8.59163/116.10771- Do all the toponyms exist in OSM (city, state, region names, etc.)?
no
no
Might be
I dont know
Here's what I think could be improved
Update the data
The text was updated successfully, but these errors were encountered: