Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Libpostal parsing Indonesian addresses with poor accuracy #645

Open
AsfarHorani opened this issue Oct 25, 2023 · 4 comments
Open

Libpostal parsing Indonesian addresses with poor accuracy #645

AsfarHorani opened this issue Oct 25, 2023 · 4 comments

Comments

@AsfarHorani
Copy link

Hi!

I was checking out libpostal, and saw something that could be improved.


My country is

Pakistan but i was working on Indonesian data for a project

Here's how I'm using libpostal

Created a docket images


Here's what I did

Jl. Arif Rahman Hakim No.5, Mataram, Nusa Tenggara Bar, 83127, Indonesia


Here's what I got

Result:

{
"road": "jl. arif rahman hakim no.5 mataram nusa tenggara bar",
"postcode": "83127",
"country": "indonesia"
}

Here's what I was expecting

Result:
Street: Jl. Arif Rahman Hakim No.5
City: Mataram
State/Province: Nusa Tenggara Bar
Postal Code: 83127
Country: Indonesia

For parsing issues, please answer "yes" or "no" to all that apply.

  • Does the input address exist in OpenStreetMap?
    yes: https://www.openstreetmap.org/search?query=Jl.%20Arif%20Rahman%20Hakim%20Mataram#map=17/-8.59163/116.10771- Do all the toponyms exist in OSM (city, state, region names, etc.)?
    no
  • If the address uses a rare/uncommon format, does changing the order of the fields yield the correct result?
    no
  • If the address does not contain city, region, etc., does adding those fields to the input improve the result?
    Might be
  • If the address contains apartment/floor/sub-building information or uncommon formatting, does removing that help? Is there any minimum form of the address that gets the right parse?
    I dont know

Here's what I think could be improved

Update the data

@albarrentine
Copy link
Contributor

albarrentine commented Feb 14, 2024

think it has trouble with the "Bar" part (maybe listed differently in OSM/GeoNames, etc.). This works fine:

Jl. Arif Rahman Hakim No.5, Mataram, Nusa Tenggara, 83127, Indonesia

{
  "road": "jl. arif rahman hakim",
  "house_number": "no.5",
  "city": "mataram",
  "state_district": "nusa tenggara",
  "postcode": "83127",
  "country": "indonesia"
}

@AsfarHorani
Copy link
Author

But it was just a sample text that I shared with you. Classification is poor in 90% of the cases for Indonesian localities

@albarrentine
Copy link
Contributor

Will probably need to check the training data (in the address_parser cli it’s possible to type .print_features and then some test addresses and it will print out a JSON representation of what the model is doing for every word in the input and can test different formulations). Could be that the municipality names from OSM are substantially different or there’s some convention being used that is different from how things are tagged in OSM/GeoNames, etc. In some cases there are things that can be in preprocessing like extracting out a regex. For instance if all the cities in the test set had something like “Bar” at the end, and in the training set they did not, it’s easy to write a regex to remove that before parsing and then let the model handle the rest and optionally add it back later. If most of the test addresses are comma-separated you can also walk backward through the string parsing and keep adding one comma-separated phrase to the parse until something becomes inconsistent (two non-adjacent phrases labeled as “road”). If so, try throwing out that component and reparsing. Generally for something like an admin there’s often a reference database/search index to look it up.

@rjurney
Copy link

rjurney commented May 3, 2024

@AsfarHorani have you tried the new Senzing parsing model? Check out the README instructions. It may help, though that isn't clear from the latest metrics: https://github.com/Senzing/libpostal-data/blob/main/files/stats/v1.1.0/Parsing_comparison_v1_1_0.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants