Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

T003: Missing data related to the mode "Road" and mode "Total Inland FT" #61

Open
hlinero opened this issue Feb 9, 2021 · 0 comments
Open
Assignees

Comments

@hlinero
Copy link
Member

hlinero commented Feb 9, 2021

Issue regarding https://stats.oecd.org/Index.aspx?DataSetCode=ITF_GOODS_TRANSPORT#

Rationale 1: Mode "Road" and Vehicle Type (VT) "All" is broken down into two VTs: "Road freight transport for hire and reward" and "Road freight transport on own account." There are years in which the "All" VT is presented but one if its components is missing.

Rationale 2: Just as mentioned before, the "Inland" mode has "All" as VT. However, the components that make up the "All" include the VT "Pipeline," "Inland Waterway," "Rail," and "Road ALL". There are years in which the "All" VT is presented but one if its components is missing.

Below is the list of countries
Albania:
- 1970 - 2002 --> Road data is missing

Belarus:
- 1970 - 1995 --> Pipeline data is missing

Bosnia-Herzegovina:
- 1998 - 2006 and 2010 --> Road data is missing

Bulgaria:
- 1970 - 1977 --> Pipeline data is missing

Canada:
- 1970 - 1994 and 2014-2018 --> Multiple data is missing

China:
- 1970 - 1977, 1979, 1981 - 1984, and 1986 - 1989 --> Road data is missing

Croatia
- 1970 - 1995 --> Multiple data is missing

Czech Republic:
- 1970 - 1992 --> Road data is missing

Denmark:
- 1971 - 1983 --> Pipeline data is missing

Greece:
- 2018

India:
- 1970 - 2004 and 2013 - 2017

Ireland:
- 1979
- Mode "Road" and VT "All" for range 1979

Japan
- 2018

Korea:
- 2018

Latvia:
- 1970 - 1994

Liechtenstein:
- 2005 - 2006

Serbia, Republic of:
- 1997 - 2018
- Mode "Road" and VT "All" for range 1997 - 2018

Spain:
- 2017 - 2018

Switzerland:
- 2018

Ukraine:
- 1990 - 2008
- Mode "Road" and VT "All" for range 1990 - 2008

United Kingdom:
- 2013 - 2018

United States:
- 2012 - 2018

Currently, the issue is handled in the code https://github.com/transportenergy/database/blob/master/item/historical/scripts/T003.py

The procedures to fix this problem are the following:

  1. ITF correct these errors in their data and republish correct data https://stats.oecd.org/Index.aspx?DataSetCode=ITF_GOODS_TRANSPORT#
  2. Implement code change https://github.com/transportenergy/database/blob/master/item/historical/scripts/T003.py removing the cleaning scripts associated with this problem
  3. Regenerate a new merged dataset
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants