Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration of ALTO metadata #12

Closed
11 tasks done
mikegerber opened this issue Apr 7, 2022 · 19 comments
Closed
11 tasks done

Integration of ALTO metadata #12

mikegerber opened this issue Apr 7, 2022 · 19 comments
Assignees

Comments

@mikegerber
Copy link
Member

mikegerber commented Apr 7, 2022

  • Should this be in here? Or in "codename altotool"?

  • What info would be relevant? What would be metadata, what would be data (count words?)

  • Include metadata from the Description section

  • Include descriptive statistics for the Layout section etc.

    • E.g. word count
    • E.g. mean STRING WC (confidence)
  • When that's done review the comments below for things we may have missed

  • Test using all available versions of ALTO

  • NER annotated ALTO should at least be identifiable

  • Include ALTO version/namespace

  • <LayoutTag ID="layouttag-marginalia" LABEL="marginalia"/>

  • Any language infos?

  • Update README that we now support ALTO

@cneud
Copy link
Member

cneud commented Apr 7, 2022

Or the tool needs a new name? Let's discuss what makes sense, e.g. libraries for parsing vs. integration

Some initial ideas on what would be relevant in ALTO:

  • Processing provenance (which software was used to produce ALTO)
  • OCR confidences
  • count of words, lines
  • presence of particular elements (e.g. graphical elements)
  • distribution of ALTO elements over a document

@mikegerber
Copy link
Member Author

The other thing we should consider: ALTO and images (+ their metadata) concern document pages, not documents. So this is either

  • to be aggregated over the whole document
  • or treated as a separate data source (indexed by document + page)
  • or both

@cneud
Copy link
Member

cneud commented Apr 7, 2022

I am leaning towards

treated as a separate data source (indexed by document + page)

as only this would allow us the most granular analysis down to page level (e.g. which pages are outliers concerning {insert feature here}, but again, let's discuss this further with the team.

@mikegerber
Copy link
Member Author

mikegerber commented Apr 8, 2022

I'm working on an "altotool", using the same techniques we use in modstool to stuff interesting stuff into a pandas DataFrame. This will be indexed by page, so any analysis to be done document-wise needs to aggregate this in meaningful way then (e.g. merge processing info, build sums of line counts etc.)

For some of this stuff there needs to done some aggregating over the page already, e.g. mean/median OCR confidence etc. (not that I think this info will be particularily useful but we shall see.)

@mikegerber
Copy link
Member Author

* distribution of ALTO elements over a document

@cneud Please clarify this.

@cneud
Copy link
Member

cneud commented Apr 8, 2022

There are and have been many use cases for info extracted from ALTO which led me to work on https://github.com/cneud/alto-tools (feel free to reuse what can be).

Please clarify this.

Think e.g. on which pages within a document do certain elements occur vs other pages

@cneud
Copy link
Member

cneud commented Apr 8, 2022

mean/median OCR confidence

When certain pages within a document are outliers wrt to the confidence scores, this would be useful to identify and investigate for example

@mikegerber mikegerber self-assigned this Apr 11, 2022
@mikegerber
Copy link
Member Author

And yeah it needs a name. While I am happy with the innovative name of "modstool", with ALTO functionality it's a bit different

@mikegerber mikegerber changed the title Consider integration of ALTO metadata Integration of ALTO metadata Apr 11, 2022
@mikegerber
Copy link
Member Author

A first version (branch feat/alto) extracts some of this information, e.g.:

Description_MeasurementUnit                                                                              pixel                  
Description_OCRProcessing_ocrProcessingStep0_processingDateTime                                     2016-08-07
Description_OCRProcessing_ocrProcessingStep0_processingSoftware_softwareCreator                          ABBYY
Description_OCRProcessing_ocrProcessingStep0_processingSoftware_softwareName           ABBYY FineReader Engine
Description_OCRProcessing_ocrProcessingStep0_processingSoftware_softwareVersion                             11
Layout_Page_ID                                                                                           Page1
Layout_Page_PHYSICAL_IMG_NR                                                                                  1
Layout_Page_HEIGHT                                                                                        2436
Layout_Page_WIDTH                                                                                         1404
Layout_Page_Page-count                                                                                       1
Layout_Page_TopMargin-count                                                                                  1
Layout_Page_LeftMargin-count                                                                                 1
Layout_Page_RightMargin-count                                                                                1
Layout_Page_BottomMargin-count                                                                               1
Layout_Page_PrintSpace-count                                                                                 1
Layout_Page_TextBlock-count                                                                                  1
Layout_Page_Shape-count                                                                                      1
Layout_Page_Polygon-count                                                                                    1
Layout_Page_TextLine-count                                                                                  40
Layout_Page_String-count                                                                                   386
Layout_Page_SP-count                                                                                       345
Layout_Page_HYP-count                                                                                        8
alto_file                                                                          alto/734008031/00000035.xml
Layout_Page_GraphicalElement-count                                                                         NaN
Layout_Page_Illustration-count                                                                             NaN
Layout_Page_ComposedBlock-count                                                                            NaN

This includes some counts of elements (*-count) and also selected attribute values (e.g. Layout_Page_HEIGHT), more to come.

A bit of a stumbling block is the diversity of ALTO variants we have, so I am going to rework this not to use a fixed XML namespace.

@mikegerber
Copy link
Member Author

From the first runs I estimate about 48h to run this over all of our (5 million?) ALTO files, which is fine with me.

@mikegerber
Copy link
Member Author

mikegerber commented May 23, 2022

Latest version in the feature branch now includes descriptive statistics on the word OCR confidence (//alto:String/@WC as an XPath expression):

Layout_Page_//alto:String/@WC-mean                                                                    0.639988
Layout_Page_//alto:String/@WC-median                                                                    0.6355
Layout_Page_//alto:String/@WC-std                                                                     0.137451
Layout_Page_//alto:String/@WC-min                                                                         0.22
Layout_Page_//alto:String/@WC-max                                                                            1

@mikegerber
Copy link
Member Author

Latest version now includes the column alto_xmlns, which is/translates to the ALTO version used.

Examples from my test data:

alto/PPN636777308/00000002.xml             http://schema.ccs-gmbh.com/ALTO
alto/734008031/00000020.xml       http://www.loc.gov/standards/alto/ns-v2#
alto/734008031/00000054.xml       http://www.loc.gov/standards/alto/ns-v2#
alto/734008031/00000098.xml       http://www.loc.gov/standards/alto/ns-v2#
alto/734008031/00000106.xml       http://www.loc.gov/standards/alto/ns-v2#
                                                    ...                   
alto/749782137/00000554.xml       http://www.loc.gov/standards/alto/ns-v2#
alto/749782137/00000252.xml       http://www.loc.gov/standards/alto/ns-v2#
alto/749782137/00000004.xml       http://www.loc.gov/standards/alto/ns-v2#
alto/749782137/00000849.xml       http://www.loc.gov/standards/alto/ns-v2#
alto/weird-ns/00000007.xml              http://www.loc.gov/standards/alto/
Name: alto_xmlns, Length: 1314, dtype: object

@mikegerber
Copy link
Member Author

NER annotated ALTO at SBB looks like this:

There's an alto:Tags tag that contains the entities (ns0 being ALTO here):

  <ns0:Tags>
    <ns0:NamedEntityTag ID="PER0" LABEL="Pentlings"/>
    <ns0:NamedEntityTag ID="LOC1" LABEL="Pentling"/>
    <ns0:NamedEntityTag ID="LOC2" LABEL="Hamm"/>
    <ns0:NamedEntityTag ID="PER4" LABEL="Hofes Pentling"/>
    <ns0:NamedEntityTag ID="LOC5" LABEL="Hofs Pentling"/>
    <ns0:NamedEntityTag ID="LOC7" LABEL="Hilbeck"/>
    <ns0:NamedEntityTag ID="PER8" LABEL="Hoff"/>
    <ns0:NamedEntityTag ID="PER9" LABEL="L i b e r"/>
    <ns0:NamedEntityTag ID="PER10" LABEL="Jhesu Christi"/>
  </ns0:Tags>

alto:Strings then reference these:

            <ns0:String CONTENT="Hofes" HEIGHT="33" HPOS="914" TAGREFS="PER4" VPOS="1396" WC="0.5019999743" WIDTH="82"/>
            <ns0:SP HPOS="997" VPOS="1398" WIDTH="21"/>
            <ns0:String CONTENT="Pentling" HEIGHT="34" HPOS="1019" TAGREFS="PER4" VPOS="1398" WC="0.5337499976" WIDTH="129"/>
            <ns0:SP HPOS="1149" VPOS="1407" WIDTH="19"/>

@mikegerber
Copy link
Member Author

Latest master now counts the above NEs in Tags_NamedEntityTag-count.

@mikegerber
Copy link
Member Author

We now count all Strings with TAGREFS in Layout_Page_//alto:String[@TAGREFS]-count (Weird naming comes from the XPath expression used). Some tagged entities span multiple String elements, not sure if and what to do about that.

@mikegerber
Copy link
Member Author

mikegerber commented Jun 17, 2022

We now count all Strings with TAGREFS in Layout_Page_//alto:String[@TAGREFS]-count (Weird naming comes from the XPath expression used). Some tagged entities span multiple String elements, not sure if and what to do about that.

TAGREFS is also used in some ALTO files to reference LayoutTags in TextBlock elements (not Strings). So technically these counts could count reference tags that are not NamedEntityTags.

However, I don't think it's currently worth the effort to check if the TAGREFS actually reference NEs and just leave it this way until we need this checking. @labusch @cneud Opinions?

@mikegerber
Copy link
Member Author

Language attributes are LANG and the deprecated language:

@mikegerber
Copy link
Member Author

Language attributes are LANG and the deprecated language:

* https://www.loc.gov/standards/alto/v4/alto-4-3.xsd

* In my test data, only the deprecated `TextBlock/@language` tags are used

Moved this to #18.

@mikegerber
Copy link
Member Author

  • <LayoutTag ID="layouttag-marginalia" LABEL="marginalia"/>

I've deviced to ignore this for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants