grobid

Python library for serializing GROBID TEI XML to dataclasses

Installation

Use pip to install:

$ pip install grobid
$ pip install grobid[json] # for JSON serializable dataclass objects

You can also download the .whl file from the release section:

$ pip install *.whl

Usage

Client

In order to convert an academic PDF to TEI XML file, we use GROBID's REST services. Specifically the processFulltextDocument endpoint.

from pathlib import Path
from grobid.models.form import Form, File
from grobid.models.response import Response

pdf_file = Path("<your-academic-article>.pdf")
with open(pdf_file, "rb") as file:
    form = Form(
        file=File(
            payload=file.read(),
            file_name=pdf_file.name,
            mime_type="application/pdf",
        )
    )
    c = Client(base_url="<base-url>", form=form)
    try:
        xml_content = c.sync_request().content  # TEI XML file in bytes
    except GrobidClientError as e:
        print(e)

where base-url is the URL of the GROBID REST service

You can use https://cloud.science-miner.com/grobid/ to test

Form

The Form class supports most of the optional parameters of the processFulltextDocument endpoint.

Parser

If you want to serialize the XML content, we can use the Parser class to create dataclasses objects.

Not all of the GROBID annoation guidelines are met, but compliance is a goal. See #1.

from grobid.tei import Parser

xml_content: bytes
parser = Parser(xml_content)
article = parser.parse()
article.to_json()  # raises RuntimeError if extra require 'json' not installed

where xml_content is the same as in Client section

Alternately, you can load the XML from a file:

from grobid.tei import Parser

with open("<your-academic-article>.xml", "rb") as xml_file:
  xml_content = xml_file.read()
  parser = Parser(xml_content)
  article = parser.parse()
  article.to_json()  # throws RuntimeError if extra require 'json' not installed

We use orjson to provide a method to_json to serialize the dataclasses into JSON. By default, orjson isn't installed, use pip install grobid[json].

License

MIT

Contributing

You are welcome to add missing features by submitting a PR, however, I won't be accepting any requests other than GROBID annotation compliance.

Disclaimer

This module was originally part of a group university project, however, all the code and tests was also authored by me.