get font data per word NOT only per block

[gustavo9b]

Hello everyone,

I noticed we can only extract font information for blocks, but not per word. Is it possible to get bbox and font data per word-token?
I need to explode at a word-token level, but I only get block and, if I do explode the blocks, I do not get the precise coordinates of the words.

If this feature is not available I would suggest to add it, it is very useful to have info per word.

PS Great work you guys, awesome package!
Best,
Gustavo Bernardino

[aleem75321]

Yes you can use Rawdict to get characters
blocks=page.get_text("rawdict",flags=11)["blocks"]

for Word
blocks=page.get_text("word",flags=11)

for more detailed read this
https://pymupdf.readthedocs.io/en/latest/app1.html#blocks

[JorjMcKie]

The "words" and "blocks" extraction variants are explicitly intended as high-speed extractions - the range in the same category as plain text variant "text".
All three have in common that text meta data differences are of no concern: text parts may have any font, font size, color, font weight or style.
As amazing as this may be, this is true for "words" as well, for example lookhereandhere. The latter string will be one item in the "words" variant, but be distributed across 4 different spans in "dict" and "rawdict". In real PDF text, I could have easily made an example with different colors and font sizes etc.

So I am afraid that there is no way to comply with this request without giving up the sense in life of this extraction variant. And this thought didn't yet even touch the question about performance ...

[gustavo9b]

Thanks for your answer @JorjMcKie ,

So that I explain myself better.

When I perform the "dict" extraction with:

blocks = page.get_text("dict", flags=11)["blocks"]
I do get meta-data for lines, spans and flags. This method is what I'm interested in, but at word level.

I deal with very intricate documents, exploding the meta-data down to word-token allows me to develop complex classification algorithms, sectioning the document in titles, subtitles and so on with greater precision. I wouldnt mind for speed if I can let the documents be cracked in the background.

What I am doing is getting the bbox for each word (with extractWORDS method) and then merging the meta-data (obtained in "dict" method) to it, using y0 and page number as common id.

If this request is too absurd ok, but please give it a thought, for advanced algorithms with ML could be very useful.

Best regards,
Gustavo Bernardino

[JorjMcKie]

If speed is not the main concern, something could be built on top of "rawdict". It anyway shares the dictionary structure with "dict", the "chars" list of dictionaries below "spans" being the only difference.
Something like this (modulo typos ...) adds a list of word dictionaries side-by-side to the "chars" list:

text_items = page.get_text("rawdict", flags=fitz.TEXTFLAGS_TEXT)  # only retrieve text blocks
for block in text_tems["blocks"]:
    for line in block["lines"]:
        for span in line["spans"]:  # we add a words list of dicts to this span
            words = []  # will contain list of word dicts
            word = ""  # stores a single word
            word_bbox = fitz.EMPTY_RECT()  # stores bbox of this word
            for char in span["chars"]:  # essentially, we skip the spaces to form words
                c = char["c"]
                if c != " ":  # add this char to word and its bbox
                    word += c
                    word_bbox |= char["bbox"]
                elif word:  # space encountered (note: 'word' could still be empty)
                    wdict = {"bbox": word_bbox, "word": word}
                    words.append(wdict)
                    word_bbox = fitz.EMPTY_RECT()
                    word = ""
            if word != "":  # word unfinished yet?
                words.append({"bbox": word_bbox, "word": word})
            span["words"] = words

[JorjMcKie]

Just to close the open loop:
My above example with a "word" consisting of 4 parts with different text properties would be delivered as 4 separate words in 4 separate spans ... although they have adjacent coordinates.
Or put the other word round: my above word will not be contained in any span.

[gustavo9b]

That is a great solution, thanks! Had no idea this could be done. Will check if it is faster than what I have come up with.

Kind regards,
Gustavo