Can preprocessing of documents be saved as text file to be read back in?

[asharm0662]

Hi,

I am following this tutorial. Under the 'Preprocessing of Documents' section, there are some functions to clean documents and write them into a document store.

Instead of writing the cleaned document text data to document store, can I write the cleaned as dictionary object to text file and read that file into memory? If yes, what is the best format to write the data JSON/dictionary?

Thank you in advance.

[bogdankostic]

Hi @asharm0662! The convert_files_to_dicts method in the tutorial you linked will return a list of dictionaries. You can easily save them as a json-file like this:

import json
with open(FILE_NAME, "w") as json_file:
    json.dump(dicts, json_file)

[asharm0662]

thank you @bogdankostic thats what i thought.