This Github repository holds the supporting documents for the paper "Evaluation of search-enabled Pre-trained Large Language Models on retrieval tasks for the PubChem Database", currently available at https://www.biorxiv.org/content/10.1101/2024.08.15.608120v1.
Supporting file 1 contains additional details for each protocol, we describe our experiences with GPT-4o in more detail, highlighting values and limitations that are specific to each case. Please note that Protocol 3 and 4 are detailed as examples in the main manuscript.
Supporting file 2 contains snippets of the gold answer and the GPT response for each of the eight protocols. The GPT prompt is listed in Figures 1-8. For reference, the complete gold answers are provided in Supporting File 2.
Supporting file 3 contains the Gold answers from protocols 1-8, through running the protocol on the PubChem web interface.