URL access fails during csv batch processing, but the log shows success

[daminthegoat]

URL access fails during csv batch processing, but the log shows success

[dosubot[bot]]

Hi @daminthegoat! I'm here to help you with any bugs, questions, or contributions. Let's work on resolving this together!

To resolve the discrepancy between the log status showing success and the actual URL access failure during CSV batch processing, ensure that the checkAnnotationBatchImportProgress function is correctly handling and reporting the job status. You might want to add more detailed logging or error messages to capture any issues more precisely.

Here is the relevant code snippet that handles the batch processing and status checking:

const checkProcess = async (jobID: string) => {
  try {
    const res = await checkAnnotationBatchImportProgress({ jobID, appId })
    setImportStatus(res.job_status)
    if (res.job_status === ProcessStatus.WAITING || res.job_status === ProcessStatus.PROCESSING)
      setTimeout(() => checkProcess(res.job_id), 2500)
    if (res.job_status === ProcessStatus.ERROR)
      notify({ type: 'error', message: `${t('appAnnotation.batchModal.runError')}` })
    if (res.job_status === ProcessStatus.COMPLETED) {
      notify({ type: 'success', message: `${t('appAnnotation.batchModal.completed')}` })
      onAdded()
      onCancel()
    }
  }
  catch (e: any) {
    notify({ type: 'error', message: `${t('appAnnotation.batchModal.runError')}${'message' in e ? `: ${e.message}` : ''}` })
  }
}

const runBatch = async (csv: File) => {
  const formData = new FormData()
  formData.append('file', csv)
  try {
    const res = await annotationBatchImport({
      url: `/apps/${appId}/annotations/batch-import`,
      body: formData,
    })
    setImportStatus(res.job_status)
    checkProcess(res.job_id)
  }
  catch (e: any) {
    notify({ type: 'error', message: `${t('appAnnotation.batchModal.runError')}${'message' in e ? `: ${e.message}` : ''}` })
  }
}

Ensure that the annotationBatchImport and checkAnnotationBatchImportProgress functions are correctly implemented and handle all possible error scenarios. Additionally, you might want to add more detailed logging within these functions to capture any issues that might not be immediately apparent from the current error messages ^[1].

For more detailed insights into URL access failures, you can leverage the Dify platform's observability feature by integrating with LangSmith. This integration allows you to track various trace information, including tool calls, inputs, outputs, error messages, and associated file URLs, which can help identify and analyze any issues related to URL access during batch processing ^[2][3].

To continue talking to Dosu, mention @dosu.