As a Data Importer I should be able to only import valid CSV files

[T-Dnzt]

Overview

The current Data Set import works well when the imported CSV files are in the expected format and the mapping is done properly. However, it lacks some validation steps to prevent the import of invalid CSV files and the mapping of fields to the wrong types.

This issue lists solutions for the problems encountered by the team when importing CSV files.

Improvements

• Switch to a Ruby CSV parser to make the steps below easier, and the whole process easier to test.
• Validate the CSV file format to ensure it will be properly imported into the system.
• Ensure one of the fields in the CSV is mapped to "Detailed call type" to enable the classification flow.
• Validate the mappings. For example, validate that if a field is mapped to the "Call Time" type, it contains a valid datetime and avoid trying to parse invalid data (which currently crashes the application).