Splitting File Function

[S-Hanly]

The extract_time function is SWEET! Can we use that to make a splitting_file function like this?

@StokesMIDE, I know you could make this function far better!

import os

def split_ide(doc, num_splits, directory_name='split_files', file_name='split'):
  """Split an IDE file (relatively) evenly"""
  table = endaq.ide.get_channel_table(doc)

  times = np.linspace(start=min(table.data.start),
                      stop=max(table.data.end),
                      num=num_splits+1
                      )
  if not os.path.exists(directory_name):
    os.makedirs(directory_name)

  i=0
  endaq.ide.extract_time(doc, 
                        out=directory_name+'/'+file_name+'_'+f'{i:05d}'+'.ide', 
                        end=times[i+1])

  for i in np.arange(2,len(times)-1):
    endaq.ide.extract_time(doc, 
                          out=directory_name+'/'+file_name+'_'+f'{i-1:05d}'+'.ide',
                          start=times[i-1],
                          end=times[i])
  i=len(times)-2
  endaq.ide.extract_time(doc, 
                        out=directory_name+'/'+file_name+'_'+f'{i:05d}'+'.ide', 
                        start=times[I])  

[StokesMIDE]

Sure. I'd implement it like the time extractor, though, which works directly on the raw IDE data.

What's the use case for the splitter now? Originally (ca. 2014), the idea was to send large files from remote locations, but that no longer seems to be much of an issue.

[S-Hanly]

It would be to then play well with endaq.batch which @CrepeGoat is working on. So someone could rip through generating many PSDs/PVSS/peak-accel etc. on segments of a larger file. I'm finding myself routinely helping folks with ~200 MB files now which we can do a PSD or something on the whole thing for sure, but it can mask certain harsh periods.