RuntimeWarning & ValueError when using endaq.plot.octave_spectrogram()

[MarcelEckh]

The Webinar I'm referring to: link
I can't attach the Jupyter Lab script nor the csv file, however the csv file is available here
I'm using a Jupyter Lab script with Python 3.9.11 (installed with asdf)

I measured vibration and I now want to analyze the data using the endaq python library. Unfortunately, I can't use the endaq.plot.octave_spectrogram() method. When I realized that (until now) only this method is not working I downloaded the file from the introduction Webinar for the library and copied the code as well. Even then, with the exact same code and data I get these Errors. I tried other methods which are mentioned previously in the Webinar and they work fine. Below is a minimum working example with the mentioned csv file and the endaq.plot.plots.rolling_min_max_envelope() method. I also added endaq.plot.octave_spectrogram() which is not working for me.
I also tested this on my private Windows PC with the same Python Version which was not installed with asdf. Same result. Every mehtod I tested works except endaq.plot.octave_spectrogram().
Yes, the csv file is in the same folder, I also tested it with only using the link like in the Webinar... no difference.

Minimum example:

# working part
import endaq

import plotly.express as px
import plotly.graph_objects as go
import pandas as pd
import numpy as np
import scipy

df_vibe = pd.read_csv('motorcycle-vibration-moving-frequency.csv',index_col=0)
df_vibe = df_vibe - df_vibe.median()
df_vibe

accel_time_labels = dict(
    xaxis_title_text='',
    yaxis_title_text='Acceleration (g)',
    legend_title_text=''
)

fig = endaq.plot.plots.rolling_min_max_envelope(
    df_vibe,
    desired_num_points=1000,
    plot_as_bars=True,
    opacity=1.0
    )
fig.update_layout(
    accel_time_labels,
    title_text='Engine Reving of Motorcycle',
)
fig.show()

# not working part
freqs, bins, Pxx, fig = endaq.plot.octave_spectrogram(df_vibe[['Z (40g)']], 
                                                      window=0.5, 
                                                      bins_per_octave=12, 
                                                      max_freq=1000, 
                                                      freq_start=40)
fig.show()

Error Message:
`/home/marcel/.asdf/installs/python/3.9.11/lib/python3.9/site-packages/endaq/calc/psd.py:169: RuntimeWarning:

empty frequency bins in re-binned PSD; original PSD's frequency spacing is too coarse

---

ValueError Traceback (most recent call last)
Input In [7], in <cell line: 1>()
----> 1 freqs, bins, Pxx, fig = endaq.plot.octave_spectrogram(df_vibe[['Z (40g)']],
2 window=0.5,
3 bins_per_octave=12,
4 max_freq=1000,
5 freq_start=40)
6 fig.show()

ValueError: not enough values to unpack (expected 4, got 2)`

[S-Hanly]

Hey @MarcelEckh, first off thank you for officially being the first non-enDAQ person to post an issue here! It's a first but not a last and although I don't want to diminish the frustration you are undoubtedly feeling, I'm excited that this is the beginning of accelerated adoption of our library! Sorry it has taken us so long to see this, I'll change my settings in GitHub to get notified every time an issue is posted! I'm also going to at my fellow contributors: @StokesMIDE @CFlaniganMide @pscheidler

I'm not a big fan of the implementation of this function, I had posted a related issue actually: #35. More on that next after I help fix the immediate issue.

The runtime errors are more messages than errors to let you know that the PSD is too coarse for the given time range. They are admittedly annoying but they can be ignored as they are "just" warnings. The software team won't like this too much but you can explicitly turn them off via (found on stackoverflow):

import warnings
warnings.filterwarnings("ignore")

The actual error you have is that the function was updated to only return two parameters not 4 so you need to change the line:
freqs, bins, Pxx, fig = endaq.plot.octave_spectrogram(df_vibe[['Z (40g)']]
to
df, fig = endaq.plot.octave_spectrogram(df_vibe[['Z (40g)']]

I showed an example of it running correctly in this colab (here's the link).

Now to how we can improve this function! I had a few issues with the current implementation:

• The returned data isn't actually in real engineering units that you can relate to anything else
• There is no ability to customize your display, see options below (all in that colab)

I really don't like that this function does a lot of calculations in the plotting module, I'd rather have rolling_psd, rolling_fft, rolling_srs functions in endaq.calc and then a single plot.3d_spectrum function that allows the users to pass the output from one of the calc functions to it. I'll clean some of this code up in the colab and do a pull request tomorrow and at you to be aware we are acting on your feedback! Speaking of, please let me know if you like or dislike what I'm proposing!

[S-Hanly]

@MarcelEckh I wanted to give you an update that I'm getting somewhere with some new functions but won't finish tonight, here's the work in progress if you want to check it out! What I have left to do is:

• Add a rolling_shock_spectrum function and test
• Add some simple examples into the docstrings (lots of examples are in the colab already though!)
• Build a new branch, test, and submit a pull request!

fft = rolling_fft(df_vibe, num_slices=200, add_resultant=True)
fig = spectrum_over_time(fft, plot_type = 'Heatmap', log_y=False, log_z=False, y_limit=200, var_to_process='Resultant')
fig.show()

psd = rolling_psd(engine, num_slices=500, add_resultant=True, octave_bins=6, fstart=4)
fig = spectrum_over_time(psd, plot_type = 'Heatmap', log_y=True, log_z=True, var_to_process='Resultant')
fig.show()

fft = rolling_fft(engine, slice_width = 1.0, add_resultant=True,
                  index_values = pd.DatetimeIndex(['2016-08-02 12:06:36', 
                                                   '2016-08-02 12:08:00', 
                                                   '2016-08-02 12:08:34',
                                                   '2016-08-02 12:11:00',
                                                   '2016-08-02 12:13:27',
                                                   '2016-08-02 12:17:21',
                                                   '2016-08-02 12:21:09'],tz='UTC'))
fig = spectrum_over_time(fft, plot_type = 'Waterfall', log_y=False, log_z=False, y_limit=250, var_to_process='Resultant', waterfall_line_sequence=False)
fig.show()

fig = spectrum_over_time(fft, plot_type = 'Surface', log_y=False, log_z=False, y_limit=200, var_to_process='Resultant')
fig.show()

fft = rolling_fft(df_vibe, num_slices=50, add_resultant=True)
fig = spectrum_over_time(fft, plot_type = 'Waterfall', log_y=False, log_z=False, y_limit=200, var_to_process='Resultant')
fig.show()

fig = spectrum_over_time(psd, plot_type = 'Animation', log_y=True, log_z=True, var_to_process='Resultant')
fig.show()

[MarcelEckh]

Hi @shanlyMIDE, I feel honored by your reply and it's absolutely great to get such good help so quickly! Thank you very much for that!
The colab you created also helps a ton and I was able to correct my code and now it works. To be honest, I completely missed the changes that were being done to this function.

Regarding your proposals: I really like them and I'd especially appreciate having the returned data in actual units.

[S-Hanly]

Alright it's been submitted! We'll let you know when we have something fully merged into development, but you can download and test this specific branch

[MarcelEckh]

Awesome! I look forward to it.

[S-Hanly]

closing this now because it is in development! #191

Here is the specific major function (that links to a lot of the other new ones): https://docs.endaq.com/en/development/endaq/plot.html#endaq.plot.spectrum_over_time

You can download this branch via:

pip install git+https://github.com/MideTechnology/endaq-python.git@development