Skip to content
This repository has been archived by the owner on Nov 25, 2021. It is now read-only.

Commit

Permalink
Merge branch 'release/1.1.0' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
StokesMIDE committed Nov 2, 2021
2 parents 1a83aa2 + f4d6a99 commit 7390d4c
Show file tree
Hide file tree
Showing 9 changed files with 189 additions and 68 deletions.
40 changes: 40 additions & 0 deletions .github/workflows/publish-to-pypi.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Publish Python 🐍 distributions 📦 to PyPI

on:
push:
branches:
- main

jobs:
build-n-publish:
name: Build and publish Python 🐍 distributions 📦 to PyPI
runs-on: ubuntu-18.04

steps:
- uses: actions/checkout@master

- name: Set up Python 3.8
uses: actions/setup-python@v1
with:
python-version: 3.8

- name: Install pypa/build
run: >-
python -m
pip install
build
--user
- name: Build a binary wheel and a source tarball
run: >-
python -m
build
--outdir dist/
.
- name: Publish distribution 📦 to PyPI
uses: pypa/gh-action-pypi-publish@master
with:
user: __token__
password: ${{ secrets.PYPI_API_TOKEN }}
verbose: true
42 changes: 42 additions & 0 deletions .github/workflows/unit-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Unit test on push and PR


on: [push, pull_request, workflow_dispatch]


jobs:
unit-tests:
name: Run Unit Tests

strategy:
fail-fast: false
matrix:
os: [windows-latest, ubuntu-latest, macos-latest]
python-version: [3.7, 3.8, 3.9]

env:
OS: ${{ matrix.os }}
PYTHON-VERSION: ${{ matrix.python-version }}

runs-on: ${{ matrix.os }}

steps:

- uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- run: python -m pip install --upgrade pip

- uses: actions/checkout@v2

- run: python -m pip install .[test]

- run: python -m pytest tests --cov=endaq.ide

- uses: codecov/codecov-action@v2
with:
token: ${{ secrets.CODECOV_TOKEN }}
env_vars: OS,PYTHON-VERSION
files: .coverage
name: ${{ matrix.os }} ${{ matrix.python-version }}
verbose: true
49 changes: 0 additions & 49 deletions .travis.yml

This file was deleted.

33 changes: 32 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ from endaq.ide import *

```python
doc = get_doc("tests/test.ide")
doc1 = get_doc("https://mide.services/software/test.ide")
doc1 = get_doc("https://info.endaq.com/hubfs/data/surgical-instrument.ide")
```

IDE files can be retrieved directly from Google Drive using a Drive 'sharable link' URL. The file must be set to allow access to "Anyone with the link."
Expand All @@ -41,6 +41,8 @@ doc3 = get_doc("tests/test.ide", start="5s", end="10s")

### Summarizing IDE files: `endaq.ide.get_channel_table()`
Once an IDE file has been loaded, `endaq.ide.get_channel_table()` will retrieve basic summary information about its contents.
Some environments, such as Jupyter Notebook or Colab, will automatically display the channel table data. From inside the
Python Console, use `get_channel_table(doc).data` to display the information, or to access the table's contents directly as a pandas `DataFrame`.


```python
Expand Down Expand Up @@ -625,3 +627,32 @@ A portion of an IDE file can be saved to another, new IDE. The source can be a l
extract_time("tests/test.ide", "doc_extracted.ide", start="0:05", end="0:10")
extract_time(doc1, "doc1_extracted.ide", start="0:05", end="0:10")
```

## Additional sample IDE recording files
Here are a number of example IDE files, which may be used with `endaq.ide`:


```python
file_urls = ['https://info.endaq.com/hubfs/data/surgical-instrument.ide',
'https://info.endaq.com/hubfs/data/97c3990f-Drive-Home_70-1616632444.ide',
'https://info.endaq.com/hubfs/data/High-Drop.ide',
'https://info.endaq.com/hubfs/data/HiTest-Shock.ide',
'https://info.endaq.com/hubfs/data/Drive-Home_01.ide',
'https://info.endaq.com/hubfs/data/Tower-of-Terror.ide',
'https://info.endaq.com/hubfs/data/Punching-Bag.ide',
'https://info.endaq.com/hubfs/data/Gun-Stock.ide',
'https://info.endaq.com/hubfs/data/Seat-Base_21.ide',
'https://info.endaq.com/hubfs/data/Seat-Top_09.ide',
'https://info.endaq.com/hubfs/data/Bolted.ide',
'https://info.endaq.com/hubfs/data/Motorcycle-Car-Crash.ide',
'https://info.endaq.com/hubfs/data/train-passing.ide',
'https://info.endaq.com/hubfs/data/baseball.ide',
'https://info.endaq.com/hubfs/data/Clean-Room-VC.ide',
'https://info.endaq.com/hubfs/data/enDAQ_Cropped.ide',
'https://info.endaq.com/hubfs/data/Drive-Home_07.ide',
'https://info.endaq.com/hubfs/data/ford_f150.ide',
'https://info.endaq.com/hubfs/data/Drive-Home.ide',
'https://info.endaq.com/hubfs/data/Mining-Data.ide',
'https://info.endaq.com/hubfs/data/Mide-Airport-Drive-Lexus-Hybrid-Dash-W8.ide']
```
These can be directly read from endaq.com using `endaq.ide.get_doc()`, as previously described.
49 changes: 48 additions & 1 deletion endaq.ide examples.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
"outputs": [],
"source": [
"doc = get_doc(\"tests/test.ide\")\n",
"doc1 = get_doc(\"https://mide.services/software/test.ide\")"
"doc1 = get_doc(\"https://info.endaq.com/hubfs/data/surgical-instrument.ide\")"
]
},
{
Expand Down Expand Up @@ -763,6 +763,53 @@
"extract_time(\"tests/test.ide\", \"doc_extracted.ide\", start=\"0:05\", end=\"0:10\")\n",
"extract_time(doc1, \"doc1_extracted.ide\", start=\"0:05\", end=\"0:10\")"
]
},
{
"cell_type": "markdown",
"id": "58cf1c20",
"metadata": {},
"source": [
"# Additional sample IDE recording files\n",
"Here are a number of example IDE files, which may be used with `endaq.ide`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8a151765",
"metadata": {},
"outputs": [],
"source": [
"file_urls = ['https://info.endaq.com/hubfs/data/surgical-instrument.ide', \n",
" 'https://info.endaq.com/hubfs/data/97c3990f-Drive-Home_70-1616632444.ide', \n",
" 'https://info.endaq.com/hubfs/data/High-Drop.ide',\n",
" 'https://info.endaq.com/hubfs/data/HiTest-Shock.ide', \n",
" 'https://info.endaq.com/hubfs/data/Drive-Home_01.ide', \n",
" 'https://info.endaq.com/hubfs/data/Tower-of-Terror.ide',\n",
" 'https://info.endaq.com/hubfs/data/Punching-Bag.ide', \n",
" 'https://info.endaq.com/hubfs/data/Gun-Stock.ide',\n",
" 'https://info.endaq.com/hubfs/data/Seat-Base_21.ide',\n",
" 'https://info.endaq.com/hubfs/data/Seat-Top_09.ide', \n",
" 'https://info.endaq.com/hubfs/data/Bolted.ide',\n",
" 'https://info.endaq.com/hubfs/data/Motorcycle-Car-Crash.ide', \n",
" 'https://info.endaq.com/hubfs/data/train-passing.ide', \n",
" 'https://info.endaq.com/hubfs/data/baseball.ide',\n",
" 'https://info.endaq.com/hubfs/data/Clean-Room-VC.ide',\n",
" 'https://info.endaq.com/hubfs/data/enDAQ_Cropped.ide',\n",
" 'https://info.endaq.com/hubfs/data/Drive-Home_07.ide',\n",
" 'https://info.endaq.com/hubfs/data/ford_f150.ide',\n",
" 'https://info.endaq.com/hubfs/data/Drive-Home.ide',\n",
" 'https://info.endaq.com/hubfs/data/Mining-Data.ide',\n",
" 'https://info.endaq.com/hubfs/data/Mide-Airport-Drive-Lexus-Hybrid-Dash-W8.ide'] "
]
},
{
"cell_type": "markdown",
"id": "d9ade002",
"metadata": {},
"source": [
"These can be directly read from endaq.com using `endaq.ide.get_doc()`, as previously described."
]
}
],
"metadata": {
Expand Down
14 changes: 7 additions & 7 deletions endaq/ide/info.py
Original file line number Diff line number Diff line change
Expand Up @@ -294,7 +294,7 @@ def get_channel_table(dataset, measurement_type=ANY, start=0, end=None,


def to_pandas(
channel: idelib.dataset.Channel,
channel: typing.Union[idelib.dataset.Channel, idelib.dataset.SubChannel],
time_mode: typing.Literal["seconds", "timedelta", "datetime"] = "datetime",
) -> pd.DataFrame:
""" Read IDE data into a pandas DataFrame.
Expand All @@ -319,10 +319,10 @@ def to_pandas(
t = t + np.datetime64(channel.dataset.lastUtcTime, "s")
elif time_mode != "timedelta":
raise ValueError(f'invalid time mode "{time_mode}"')

if hasattr(channel, "subchannels"):
columns = [sch.name for sch in channel.subchannels]
else:
columns = [channel.name]

result = pd.DataFrame(
data, index=t, columns=[sch.name for sch in channel.subchannels]
)
result.index.name = "timestamp"

return result
return pd.DataFrame(data, index=pd.Series(t, name="timestamp"), columns=columns)
4 changes: 2 additions & 2 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
numpy>=1.16.6
ebmlite>=3.0.0
idelib>=3.2.0
idelib>=3.2.3
jinja2
pandas
pandas>=1.3
requests
8 changes: 3 additions & 5 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@
INSTALL_REQUIRES = [
"numpy>=1.16.6",
"ebmlite>=3.0.0",
"idelib>=3.2.0",
"idelib>=3.2.3",
"requests",
"pandas",
"pandas>=1.3",
"jinja2", # required for pandas.DataFrame.style
]

Expand All @@ -23,7 +23,7 @@

setuptools.setup(
name='endaq-ide',
version='1.0.0b1',
version='1.1.0',
author='Mide Technology',
author_email='[email protected]',
description='A comprehensive, user-centric Python API for working with enDAQ data and devices',
Expand All @@ -34,8 +34,6 @@
classifiers=['Development Status :: 4 - Beta',
'License :: OSI Approved :: MIT License',
'Natural Language :: English',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
Expand Down
18 changes: 15 additions & 3 deletions tests/test_info.py
Original file line number Diff line number Diff line change
Expand Up @@ -125,15 +125,27 @@ def test_channel_table_ranges(self):
self.assertListEqual(list(ct3.data['end']), list(ct2.data['end']))


@pytest.mark.parametrize("time_mode", ["seconds", "timedelta", "datetime"])
def test_to_pandas(test_IDE, time_mode):
@pytest.mark.parametrize("time_mode, subchannel", [
("seconds", False),
("timedelta", False),
("datetime", False),
("seconds", True),
("timedelta", True),
("datetime", True),
])
def test_to_pandas(test_IDE, time_mode, subchannel):
channel = test_IDE.channels[32]
if subchannel:
channel = channel.subchannels[0]
eventarray = channel.getSession()

result = info.to_pandas(channel, time_mode=time_mode)

assert len(result) == len(eventarray)
assert result.columns.tolist() == [sch.name for sch in channel.subchannels]
if subchannel:
assert result.columns.tolist() == [channel.name]
else:
assert result.columns.tolist() == [sch.name for sch in channel.subchannels]
assert np.issubdtype(
result.index.values.dtype,
dict(seconds=np.float64, timedelta=np.timedelta64, datetime=np.datetime64)[time_mode],
Expand Down

0 comments on commit 7390d4c

Please sign in to comment.