Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update C++ codebase for handling of feature dependencies [vcs: #minor] #334

Merged
merged 94 commits into from
Jan 10, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
0e6dbcd
throw runtime_error, rm exit(1) in featuretype&calcfeatures
anilbey Oct 20, 2023
5cc3c5e
rename type->input_type
anilbey Oct 20, 2023
2a84582
throw EfelAssertionError to avoid exit(-1)
anilbey Oct 20, 2023
84626df
add test to trigger C++->Python AssertionError
anilbey Oct 20, 2023
954613a
remove featurename.find(";") check
anilbey Oct 20, 2023
e9cbcc8
typo in docstring
anilbey Oct 20, 2023
443e083
add feature's name to the error message
anilbey Oct 20, 2023
f11e36d
Revert "remove featurename.find(";") check"
anilbey Oct 20, 2023
108ca85
remove setversion function
anilbey Oct 20, 2023
71f687d
draft: removing alternative wildcard syntax
anilbey Oct 20, 2023
2ba460d
merge master
anilbey Oct 25, 2023
3748628
remove variable features in LibV5
anilbey Oct 25, 2023
9fa5fa4
remove variable (alias) features in LibV2
anilbey Nov 3, 2023
f07895b
simplify feature pointers representation in cppcore, remove wildcards
anilbey Nov 3, 2023
655cd03
make AddUniqueItem void
anilbey Nov 3, 2023
34cfb4d
remove empty function getDependencyList
anilbey Nov 3, 2023
b8f629a
directly check stream's state after opening the file
anilbey Nov 3, 2023
d91d476
remove dead code in efel and cfeature
anilbey Nov 3, 2023
9172cc7
merge efel into cppcore
anilbey Nov 3, 2023
8ed52a8
add template to getParam
anilbey Nov 6, 2023
786f636
add getFeatures fn to get all dependent features
anilbey Nov 6, 2023
6a80aeb
LibV5.cpp update until time_to_last_spike
anilbey Nov 6, 2023
cf52d1d
libv5 update ISI computations
anilbey Nov 6, 2023
1abfbac
LibV5 remove ISI first, second etc. duplication
anilbey Nov 6, 2023
32eb094
calculateInvISI throw except instead of return 0
anilbey Nov 6, 2023
13c21f6
use getFeatures in depolarized_base
anilbey Nov 6, 2023
9ad88fd
use getFeatures in steady_state_hyper
anilbey Nov 6, 2023
4de7876
depolarized_based consider retVal==0 failure
anilbey Nov 7, 2023
3c86f4e
use getFeatures in LibV2
anilbey Nov 7, 2023
4d36018
use getFeatures in LibV1
anilbey Nov 7, 2023
65e8f60
using to replace scope resolution
anilbey Nov 8, 2023
d0084a3
define distinct errors: FeatureComputationError, EmptyFeatureError
anilbey Nov 8, 2023
84b3020
use getFeature/s in LibV5 for consistent handling of edge cases
anilbey Nov 9, 2023
841f2b3
remove unnecessary initializer_list
anilbey Nov 14, 2023
ffd9d03
min_AHP_indices to use getFeatures
anilbey Nov 14, 2023
90a5dc6
AHP_depth_abs with getFeature
anilbey Nov 14, 2023
749e002
spike_half_width to use getFeatures
anilbey Nov 14, 2023
932b1b1
AP_begin_indices to use getFeatures
anilbey Nov 14, 2023
35a8450
use getFeatures in current_base
anilbey Nov 14, 2023
0510877
getfeatures in burst_begin_indices
anilbey Nov 14, 2023
378c278
decay_time_constant_after_stim to use getFeatures
anilbey Nov 15, 2023
8ded463
libv5 exception handling
anilbey Nov 15, 2023
a21b653
remove getVec completely
anilbey Nov 16, 2023
f3b3a73
Merge branch 'master' into wildcards
anilbey Nov 29, 2023
4e6514b
make format
anilbey Nov 29, 2023
f7425b8
Merge branch 'master' into wildcards
anilbey Dec 12, 2023
d1101d9
replace throw EmptyFeatureError with return -1 for consistency
anilbey Dec 14, 2023
a4335ec
Merge branch 'wildcards' of https://github.com/BlueBrain/eFEL into wi…
anilbey Dec 14, 2023
bde76af
remove redundant else condition
anilbey Dec 14, 2023
d6848c3
avoid fetching ignore_first_ISI twice
anilbey Dec 14, 2023
3ca2801
in the spikewidth2 error message mention spikewidth2
anilbey Dec 14, 2023
9b9d382
update maximum_voltage's success case to be same asminimum_voltages
anilbey Dec 14, 2023
d5d5ed7
make peakvoltage vector a reference in amp_drop_first_last
anilbey Dec 14, 2023
e7907c8
remove obsolete empty feature checks in spike_width1
anilbey Dec 14, 2023
e790e16
remove else after throw
anilbey Dec 14, 2023
f3adcc3
update new line in code comment for readability
anilbey Dec 14, 2023
83a57b1
replace GErrorStr+1 and return -1 via exception
anilbey Dec 14, 2023
444bfe7
add test to cover single spike case in min_voltage_between_spikes
anilbey Dec 14, 2023
5a5c22d
cover stimulus_current==0.0 edge case in ohmic resistance features
anilbey Dec 14, 2023
d7c576f
remove else after throw in 3 LibV5 places
anilbey Dec 14, 2023
944140a
typo in test name
anilbey Dec 15, 2023
b3a33f3
test_ohmic_input_resistance_zero_stimulus_current update stim params
anilbey Dec 15, 2023
1253dd3
update fail case for spikecount & spikecount_stimint
anilbey Dec 19, 2023
5784c06
add test calling all features on constant trace
anilbey Dec 19, 2023
c101450
remove re imported import in test
anilbey Dec 19, 2023
2ff86ed
merge master into wildcards
anilbey Dec 19, 2023
267220d
impute Spikecount missing val to 0.0 in GranuleCellDeap1.ipynb
anilbey Dec 19, 2023
9e9286f
remove unnecessary variable
anilbey Dec 21, 2023
241a2d7
move Spikecount implementation to Python
anilbey Dec 21, 2023
46007d2
change doc of Spikecount as Python efeature
anilbey Dec 21, 2023
818e3a1
move Spikecount_stimint to Python
anilbey Dec 21, 2023
f778c6a
add validation module with check_ais_initiation
anilbey Jan 4, 2024
eebc16e
add bpap_attenuation as a multitrace feature
anilbey Jan 4, 2024
31b7212
move burst_number and strict_burst_number to Python
anilbey Jan 5, 2024
4c1aae3
update impedance, remove spikecount is None check
anilbey Jan 5, 2024
d287792
add CHANGELOG.md
anilbey Jan 5, 2024
7a2594d
Merge branch 'master' into wildcards
anilbey Jan 5, 2024
133a48e
Merge branch 'master' into wildcards
anilbey Jan 8, 2024
cc4f705
docs: add multitrace and validation modules to API
anilbey Jan 8, 2024
c02f4c9
Merge branch 'master' into wildcards
anilbey Jan 8, 2024
df224bd
Docs: add name change warnings
anilbey Jan 8, 2024
c7ff80a
Merge branch 'wildcards' of github.com:BlueBrain/eFEL into wildcards
anilbey Jan 8, 2024
082f205
add int/double template instantiations to cfeature
anilbey Jan 9, 2024
c7a4cc9
remove getDistance_cpp, use python implementation
anilbey Jan 9, 2024
5f8881b
move trace_check to pyfeatures
anilbey Jan 9, 2024
b6940d1
lint fix
anilbey Jan 9, 2024
87cc375
add from __future__ import annotations
anilbey Jan 9, 2024
41b087e
add stimulus_current to test_allfeatures_on_constant_V
anilbey Jan 9, 2024
f39e7eb
allow both spike_count and Spikecount
anilbey Jan 9, 2024
489c247
move validation.py inside pyfeatures
anilbey Jan 9, 2024
c6033de
Revert "add stimulus_current to test_allfeatures_on_constant_V"
anilbey Jan 9, 2024
2682494
add spike_count expect it to be 0 for allfeatures_on_constant_voltage
anilbey Jan 9, 2024
43bf8b3
encourage use of spike_count instead of Spikecount
anilbey Jan 9, 2024
bc15caa
update deprecation note in the docs
anilbey Jan 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions efel/DependencyV5.txt
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,7 @@ LibV1:maximum_voltage #LibV1:interpolate
LibV1:minimum_voltage #LibV1:interpolate
LibV1:steady_state_voltage #LibV1:interpolate
LibV3:depolarized_base #LibV5:AP_end_indices #LibV5:AP_begin_indices #LibV1:interpolate
LibV1:ISI_CV #LibV1:ISI_values #LibV1:interpolate
LibV1:Spikecount #LibV5:peak_indices #LibV1:interpolate
LibV1:ISI_CV #LibV1:ISI_values #LibV1:interpolate
LibV5:Spikecount_stimint #LibV1:peak_time #LibV1:interpolate
LibV1:AHP_depth #LibV5:voltage_base #LibV5:min_AHP_values #LibV1:interpolate
LibV1:AHP_depth_slow #LibV5:voltage_base #LibV1:AHP_depth_abs_slow #LibV1:interpolate
Expand Down
1 change: 0 additions & 1 deletion efel/cppcore/FillFptrTable.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,6 @@ int FillFptrTable() {
FptrTableV1["ISI_CV"] = &LibV1::ISI_CV;
FptrTableV1["AHP_depth_abs_slow"] = &LibV1::AHP_depth_abs_slow;
FptrTableV1["AHP_slow_time"] = &LibV1::AHP_slow_time;
FptrTableV1["Spikecount"] = &LibV1::Spikecount;
FptrTableV1["AHP_depth"] = &LibV1::AHP_depth;
FptrTableV1["AHP_depth_slow"] = &LibV1::AHP_depth_slow;
FptrTableV1["burst_number"] = &LibV1::burst_number;
Expand Down
11 changes: 0 additions & 11 deletions efel/cppcore/LibV1.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -76,17 +76,6 @@ int LibV1::interpolate(mapStr2intVec& IntFeatureData,
return 1;
}

// *** Spikecount ***
int LibV1::Spikecount(mapStr2intVec& IntFeatureData,
mapStr2doubleVec& DoubleFeatureData,
mapStr2Str& StringData) {
size_t spikecount_value = getFeature(IntFeatureData, {"peak_indices"}).size();
vector<int> spikecount(1, spikecount_value);
setVec(IntFeatureData, StringData, "Spikecount", spikecount);
return spikecount_value;
}
// end of Spikecount

int LibV1::ISI_values(mapStr2intVec& IntFeatureData,
mapStr2doubleVec& DoubleFeatureData,
mapStr2Str& StringData) {
Expand Down
2 changes: 0 additions & 2 deletions efel/cppcore/LibV1.h
Original file line number Diff line number Diff line change
Expand Up @@ -116,8 +116,6 @@ int AHP_depth_abs_slow(mapStr2intVec& IntFeatureData,
mapStr2Str& StringData);
int AHP_slow_time(mapStr2intVec& IntFeatureData,
mapStr2doubleVec& DoubleFeatureData, mapStr2Str& StringData);
int Spikecount(mapStr2intVec& IntFeatureData,
mapStr2doubleVec& DoubleFeatureData, mapStr2Str& StringData);
int AHP_depth(mapStr2intVec& IntFeatureData,
mapStr2doubleVec& DoubleFeatureData, mapStr2Str& StringData);
int AHP_depth_slow(mapStr2intVec& IntFeatureData,
Expand Down
4 changes: 1 addition & 3 deletions efel/cppcore/cfeature.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,6 @@ void cFeature::fillfeaturetypes() {
featuretypes["ISI_CV"] = "double";
featuretypes["AHP_depth_abs_slow"] = "double";
featuretypes["AHP_slow_time"] = "double";
featuretypes["Spikecount"] = "int";
featuretypes["Spikecount_stimint"] = "int";
featuretypes["AHP_depth"] = "double";
featuretypes["AHP_depth_slow"] = "double";
Expand Down Expand Up @@ -257,8 +256,7 @@ int cFeature::calc_features(const std::string& name) {

bool last_failed = false;

for (const auto& pfptrstring : lookup_it->second) {
feature_function function = pfptrstring;
for (const feature_function& function : lookup_it->second) {
setFeatureString("params", "");

if (function(mapIntData, mapDoubleData, mapStrData) < 0) {
Expand Down
23 changes: 19 additions & 4 deletions efel/pyfeatures/pyfeatures.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""

from typing_extensions import deprecated

import numpy
import efel.cppcore
Expand All @@ -41,6 +42,7 @@
'initburst_sahp_ssse',
'depol_block',
'depol_block_bool',
'Spikecount',
'spikes_per_burst',
'spikes_per_burst_diff',
'spikes_in_burst1_burst2_diff',
Expand All @@ -59,6 +61,19 @@ def time():
return _get_cpp_feature("time")


@deprecated("Use spike_count instead.")
def Spikecount() -> numpy.ndarray:
return spike_count()


def spike_count() -> numpy.ndarray:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to have a new spike_count feature and deprecating Spikecount? Also since it stays as Spikecount in the documentation.
Same question for Spikecount_stimint

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the future I thought keeping Python naming convention would be better. I wanted to deprecate it now so that if we change the naming convention in the future we can say: this was deprecated 2 major releases ago.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally all features should have a consistent naming. Most of them are underscore separated already e.g.
adaptation_index or ohmic_input_resistance. Spikecount was an exception. What do you think?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, sounds good. But then maybe we should update the documentation also.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see Spikecount has been used in several places in BluePyOpt and BluePyEModel. There can be lot of warnings from old code at multiple places as it is one of the most common features.

Do all the other features follow consistent naming convention? If not, can we keep the name and feature Spikecount ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Thanks!

Copy link
Contributor Author

@anilbey anilbey Jan 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While testing bluepyopt, bluepyefe, bluepyemodel I will make sure this warning does not occur multiple times.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't you add spike_count in all_pyfeatures to make it a valid feature? If you want to deprecate Spikecount I mean. Also modify documentation so that feature name is spike_count while keeping a note for Spikecount explaining that it is deprecated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think "spike_count" is there already.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updating the docs

"""Get spike count."""
peak_indices = _get_cpp_feature("peak_indices")
if peak_indices is None:
return numpy.array([0])
return numpy.array([peak_indices.size])


def impedance():
from scipy.ndimage.filters import gaussian_filter1d

Expand All @@ -71,10 +86,10 @@ def impedance():
if current_trace is not None:
holding_current = _get_cpp_feature("current_base")
normalized_current = current_trace - holding_current
spike_count = _get_cpp_feature("Spikecount")
if spike_count is None:
spike_count = 0
if spike_count < 1: # if there is no spikes in ZAP
n_spikes = spike_count()
if n_spikes is None:
n_spikes = 0
if n_spikes < 1: # if there is no spikes in ZAP
fft_volt = numpy.fft.fft(normalized_voltage)
fft_cur = numpy.fft.fft(normalized_current)
if any(fft_cur) == 0:
Expand Down
3 changes: 1 addition & 2 deletions examples/deap/GranuleCell1/GranuleCellDeap1.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,6 @@
" times.append(time)\n",
" voltages.append(voltage)\n",
"\n",
" feature_values[0][\"Spikecount\"] = 0.0 # replace None with 0.0 \n",
" return feature_values, times, voltages\n",
"\n",
"orig_features, orig_times, orig_voltages = get_features(orig_conductances)"
Expand Down Expand Up @@ -406,7 +405,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
"version": "3.10.12"
}
},
"nbformat": 4,
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@
name="efel",
version=versioneer.get_version(),
cmdclass=versioneer.get_cmdclass(),
install_requires=['numpy>=1.6', 'neo>=0.5.2'],
install_requires=['numpy>=1.6', 'neo>=0.5.2', 'typing-extensions>=4.8.0'],
packages=['efel', 'efel.pyfeatures', 'efel.units'],
author="BlueBrain Project, EPFL",
maintainer="Werner Van Geit",
Expand Down
1 change: 0 additions & 1 deletion tests/DependencyV5_LibV5peakindices.txt
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ LibV1:interpolate
LibV1:steady_state_voltage
LibV3:depolarized_base
LibV1:ISI_CV #LibV1:ISI_values
LibV1:Spikecount #LibV5:peak_indices
LibV1:AHP_depth #LibV5:voltage_base #LibV5:min_AHP_values
LibV2:AP_rise_indices #LibV5:peak_indices #LibV5:AP_begin_indices
LibV2:AP_end_indices #LibV5:peak_indices
Expand Down
2 changes: 1 addition & 1 deletion tests/test_allfeatures.py
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ def test_allfeatures_on_constant_voltage():
"sag_amplitude", "voltage_after_stim", "steady_state_hyper",
"steady_state_voltage", "steady_state_voltage_stimend",
"voltage_deflection", "voltage_deflection_begin", "voltage_deflection_vb_ssse",
"depol_block", "depol_block_bool", "voltage_base"
"depol_block", "depol_block_bool", "voltage_base", "Spikecount"
]

for field in array_fields:
Expand Down
6 changes: 2 additions & 4 deletions tests/test_basic.py
Original file line number Diff line number Diff line change
Expand Up @@ -626,7 +626,6 @@ def test_strict_stiminterval():
peak_indices = feature_values[0]['peak_indices']
peak_time = feature_values[0]['peak_time']
spikecount = feature_values[0]['Spikecount']

assert len(peak_indices) == n_of_spikes
assert len(peak_time) == n_of_spikes
assert spikecount == n_of_spikes
Expand Down Expand Up @@ -1132,7 +1131,6 @@ def test_getDistance_trace_check():
trace['stim_start'] = [10]
trace['stim_end'] = [70]
traces.append(trace)

numpy.testing.assert_allclose(
efel.getDistance(trace, 'Spikecount', 0, 1), 3.0
)
Expand Down Expand Up @@ -1638,8 +1636,8 @@ def test_spikecount2():
[trace],
features)

spikecount = feature_values[0]['Spikecount']
assert spikecount is None
spikecount = feature_values[0]['Spikecount'][0]
assert spikecount == 0


def test_min_voltage_between_spikes1():
Expand Down
Loading