Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic curvature #974

Merged
merged 84 commits into from
Nov 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
9be0604
Add beginnings of curvature file
SylviaWhittle Sep 11, 2024
541a514
Add angle_diff_signed
SylviaWhittle Sep 11, 2024
7e7cfdd
Add test_angle_diff_signed
SylviaWhittle Sep 11, 2024
6569f94
Add discrete_angle_difference_per_nm_circular
SylviaWhittle Oct 1, 2024
138ab89
Fix normalisation and flipped subtractions in first and last index
SylviaWhittle Oct 1, 2024
7fa02ef
Add test for discrete_angle_difference_per_nm_circular
SylviaWhittle Oct 1, 2024
c5be319
Add find_curvature_defects_simple_threshold
SylviaWhittle Oct 2, 2024
bce731c
Add test for find_curvature_defects_simple_threshold
SylviaWhittle Oct 2, 2024
ee6c8af
Add calculate_trace_distances_to_last_points_circular
SylviaWhittle Oct 3, 2024
c262ad9
Add calculate_distances_between_distances_circular
SylviaWhittle Oct 3, 2024
9701b20
Add test_trace_distances_to_last_points_circular
SylviaWhittle Oct 3, 2024
de0c0b2
Add test_calculate_distances_between_defects_circular
SylviaWhittle Oct 3, 2024
6ee2bc5
Add calculate_number_of_defects
SylviaWhittle Oct 4, 2024
e175e87
add test_caclculate_number_of_defects
SylviaWhittle Oct 4, 2024
7cc6b0b
Add calculate_curvature_stats_single_grain_circular
SylviaWhittle Oct 4, 2024
f4cb94c
Add calculate_curvature_stats_image
SylviaWhittle Oct 4, 2024
f5013f8
Add plot_curvatures
SylviaWhittle Oct 4, 2024
f8734cd
Add plot_curvatures_individual_grains
SylviaWhittle Oct 4, 2024
39f3295
Add test_plot_curvature
SylviaWhittle Oct 4, 2024
6bcb570
Add run_curvature_stats
SylviaWhittle Oct 4, 2024
acf0c56
Fix test_angle_diff_signed
SylviaWhittle Oct 20, 2024
7326fad
Add test_calculate_curvature_stats_image
SylviaWhittle Oct 20, 2024
613b1cf
Disable defects functions and add linear version of angles_per_nm
SylviaWhittle Oct 20, 2024
a70a773
Fix plotting for updated curvature functions and remove defects funct…
SylviaWhittle Oct 20, 2024
01f0fce
Disable plotting test
SylviaWhittle Oct 20, 2024
ec0dabd
Integrate curvature stats into processing.py
SylviaWhittle Oct 20, 2024
cfb9cf6
Add curvature config option: run
SylviaWhittle Oct 20, 2024
beb263d
Add 'pad_width' to disordered tracing output for later grain alignment
SylviaWhittle Oct 20, 2024
8775c6e
Fix disordered tracing test given addition of 'pad_width'
SylviaWhittle Oct 20, 2024
311a6c0
Add curvature config to tests
SylviaWhittle Oct 20, 2024
cfb9074
Fix curvature tests and disable all relating to defects
SylviaWhittle Oct 20, 2024
76d60e4
Fix processing tests
SylviaWhittle Oct 20, 2024
b48bf08
[pre-commit.ci] Fixing issues with pre-commit
pre-commit-ci[bot] Oct 20, 2024
016782a
Make curvature plotting functions part of the Images class
SylviaWhittle Oct 22, 2024
1132050
Add whole image curvature plot and individual grain curvature plots f…
SylviaWhittle Oct 22, 2024
011c0c6
Add plots and config to validation
SylviaWhittle Oct 22, 2024
7ab8c91
Add test for plot_curvatures
SylviaWhittle Oct 23, 2024
d72d053
Add beginnings of curvature file
SylviaWhittle Sep 11, 2024
a963492
Add angle_diff_signed
SylviaWhittle Sep 11, 2024
90e8722
Add test_angle_diff_signed
SylviaWhittle Sep 11, 2024
9b15cb8
Add discrete_angle_difference_per_nm_circular
SylviaWhittle Oct 1, 2024
e702aea
Fix normalisation and flipped subtractions in first and last index
SylviaWhittle Oct 1, 2024
307a89b
Add test for discrete_angle_difference_per_nm_circular
SylviaWhittle Oct 1, 2024
e2356ab
Add find_curvature_defects_simple_threshold
SylviaWhittle Oct 2, 2024
2a76b75
Add test for find_curvature_defects_simple_threshold
SylviaWhittle Oct 2, 2024
05f22d7
Add calculate_trace_distances_to_last_points_circular
SylviaWhittle Oct 3, 2024
c9af571
Add calculate_distances_between_distances_circular
SylviaWhittle Oct 3, 2024
6dac959
Add test_trace_distances_to_last_points_circular
SylviaWhittle Oct 3, 2024
a71aeeb
Add test_calculate_distances_between_defects_circular
SylviaWhittle Oct 3, 2024
533ba7b
Add calculate_number_of_defects
SylviaWhittle Oct 4, 2024
ccf0acd
add test_caclculate_number_of_defects
SylviaWhittle Oct 4, 2024
b559441
Add calculate_curvature_stats_single_grain_circular
SylviaWhittle Oct 4, 2024
5525cd7
Add calculate_curvature_stats_image
SylviaWhittle Oct 4, 2024
ef895c5
Add plot_curvatures
SylviaWhittle Oct 4, 2024
f1fef84
Add plot_curvatures_individual_grains
SylviaWhittle Oct 4, 2024
5b6bb88
Add test_plot_curvature
SylviaWhittle Oct 4, 2024
378ca58
Add run_curvature_stats
SylviaWhittle Oct 4, 2024
7f5f389
Fix test_angle_diff_signed
SylviaWhittle Oct 20, 2024
0b1f0e1
Add test_calculate_curvature_stats_image
SylviaWhittle Oct 20, 2024
da3fff4
Disable defects functions and add linear version of angles_per_nm
SylviaWhittle Oct 20, 2024
7757dbd
Fix plotting for updated curvature functions and remove defects funct…
SylviaWhittle Oct 20, 2024
a9a647e
Disable plotting test
SylviaWhittle Oct 20, 2024
25e6847
Integrate curvature stats into processing.py
SylviaWhittle Oct 20, 2024
d4dc4c1
Add curvature config option: run
SylviaWhittle Oct 20, 2024
b982bde
Add 'pad_width' to disordered tracing output for later grain alignment
SylviaWhittle Oct 20, 2024
767ee71
Fix disordered tracing test given addition of 'pad_width'
SylviaWhittle Oct 20, 2024
f44992f
Add curvature config to tests
SylviaWhittle Oct 20, 2024
8b9e2b2
Fix curvature tests and disable all relating to defects
SylviaWhittle Oct 20, 2024
0dd6823
[pre-commit.ci] Fixing issues with pre-commit
pre-commit-ci[bot] Oct 20, 2024
3873a65
Make curvature plotting functions part of the Images class
SylviaWhittle Oct 22, 2024
0ed667e
Add whole image curvature plot and individual grain curvature plots f…
SylviaWhittle Oct 22, 2024
4d683aa
Add plots and config to validation
SylviaWhittle Oct 22, 2024
84acd11
Add test for plot_curvatures
SylviaWhittle Oct 23, 2024
0a78655
Fix: Formatting from rebase
SylviaWhittle Nov 7, 2024
a76b333
Fix tests: after rebase
SylviaWhittle Nov 7, 2024
9daab07
Merge remote-tracking branch 'origin/SylviaWhittle/curvature' into Sy…
SylviaWhittle Nov 7, 2024
3f42cae
[pre-commit.ci] Fixing issues with pre-commit
pre-commit-ci[bot] Nov 7, 2024
e4928b2
Merge main into curvature
SylviaWhittle Nov 18, 2024
e6ef86a
Remove: Add defect code
SylviaWhittle Nov 18, 2024
6dbfad1
Fix: test_process_scan_both: topostats file needed updating to contai…
SylviaWhittle Nov 19, 2024
1bd174d
Fix: test_calculate_curvature_stats_image: use absolute curvature values
SylviaWhittle Nov 19, 2024
695ac3c
Merge main into curvature
SylviaWhittle Nov 19, 2024
7d0bb3e
Fix: test_process_scan_both: Merge conflict
SylviaWhittle Nov 19, 2024
b368847
Add: test_process_scan_stages parametrisation for when curvature sta…
SylviaWhittle Nov 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
421 changes: 421 additions & 0 deletions tests/measure/test_curvature.py

Large diffs are not rendered by default.

Binary file added tests/resources/img/test_plot_curvatures.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/resources/process_scan_topostats_file_regtest.topostats
Binary file not shown.
Binary file not shown.
Binary file not shown.
149 changes: 149 additions & 0 deletions tests/test_plottingfuncs.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,155 @@ def test_dilate_binary_image(binary_image: np.ndarray, dilation_iterations: int,
rng = np.random.default_rng()


@pytest.mark.mpl_image_compare(baseline_dir="resources/img/")
def test_plot_curvatures(tmp_path: Path) -> None:
"""Test plotting of curvatures."""
image = np.array(
[
[0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2],
[0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2],
[0.1, 0.2, 1.1, 1.1, 1.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2],
[0.1, 0.2, 1.1, 0.2, 1.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2],
[0.1, 0.2, 1.1, 1.1, 1.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2],
[0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2],
[0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2],
[0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2],
[0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.2, 0.1, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2],
[0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.2, 1.2, 1.1, 1.1, 1.1, 1.1, 0.2, 0.1],
[0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.2, 0.1, 0.1, 1.1, 1.1, 0.2, 0.1, 1.2, 1.1, 0.2],
[0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.2, 1.1, 0.2, 0.1, 0.2, 0.1, 1.1, 0.1],
[0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.2, 0.1, 0.1, 1.1, 0.1, 0.2, 0.1, 0.2, 1.1, 0.2],
[0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.2, 0.1, 1.1, 0.1, 0.2, 0.1, 1.1, 0.1],
[0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.2, 0.1, 0.1, 0.2, 1.1, 1.1, 1.1, 1.1, 0.1, 0.2],
[0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1],
]
)

cropped_images = {
"grain_0": {
"original_image": np.array(
[
[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
[0.0, 0.2, 0.1, 0.2, 0.1, 0.2, 0.0],
[0.0, 0.2, 1.1, 1.1, 1.1, 0.2, 0.0],
[0.0, 0.2, 1.1, 0.2, 1.1, 0.2, 0.0],
[0.0, 0.2, 1.1, 1.1, 1.1, 0.2, 0.0],
[0.0, 0.2, 0.1, 0.2, 0.1, 0.2, 0.0],
[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
]
),
"bbox": [1, 1, 5, 5],
"pad_width": 1,
},
"grain_1": {
"original_image": np.array(
[
[0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2],
[0.2, 1.2, 1.1, 1.1, 1.1, 1.1, 0.2, 0.1],
[0.1, 1.1, 1.1, 0.2, 0.1, 1.2, 1.1, 0.2],
[0.2, 1.1, 0.2, 0.1, 0.2, 0.1, 1.1, 0.1],
[0.1, 1.1, 0.1, 0.2, 0.1, 0.2, 1.1, 0.2],
[0.2, 0.1, 1.1, 0.1, 0.2, 0.1, 1.1, 0.1],
[0.1, 0.2, 1.1, 1.1, 1.1, 1.1, 0.1, 0.2],
[0.2, 0.1, 0.2, 0.1, 0.2, 0.1, 0.2, 0.1],
]
),
"bbox": [8, 8, 15, 15],
"pad_width": 0,
},
}

grains_curvture_stats_dict = {
"grain_0": {"mol_0": np.array([0.2, 0.0, 0.2, 0.0, 0.2, 0.0, 0.2, 0.0])},
"grain_1": {
"mol_0": np.array(
[
0.1,
0.2,
0.1,
0.2,
0.1,
0.2,
0.1,
0.2,
0.1,
0.2,
0.1,
0.2,
0.1,
0.2,
0.1,
0.2,
0.1,
0.2,
]
)
},
}

all_grain_smoothed_data = {
"grain_0": {
"mol_0": {
"spline_coords": np.array(
[
[2.5, 2.5],
[2.5, 3.5],
[2.5, 4.5],
[3.5, 4.5],
[4.5, 4.5],
[4.5, 3.5],
[4.5, 2.5],
[3.5, 2.5],
]
),
},
},
"grain_1": {
"mol_0": {
"spline_coords": np.array(
[
[1.5, 1.5],
[1.5, 2.5],
[1.5, 3.5],
[1.5, 4.5],
[1.5, 5.5],
[2.5, 5.5],
[2.5, 6.5],
[3.5, 6.5],
[4.5, 6.5],
[5.5, 6.5],
[6.5, 5.5],
[6.5, 4.5],
[6.5, 3.5],
[6.5, 2.5],
[5.5, 2.5],
[4.5, 1.5],
[3.5, 1.5],
[2.5, 1.5],
],
)
}
},
}

fig, _ = Images(
np.array([[0, 0], [0, 0]]),
output_dir=tmp_path,
filename="Curvature",
image_type="non-binary",
savefig_dpi=200,
core_set=True,
).plot_curvatures(
image=image,
cropped_images=cropped_images,
grains_curvature_stats_dict=grains_curvture_stats_dict,
all_grain_smoothed_data=all_grain_smoothed_data,
colourmap_normalisation_bounds=[-0.2, 0.2],
)

return fig


@pytest.mark.parametrize(
("masked_array", "axes_colorbar", "region_properties"),
[(rng.random((10, 10)), True, None), (None, True, None), (None, False, True)],
Expand Down
85 changes: 57 additions & 28 deletions tests/test_processing.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ def test_process_scan_below(regtest, tmp_path, process_scan_config: dict, load_s
nodestats_config=process_scan_config["nodestats"],
ordered_tracing_config=process_scan_config["ordered_tracing"],
splining_config=process_scan_config["splining"],
curvature_config=process_scan_config["curvature"],
plotting_config=process_scan_config["plotting"],
output_dir=tmp_path,
)
Expand Down Expand Up @@ -76,6 +77,7 @@ def test_process_scan_below_height_profiles(tmp_path, process_scan_config: dict,
nodestats_config=process_scan_config["nodestats"],
ordered_tracing_config=process_scan_config["ordered_tracing"],
splining_config=process_scan_config["splining"],
curvature_config=process_scan_config["curvature"],
plotting_config=process_scan_config["plotting"],
output_dir=tmp_path,
)
Expand Down Expand Up @@ -110,6 +112,7 @@ def test_process_scan_above(regtest, tmp_path, process_scan_config: dict, load_s
nodestats_config=process_scan_config["nodestats"],
ordered_tracing_config=process_scan_config["ordered_tracing"],
splining_config=process_scan_config["splining"],
curvature_config=process_scan_config["curvature"],
plotting_config=process_scan_config["plotting"],
output_dir=tmp_path,
)
Expand All @@ -136,6 +139,7 @@ def test_process_scan_above_height_profiles(tmp_path, process_scan_config: dict,
nodestats_config=process_scan_config["nodestats"],
ordered_tracing_config=process_scan_config["ordered_tracing"],
splining_config=process_scan_config["splining"],
curvature_config=process_scan_config["curvature"],
plotting_config=process_scan_config["plotting"],
output_dir=tmp_path,
)
Expand Down Expand Up @@ -172,6 +176,7 @@ def test_process_scan_both(regtest, tmp_path, process_scan_config: dict, load_sc
nodestats_config=process_scan_config["nodestats"],
ordered_tracing_config=process_scan_config["ordered_tracing"],
splining_config=process_scan_config["splining"],
curvature_config=process_scan_config["curvature"],
plotting_config=process_scan_config["plotting"],
output_dir=tmp_path,
)
Expand Down Expand Up @@ -226,6 +231,7 @@ def test_save_cropped_grains(
nodestats_config=process_scan_config["nodestats"],
ordered_tracing_config=process_scan_config["ordered_tracing"],
splining_config=process_scan_config["splining"],
curvature_config=process_scan_config["curvature"],
plotting_config=process_scan_config["plotting"],
output_dir=tmp_path,
)
Expand Down Expand Up @@ -274,6 +280,7 @@ def test_save_format(process_scan_config: dict, load_scan_data: LoadScans, tmp_p
nodestats_config=process_scan_config["nodestats"],
ordered_tracing_config=process_scan_config["ordered_tracing"],
splining_config=process_scan_config["splining"],
curvature_config=process_scan_config["curvature"],
plotting_config=process_scan_config["plotting"],
output_dir=tmp_path,
)
Expand Down Expand Up @@ -558,58 +565,76 @@ def test_check_run_steps(
"nodestats_run",
"ordered_tracing_run",
"splining_run",
"curvature_run",
"log_msg1",
"log_msg2",
),
[
pytest.param(
False,
False,
False,
False,
False,
False,
False,
False, # Filters
False, # Grains
False, # Grainstats
False, # Disordered tracing
False, # Nodestats
False, # Ordered tracing
False, # Splining
False, # Curvature
"You have not included running the initial filter stage.",
"Please check your configuration file.",
id="All stages are disabled",
),
pytest.param(
True,
False,
False,
False,
False,
False,
False,
True, # Filters
False, # Grains
False, # Grainstats
False, # Disordered tracing
False, # Nodestats
False, # Ordered tracing
False, # Splining
False, # Curvature
"Detection of grains disabled, GrainStats will not be run.",
"",
id="Only filtering enabled",
),
pytest.param(
True,
True,
False,
False,
False,
False,
False,
True, # Filters
True, # Grains
False, # Grainstats
False, # Disordered tracing
False, # Nodestats
False, # Ordered tracing
False, # Splining
False, # Curvature
"Calculation of grainstats disabled, returning empty dataframe and empty height_profiles.",
"",
id="Filtering and Grain enabled",
),
pytest.param(
True,
True,
True,
False,
False,
False,
False,
True, # Filter
True, # Grains
True, # Grainstats
False, # Disordered Tracing
False, # Nodestats
False, # Ordered tracing
False, # Splining
False, # Curvature
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we should have a test that checks if the Curvature option is True which would require all prior steps?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like this? (New commits)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

b368847 checks everything upto Splining is disabled and Curvature is disabled and others tests check that output of curvature are produced if enabled.

"Processing grain",
"Calculation of Disordered Tracing disabled, returning empty dictionary.",
id="Filtering, Grain and GrainStats enabled",
),
pytest.param(
True, # Filter
True, # Grains
True, # Grainstats
True, # Disordered Tracing
True, # Nodestats
True, # Ordered tracing
True, # Splining
False, # Curvature
"Processing grain",
"Calculation of Curvature Stats disabled, returning None.",
id="All stages enabled",
),
# @ns-rse 2024-09-13 : Parameters need updating so test is performed.
# pytest.param(
# True,
Expand All @@ -633,6 +658,7 @@ def test_process_stages(
nodestats_run: bool,
ordered_tracing_run: bool,
splining_run: bool,
curvature_run: bool,
log_msg1: str,
log_msg2: str,
caplog,
Expand All @@ -652,6 +678,7 @@ def test_process_stages(
process_scan_config["nodestats"]["run"] = nodestats_run
process_scan_config["ordered_tracing"]["run"] = ordered_tracing_run
process_scan_config["splining"]["run"] = splining_run
process_scan_config["curvature"]["run"] = curvature_run
_, _, _, _, _, _ = process_scan(
topostats_object=img_dic["minicircle_small"],
base_dir=BASE_DIR,
Expand All @@ -662,6 +689,7 @@ def test_process_stages(
nodestats_config=process_scan_config["nodestats"],
ordered_tracing_config=process_scan_config["ordered_tracing"],
splining_config=process_scan_config["splining"],
curvature_config=process_scan_config["curvature"],
plotting_config=process_scan_config["plotting"],
output_dir=tmp_path,
)
Expand All @@ -685,6 +713,7 @@ def test_process_scan_no_grains(process_scan_config: dict, load_scan_data: LoadS
nodestats_config=process_scan_config["nodestats"],
ordered_tracing_config=process_scan_config["ordered_tracing"],
splining_config=process_scan_config["splining"],
curvature_config=process_scan_config["curvature"],
plotting_config=process_scan_config["plotting"],
output_dir=tmp_path,
)
Expand Down
3 changes: 3 additions & 0 deletions topostats/default_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,9 @@ splining:
spline_linear_smoothing: 5.0 # The amount of smoothing to apply to linear features.
spline_circular_smoothing: 5.0 # The amount of smoothing to apply to circular features.
spline_degree: 3 # The polynomial degree of the spline.
curvature:
run: true # Options : true, false
colourmap_normalisation_bounds: [-0.5, 0.5] # Radians per nm to normalise the colourmap to.
plotting:
run: true # Options : true, false
style: topostats.mplstyle # Options : topostats.mplstyle or path to a matplotlibrc params file
Expand Down
Loading