Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pt: refactor data stat #3285

Merged
merged 42 commits into from
Feb 18, 2024
Merged

pt: refactor data stat #3285

merged 42 commits into from
Feb 18, 2024

Conversation

njzjz
Copy link
Member

@njzjz njzjz commented Feb 17, 2024

No description provided.

njzjz added 30 commits February 16, 2024 15:34
@njzjz
checkpoint
f352a67 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
Merge branch 'devel' into env-mat-stat
b2b48b2
@njzjz
record sum
1c3e2bb 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
compute_std
d6bf4ab 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
protection
4f029b0 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
save stat
afd6d6a 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
get std
2f67aac 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
compute avg
89d413c 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
sea looks good
8b6e24e 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
rm init_desc_stat
4736ae5 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
rm get_stat_name
6cb1275 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
rewrite compute_input_stats
b99d330 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
hybrid
da1e72d 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
fix hash
959583a 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
se atten
524366d 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
to make it work
87f0d85 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
compute_or_load_stat
2e815ea 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
init
ed34d59 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
fix shape
d04d16c 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
fix concat
2580f8e 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
make it work
2771b1e 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
rm save_stats and load_stats
3eb5577 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
assert_allclose
3ad5484 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
fix shape
2b9bbd8 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
add env mat type
7d40b9f 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
remove print
a55e21f 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
merge methods
c34622d 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
merge
af0711c 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
clean
37e9b28 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
clean
2c63335 
Signed-off-by: Jinzhe Zeng <[email protected]>
njzjz added 2 commits February 16, 2024 21:23
@njzjz
fix load stats
d343c32 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
rm process_stat_path
6d8955a 
Signed-off-by: Jinzhe Zeng <[email protected]>
@codecov Codecov
Copy link

codecov bot commented Feb 17, 2024 •
edited
Loading

Codecov Report

Attention: 70 lines in your changes are missing coverage. Please review.

Comparison is base (9053caf) 75.06% compared to head (2ad6990) 74.84%.

Files Patch % Lines
deepmd/utils/path.py 63.15% 21 Missing ⚠️
deepmd/utils/env_mat_stat.py 75.43% 14 Missing ⚠️
deepmd/pt/model/descriptor/descriptor.py 42.85% 8 Missing ⚠️
deepmd/pt/entrypoints/main.py 45.45% 6 Missing ⚠️
deepmd/pt/model/descriptor/repformers.py 73.33% 4 Missing ⚠️
deepmd/pt/model/descriptor/se_a.py 80.00% 4 Missing ⚠️
deepmd/pt/model/descriptor/se_atten.py 73.33% 4 Missing ⚠️
deepmd/pt/model/task/ener.py 66.66% 3 Missing ⚠️
deepmd/pt/model/atomic_model/dp_atomic_model.py 77.77% 2 Missing ⚠️
deepmd/pt/utils/env_mat_stat.py 96.96% 2 Missing ⚠️
... and 2 more
Additional details and impacted files 
@@            Coverage Diff             @@
##            devel    #3285      +/-   ##
==========================================
- Coverage   75.06%   74.84%   -0.22%     
==========================================
  Files         387      389       +2     
  Lines       33640    33587      -53     
  Branches     1604     1604              
==========================================
- Hits        25251    25139     -112     
- Misses       7528     7587      +59     
  Partials      861      861

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

njzjz added 2 commits February 16, 2024 21:38
@njzjz
fix py38 compatibility
582451e 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
fix typo
1c1c1a5 
Signed-off-by: Jinzhe Zeng <[email protected]>
wanghan-iapcm
wanghan-iapcm reviewed Feb 17, 2024
View reviewed changes
Copy link
Collaborator

@wanghan-iapcm wanghan-iapcm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the case of mixed_type, or not distinguish_type is not covered by UTs.

deepmd/pt/utils/env_mat_stat.py Show resolved Hide resolved
deepmd/pt/utils/env_mat_stat.py Outdated Show resolved Hide resolved
source/tests/pt/test_stat.py Outdated Show resolved Hide resolved
njzjz added 4 commits February 17, 2024 17:39
@njzjz
rm unused compute_std
b47014b 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
bugfix
ef5a92e 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
Merge branch 'devel' into env-mat-stat
367f472 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
make test work
a8aef18 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
Copy link
Member Author

njzjz commented Feb 18, 2024

It seems that the case of mixed_type, or not distinguish_type is not covered by UTs.

Resolved.

@njzjz njzjz requested a review from wanghan-iapcm February 18, 2024 01:18
njzjz added 2 commits February 18, 2024 02:51
@njzjz
add type_map to stat_file_path
6a83465 
Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz
update share_params
ba688a9 
Signed-off-by: Jinzhe Zeng <[email protected]>
@wanghan-iapcm wanghan-iapcm added this pull request to the merge queue Feb 18, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 18, 2024
@wanghan-iapcm wanghan-iapcm merged commit c55ceac into deepmodeling:devel Feb 18, 2024
49 checks passed
njzjz added a commit to njzjz/deepmd-kit that referenced this pull request Feb 18, 2024
@njzjz
pt: process frames in parallel for env mat stat
117acd9 
Resolves deepmodeling#3285 (comment)

Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz njzjz mentioned this pull request Feb 18, 2024
Merged
github-merge-queue bot pushed a commit that referenced this pull request Feb 19, 2024
@njzjz
pt: process frames in parallel for env mat stat (#3293)
ab35468 
Resolves
#3285 (comment).

We don't even need to consider whether the data uses mixed type in this
piece of code. The data can be reshaped to the atom level, and type
masks can be used to get an environmental matrix with certain types. No
frame-level things are involved here.

---------

Signed-off-by: Jinzhe Zeng <[email protected]>
@njzjz njzjz mentioned this pull request Apr 2, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@iProzd iProzd iProzd approved these changes

@wanghan-iapcm wanghan-iapcm Awaiting requested review from wanghan-iapcm

Assignees
No one assigned
Labels
Python
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@njzjz @iProzd @wanghan-iapcm