Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Increasing rel_err due to test failures. #1836

Merged
merged 1 commit into from
May 24, 2022
Merged

Increasing rel_err due to test failures. #1836

merged 1 commit into from
May 24, 2022

Conversation

joshua-cogliati-inl
Copy link
Contributor

@joshua-cogliati-inl joshua-cogliati-inl commented May 23, 2022 •
edited by wangcj05
Loading

Pull Request Description

What issue does this change request address?

#1806
This addresses failures in tests like: https://civet.inl.gov/job/1052344/

DIFF in tests/framework/PostProcessors/TSACharacterizer/Basic/chz.csv: 
  Different values in ARMA__signal_fa__constant for 0.0193421236263 and 0.0184069534627
  | Difference | statistics:
    MEAN    diff.: 9.417981434e-04
    LARGEST diff.: 5.080526582e-02
(371F1/801) Diff   (  5.36sec)tests/framework/PostProcessors/TSACharacterizer/Basic

DIFF in tests/framework/ROM/TimeSeries/SyntheticHistory/ARMA/samples_1.csv: 
  Different values in signal0 for 0.000200007514986 and 0.000288985523409
  | Difference | statistics:
    MEAN    diff.: 3.070426729e-04
    LARGEST diff.: 3.078978053e-01
(531F2/801) Diff   (  3.98sec)tests/framework/ROM/TimeSeries/SyntheticHistory/ARMA
What are the significant changes in functionality due to this change request?

Increases the rel_err for two unstable tests.

For Change Control Board: Change Request Review

The following review must be completed by an authorized member of the Change Control Board.

  • 1. Review all computer code.
  • 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
  • 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
  • 4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
  • 5. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options. Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in XML block, the node <internalParallel> to True.
  • 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change. The requirements and the requirements test shall be in sync.
  • 7. The merge request must reference an issue. If the issue is closed, the issue close checklist shall be done.
  • 8. If an analytic test is changed/added is the the analytic documentation updated/added?
  • 9. If any test used as a basis for documentation examples (currently found in raven/tests/framework/user_guide and raven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.

wangcj05
wangcj05 approved these changes May 23, 2022
View reviewed changes
Copy link
Collaborator

@wangcj05 wangcj05 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No actual code changes, but rel_err update due to potential libraries update,

@wangcj05
Copy link
Collaborator

checklist satisfied, and PR can be merged.

@wangcj05 wangcj05 merged commit 71b13df into idaholab:devel May 24, 2022
@joshua-cogliati-inl joshua-cogliati-inl deleted the arma_rel_err branch May 24, 2022 13:30
dgarrett622 pushed a commit to dgarrett622/raven that referenced this pull request Jun 1, 2022
@joshua-cogliati-inl @dgarrett622
Increasing rel_err due to test failures. (idaholab#1836)
ab6d140
PaulTalbot-INL added a commit that referenced this pull request Jun 15, 2022
@dgarrett622 @PaulTalbot-INL
@wanghy-anl @joshua-cogliati-inl @wangcj05 @dylanjm
Re-running RAVEN workflows in Python/Jupyter environments (#1843)
5ae6e56 
* idiot-proof getEntity method

* splitting and cleaning XMLread

* fix missing f-statement

* update Simulation to allow subsequent run() calls

* formatting updates

* create flushOutputDataObject in DataObject and DataSet to reinitialize output DataObjects for successive workflow runs

* fixing assembler issue

* removing getMessageHandler from builtins

* removing duplicate import statement

* fixing flushOutputDataObject for when 'prefix' is not a part of the object

* restart time each time a simulation is run

* ensure time resets when simulation reinitialized

* plots generate when RAVEN workflow rerun

* fixing DeprecationWarning message

* fixing printing issue for HistorySet

* reset warning messages if workflow ran previously

* fixing colorMap specified does not exist error

* fixing 'Tried to add new data to cNDarray' for PostProcessor tests

* fixing 'Tried to add new data to cNDarray' for ARMA tests

* fixing h5py 'Unable to create group' issue

* fix issue with controlFunction in LogicalModel

* HybridModel can be re-run in workflow

* fixing GeneralPlot.py for re-running OutStreams tests

* fix ['prefix'] not in index error

* fixing issue seen with GeneticAlgorithm

* reset Steps, SolutionExport, and GradientDescent Optimizer for rerunning workflows

* finish up flushing GradientDescent Optimizer

* flushing GeneticAlgorithm Optimizer

* finish flushing SimulatedAnnealing

* flushing AdaptiveMonteCarlo and other Samplers

* improving flushDataObject

* flushing Sobol Sampler

* flushing AdaptiveSparseGrid

* fixing NetCDF issue where values were appended instead of overwritten

* adding flushSampler to Stratified

* updating _inputMetaVars for HistorySet

* flush Databases using (mostly) existing initializeDatabase

* flushing additional Samplers

* fixing re-initialization of HDF5 databases for re-running workflows

* simplifying Sampler inheritance

* formatting touched files

* flushing metadataKeys for Samplers and Optimizers

* adjust spacing for Run complete! message

* updating tests to check for re-running RAVEN workflow

* fixing f-string issue

* fix new test for re-running RAVEN workflow

* vargroups in rrr dataobjects (#1823)

* Farm submodule update (#1826)

* update FARM submodule version

* Parallel improvements (#1825)

* If ray instatiated outside, use it.

Basically, before, ray was only used if nodes was setup,
such as with MPI mode.

* No longer automatically adds mpi mode to inner ravens.

* Make it a bigger test, and switch to internalParallel

* Adding useful debugging information.

* Wait for servers to finish starting.

Otherwise they become zombies.

* Only print out changed part.

* Adding debugging info, and force status to disk.

* Setting port to 0 so ray chooses an available port.

This adds port as a parameter to the starting ray function,
and sends in 0.  This tells ray to choose an available port,
instead of erroring if port 6379 is not available.

* Find correct ray start

`ray start sometimes appears, so add a space before it.

* Don't start JobHandler if running remotely.

If we are going to be running remotely, then we should
not start job handler. Otherwise, we have to start ray or threads, only
to seconds later shutdown them down.

* Update scikit learn to 0.24 (#1831)

* Remove old pre-0.17 compatibility code.

* Properly handling 'None' and 'most_frequent' outlier_label

Checking if outlier_label exists before looking at it.

* Fixing default to be float.

* changing sklearn to 0.24

* Regolding and test modification for sklearn 0.24

* Doubling relative error.

* Labels are somewhat random, so don't print them.

Just check that all the means are the same.

* Updating generated documents.

* Removing check for version for StackingRegressor since 0.24 is now minimum.

* Adding StackingRegressor

* Setup changes (#1748)

* Converting to ravenframework

* Fix pluginhandler.

* Making pyDOE use relative imports.

* Adding __init__.py so lazy is found.

* pyDOE now really in contrib.

* Make checking libraries optional.

* Updating for raven package.

* Fixing library_report.

* There was both a CodeInterfaces directory and a CodeInterfaces.py This caused problems.

* Support python 2.0 for utils.

* Skip library check if library handler not found.

* Adds ability to generate setup.cfg requirements section.

Usage:
python3 ./scripts/library_handler.py pip --action=setup.cfg > setup.cfg

* Updating things from review.

* Increasing rel_err due to test failures. (#1836)

* update LOGOS submodule (#1835)

* HERON Submodule Update (#1837)

* reverting PythonRaven to address issues caused by recent changes

* reverting PythonRaven again due to circular imports

* reverting getMessageHandler

* fixing formatting in DataSet.py

* adding resetHybridModel method

* formatting GradientDescent

* restoring commented code block and fixing variable name

* formatting comments

* updating to common 'flush' method

* adding resetSimulation functionality

* formatting f-string in MultiRun

Co-authored-by: Paul Talbot <[email protected]>
Co-authored-by: Haoyu Wang <[email protected]>
Co-authored-by: Joshua J. Cogliati <[email protected]>
Co-authored-by: Congjian Wang - INL <[email protected]>
Co-authored-by: Dylan McDowell <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wangcj05 wangcj05 wangcj05 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@joshua-cogliati-inl @wangcj05