Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add method to log custom visualizations in Tracker #149

Open
mibarguen opened this issue Dec 23, 2021 · 1 comment
Open

Add method to log custom visualizations in Tracker #149

mibarguen opened this issue Dec 23, 2021 · 1 comment

Comments

@mibarguen
Copy link

mibarguen commented Dec 23, 2021

Is your feature request related to a problem? Please describe.
My feature is not related to a specific problem. However, it would be useful to have the ability to store custom plots that evaluate the results of each experiment. For example, we could store Shapley Feature Explanation plots or simple feature importance plots for each experiment. It would be helpful to have the ability to upload these images/plots as part of each trial.

Describe the solution you'd like

with Tracker.create(display_name="Preprocessing", sagemaker_boto_client=sm) as tracker:
    feat_importances = pd.DataFrame(model.feature_importances_, index=iris.feature_names, columns=["Importance"])
    feat_importances.sort_values(by='Importance', ascending=False, inplace=True)
    ft_importances_plot = feat_importances.plot(kind='bar', figsize=(8,6))
    tracker.log_figure(name="feature-importances", ft_importances_plot)

Ideally, the Tracker class would have a method called log_figure that would allow us to log any figure that we would consider useful.

Describe alternatives you've considered
I have considered using the log_artifact method of the Tracker class. This method is not implemented specifically for visualizations.

Additional context
I believe Comet has this feature implemented Experiment.log_figure method. It would be extremely useful to add this feature to sagemaker-experiments.

Thank you!

@helinmik
Copy link

helinmik commented Nov 3, 2022

+1 to this. Logging images and custom plots feels like a must have feature when trying to evaluate a model. This is supported also in other similar tools as Tensorboard and MLFlow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants