fix: all transform_inputs to handle cases where index column is set as time_col #248

Yibei990826 · 2024-03-08T22:51:50Z

Related to issue #208:
When running the df bellow:

	value
timestamp
2007-12-10	9.590761
2007-12-11	8.519590
2007-12-12	8.183677
2007-12-13	8.072467
2007-12-14	7.893572

The original code would generate error, which is resulted from not successfully recognizing index column and its column name:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
[<ipython-input-15-724adc6484c1>](https://localhost:8080/#) in <cell line: 1>()
----> 1 timegpt_anomalies_df = timegpt.detect_anomalies(pm_df, time_col='timestamp', target_col='value', freq='D')
      2 timegpt_anomalies_df.head()

6 frames
[/usr/local/lib/python3.10/dist-packages/nixtlats/timegpt.py](https://localhost:8080/#) in detect_anomalies(self, df, freq, id_col, time_col, target_col, level, clean_ex_first, validate_token, date_features, date_features_to_one_hot, model, num_partitions)
   1251         """
   1252         if isinstance(df, pd.DataFrame):
-> 1253             return self._detect_anomalies(
   1254                 df=df,
   1255                 freq=freq,

[/usr/local/lib/python3.10/dist-packages/nixtlats/timegpt.py](https://localhost:8080/#) in wrapper(self, *args, **kwargs)
    663                 f'supported models: {", ".join(self.supported_models)}'
    664             )
--> 665         return func(self, *args, **kwargs)
    666 
    667     return wrapper

[/usr/local/lib/python3.10/dist-packages/nixtlats/timegpt.py](https://localhost:8080/#) in wrapper(self, num_partitions, **kwargs)
    681     def wrapper(self, num_partitions, **kwargs):
    682         if num_partitions is None or num_partitions == 1:
--> 683             return func(self, **kwargs, num_partitions=1)
    684         df = kwargs.pop("df")
    685         X_df = kwargs.pop("X_df", None)

[/usr/local/lib/python3.10/dist-packages/nixtlats/timegpt.py](https://localhost:8080/#) in _detect_anomalies(self, df, freq, id_col, time_col, target_col, level, clean_ex_first, validate_token, date_features, date_features_to_one_hot, model, num_partitions)
    860             max_wait_time=self.max_wait_time,
    861         )
--> 862         anomalies_df = timegpt_model.detect_anomalies(df=df)
    863         self.weights_x = timegpt_model.weights_x
    864         return anomalies_df

[/usr/local/lib/python3.10/dist-packages/nixtlats/timegpt.py](https://localhost:8080/#) in detect_anomalies(self, df)
    565         # ie X_df = [X_df_history, X_df]
    566         # exogenous variables are passed after df
--> 567         df, _ = self.transform_inputs(df=df, X_df=None)
    568         main_logger.info("Preprocessing dataframes...")
    569         Y_df, X_df = self.preprocess_dataframes(df=df, X_df=None)

[/usr/local/lib/python3.10/dist-packages/nixtlats/timegpt.py](https://localhost:8080/#) in transform_inputs(self, df, X_df)
    204         }
    205         df = df.rename(columns=renamer)
--> 206         if df.dtypes.ds != "object":
    207             df["ds"] = df["ds"].astype(str)
    208         if "unique_id" not in df.columns:

[/usr/local/lib/python3.10/dist-packages/pandas/core/generic.py](https://localhost:8080/#) in __getattr__(self, name)
   5900         ):
   5901             return self[name]
-> 5902         return object.__getattribute__(self, name)
   5903 
   5904     def __setattr__(self, name: str, value) -> None:

AttributeError: 'Series' object has no attribute 'ds'

CLAassistant · 2024-03-08T22:51:56Z

All committers have signed the CLA.

github-actions · 2024-03-08T22:53:29Z

Experiment 1: air-passengers

Description:

variable	experiment
h	12
season_length	12
freq	MS
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	12.6793	11.0623	47.8333	76
mape	0.027	0.0232	0.0999	0.1425
mse	213.936	199.132	2571.33	10604.2
total_time	3.4273	4.4439	0.0092	0.0052

Plot:

Experiment 2: air-passengers

Description:

variable	experiment
h	24
season_length	12
freq	MS
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	58.1031	58.4587	71.25	115.25
mape	0.1257	0.1267	0.1552	0.2358
mse	4040.21	4110.79	5928.17	18859.2
total_time	7.817	9.9332	0.0062	0.0056

Plot:

Experiment 3: electricity-multiple-series

Description:

variable	experiment
h	24
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	142.394	196.363	269.23	1331.02
mape	0.0203	0.0234	0.0304	0.1692
mse	63464.7	123119	213677	4.68961e+06
total_time	6.0905	4.9839	0.0086	0.0074

Plot:

Experiment 4: electricity-multiple-series

Description:

variable	experiment
h	168
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	522.427	353.528	398.956	1119.26
mape	0.069	0.0454	0.0512	0.1583
mse	966294	422332	656723	3.17316e+06
total_time	5.0144	3.7813	0.0082	0.0076

Plot:

Experiment 5: electricity-multiple-series

Description:

variable	experiment
h	336
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	478.362	361.033	602.926	1340.95
mape	0.0622	0.046	0.0787	0.17
mse	805039	441118	1.61572e+06	6.04619e+06
total_time	5.3238	4.8189	0.0084	0.0078

Plot:

…ial (#239)

review-notebook-app · 2024-03-12T18:25:19Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

github-actions · 2024-03-12T18:27:01Z

Experiment Results

Experiment 1: air-passengers

Description:

variable	experiment
h	12
season_length	12
freq	MS
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	12.6793	11.0623	47.8333	76
mape	0.027	0.0232	0.0999	0.1425
mse	213.936	199.132	2571.33	10604.2
total_time	4.5173	3.5914	0.0093	0.0055

Plot:

Experiment 2: air-passengers

Description:

variable	experiment
h	24
season_length	12
freq	MS
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	58.1031	58.4587	71.25	115.25
mape	0.1257	0.1267	0.1552	0.2358
mse	4040.21	4110.79	5928.17	18859.2
total_time	9.6629	9.8716	0.0062	0.0054

Plot:

Experiment 3: electricity-multiple-series

Description:

variable	experiment
h	24
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	142.394	196.363	269.23	1331.02
mape	0.0203	0.0234	0.0304	0.1692
mse	63464.7	123119	213677	4.68961e+06
total_time	10.6422	4.4929	0.0085	0.0078

Plot:

Experiment 4: electricity-multiple-series

Description:

variable	experiment
h	168
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	522.427	353.528	398.956	1119.26
mape	0.069	0.0454	0.0512	0.1583
mse	966294	422332	656723	3.17316e+06
total_time	6.4141	4.7437	0.0085	0.0077

Plot:

Experiment 5: electricity-multiple-series

Description:

variable	experiment
h	336
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	478.362	361.033	602.926	1340.95
mape	0.0622	0.046	0.0787	0.17
mse	805039	441118	1.61572e+06	6.04619e+06
total_time	7.5091	4.75	0.0083	0.0076

Plot:

github-actions · 2024-03-15T05:57:50Z

Experiment Results

Experiment 1: air-passengers

Description:

variable	experiment
h	12
season_length	12
freq	MS
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	12.6793	11.0623	47.8333	76
mape	0.027	0.0232	0.0999	0.1425
mse	213.936	199.132	2571.33	10604.2
total_time	9.8087	7.5149	0.0094	0.0055

Plot:

Experiment 2: air-passengers

Description:

variable	experiment
h	24
season_length	12
freq	MS
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	58.1031	58.4587	71.25	115.25
mape	0.1257	0.1267	0.1552	0.2358
mse	4040.21	4110.79	5928.17	18859.2
total_time	6.5513	3.8154	0.0061	0.0052

Plot:

Experiment 3: electricity-multiple-series

Description:

variable	experiment
h	24
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	142.394	196.363	269.23	1331.02
mape	0.0203	0.0234	0.0304	0.1692
mse	63464.7	123119	213677	4.68961e+06
total_time	4.883	3.7156	0.0085	0.0077

Plot:

Experiment 4: electricity-multiple-series

Description:

variable	experiment
h	168
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	522.427	353.528	398.956	1119.26
mape	0.069	0.0454	0.0512	0.1583
mse	966294	422332	656723	3.17316e+06
total_time	5.1339	4.8501	0.0083	0.008

Plot:

Experiment 5: electricity-multiple-series

Description:

variable	experiment
h	336
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	478.362	361.033	602.926	1340.95
mape	0.0622	0.046	0.0787	0.17
mse	805039	441118	1.61572e+06	6.04619e+06
total_time	7.8826	6.4333	0.0083	0.008

Plot:

github-actions · 2024-03-15T06:52:04Z

Experiment Results

Experiment 1: air-passengers

Description:

variable	experiment
h	12
season_length	12
freq	MS
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	12.6793	11.0623	47.8333	76
mape	0.027	0.0232	0.0999	0.1425
mse	213.936	199.132	2571.33	10604.2
total_time	2.606	3.7618	0.0093	0.0053

Plot:

Experiment 2: air-passengers

Description:

variable	experiment
h	24
season_length	12
freq	MS
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	58.1031	58.4587	71.25	115.25
mape	0.1257	0.1267	0.1552	0.2358
mse	4040.21	4110.79	5928.17	18859.2
total_time	4.2367	5.8082	0.0065	0.0056

Plot:

Experiment 3: electricity-multiple-series

Description:

variable	experiment
h	24
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	142.394	196.363	269.23	1331.02
mape	0.0203	0.0234	0.0304	0.1692
mse	63464.7	123119	213677	4.68961e+06
total_time	9.3149	10.822	0.0085	0.0075

Plot:

Experiment 4: electricity-multiple-series

Description:

variable	experiment
h	168
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	522.427	353.528	398.956	1119.26
mape	0.069	0.0454	0.0512	0.1583
mse	966294	422332	656723	3.17316e+06
total_time	7.7652	3.3356	0.0082	0.0089

Plot:

Experiment 5: electricity-multiple-series

Description:

variable	experiment
h	336
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	478.362	361.033	602.926	1340.95
mape	0.0622	0.046	0.0787	0.17
mse	805039	441118	1.61572e+06	6.04619e+06
total_time	5.8391	3.5846	0.0085	0.0081

Plot:

MMenchero and others added 3 commits March 5, 2024 23:51

WIP: Added new sections to FAQs

a548022

feat: Added new sections to FAQs

68194d6

fix transform_inputs when time col is index col

8780767

Yibei990826 marked this pull request as draft March 8, 2024 22:52

Yibei990826 marked this pull request as ready for review March 8, 2024 23:28

marcopeix and others added 4 commits March 8, 2024 17:39

Add finetuning loss function tutorial and link it to finetuning tutor…

4134175

…ial (#239)

chore: collapse experiment results in pr (#247)

2f98dcf

WIP: New version with suggested changes

a26ae63

used nbdev to update

9d0b789

MMenchero and others added 11 commits March 12, 2024 18:57

Added complete FAQs, new tutorial and updated sidebar

f6b9f45

Added feedback given by Max

7449b17

[DOC] - Tutorial on long-horizon forecasting (#249)

21d23aa

add pre-commit (#252)

2af94d4

Added additional feedback

e5ccf7a

Minor change

725ff49

Merge remote-tracking branch 'origin' into feat/new_faqs

d290ff6

Feat/new faqs (#250)

4ca9418

fix transform_inputs when time col is index col

810a55f

used nbdev to update

5923875

added testing

7e0c964

Update .gitignore

de17ecc

Yibei990826 closed this Mar 15, 2024

Yibei990826 deleted the fix/transform_inputs branch March 15, 2024 17:29

fix: all transform_inputs to handle cases where index column is set as time_col #248

fix: all transform_inputs to handle cases where index column is set as time_col #248

Conversation

Yibei990826 commented Mar 8, 2024 • edited Loading

CLAassistant commented Mar 8, 2024 • edited Loading

github-actions bot commented Mar 8, 2024

Experiment 1: air-passengers

Description:

Results:

Plot:

Experiment 2: air-passengers

Description:

Results:

Plot:

Experiment 3: electricity-multiple-series

Description:

Results:

Plot:

Experiment 4: electricity-multiple-series

Description:

Results:

Plot:

Experiment 5: electricity-multiple-series

Description:

Results:

Plot:

review-notebook-app bot commented Mar 12, 2024

github-actions bot commented Mar 12, 2024

Experiment 1: air-passengers

Description:

Results:

Plot:

Experiment 2: air-passengers

Description:

Results:

Plot:

Experiment 3: electricity-multiple-series

Description:

Results:

Plot:

Experiment 4: electricity-multiple-series

Description:

Results:

Plot:

Experiment 5: electricity-multiple-series

Description:

Results:

Plot:

github-actions bot commented Mar 15, 2024

Experiment 1: air-passengers

Description:

Results:

Plot:

Experiment 2: air-passengers

Description:

Results:

Plot:

Experiment 3: electricity-multiple-series

Description:

Results:

Plot:

Experiment 4: electricity-multiple-series

Description:

Results:

Plot:

Experiment 5: electricity-multiple-series

Description:

Results:

Plot:

github-actions bot commented Mar 15, 2024

Experiment 1: air-passengers

Description:

Results:

Plot:

Experiment 2: air-passengers

Description:

Results:

Plot:

Experiment 3: electricity-multiple-series

Description:

Yibei990826 commented Mar 8, 2024 •

edited

Loading

CLAassistant commented Mar 8, 2024 •

edited

Loading