Remove the dependency on pygam

[jeongyoonlee]

Is your feature request related to a problem? Please describe.
pygam pins scipy < 1.12, which blocks the use of numpy 2.x and Cython 3.x.

Describe the solution you'd like
Currently, it is used in propensity.calibrate().

We can evaluate alternative calibration methods, e.g., scikit-learn's probability calibration.

Describe alternatives you've considered
Alternatively, we can work on pygam directly to see if we can make it work with scipy >= 1.12

Additional context
N/A

[ras44]

hi @jeongyoonlee I took a quick look at this in this notebook:

https://github.com/ras44/causalml/blob/ras44/remove_pygam_dep_796_dev/docs/examples/pygam_removal_and_testing.ipynb

If we look at log_loss and brier_score_loss independently as indicators of calibration quality (as described here), then it looks like we should be able to swap out pygam with sci-kit learn's IsotonicRegression and either maintain or improve performance:

def calibrate(ps, treatment):
    """Calibrate propensity scores with IsotonicRegression.

    Args:
        ps (numpy.array): a propensity score vector
        treatment (numpy.array): a binary treatment vector (0: control, 1: treated)

    Returns:
        (numpy.array): a calibrated propensity score vector
    """
    pm_ir = IsotonicRegression(out_of_bounds="clip")
    ps_ir = pm_ir.fit_transform(ps, treatment)

    return ps_ir

At the end of the notebook above, I show that IsotonicRegression performs better than or equal to pygam for a couple test cases: