discussion.tex

%\section{Discussion (3 pages)}
%\label{sec:discussion}

%\subsection{Application to real world case}
\section{Application to the Higgs case}
\label{sec:discussion}
\label{sec:discussion:higgs}

%HOW APPLIED TO HIGGS ANALYSIS.

The actual CMS
%$H \rightarrow \gamma\gamma$ 
Higgs to two photon analysis~\cite{ref:introduction:legacy}
is significantly more
complex than the simplified version used here. In particular, the 2011 
and 2012 data
samples are split into eleven and fourteen categories, respectively, which have
differing signal to background ratios.
Because the categories (by definition) have different selection criteria,
they can have different background shapes.
There is no {\it a priori} reason to make any assumptions that the functions
used in each category should be the same. Hence, each category should be
tested with all functions, in a similar way to the above.

A major complication then arises because there are common systematic effects
across the categories, arising from nuisance parameters in the signal
model.
In the absence of these common nuisance parameters,
the different categories could be profiled independently, each using the
minimum envelope technique to produce an envelope curve per category.
These could then
be summed to give the overall profile curve. However, with common
nuisance parameters, all categories must be profiled at the same time.
Since minimisation code to handle the 
discrete nuisance parameter identifying the
function seems difficult, in practical terms, this means that all possible
combinations of each function in each category must be fitted.
The minimum envelope made from the results of all these combinations would
then be found. While this is conceptually straightforward, the actual
implementation is prohibitive. For example, using the 16 functions
discussed in this paper, there would be $16^{11} (\sim 10^{13}$) combinations 
of functions to be fitted for the 2011 data sample and $16^{14} (\sim 10^{17}$)
combinations for the 2012 data sample.
Instead of considering all possible combinations of functions, the function 
which minimizes the likelihood and its parameters can be determined through 
an iterative procedure. The first step is to determine the set of functions 
which best fit the data, in each category, fixing the values of all 
signal nuisance parameters.
Since the signal parameter, $\mu$, is also fixed for a given point in the scan 
no free parameters are correlated between categories.
Finding the functions which minimize the likelihood in 
this case then reduces to the simple case, namely finding the minimum in each 
category independently. 
Once these functions, and their parameters, are found, the choice of function 
is then fixed while the signal nuisance parameters are freed and a 
single fit of these
nuisance parameters and chosen background parameters is performed.
If no significant change in the likelihood is observed, the overall minimum
has been 
achieved. However, if this is not the case, the nuisances must be fixed again 
and the procedure to find the best fit functions given the new values of the 
nuisance parameters is repeated.
This iterative procedure continues until the likelihood remains stable.
This procedure dramatically reduces the number of fits to be performed 
to minimize the likelihood and is found to produce identical results when 
compared to trying all combinations in toy examples. 
The bias and coverage properties of the discrete profiling method were tested 
extensively in the CMS Higgs analysis using a similar procedure to that described in 
Section~\ref{sec:correction:bias}. The method was found to give consistently good 
coverage and negligible bias.


%\subsection{Envelope smoothing}
%\label{sec:discussion:smoothing}

%DISCRETE FUNCTIONS MEAN NON-SMOOTH ENVELOPE

%APPLY SMOOTHING USING [SUM \nll EXP(-\nll/2)]/[SUM EXP(-\nll/2)]?


%\subsection{Bayesian equivalent method}
%\label{sec:discussion:bayesian}