forecasts.Rmd

---
title: "Forecasting in IR"
subtitle: "Security Policy Forecasting Tournament"
author: "Alexander Sacharow"
date: "02 May 2017"
output: 
  ioslides_presentation:
    css: general/tables.css
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = FALSE)

# Clear Global environment
rm(list=ls())

## Setting Working directory
try(setwd("D:/Eigene Datein/Dokumente/Uni/Hertie/Materials/Master thesis/SecurityPolicyForecastingTournament"), silent = TRUE)

library(knitr)

source("main.R")
```

## Why Forecasting? {.text40}

<div class="centered">
What do we want to achieve with forecasting?
</div>

## Why Forecasting?

<div class="columns-2">

<div style="color:#C02F39;">**Authors: Scientists, Analysts**</div>

- to get it right

    - value in itself?

- test theories (Friedman, 1966; Ward, 2016)

- Central function of science next to explanation

    - how much can we know?

<div style="color:#C02F39;">**Users: Policy makers etc.**</div>

- basis of decision making

    - influence future

    - show alternatives

- Early-Warning

    - Basis for preparation
</div>

## Is Forecasting Possible?

<div class="centered">

<p style="font-size: 40px">
Is successful forecasting in international politics possible at all?
</p>

</div>


## Is Forecasting Possible?

<div class="columns-2">

<div style="color:#C02F39;">**The Skeptics**</div>

- Politics are like *Clouds* (Almond et al., 1977)

- Non-Linearity
   
    - limit to conclusions from the past
   
- *Black swan* (Taleb, 2007)
   
    - High impact events are often unexpected
   
    - e.g. discovery of America, Arab Spring

<div style="color:#C02F39;">**The Optimists**</div>

- *The poets*
   
    - Yes, but without probabilities (Scenarios, Mega trends)
  
- *The mathematicians*

    - Yes, with sufficient data (Chourci, 1974; Ward, 2016)
  
- With crowds
  
    - Groups, competitions, markets

</div>

## Forecasting methods - overview

1. **Strategic Foresight**

2. **Statistical models**

3. **Forecasting competitions**

4. Intuitive forecasts

5. Expert forecasts

6. Prediction markets

7. Intelligence Estimates

## Strategic Foresight

Example: [European Union in the World 2025](http://www.dahrendorf-forum.eu/wp-content/uploads/2016/05/Dahrendorf_Analysis_EU_2025_15.pdf) by Dahrendorf Forum

- Essentially: Scenario analysis

- originated and widely used in the corporate sector

    - Shell Foresight 1960s (Pierre Wack)

<div style="color:#C02F39;">**Advantage:**</div>

- No content restrictions

- consistency of scenarios as key quality criteria

<div style="color:#C02F39;">**Disadvantage:**</div> 

- No likelihood judgements (policy maker ask for it)

- how can scenarios be verify?

## Statistical Models

Example: *World-Wide Integrated Crisis Early Warning System (W-ICEWS)* [(Preview)](http://nbviewer.jupyter.org/gist/dmasad/f79ce5abfd4fb61d253b)

<div style="color:#C02F39;">**Advantage:**</div>

- can be verified

- less decision biases

<div style="color:#C02F39;">**Disadvantage:**</div>

- requires measureable events

    - e.g. number of  protesters / injured / death
    
- not applicable to many relevant questions

    - e.g. International Court of Justice decision forecast
    
- data often not existing / too expensive

## Forecasting competitions

Example: [The Good Judgement Project](www.gjopen.com)

- for geopolitics relatively new (first in 2011)

- combine individual judgements and track record keeping of formal model

- Researchers: Philip Tetlock and Barbara Mellers

  <div align="center">
 <img src="general/superforecasting.jpg" width=160 height=250>
 </div>

# Security Policy Forecasting Tournament

## Security Policy Forecasting Tournament

- 231 participants in total

    - 53 Hertie students participated (48 on time, 5 late)

- short-term: 2.5 months

    - other tournaments include question up to one / one and a half years

- 24 questions

    - restricted to topics of the Security Policy class
    
    - questions were externally reviewed before the tournament

## Performance

```{r, echo = FALSE, fig.width=4, fig.height=2.6, out.extra='style="float:left"'}
response.uni(9) + geom_vline(xintercept=0,  colour="blue", linetype="dashed", size=1.5)

response.uni(16) + geom_vline(xintercept=1,  colour="blue", linetype="dashed", size=1.5)
```

```{r, echo = FALSE, fig.width=4, fig.height=2.6, out.extra='style="float:left"'}
response.uni(13) + geom_vline(xintercept=0,  colour="blue", linetype="dashed", size=1.5)

response.uni(6) + geom_vline(xintercept=0,  colour="blue", linetype="dashed", size=1.5)
```

## Score distribution

- Score: Brier score

    - intuition: closeness to truth
    
    - mathematical: squared errors

- Hertie students compared to other participants


```{r, echo = FALSE, fig.width=5, fig.height=3}
ggplot(SB, aes(x = brier.avg, fill = part.group)) +
     geom_histogram(binwidth=.1, position="dodge") + # bar type
     #geom_density(alpha=.3) +
     theme_bw() +
     theme(#legend.title=element_blank(),
           axis.title = element_text(size=18, colour = "#696969", family = "serif"), # Labels axis font size
           axis.text = element_text(size=14, colour = "#696969"),
           axis.line = element_line(colour = "#696969"), 
           axis.ticks = element_line(colour = "#696969")) +  
     labs( # title = "Brier score distribution",
          fill = "Participant group",
          x = "Brier score",
          y = "Frequency") # labels))
```


## The research in the background

data used for testing, e.g.

- **Hypothesis 2: The marginal added value of time spend on forecasting is positive and decreases over time**

```{r, echo = FALSE, fig.width=5, fig.height=3.5, out.extra='style="float:center"'}
cor.brier.time.log.plot
```

## Scoreboard

```{r, echo = FALSE, message=FALSE, warning=FALSE}

SB %>% merge(read.csv("personalData.csv")[, c("ResponseId", "id.uni")], 
                      by = "ResponseId") %>% filter(part.group == "uni") %>% arrange(brier.avg) %>% 
     select(HertieID = id.uni, brier.avg) %>%  head(10) %>% mutate(rank = row_number()) %>% kable(format = "html")
```
## Questions {.text40}

<div class="centered" style="top:35%; position: fixed;">
Thank you for your attention
</div>