-
Notifications
You must be signed in to change notification settings - Fork 36
/
Copy pathannotated.Rmd
417 lines (272 loc) · 27.4 KB
/
annotated.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
# Annotated Bibliography
This is less an annotated and more of a citation and link dump while I move the references into the main text.
## Textbooks
- Kruschke (2015) *Doing Bayesian data analysis* [@Kruschke2015a] Another accessible introduction aimed at psychology.
[Website](https://sites.google.com/site/doingbayesiandataanalysis/) with additional material.
- McElreath (2016) *Statistical rethinking* [@McElreath2016a] An accessible introduction to Bayesian stats; effectively an intro-stats/linear models course taught from a Bayesian perspective.
- [GitHub page](https://github.com/rmcelreath/rethinking)
- [Course page](http://xcelab.net/rm/statistical-rethinking/)
- Lee (2012) *Bayesian Statistics : An Introduction* [@Lee2012a]
- Marin and Robert (2015) *Bayesian Essentials with R* [@MarinRobert2014a]
and [solutions manual](https://arxiv.org/pdf/1503.04662.pdf)
- Robert and Casella. 2009. *Introducing Monte Carlo Methods with R* [@RobertCasella2009a]
- Robert and Casella. 2004. *Monte Carlo statistical methods* [@RobertCasella2004a]
- Albert (2009) *Bayesian Computation with R* [@Albert2009a]
- Jackman (2009) *Bayesian Analysis for the Social Sciences* [@Jackman2009a] Covers commonly used models in the social sciences. Largely covers Gibbs sampling methods and
- Hoff (2009) *A First Course in Bayesian Statistical Methods* [@Hoff2009a]
- Gelman, Carlin, Stern, Dunson, and Vehtari (2013) *Bayesian data analysis* (3rd Edition) [@BDA3]
- Gelman, and Hill (2007) *Data analysis using regression and multilevel/hierarchical models* [@GelmanHill2007a] An accessible introduction to to linear models and multilevel models.
- Efron and Hastie (2016) *Computer Age Statistical Inference: Algorithms, Evidence, and Data Science* This is a unique work that blends an overview of statistical methods with a history of statistics. [@EfronHastie2016a]
- Robert (2007) *The Bayesian Choice* A statistics graduate-level book on Bayesian statistics.
- Berger (1993) *Statistical Decision Theory and Bayesian Analysis* [@Berger1993a] The classic book on Bayesian inference and decision theory.
The underlying statistical theory is still relevant even if its date makes the computational aspects less so.
- Murphy (2012) *Machine Learning: A Probabilistic Perspective* [@Murphy2012a] A machine learning book with a heavy Bayesian influence.
- MacKay (2003) *Information Theory, Inference, and Learning Algorithms* [URL](https://www.ece.uvic.ca/~agullive/Mackay.pdf).
[@MacKay2003a] On information theory, but combines it with Bayesian statistics, and is ultimately about learning and evidence.
Lectures from the course are available [here](http://videolectures.net/course_information_theory_pattern_recognition/).
- Gelman and Hill (2007) *Data Analysis Using Regression and Multilevel/Hierarchical Models* [@GelmanHill2007a]
- Gelman, Carlin, Stern, Dunson, Vehtari, and Rubin (2013) *Bayesian Data Analysis* 3rd ed.
- Jackman, Simon. 2009. *Bayesian Analysis for the Social Sciences* [@Jackman2009a]
- Lynch, Scott M. 2007. *Introduction to Applied Bayesian Statistics and Estimation for Social Scientists*
- Lunn, Jackson, Best, Thomas, and Spiegelhalter (2012) *The BUGS Book: A Practical Introduction to Bayesian Analysis* [@LunnJacksonBestEtAl2012a]
- Peter Hoff. 2009. *A First Course in Bayesian Statistical Methods* [@Hoff2009a]
- Congdon. 2014. Applied Bayesian Modeling.
- Marin and Roberts. 2014. [Bayesian Essentials with R.](http://www.springer.com/us/book/9781461486862)
- Robert and Casella. *Introducing Monte Carlo Methods with R* [@RobertCasella2009a]
## Syllabi
- Ryan Bakker and Johannes Karreth, "Introduction to Applied Bayesian Modeling" ICPSR. Summer 2016. [Syllabus](http://www.jkarreth.net/files/bayes2016.pdf); [code](https://github.com/jkarreth/Bayes)
- Justin Esarey. "Advanced Topics in Political Methodology: Bayesian Statistics" Winter 2015. [Syllabus](http://jee3.web.rice.edu/POLS506-syllabus-2015.pdf); [Lectures](http://jee3.web.rice.edu/teaching.htm).
- Kruschke. [Doing Bayesian Data Analysis site](https://sites.google.com/site/doingbayesiandataanalysis/).
- Nick Beauchamp. "Bayesian Methods." NYU. [syllabus](http://www.democraticwriting.com/work/Beauchamp_bayesian_syllabus.pdf).
- Alex Tanhk. "Bayesian Methods for the Social Sciences" U of Wisconsin. Spring 2017. [syllabus](https://polisci.wisc.edu/sites/polisci.wisc.edu/files/documents/syllabi/PS%20919%20.pdf).
- MTH225 Statistics for Science Spring 2016. [github website](https://github.com/equinn1/MTH225_Spring2016).
- Ben Goodrich, "Bayesian Statistics for Social Sciences" Columbia University. Spring 2016.
- Bakker. "Introduction to Applied Bayesian Analysis" University of Georgia. [syllabus](http://spia.uga.edu/faculty_pages/rbakker/bayes/bayes2016_maymester.pdf); [site](http://spia.uga.edu/faculty_pages/rbakker/bayes/POLS%20Bayes.htm)
- Myimoto. "Advances in Quantitative Psychology: Bayesian Statistics, Modeling & Reasoning" U of Washington. Winter 2017. [site](http://faculty.washington.edu/jmiyamot/p548/p548-set.htm)
- Neil Frazer. Bayesian Data Analysis. Hawaii. Spring 2017. [syllabus](http://www.soest.hawaii.edu/GG/resources/syllabi-S17/gg695-s17-syl.pdf)
- Lopes. 2016. Bayesian Statistical Learning: Readings in Statistics and Econometrics. [syllabus](http://hedibert.org/current-teaching/).
- Lopes. 2012 [Simulation-based approaches to modern Bayesian econometrics](http://hedibert.org/simulation-based-approaches-to-modern-bayesian-econometrics/). Short course.
- Lopes. 2015. Bayesian Econometrics. [syllabus](http://hedibert.org/current-teaching/).
## Topics
## Bayes' Theorem
- Puga, Kryzwinski, and Altman (2015) "[Points of significance: Bayes’ theorem](https://dx.doi.org/10.1038/nmeth.3335)" *Nature Methods*
## Article Length Introductions to Bayesian Statistics
- Stan Modeling 2.17. Ch. 29. "Bayesian Inference"
- Michael Clarke [Bayesian Basics](https://m-clark.github.io/docs/IntroBayes.html).
- Eddy (2004) "[What is Bayesian Statistics](https://dx.doi.org/10.1038/nbt0904-1177)" *Nature Biotechnology*
- Jackman. 2004. Bayesian Analysis for Political Research. *Annual Review of Political Science* DOI:10.1146/annurev.polisci.7.012003.104706.
- Kruschke, J.K. & Liddell, T.M. Psychon Bull Rev (2017). doi:10.3758/s13423-016-1221-4 - Kruschke and Liddell (2017) "[Bayesian new statistics: hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective](https://dx.doi.org/10.3758/s13423-016-1221-4)"
### Why Bayesian
- Jim Savage. [Why learn Bayesian Modeling?](http://modernstatisticalworkflow.blogspot.com/2017/04/why-learn-bayesian-modeling.html) April 10, 2017.
### Modern Statistical Workflow
- Savage, Jaim. 2017. [A Brief Introduction to Econometrics in Stan](https://khakieconomics.github.io/half_day_course/msw.html)
- Betancourt, Michael. [Robust Statistical Workflow with RStan](http://mc-stan.org/users/documentation/case-studies/rstan_workflow.html)
- Stan Modeling Guide "Model Building as Software Development"
- Gabry, J., Simpson, D., Vehtari, A., Betancourt, M., Gelman, A. (2018). [Visualization in Bayesian workflow](https://arxiv.org/abs/1709.01449)
### Bayesian Philosophy
- Gelman (2008) "[Objections to Bayesian Statistics](https://dx.doi.org/10.1214/08-ba318)" *Bayesian Analysis*
- Gelman and Shalizi (2012) "[Philosophy and the practice of Bayesian statistics](https://dx.doi.org/10.1111/j.2044-8317.2011.02037.x)" *British Journal of Mathematical and Statistical Psychology*
- Borsboom and Haig (2012) "[How to practice Bayesian statistics outside the Bayesian church: What philosophy for Bayesian statistical modelling?](10.1111/j.2044-8317.2012.02062.x)" *British Journal of Mathematical and Statistical Psychology*
- Berger and Berry (1988) "[Statistical Analysis and the Illusion of Objectivity](http://www.jstor.org/stable/27855070)" *American Scientist*
American Scientist 1988
- Efron (2010) "[The Future of Indirect Evidence](https://dx.doi.org/10.1214/09-STS308)"
- Efron (1986) "[Why Isn’t Everyone a Bayesian?](https://dx.doi.org/10.1080/00031305.1986.10475342)" *American Statistician* [@Efron1986a]. See comments @Chernoff1986a, @Lindley1986a, @Morris1986a, @Smith1986a, @Press1986a, @Efron1986b.
- [Philosophy and the practice of Bayesian statistics in the social sciences](http://www.stat.columbia.edu/~gelman/research/published/philosophy_chapter.pdf)
- Rubin (1984) Rubin, [Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician](http://projecteuclid.org/euclid.aos/1176346785)
- Andrew Gelman Induction and Deduction in Bayesian Data Analysis
- Berger (2013) "[Could Fisher, Jeffreys and Neyman Have Agreed on Testing?](https://dx.doi.org/10.1214/ss/1056397485) *Statistical Science*
### Bayesian Hypothesis Testing
- Gross, J. H. (2015) "[Testing What Matters (If You Must Test at All): A Context-Driven Approach to Substantive and Statistical Significance](https://dx.doi.org/10.1111/ajps.12149)" *American Journal of Political Science* [@Gross2014a]
### Bayesian Frequentist Debates
- [Bayesians and Frequentists : Models, Assumptions, and Inference](http://www.stat.ufl.edu/archived/casella/Talks/BayesRefresher.pdf) (slides)
- Kass [Statistical Inference: The Big Picture](https://arxiv.org/pdf/1106.2895v2.pdf)
- Noah Smith [Bayesian vs. Frequentist: Is there any "there" there?](http://noahpinionblog.blogspot.com/2013/01/bayesian-vs-frequentist-is-there-any.html)
- Kass [Kinds of Bayesians](http://www.stat.cmu.edu/~kass/papers/kinds.pdf)
- Anthony O'Hagan. Science, Subjectivity and Software (Comments on the articles by Berger and Goldstein)
- VanderPlas (2014) Frequentism and Bayesianism: A Python-driven Primer. [posts](http://jakevdp.github.io/blog/2014/06/14/frequentism-and-bayesianism-4-bayesian-in-python/)
### Categorical
- Agresti. [Bayesian Inference for Categorical Data Analysis](http://www.stat.ufl.edu/~aa/cda2/bayes.pdf)
- Gelman. 2008. "A weakly informative default prior distribution for logistic and other regression models"
- Rainey. 2016. "Dealing with Separation in Logistic Regression Models" *Political Analysis*
- @WechslerIzbickiEsteves2013a "A Bayesian look at nonidentifiability: a simple example""
### Variable Selection
- @GhoshGhattas2015a Ghosh and Ghattas (2015) "Bayesian Variable Selection Under Collinearity" *American Statistician*
- Scott and Berger (2011) "[Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem](https://dx.doi.org/10.1214/10-Aos792)" *Annals of Statistics* [@ScottBerger2010a]
- Ishwaran and Rao (2005) "[Spike and slab variable selection: Frequentist and Bayesian strategies](https://dx.doi.org/10.1214/009053604000001147)" *Annals of Statistics*
- Ishwaran, Kogalur, and Rao (2010) "[spikeslab: prediction and variable selection using spike and slab regression](ttps://journal.r-project.org/archive/2010-2/RJournal_2010-2_Ishwaran~et~al.pdf)" *R Journal*
- Polson and Scott. "[Shrink globally, act locally: sparse Bayesian regularization and prediction](https://dx.doi.org/10.1093/acprof:oso/9780199694587.003.0017)" *Bayesian Statistics*
- [Projection predictive variable selection using Stan + R](https://arxiv.org/abs/1508.02502)
- [Lasso Meets Horseshoe](https://arxiv.org/pdf/1706.10179.pdf)
- Piironen and Vehtari, [Sparsity information and regularization in the horseshoe and other shrinkage priors](https://arxiv.org/pdf/1706.10179.pdf)
- Hahn and Carvalho. [Decoupling Shrinkage And Selection In Bayesian Linear Models: A Posterior Summary Perspective](https://arxiv.org/pdf/1408.0464.pdf)
- Michael Betancourt [Bayes Sparse Regression](https://betanalpha.github.io/assets/case_studies/bayes_sparse_regression.html)
### Multiple Testing
- Gelman, Hill, and Yajima (2012) "[Why we (Usually) don’t have to worry about multiple comparisons](https://dx.doi.org/10.1080/19345747.2011.618213)" *Journal of Research on Educational Effectiveness*
### Rare Events
- King and Zheng. 2001. "[Explaining Rare Events in International Relations](https://doi.org/10.1162/00208180152507597)
- King, Gary, and Langche Zeng. 2001. "Logistic Regression in Rare Events Data"
### Identifiability
- Weschler et al. 2013. [A Bayesian Look at Nonidentifiability: A Simple Example](http://dx.doi.org/10.1080/00031305.2013.778787)
### Shrinkage
- Efron and Morris (1975) "[Data Analysis Using Stein's Estimator and its Generalizations](https://dx.doi.org/10.1080/01621459.1975.10479864)" *JASA* [@EfronMorris1975a]
## Software
Software for general purpose Bayesian computation are called [probabilistic programming languages](https://en.wikipedia.org/wiki/Probabilistic_programming_language).
- [Stan](http://mc-stan.org/)
- Joseph Rickert. 2016. [R Stan and Statistics](https://www.r-bloggers.com/r-stan-and-bayesian-statistics/)
- BUGS modeling language. Models are specified in a different language.
- [NIMBLE](https://r-nimble.org/) A very new BUGS-like language that works with R.
- [JAGS](http://mcmc-jags.sourceforge.net/) Gibbs/MCMC based
- [WinBUGS](https://www.mrc-bsu.cam.ac.uk/software/bugs/the-bugs-project-winbugs/) Gibbs and MCMC based software. It was
one of the first but is now obsolete and unmaintained. Use JAGS or Stan instead.
- [OpenBUGS](http://www.openbugs.net/w/FrontPage) The continuation of the WinBUGS project. Also no longer well maintained. Use JAGS or Stan instead.
- R has multiple packages that implement some Bayesian methods. See the [Bayesian Task View](https://cran.r-project.org/web/views/Bayesian.html)
- [LearnBayes](https://cran.r-project.org/web/packages/LearnBayes/index.html)
- [TeachBayes](https://cran.r-project.org/web/packages/TeachBayes/index.html)
- Python
- [PyMC](https://pymc-devs.github.io/pymc3/) Very complete general-purpose Python package for Bayesian Analysis
- The various Machine learning packages like [scikit-learn](http://scikit-learn.org/stable/).
- [Edward](https://github.com/blei-lab/edward). By David Blei. Deep generative models, variational inference. Runs
on TensorFlow. Implements variational and HMC methods, as well as optimization.
- [Church](http://projects.csail.mit.edu/church/wiki/Church) and [Anglican](http://www.robots.ox.ac.uk/~fwood/anglican/literature/index.html) are Lisp-based inference programs.
- Stata: Since [version 14](http://www.stata.com/new-in-stata/bayesian-analysis/) it can estimate some Bayesian models. It uses Metropolis-Hastings and Gibbs methods.
- Julia
- [Mamba](https://mambajl.readthedocs.io/en/latest/) MCMC supporting multiple methods including Gibbs, MH, HMC, slice
### Stan
Official Stan-dev R packages:
- [rstan](https://cran.r-project.org/web/packages/rstan/index.html)
- [rstanarm](https://cran.r-project.org/web/packages/rstanarm/index.html)
- [bayesplot](https://cran.r-project.org/web/packages/bayesplot/index.html)
- [ShinyStan](https://cran.r-project.org/web/packages/shinystan/index.html)
- [loo](https://github.com/stan-dev/loo)
Others:
- [brms](https://github.com/paul-buerkner/brms) Bayesian generalized non-linear multilevel models using Stan
- [ggmcmc](https://cran.r-project.org/web/packages/ggmcmc/index.html)
### Diagrams
#### DAGs and Plate Notation
See [Plate notation](https://en.wikipedia.org/wiki/Plate_notation)
- [tikz-bayesnet](https://github.com/jluttine/tikz-bayesnet) A TikZ library for drawing Bayesian networks
- [Daf](http://daft-pgm.org/) A python package to draw DAGs
- Relevant Stack Overflow questions:
- [Software for drawing Bayesian networks](http://stats.stackexchange.com/questions/16750/software-for-drawing-bayesian-networks-graphical-models) Stack Overflow.
- [TikZ Example](http://www.texample.net/tikz/examples/bayes/)
- [how to draw plate indices in graphical model in TikZ](http://tex.stackexchange.com/questions/199734/how-to-draw-plate-indices-in-graphical-model-by-tikz)
- [Can I have automatically adjusted plates in a graphical model?](http://tex.stackexchange.com/questions/11751/can-i-have-automatically-adjusted-plates-in-a-graphical-model?rq=1)
#### Kruschke Diagrams
Diagrams in the style of Kruschke's *Doing Bayesian Analysis*:
- [LibreOffice Draw Templates](http://www.sumsar.net/blog/2013/10/diy-kruschke-style-diagrams/)
- Blog posts
- <http://doingbayesiandataanalysis.blogspot.se/2012/05/graphical-model-diagrams-in-doing.html>
- <http://doingbayesiandataanalysis.blogspot.se/2012/05/hierarchical-diagrams-read-bottom-to.html>
- <http://doingbayesiandataanalysis.blogspot.se/2013/10/diagrams-for-hierarchical-models-we.html>
- [R scripts](https://github.com/rasmusab/distribution_diagrams)
- [TikZ scripts](https://github.com/yozw/bayesdiagram)
#### Venn Diagrams/Eikosograms
- Oldford and W.H. Cherry. 2006. "Picturing Probability: the poverty of Venn diagrams, the richness of Eikosograms"
### Priors
- Betancourt (2017) "[How the shape of a weakly informative prior affects inferences](http://mc-stan.org/documentation/case-studies/weakly_informative_shapes.html)" *Stan Case Studies*
- Stan, [Prior Choice Recommendations](https://github.com/stan-dev/stan/wiki/Prior-Choice-Recommendations)
## Bayesian Model Averaging
- Montgomery, Hollenbach and Ward (2012) "[Improving Predictions Using Ensemble Bayesian Model Averaging](https://dx.doi.org/10.1093/pan/mps002)" *Political Analysis*
- Montgomery and Nyhan (2011) [Bayesian Model Averaging: Theoretical Developments and Practical Applications](https://dx.doi.org/10.1093/pan/mpq001)
- [BMA Package](https://CRAN.R-project.org/package=BMA)
- [BMS Package](https://CRAN.R-project.org/package=BMS)
- [BAS Package](https://CRAN.R-project.org/package=BAS)
- Amini and Parmeter (2011) "[Bayesian Model Averaging in R](https://dx.doi.org/10.3233/JEM-2011-0350)" *Journal of Economic and Social Measurement*
- Fragoso and Neto (2015) [Bayesian model averaging: A systematic review and conceptual classification](http://arxiv.org/pdf/1509.08864v1:PDF) [@FragosoNeto2015a]
- Ley and Steel (2012) "[Mixtures of g-priors for Bayesian model averaging with economic applications](https://dx.doi.org/10.1016/j.jeconom.2012.06.009)" *Journal of Econometrics*
- Ley and Steel (2009) "[On the effect of prior assumptions in Bayesian model averaging with applications to growth regression](https://dx.doi.org/10.1002/jae.1057)" *Journal of Applied Econometrics*
- Volinsky, Raftery, Madigan, and Hoeting (1999) "[Bayesian model averaging: A Tutorial](https://dx.doi.org/10.1214/ss/1009212519)" *Statistical Science*
## Multilevel Modeling
- Stegmueller (2013), "[How Many Countries for Multilevel Modeling? A Comparison of Frequentist and Bayesian Approaches](https://dx.doi.org/10.1111n/ajps.12001)" *American Journal of Political Science* [@Stegmueller2013a]
- Shor, Bafumi, Keele, and Park (2007) "[A Bayesian multilevel modeling approach to time-series cross-sectional data](https://dx.doi.org/10.1093/pan/mpm006)" *Political Analysis*
- Beck and Katz (2007) "[Random coefficient models for time-series—cross-section data: Monte Carlo experiments](https://dx.doi.org/10.1093/pan/mpl001)" *Political Analysis* [@BeckKatz2007a]
- Western and Jackman (1994). "[Bayesian Inference for Comparative Research](https://dx.doi.org/10.2307/2944713)" *American Political Science Review* [@WesternJackman1994a]
- Anderson and Fetner. 2008. "[Economic inequality and intolerance: attitudes toward homosexuality in 35 democracies](https://dx.doi.org/10.1111/j.1540-5907.2008.00352.x)" *American Journal of Political Science*
## Mixture Models
- Imai, K. and Tingley, D. (2012) "[A Statistical Method for Empirical Testing of Competing Theories](https://dx.doi.org/10.1111/j.1540-5907.2011.00555.x)" *AJPS*
## Inference
### Discussion of Bayesian Inference
- Lindley. The Analysis of Experimental Data: The Appreciation of Tea and Wine
## Model Checking
### Posterior Predictive Checks
- Gelman, Andrew (2007) "[A Bayesian Formulation of Exploratory Data Analysis and Goodness-of-fit Testing](10.1111/j.1751-5823.2003.tb00203.x)" *International Statistical Review*
- Gelman, Meng, Stern (1996) "Posterior Predictive Fitness Via Realized Discrepencies"
- Kruschke. Posterior predictive checks can and should be Bayesian: Comment on Gelman and Shalizi, ‘Philosophy and the practice of Bayesian statistics
- [Confusions about posterior predictive checks](http://andrewgelman.com/2009/02/07/confusions_abou/)
- Gabry, Jonah. [Graphical posterior predictive checks using the bayesplot package](https://cran.r-project.org/web/packages/bayesplot/vignettes/graphical-ppcs.html)
### Prediction Criteria
- Gelman, Andrew, Jessica Hwang, and Aki Vehtari. 2014. “Understanding Predictive Information Criteria for Bayesian Models.” *Statistics and Computing*
- Vehtari, Gelman, and Gabry. 2016 [Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC](http://www.stat.columbia.edu/~gelman/research/unpublished/loo_stan.pdf)
- Vehtari and Lampinen (2002) [Bayesian model assessment and comparison using cross-validation predictive densities](https://doi.org/10.1162/08997660260293292)
- Vehtari and Ojanen (2012) "[A survey of Bayesian predictive methods for model assessment, selection and comparison](https://dx.doi.org/10.1214/12-SS102)"
### Software Validation
- Cook, Gelman, and Rubin (2006) "[Validation of Software for Bayesian Models Using Posterior Quantiles](https://dx.doi.org/10.1198/106186006X136976)" and [Correction](https://doi.org/10.1080/10618600.2017.1377082)
- Savage, Jim. [An easy way to simulate fake data from your Stan model](http://modernstatisticalworkflow.blogspot.com/2017/04/an-easy-way-to-simulate-fake-data-from.html)
- [Stan Best Practices](https://github.com/stan-dev/stan/wiki/Stan-Best-Practices)
## Hierarchical Modeling
- Kruschke and Vanpaeml "[Bayesian Estimation in Hierarchical Models](http://www.indiana.edu/~kruschke/articles/KruschkeVanpaemel2015.pdf)"
- Park, Gelman, and Bafumi (2004) "[Bayesian Multilevel Estimation with Poststratification: State-Level Estimates from National Polls](https://dx.doi.org/10.1093/pan/mph024)" *Political Analysis*
- Lax and Phillips. 2009. "How Should We Estimate Public Opinion in the States?" *AJPS*
## Shrinkage/Regularization
- Piironen and Vehtari. 2016. [On the Hyperprior Choice for the Global Shrinkage Parameter in the Horseshoe Prior](https://arxiv.org/abs/1610.05559)
- Lopes. 2015. [Bayesian Regularization](http://hedibert.org/wp-content/uploads/2015/12/BayesianRegularization.pdf) slides.
## Empirical Bayes
- Berger (2006) "[The case for objective Bayesian analysis](https://dx.doi.org/10.1214/06-BA115)" *Bayesian Analysis*
- Efron (2014) "[Frequentist accuracy of Bayesian estimates](https://dx.doi.org/10.1111%2Frssb.12080)" *JRSS B*
- Efron (2010) "[The Future of Indirect Evidence](https://dx.doi.org/10.1214/09-sts308)" *Statistical Science*
## History of Bayesian Statistics
- Robert and Casella (2011) "[A Short History of Markov Chain Monte Carlo: Subjective Recollections from Incomplete Data](http://dx.doi.org/10.1214/10-STS351)" *Statistical Science*
- Stigler (2018) "[Richard Price, the first Bayesian](https://doi.org/10.1214/17-STS635)" *Statistical Science* [@Stigler2018a]
- Stigler (1983) "[Who discovered Bayes's theorem?](http://www.jstor.org/stable/2682766)" *American Statistician* [@Stigler1983a]
- Fienberg (2006) "[When did Bayesian Inference Become "Bayesian"?](https://doi.org/10.1214/06-BA101)" *Bayesian Analysis* [@Fienberg2006a]
## Sampling Difficulties
- Carpenter (2017) "[Typical sets and the curse of dimensionality](http://mc-stan.org/users/documentation/case-studies/curse-dims.html)" *Stan Case Studies*
- Betancourt (2017) "[Diagnosing biased inference with divergences](http://mc-stan.org/users/documentation/case-studies/divergences_and_bias.html)" *Stan Case Studies*
- Betancourt (2016) "[Diagnosing suboptimal cotangent disintegrations in Hamiltonian Monte Carlo](http://arxiv.org/pdf/1604.00695v1:PDF)"
- Betancourt and Girolami (2013) "Hamiltonian Monte Carlo for Hierarchical Models"
## Complicated Estimation and Testing
- King, Tomz, and Wittenberg (2000) "[Making the most of statistical analyses: improving interpretation and presentation](https://dx.doi.org/10.2307/2669316)" Propose a pseudo-Bayesian method.
- Golder "[Interactions](http://mattgolder.com/interactions)". See referenced papers.
- Hanmer and Kalkan (2012) "[Behind the curve: clarifying the best approach to calculating predicted probabilities and marginal effects from limited dependent variable models](https://dx.doi.org/10.1111/j.1540-5907.2012.00602.x)" *American Journal of Political Science*
## Pooling Polls
- Jackman (2000) "[Pooling the Polls over an Election Campaign](https://dx.doi.org/10.1080/10361140500302472)" *Australian Journal of Political Science*
- Linzer (2013) "[Dynamic Bayesian forecasting of presidential elections in the States](http://dx.doi.org/10.1080/01621459.2012.737735)" *JASA*
## Visualizing MCMC Methods
- <https://chi-feng.github.io/mcmc-demo/>
- <https://mimno.infosci.cornell.edu/hmc/> and <http://www.mimno.org/articles/hmc/>
- <http://twiecki.github.io/blog/2014/01/02/visualizing-mcmc/>
- <https://ridlow.wordpress.com/category/animation/>
- <http://people.math.aau.dk/~kkb/Undervisning/Bayes14/sorenh/docs/sampling-notes.pdf>
- <https://rpubs.com/mv2521/mcmc-animation>
- <http://blog.revolutionanalytics.com/2013/09/an-animated-peek-into-the-workings-of-bayesian-statistics.html>
- <https://people.duke.edu/~ccc14/sta-663/Animation.html>
- <https://artax.karlin.mff.cuni.cz/r-help/library/asbio/html/anm.mc.bvn.html>
- <https://groups.google.com/forum/#!topic/stan-users/nOk80xTlSyE>
- <https://www.youtube.com/watch?v=Vv3f0QNWvWQ>
- <https://theclevermachine.wordpress.com/2012/11/18/mcmc-hamiltnonian-monte-carlo-a-k-a-hybrid-monte-carlo/>
- <https://www.youtube.com/watch?v=pHsuIaPbNbY&list=PLqdbxUnkqOw2nKn7VxYqIrKWcqRkQYOsF&index=11>
- <http://arogozhnikov.github.io/2016/12/19/markov_chain_monte_carlo.html>
## Bayesian point estimation / Decision
- Stan Modeling Language. Ch 32. Bayesian Point Estimation.
- [Modes, Medians and Means: A Unifying Perspective](http://www.johnmyleswhite.com/notebook/2013/03/22/modes-medians-and-means-an-unifying-perspective/). Not explicitly motivated with Bayesian decision theory; nevertheless, it is a good intuitive explanation of these estimators.
- [The Impact of Reparameterization on Point Estimates]( http://mc-stan.org/users/documentation/case-studies/mle-params.html)
- [Rainey](https://github.com/carlislerainey/transformation-induced-bias)
## Stan Modeling Language
- Ch 1--8 Introduction.
- pay attention to Ch 1, 8. skim the rest. know where to look for help.
- Ch 28. Optimizing Stan Code for Efficiency (Neal's funnel, reparameterization, vectorization)
- Ch 22. Reparameterization and change of variables
- Ch 23. Customized
- Ch 24. User-defined functions
- Ch 25. problematic posteriors
- Ch 29. Bayesian Data Analysis
- Ch 30. Markov Chain Monte Carlo Sampling (R hat, ESS, convergence, thinning)
- Ch 31. Penalized MLE
- Ch 32. Bayesian Point Estimation
- Ch 34. Hamiltonian Monte Carlo Sampling
- Ch 35. Transformations of Constrained Variables - changes of variables.
## Bayes Factors
- [Lindley's Paradox](https://en.wikipedia.org/wiki/Lindley%27s_paradox)
- [Bayes' Factors](https://en.wikipedia.org/wiki/Bayes_factor)
- Robert (2016) [The expected demise of the Bayes factor](https://arxiv.org/pdf/1506.08292.pdf) [@Robert2016a].
- Kass and Raftery (1995) "Bayes factors" [@KassRaftery1995a]