Bayesian analysis
This page is a encyclopedia article.
The page identifier is Op_en3792 

Moderator:Jouni (see all) 

Upload data

The text below is based on the review document put together in the first phase of Intarese project. The original document can be found here: Bayes review
Additional information on Bayesian statistics (note: external link to Wikipedia).
KTL (M. Hujo)
Contents
Basic ideas
Most researchers first meet with concepts of statistics through the frequentist paradigm. Bayesian statistics offers us an alternative to frequentist methods.
Bayesian thinking and modeling is based on probability distributions. Very basic concepts in Bayesian analysis are prior and posterior distributions. A prior distribution p(θ) summarizes our existing knowledge on θ (before data are seen). It can, for example, describe the opinion of a specialist and therefore Bayesian modelling requires that we accept the concept of subjective probability. A posterior distribution p(θdata) describes our updated knowledge after we have seen data. A posterior is formed by combining a prior and likelihood p(dataθ) (derived using the same techniques as in frequentist statistics) using Bayes' formula,
As we can see, this is a natural mechanism for learning; it gives a direct answer to the question: "How do data change our belief in the matter we are studying?"
From the above we can also see that one of the main differences between frequentist and Bayesian analyses lies in whether we use only likelihoods or whether we also use prior distributions. The prior distribution allows us to make use of information from earlier studies. We summarize this information with our prior distribution and then use Bayes' formula and our own data to update our knowledge. If we do not have any previous information on the issue we are studying we may use the socalled uninformative prior meaning that our prior distribution does not contain much information, for example a normal distribution with a large variance. As we see from Bayes' formula, the use of the uninformative prior lets data define our posterior distribution.
Of course there are also differences between Bayesian and frequentistic statistics. One of these differences is the way of thinking. In frequentistic analysis the parameter θ is taken to be fixed (albeit unknown) and data is considered to be random, whereas Bayesian statisticians would say that θ is uncertain and follows a probability distribution while data is taken to be fixed.
It is important to note that Bayesian analysis carefully distinguishes between p(θ  data) and p(data  θ) and all inference from Bayesian analysis is based on a posterior distribution, which is a true probability distribution. Thus Baysian analysis ensures natural intepretations for our estimators and probability intervals. More on the basics of Bayesian analysis can be found for example in [1] and [2].
Markov chain Monte Carlo
In practice it is not straightforward to compute an arbitrary posterior distribution, but we can sample from it. For sampling we may use the Markov chain Monte Carlo (MCMC) concept. In simple terms, the idea is to construct a Markov chain such that it has the desired posterior distribution as its limiting distribution. Then we simulate this chain and get a sample from the desired distribution. Perhaps the most commonly used software writen for MCMC is called BUGS which is an abbreviation of 'Bayesian statistics Using Gibbs Sampling'.
Bayesian metaanalysis
A metaanalysis is a statistical procedure in which the results of several independent studies are integrated. There are always some basic issues to be considered in meta analysis such as the choice between using a fixedeffects model or a randomeffects model, the treament of small studies and incorporation of studyspecific covariates. For example, analysis of the results of a clinical trial with many health care centers involved. The centers may differ in their patient pool e.g. number of patients, overall health level and age or the quality of the health care they provide. It is widely recognized that it is important to explicitly deal with the heterogeneity of the studies through randomeffects models, in which for each center there is a centerspecific "true effect" included. Bayesian methods allow us to deal with these problems within a unified framework (cf. [5]). A major advantage in the Bayesian approach is the ease with which one can include studyspecific covariates and that inference concerning the studyspecific effects is done a natural manner through the posterior distributions. Compared to classical methods, the Bayesian approach potentially gives a more complete representation of betweenstudy heterogeneity and a more transparent and intuitive reporting of results. Of course the cost of these benefits is that a prior specification is required.
To shed more light on Bayesian metaanalysis, let us now give a relatively simple example. Assume that we have n studies. We are interested in average µ of the parameters θ_{j} , j = 1,..,n. Using information available from these studies we calculate a point estimator y_{j} for parameter θ_{j} . The first stage of the hierarchical Bayesian model assumes that the point estimators conditioned on parameters are e.g. normally distributed i.e.
We can simplify this model by assuming that σ_{j} is known. This simplification does not have much effect if sample sizes for each study are large enough. As an estimator of σ_{j} we can take for example the sampling variance of a point estimator y_{j}.
The second stage of our model assumes normality for θ_{j} conditioned on hyperparameters μ ja τ ,
Finally, we assume noninformative hyperpriors for μ and τ . The analysis of our metaanalysis model follows a now normal Bayesian procedure, the inference is again based on posterior distribution p(μ  data).
More reading on the applications of Bayesian metaanalysis can be found in [3] and [4].
Combining information from different types of studies
Let us now consider an example on air pollution (fine particles) and its health effects measured by health test. We are interested in the relationship of personal exposure to fine particulate matter and the associated health effect. For health effect we have binary data y, y ~ Ber(p), given by a health test (st depression) where one indicates a health problem and zero stands for no problem. We also have data on the ambient concentration of the fine particle matter, denoted by variable z_{1}, for each day of the health test. What we do not have are data on personal exposure for health measurement days, which we denote by x_{1}. So there is a missing piece of information between personal exposure and the health test. However, we do have another data set that connects ambient concentration to personal exposure. In this second data set we denote ambient concentration by z_{2} and personal exposure by x_{2}. The solution to our problem now is a model consisting of two parts. First part is logistic health effect model which assumes that
where c stands for all confounding variables required in model. The second part is a
linear regression model
where we obtain estimates to personal exposure on health test days. Parameters a
and b are estimated from
using our second data set. One of the advantages of using Bayesian statistics here to analyze
the relationship between personal exposure and a health effect is that the model takes into account
our uncertainty of x_{1} in (1.1).
To summarize the above we have
To complete our model we set some prior distributions for parameters ά, β_{1}, β_{2}
and for parameters a and b. The posterior distribution of this model can be simulated
using the MCMC methods described above. This kind of idea is applied for example in [3].
Final remarks
Bayesian methods offer a rather flexible tool to combine studies and there is some software available to use with Bayesian analysis. Intepretations for Bayesian estimators and probability intervals are very natural because Bayesian analysis is based on true probability distributions. For example a Bayesian 95% probability interval should be interpreted such that a parameter of interest µ lies in that interval with a probability of 0.95. A frequentistic confidence interval is a random interval meaning that if we would repeat our calculations 100 times with new data each time then, on average, 95 of our confidence intervals would contain θ.
References
[1] P. Congdon. Applied Bayesian Modelling. Wiley series in probability and statistics 2003.
[2] A. Gelman, J. B. Carlin, H. S. Stern, D.B. Rubin. Bayesian data analysis. Chap
man & Hall 2004.
[3] F. Dominici, J. M. Samet, S. L. Zeger A measurement error model for timeseries
studies of air pollution and mortality. Biostatistics (2000), 1, pp. 157175
[4] J. M. Samet, F. Dominici, F. C. Curriero, I. Coursac, S. L. Zeger. Fine partuculate
air pollution and mortality in 20 U.S. cities. The New Enland Journal of Medicine
(2000), 342, 24, pp. 17421749
[5] T. Smith, D. Spiegelhalter, A. Thomas Bayesian approaches to randomeffects
metaanalysis: a comparative study. Statistics in Medicine (1995) 14(24) pp. 268599.