Bernouilli / Binomial distribution likelihood with uniform prior
As we can see in this post, the output or posterior will be beta distribution if we use uniform prior to bernouilli / binomial distribution. The generalization of bernouilli / binomial distribution with uniform prior is like this. f(˜y|θ)=θΣyi(1−θ)n−Σyif(θ)=I0≤θ≤1
Through the previous equation, we can see posterior for theta given y follows a beta distribution. We can use this relation to make general sentense. Actually, the uniform distribution is a kind of beta distribution of parameter 1,1 and any beta distribution is conjugate for the bernouilli distribution. f(θ)=Γ(α+β)Γ(α)Γ(β)θα−1(1−θ)β−1I0≤θ≤1
Now, we can say conjugae family. It is a family of distribution is referred to as conjugate if when we use member of that family as a prior, we get another member of that familiy as our posterior. In the previous case, the beta distribution is conjugate for the bernouilli distribution. Because this relationship makes calculation much simple, it is better to use conjugate distribution as a prior. One thing we need to know is that when we use beta distribution or other distribution as a prior, there are another parameter like alpha and beta. We call this parameter as hyper parameter.
Posterior mean and effective sample size
In the case of bernouilli likelihood with N observation using beta prior, now we can write the posterior predictive distribution is like this. Also, we can set effective sample size using hyper parameter. prior∼beta(α,β)
If we observe event in different time, it is not problem in bayesian paradigm. The only thing need to do is updating posterior sequentially. Because we update it sequentially, it has identical result whether we perform at one time or seperately. In the case of frequentist paradigm, it is different case if we perform in different days.