Today, we will estimate the unknown parameters of a mixture of Exponential distributions using Expectation-Maximization. I will use the notes provided by the TA for the previous semester. mixtureem.pdf
1. To begin with, use the following code snippet to simulate from a mixture of Exponentials:
N = 100 #Number of observations p = .5 #Mixing parameter lambdas = c(1.2, 4.8) #Exponential rate parameter for each distribution Z = sample(c(1,2), N, replace = TRUE,prob=c(1-p,p)) Z X = sapply(Z,function(c) rexp(n = 1,rate = lambdas[c])) #goes through each entry in Z and applies the function to it. hist(X) #it is pretty hard to see a mixture of exponentials.
2. Write an Expectation-Maximization function to fit the data X simulated above (treat the parameters as unknown) by following Section 5 in the notes. It should return the history of the parameters at each iteration and the history of the observed log-likelihood at each iteration.
3.It is important to verify that the observed log-likelihood is increasing at each iteration. It is possible that you implement EM, which would seem right at first but the observed log-likelihood would occasionally decrease. The best way to check it, would be to draw a plot for the observed likelihood.
Here are the solutions:lab14.r