Today, we will estimate the unknown parameters of a mixture of Exponential distributions using Expectation-Maximization. I will use the notes provided by the TA for the previous semester. mixtureem.pdf

1. To begin with, use the following code snippet to simulate from a mixture of Exponentials:

 N  = 100  		    #Number of observations
 p = .5                      #Mixing parameter
 lambdas = c(1.2, 4.8)       #Exponential rate parameter for each distribution
 Z = sample(c(1,2), N, replace = TRUE,prob=c(1-p,p))
 Z
 X = sapply(Z,function(c) rexp(n = 1,rate = lambdas[c]))   #goes through each entry in Z and     applies the function to it.
 hist(X) #it is pretty hard to see a mixture of exponentials.

2. Write an Expectation-Maximization function to fit the data X simulated above (treat the parameters as unknown) by following Section 5 in the notes. It should return the history of the parameters at each iteration and the history of the observed log-likelihood at each iteration.

3.It is important to verify that the observed log-likelihood is increasing at each iteration. It is possible that you implement EM, which would seem right at first but the observed log-likelihood would occasionally decrease. The best way to check it, would be to draw a plot for the observed likelihood.

Here are the solutions:lab14.r