library(tidyverse)
library(broom)
library(modelsummary)
library(marginaleffects)
library(MASS, exclude = "select")
library(pscl)
library(estimatr)
library(juanr)Problem Set 4: MLE and Count Models
Due: Tuesday, May 5 by 11:59 PM. Push your completed .qmd and rendered .html to your GitHub repository.
Format: Write a self-contained Quarto document (.qmd) that renders to HTML. Your document should read as a short report — include brief prose explaining your reasoning, not just code and output.
Libraries you will likely need:
Question 1: The full logit workflow on Fearon & Laitin
Use the fearon dataset from {juanr} — a country-year panel from Fearon & Laitin (2003) on civil-war onset. The outcome onset is recorded as the number of new conflicts that began in a given country-year (mostly 0, sometimes 1, occasionally more). For this question we’ll treat onset as a binary event: did any new conflict start in this country-year?
The substantive question: does linguistic diversity predict civil war onset? numlang is the number of distinct languages spoken in the country.
Prepare the data. Construct a binary
any_onsetvariable that equals 1 ifonset > 0and 0 otherwise. Drop observations with missing values onnumlang,polity2, orpop. Report how many country-years are in your analysis sample, and the share withany_onset == 1(this is the base rate).Fit a logistic regression of
any_onsetonnumlang, controlling forpolity2andlog(pop). Report the model in a regression table.Plot predicted \(\Pr(\text{any\_onset} = 1)\) across the range of
numlang, holdingpolity2andlog(pop)at sensible values (e.g., their sample medians or means). Include a 95% confidence ribbon. Briefly describe the shape of the curve.Report the average marginal effect (AME) of
numlangon \(\Pr(\text{any\_onset} = 1)\). This is your headline number.Anchor the AME against the base rate from (a). The AME tells you the change in probability for a one-unit increase in
numlang; the base rate tells you what “typical” risk looks like. Translate the AME into a substantively meaningful statement — e.g., what does adding 10 more languages do to onset risk relative to the average country-year’s risk?Write one sentence suitable for the results section of a paper. Include a point estimate, a sense of magnitude, and an honest acknowledgement of the uncertainty around it.
Question 2: Searching for the MLE of numlang by hand
In Week 4 we wrote a log-likelihood function for the logit and searched for \(\hat\beta_1\) by plugging in candidate values, evaluating \(\ell(\beta)\), and seeing which one made the data most probable. We did this with a single covariate. Now you will do it with the full model from Question 1. You’ll only search over one parameter, \(\beta_{\texttt{numlang}}\). The other coefficients (\(\beta_0\), \(\beta_{\texttt{polity2}}\), \(\beta_{\texttt{logpop}}\)) will be held fixed at the MLE values that glm() returned in Q1.
Recall that for the logit,
\[p_i = \frac{1}{1 + \exp\!\left(-\left[\beta_0 + \beta_{\texttt{numlang}} \cdot \texttt{numlang}_i + \beta_{\texttt{polity2}} \cdot \texttt{polity2}_i + \beta_{\texttt{logpop}} \cdot \log(\texttt{pop}_i)\right]\right)}\]
and the log-likelihood is
\[\ell(\beta) = \sum_{i=1}^n \Bigl[\, Y_i \log p_i + (1 - Y_i)\log(1 - p_i)\,\Bigr].\]
Pull the MLE values of \(\beta_0\), \(\beta_{\texttt{polity2}}\), and \(\beta_{\texttt{logpop}}\) from the
glm()model you fit in Q1 (usecoef()). Store them as scalars (e.g.,b0_hat,b_polity_hat,b_logpop_hat). Also pull the MLE of \(\beta_{\texttt{numlang}}\) and store it separately asb_numlang_mle.Write a function
log_lik(b_numlang)that takes a candidate value for \(\beta_{\texttt{numlang}}\) and returns the log-likelihood of the data under that value, holding the other three coefficients fixed at their MLE. Inside the function, compute \(p_i\) for every observation, then return \(\sum_i [Y_i \log p_i + (1 - Y_i)\log(1 - p_i)]\). Test it: calllog_lik(b_numlang_mle)— this should give you the log-likelihood at the MLE (compare tologLik()of yourglmfit; they should match).Build a grid of candidate values for \(\beta_{\texttt{numlang}}\) that brackets the MLE on both sides (e.g., 100 values below
b_numlang_mleand 100 values aboveb_numlang_mle— adjust the window so the curve of the likelihood is clearly visible). For each value, evaluatelog_lik()and store the result.purrr::map_dbl()is the natural tool here.Plot the log-likelihood curve: candidate \(\beta_{\texttt{numlang}}\) on the x-axis, \(\ell(\beta)\) on the y-axis. Add a vertical line at
b_numlang_mle(the valueglm()found). Does the peak of your hand-computed curve sit on top ofglm()’s estimate? Briefly explain why the curve peaks where it does — what does it mean, in plain language, for the data to be “most probable” at that value of \(\beta_{\texttt{numlang}}\)?
This section is optional. It is intended for students building a methods portfolio for the quantitative methods subfield.
Now relax the “hold the others fixed” assumption. Modify your function so it takes a vector beta = c(b0, b_numlang, b_polity, b_logpop) of all four parameters and returns the log-likelihood. Pass this function to optim() (with control = list(fnscale = -1) to maximize rather than minimize, and a sensible starting vector like rep(0, 4)). Compare optim()’s estimated \(\hat\beta\) vector against coef() from your glm() fit. They should agree to several decimal places. In one or two sentences: what does this tell you about what glm() is doing under the hood?
Question 3: Computing the slope at four points
Looking at your log-likelihood curve from Q2, you can see where the peak is. But an algorithm searching for the MLE can’t see the curve — it can only evaluate \(\ell\) at specific values of \(\beta\). To know which way to step, it computes the slope (the gradient) by evaluating \(\ell\) at the current point and at one tiny step away. This is what we did in lecture (see the “Gradient” slide from Week 4 — the code there is your model).
You’re going to compute the slope at four candidate values of \(\beta_{\texttt{numlang}}\), chosen to highlight what the slope tells the algorithm:
- Two values to the left of the MLE: one close to
b_numlang_mle, one far to the left - Two values to the right of the MLE: one close, one far to the right
You pick the actual numbers — “close” and “far” are relative to the curve you plotted in Q2. Use the same log_lik() function you wrote there.
- For each of your four candidate values \(\beta^{(0)}\), compute the slope using the finite-difference recipe from the slide:
\[\text{slope} \;\approx\; \frac{\ell(\beta^{(0)} + \epsilon) - \ell(\beta^{(0)})}{\epsilon}\]
with a small \(\epsilon\) (e.g., epsilon = 0.0001). Report the four slopes in a small table (candidate \(\beta\), \(\ell(\beta^{(0)})\), \(\ell(\beta^{(0)} + \epsilon)\), slope).
Interpret the sign of each slope. For each of your four points: if an algorithm were standing there, which direction (left or right) would the slope tell it to step? Does that line up with where you know the MLE actually is?
Interpret the magnitude of each slope. Compare the two left-side slopes (close vs. far) and the two right-side slopes (close vs. far). Where is the slope largest in absolute value, and where is it smallest? In one or two sentences, explain why an MLE search algorithm would want to take big steps where the slope is big and small steps where the slope is small — and what would go wrong if it ignored that and took the same-size step every time.
The slope tells the algorithm which direction to step — but not exactly how far. For that we need to know how fast the slope itself is changing, i.e., the Hessian (the curvature of the log-likelihood). In lecture we computed it by applying the same nudge trick “one level up”: three evaluations of \(\ell\) give two slopes, and two slopes give one Hessian (see the Hessian slide from Week 4).
For each of your four candidate \(\beta^{(0)}\) values, evaluate \(\ell\) at \(\beta^{(0)}\), \(\beta^{(0)} + \epsilon\), and \(\beta^{(0)} + 2\epsilon\). Compute two slopes from those three points, then compute the Hessian as
\[\text{Hessian} \;\approx\; \frac{\text{slope}_2 - \text{slope}_1}{\epsilon}.\]
Report the four Hessians alongside the slopes from (a). Then answer: how does \(|\text{Hessian}|\) compare across your four points?
The pattern here is not as clean as it was for the slope in part (c), and that’s the point of the question. A few things to notice:
- Sign. The Hessian should be negative at every point you evaluate (you’re sitting near the top of a hill — the curve is concave down). If you got a positive Hessian, recheck your finite-difference arithmetic.
- Magnitude. Unlike the slope, \(|\text{Hessian}|\) does not grow steadily as you move away from the MLE. For most log-likelihoods it is largest near the peak (where the curve is most sharply bowed) and smaller in the tails (where the curve flattens out). Don’t be surprised if your four values look noisy or close to each other — the curvature changes much more gently than the slope does.
- What the algorithm uses it for. The slope tells the optimizer which direction to go; the Hessian tells it how far. A big \(|\text{Hessian}|\) (sharp curvature) means “the slope is changing fast — take a small step.” A small \(|\text{Hessian}|\) (flat curve) means “the slope is barely changing — you can afford a big step.” So the right way to describe your numbers is in terms of what step size the algorithm would take at each point, not in terms of a clean monotone pattern.
Two or three sentences is plenty.
Question 4: How sample size sharpens the likelihood
In Q2 you plotted the log-likelihood curve for \(\beta_{\texttt{numlang}}\) using the full Fearon sample. The peak sat right where glm() said it would. But the shape of that peak — how sharp or how flat — isn’t just a property of the model. It depends on how much data you have. More data means a sharper peak, which means a more precisely estimated \(\hat\beta\) and a smaller standard error. This is the intuition behind Fisher information (see the Standard errors slide from Week 4).
You’re going to demonstrate this empirically by re-running your Q2 grid search on subsamples of the Fearon data and overlaying the log-likelihood curves.
Pick three sample sizes spanning roughly two orders of magnitude — e.g., \(n = 100\), \(n = 500\), and the full sample (whatever your Q1 cleaned-data \(n\) is). Use
slice_sample()to draw a random subsample at each smaller \(n\) from your cleaned data (set a seed for reproducibility).For each subsample, refit the
glm()from Q1 to get that subsample’s MLE for all four coefficients (the betas will be slightly different in each subsample — that’s fine and expected). Then build alog_lik()function for that subsample (same recipe as Q2: it should close over the subsample’sY,numlang,polity2,log(pop), and the three “other” MLEs). Evaluate it on the same kind of grid you used in Q2, bracketing that subsample’sb_numlang_mleon both sides.Plot all three log-likelihood curves on the same axes, colored by sample size. Two important plotting moves to make the comparison visible:
- Recenter: subtract each curve’s own maximum so all three peaks sit at \(\ell - \max(\ell) = 0\). Without this, the curves will live on wildly different vertical scales and you won’t be able to compare their shapes.
- Use a common x-axis window wide enough to show curvature in the smallest-\(n\) curve (the broad one) and narrow enough that the shape of the largest-\(n\) curve (the sharp one) is still visible.
Describe what happens to the curve as \(n\) grows.
Question 5: Counting affairs — OLS, Poisson, and Negative Binomial
Switch datasets. Use the affairs data from {juanr} — a 1978 survey of 601 respondents (Fair 1978). The outcome n_affairs is the self-reported number of extramarital affairs in the past year. Most respondents report zero; a small minority report many. This is a classic count outcome.
We’ll model n_affairs as a function of:
marriage_rating(1 = very unhappy, 5 = very happy — the focal predictor)occup(occupation code, 1–7)relig(religiosity, 1–5)years_married(years married)
Fit three models with the same right-hand side and the same data:
- OLS with robust SEs:
estimatr::lm_robust(n_affairs ~ marriage_rating + occup + relig + years_married, data = affairs, se_type = "HC2") - Poisson:
glm(..., family = poisson, data = affairs) - Negative binomial:
MASS::glm.nb(..., data = affairs)
Display all three side-by-side in a single regression table using
modelsummary(). Briefly note where the three columns agree (sign) and where they disagree (standard errors — compare Poisson SEs to NB SEs in particular, and explain why they differ).- OLS with robust SEs:
Plot predicted
n_affairsasmarriage_ratingincreases, with one curve per model (colored by model). Holdoccup,relig, andyears_marriedat sensible reference values (e.g., their sample medians). The cleanest way to do this ismarginaleffects::predictions(), run separately for each model and then bound together with amodelcolumn for the color aesthetic. Include 95% confidence ribbons or error bars.In two or three sentences: where do the three predicted-value curves agree, and where do they differ?
We didn’t get to the negative binomial model in class — I’ll cover it next week. Give it a try anyway: MASS::glm.nb() works just like glm(), and you can run all the same marginaleffects::predictions() machinery on it. If you’d rather wait, drop the NB column from your table and the NB curve from your plot, and answer 5(a)–(c) with just OLS and Poisson. No penalty either way.