Multilevel Analysis/Hierarchical Linear Modeling
Last revised: December 6, 2016 General Information
Examples of Papers that Use Multilevel Models
Handy program and links
Questions or problems regarding this site should be sent to
There are 2 things that you need before you can do this:
General Information (MSword format):
Nov 3: Final exam is posted along with data.
Oct 18: Class on Thursday will be Open office hours for extra help with R, SAS, etc.
Oct 17: Lab 3 will be Tuesday October 18: R at 9:00-10:30 and SAS at 10:30-12:00.
- Dec 6: new version of hlmRsq function for R.
- Nov 29: Posted last answer keys, SAS and R code.
- Nov 28: Below are links to stat presentation on bayesian estimation of multilevel models presented by Chuck Huber
- Nov 28: Posted up-dated notes and examples for logistic regression.
- Nov 14: New link to revised wiki with in instructions on how to logon to webstore server. Unless you are on a computer hardwired to the campus network, you must you VPN.
- Nov 10: I have posted the following related to this morning's lab and corresponding homework:
- SAS code created during lab this morning.
- Corrected version of SAS lab instructions (i.e., the model to use when computing R2's and meta R2).
- Corrected version of R homework (i.e., I struck out a variable from the model you should use when
computing R2 and R2meta. Now the models fit in lab correspond to what you need to look at in homework).
- Nov 8: Answer keys and computer code for homework 5 posted below.
- Nov 7: New/revised postings related to R:
- Lecture notes on inference of marginal model now has R estimated empirical standard errors.
- R code that goes with inference of the marginal model now has R for estimated empirical standard errors
using most rececent version of the robust function.
- Revised and more stable robust function replaces old one. The new one does not have error regarding
cut1 and/or cut2, and I added 3 new lines of code so that the function should find the input model, the
response vector and the id. I tested this version of the function using simmulated data and hsb data (results
and cases are in comment before the function. I did not try to use this in R studio.
- Revised version of the contrast function replaces old one. Based on errors made in lab, the revised function
checks for common input errors, and if found, prints an error message stating the problem detected.
- R and SAS code and answer keys posted
- R code for lecture notes on model building is posted along with data. There are some graphs not covered in class that are
described in the lme4 documentation
Sept 29: R lab instructions re-posted with typos fixed.
Sept 22: Next lab will the Thurday Sept 29th. R at 9:00-10:30 and SAS at 10:30-12:00.
Sept 20: Wes's office hours for rest of semester: Wed 1:30-3:30, i224B Col. Wolf School, 403 East Healey (at corner of 4th & Healey) or if you would like a map http://ada.fs.illinois.edu/0167.html
Sept 18: Homework #3 is due on THURSDAY Sept 22.
Sept 12: up-dated notes on inference for marginal model.
Sept 8: R script and SAS code posted below.
Lab 1 materials for both SAS and R are posted below. IF you are using SAS, make sure that your can access it. If you are using R, make sure that it is installed on your computer (you may want to get the package lmerTest beforehand).
- Lab instructions and data are posted
- Homework 5 is posted
Aug 30: Homework is due at the beginning of lecture. If you are sick then give it to a friend or e-mail it to both
Wes and Carolyn.
Aug 30: If you come to the lab session at 9am for an introduction to SAS, bring your laptop and put data file on
your remote desktop account (if using).
- Homework #2 is due in class on Tuesday, September 6. If you want to see me in office hours before
turning in the homework, may turn it in by 5pm to my mailbox on the 3rd floor of Education. Harcopy
is much prefered (e-mail gets lost in my mailbox
- The first computer lab on random intercept models will be Thursday September 8th. Those using R should
attend the 9:00-10:30 session and those using SAS should attend the 10:30-12 session. Be sure to
bring you (charged) laptop and it is useful to down load materials ahead of time. These will be
posted in the near future.
- The wiki that was online has been fixed thanks to our trusty IT staff.
- To move files between the server and your laptop:
- For PC users, just "copy" and "paste" file to/from laptop and server.
- For MAC users, instructions are here and
a screen shot is here.
- Introduction to SAS will be Tuesday Sept 1, 9-10am in room 22 Education. Bring you laptop.
- The lab sessions when we have them will be 9-10:30 for R and 10:30-12 for SAS and mostly likely
be on Thursdays.
Aug 25: See below for the first homework assignement, which is due Tuesday August 30.
Aug 24: Those who came to class but weren't registered and signed the attendence sheet can all register.
You have to send me your UIN and netid so that I can put in a request for an override.
Draft chapters on GLM, GLMM, and LLM (i.e., HLM).
Resources for R users:
Computing: We will be running statistical software (SAS) using remote desktop to a university server. Here are the instructions:
remote desktop connection. Here is the file referred to in the instructions as an RDP file. ---
Use "Save link as" when you download the RDP file and save it to a convient location (e.g., desktop).
When you run the RDP application and it will run remote desktop application and set up configurations such that you will be able to directly open and save files from the remote server to your computer (and visa versa).
If you don't want to do these things, just use your remote desktop program and enter "remote2.webstore.illinois.edu".
- Your computer must be hooked up to the internet. If you access the internet via campus wireless, home, hotel, Espresso Royal, or elsewhere), you will eed
to use VPN (Virtual Private Networking) to securely connect to the campus internet.
- You must be registered for the class. If you reccently registered, you may not yet have permission to login yet.
Lectures Notes: (I am up-dating throughout the semester)
- Introduction ( up-dated Fall 2016).
SAS that generated (most of) the statistics and graphs in the lecture notes:
- Models for clustered data: Fixed and random
effects ANOVA and multiple regression. up-dated Fall 2016
SAS and R code that reproduces the statistics, graphs, etc. in the lecture notes:
- Optional: Introduction to SAS to be held....Tuesday Aug 30, 9am in room 22 Education.
- Random Intercept Models. up-dated Fall 2016
Fits ANCOVA model to NELS88, N=10 data (includes centering a variable, model
fitting using GLM, and SAS/GRAPH of model).
Creates SAS data set of level 1 data for the High School and Beyond data.
Creates SAS data set of level 2 data for the High School and Beyond data.
Merges level 1 and level 2 high school and beyond sas datasets.
SAS/GRAPHS for looking between and within variability of SES in the high
school and beyond data.
SAS PROC MIXED and fitting random intercept models (includes centering SES)...and some graphics.
- R: (New 2016)
- For the NELS example, use R from note on previous lecture on models for clustered data.
- HSB1data.txt. Student level data
- HSB2data.txt. School level data
- R_hsb_rand_intercept.txt. R code that
reproduces everything in lecture notes. You will need to change the "setwd" to where you put the data
and install package lmer, lattice and lmerTest.
- SAS for HLM and a little R. Up-dated Fall 2016
These will be used in lecture (they can also be used to reproduce the models for HSB data in
the lecture notes on random intercept models):
- This is from the introduction to SAS session:
- R: See R code under Notes on Random Intercept Models
- Random Intercept and Slopes Models . (up-dated Fall 2016)
- Estimation of Marginal Model. (up-dated fall 2016)
Statistical Inference: Marginal Model. (up-dated Fall 2017)
- R: (new fall 2016)
- HSB1.txt Can use to try out functions.
- HSB2.txt Can use to try out functions.
- hsb.sas This SAS code can used to compare results of R and SAS.
- robust. This R code is a function that computes robust ("sandwiche") standard errors for fixed effects. Be sure to read comments at the top regarding use.
- hlmRsq. This R code is a function that takes results from lmer and computes R1sq and R2sq as described in Snijders, TAB, Bosker, RJ (1994). Modeled variance in two-level models." Sociological Methods & Research, 22, 342-363. This works for both random intercept and random intercept and slope models. Be sure to read comments at the top before use.
- contrast.txt This R code is a function does contrasts. It takes as input model and a vector/matrix of contrasts and computes both F and Wald test statistics.
- Illustrates some of things done in lecture (i.e., computing roubst se, computing R squares, and finding mixture p-values).
- Power and sample size:
- Xiaofeng Steven Liu (2014). Statistical Power Analysis for the Social and Behavioral Sciences: Basic and Advanced Techinques.
Routledge: NY. This book contains explanation of procedures and code for R, SAS and SPSS. If you search google scholar
- Optimal Design Software Program and documenation from Raudenbush group. (PC only)
- PINT Program and documentation from Snijders. (PC only)
- Google "Xiaofeng Liu power hlm", you will find some of his papers on power and HLM (I think he was a student of Raudenbush).
- New SAS programs:
- R: (new fall 2016)
Longitudinal Data Analysis
Lecture notes on Serial Correlation.. (not up-dated---may not cover)
HLM for Riesby data. (data from Hedeker web-site plus additional
analyses I did).
Error Structures Simulation:
Multilevel Logistic Regression. (up-dated 2016)
Computer Lab Sessions: Bring laptop
Computer Lab Session 4: Thursday November 10, 2016.
- Computer Lab Session 0 (optional): Introduction to SAS ---> Tuesday 9am, Aug 30, 3016
- Computer Lab Session 1: (bring laptop) ---> Thursday September 8, 2016
- Computer Lab Session 2: (bring laptop)
- Computer Lab Session 3: bring laptop. date: Tuesday October 17, 2016
- Computer Lab instructions.
- We will use data and programs from previous labs 1 and 2. For youor convience, here is modified version for lab3.
- Data for computer lab 3
- R script for things demonstrated during lab.
This R code is a function that computes robust ("sandwiche") standard errors for fixed effects. Be sure to read comments at the top regarding use.
- hlmRsq. T
his R code is a function that takes results from lmer and computes R1sq and R2sq as described in Snijders, TAB, Bosker, RJ (1994). Modeled variance in two-level models."
Sociological Methods & Research, 22, 342-363. This works for both random intercept
and random intercept and slope models. Be sure to read comments at the top before use.
This R code is a function does contrasts. It takes as input model and a vector/matrix of
contrasts and computes both F and Wald test statistics.
- R code to get answers.
- Homwork 1:
- Homwork 2:
- Homwork 3:
- Homwork 4: Due Tuesday October, 18, 2016
- Homework 5: Thursday November 3, 2016
- Homework 6: Thursday Nov 17, 2016
- Final Exam and Projects: Hardcopy is due 3:45pm Friday December 9th (offices are sometimes locked around 4:00pm).
Pentalty for late finals or projects is 10 points (out of 100 points) per weekday.
(e.g., turn in Monday, 10 point deduction; turn in Tuesday 20 point, etc).
Examples of Papers that Use Multilevel Models
- Payne, B.R., Gao, X., Noh, S.R., Anderson, C.J., Stine-Morrow, E.A.L. (2011). The effects of print
exposure on sentence processing and memory in older adulats: Evidence for efficiency and reserve. Aging, Neuropsychology, and Cognition.
Some examples of crossed random effects, skewed responses (i.e., reaction times), and discrete response (i.e., Poisson).
- Segerstrom, S.C. & Sephton, S.E. (2010). Optimistic expectanices and cell-mediated immunity: The role
of positive affect. Psychological Science, 21, 448-455.
Example of where cluster centered level one variable is substantive (theoretical) interest. The response variable is numerical/continuous.
- Allen, N.E., Todd, N.R., Anderson, C.J., Davis, S.M., Javdani, Bruehler, V., & Dorsey, H.
(2013). Council-Based approaches to intimate partner violence: Evidence for distal change in system response. American Journal of Community
Psychology, 52, 1-12.
Example of a longitudinal study with creative centering of time. The response variable was a rate (probability).
- Poteat, V.P. & Anderson, C.J. (2012). Developmental changes in sexual prejudice from early to late
adolescence: The effects of gener, race, and ideology on different patterns of change. Developmental Psychology, 48, 1403-1415.
Example of an accelerated longitudinal design.
- Examples from Tom Snijders course web-page where multilevel models have been used. (click on "info course multilevel" on left and go to
bottom of page. These papers cover a range of topics (e.g., political science, sociology, school psychology, criminology, medicine, and others).
Example SAS Programs (ascii/text format):
Examples from Snijders & Bosker using SAS
Examples from Chapter 4 of Kreft & de Leeuw (provided and written
by Carol Nickerson):
Create SAS data for examples in Chapters 4 and 5.
Example 2-level analyses from Chapter 4 (random intercept models).
Example 2-level analyses from Chapter 5 (random intercept and
Examples analyses from Chapter 12 (longitudinal data analysis),
including creating sas dataset.
Handy Programs and Links:
SAS code that creates data set and fits models reported in Kreft &
Raw data file that is used as input to school23.sas.
Ones specific to multilevel modeling:
A FORTRAN program that computes large sample confidence intervals for a
A FORTRAN program that computes p-values and (bonferroni) critical values
for the standard normal, chi-squared, t, and F distributions (and for correlations).
For users of PC type computers,
is an executable (i.e. already compiled) program.