How Structural Equation Modeling is Ruining Family Research

How I feel when I see a structural equation model of cross-sectional data with 15 latent variables predicting 3 different outcomes. photo credit: BrittneyBush via photopin cc

Please find this post at: https://clairekampdush.com/2014/12/05/semfamilyresearch/

Look out readers! This is my first of a series of posts I am working on related to causal analysis.

About two weeks ago, I attended the National Council on Family Relations (NCFR) annual meeting. It is a long meeting – I usually get there on Tuesday, and don’t leave until Saturday. And, while at the conference, I go from 7 am until 9 pm. In fact, I didn’t even leave the hotel two days! This might explain why I went off the rails during the Q&A at the last session I went to, but what set me off was a theme I saw throughout the conference.

Here is the crux of the problem: early career scholars are so focused on fancy statistics (i.e. structural equation modeling, latent class analysis) that they 1) forget about theory and the justification of their research question, and 2) present papers so complicated that no average person can understand, and even the non-average person who has a PhD and tenure cannot understand it.  But, I do not want to lay all the blame on early career scholars – we senior scholars are the ones creating these monsters!!

In most graduate programs, students are required to take several methods courses. I was required to take at least 6 when I was in grad school, and our students are required to take at least 6 as well. Unfortunately though what is happening is that we are overemphasizing the importance of cutting-edge methods and statistics, and underemphasizing the importance of constructing coherent research questions that have strong theoretical justifications.

Why are these complicated statistics ruining family science? They are ruining family science because they are making conferences boring and incoherent, and leading to the rejection of papers from these family science scholars, and a lack of publications can make it hard for these students to get a job. I talked to several colleagues and students about this issue, and each could give me examples of presentations they went to where they could not even figure out what the research question was, or why they should care.

I feel bad because early career scholars work incredibly hard and create these amazing complex models, including latent classes of trajectories from several waves of data, or structural equation models of cross-sectional data, with lots of arrows and boxes, but these models are so complicated that I cannot even understand them, or at least, I cannot understand them in 12 minutes. Or, the language interpreting a cross-sectional structural equation model is causal and using words like “effect” with arrows pointing directionally from an IV to a DV measured at the same time point, with no discussion of potential reciprocal associations (more on why this is an issue next week).

The smart early career scholars turn these models into papers, but too often these papers include just a few pages focused on the theory and what the actual research question is, and instead include several pages devoted to the methods and results, and a short discussion. I often see these papers rejected, because reviewers don’t understand why the question is important, or sometimes, even what the question is. So, again I come back to – we are focusing too much on fancy statistics and methods in our grad programs, and not emphasizing understandability, theoretical importance, and readability enough.

So, what to do? If you are an early career scholar who has an upcoming presentation, try presenting your work to a friend – preferably a friend who has no idea what your method is, or even better, does not understand your research topic. After your presentation, have your friend tell you what they thought your research question was and what your results were, and what the take home message was. Is your friend right? If not – revise until a friend can answer these questions correctly.

If you are an early career scholar getting ready to submit one of these papers, congratulations! But, before you submit, make sure you have a friend read it, and ask your friend to tell you 1) what your research question is, 2) why it is important, 3) how your work advances your research area, and 4) what the results were and why they were important. Again, if your friend is off-base, or if your friend cannot even answer these questions, revise your paper and focus on the theory and research questions just as much as the methods and results.