What do sociologists mean when they say "cause?"

6/21/2013 04:22:00 PM
In economics, it is extraordinarily hard to get good enough data to estimate causal effects. But the concept of causality in economics is usually pretty straight forward: we want to know if we change policy X by a certain amount, how much will outcome Y change. The response of Y to our deliberate change in X is the "causal" effect. As I said, this is very difficult to estimate, but very easy to understand.

But what's "causal" in sociology? Sociologists don't live in the same policy-dominated world that economists do. They live in a much subtler world of cultural habits, personalities, and lifestyles that aren't part of the political machinery, and can't be changed by legislative fiat. So if you can't pass a piece of legislation or promulgate a policy that changes X, what does it mean when they say X "causes" Y?

I have a particular sociological study in mind. Actually, I should say "study," because at this point it has been thoroughly discredited, and is currently under investigation for ethical conflicts. I'm referring to the infamous paper by Mark Regnerus on children of parents who have had same-sex relationships. Now, if that sounds like a strangely wordy way to describe what the paper is about, believe me, you can't simplify the wording without misrepresenting the paper. In amicus briefs for the recent supreme court Prop 8 case, anti-gay groups cited Regerus's paper as evidence that children do worse in same-sex households than in opposite-sex households. That is, the proponents of prop 8 are making a very specific causal claim: being raised by same-sex parents causes worse outcomes for the children.

Here's how economists think about causality. Suppose we observe some data. There is some outcome measure $Y$ and an explanatory variable $X$, and we want to know whether $X$ "causes" $Y$. For simplicity, suppose $X$ is just a binary variable that can either be $0$ or $1$, so what we mean by "causal" is that if we change $X$ from $0$ to $1$ (or the other way around), then $Y$ will change for that same individual observation. That is, if $y_{i,0}$ is the observed outcome for observation$i$ when $x_i=0$, and $y_{i,1}$ is the what would have been the observed outcome in the counter-factual where $x_i=1$, then "causal" means that $y_{i,0} \not= y_{i,1}$ for some particular $i$. Now, in the real world, we can't literally observe both $y_{i,0}$ and $y_{i,1}$ for the same observation $i$--statisticians try to get around this by comparing some observations $y_{i,0}$ to other observations $y_{j,1}$ and then sort out how much of the difference between those was due to switching $X$ from $0$ to $1$, rather than from switching from $i$ to $j$.

Prop 8 advocates used the Regnerus study to make a causal claim about the effect of same-sex parents on child outcomes. So in their claim, $Y$ is the measure of child outcomes, and $X=1$ if both parents are of the same-sex, while $X=0$ if the parents are of opposite sexes. Right away we can see a glaring problem with how Prop 8 advocates are misusing the Regnerus study: in the actual paper, he defined $X=0$ as being opposite-sex biological parents, but defined $X=1$ as being at least one parent who, at some point in their life, had a same-sex relationship. In some sense, you might argue that $X=1$ implies the child had a "gay parent" (though that's dubious itself), but it most definitely does not imply that the child was raised by a same-sex couple--indeed, out of 2,988 participants in the study, precisely 2 were actually raised by a same-sex couple. Moreover, almost all of the other observations or which $X=1$ in the study were actually raised exclusively in their biological, heterosexual parent's household, without the "gay" biological parent present for a majority of their childhood.

So here's my point. Let's toss Regnerus aside for a moment and design a thought experiment from scratch. What would be a causal effect of  gay parenting on child outcome? Well, using the economic definition of causality above, we want to observe the outcomes of a particular child with two opposite-sex parents (that is, we observe $y_{i,0}$), and then observe the counter-factual world in which we observe that exact same child, but where we've switched his parents from opposite-sex to same-sex (that is, observe $y_{i,1}$). But, here's the thing: we cannot, even in theory, observe $y_{i,1}$ because in that counter-factual, observation $i$ would not even exist--same-sex partners cannot have a biological child. Instead the gay couple would have to either adopt, or find a third party to have a baby.   Hence, even in a theoretical sense, we cannot distinguish between the causal effects of same-sex parenting versus the causal effect of adoption or third-party biological parents. The latter two are, just among heterosexual families, associated with worse outcomes in the data, on average.

So in this context, what do we consider causal? Is it the difference between a child who lived with two same-sex adoptive parents and the same child living with his biological parents? If there was a third party, is the "causal effect" of gay parenting the difference between the outcome when the child lives with the gay couple versus living with the one gay biological parent and the other (maybe heterosexual) biological parent who didn't get custody? Or is it the comparison between the gay couple and the biological parents, but switching the gay biological parent to straight as well? Any of those definitions of "causal" are, to various extents, defensible, but also problematic--there is, in fact, no way to think about causality here without considering changes in more than one variable at a time. And of course, the data limitations make these even more problematic.

Yet, Regnerus didn't attempt any of those comparisons. Instead, he just calculated the difference in outcomes between children with two opposite-sex biological parents who never divorced, adopted, remarried or used surrogates, to children who had at least one parent who had had a homosexual relationship sometime in their life. No attempt was ever made to distinguish between the causal effect of going from $X=0$ to $X=1$ versus the effect of going from observation $i$ to $j$.

In some abstract sense, I suppose, we'd like to forecast what the effect of increasing the share of children raised by same-sex households would be on children's outcomes. That would qualify as "causal," but I still don't think it would be meaningful--we can't change the proportion of same-sex parents by legislative fiat, and I doubt there's much in sociology that can be changed by legislative fiat. So I'm not sure what a sociologist means when he says "cause."

Perhaps the moral here is this: if you are going to submit an empirical study as evidence in a court case, there damn well better be a policy instrument somewhere in the study.
erik 6/28/2013 09:02:00 PM
fascinating post. the problem of causality regarding this question is further complicated by this fact: in order to adopt or have a third-party serve as a proxy for child-birth, it usually requires a certain socio-economic status. So while one is getting the outcome from all men-women couples, that is likely not the case on the other side.
Matthew Martin 6/28/2013 10:25:00 PM
Good point. There have been several studies showing that children raised in intact same-sex households actually do better than intact opposite-sex households. Some of them attempted to control for socio-economic factors, but if your data is non-random, the estimate is still likely to be "biased." I use quotes there because, from a policy perspective, it is not at all clear to me why we would care about the outcomes of the hypothetical children of parents without actual children.