Monday, January 12, 2015

The difference between cost and waste

Healthcare journalism has been flooded with anecdotes of outrageous hospital charges for mundane things. Here's a $1,206 toenail clipping. Here's $9,000 to put a band aid on a cut finger. I could fill several blog posts just with links to these kinds of anecdotes, but I will stop there.

One criticism I have of these posts is that they don't distinguish between cost and efficiency. Consider the toenail guy. Here's his enumerated charges:
  • Office Visit -- $248.00
  • Biopsy -- $182.00
  • Treatment Room -- $328.00
  • AMB Clinic -- $117.00
  • Pathology Lab -- $165.00
  • Laboratory Services -- $166.00
  • Total -- $1206.
The vast majority of the charges here--the office visit, most of the biopsy, the treatment room, and AMB clinic--are charges for hospital time rather than goods. But we already know that they didn't waste a significant amount of hospital time, because the whole thing took "about 15 minutes." The main source of actual waste here is the running of what turned out to be unnecessary lab tests on the sample, but my guess is that even here the vast majority of that $165 was for lab time, and that these particular tests did not actually take that much time. This is to say that an obviously outrageous price tag needn't be indicative of an outrageous quantity of resources wasted.

The way I think about hospital prices is this: they don't really calculate charges on a patient-individual basis. Instead, patients are standardized into surprisingly broad sets of codes, and the hospital charges it's average cost per each code applied to the patient. So while this particular individual had only a tiny toenail clipping, the hospital needs to charge a high enough price for all biopsies to be able to cover the biopsies that actually are expensive to administer. The guy in this anecdote wasn't paying just for his own biopsy, but also cross-subsidizing a more expensive, trickier case somewhere else in the hospital. When averaged across all patients, this hospital could be running an extraordinarily lean and efficient operation. Or it could be scamming people out of billions. Anecdotes of individual experiences do not provide any evidence on this at all.

To make the point more general, we tend to talk about US healthcare spending as if more spending for the same outcomes is necessarily inefficient, when in reality much of this is actually a matter of distribution rather than efficiency. As a society, we may decide that doctors deserve to be paid more because they perform a great service to society. Or we may decide they don't deserve as much money as they already get. These are social choices and can neither be right or wrong. But regardless of how much we choose to pay doctors, if the quantity supplied and quantity demanded do not change, then neither does the level of efficiency.

Outrageous prices necessarily involve redistribution from consumers to producers. Whether that is inefficient depends on what happens to supply and demand. There are ways in which high prices cause inefficiency: first, they incentivize unnecessary procedures that doctors would not otherwise have done. Was this the case in the toenail clipping story? Doesn't look like it to me--they took a biopsy, ran a test, and discharged the guy within 15 minutes. Maybe the test was not necessary, maybe it was. But it doesn't look like they did a lot here that wasn't justified. So, the supply side does not actually look terribly wasteful to me.

The other way high prices can cause inefficiency is if consumers opt not to buy because the price is higher than it needs to be. The healthcare not administered to patients who can't afford outrageously high hospital prices is inefficiency. But I'm somewhat skeptical that this particular factor causes much inefficiency in our system. As I mentioned above, outrageous prices are usually a sign of extensive within-code redistribution from easy to complex patients. One reason hospitals do this is to avoid losing complex patients who can't afford to pay their true individual-level cost of care, and to the extent this is the case, then the outrageous prices actually increase rather than decrease overall access to care.* Moreover, most people have insurance, which like the hospital, averages costs across patients so that within-code price variation matters little.

That's not to say that outrageous prices never cause inefficiencies, but I think this is a substantially smaller problem than it is made out to be. The much bigger issues are:
  • the fairness of the distribution of income between hospitals, insurers, and patients, and
  • medical overuse.

*Aside: a common dictum in economics is that agents must internalize their own marginal costs. A less well known result is that fixed costs should be divied up according to willingness-to-pay. Hence, easy patients pay to keep the roof over the OR, so complex surgeries can happen at marginal cost. I'm really making the standard price-discrimination-as-welfare-improving argument here.

Thursday, January 8, 2015

Fallacy of composition and life expectancy

Bill Gardner has a nice post on what the International Journal of Epidemiology calls "lagged selection bias" in how we analyze life expectancy, where we incorrectly take the difference in life expectancy for a certain sub-population and claim that life expectancy has "increased" or "decreased" for that group. This procedure is not valid. I learned about this concept in the context of Labor Economics, where the more common term is "fallacy of composition." These are the same concepts.

Gardner's post mentioned studies of female mortality, which had reported that women's life expectancy was lower in some US counties than 20 years earlier, and concludes that life expectancy has fallen for women in these counties. Because of the fallacy of composition, this conclusion does not logically follow. To drive that point home, here's a numerical example. Suppose there are two counties: Healthypeopleshire and Unhealthypeopleshire. We perform surveys over two periods to calculate life expectancy in each county each period. In the first survey, there is one resident of Healthypeopleshire with a life expectancy of 90 years, and two residents of Unhealthypeopleshire with life expectancies of 70 and 56 years respectively.
Residents of each county, with their life expectancies indicated. Our survey, however, we see only the average life expectancy for each county which is 90 for Healthypeopleshire and 63 for Unhealthypeopleshire.
After a vigorous public health campaign, life expectancy rises dramatically for all individuals in period 2:
Ann and Brad both saw their life expectancies rise by 5 years, while Chris's rose by 4 years. Nevertheless, life expectancy fell in both Healthypeopleshire and Unhealthypeopleshire.
Incidentally, one effect of our public health campaign is that Brad left Unhealthypeopleshire to go live the ways of the healthy people in Healthypeopleshire. Because Brad has a lower expectancy than the other residents of Healthypeopleshire, but a higher expectancy of the other residents of Unhealthypeopleshire, our survey actually shows that life expectancy dropped in both counties, by 5 and 3 years respectively, even though it increased dramatically overall!

Wednesday, December 31, 2014

Why I'm worried about New York City

The New York post reports that the NYPD has gone on strike:
"It’s not a slowdown — it’s a virtual work stoppage. NYPD traffic tickets and summonses for minor offenses have dropped off by a staggering 94 percent following the execution of two cops — as officers feel betrayed by the mayor and fear for their safety, The Post has learned....Citations for traffic violations fell by 94 percent, from 10,069 to 587, during that time frame. Summonses for low-level offenses like public drinking and urination also plunged 94 percent — from 4,831 to 300. Even parking violations are way down, dropping by 92 percent, from 14,699 to 1,241. Drug arrests by cops assigned to the NYPD’s Organized Crime Control Bureau — which are part of the overall number — dropped by 84 percent, from 382 to 63."
As a Cincinnatian, I've seen this all before:
"Nearly two months after an officer was indicted for killing an unarmed black suspect, Cincinnati police are so demoralized that they are ignoring some minor crimes and looking for jobs in the suburbs. Arrests are down 35 percent compared with May and June a year ago. Revenue from traffic tickets is down significantly — drivers paid $25,000 during May, compared with more than $90,000 in fines the same time last year. Judges and lawyers report lighter dockets as fewer defendants appear in court."
That's from the Cincinnati Enquirer, a few months after the outbreak of the 2001 race riots. The police protest, then and now, has been attributed as the cause of a resurgence in Cincinnati violence for a decade and a half that still has not subsided:
While violent crime in Cincinnati was falling prior to the riots and continued to fall nationally after, it rose very sharply in Cincinnati after the riots, peaking not much below the historic high before finally starting to drop around 2007-8. The rate remains above pre-riot levels in 1999 and 2000. No clue why 1997-8 data for Cincinnati is missing. Data comes from the US Department of Justice's UCR data.
Whether the spike in crime was due to the police strike or the riot itself is irrelevant--I certainly understand their frustrations, but at a time when police were most needed, they abandoned their posts. The Federalist argues that this is criminal:
"we’ve seen the type of escalating activity in the city which would be more recognizable as the preview to a messy Latin American coup d’etat. The latest is a form of purposeful sabotage on the part of the NYPD, which is now actively shirking its duty to enforce the law. ...Mayor de Blasio should’ve responded to the backs turning by firing people immediately. The NYPD needed to be reminded that chain of command exists, and that they are not at the top of it. Instead, what New York City is experiencing now amounts to nothing less than open rebellion by the lone armed force under the worst kind of weakened junta, one led by a figure ideologically radical and personally weak, who has lost control of his bureaucracies and may soon be devoured by them."
I don't use the term "criminal" metaphorically: New York law, same as Ohio law, prohibits police strikes and prescribes disciplinary action against those who do.

Some good thoughts on economic pedagogy

I'm not going to blog about this now, but just a few (not necessarily new) posts that I thought were worth reading, on the way we teach (and do) economics:

Tuesday, December 30, 2014

No, you can't control for that

EARLIER THIS month, Ezra Klein wrote a post about the use of statistical "controls" in academic studies. If statistics isn't your thing, then the background: "controls" are just the list of variables you include in your statistical model other than the the variable whose effect you are attempting to estimate. Klein says

"You see it all the time in studies. "We controlled for..." And then the list starts. The longer the better. Income. Age. Race. Religion. Height. Hair color. Sexual preference. Crossfit attendance. Love of parents. Coke or Pepsi. The more things you can control for, the stronger your study is—or, at least, the stronger your study seems. Controls give the feeling of specificity, of precision."
Is this really what people think of controls? Do the study authors put this much faith in the value of controls? I'm genuinely baffled if they do.

Klien went on to mention the downside of including too many controls

"But sometimes, you can control for too much. Sometimes you end up controlling for the thing you're trying to measure."
--basically, if the thing you care about affects the things you control for which in turn affect the outcome you care about, then your estimate is missing a portion of the effect you care about. But there's a deeper problem with the sentiment that Klein described. Adding the right controls won't do anything to guarantee that your estimate is causal. And the best study designs are the ones that minimize the number of control variables needed. First, let's examine what controls variables are for.

Suppose you want to estimate the effect of variable X on variable Y. The third variable Z also affects Y but also affects X so that X and Z are correlated. I get complaints whenever I put math on here, so here's a graph of the causal relationships:

X and Z both affect Y, but are correlated with each other because Z also affects X.
In this system, X and Z are both exogenous to Y, and if we regress X and Z on Y the coefficient of X will be an unbiased estimate of the true (linear) causal effect of X on Y. But if we omit Z from this, we get a biased estimate as the regression will falsely attribute some of the variation in Y to X, because we haven't told it about Z. I've concocted a simulation you can run in R to see this--it runs a simulation of the causal relationship above 1000 times, and records the proportion of the time that the true causal effect of X on Y specified in the program does not fall within the 95 percent confidence interval of the regression coefficient, both when Z is and is not included in the regression. With Z included, the true coefficient is outside the confidence interval just 5 percent of the time, exactly as it should be, while it falls outside the interval 100 percent of the time when Z is excluded. This is omitted variables bias. It applies when both these conditions hold:
  1. the control variable is correlated with your independent variable of interest, and
  2. the control variable has a non-zero effect on the outcome variable.
It's unfortunately pretty common for applied papers to lump any and all variables related to their outcome variable as controls in a regression, but you really shouldn't control for something unless both points above apply. Under classical conditions, including too many variables shouldn't bias the estimate, but in practice you'll get better results by excluding variables whenever it is valid to do so.

Frequently, though, I notice commentators and study authors perform a subtle bait-and-switch when they talk about controlling for various factors. For example, suppose we want to estimate the effect of the number of police officers on crime rates. Crime rates vary by neighborhood, and governments typically assign more police officers to higher crime neighborhoods. This situation sounds superficially like the paragraph above: two regressors, neighborhood and number of police, which are correlated with each other and which both affect crime rates. A naive researcher might regress police on crime, controlling for neighborhood, and claim that the estimate is causal. But it's not! There's been a subtle shift from discussion of neighborhood effects given a certain number of police on crime to the effects of crime on the number of police in those neighborhoods. There's no way to control for the latter. If you actually did this you'd probably conclude that police presence has virtually no effect on crime, but that's wrong.

The causality chart in the example above actually looks like this:

X and Z both affect Y, but are correlated with each other because Z also affects X. In addition, there is feedback whereby Y affects Z and, by extension X as well. This system is endogenous.
Because governments assign more cops to high-crime neighborhoods, and more police reduces the crime rate in those neighborhoods, your data is not going to show as strong of a correlation between police and crime as it would otherwise. OLS regression has no way to pick apart the effect of police on crime from the effect--via policy makers who watch crime rates--of crime on police numbers. This variation simply does not exist in the data, so the correlation cannot be estimated. This is endogenous variables bias, and there's no number of control variables you can add that will help.

While you can certainly have both endogeneity and omitted variables bias, in my simulation with endogeneity added, the estimate was wrong 100 percent of the time with Z in the regression, versus 100 percent of the time with Z omitted.

What we need to get a true causal estimate of the effect of police on crime is to identify a source of exogenous variation in police levels. This is different than adding control variables. One strategy would be to perform an experiment by randomly assigning police officers to neighborhoods. That would eliminate the endogeneity because randomized police presence would mean that crime rates have no effect on police presence, so it is safe to attribute variation in crime rates to the variation in police levels. Incidentally, we don't actually have to "control" for neighborhood effects now, because with perfect randomization neighborhoods are no longer correlated with levels of police, so there's no omitted variables bias either. This is what I mean when I say that the best study designs are the ones that minimize the number of control variables--the closer to random our treatment variable is, the less of both endogeneity and omitted variables bias we are likely to face.

For the most part, having a very long list of control variables is actually evidence of the weakness of the underlying study design. Controlling for these variables is better than not when there is risk of omitted variables bias, but a design that has such risks is considerably weaker than one that does not. Bottom line: be suspicious whenever a paper says "controlling for ____". There's a good chance you can't actually control for that.