The Last Word On Nothing

Earlier this year, I made the case for tracking outcomes. As we enter 2015, now is a natural time to reflect on the year that was. As I do every December, I’ve spent some time this month evaluating my work performance, my accomplishments and my failures, and, as always, the process has led me to insights that I would have otherwise missed. Here, I argue that everyone can benefit from tracking outcomes, and I explain why sometimes we resist.

Earlier this year, I installed a little program on my computer that tracks how I spend my time. At the end of the day, it can tell me how many minutes I spent editing a specific document, how long it took me to write a blog post and how much time I spent surfing the internet or checking email. The time tracker is part of my ongoing experiment on how to better manage my time. I’d been playing around with different tools for a while when it occurred to me that I didn’t actually know where all my time was going. So I started collecting data.

The results were enlightening. I was certain that social media and LOL cats were hogging too much time, but after tracking my numbers for a little while, I discovered that those diversions were just little blips. The data showed me that email was my actual number one time suck. I’d had no idea it was so bad, probably because internet surfing feels like guilty pleasure, while email feels like work.

Simply identifying the problem represented a huge step toward fixing it. Within a week, I had doubled my productivity score and cut in half the amount of time I was wasting on email. I didn’t take any drastic measures. I added a couple new filters to improve my email triage, but mostly I just paid attention. With the little timer window watching me, I automatically became more mindful of my habits.

I’ll never be one of those people who tracks every step and quantifies every possible aspect of their lives, but I’ve become a believer in tracking how I’m doing in areas I’d like to improve. Yes, tracking outcomes is often tedious, but it’s worth doing, because it turns out that we’re not very good at judging our performance. Most people think they’re above average, and this is true across disciplines. For instance, a 2006 study published in JAMA found that, “physicians have a limited ability to accurately self-assess,” and a 2012 study found that doctors overestimate the value of the care they provide. 

I was thinking about this recently while working on a Washington Post column about platelet-rich plasma treatments (PRP), a heavily-marketed intervention to treat an array of orthopedic conditions like tendonitis, muscle strains and osteoarthritis. The PRP market is expected to grow to $126 million by 2016, despite scant evidence that it works.

Many (mostly small) studies exist, but the results are mixed, at best. A Cochrane review published earlier this year found that “there is currently insufficient evidence to support the use of [platelet-rich therapy] for treating musculoskeletal soft tissue injuries.” Similarly, the conclusion of a British Journal of Sports Medicine paper published this year is found in its title, “Strong evidence against platelet-rich plasma injections for chronic lateral epicondylar tendinopathy: a systematic review.” (LET is also known as tennis elbow.)

Given its widespread use, you’d think that PRP has cleared some kind of standard for effectiveness. You’d be wrong. It turns out that PRP (along with many other orthopedic procedures and devices) don’t fall under FDA regulation. As bioethicist Karen Maschke at the Hastings Center told me, the clinics and hospitals that offer it are “engaging in entrepreneurial medicine.” They’re under no obligation to show that it works.

To make matters worse, there’s no standardized methodology for delivering PRP, so it’s difficult to even compare studies, since there are so many different preparations, protocols and dosing schemes for delivering it. It’s the wild west out there.

Why are doctors so keen to offer a treatment that, upon inspection, looks promising, but not ready for prime time? Because even absent definitive evidence, those who provide PRP believe in it, and belief is a powerful driver in medicine. “We don’t have enough information to show that it works, but we believe it has great potential to work,” Frederick Azar told me. Azar is president of the American Academy of Orthopaedic Surgeons.

I understand the appeal. Injured athletes will try anything to return to play, and if they’ve exhausted other options, why not offer a treatment that can at least offer them hope? (“Platelet-rich placebo?” was the title of one paper published last year.) It might not help, but as long as you don’t think it will hurt either, it makes everyone feel better to do something.

And maybe the injured person isn’t directly harmed. Maybe, even if his tendonitis doesn’t heal any faster, he really does feel better, because he’s doing something. Passengers at the Houston airport stopped complaining about baggage claim wait times after the walkway to the carousel was rerouted so that they spent far more time walking, but virtually none waiting. It’s probable that patients who get PRP will come away highly satisfied, because even if PRP doesn’t actually help them, it makes them feel better by giving them something empowering to do while they wait. Meanwhile, doctors also feel better, because they get the warm feeling of having helped someone and some extra income for doing so.

It’s tempting to view this as a win-win situation, until you start to think about the implications. First, we know that once a treatment becomes established, it’s very, very difficult to walk it back, even when studies show that it doesn’t work. Medical costs in the U.S. are higher than in other western countries, but all that spending isn’t improving outcomes and our infatuation with expensive, high-tech treatments that don’t improve outcomes are one reason. 

We could turn this around if every physician and medical practice made outcome tracking a priority. This is already happening on a larger scale (most insurance companies won’t pay for PRP, because it’s unverified), but individual doctors and medical practices also have a role to play. If you’re going to offer an experimental treatment, do yourself and your patients a favor and measure how it works. With the advent of electronic medical records, this should become easier to do.

Yet when I interviewed doctors who offer PRP, I found that none of them were tracking the results of PRP in a systematic way. When I asked them what proportion of their patients benefitted, they could only offer stabs in the dark.

And that gets me to another thing that tracking my time has taught me. Sometimes, I don’t want to know. My tracker most often tells me I’m spending it less wisely than I want to, and some days, I prefer the bliss of ignorance.

Illustration by Shutterstock.com