The Last Word On Nothing

Caveat: I’m possibly having something like a Cassie-Willyard-Hubble-Moment here – in this case, I learn something new, don’t quite understand it but get all excited about it, and it’s, you know, wrong.  Never mind because I’m all excited anyway because science has found a new way of being confident that what you know is right.  Ok, confident enough that what you know is right enough.

I was reading about a defense department war game that took place completely inside a computer.  Individual virtual terrorists worked together on an attack; the defense department had to catch them first and needed to find the method that would work best.   So put the terrorists’ behaviors and tactics into a virtual world along with the defenders’ methods, and see whether a given method was any good at finding terrorists.  No? you missed some terrorists and got blown up?  Try another method until you find one that works.

I don’t think this war game was ever actually created but what do I know.  Nor do I know whether this is actually an accurate description of the war game, nor whether it’s actually as similar to a game astronomers play as I think it is.

Astronomers call their war games, simulations, and they’re traditionally played between theorists and observers.  Theorists who want to know if they have the right idea about, say, the way galaxies are born, put gas, gravity, and physics into a computer to see whether what comes out looks like the actual galaxies that observers see.  Simulations are technically awesome – that word meant literally – and beautiful.  But they’re not exactly what the defense department was doing, testing a method.

Astronomers do that too, by using what they call mock data.  Their problem is not looking through a lot of people and identifying the terrorists but sorting through terabytes of data and finding, say, the clusters of galaxies.  But how? using what methods?  This is trickier than it sounds.  Maybe they’re got a good method for finding clusters and they use it to look through the terabytes and find no clusters.  Does that mean the clusters are really not there? Or that their method might be good but it’s not perfect?

So they make a mock universe full of mock galaxies, with realistic colors and motions and distances, and put it all into a computer programmed with their method of finding clusters, plus – and here’s the thing – a certain number of clusters.  Then they run the simulation, and if the number of clusters the method finds is the same as the number they put in, then hey! The method works!  “You’re comparing ‘out’ to ‘in,'” says Molly Peeples who’s running a workshop at her home in the Space Telescope Science Institute called Mocking the Universe,  “And no real data are harmed in this experiment.”

This seems to me so ingenious.  Can you trust your method of finding something? Is the thing that the method finds even real?  Is that a needle or just some random piece of hay?  Ask a question, know the answer, then see whether your method of answering finds it.

“It’s a way of thinking that’s getting more common,” Peeples says.  She thinks particle physicists use this way of thinking to figure out whether, in their floods of data, they’ve just seen – for instance — the only magnetic monopole in the universe.  Or maybe with the war games, the defense department inputs a certain number of terrorists and checks to see if their surveillance finds them all.  Or maybe input a certain number of needles into your haystack to see if your search methods find them.  “To measure something complicated,” Peeples says, “you want to know if what you find is real.  You’re testing your confidence in the measurement.”

I swear, this is most of the reason I like astrophysicists so much:  like every other one of God’s children, they don’t know whether they can believe what they see; but unlike the rest of us, they’ve figured out clever work-arounds by which they can quantify confidence.  I’m still not sure I’ve gotten it right — if you have any ideas please let me know — nor am I sure I can use this in real life.  But I’m working on it.

__________

soldier with haystack – United States Forces Iraq; haystack with possible needle – theilr; both via Flickr