Locklin on science

HFT using neutrino physics: Stats Jackassery

Posted in physics, stats jackass of the month by Scott Locklin on June 18, 2012

I’ve always wondered what good electroweak theory could ever do for anybody, technologically speaking. The unification theory between electrical  and magnetic forces produced huge technological benefits for humanity; pretty much all electrical, electronic and radio technology is the result -and the technological results happened quickly. Physicists work feverishly on unification theories, more or less because electromagnetic theory was so damned important to humanity. Electromagnetism was unified with the “weak field” way back in 1973 or so (or 1968, depending on if you count the theory before the experiment, which I don’t), and Salaam, Glashow and Weinberg were awarded the Nobel Prize for it in 1979. Call it a round 40 years ago. 40 years after Maxwell’s equations (the unification theory between electricity and magnetism) were written down, humans were using electromechanical power on a wide scale, and radio was already being used (in financial applications no less). Not so much has happened technologically since folks invented electroweak theory.

Espen Haug has apparently spoken of  a potential use for electroweak theory. It got carried by Forbes. His idea, which I assume was somewhat in jest, was using electroweak theory to do high frequency trading. Because neutrinos don’t interact strongly with the rest of nature (that’s why they call it the “weak force”), you can transmit a beam of them through the earth. Basically, Haug noticed that “through the earth” is a much more straight line than “across the earth” which is how signals are generally transmitted. The perimeter of a circle is longer than its diameter. Something which has been known since people started drawing circles in patches of dirt. Therefore, you can potentially trade ahead of price movements in far-away exchanges.

The problem with this, of course, is the fact that any beam of particles which can be transmitted through a giant piece of iron and silicon like the earth can’t be easily detected by anyone. There is a reason they call it the “weak force.” It’s really weak! Detecting any neutrinos at all is a pretty neat trick.  If we wait around a long time, and have really big detectors and a lot of neutrinos coming from somewhere, we can see a neutrino interact with a proton once in a while. People do this sort of thing for a living. It’s fairly important stuff for cosmology, astrophysics and high energy physics. Measurements are difficult, so any experiment involving neutrinos pushes knowledge forward.

I had thought about doing a Shannon type calculation, making some guesses as to neutrino flux humans are capable of producing and transmitting through the earth, and looking at cross sections of the best detectors, to see what kind of information can be transmitted in this way. Another way to think about it, how long do you have to sit around at your detector and count things to see an unambiguous signal in your Poisson noise? If it’s longer than a few milliseconds, you can’t do this trick and make HFT front-runny money.  I don’t know much about neutrino detectors, but I do know that the best ones are size of large scale mining installations, and the time frames for looking for interesting signals are measured in years. It turns out someone already did the hard work for me experimentally by building a neutrino telegraph.

The MINERVA detector has been used for this purpose already, in concert with a beam of neutrinos from Fermilab, which is probably close to the best we can do for making lots of neutrinos. The bit rate is reported as 0.1 bits/second, with a 1% error rate. It was also only through 240 meters of rock (it was about 1km total), as opposed to the diameter of the earth, which is 12750 kilometers. No high frequency trading is going to happen at 0.1 bits/second, or whatever lower rate one can get transmitting the beam through some large chord of the earth’s diameter, assuming you can do that at all.

There are other problems with the idea. How do you modulate a neutrino beam? Can you do it on a millisecond timescale? Maybe you can, but accelerators are big giant things, and doing things like ramping magnetic fields in them up and down to change the energy or amplitude of neutrinos, or accelerate a bit string of neutrino-making protons clumps takes a long time. Making an atom smasher which makes a lot of neutrinos … well, I’m guessing it will be even bigger than Fermilab, which is pretty damn big. I also don’t have a good idea of how collimated a beam of neutrinos are. My guess would be, “not very.” But even if you could make a neutrino ray with a laser-like milliradian divergence (almost certainly impossible), the beam radius on the other end of the earth will be measured in kilometers. This would imply that a detector at the other end would have to be very big indeed. Or else someone else could build a detector within the beam radius and see the same thing.

On the other end of things, the detector in the MINERVA experiment would indeed “fit in a basement” at someone’s trading office; it was only 5 tons of scintillators. Putting aside the beam divergence issue, this would work a lot better if it was a lot bigger. The more mass you have, the more neutrinos you can see. That’s why folks do things like using a cubic kilometer of antarctic ice pack as a detector. Assuming you could scale up the bit rate by increasing the detector size, maybe if you built one 100,000 times bigger, that would be good enough? I’m guessing that 500,000 tons of detector might cost a bit of money. I suppose it is possible, if unlikely. Submarine cables from San Francisco to New Zealand are around 80,000 tons, rather expensive, and not as complex.

Something tells me the HFT boys  aren’t going to be running triangle arb on neutrino signals, like, ever. Nice funding attempt though.

I don’t think Espen Haug deserves the stats jackass award, as he’s a serious guy who knows about noise distributions. He also hasn’t written any papers on the subject. Similar comments apply to the aptly named neutrino physicist, John Learned who presumably knows about detector noise, and isn’t actively agitating for neutrino ansibles.  Bruce Dorminey wrote it; he should definitely know better. Either way, Forbes published this, and it doesn’t even pass a sniff test: someone there was jackassed enough to publish this without disclaimers. For that, Bruce and his credulous numskull editors at Forbes Magazine are, stats jackass of the month:

Yet another stab at explaining the debt crisis

Posted in finance journalism by Scott Locklin on December 7, 2011

Why Michael Lewis Annoys the Bejeepers out of Me

Posted in finance journalism by Scott Locklin on August 29, 2011

Michael Lewis is the preeminent financial journalist of our age. In many ways, Michael Lewis is the only financial journalist of our age. No other author on finance is so widely read. His articles are widely taken as something like the conventional wisdom. This is a great tragedy, as, despite the fact that Michael Lewis is unarguably a great writer, he’s a terrible journalist. Reading a Michael Lewis article on finance is much like watching the evening news. It gives you the impression that you’re well informed, but in reality, you’ve been deceived by noise.

Consider his latest Vanity Fair missive. Michael Lewis attempts to get to the bottom of the German side of the financial crisis. This is an interesting and tremendously important subject. Why? Because Germans didn’t have a financial crisis. This, despite the fact that the German Landesbanks were the counterparties in a good fraction of the printing of shitty bonds (aka, Germans own a lot of the worthless bonds printed up by American banks). This despite the fact that the Germans own a bunch of shitty Greek government paper. This despite the fact that Germany was incinerated and invaded in WW-2, and half her territory and a third of her population incarcerated under communism until 1989. Yet, Germany is a prosperous and pleasant nation to live in; one of the best in the world. Germany manages to have lower unemployment than the US, despite all their unions and socialistic regulations for hiring and firing: laws which Harvard economist ding a lings will insist would be the ruination of the American economy. How did the Germans manage this?

Michael Lewis doesn’t know; he’s too busy making turd jokes. I’m not exaggerating. Some jackass gave him Alan Dundes’ imbecilic, “Life is a Chicken Coop Ladder.” This is a book documenting alleged German folklore surrounding scatology, and drawing the conclusion that Germans are prone to killing Jews because of inadequate toilet training. If you think that’s some kind of joke: it isn’t; the guy who wrote it is completely serious. The local Kraut-hating bigot who insisted I read it was serious as well. Lewis takes it at face value also, and uses it as an excuse to regale us with Teutonic turd humor for a significant fraction of the essay.

Lewis’ article is some funny and well written bigotry, including the obligatory Nazi jokes all English speaking writers are required to make when discussing German culture. Of course, if he did an article on some cannibalistic African hell hole in the same “light hearted” spirit, he’d be hounded by the Southern Poverty Law Center for the rest of his life as an evil cousin-fucking racist. Not that there is anything wrong with that: some of my best friends and all that. I’m just pointing it out.

Lewis has, according to his report, spent a grand total of about a week of his adult life in Germany. His report, like the rest of the VF roadtrip series, is an adolescent travel story: some ethnic stereotypes, with some bad financial analysis thrown in. It’s true, the Landesbanks were royally ass-raped by shady American bankers. It’s also true that Germans tend to be gullible and trust in authorities, such as the American credit rating agencies. This is a racial characteristic of virtually all Germans I have known; German-American or straight from the tap. It is also true that Germans tend to be orderly people with a deep seated thirst for justice, equity and social harmony. I don’t think this has anything to do with poop, and I don’t think Germans have any more interest in turds or mud wrestling than anybody else. The real story of Germany is their banking systems is awesome, despite the fact that the actual banks were run by retards (rather than thieves), and their economy is doing really well. Somehow the great journalist finds it more interesting making poop jokes, rather than getting to the bottom of this. Why do the Germans do so well, despite their getting the wrong end of the financial disaster?

The last Lewis article I read was about Iceland. Again, he gives a fair, if bigoted characterization of Icelanders: fearless risk takers who belong on fishing boats or strongman contests rather than trading floors. No real argument from me on that one: Icelanders are brave men. His silly prescription to heal the Icelandic economy? Let Icelandic feminists run their banks. I’ve got a better idea: why not let Germans run their banks? Germans have a longer track record, and have unarguably contributed more to banking, human civilization and the ordering thereof than all the feminists in all nations in all of human history.

The important story from Iceland, he missed completely: they told the EU and the world financial system to eat shit, and are actually better off for it. It was obvious they were going to, but Lewis didn’t raise the possibility in his 2009 article, or any subsequent ones. The other real story of the Iceland tragedy: there are actual villains, and they’re not brave honest-faced Icelandic fishermen. Lewis is too chicken shit to name them (he could have; I’m pretty sure wikileaks did). I’ll name them: the Tchenguiz Brothers -a couple of Iraqi guys from the Anti-Semitic central casting department who ripped Kaupthing off to the tune of 2 Billion dollars, and the fat bag of fecal debris who let them get away with it: Sigurdur Einarsson.

Lewis’ assertions about the alleged German love of turds doesn’t even pass the sniff test. He claims the Germans have the only toilet museum in the world? Well, I know of one in South Korea off the top of my head. Feeding the phrase into google nets one in New Delhi, one in Massachusetts, one in Texas, one in Ukraine, one in Great Britain and at least one on the goddamned internet. Who checks Lewis’ facts? My editor would shit on me from a tremendous height for making such an unsubstantiated remark, and he’s not even a little bit German. Similarly, Lewis got an assertion about Irish parliamentary proceedings wrong too: mostly because it got in the way of a good racial stereotype about my lazy, potato-eating, Bog Negro cousins. If he can’t get crappy little facts like this right … why should anyone believe him about anything else?

It’s difficult to resist mentioning Lewis the prophet. The way he runs around unexplaining things, you’d think he predicted our present financial predicament. He seems to claim to: Liars Poker was his prediction, dontcha know. While it was a great book, his memory is faulty. The internet is forever, or so I am told. Right before the poop hit the prop in 2007, Lewis was making farty noises at the heavy hitters at Davos who were warning of immanent dangers, calling them, “wimps, ninnies and pointless skeptics.” I owe Janet Tavakoli an apology: she was right about the guy. I’ll go out on a limb as a forecaster: his assertion that the Germans will continue to pay for lazy Greeks and other lazy Southern Europeans is dead wrong. I haven’t been to Germany in over a decade, excepting for airport fly throughs, and I know that for a fact.

Michael Lewis is part of the establishment. His position in the financial community is based on celebrity and patronage rather than actually knowing anything. The man lives in Berkeley for crying out loud. If you want to really understand what is going on in the world, you’re not going to get anything out of listening to a perfumed prince of the media-banking-government complex. No, I don’t think Lewis is in on any conspiracies; I just think it’s easier and more comfortable for him to be funny and misleading than right.

Does anyone at The Atlantic understand statistics?

Posted in finance journalism, stats jackass of the month by Scott Locklin on March 24, 2011

At least they’re not pimping talking points for the oligarchy this time: it’s just general journalistic imbecility. No, Anne Hathaway news does not drive Berkshire Hathaway price changes. No, I haven’t tried to do this regression, nor will I ever try to do this regression, because I’m not as statistically retarded as people who think the Huffington Post is anything but an exotic white noise signal generated by the amygdalas of neurotic liberal arts majors.

If anyone reading my blog has fallen victim to the latest installment of the Atlantic’s regularly scheduled moronathon, please see David Leinweber’s excellent and hysterically funny paper, Stupid Data Miner Tricks: Overfitting the S&P500. In it, he uses almost twice as many data points as the HuffPo mouth breather to show a nearly perfect correlation between Bangladeshi butter production and the S&P500. Adding in sheep population also made the regression better. Professor Leinweber’s paper is a classic in the field: anyone who cares about doing statistics properly should read and internalize its lessons.

As for the hedge fund consultant guy they interviewed, John Bates: I hope you are suitably embarrassed or they egregiously misquoted you. Otherwise, I hope I never have to fix one of your messes. You should hang your head in epic shame for your apparent donkey-like lack of understanding of even the most rudimentary ideas about spurious correlation. Until you make amends and grovel in shame before your professional peers for misleading the public about a six data point spurious correlation in exchange for a little publicity, I hereby award you with the very first “Locklin on science statistical jackass of the month” prize:

Enjoy your prize. If I were in charge of the guild, it would be the stockade and rotten cabbages for you. Progress software? Not the guys that make Apama? Some pimp tried to recruit me to that outfit. I couldn’t understand why a CEP would be written in Java, just as I now can’t even imagine working for a company whose CTO doesn’t understand regression.