Locklin on science

On the Empire of the Ants

Posted in brainz, information theory by Scott Locklin on July 2, 2013

The internet is generally a wasteland of cat memes and political invective. Once in a while it serves its original purpose in disseminating new ideas. I stumbled across Boris Ryabko‘s little corner of the web while researching compression learning algorithms (which, BTW, are much more fundamental and important than crap like ARIMA). In it, I found one of the nicest little curiosity driven  papers I’ve come across in some time. Ryabko and his coworker, Zhanna Reznikova, measured the information processing abilities of ants, and the information capacity of ant languages. Download it here. There was also a plenary talk at an IEEE conference you can download here.


In our degenerate age where people think cell phone apps are innovations,  it is probably necessary to explain why this is a glorious piece of work. Science is an exercise in curiosity about nature. It is a process. It sometimes involves complex and costly apparatus, or the resources of giant institutes. Sometimes it involves looking at ants in an ant farm, and knowing some clever math. Many people are gobsmacked by the technological gizmos used to do science. They think the giant S&M dungeons of tokomaks and synchro-cyclotrons are science. Those aren’t science; they’re tools. The end product; the insights into nature -that is what is important. Professors Ryabko and Reznikova did something a kid could understand the implications of, but no kid could actually do. The fact that they did it at all indicates they have the child-like curiosity and love for nature that is the true spirit of scientific enquiry. As far as I am concerned, Ryabko and Reznikova are real scientists. The thousands of co-authors on the Higgs paper; able technicians I am sure, but their contributions are a widows mite to the gold sovereign of Ryabko and Reznikova.

Theory: ants are smart, and they talk with their antennae. How smart are they, and how much information can they transfer with their antennae language? Here’s a video of talking ants from Professor Reznikova’s webpage:

Experiment: to figure out how much information they can transfer, starve some ants (hey, it’s for science), stick some food at random places in a binary tree, and see how fast they can tell the other ants about it. Here’s a video clip of the setup. Each fork in the path of a physical binary tree represents 1 bit of information, just as it does on your computer. Paint the ants so you know which is which. When a scout ant finds the food, you remove the maze, and put in place an identical one to avoid their sniffing the ant trails or the food in it.  This way, the only way for the other ants to find the fork the food was in is via actual ant communication. Time the ant communication between the scout ant and other foragers (takes longer than 30 seconds, apparently). Result: F. sanguinea can transmit around 0.74 bits a minute.  F. polyctena can do 1.1 bits a minute.


Experiment: to figure out if ants are smart, see if they can pass on maze information in a compressed way. LRLRLRLRLRLR is a lot simpler in an information theoretical sense than an equal length random sequence of lefts and rights. Telephone transmission and MP3 players have this sort of compression baked into them to make storage and transmission more efficient.  If ants can communicate directions for a regular maze faster than a random one, they’re kind of smart. Result: in fact, this turns out to be the case.

Experiment: to find out if ants are smart, see if they can count. Stick them in a comb or hub shaped maze where there is food at the end of one of the 25 or more forks (you can see some of the mazes here). The only way the poor ant can tell other ants about it is if he says something like “seventeenth one to the left.” Or, in the case of one of the variants of this experiment,  something more like”3 over from the one the crazy Russian usually puts the food in.” Yep, you can see it plain as pie in the plots: ants have a hard time explaining “number 30” and a much easier time of saying, “two over from the one the food is usually in.” Ants can do math.


The power of information theory is not appreciated as it should be. We use the products of it every time we fire up a computer or a cell phone, but it is applicable in many areas where a mention of “Shannon entropy” will be met with a shrug. Learning about the Empire of the Ants is just one example.

People in the SETI project are looking for  alien ham radios on other planets. I’ve often wondered why people think they’ll be able to recognize an alien language as such. Sophisticated information encoding systems look an awful lot like noise. The English language isn’t particularly sophisticated as an encoding system. Its compressibility indicates this. If I were an alien, I might use very compressed signals (sort of like we do with some of our electronic communications). It might look an awful lot like noise.

We have yet to communicate  with dolphins. We’re pretty sure they have interesting things to say, via an information theoretical result called Zipf’s law (though others disagree,  it seems likely they’re saying something pretty complex). There are  better techniques to “decompress” dolphin vocalizations than Zipf’s law: I use some of them looking for patterns in economic systems. Unfortunately marine biologists are usually not current with information theoretical tools, and the types of people who are familiar with such tools are busy working for the NSA and Rentech. Should I ever make my pile of dough and retire, I’ll hopefully have enough loot to strap a couple of tape recorders to the dolphins. It seems something worth doing.

The beautiful result of Ryabko and Reznikova points the way forward. A low budget, high concept experiment, done with stopwatches, paint and miniature plastic ant habitrails produced this beautiful result on insect intelligence. It is such a simple experiment, anyone with some time and some ants could have done it! This sort of “small science” seems rare these days; people are more interested in big budget things, designed to answer questions about minutae, rather than interesting things about the world around us. I don’t know if we have the spirit to do such “small science” in America any longer.  American scientists seem like bureaucratized lemmings, hypnotized by budgets, much like the poor ants are hypnotized by sugar water. The Rube-Goldberg nature of this experiment could only be done by a nation of curious tinkerers; something we no longer seem to have here.

Dolphin language could have been decoded decades ago. While it is sad that such studies haven’t been done yet, it leaves open new frontiers for creative young scientists today. Stop whining about your budget and get to work!


Mormon nuclear fusion

Posted in Design, energy by Scott Locklin on July 2, 2013

Most of you have never heard of Philo T. Farnsworth. Philo T. Farnsworth is famous for never getting credit for inventing the Television machine.  I never thought Television was particularly interesting (either as a device, or in any other way), though I have to admit, the Television machine is a pretty impressive accomplishment for a 14 year old farm-boy Mormon. Even more impressive was his successful attempt to build a fusion machine.


Farnsworth, like all good inventors, took a workman like approach to nuclear fusion. Thousands of morons (as opposed to Mormons) in the scientific establishment have been trying for literally, decades, to the tune of hundreds of billions of dollars, to achieve what Farnsworth did, using what amounts to a pile of junk.  His solution is still considered pretty  good, and if it were given a fair trial, it might even beat the billion dollar efforts out there in achieving break-even (aka as much fusion energy out as was put in). The Navy recently revived the idea in the form of “Polywell Fusion.” It’s so simple, anyone can build a Farnsworth Fusor in his basement; there are websites devoted to hobbyist efforts. Kids regularly build these things for science fair projects. That’s how dumb and easy they are. The most complicated thing about them is the vacuum pump they use.


The “big science” buffoons use magnetic confinement; a copy of a Soviet idea that never went anywhere. You end up with a giant toroidal machine, with megawatts of energy going to keep the fusion plasma contained in place. Farnsworth’s idea just used some rings of metal to more or less passively keep the ionized fluid in place. It’s such a simple device, you could construct one out of TV and refrigerator parts, with the electrostatic rings made of old coat hangers. Such machines are used commercially as neutron sources, as they produce lots of fusion reactions (though nowhere near breakeven thus far).


Farnsworth was probably the last great American inventor. I’d like to think there will be great inventing men to come after him, but I’m pretty sure it won’t happen here any more, as the continuity is gone. Independent inventing men like Edison, Tesla and the Maxim brothers are part of America’s tradition; Farnsworth was the last of the great ones. Now we think of men as inventors when they write some crap piece of software. Farnsworth was uneducated by modern lights; only a few years of college. He was an actual farm boy, and he thought of television while ploughing the fields. TV is a rastering process, like ploughing fields. Yet, he invented all manner of machines, as well as being an accomplished mathematician.

Why won’t there be any more like him? The tinkering mentality is gone. Guys from the midwest  in the early 20th century were tinkerers who fixed things because they had to in those days. You can’t really understand physical reality by screwing around with CAD and computer models. You can only understand physical reality by, well, tinkering with it. My pal Rodrigo recently sent me a Tom Wolfe essay (about Intel’s Bob Noyce, primarily) which illustrates the point, and also demonstrates why modern bureaucratic space flight is such a galloping failure:

The engineers who fulfilled one of man’s most ancient dreams, that of traveling to the moon, came from the same background, the small towns of the Midwest and the West. After the triumph of Apollo 11, when Neil Armstrong and Buzz Aldrin became the first mortals to walk on the moon, NASA’s administrator, Tom Paine, happened to remark in conversation: “This was the triumph of the squares. ” A reporter overheard him; and did the press ever have a time with that! But Paine had come up with a penetrating insight. As it says in the Book of Matthew, the last shall be first. It was engineers from the supposedly backward and narrow-minded boondocks who had provided not only the genius but also the passion and the daring that won the space race and carried out John F. Kennedy’s exhortation, back in 1961. to put a man on the moon “before this decade is out.” The passion and the daring of these engineers was as remarkable as their talent. Time after time they had to shake off the meddling hands of timid souls from back east. The contribution of MIT to Project Mercury was minus one. The minus one was Jerome Wiesner of the MIT electronic research lab who was brought in by Kennedy as a special adviser to straighten out the space program when it seemed to be faltering early in 1961. Wiesner kept flinching when he saw what NASA’s boondockers were preparing to do. He tried to persuade to forfeit the manned space race to the Soviets and concentrate instead on unmanned scientific missions. The boondockers of Project Mercury, starting with the project’s director, Bob Gilruth, an aeronautical engineer from Nashwauk, Minnesota, dodged Wiesner for months, like moonshiners evading a roadblock, until they got astronaut Alan Shepard launched on the first Mercury mission. Who had time to waste on players as behind the times as Jerome Wiesner and the Massachusetts Institute of Technology…out here on technology’s leading edge?

Just why was it that small-town boys from the Middle West dominated the engineering frontiers? Noyce concluded it was because in a small town you became a technician, a tinker, an engineer, and an and inventor, by necessity.


Of course, Farnsworth was hounded by scumbags for most of his life. David Sarnoff, the evil weasel who founded NBC, and early patent troll, attempted to sue Farnsworth to penury. He ultimately failed in this endeavor, though the mind reels at the injustice of a towering genius like Farnsworth having to pay any attention to such nonsense. Who knows what wonders Farnsworth may have come up with had he been free to pursue his interests, rather than being tied up in pointless patent disputes with sleazeballs?

Consider Philo Farnsworth the next time someone tells you we live in an era of scientific progress. Where are our Philo Farnsworths today? They certainly aren’t laboring in a make work program in some government lab, nor do they seem to be inventing anything particularly interesting.





https://www.neco.navy.mil/synopsis_file/N6893609C0125%20_Redacted_JA.pdf (the navy can’t update their security certs, apparently).

BTC bubbles

Posted in econophysics by Scott Locklin on April 17, 2013

Not surprisingly, Bitcoin prices are well described by the  log periodic power laws describing the dynamics of bubbles. A reminder of what a LPPL model looks like; here is a simple one:

\log(p(t)) = A + B(t_c - t)^\beta + C(t_c - t)^\beta \cos( \omega \log(t_c-t)+\phi)

I didn’t profit from this. I thought of applying LPPL to the BTC bubble well before the crash during a bullshit session with a friend, but I didn’t run the analysis until after. I have better things to do with my time than play with weird monopoly money, and the “exchanges” presently offering shorts are not even close to useful. I also think anyone who trades on LPPL is basically gambling. The most interesting parameter, t_c is hardest to fit, and, well, with all those parameters I could fit a whole lot of elephants. Just the same it is a useful enough concept to justify further research. No, I won’t be telling the world about that research on my blog. A man’s got to eat, after all. Doing bubble physics costs money.

If you don’t know about LPPL models, click on these two helpful links. The “hand wavey” idea is, if the price is formed by market participants looking at what other market participants are doing, as with Dutch tulips, pets.com, and market prices in various eras, the price is an irrational bubble which will eventually burst. This isn’t an original idea: Charles Mackay was talking about it 180 years ago. The original idea is mapping this behavior onto an Ising model,  running some renormalization group theory on it, and fitting to the result to get a forecast of bubble burstings.  Sornette, Ledoit,  Johanson, Bouchaud and Freund did it and told the world about it; may the eternal void bless them with healthy returns for being kind enough to share this interesting idea with us.

Here’s a plot of BTC close prices from MtGox (via quandl), with the LPPL model fit 10 days before the bubble pop. I wasn’t real careful with the fit; no unit root tests were done, no probabilistic estimates were made and no Ornstein Uhlenbeck processes were taken into account. This is just curve fitting. The result is compelling enough to talk about. As you can see, with these parameters, the out of sample top is fit fairly well. Amusingly, so is the decline.


What can we learn from this? You can see a “fair value” of around $20/BTC due to be hit in a few weeks, with perhaps a full mean reversion to $10/BTC.  BTC doesn’t seem to have a helpful “anti-bubble” decay; if anything, it is decaying faster than expected so far (it is possible I mis-fit the \omega). The fit parameters for this version of the model tell us a few interesting things about the herding behavior which you can read about in Sornette’s book.

I don’t have any strong opinions about using BTC as a currency. I think most of its enthusiasts  are naive and do not understand the nature of money and what it is good for. I do think BTC would work a lot better as a store of value with a properly functioning foreign exchange futures market. There are no properly functioning BTC futures exchanges at present; just an assortment of dreamers and borderline crooks cashing in on hype. This is more of an engineering and legal problem than it is an inherent problem with using BTC as a currency. The way things are presently set up, without shorts, any extra media attention will result only in people buying the damn things. Without the ability to easily short them, price discovery is impossible, and herding behavior is the rule. It ain’t a market without shorts. It’s a bubble maker. Shorts don’t guarantee there will be no bubbles; we see plenty in shortable markets, but a lack of shorts will virtually guarantee future BTC bubbles.

The enigma of the Ford paradox

Posted in chaos, physics by Scott Locklin on March 7, 2013

“God plays dice with the Universe. But they’re loaded dice. And the main objective of physics now is to find out what rules were they and how we can use them for our own ends.” -Joe Ford


Joe Ford was one of the greats of “Chaos Theory.” He is largely responsible for turning this into a topic of interest in the West (the Soviets invented much of it independently) through his founding of the journal Physica D. It is one of the indignities of physics history that he isn’t more widely recognized for his contributions. I never met the guy, as he died around the time I began studying his ideas, but my former colleagues sing his praises as a great scientist and a fine man. One of his lost ideas, working with student Matthias Ilg and coworker Giorgio Mantica, is the “Ford paradox.” The Ford paradox is so obscure, a google search on it only turns up comments by me. This is a bloody shame, as it is extremely interesting.

Definitions: In dynamical systems theory, we call the motion of a constrained system an “orbit.” No need to think of planets here; they are associated with the word “orbit” because they were the first orbital systems formally studied. It’s obvious what an orbit is if you look at the Hamiltonian, but for now, just consider an orbit to be some kind of constrained motion.

In most nontrivial dynamical systems theory, we also define something called the phase space.” The phase space is that which fully defines the dynamical state of the system. In mechanics, the general convention is to define it by position and momentum of the objects under study. If the object is constrained to travel in a plane and its mass doesn’t change, like, say, a pendulum, you only have two variables; angular position, and its time derivative, and you can easily visualize the phase space:


For my last definition, I will define the spectrum for the purposes of this exposition. The spectrum is the Fourier transform with respect to time of the orbits. Effectively, it is the energy levels of the dynamical system. If you know the energy and the structure of the phase space, classically speaking, you know what the motion is.

Consider a chaotic system, such as the double pendulum. Double pendulums, as you might expect, have two moving parts, so the phase space is four dimensional, but we can just look at the angle of the bottom most pendulum with respect to the upper pendulum:


If you break down the phase space into regions, and assign a string to each region, one can characterize  chaos by the length of the string in bits. If it is a repeated string, the system is non-chaotic. Chaotic systems are random number generators. They generate random strings. This is one of the fundamental results of modern dynamical systems theory.  A periodic orbit can be reduced to simple sequences, like: {1 0 1 0 1 0}, {1 1 0 1 1 0 1 1 0}. Effectively, periodic orbits are integers. Chaotic orbits have no simple repeating sequences. Chaotic orbits look like real numbers. Not floats which can be represented in a couple of bytes: actual real numbers, like  base of the natural log e or \pi or the golden ratio \phi . In a very real sense, chaotic orbits generate new information. Chaotic randomness sounds like the opposite of information, but noisy signals contain lots of information. Otherwise, qua information theory, you could represent the noise with a simple string, identify it, and remove it.  People have invented  mechanical computers that work on this principle. This fact also underlies the workings of many machine learning algorithms. Joe Ford had an extremely witty quoteable about this: “Evolution is chaos with feedback.”

This is all immediately obvious when you view the phase space for a chaotic system, versus a non-chaotic system. Here is a phase space for the end pendulum of a double pendulum at a non-chaotic set of parameters: it behaves more or less like a simple pendulum. My plots are in radians (unlike the above one for a normal pendulum, which I found somewhere else), but otherwise, you should see some familiar features:


It looks squished because, well, it is a bipendulum. The bottom which looks like  lines instead of distorted ellipses  is where the lower pendulum flips over the upper pendulum. The important thing to notice is, the orbits are all closed paths. If you divided the phase space into two regions, the path defined string would reduce to something like {1 0 1 0 1 0…} (or in the lower case { 0 0 0 0…}) forever.

Next, we examine a partially chaotic regime. The chaotic parts of the phase space look like fuzz, because we don’t know where the pendulum will be on the phase space at any given instant. There are still some periodic orbits here. Some look reminiscent of the non-chaotic orbits. Others would require longer strings to describe fully.  What you should get from this; the orbits in the chaotic regions are random. Maybe the next point in time will be a 1. Maybe a 0. So, we’re generating new information here. The chaotic parts and not so chaotic parts are defined on a manifold. Studying the geometry of these manifolds is much of the business of dynamical systems theory. Non-chaotic systems always fall on a torus shaped manifold. You can see in the phase space that they even look like slices of a torus. Chaotic systems are, by definition, not on a torus. They’re on a really weird manifold.


Finally: a really chaotic double pendulum. There are almost no periodic orbits left here; it’s all motion in chaotic, and the path the double pendulum follows generates random bits on virtually any path available to it in the phase space:


Now, consider quantum mechanics. In QM, we can’t observe the position and momentum of an object with infinite precision, so the phase space is “fuzzy.” I don’t feel like plotting this out using Husimi functions, but the ultimate result of it is the chaotic regions are smoothed over. Since the universe can’t know the exact trajectory of the object, it must remain agnostic as to the  path taken. The spectrum of a quantum mechanical orbital system looks like … a bunch of periodic orbits. The quantum spectrum vaguely resembles the parts of the classical phase space that look like slices of a torus. I believe it was W.P. Reinhardt who waggishly called this the “vague tori.” He also said, “the vague tori, being of too indistinct a character to object, are then heavily exploited…” Quantum chaologists are damn funny.

This may seem subtle, but according to quantum mechanics, the “motion” is completely defined by periodic orbits. There are no chaotic orbits in quantum mechanics.  In other words, you have a small set of  periodic orbits which completely define the quantum system. If the orbits are all periodic, there is  less information content than orbits which are chaotic. If this sort of thing is true in general, it indicates that classical physics could be a more fundamental theory than quantum mechanics.

As an interesting aside: we can see neat things in the statistics of the quantum spectrum when the classical equivalent is chaotic; the spectrum looks like the eigenvalues of a random matrix. Since quantum mechanics can be studied as matrix theory, this was a somewhat expected result. Eigenvalues of a random matrix were studied at great length by people interested in the spectra of nuclei, though the nuclear randomness comes from the complexity of the nucleus (aka, all the many protons and neutrons), rather than the complexity of the underlying classical dynamics.  Still, it was pretty interesting when folks first noticed it in simple atomic systems with classically chaotic dynamics. The quantum spectra of a classically non-chaotic system are more or less near neighbor Poisson distributed. Quantum spectra repulse one another. You know something is up when near neighbor spectral distribution starts to look like this:

wigner distribution

Random matrix theory is now used by folks in acoustics. Since sound is wave mechanics, and since wave mechanics can be approximated in the short wavelength regime by particles, the same spectral properties apply.  One can design better concert hall acoustics by making the “short wavelength” regime chaotic. This way there are no dead spots or resonances in the concert hall. Same thing applies to acoustically invisible submarines. I may expand upon this, and its relationship to financial and machine learning problems in a later blog post. Spectral analysis is important everywhere.

Returning from the aside to the Ford paradox. Our chaotic pendulum is happily chugging along producing random bits we can use to, I dunno, encrypt stuff or otherwise perform computations. But, QM orbits behave like classical periodic orbits, albeit ones that don’t like standing too close to one another. If quantum mechanics is the ultimate theory of the universe: where do the long strings of random bits come from in a classically chaotic system? Since people believe that QM is the ultimate  law of the universe, somehow we must be able to recover all of classical physics from quantum mechanics. This includes information generating systems like the paths of chaotic orbits. If we can’t derive such chaotic orbits from a QM model, that indicates that QM might not be the ultimate law of nature. Either that, or our understanding of QM  is incomplete. Is there a point where the fuzzy QM picture turn into the classical bit generating picture? If so, what does it look like in the transition?

I’ve had  physicists tell me that this is “trivial,” and that the “correspondence principle” handles this case. The problem is, classically chaotic systems egregiously violate the correspondence principle. Classically chaotic systems generate  information over time. Quantum mechanical systems are completely defined by stationary periodic orbits. To say the “correspondence principle handles this” is to merely assert that we’ll always get the correct answer, when, in fact, there are two different answers. The Ford paradox is asking the question: if QM is the ultimate theory of nature, where do the long bit strings in a classically chaotic dynamical system come from? How is the classical chaotic manifold  constructed from quantum mechanical fundamentals?

Joe Ford was a scientist’s scientist who understood that “the true method of knowledge is experiment.” He suggested we go build one of these crazy things and see what happens, rather than simply yakking about it. Why not  build a set of small and precise double pendulums and see what happens? The double pendulum is pretty good, in that its classical mechanics has been exhaustively studied. If you make a small enough one, and study it on the right time scales, quantum mechanics should apply. In principle, you can make a bunch of them of various sizes, excite them to the chaotic manifold, and watch the dynamics unfold.  You should also do this in simulation, of course. My pal Luca made some steps in that direction.  This experiment could also be done with other kinds of classically chaotic systems; perhaps the stadium problem is the right approach. Nobody, to my knowledge, is thinking of doing this experiment, though there are many potential ways to do it.

It’s possible Joe Ford and I have misunderstood things. It is possible that spectral theory and the idea of the “quantum break time” answers the question sufficiently. But the question has not to my knowledge been rigorously answered. It seems to me much a more interesting question than the ones posed by cosmology and high energy physics. For one thing, it is an answerable question with available experimental tests. For another, it probably has real-world consequences in all kinds of places. Finally, it is probably a  productive approach to unifying information theory with quantum mechanics, which many people agree is worth doing. More so than playing  games postulating quantum computers. Even if you are a quantum computing enthusiast, this should be an interesting question. Do the bits in the long chaotic string exist in a superposition of states, only made actual by observation? If that is so, does the measurement produce the randomness? What if I measure differently?

But alas, until someone answers the question, I’ll have to ponder it myself.

Edit add:
For people with a background in physics who want to understand the information theory behind this idea, the following paper is useful:

“The Arnol’d Cat: Failure of the Correspondence Principle” J. Ford, G. Mantica, G. H. Ristow, Physica D, Volume 50, Issue 3, July 1991, Pages 493–520