# Locklin on science

## Ruling engines and lapping the ultimate screw

Posted in Design, metalshop, Progress by Scott Locklin on April 16, 2022

The story of the ruling engine is one of those bizarro incredibly important things that has slipped into obscurity, only really known by people still directly involved in this sort of thing. I was briefly involved in this area working at LBNL’s Advanced Light Source, measuring diffraction gratings, their efficiencies, and attempting to estimate how well they’d work in presence of error. I promptly forgot almost all of it in favor of learning how to pants goth girls or whatever I repurposed that set of brain cells for, but it’s still in there rattling around somewhere.

Diffraction gratings are those little rainbow thingees on your credit card. Or if you’re old, you remember the rainbow patterns on CDs, those were sort of ad-hoc diffraction gratings. Ultimately it is a set of very precise lines across a mirror substrate. There are all kinds of profiles and shapes of diffraction gratings for different purposes, but they all work roughly the same way. Different wavelengths of light are reflected into different angles via constructive interference. The simple grating equation is $\sin(\theta_m)=\sin(\theta) + m \frac{\lambda}{\Lambda}$ where $m$ is the diffracted order, $\theta_m$ is the angle of the diffracted order, $\Lambda$ is the periodicity of the grating, and $\lambda$ is the wavelength of the light diffracted.

This is a long winded way of saying if you reflect light on a grating, it will make a nice rainbow pattern. If you make a slit out of razorblades (this is basically what people use) perpendicular to the first order diffraction angle, you get a monochromator or spectrograph, depending on how you use it. This means you can resolve narrow lines in the spectra of whatever it is you’re looking at. Of course, nothing is perfect, least of all diffraction gratings. There’s a figure of merit in spectroscopy called resolving power; $R = \frac{\lambda}{\Delta \lambda}$ where $\lambda$ is the approximate wavelength of interest and $\Delta \lambda$ is the narrowness of line you want to resolve. It’s easy to show that R is proportional to the number of coherently illuminated perfect grating lines, and that any error in grating line shape or tracking will cause R to be smaller. So  if you want to discover quantum mechanics, you need to make some nice lines otherwise you’re wasting your time. Oh yeah, and obviously if you want to resolve smaller wavelengths of light, say, in the UV, you need to rule your gratings with smaller lines.

Over complex representation of a monochromator or spectrograph

Now a days we have a number of ways of making gratings, but the first way (still important and used) is using a ruling engine, which is a very fine machine tool which mechanically draws lines on a substrate using a diamond anvil. The first important such tool was Rowland’s mentioned several times now; literally the machine that launched American physics and made quantum mechanics possible. There were gratings made before, but Rowland’s was the first to make useful gratings repeatedly. For decades it was the only one capable of making decent gratings; like a machine made by super intelligent alien beings that nobody else can figure out. For decades after this, all the subsequent ruling engines that worked were Rowland designs. The first successful ruling engine which wasn’t a Rowland design is the topic of the rest of this blog; that invented by the underappreciated experimental physicist John Donovan Strong (I’ve definitely been in the same room as him early in my career, but I can’t say I remember anything about him –his book is amazing BTW). This is the type of ruling engine still used today, more or less, with some additional complications of using feedback mechanisms made possible by electronics over the years. I’m following Strong’s article from 1951 as well as a couple of  Scientific American articles.

The original Rowland machine was a sort of overgrown and ultra precise metal shaper (or for a more familiar example; a grocery store meat slicer). Strong took his design cues from the much more uncommon metal planer. The difference, Rowland’s machine advanced the relatively heavy grating blank using the precision screw, making the screw subject to mechanical deformation and stick slip, while moving the diamond using ways that could wear out.  Remember, this thing is making long, straight lines, very precisely on the order of 1000/2000 lines per millimeter; a perfect line every 500-1000 nanometers. Real nanotechnology; not the imaginary kind done with Schroedinger’s equation and pixie dust. For contrast, an atom is around a tenth of a nanometer. While they call the latest semiconductor technology 14nm, it’s really more like 100nm, and diffraction gratings built with screws were doing that, over much larger areas than a defect free wafer more than 140 years ago using doodads such as these very precise screws. There were seven major sources of error with this design in absence of mechanical or manufacturing defects, to give an idea of the type of thing involved here; they were referred to as the “seven demons.”

1. Stick slip/lubrication forces of the various moving parts caused large irregularities.
2. Wear in the various parts of the engine were also hugely important; the carriage might travel miles in ruling a grating and the Rowland carriage was a big beefy object.
3. The metal parts also contain locked-up stresses from creation from raw ore to machining; as the machine ages, the stresses relieve and the perfect surfaces deform.
4. Creep also takes place from external forces; sag, motion, weight support.
5. Any vibration may cause bad gratings to be made; one worker correlated his grating defects to the swaying of trees outside the building (this is huge with optics in general, especially in current year with all kinds of machinery around and driving by).
6. Dust of course is a big problem; get dust under the diamond cutter or in the screw/nut interface and you’re, well, screwed.
7. Finally, the heat radiated by a human body can cause sufficient creep in the engine to ruin a grating.

Strong’s gizmo obviated the stick slip problem by moving the diamond rather than the grating blank, removing the ways for moving the diamond, and improving both the lubrication of the screwing mechanisms, and the alignment techniques.  His thing used two precision screws to advance the diamond, and as they’re pointing in opposite directions, they can cancel out pressure and sag errors as well as angular “fanning” errors in the grating ruling (Rowland’s machine had microradian misalignments that borked the resolving power via this fanning effect; a microradian across a few inches is easily a wavelength of green light). Downside; you need two nice screws instead of just one.

Strong’s exposition was fascinating. He points out that precision in his day was entirely “primitive methods.” Aka geometry, averaging and lapping compounds. The dividing heads on the screws for making microscopic motions were self lapped in place on an oil bath. Instead of a kinematic mounting system for moving the grating, he overconstrained it with multiple ways which averaged out to a nice straight line.

Strong was a great scientist who understood machinery and tooling in great detail. He also had a couple of helpers he credited with his success. One of them was Wilbur Perry, an engineer trained at WPI. Before he went to school he made a bunch of telescopes, and was a proud member of the Springfield Vermont telescope makers society, which still maintains a clubhouse. Let me emphasize the implications of this: a tiny town of a few thousand people had a telescope makers society at the turn of the century, when telescopes were still high technology, and they endowed it well enough it is still physically there. That’s sort of like a small town of a few thousand people having its own privately owned MEMS fab in the 1990s when this became a more common technology. Social capital is highly underappreciated and they had lots of it in those days. Strong himself got many of his ideas for the ruling engine from hanging out in a club he founded with John Anderson (the previous John Hopkins Rowland engine driver); the “100-to-1 shot club.” Some nice oral history before it fades away: an interview with Henry Victor Neher:

NEHER: This was a small group that was formed at Caltech in about 1934 or ’35. The
way it originated was this. John Anderson, who was at the Mount Wilson Observatory, had an office at Caltech when he was working on the 200-inch telescope, back in the thirties. One of the members of the staff was a young fellow by the name of John Strong [professor of physics and astrophysics, 1937-1942], who had his experimental equipment in the same room in Bridge as I did. John Strong was over talking to John Anderson one day. John Strong was always interested in ideas of one sort or another. He was an inventor if there ever was one. John thought that there ought to be a group that considered far-out ideas of one sort or another.

INTERVIEWER: For example.

NEHER: Primarily ideas connected with something scientific or mechanical, or something of that sort. And John Anderson said, “Well, what you are suggesting is to discuss things that have one chance in a hundred of working.” And so, this is the way the 100-to-1 Shot Club was formed. They got a group together which consisted of John Strong, John Anderson, Russell Porter, Roger Hayward—who did that picture up there above the fireplace—and then some others not connected with the Institute, like Byron Graves. And there were a couple of patent attorneys in the group.
Well, I didn’t get into it right away. I guess it was about 1936 or ’37 before I became associated with it. We met once a month at various members’ homes. It was mostly discussions of ideas in connection with astronomy or with physics. There may have been some mechanical things. One of the members was George Mitchell, who designed and made the Mitchell camera that was used in Hollywood for years. Another was George Beadle [professor of biology 1946-1961], who joined after World War II.

INTERVIEWER: Did anything ever come out of it?

NEHER: No. It wasn’t meant to be that. It was just a place where you could just discuss anything you wanted.

Or as Strong himself put it:

We called it the “100 to 1 Shot Club.” We met at various member’s houses at Palomar; in the Mohave Desert; etc. — about 6 or 7 times a year. It was called by the name mentioned to indicate that our considerations (like: Does the water spin in a contrary way in the Southern hemisphere when it runs out of the bath tub? — etc.) were restricted to topics that were fantastic by a factor of 100:1 over scientific. The dozen members included: Trim Barkelov — patent council for Paramount Pictures Roger Hayward — artist and architect Victor Neher laboratory roommate George Mitchell — millionaire manufacturer of the Mitchell camera; a former Hollywood camera man Byron Graves — an amateur astronomer and retired executive from Ford Co. in Detroit John Anderson — my boss Jack McMorris — a chemist (and disappointed concert pianist) George Worrell — successor to Mitchell at the Camera plant Milton Humason — astronomer I mention this because it was a group worthy to go down in history.

The importance of such clubs can’t be overestimated. They’re everywhere in the annals of technological history; from Wernher von Braun and company’s rocket club, to the famous Lunar society, to the X club even the Bohemian Club was responsible for the US nuclear weapons program. Most great human ventures have started in some sort of men’s club. And yes, they were/are men’s clubs, u mad? As my pal BAP put it, only the most depraved ancient Greek tyrants would ban men’s associations:

A brotherhood of men in this form is the foundation of all higher life in general: there is a certain madness, an enthusiasm that exists also in a community of true scientists or artists…. it is totally forbidden in our time…. the dedication, severity, focus and enthusiasm necessary to sustain true scientific enterprise are forbidden because they make women and weaklings uncomfortable.

To remove stress in the screw blanks, he had two garbage cans with inner cells, one for heat the other for dry ice,  so he could stress relieve the screws before the finish cuts. He dipped them in what he called “tincture of skunk cabbage” (overheated Mazola corn oil at 400F, 100F) and “hobo cocktails” (dry ice and alcohol at 10, -60 and -100F). He did this stress relief cycle 50 times per screw.

When he moved on to lapping, he rigged up a tape recorder which kept a record of the torque  of the screw being lapped in its giant split nut. This way he could keep track of progress on the lapping process and the recorder would tell him when there was a burr or requirement for more lapping compound. Mind you this is a 1940s era tape recorder, so in addition to being a great machinist, he must have known a thing or two about electronics back in the vacuum tube era. He also rigged up a motor mechanism and ran the thing on his wall in his basement.

Apparently the whole new ruling engine worked the first time, which is a minor miracle. A hugely successful scientific breakthrough done with a sort of miniature Klein type-1 organization. More of a Klein type-1 A-team;  a common type of group for successful experimental physics ventures. The origins in a couple of men’s clubs and a couple of obscure working class geniuses makes it all the more sweet.

It’s also an object lesson in why current year can’t have nice things. No men’s clubs thanks to various vile and pathetic tyrannies. No working class craftsmen making things in matter. No physicists who understand how a fucking screw works (who worked on the Kansas wheat harvests).  And tens of thousands of nincompoops fiddling around on a computer instead of learning how matter works with the eyes and fingers. The very idea of using such mechanical creativity by talking to other artificers and hammer and tongs precision work is anathema to current year bugmen. I’m pretty sure they’d find a way to call the whole project sexist and racist because they don’t understand how a fucking screw lap works either.

That is the biggest advance in the grating art that I am responsible for. I also made an advance in the lapping of lead screws that is recognized in industry. I developed several techniques which are useful in precision machine tool practice. And that was a consequence of the work on ruling engines. But my work on ruling engines, in a sense, was supernumerary, because now the control of the relative position of the ruling engines components is accomplished by interferometry. Here Harrison was the pioneer.

## From ancient Gears and Screws to Quantum Mechanics

Posted in Design, metalshop, Progress by Scott Locklin on April 10, 2022

The geared mechanical clock, like the pipe organ and the Gothic cathedral is a defining symbol of Western Civilization. Division of the day into mechanically measured hours  unrelated to the movements of the sun is a mechanical symbol of the defeat of the tyranny of nature by human ingenuity and machine culture. The hours of the day used to be something measured locally by the position of the sun. The liturgy of the hours of the Catholic Church caused north-western Europeans to go all spergy and design intricate machines to tell the monks when to say their prayers, rather than using arbitrary times. After all in the sperdo north, it’s often cloudy or dark very early or barely at all: you need something better than the sun to tell the time.

There’s an oldest surviving clock; that of the Salisbury Cathedral (allegedly 1386). It’s an interesting enough looking mechanism, foliot and verge escapement (the first known mechanical escapement for counting the seconds); you can see it running here. These early clocks had the advantage over water clocks in that you didn’t need to haul water up the tower, and they didn’t freeze in the cold northern winter.

One of the interesting mysteries of technological history; nobody knows where gears came from. A gear is sort of like a wheel, or a pulley system, both of which existed long before the gear. There are claims that the Chinese had them before anyone else; the south-facing wagon is a postulated example, though the first document of it was by Yen-Su in the 11th century, long after such mechanisms were in common use in the West.  As with most of early Chinese history, this isn’t well documented and it may have been nonsense. Unless they influenced the Greeks directly, which doesn’t seem to have ever happened otherwise, the Chinese developments weren’t important in a world historical context.

As with most things, the first documented gears are Greek. Aristotle wrote about them in his physics book around 340BC -around the time of Alexander the Great. Ctesibius was first we know of to write about the things (~250BC) being used in interesting ways; his stuff all lost, but written about by later thinkers; he also invented the pipe organ. It has been suggested that water wheels using lantern pinions were the first gears: we learn of them via Vitruvius (probably originally Ctesibius). We know that Heron of Alexandra had well developed gear trains; he described some effectively like the backgears in a lathe. Archimedes invented the worm gear and pinion used in modern clockworks; possibly also the spiral bevel gears used in differentials.

The most shockingly advanced early geared mechanism is the Antikytheria mechanism. It’s one of those things people didn’t for some reason expect, but if you read old astronomy books, I’m virtually certain such mechanisms are much older. The epicyclic theories of Eudoxus (375BC) and Callippus (330 BC) were pretty explicitly gearworks; later expanded by Hipparchus and Claudius Ptolemy, who was contemporary with the Antikytheria mechanism. It’s entirely possible there were no gearworks before Posidonius (maybe) brought us the Antikytheria mechanism. I suppose it’s possible there were no gearworks after. But it seems vastly more likely we didn’t just randomly pick up a unique space alien technology toy off the sea bed, and there are probably more such treasures still buried in other places, perhaps even sitting somewhere in a Museum storage closet.

My pet theory, for which there is exactly zero evidence, is that gears were very ancient and lost with the late Bronze age collapse. Certainly they had brasses and small drills and the ability to fabricate elaborate objects out of much harder materials. Since the Greeks didn’t mention where they got the basic gear idea, I’m assuming it existed before they started making more clever versions of it. I suppose such things could have existed in some other culture (Sumerian, Egyptian, whatever), but it’s my pet theory; feel free to come up with your own.

The Screw might have been an invention of Archimedes as well, though some historians attribute it to a more forgotten artificer called Archytas (my pal Eudoxus‘ teacher from the time of Plato). Screws were used in the Mediterranean region for olive and grape presses. There is a widespread misapprehension that the science of the ancient Greeks was some kind of theoretical construct: not so. The mechanical and scientific ideas of the great ancient philosophers and the Alexandrian Museum were used by ordinary people on a daily basis. From screw presses to waterwheel contraptions, the Hellenistic and Roman world benefited from the applications of Greek thought.

Gears allow one to change the plane of rotary motion, or the angular velocity of the rotary motion. Screws turn rotary motion into linear motion, generally considerably stepped down in velocity. You need both to make something like a modern clock or a mechanical lathe. Screws are commonplace now and used everywhere, but they really are a wonder. An inclined plane wrapped around a cylinder. Early screws were made with tools like hammer and chisel, with taps and dies made in the same way, various kinds of ingenious mechanisms to assist the process.

Making the first set of standardized and precise screws took until around 1800, culminating in Henry Maudslay‘s screw cutting lathe which was one of the most important inventions in human history. The screw cutting lathe required a screw and gears, combined together on a rigid lathe bed. The lathe bed is effectively a plane, allowing for precise motion. You can make these with a chisel and scraper/file out of arbitrary chunks of steel or cast iron; hobbyists still carve and scrape their own lathe beds. The leadscrew allowed the cutter to automatically move along a piece of rotating stock to cut another thread in a piece of rotating stock. Changegears allowed one to cut arbitrary threads from stock, by altering the ratio of screw linear motion to workstock rotation. With all these ingredients you can move a cutter along a piece of rotating screw blank an arbitrary amount, making arbitrary pitch screws. What’s more, you can amplify the accuracy of your leadscrew to a certain extent. There are heroic tales of Maudslay creating his master leadscrew that are in their own way as glorious and Promethean as Benvenuto Cellini casting his Perseus statue.

Maudslay is one of those guys who created a whole center of excellence around himself; he was a blacksmith/locksmith who built a classic Klein type-1 organization. He invented all manner of clever devices we now take for granted, from micrometers to various kinds of steam engine and telescope; he was even involved with the father of Isambard Kingdom Brunel building machines for his various ventures. Maudsley’s students (it wasn’t a school, it was a high technology business) went on to make numerous further innovations and form their own high technology companies. Joseph Whitworth invented all kinds of machine screw standards (BSW still in use today) machine tools and measuring devices, guns and so on and became enormously wealthy. Joseph Clement built the first version of the difference engine.  William Muir manufactured machine tools, Richard Roberts made locomotives and  power looms, and James Nasmyth invented the steam hammer and shaping machine.There aren’t any substantive books written about this amazing group of men, and the one I know about is expensive and out of print, but if done properly it would make an excellent case study of a Klein type-1 organization. All of these guys were giants of invention and industry and they all got stinking rich inventing new technologies and increasing man’s power over nature. There’s a sort of pamphlet about Maudsley available on archive, which is slightly better than nothing.  I assume there were contemporaries who wrote about them, but they’re mostly forgotten today.

Back to screws; using a screw, you can precisely position things on a nanoscale. I’ve done it, using these little buggers called picomotors. You can buy big giant screws made in temperature controlled oil baths which are capable of similar tricks while retaining their accuracy as well. It boggled my mind when I first read about how this is accomplished; basically the same way most mechanical accuracy is achieved; by lapping with abrasives. You can read all about it in the old Wayne Moore book “Foundations of Mechanical Accuracy.”

Which brings me back to Henry Rowland, father of American physics. It was Rowland who invented the techniques for creating the ultra precise nanoscale screw by manually lapping the screw in a giant split nut. He did this to create diffraction gratings using a “ruling engline.” Diffraction gratings are responsible for the origin of modern physics, as scientists needed them to resolve atomic spectra. And of course as I said in the previous blog, Rowland was by his own self largely responsible for American physics activities in general.

The story of the ultra-precise screw and ruling engine is so insanely awesome I’ll dedicate a later blerg or two to the topic.

## Office chairs are a scam

Posted in Design by Scott Locklin on April 23, 2021

A younger self employed pal asked me if he should dump $2k on a Herman Miller Aeron recently. The TLDR is “no.” It’s like spending$2k on a pair of sneakers blessed by a devil worshipping blood-drinking pothead. Worse; unlike the $2k devil worshipping blood sneakers (which I suppose will eventually be mandatory for corporate dress codes), it’s become something that everyone unquestionably accepts. Let’s do a little history here; there was a first office chair, and it existed for a reason. Originally it was because Charles Darwin had a giant room filled with biological specimens and he needed to flit from object to object to invent evolution. So he stuck some wheels on a comfy Victorian chair. I’ve used caster-chairs in laboratories for their intended purpose. They’re great and they exist for a reason. Thank you based Charles Darwin for inventing laboratory caster chair. Darwin chair; respek Later, such chairs were used as an integral part in early information technologies systems. Aka office workers with filing cabinets; yards and yards of filing cabinets, performing essentially the same functions done by a roughly equivalent volume of IT professionals and computer databases. The little casters on the bottom of the chair; that’s for flitting from cabinet to cabinet without getting up. They’re sort of like the actuator arms of the R/W head of magnetic spinney disk drives. The innovation in this era of office chair is they have a swivel mechanism, making it even easier to maneuver around, bring files back to your desk, and do clerical work with a typewriter and Marchant calculator. This also made total sense and it was both healthy and civilized having such aids to efficiency in offices. At some point they put another joint in the things so self-important executives (who were actually glorified clerks: otherwise they’d have sat in stuffed armchairs) could sit in such a chair with their feet on the desk; this was never needed, but it was popular anyway. I inherited such a 50s era chair from a friend and kind of wish I had kept it going by bringing the metal parts to the machine shop, as it looked cool at least. By the 70s, such chairs were pretty universal and made out of plastic. By then, information technology was an important factor, so swiveling and rolling past the filing cabinet was vastly less important. However physical decay and the workman’s comp lawsuit had become important factors in the workplace. People sued their employers for having a bad back. In reality, your bad back is some combination of genetic factors and poor maintenance. But whatever: this is when ergonomics became a factor in office chair design. Now, the reality is, there is absolutely no such thing as “ergonomics.” Nobody who isn’t a 60s era fighter plane or space capsule designer, has any idea how to make a chair that is “ergonomic” -nobody really has any idea what “ergonomic” means in terms of office chairs. It’s just a sort of virtue word with vaguely medical connotations. Cargo cult science at its best, designed to ward off bad juju like lawsuits. This is where it started to go bad really quickly. Herman Miller who invented the office cubicle: the second most dystopian form of office, started producing “ergonomic” chairs festooned with gas-lift levers to alter the height, and lots of sliding doodads to move arm rests and knee flex points and so on. The idea was based on factory/industrial designs. Cube farms are a sort of production line for clerical work. I’m pretty sure casters were completely irrelevant at this point, but since cube farms were all covered in filthy plastic carpet, and non-casters would ruin the shitty rugs, the casters stayed. Rotations same thing. And I guess you could still put your feet up in your smelly little carpeted cube. the ergon; this atrocity is considered “iconic” Finally we come to the Herman Miller Aeron chair; the throne of the bugman open office. Add even more slidey “ergonomic” pieces. Because we’re doing “ergonomics” don’t you know. You can raise and lower the arms, cant the thing at all kinds of odd angles, roll around; it is endlessly ergonomic (whatever that means). Truly the Aeron chair is like the restaurant with 100 specialties on the menu; infinite choices for propping up your ridiculous clerical meatsack. Since by the 90s not only were office workers completely gelatinous and in need of 10,000 adjustments to hold them in front of their computers against the ravages of gravity, they were also enormously fat sweathogs. The plastic carpeted cushions of Herman Miller’s Ergon gave way to a sort of polyester hammock-net so they wouldn’t develop grotesque sweat and fart stains on their office chairs and kakhis and cotton shirts. Something that was somehow missing when everyone wore wool and weren’t disgusting gelatinous ham planets. The chair itself, as a chair is, like most aspects of modernity, almost unspeakably ugly: it looks like some kind of arachnid or brine shrimp. I came to the realization that office chairs are a scam when I was writing my Ph.D. thesis. Up until that point in life, I had spent most of my time as a free range physicist, bounding up and down vacuum chamber platforms, dashing to the machine shop, rolling the helium leak checker over and crawling over piles of conflat flanges; even practicing martial arts with the Filipino janitor who studied with Remy Presas. Suddenly though, I was strapped to a chair answering emails, doing Fortran, gnuplot and LaTeX all day; then goofing off by making fun of people on the internet. I actually developed mild carpal tunnel and a sore lower back. Since I worked for the government they sent a safety and ergonomics expert over who upgraded my chair from an Ergon looking thing to something more like an Aeron and gave me training and a new “ergonomic” keyboard. It very obviously didn’t help one whit. Moving to the LBNL physics library and sitting on hard wood and metal chairs from 50s Army offices helped some. Going back to the gym and lifting heavy objects cured it for all time. Your body isn’t a tub of shit when you exercise and so you can happily sit on a log or boulder and be just fine. Best open office chair I had was one of those exercise balls; it kept my posture good and was way more comfy than any other kind of plastic chair could be. Plus it flexed on my coworkers who would get sore in the midsection if they attempted it. For my home-office; I mostly sit in tub chairs. It’s been right for 250+ years; peak chair. The more “advanced” a chair gets, the more degenerate it is. If you train your body to decant itself into some arbitrary chair shape (with 1001 “ergonomic” positions) like a living jello mold or octopus hiding in a mayonnaise jar, you will eventually inevitably experience back problems. This happens because despite your bugman lifestyle, you actually are a vertebrate. Look at how the spinal column is made; it’s not made to be held up by stupid chairs; all those little pointy deely boppers on your spine are muscle attachments. Those muscles are meant to never relax: they’re structural, just as much as the ligament, cartilage and bone in your spinal column. That’s why people with back problems do better sleeping on the floor rather than a waterbed. The other thing that makes these things a scam; in addition to being made of materials which are both fragile and uncleanable (unlike, say, steel, wood, leather or even hard plastic) it is made up of numerous fragile parts which wear out and break. People own leather tub chairs for decades; assuming there is no plastic foam in them, for centuries even. Literally all of this could be discerned if people paid attention or had the rudiments of common sense. Instead of using humble powers of observation, office drones have been brainwashed into uncritically accepting the claims of the vendors of these ridiculous contrivances. Instead of demanding a comfy tub chair and their own damn office, modern invertebrates accept the swindles of their bugman overlords, and feel important sitting in horrific open offices on their intricate, smelly and insubstantial$2000 plastic thrones. Anyway when you get back to the office after the covidiocy abates, tell them to sell the chair and give you an office with a regular chair in it; even a metal folding chair or a stool in your own office is an improvement. They won’t, because the chairs cost way less than urban commercial real estate, but that’s the future you should be aiming for anyway. At least as good as the one portrayed in Brazil.

## Woo for its own sake

Posted in Design, tools by Scott Locklin on January 8, 2021

Software development is a funny profession. It covers people who do stuff ranging from register twiddling in device drivers and OS guts, to people who serve web content, to “big data” statisticians, to devops infrastructure, to people who write javascript and html front ends on electron apps. To a certain extent, software engineering is grossly underpaid. If software engineers were allowed to capture more of the value they create, we’d have vastly fewer billionaires and more software engineers with normal upper middle class lifestyles, such as houses owned in the clear and successful reproductive lifecycles. The underpaid are often compensated in self esteem.

By “compensated in self esteem” I don’t mean they have high self esteem; I mean the manager saying “dude yer so fookin smart brah” kind. This is the same brainlet payment system in place in the present day “hard sciences” with people writing bullshit papers nobody cares about, or, like, journalists and other “twitter activists” who believe themselves to be intellectual workers rather than the snitches and witch hunters they actually are. Basically, nerd gets a pat on the head instead of a paycheck.

Once in a while, independent minded programmers demand more. They may or may not be “so fookin smart,” but they think they are. Their day jobs consist of unpleasant plumbing tasks, keeping various Rube Goldberg contraptions functioning and generally eating soylent and larva-burgers and claiming to like it. As such, most programmers long to do something fancy, like develop a web server based on Category Theory, or write a stack of really cool lisp macros for generating ad server callbacks, or add some weird new programming language of dubious utility to an already complex and fragile stack.

Allowing your unicycle-riding silver pants mentat to write the prototype in Haskell to keep him from getting a job at the Hedge Fund may make some HR sense. But if you’re going to rewrite the thing in Java so a bunch of offshore midwits can keep it running, maybe the “adulting” thing to do is just write it in Java in the first place.

I’m not shitting on Haskell in particular, though there is an argument to be made for looking askance at using it in production. Haskell is mostly a researchy/academicy language. I don’t know, but I strongly suspect its run of the mill libraries dealing with stuff like network and storage is weak and not fully debugged. Why do I suspect this? In part from casual observation, but also from sociology. Haskell is a fancy language with people doing fancy things in it. One of the valuable things about popular but boring languages is that the code has been traversed many times, and routine stuff you’re likely to use in production is probably well debugged. This isn’t always true, but it’s mostly true. The other benefit to boring languages is people concentrate on the problem, rather than the interesting complexities of the language itself.

You see it in smaller ways too; people who feel like every line of code has to be innovative: new elliptic curves, new network protocols, new block ciphers, new ZNP systems; to a crucial money oriented application that would have been really cool and have a much smaller attack surface if you had bestowed only one innovation on it. I guess this sort of thing is like bike-shedding or Yak-shaving, but it’s really something more perverse. If you have a job doing shit with computers, you are presumably solving real world problems which someone pays for. Maybe, you know, you should solve the problem instead of being a unicycle riding silver pants juggling chainsaws.

You see a lot of it in the cryptocurrency community, in part because there is enough money floating around, the lunatics are often running the asylum, in part for its undeserved reputation as being complicated (it’s just a shared database with rules and checksums; Bram more or less did the hard part in the summer of 2000 while my buddy Gerald was sleeping on his couch). For example: this atrocity by Gnosis. Gnosis is an interesting project which I hope is around for a long time. They’re doing a ton of very difficult things. Recently they decided to offer multi-token batch auctions. Why? I have no freaking idea. It’s about as necessary and in demand as riding to work in silver pants on a unicycle. Worse though: from an engineering perspective, it involves mixed integer programming, which is, as every sane person knows, NP-hard.

This is a danger in putting software developers or programmers in charge. These guys are often child-like in their enthusiasm for new and shiny things. Engineers are different: they’re trying to solve a problem. Engineers understand it’s OK to solve the problem with ephemeral, trashy, but fast-to-market solutions if the product manager is going to change it all next week. Engineers also plan for the future when the software is critical infrastructure that lives and fortunes may depend on. Engineers don’t build things that require mixed integer programming unless it’s absolutely necessary to solve a real world problem. If they juggle on unicycles, they do it on their own time; not at work.

Consider an engineering solution for critical infrastructure from a previous era; that of providing motive power for small fishing boats. Motors were vastly superior to sail for this task. In the early days of motorized fishing, in some cases until fairly recently, there was no radio to call for help if something goes wrong. You’re out there in the vastness on your own; possibly by yourself, with nothing but your wits and your vessel. There’s probably not much in the way of supply lines when you’re at shore either. So the motors of the early days were extremely reliable. Few, robust moving parts, simple two stroke semi diesel operation, runs on any fuel, requires no electricity to start; just an old fashioned vaporizing torch which runs on your fuel; in a pinch you could start a fire of log books. You glance at such a thing and you know it is designed for robust operation. Indeed the same engines have been used more or less continuously for decades; they only turn at 500 rpm, and drive the propeller directly rather than through a gearbox.

Such engines are useful enough they remain in use to this day; new ones of roughly this design are still sold by the Sabb company in Norway. They’re not as environmentally friendly or fuel efficient as modern ones (though close in the latter measure), but they’re definitely more reliable where it counts. When you look at this in the engine room, you are filled with confidence Mr. Scott will keep the warp drives running. If you find some jackass on a unicycle back there (who will probably try to stick a solar powered Sterling engine in the thing), maybe not so much.

I don’t think long term software engineering looks much different from this. Stuff you can trust looks like a giant one-piston semidiesel. You make it out of well known, well traversed and well tested parts. There are a couple of well regarded essays on the boringness yet awesomeness of golang. Despite abundant disagreement I think there is a lot to that. Nobody writes code in golang because of its extreme beauty or interesting abstractions. It is a boring garbage collected thing that looks like C for grownups, or Java not designed by 90s era object oriented nanotech fearing imbeciles. I think it bothers a lot of people that it’s not complicated enough. I’m not shilling for it, but I think anyone who overlooks it for network oriented coding because it’s boring or they think it’s “slow” because it doesn’t use functors or borrow checkers or whatever is a unicycle riding idiot though. Again looking at blockchain land; Geth (written in golang) has mostly been a rock, where the (Rust) Parity team struggles to maintain parity with feature roll outs and eventually exploded into multiple code bases the last time I checked. There’s zero perceptible performance difference between them.

There’s a Joel Spolsky on (Peter Seibel interview with) JWZ which I always related to on complexification of the software process:

One principle duct tape programmers understand well is that any kind of coding technique that’s even slightly complicated is going to doom your project. Duct tape programmers tend to avoid C++, templates, multiple inheritance, multithreading, COM, CORBA, and a host of other technologies that are all totally reasonable, when you think long and hard about them, but are, honestly, just a little bit too hard for the human brain.

Sure, there’s nothing officially wrong with trying to write multithreaded code in C++ on Windows using COM. But it’s prone to disastrous bugs, the kind of bugs that only happen under very specific timing scenarios, because our brains are not, honestly, good enough to write this kind of code. Mediocre programmers are, frankly, defensive about this, and they don’t want to admit that they’re not able to write this super-complicated code, so they let the bullies on their team plow away with some godforsaken template architecture in C++ because otherwise they’d have to admit that they just don’t feel smart enough to use what would otherwise be a perfectly good programming technique FOR SPOCK. Duct tape programmers don’t give a shit what you think about them. They stick to simple basic and easy to use tools and use the extra brainpower that these tools leave them to write more useful features for their customers.

I don’t think this captures the perverseness and destructiveness of people who try to get fancy for no reason, nor do I think JWZ was a “duct tape programmer” -he was an engineer, and that’s why his products actually shipped.

I say this as an aficionado of a couple of fancy and specialized languages I use on a regular basis. I know that it is possible to increase programmer productivity through language choice, and often times, runtime performance really doesn’t suffer. Languages like OCaML, APL and Lisp have demonstrated that small teams can deliver complex high performance software that works reliably. Delphi and Labview are other examples of high productivity languages; the former for its amazing IDE, and the latter for representing state machines as flow charts and providing useful modules for hardware. The problem is that large teams probably can’t deliver complex high performance software that works reliably using these tools. One also must pay a high price up front in learning to deal with them at all, depending on where you come from (not so much with Labview). From a hiring manager or engineer’s perspective, the choice to develop in a weird high productivity language is fraught. What happens if the thing crashes at 4 in the morning? Do you have enough spare people someone can be raised on the telephone to fix it? What if it’s something up the dependency tree written by an eccentric who is usually mountaineering in the Alps? For mission critical production code, the human machine that keeps it running can’t be ignored. If your mentat gets hit by a bus or joins the circus as a unicycle juggler and the code breaks in production you’re in deep sheeyit. The idea that it won’t ever break because muh technology is retarded and the towers of jelly that are modern OS/language/framework stacks are almost without exception going to break when you update things.

The “don’t get fancy” maxim applies in spades to something like data science. There are abundant reasons to just use Naive Bayes in production code for something like sentiment analysis. They’re easy to debug and they have a trivial semi-supervised mode using the EM algorithm if you’re short of data. For unsupervised clustering or decomposition it’s hard to beat geometric approaches like single-linkage/dbscan or PCA. For regression or classification models, linear regression is pretty good, or gradient boost/random forest/KNN. Most of the time, your real problem is shitty data, so using the most accurate  tool is completely useless.

Using the latest tool is even worse. 99 times out of 100, the latest woo in machine learning is not an actual improvement over existing techniques. 100% of the time it is touted as a great revolution because it beat some other technique … on a carefully curated data set. Such results are trumpeted by the researcher because …. WTF else do you expect them to do? They just spent a year or two developing a new technique; the professor is trying to get tenure or be a big kahuna, and the student is trying to get a job by being expert in the new technique. What are they going to tell you? That their new technique was kind of dumb and worthless?

I’ve fallen for this a number of times now; I will admit my sins. I fooled around a bit with t-SNE while I was at Ayasdi, and I could never get it to do anything sane. I just assumed I was a moron who couldn’t use this advanced piece of technology. No, actually, t-SNE is kind of bullshit; a glorified random number generator that once in a while randomly finds an interesting embedding. SAX looked cool because it embodied some ideas I had been fooling around with for almost a decade, but even the author admits it is horse shit. At this point when some new thing comes along, especially if people are talking about it in weeb-land forums, I pretty much ignore it, unless it is being touted to me by a person who has actually used it on a substantive problem with unambiguously excellent results. Matrix profiles looks like one of these; SAX dude dreamed it up, and like SAX, it appears to be an arbitrary collection of vaguely common sense things to do that’s pretty equivalent to any number of similar techniques dating back over the last 40 years.

There are innovations in data science tools. But most of them since boosting are pretty marginal in their returns, or only apply to corner cases you’re unlikely to encounter.  Some make it easier to see what’s going on, some find problems with statistical estimators, but mostly you’re going to get better payoff by getting better at the basics. Everyone is so in love with woo, the guy who can actually do a solid estimate of mean differences is going to provide a lot more value than the guy who knows about the latest PR release from UC Riverside.

Good old numerical linear algebra, which everyone roundly ignores, is a more interesting subject than machine learning in current year.  How many of you know about using CUR decompositions in your PCA calculations? Ever look at some sloppy PCA and wonder which rows/columns produced most of the variance? Well, that’s what a CUR decomposition is. Obviously looking at the top 3 most important of each isn’t going to be as accurate as looking at the regular PCA, but it sure can be helpful. Nuclear Norm and non-negative matrix factorizations all look like they do useful things. They don’t get shilled; just quietly used by engineering types who find them helpful.

I’m tooling up a small machine shop again, and it makes me wonder what shops for the creation of physical mechanisms would look like if this mindset were pervasive. The archetypical small shop has always had a lathe in it. Probably the first thing after you get tired of hacksawing up material; a bandsaw or powered hacksaw. Small endmill, rotary sharpener, and you’re off to the races; generally building up more tooling for whatever steam engines, clocks or automatons you feel like building. I’m imagining the archetypical unicycle-juggler buying a shop full of solid printers and weird CNC machines and forgetting to buy cutters, hacksaws, files and machinist squares. As if files and machinist squares are beneath them in current year.