It seems that every day, some computer monkey comes up with a new and more groovy serialization protocol.
In the beginning, there was ASN.1 and XDR, and it was good. I think ASN.1 came first, and like many old things, it was very efficient. XDR was easier to use. At some point, probably before ASN.1, people noticed you could serialize things using stuff like s-expressions for a human readable JSON like format.
Today, we have an insane profusion of serializers. CORBA (which always sucked), Thrift, protocol buffers, Messagepack, Avro, BSON, BERT, Property Lists, Bencode (Bram … how could you?), Hessian, ICE, Etch, CapnProto (because he didn’t get it right the first time), SNAC, Dbus, MUSCLE, YAML, SXDF, XML-RPC, MIME, FIX, FAST, JSON, serialization in Python, R, PHP, ROOT and Perl… Somehow this is seen as progress.
Like many modern evils, I trace this one to Java and Google. You see, Google needed a serialization protocol across thousands of machines which had versioning. They probably did the obvious thing of tinkering with XDR by sticking a required header on it which allowed for versioning, then noticed that Intel chips are not Big Endian the way Sun chips were, and decided to write their own semi shitty versioning version of XDR … along with their own (unarguably shitty) version of RPC. Everything has been downhill since then. Facebook couldn’t possibly use something written at Google, so they built “Thrift,” which hardly lives up to its name, but at least has a less shitty version of RPC in it. Java monkeys eventually noticed how slow XML was between garbage collects and wrote the slightly less shitty but still completely missing the point Avro. From there, every ambitious and fastidious programmer out there seems to have come up with something which suits their particular use case, but doesn’t really differ much in performance or capabilities from the classics.
The result of all this is that, instead of having a computer ecosystem where anything can talk to anything else, we have a veritable tower of babel where nothing talks to anything else. Imagine if there were 40 competing and completely mutually unintelligible versions of html or text encodings: that’s how I see the state of serialization today. Having all these choices isn’t good for anything: it’s just anarchy. There really should be a one size fits all minimal serialization protocol, just the same way there is a one size fits all network protocol which moves data around the entire internet, and, like UTF-8. You can have two flavors of the same thing: one S-exp like which a human can read, and one which is more efficient. I guess it should be little-endian, since we all live in Intel’s world now, but otherwise, it doesn’t need to do anything but run everywhere.
IMO, this is a social problem, not a computer science problem. The actual problem was solved in the 80s with crap like XDR and S-expressions which provide fast binary and human readable/self describable representations of data. Everything else is just commentary on this, and it only gets written because it’s kind of easy for a guy with a bachelors degree in CS to write one, and more fun to dorks than solving real problems like fixing bugs. Ultimately this profusion creates more problems than creating a new one solves: you have to make the generator/parser work on multiple languages and platforms, and each implementation on each language/platform will be of varying quality.
I’m a huge proponent of XDR, because it’s the first one I used (along with RPC and rpcgen), because it is Unixy, and because most of the important pieces of the internet and unix ecosystem were based on it. A little endian superset of this with a JSON style human semi-readable form, and an optional self-description field, and you’ve solved all possible serialization problems which sane people are confronted with. People can then concentrate on writing correct super-XDR extensions to get all their weird corner cases covered, and I will not be grouchy any more.
It also bugs the hell out of me that people idiotically serialize data when they don’t have to (I’m looking at you, Spark jackanapes), but that’s another rant.
Oh yeah, I do like Messagepack; it’s pretty cool.
I came across this video recently. It is a think piece by the Ford motor company and a long dead electronics firm called Philco showing what the future will be like from the perspective of 1967. It’s a nice imaginative vista from a time of great technological optimism: 1967. They were close to accomplishing the moon shot, and the Mach-3 Boeing SST had only recently been announced. From the perspective of a technologist alive in those days, life could have ended up like this. The set of things people worried about technological solutions and conveniences they thought would be cool are also interesting. It is kind of sad comparing this bold imagined future (only 32 years away from when the video was made) to our actually existing shabby 1999 +17y future. It’s 21 minutes, so if you don’t have 21 minutes to watch the whole thing, you can read my comments.
The husband of the house is an astrophysicist (working a remote day job on Mars Colonization no less) with a hobby doing … botany. He’s got a lab at home and is trying to breed a super peach with a protective tangerine skin. This is wildly unrealistic, even if they had thought of genetic engineering back then, and as far as I know, nobody is breeding crazy fruits today, let alone doing so as a hobby. Obviously nobody is colonizing Mars. Still, food and novelty was apparently considered important in 1967, so it is kind of endearing they gave the astrophysicist this kind of hobby. Most astrophysicists I know work 80 hour weeks and have hobbies like looking at youtube videos and grousing about funding levels.
The house of tomorrow has a central computer where all kinds of stuff is stored in its “memory banks.” There is really no reason why people distribute their data all over creation the way they do now; the future from 1967 looked a lot more sane and safe in this regard. Memory banks and computers in this video look a lot like the computers, TVs and radios of 1967. They’re kind of cool looking, like a bit CAMAC crate or IBM mainframe.
The kid (single child to upper middle class parents; good prediction) seems to be homeschooled by teaching machines. This is quite technically feasible these days, but not so many people work at home in our shabby future of 2016 that this is done regularly.
They chat with each other electronically. Their future used a sort of video intercom, which is a lot more interesting than our actual crummy future, where people furiously thumb-type text messages to each other from across the dinner table, rather than video calling from the other room. They also didn’t predict chatroulette.
Dinner is pre-processed and stored in some kind of central silo which microwaves dinner for everybody, based on their nutritional requirements and how fat they’re getting; all done in less than 2 minutes. The upside to our shabby future present is people don’t like icky but futuristic seeming TV dinners as much as they did in the 60s. Our shitty future equivalent, I guess, at least in the Bay Area, we have “services” which deliver food to your house unmade, and you have a bonding experience with your significant other following the directions and making the food. Or we just go to the grocery store like they did in 1967. There are probably apps which claim to track calories for people, but in shitty future now pretty much everyone is disgustingly fat. Oh yeah, in the future of 1967, dishwashers are obsolete; everyone throws their (seemingly perfectly reusable) plates away. Little did they know in 1967, landfills would become a political problem.
Lots of clothes in the 1967 future will be as disposable as the plates and silverware. The ones that you want to keep are ultrasound dry cleaned using a frightening closet which seems quite exposed to the rest of the house, despite shooting visible clouds of noxious chemicals all over the place. People in the 1967 future weren’t as petrified of chemicals as we are now. Frankly their self cleaning closet gives even me the creeps. I don’t even like using moth balls. Hot link to scary cleaning closet here.
In the 1967 future, the Mrs. of the household can buy stuff “online,” which was a pretty good guess. Of course, their “online” is from some kind of live video feed. The idea of a website (or a mouse or keyboard) hadn’t occurred to them yet. And the bank is also accessible through some other kind of computerized console, as is a “home post office” which I guess was a form of email. Though their email system works in cursive in this example. I am guessing that typewriter style keyboards were seen as a specialized skill in those days, and “push button” was seen as more futuristic.
The house is powered by a hydrogen fuel cell for some reason, and “pure water” is a useful byproduct. Maybe in the 1967 future plumbing will be depreciated. In their 1967-vantage future, despite breeding crazy peaches and eating all their food from the microwave-refrigerator food dispensing machine, they’ll get strange undersea fruits from hydro-cultured underwater farms. 1960s futurology was filled with fantasies of growing things under water; science fiction from those days seemed to think we’d all be algae eaters in the future. I was never able to figure that out. I guess humanity obviated this with the green revolution, which was not something which was particularly predictable from those days.
The home gym is fun. It features a medical scanner which scans you while reclining on an Eames style couch and makes exercise suggestions; something that doesn’t exist anywhere, probably never will, despite all the DARPA requests for such a thing. Pretty much the same thing as in Star Trek’s “sick bay.” There’s lots of funny old timey exercise equipment in the gym, some of which has made a recent comeback; exercise clubs, gymnastic rings, chest expander. I don’t think they predicted the come back of such devices: those were probably cutting edge in 1967. Oh yeah, the medical scanner sends data back to the community medical center: HIPPA records apparently don’t apply in 1967 future, as opposed to our present shitty future, because people didn’t think of themselves as living in a sinister oligopoly careening towards totalitarianism as we do now.
In 1967 future, you video call your far away buddy to make travel plans, just like now on skype. But in 1967 future you could pick between a golf course in Monterey and one in Mexico City for a casual afternoon of golf, depending on the weather forecast. Because in those days, it seemed inevitable that supersonic or even hypersonic air travel be cheap and convenient. They had no way of knowing the oil crisis would come, just as they had no way of knowing you’d need to arrive 3 hours early to the airport because of imbecile US foreign policy hubris. Remember you didn’t even need a photo ID to get on a plane until 1999 or so; you could go to the airport with a bundle of cash and fly anywhere you wanted to; just like in 1967. In a later scene in the video, pals from Philippines and Paris show up for a house party, because, again, supersonic (maybe hypersonic) flight is super cheap in the 1967 future.
Hobbies in the future: the lady of the house has a fine arts degree and makes pots at home. I actually know a few people like this, and suspect there were people like this in 1967, but it’s really more of an upper middle class thing than a future thing. It’s arguably more upper middle class now for the missus to work for a non-profit. Video games in 1967 future seemed to be restricted to chess. 1999 shabby future had stuff like Castle Wolfenstein and was legitimately less shitty than the imagined 1967 future. It’s probably better for kids to play computer chess though.
Parties in the 1967 future looked better than modern parties; people dressed stylishly and listened to decent music while having enlightened conversation. This is pretty rare these days, though I suppose people do often have “parties” centered around the TV the way they did.
The 1999 future as envisioned in 1967 seemed like a nice place. Everything is convenient. People spent a lot of time bettering themselves with productive hobbies; making artistic pots and breeding interesting plants when they’re not doing a man’s work sending people to colonize Mars or playing duets with your child on a giant synthesizer. Friendships were cultivated all over the world, and travel was trivial and cheap. People in the 1967 envisioned future were apparently very worried about getting fat; I can only speculate that this was an actual concern of 1967, which is probably why everyone looks so slim in those old timey “people watching the moon shot” photos. I’m not sure what happened to that; perhaps cheap insulin has made people worry about it less. People in 1967 were also very concerned with overpopulation and foodstuffs to feed the teeming masses, which is why food came up so much in the video, and why the future family only had one offspring. While the 1967 envisioned future seemed preternaturally clean and environmentally sound, upper middle class neuroses now a days are more overtly concerned with pollution and environmental issues. I am guessing the household conveniences of disposable dishes, self-cleaning closets and pre-made meals were some technical reflection on the cultural changes between the sexes brewing in the 60s. In 1967 it probably seemed like you could solve these looming cultural upheavals using technology; just give the missus some self-cleaning closets and a machine which does the cooking. I couldn’t help but think that the Housewife of the future seemed a little bored. Honestly the whole family seemed pretty spaced out and lost, but perhaps that’s because plot, characterization and motivation in industrial videos is not always a priority.
They did guess that computers would be important in the home, which was far from obvious at that point. They also guessed that some kind of networked computer system would be routine, which was a very good guess, as computer networks were entirely military up to that point. Oh yeah, and unlike lots of science fiction movies, the screens of the future were flat, rather than CRT based.
It would be interesting to find a modern “home of the future” video by a modern industrial concern; maybe there is one by Microsoft or Apple. I doubt as their future is as interesting and healthy seeming as this future. Perhaps some visionary should attempt this, if only for aspiration purposes.
Putting aside the fumes of smug self-satisfaction which shimmer over the audience, not mentioning the vacuous imbecilities regurgitated by most of the liberal arts talks, not even considering the $6000 seat price for attending this precious wankfest: sometimes, the “science” is often so unbearably bad as to defy parody. What am I talking about? Well, here’s some crank making perpetual motion machine and unified field theory claims at a TEDx conference. His lecture makes absolutely no sense; it is a schizoid word-salad of vaguely virtuous sounding quasi-technical phrases. The audience, gawdblessem, claps:
( Hat tip to Carl Zimmer for this stinker.)
The inchoate chattering is that of the managerial upper middle classes giving each other sloppy wet tongue kisses for being so incredibly aware, creative and wonderful masters of the universe. The subject matter is pure beigeism, from revolutionary musings that conservatives might actually be human beings, to precocious teenagers who fuse out a couple of neutrons in their kitchen, to superhero teachers, to TV noodle theorists, to, well, Jane freaking Fonda. The whole thing is so fey and revolting it makes me puke liverwurst in my mouth. I can almost smell the ripe stew of VC types sporting dockers pantaloons, the gaseous emanations of powerpoint and keynote slides, the toxic outgassing of parked Segways, the vegetarian salami stench of upper level human resources directors.
The great Henry Louis Mencken used to laugh at the pretensions of the Chautauqua lecturer of his day, “belaboring and fermenting the hinds with his Message from the New Jerusalem.” The average Chautauqua lecturer talking to the hayseeds is a regular Marcus Tullius Cicero in comparison to their modern equivalent; certainly he was allowed to express thoughts more complex than ones which fit into an 18 minute presentation. While the yokel Chautauqua audience may have been less educated than the modern equivalent, they were also better dressed, better mannered, and nowhere near as smug. The two audiences appear equivalent in their superstition. The old audience believing in god and country, and the new one believing in sustainable, non-violent, managerial, egalitarian scientism. The old audience was geographically provincial; the new audience is socially provincial, probably regarding an infantryman or a cafeteria worker as a kind of exotic plankton; perhaps something akin to a sea urchin or bivalve.
I’m happy for the TED talks to continue. Why not concentrate all the smug no-goodniks in one place, for when the revolution begins? Imagine how many natural resources we can save by shelling their conference center, rather than hunting them down individually! I’m not happy if people keep enthusing over it and sending me dumb links to it.
With apologies to the master for turning a few of his delicious phrases to my purpose:
” The simple fact is that most of man’s thinking is stupid, pointless, and injurious to him. Of all animals, indeed, he seems the least capable of arriving at accurate judgments in the matters that most desperately affect his welfare. Try to imagine a rat, in the realm of rat ideas, arriving at a notion as violently in contempt of plausibility as the notion, say, of ‘social venture,’ or that of sustainability, or that of social media saving humanity, or that of mental telepathy. Try to think of a congregation of educated rats gravely listening to such disgusting intellectual rubbish as was in the public bulls of Eve Ensler. Man’s natural instinct, in fact, is never toward what is sound and true; it is toward what is specious and false. Let any great nation of modern times be confronted by two conflicting propositions, the one grounded upon the utmost probability and reasonableness and the other upon the most glaring error, and it will almost invariably embrace the latter. It is so in politics, which consists wholly of a succession of unintelligent crazes, many of them so idiotic that they exist only as battle-cries and shibboleths and are not reducible to logical statement at all. It is so in religion and ideology, which, like poetry, is simply a concerted effort to deny the most obvious realities. It is so in nearly every field of thought. The ideas that conquer the race most rapidly and arouse the wildest enthusiasm and are held most tenaciously are precisely the ideas that are most insane. This has been true since the first ‘advanced’ gorilla put on underwear, cultivated a frown and began his first lecture tour in the first chautauqua, and it will be so until the high gods, tired of the farce at last, obliterate the race with one great, final blast of fire, mustard gas and streptcocci.”
It’s true, there are probably thousands of papers this bad, but this one made the papers, so someone had to make fun of them.