Locklin on science

Open problems in Robotics

Posted in brainz, Open problems by Scott Locklin on July 29, 2020

Robotics is one of those things the business funny papers regularly wonder about; it seems like consumer robotics is a revolutionary trillion dollar market which is perpetually  20-years away -more or less like nuclear fusion.

I had contemplated fiddling with robotics in hopes of building something that would do a useful science-fictiony thing, like go fetch me a beer from the refrigerator. Seemed like a nice way of fucking around with math, the machine shop and ending up with something cool and useful to fiddle with.  To do this, my beer fetching robot would have to navigate my potentially cluttered apartment to the refrigerator, open the door, look for the arbitrarily shaped/sized beer bottle amidst the ketchup bottles, jars of herring, broccoli and other such irrelevant objects, move things out of the way, grasp the bottle and return to me. After conversing with a world-renowned expert in autonomous vehicles; a subset of robotics,  I was informed that this isn’t really possible. All the actions I described above are open problems. Sure, you could do some ridiculous workaround that makes it look like autonomous behavior. I could also train a monkey or a dog to do the same thing, or get up and get the damn beer myself.

There really aren’t any lists in open problems in robotics, I am assuming because it would be a depressingly long litany. I figured I would assemble one; one which I assume will be gratuitously incomplete and occasionally wrong, but which makes up for all that by actually existing. Like my list of open problems in physics and astronomy, I could very well be wrong about some of these, or behind the times, and since my expertise consists in google and 5-10 year old conversations with a cool dude between deadlifts, but it seems worth doing.

  1. Motion planning is an actual area of research, with its own journals, schools of thought, experts and sets of open problems. Things like, “how do I get my robot from point A to point B without falling into a canyon, getting stuck, or being able to deal generally with obstacles” are not solved problems. Even things like a model of where the robot is, with respect to the surroundings: totally an open problem. How to know where your manipulator is in space, and how to get it somewhere else; open problem. Obviously beer fetching robots need to do all kinds of motion planning. Any potential solution will be ad-hoc and useless for the general case of, say, fetching a screw from a bin in the machine shop.
  2. Multiaxis singularities -this one blew my mind. Imagine you have a robot arm bolted to the ground. You want to teach the stupid thing to paint a car or something. There are actual singularities possible in the equations of motion; and it is more or less an underconstrained problem. I guess there are workarounds for this at this point, but they all have different tradeoffs. It’s as open a problem as motion planning on a macro scale.
  3. Simultaneous Location and Mapping. SLAM for short. When you enter a room, your brain knows exactly where your body is, and makes a map of the surroundings. Robots have a hard time with this. There are any number of solutions to the problem, but ultimately the most useful one is to make a really good map in advance. Having a vague or topological map or some kind of prior as to the environment: these are all completely different problems which seem like they should have a common solution, but don’t. While there are solutions to some problems available, they’re not general and definitely not turn-key to where there would be a SLAM module you can buy for your robot. I could program my beer robot to know all about my room, but there’s always going to be new obstacles (a pair of shoes, a book) which aren’t in its model. It needs SLAM to deal.
  4. Lost Robot Problem. Related; if I wake up, and my friends moved my bed to another room; we’ll all have a laugh. Most robots won’t know what to do if it loses track of its location. It will need a strategy to deal with this. The strategies are not general. It’s extremely likely I turn on my beer robot in different positions and locations in the room, and it will have to deal with that. Now imagine I put it somewhere else in the apartment building.
  5. Object manipulation and haptic feedback. Hugely not done yet. The human hand is an amazing thing, and robot manipulators are nowhere near being able to manipulate with haptic feedback or even simply manipulate real world objects based on visual recognition. Even something like picking up a stationary object with a simple graspable plane is a huge unsolved problem people publish on all the time. My beer robot could have a special manipulator designed to grasp a specific kind of beer bottle, or a lot of models of shapes of beer bottles, but if I ask the same robot to fetch me a carrot or a jar of mayo, I’m shit out of luck.
  6. Depth estimation. A sort of subset of object manipulation; you’d figure a robot with binocular vision, or even simply the ability to poke at an object and see it move is something pretty simple to do. It’s very much an open problem. Depth estimation is a problem for my beer-fetching robot, even if the beer is in the same place in the refrigerator every time (the robot won’t be, depending on its trajectory).
  7. Position estimation of moving objects. If you can’t know how far away an object is, you’re sure going to have a hard time estimating what a moving object is doing. Lt. Data ain’t gonna be playing baseball any time soon. If my beer robot had a human-looking bottle opener, it would need a technology like this.
  8. Affordance discovery how to predict what an object you interact with will do when you interact with it.  In my example; the robot would need a model for how objects are likely to behave in moving them aside in searching my refrigerator for a beer bottle.
  9. Scene understanding: this one should be obvious. We’re just at the point where image recognition is useful: I drove an Audi on the autobahn which could detect and somewhat adhere to the lines on the highway. I’m pretty sure it eventually would have detected the truck stopped in the middle of the road in front of me, but despite this fairly trivial “you’re going to turn into road pizza” if(object_in_front) {apply_break} level of understanding, it showed no evidence of being capable of this much reasoning. Totally open problem. I’ll point out that the humble housefly has no problem understanding the concept of “shit in front of you; avoid,” making robots and Audi brains vastly inferior to the housefly. Even putting the obvious problem aside; imagine if your robot is tasked with getting me a beer out of the refrigerator and there is a bottle of ketchup obscuring the beer. The robot will be unable to deal. Even with a 3-d model of the concept of beer bottle and the ketchup bottle which is absurdly complex to program the robot with.


several of the above problems illustrated



There’s something called the Moravec paradox which I’ve mentioned in the past.

“it is comparatively easy to make computers exhibit adult level performance on intelligence tests or playing checkers, and difficult or impossible to give them the skills of a one-year-old when it comes to perception and mobility”

Robotics embodies the Moravec paradox. There’s a sort of corollary to this that people who work in the tiny field of “actual AI” (as opposed to ML ding dongs who got above their station) used to know about. This was before the marketing departments of google and other frauds made objective thought about this impossible. The idea is that intelligence and consciousness arose spontaneously out of biological motion control systems.

I think the idea comes from Roger Sperry, but whatever, it used to be widely known and at least somewhat accepted. Those biological motion control systems exist even on a microscopic level; even unicellular creatures like the paramecium, or primitive animals without real nervous systems like the hydra are capable of solving problems that we can’t do even in the general case with the latest NVIDIA supercomputer. While robotics is a noble calling and the roboticists solve devilishly hard problems, animal behavior ought to give a big old hint that they’re not doing it right.



Guys like Rodney Brooks seemed to accept this and built various robots that would learn how to walk using primitive hardware and feedback oriented ideas rather than programmed ideas. There was even a name for this; “Nouvelle AI.” No idea what happened to those ideas; I suppose they were too hard to make progress on, though the early results were impressive looking. Now Dr Brooks has a blog where he opines hilarious things like flying cars and “real soon now” autonomous vehicles are right around the corner.

I’ll go out on a limb and say I think current year Rodney Brooks is wrong about autonomous vehicles, but I think 80s Rodney Brooks was probably on the right path. Maybe it was too hard to go down the correct path: that’s often the way. We all know emergent systems are super important in all manner of phenomena, but we have no mathematics or models to deal with them. So we end up with useless horse shit like GPT-3.

It’s probably the case that, at minimum, a genuine “AI” would need to have a physical form and be capable of interacting with its environment. Many of the proposed algorithmic solutions to the problems listed above are NP-hard problems. To me, this implies that crap involving computers such as we use is wrong. We do approximately solve NP-hard problems in other ways all the time; you can do it with soap bubbles, but the design of the “computer” is vastly different from the von Neumann machine: it’s an analog machine where we don’t care about infinite accuracy.

You can see some of this in various proposed neuromorphic computing models: it’s abundantly obvious that nothing like stochastic gradient descent or contrastive divergence is happening in biological neurons. Spiking models like a liquid state machine are closer to how a primitive nervous system works, and they’re fairly difficult to simulate on Von Neumann hardware (some NPC is about to burble “Church Turing thesis” at me: don’t). I think it likely that many robot open problems could be solved using something more like a simulacrum of a simple nervous system than writing python code in ROS.

But really, all I know about robotics is that it’s pretty difficult.

45 Responses

Subscribe to comments with RSS.

  1. asciilifeform said, on July 29, 2020 at 5:04 pm

    The Von Neumann model of computation is simply crippling, even in “crunching” (e.g. chemical modeling) applications that have nothing to do with mechanical mules. Observe that 1950s rocketry was ~100% driven by analog servo controllers (and without even bothering to e.g. “simulate neuron” — worked great.)

    On the hardware side, the robotic animals are only superficially (i.e. for rube audience) biomimetic — actual animals do not have hinges for joints, and their prime mover is finely-controlled muscle tissue, rather than motors supplying gigantic torque directly at a joint. (But there is ~0 incentive to develop proper electronic muscle, because of 1st paragraph.)

    Not even to mention that “fetch beer from fridge” is not a market that can cover the R&D outlay for anything in this vein. “Bigdog” et al are simply boondoggles, rather than commercial products in the usual sense. They are not designed to work — but to impress the gullible; in the fine tradition of “Eliza”.

    • Scott Locklin said, on July 29, 2020 at 6:21 pm

      I really have no idea what “computer scientists” in current year get up to. Seems like it should be what they’re doing! I guess they’d rather pay theorists to tell them new architectures aren’t worth doing, than pay people to innovate.

      • asciilifeform said, on July 29, 2020 at 7:13 pm

        >…what “computer scientists” in current year get up to…

        They’re… cranking out GBs of LaTeX, pushing out LPUs, killing trees, earning diplomas and tenure, precisely like the “humanities” fluff wankers — but instead being abusers of equations and plots rather than of words. See e.g. the anon commenter’s links — primo examples.

        Meanwhile the only “innovations” involved are in new methods to part the fool from his money. And even there, not much in the way of novelty: the academic “jam tomorrow” racket is essentially the same today as it was during the reign of Reagan. Albeit featuring ever-larger dollar sums and ever-smaller outlay on physical props.

        • Walt said, on August 8, 2020 at 3:26 pm

          I watched “The 30 Million Line Problem” on YouTube and realized even software engineering is regressing significantly and I wasn’t imagining all the bugs and defects in the software I use every day.

          What problems in electrical engineering and computer science are actually worth working on these days? Almost every tech company is focused on software and services. This is where 90% of the investment goes. Most of our semiconductors are now made in Asia and are often pirated, repackaged versions of US-designed devices. On the “defense” side, we’re building Rube Goldberg contraptions that seem to be buggier digital versions of the previous generation’s mostly analog tech. Meanwhile the tech doesn’t help us win any wars.

          Working on amateur radio is fun but doesn’t pay the bills.

          I’ve been driven to the airport in foreign countries by cab drivers with advanced technical degrees. This seems to be where we’re heading. I’m seriously considering other fields.

          • asciilifeform said, on August 8, 2020 at 3:41 pm

            > What problems in electrical engineering and computer science are actually worth working on these days?

            Depends what means “worth working on.” If you only want “get rich”, then “choose your parents carefully” and/or play the SV lotteries. But there are actual open problems worth solving.

            Desktop semiconductor fab, for instance. (Don’t ask how. It’s a 100% open problem, and the traditional methods are a political dead end, they’re inherently centralizing technologies, to just about the same degree as uranium refinement — what with piped elemental fluorine, etc., $B+ entry barrier.)

            Or, the old Holy Grail: room-temp. supercon.

            Or, if you want something easier, that doesn’t require new “martian” physics: below-noise-threshold pirate radio (carrier-less “UWB” minus the bureaucracy.) P2P comms that make triangulation and suppression by spectrum-auctioning state busybodies impossible. Entirely doable with existing off-the-shelf components.

            • Walt said, on August 8, 2020 at 3:58 pm

              By “worth working on” I meant, “Interesting and pays the bills.” I thought this is what I was going to be doing when I got an EE degree but welp.

              Desktop fab is going to be a must to maintain our current level of technology since these big, centralized fabs are hothouse flowers in that they require extreme political stability to operate and we’re headed in the direction of SA which is now regressing technologically. I’m skeptical about Asia to keep producing what it never invented. Still, how do you make a clean room at home and keep from blowing yourself to kingdom come?

              Carrierless UWB sounds fascinating but I wonder about the atmospheric effects on propagation. I suppose by “UWB,” you mean, “stays below low UHF?” I’ve struggled to create new device layouts in the free tools like KiCAD. I’ve wanted to do a 3-layer FRP tranciever board. Still, this probably isn’t going to sell well.

              I may need to get into something altogether that is home-based. The least satisfying aspect of any job is bureaucracy, and tech has that in spades these days.

              • asciilifeform said, on August 8, 2020 at 5:11 pm

                Desktop fab is a must if you want a reproducible, “fits-in-head” comp where the OS is 3K demonstrably-correct lines rather than 30 million of buffer-overflowing C garbage. And if you want CPU w/out NSAware, etc. And yes, if you want a post-empire computing that doesn’t consist of x86 excavated from junkyards for 500 years.

                > atmospheric effects on propagation

                Well-documented for 1-30MHz. And there’s no particular reason you can’t UWB the entire ionosphere-bounceable spectrum. For short-range comms, however, there is no particular reason to concern yourself with the ionosphere.

                > I’ve struggled to create new device layouts in the free tools like KiCAD

                AFAIK all of the opensores CAD tooling still consists of barely-usable “hair shirts”. And folks who actually want to get something done — rely on warez.

                > this probably isn’t going to sell well

                If you’re counting on “solve open problem of the century, get fabulously rich” you’ve some surprises coming. The kind E. H. Armstrong, P. Farnsworth, J. E. Lilienfeld got.

                • Raul Miller said, on August 8, 2020 at 5:37 pm

                  Desktop fab would presumably start with a vacuum chamber, and would probably use ion beam epitaxy — so basically in the same general ballpark as a scanning tunnelling electron microscope — just a bit more heavy handed, since you’re more interested in depositing material than picking up signals. If you’re working with an outfit like spacex, you’ve also got some interesting potential alternatives for vacuum chambers (which suggests that someone willing to spend a decade or two building out techniques might have a lucrative career ahead of them). Spendy stuff if you don’t have the right contacts (and of course any initial work would have to be low-end, because better quality work requires a solid base of experience).

                  You could also work with fitting components together, of course, classic electronics builds with solder and/or sockets or whatever.

                  But anything you do on an individual scale is going to be limited in scope, also. And, a lot of the large scale stupidities we’re seeing are things which have been creeping up on us for decades. If you think purely in terms of profit, rather than in terms of maintaining and improving existing systems, or resurrecting older systems (the factory type things, not the software side of things), I think you’re going to be out of luck.

                  That said… have I mentioned greenarrays here yet? In terms of media savvy, level it’s boring and clunky looking stuff. But if you can get past that, it has some rather interesting possibilities.

                  Long story short, though — we’ve got so much “lost technology” that in our current environment that’s probably where we will be getting most of our “gains”.


                  “Don’t worry about people stealing your ideas. If your ideas are any good, you’ll have to ram them down people’s throats.” — attributed to Howard H. Aiken

                  • asciilifeform said, on August 8, 2020 at 5:43 pm

                    > If you’re working with an outfit like spacex, you’ve also got some interesting potential alternatives for vacuum chambers

                    If you work for SpaceX, you can afford to simply fork over the 100K$ or so for a conventional fab run at e.g. TMSC.

                    IMHO it is foolish to expect “garage experimenter”-empowering innovation from a golden toilet USG contractor, for that matter.

                    > But anything you do on an individual scale is going to be limited in scope, also.

                    Keep in mind that, for instance, the first electron microscope in USSR was in fact built in besieged Leningrad. By half-starved graduate students, and out of materials that you probably wouldn’t pick up if you saw them in a dumpster.

                    Many if not most of the limits suffered by current-day practitioners are really in their heads, rather than bank accounts.

                    > greenarrays

                    Is it still in business? Their product seems to have vanished from dealer catalogues, last I knew.

                    • Raul Miller said, on August 8, 2020 at 8:14 pm

                      The point I was hinting at in the context of SpaceX was that space itself might serve as a high quality fabrication vacuum mechanism, if some of the practical issues can be resolved, and that given the cost constraints of space launches and the need for maintenance of systems on the scale of what they’ve been working towards, that this is a likely area of future activity.

                      As for greenarrays, have you tried http://www.greenarraychips.com/home/contact/index.html ?

                    • asciilifeform said, on August 8, 2020 at 8:32 pm

                      IMHO it is more than a little strange to consider “do it in orbit” as approach to the “cheap individual semiconductor fab” problem. Considering that anything taken to orbit is “bought for own weight in pure gold” and this is likely to remain the case indefinitely while such travel still relies on chemical rocketry.

                      Not to mention that drawing a high vacuum in your kitchen isn’t *that* expensive, the required gear costs likely less than the computer you are posting from. Vacuum gear is neither 1st nor 10th on the list of cost obstacles to amateur CPU fabrication.

                    • Raul Miller said, on August 9, 2020 at 1:35 am

                      SpaceX is a little strange, sure. But you should take another look at what you wrote, to distinguish between costs on the ground and costs in space.

                      If Musk manages to approach his target of eventually sending a million people to mars, there’s going to be a lot of activity. And, sure, maybe using standardized parts it will be cheap enough to carry enough spares for all of them.

                      But space is a hostile environment for electronics — moreso than in an atmosphere because of radiation issues. So as a general rule the components used in space are significantly “retro” so that they have enough bulk to resist random changes. But, also, scaling things up, this means we can expect some significant problems, which will need repairs.

                      Meanwhile, there should not need to be much mass to an epitaxy system (and hopefully even a substrate cleaning system) once you get rid of the mechanisms needed to create a high vacuum in an atmosphere.

                    • asciilifeform said, on August 9, 2020 at 1:44 am

                      > If Musk manages to approach his target of eventually sending a million people to mars…

                      Do you seriously believe this?!

                    • Raul Miller said, on August 9, 2020 at 4:51 am

                      Please be more specific. Unbridled scepticism suggests too many possible issues for meaningful discussion.

                      What are you asking about?

                    • Walt said, on August 8, 2020 at 8:53 pm

                      Russians do a lot more with a lot less than we do. This gives me some hope that tinkerers can accomplish something innovative, but how do you make it pay the monthly bills? How do you see outside the constraints imposed by your current situation? Everyone believes you need a lot of money and a high tech fab for R&D.

                    • asciilifeform said, on August 9, 2020 at 12:11 am

                      The trick is to not think yourself into the corner where “I need $B+”. Everything else necessarily follows from that.

                      Re: modern-day Russia — as I understand, thoroughly-sad, and culturally moribund, no less than USA — albeit with top-dollar PR, to sell gullible folks on being “cultural alternative”.

                • Walt said, on August 8, 2020 at 8:57 pm

                  Are people using apertures with fractions of a wavelength for 1-30 MHz? Generally people are using big apertures for HF though I have heard of people Dxing on partial-wavelength apertures.

                  “rely on warez?” do you mean a tool you have to buy?

                  I’m not trying to get fabulously rich, just earn a living.

                  • asciilifeform said, on August 9, 2020 at 12:04 am

                    > apertures

                    For true “UWB”, you’re stuck with something like a folded dipole, i.e. entirely untuned antenna. (If receive-only, can use e.g. Beverage antenna.)

                    For clarity: I was speaking of genuinely carrier-less, spectrum-carpeting, and — ideally — below-noise-floor UWB. Where the only thing you want from the antenna is a maximally-flat spectral response, because you are in fact throwing something close to square pulses through it.

                    There will be no stranger-to-stranger “DXing” on pirate UWB — definitionally it ought to be indistinguishable from thermal noise for anyone who lacks the cipher key.

                    > a tool you have to buy?

                    Buy (if you have $500+/mo. to pay eternally, AFAIK all the CAD vendors have moved to “subscription” pay-per-click models) if idiot. If smart, “buy”.

                    > I’m not trying to get fabulously rich, just earn a living.

                    Presumably you are already earning one somehow?

                    • Walt said, on August 9, 2020 at 3:12 am

                      I am but I see where it’s all headed: ideological or identitarian requirements for hiring such as affirming BLM or being a PoC, reciting their version of the 14 Words, etc. Less and less focus is on the technology or the customers. More and more focus is on nonsense like SaaS, services and Wokeness. The Great Awokening is upon us. The age of innovation is over. I just can’t see spending further decades in an industry that’s not heading anywhere but off a cliff.

                    • asciilifeform said, on August 9, 2020 at 3:38 am

                      > I am but I see where it’s all headed: ideological or identitarian requirements for hiring such as affirming BLM or being a PoC, reciting their version of the 14 Words

                      Aah but I wasn’t talking about corporate industry and its garbage offerings.

                      But rather of things that you can build in your kitchen and no one can stop you. Just like “all of Washington’s horses and all of Washington’s men” can’t stop Iraqi teenagers from building IEDs.

                    • Walt said, on August 10, 2020 at 3:55 pm

                      Do you have a link on HF UWB? I couldn’t find much about it. I have heard of HFLink or ALE but anyone can listen to this

                    • asciilifeform said, on August 10, 2020 at 4:01 pm

                      AFAIK there is no public “cookbook” specifically on HF UWB — it’s ipso facto “pirate” in any major country, as there are 1930s-era treaties mandating spectrum fascism, at least on paper, everywhere.

                      However, all you have to do is to solve the Gaussian monocycle equation for the desired band coverage, and design the appropriate circuits to generate/receive the pulses, exactly as described in the Officially-blessed UHF-UWB literature.

              • asciilifeform said, on August 8, 2020 at 5:38 pm

                Re: clean-room — obviously you would want an entirely new process that doesn’t rely on clean rooms, high vacuum, particle beams, gaseous halogens, etc.; and something rather more like B&W photography in terms of tech level. The exact “how” is more or less an unknown-unknown, there is AFAIK zero publicly-visible progress in this area.

                I suspect the pill — if and when it is found — will consist of a repurposed tool from a wholly-unrelated field, e.g. electrophoretic gel “blots” with organic semiconductors instead of pigment.

          • Scott Locklin said, on August 8, 2020 at 4:40 pm

            Decentralized radio is a good idea.

            Decentralized systems is easily the hottest thing right now.


            Lots of gentleman amateurs working in the space, making important contributions, and as most of the cryptographers are serious people, as opposed to modern glad handling EEOC wankers such as in physics, the academic overlap is good and constructive too. In particular the applications of stuff like ZK-SNARK and ZK-STARK ideas haven’t played out fully yet and are very powerful.

            Most of what you’re going to see on the subject is going to be sociopathic grifters gabbling on about bitcoin, but it’s a real innovation, taking place in a time when social trust is plummeting. Not limited to currency applications; I consider bittorrent and stuff like Jules Wattenberg to be close relatives.


            If you want to do EE type stuff, building a hardware “VM” with verifiable calculations (aka zkvm type things) or secure elements is stuff that needs doing. I think asciilifeform built one at some point.

            • asciilifeform said, on August 8, 2020 at 4:53 pm

              > ZK-SNARK

              Do you know of any ZKP schemes that don’t rely on trusting some joker to have “destroyed, really, I promise!” the initialization register?

              > sociopathic grifters gabbling on

              So far I’ve only seen ZKPism pushed by specifically these people. (And also by certain USG contractors; specifically one where I personally slaved for a time, and was assigned to a homomorphism “golden toilet” project; and eventually quit in disgust when realized that the entire field is based on fraudulent pseudomathematics.)

              > secure elements is stuff that needs doing. I think asciilifeform built one at some point.

              I built a plain old numerics stack where e.g. +/-/*, modular exponentiation, etc. in constant time and electrical current. No magical ZKPism involved, nor any unproven hardness conjectures.

              • Scott Locklin said, on August 8, 2020 at 6:29 pm

                Eli Ben Sasson’s ZK-STARK doesn’t need trusted setup for the reference string. Neither does most of the latest generation of Snark gizmoes. Of course none of them produce proofs as lean as Groth-16 either, but you could always use Groth with your own ceremony and burn a piece yourself.

                It’s very possible these things all fall apart some day. Starks least likely; unless you believe common hashing functions are pwned it’s pretty legit -no weird elliptic curves.

                I suppose it could all fall apart, but in the meanwhile, not having to trust your broker is a nice idea, and there are definitely use cases not thought of yet.

                • asciilifeform said, on August 8, 2020 at 6:46 pm

                  If I’m required to “trust the broker” to any degree, I may as well use PayPal et al.

                  IMHO all crypto which requires anything resembling “burn ceremonies” is intrinsically fraudulent. I trust shitcoin authors exactly as far as I can throw them.

                  Nor do I have much trust in any cipher system to any extent more complicated than RSA. IMHO even the use of ECDSA in traditional Bitcoin is a major weakness. “Quantumism” is simply USG propaganda to get folks off RSA (observe that NSA itself publicly retracted its “use ECC in gov. systems” decree a few years ago.)

                  All of these various “hot, new, modern” items are in my mind simply elaborate schemes to defraud Bitcoin holders of their stashes. (Either directly, or by conning them into trading their supply-capped coins for a shitcoin with unknown supply cap; or via bizarre transfer logic, e.g. the infamous DAO; and in many other ways.)

                  • Walt said, on August 8, 2020 at 9:04 pm

                    The great think about radio is that you can control how much power you radiate and where you radiate it. The information transmitted can only be received by someone close-enough with enough sensitivity. If you’re trying to create a secure neighborhood radio net, just use the minimal power. This is always the first consideration secure considerations, especially since the USA is already a highly non-permissive environment when it comes to encrypted radio communications. It’s pretty depressing.

                    All other communications nowadays including cellular is converted to an internet packet and who knows who receives that packet? They can be duplicated an routed infinitely.

                    • asciilifeform said, on August 9, 2020 at 12:07 am

                      The promise of pirate HF UWB (vs. something like neighbourhood UHF) is potentially to break out of the state monopoly on long-haul comms. Currently ~entire planet is at the mercy of half a dozen state-controlled fiber conglomerates.

                    • Raul Miller said, on August 9, 2020 at 1:16 am

                      Have you looked at what computers radiate on the rf spectrum?

                      Do you know anyone who has?

                      Yes, the FCC has relevant regulations, but computers nowadays are mostly imports and the fcc stickers are easily forged or misused. Some interesting war stories here, but no neighborhood activities that I am aware of.

  2. Anonymous said, on July 29, 2020 at 5:58 pm

    One area that’s seen very good progress is perception (depth estimation, etc), driven in part by deep learning. Most autonomous vehicle projects use a combination of laser range finding (lidar scanning over 1M a second) with cameras & radar that all complement each other. LIDAR gives accurate depth returns, but suffers at detecting objects at long range (>200m) when the point cloud returned is sparse, and gets spooked by fog, snow, rain, folliage, etc. High res cameras can detect cars, pedestrians, etc at long range, but aren’t as accurate at depth perception. Radar can see through the fog & rain, but has other issues.
    One major research direction is fusing all of this together inside of neural networks (instead of generating separate outputs for each sensor modality and then fusing with a tracking algorithm / optimization method), which gets better results than having separate pipelines and fusing after [1].
    The going strategy in some of these AV projects is to use larger neural net backbones [2] coupled with more data, assuming it will continue to scale [3] (what I imagine you’d take issue with).
    Now there are some obvious problems, like DL lack of common sense [4]. Secondly, this uses a tonne of compute. There’s a reason Waymo is using minivans – current AV test vehicles are approaching a petaflop of compute, faster than any existing super computer in 2005. The power required is enough that it would potentially handicap the range (presuming they have an electric drivetrain).
    The other issue is hand engineering what data is transferred from the detection pipeline to the motion planning pipeline. In addition to location & velocity, now planners are starting to propagate turn signal information, but this requires reworking the entire pipeline. Imaginable even more things will need to be propagated, making this process very hand-engineered and potentially infeasible. However, the alternate to this is terrifying – end to end deep learned models that go from visual inputs to control outputs. There are some startups doing this [5], but I wonder how a fully black box pipeline would get regulatory sign off. The founder & CEO is a recent Cambridge PhD grad who at least seems humble about the limits of deep learning [6] and willing to address the job displacement issues [7], but we’ll see how their approach turns out.
    I really doubt an autonomous vehicle that works at all latitudes and in all conditions will be feasible until (if ever) a general AI is developed to handle the long tail of events. Even proving that an AV is safer than a human driver may take on the order of ~100M miles, after all the R&D work is complete [8].

    Formerly state of the art paper on fusing LIDAR with camera returns by Uber ATGs chief scientist. She got her PhD in autonomous robotics in 2006 & her earlier papers are GPs and other non deep learning techniques, so I think DL actually works better for this application.
    Latest SOTA object detector from Google

    Click to access 1911.09070.pdf


    Click to access 1712.00409.pdf

    Here’s the Waymo guy talking about how a bunch of kids who stole a stop sign & were carrying it on bikes and how it caused problems for their car: https://youtu.be/Q0nGo2-y0xY?t=355

    • Scott Locklin said, on July 29, 2020 at 6:08 pm

      Thanks for a meaty and useful comment. I figure Lidar makes depth perception a solved problem in a sort of “cheating” sense (aka my beer robot would have eyes but no lidar). But … mapping the Lidar identified objects to the DL identified objects to whatever ontology or internal map seems like an open problem.

      I remember the last time I was looking at the SBIR funny papers in 2015 or so, data fusion was still a huge problem for the F35. I assume they have some problems solved well enough to lob missiles at bad guys, but that’s a lot different than the beer robot rooting around in my refrigerator.

  3. Igor Bukanov said, on July 29, 2020 at 8:17 pm

    A few days ago out of curiosity I tried Tesla 3 autopilot on a mountain road in Norway. I switched it off after few minutes.

    The first problem was that it tried to stick to the middle of the lane even when there are many turns around a mountain resulting in nausea feeling. Any sensible driver will smooth that by driving towards the edges of the lane. Then at one point the line turned into two to allow for overtaking. The car “perceived” that only few meters before the marking separating the new lanes appeared in the middle of the old lane and then rather violently moved to the left, not right as one is supposed to do. That was enough.

    On the other hand the cruise control with radar tracking of the car in front is fantastic. It literally sticks to the car and brakes properly and smoothly if the car in front slows down. But then I suspect that it is just solving physical equations, not machine learning.

  4. Raul Miller said, on July 29, 2020 at 8:24 pm

    Yes, if I were inclined to push this field forwards, I would focus on the “cyborg” or “human enhancement” side of things rather than the “completely autonomous” side of things.

    The issues that could be tackled include:

    (*) Gathering additional information about the environment (I remember a guy using ultrasound to locate pipes, for example),

    (*) Dealing safely with unpleasant or maybe even dangerous things,

    (*) Using leverage or other mechanisms to move heavy things.

    All of these would involve tradeoffs, and (especially in the early stages), frustrations. Some approaches would also look completely dorky and/or invite other criticisms from instant experts.

    There might also be some use for “learning algorithms” (neural nets, genetic search, or maybe even just regular searching and/or sorting and/or statistics — the stuff that we hear the “AI” label get applied to), but that’s only going to take you so far.

    • Rickey said, on July 30, 2020 at 2:34 am

      That is already being done with aerial drones such as Reaper, remotely operated underwater vehicles (ROV’s) and planetary rovers. Even forklifts apply if you want to get really basic. It seems the best approach right now is to extend human reach and perception so a person can act at a distance through extended control without having to go into harms way. Money would be better spent developing better interface and communications systems. Maybe I am missing something, but fully autonomous robots seem to be a solution looking for a problem.

  5. pindash91 said, on July 29, 2020 at 11:59 pm

    I know of one person who is doing exactly this and building artificial reflexes, check out. He came to this conclusion by realizing that the brain is not a computer but a giant gland attached to reflexes. https://jlettvin.github.io/gaze/gaze.html

  6. Bruce said, on July 30, 2020 at 7:06 am

    A while back, I tried an echo state network to solve a difficult prediction problem. It worked well enough to drive home that gradient descent isn’t really needed in neural network learning. I suspect the filtered randomness of the ESN could play a role in biological networks.

    On a different note, the old Hebbian notion that biological neurons which fire together wire together seems to be making a minor comeback. 90 year old Bernard Widrow, who worked on early neural nets in the 50s and 60s, has published quite a few recent papers on his Hebbian-LMS method, which does seem to be successful at unsupervised clustering.

    • Scott Locklin said, on July 30, 2020 at 1:35 pm

      I had some interesting results from ESNs as well. At some point I’d like to fool around with LSM simulators as well.

      I hadn’t heard of Hebbian-LMS; that’s pretty neat.

      I figure fooling around with the latest DL atrocity in Keras is a lot less interesting than going over these old ideas that we have more computational power to throw at. Reservoir computing is pretty weird that it does anything at all; doesn’t seem to make anyone curious.

    • Raul Miller said, on July 31, 2020 at 7:21 pm

      When you say “gradient descent isn’t really needed” — do you mean “not needed for some basic functionality” or do you mean “did not measurably improve X” (where X is some specific quality or qualities, such as learning rate or degree of final convergence, etc)?

      (I am a big fan of minimalism, but I am also acutely aware of its limitations, so I want to understand…)

      • Scott Locklin said, on August 1, 2020 at 3:37 pm

        I think he means (or at least I agree with the statement) something like “neural nets do interesting things without gradient descent approaches.” An ESN for example is a random matrix with a linear regression output layer (matrix inverse), and it does amazing things.

        • Raul Miller said, on August 2, 2020 at 11:02 pm

          Gradient descent is about training the network. So I am going to guess that you are saying that ESN does not train the random matrix but instead it just keeps those initial random values? And that the “training” consists of building that output layer for the random matrix?

          • Scott Locklin said, on August 3, 2020 at 6:33 pm

            Correct. You just fit the output layer using linear regression. You can fit it with something more complicated, but why bother; it doesn’t help.

            I have a (very shitty, probably broken) ESN written in J. I’ll try to write a correct one and talk about it some time; maybe make a “lab” for people to fool with.

            Or if you want to write one, since your tacit code is so sweet:

  7. thenuttynetter said, on August 19, 2020 at 10:42 pm

    “You can see some of this in various proposed neuromorphic computing models: it’s abundantly obvious that nothing like stochastic gradient descent or contrastive divergence is happening in biological neurons. Spiking models like a liquid state machine are closer to how a primitive nervous system works, and they’re fairly difficult to simulate on Von Neumann hardware (some NPC is about to burble “Church Turing thesis” at me: don’t). I think it likely that many robot open problems could be solved using something more like a simulacrum of a simple nervous system than writing python code in ROS.”

    Can you clarify, functionally, what you think is promising about spiking neurons or liquid state machines? In what ways do you think it could be better than today’s neural networks?

    • Scott Locklin said, on August 19, 2020 at 11:52 pm

      Well for one thing, biological brains seem to work that way. For another thing, as I very clearly stated: there aren’t so many people looking there, so it’s more likely there’s juice there than in the latest dweeb learning atrocity where every jackass with a video card has been looking.

  8. […] Open Problems in Robotics 417 by haltingproblem | 218 comments . […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: