Locklin on science

Torch7 notes

To be updated as I go along:

My  difficulties with torch and lua might be helpful to someone else in the future…. Torch is pretty cool, when it works. I can’t imagine having to deploy it in an industrial setting at my present state of Torchitude. The deploy process seems too insane at present. Still, anything even tangentially associated with les phr0gs  is worth figuring out, as I am still a big fan of Lush.

One source of confusion: they changed a lot recently, and old documentation doesn’t help. One can still pull this from github, build it and it will work. Unfortunately, it is an old and depreciated version of torch7 (though Leon Bottou seems to be updating this version regularly). This is the one that can still be invoked with commands like “torch” and “torch-rocks”  which is what exists in most of the documentation. For example, everything mentioned here seems to only apply to the old torch. Also, the build instructions for torch-svm don’t work,  because it assumes torch-pkg. The manual and index are not where the nyu webpage says they are. Manuals and index are pretty important. The old torch package gave you these with the qttorch IDE. It is quite frustrating as a n00b in an ecosystem where virtually all the documentation, right down to what the executable is called, is broken. They should have called the present version Torch8 or something. Or waited to change everything after the migration to Torch9. I’m sure Torch initiates don’t have a problem if they can walk down the hall and talk to 20 guys using the tool, but reading a year’s worth of torch7 google group postings didn’t help much.

For screwing around with scripts and learning things, I’d actually recommend using this, with “torch -ide” to start up Leon Bottou’s cool little QT based ide. If you can’t get started on your machine, perhaps run the first “easy install” script listed on the main website to get the dependencies. The merely curious should stop here. People dealing with “official torch” should read on.


 

Mainline Torch

At present, using the recommended easy install is not easy.  One must remove all existing versions of lua it seems, to avoid conflicts. NBD, but previous efforts hid things in moonrocks, /usr/local/lib/lua /usr/local/bin/lua, my home directory, and a few other places I am probably forgetting.

Even after cleaning everything, the ezbuild doesn’t work on any of my ubuntu 12.04 machines, and dies with the unhelpful:

 

 Failed loading manifest for /usr/local/lib/luarocks/rocks: cannot open /usr/local/lib/luarocks/rocks/manifest: No such file or directory — corrupted local rocks tree? Cannot install new version.
Error. Exiting.

sudo lua-admin make-manifest

will fix this.

The way this easy install thing works, it downloads and builds their version of luajit, which it uses to build everything else. For some reason it doesn’t think to create this manifest file, which I guess is needed by everything else to add things to luarocks.

Plots and native IDE:

Next, when one tries to use qttorch, this can’t be found. It gets built, and one can see it, but…

“module ‘qt’ not found:No LuaRocks module found for qt”

and

module ‘qtlua’ not found:No LuaRocks module found for qtlua

qttorch, which worked in the previous “torch-distro” version is pretty good stuff. Lame.

Well, it turns out that one can run qttorch, but you have to know that you need to start it from qlua. So, qlua -qttorch.

This doesn’t resemble the old qttorch, which came with a nice IDE (I guess qtgui in the old ‘package’ thing), but it works for plots.

gfx.go (aka gfx.js when you install it; it only becomes .go when you load it) – Nodejs seems to be broken on Ubuntu 12.04.  This seems to help for installing NPM and get through the rest of the install. It doesn’t work with the old torch-rocks install.

Emacs:

To use qtlua with emacs, change the defaults for lua-mode.el  to the following values. For gfx.go, change “qlua” to “th” and get rid of the “-lqttorch” since it won’t work. Emacs sometimes makes the colors display funny if you start the repl the emacs way. It stops happening if you fire up a th session in a shell first. Don’t ask me: emacs mysteries. It also works fine from a shell.
(defcustom lua-default-application “qlua”
….
(defcustom lua-default-command-switches (list “-lqttorch” “-lenv” “-i”)

Emacs tab complete:

Downside to emacs: autocomplete doesn’t work. You can make it work with the old *term* trick, but that’s pretty much useless. This is a general problem with emacs, from time immemorial. There are various workarounds for getting it to work with bash and (mostly unsuccessfully) ipython.

 Zerobrane:

I guess this is the alternative to emacs. Get it here. It’s huge, as it contains all kinds of binaries for different platforms. Run it with /path/to/zbs-torch/zbstudio.sh (set up an alias of course).

Autocomplete doesn’t work here either, and while it has some standard IDE features, stuff like closing the stack trace window doesn’t work in the usual way, and screws up the viewing of the file of interest. Not acceptable.

Getting help:

The “ezbuild” doesn’t seem to come with the help functionality, which confused me for a while. When Lua can’t find something, it thinks it is a global variable: awesome. Apparently installing env (sudo luarocks install env), and loading it will make things all better. Some of the functions are undocumented, but at least you can get at what is there.

Loading data:

Some people experience difficulties with csvigo for loading larger data files. Torch7, like Lush, has a sort of “backdoor native format” for saving and loading arrays. I wrote a simple script for turning a simple (double precision, no header, no extra carriage returns) CSV into this Torch7  format. You can find it at this gist. I’m sure it will work on ints as well if you modify it a bit (doubles to longs/shorts/whatever), but it will never work on categorical or mixed data.

Other packages to check out using luarocks:

Stuff is now installed using “luarocks” rather than “torchrocks.”

nnx  an extension to the neural net package.

unsup -unsupervised learning

nngraph

manifold -manifold learning

fex -image processing (SIFT & friends)

fftw3 -good old fftw for the FFT

csvigo -some of the demos use an older package called csv; use this instead, and replace all the csv. with csvigo.

Packages which don’t load with luarocks, but I wish did (they may build from sources):

Building from sources: “sudo luarocks make” in the cloned githubs generally works, but because there is no consistency between these tools, sometimes you need to use cmake/make, or just run from the folder like matlab.

torch-svm doesn’t build. Instructions to build here. Works using “old torch.”

rbm_toolbox “Execute your scripts from the repository folder” -untested

spaghetti

 

Resources:

Tutorial here, of unknown applicability to the most recent version of Torch7.

Cheatsheet here

Demos here..


What is Torch7?

Torch7 is basically a subset of Lush, written in Lua. The “tensor” system is virtually identical to the -IDX- system for array processing in Lush. Many of the idioms used are identical, and I’m pretty sure a good fraction of the C code is the same. Presumably this made translating Lenet and friends simple. From an engineering and professional standpoint, I get why they did this. If you’re a machine learning researcher, you don’t want to be mucking about in the weeds playing with home made compiler internals. I stopped using Lush, as I didn’t feel like writing a lot of C++ bindings and dealing with the problems binutils has in accessing them. LuaJIT provides easy ways of doing this. That, a large community who maintains the LuaJIT compiler, and who presumably did a few useful things like link to CUDA (the reason why I am using it) made it a good, practical engineering choice. The embedability of Lua (push things to a stack, call functions, basically) also makes it an attractive platform: I may try embedding to J or R at some point. You do get the Lush like experience of having access to compiler and interactive environment, though the repl is much more primitive. A big advantage (and horror) of Torch7 over Lush is its very weak typing. Lush had to compile and use separate functions to deal with Floats, for example. Torch7 seems to not notice.

The “torch-rocks/luarocks” package management system is meant to be easy to use, but it leaves something to be desired at the present state of code discipline. It seems old packages such as csv that are being replaced by things named with lots of dashes (lua—csv for csv for “New Torch7”). One can easily screw up an install of Torch by installing a bad package using *rocks: I have done so many times now. I prefer to use the cmake scripts. I’m pretty sure luarocks depends on them anyway.

Downsides: Lua ain’t Lush. Lua is much simpler and more primitive: the default of making all variables global rather demonstrates that it isn’t much of a systems programming language. Still, it works, and people have been using Lua for many impressive things for two decades now. One could think of it as intermediate, human readable code. Maybe some day some kind soul will make something like Lush target Lua code for later compilation.

3 Responses

Subscribe to comments with RSS.

  1. smthchntla said, on August 18, 2014 at 12:54 pm

    just came across your notes. if you think any part of your notes would be helpful for others, considering adding condensed points to the Cheatsheet (it’s an open wiki).

    > It is quite frustrating as a n00b in an ecosystem where virtually all the documentation, right down to what the executable is called, is broken.

    This is completely unacceptable of course, and this is why I started the Cheatsheet (after months of frustration where noobs dont really know what to do, where to go etc.).

    w.r.t. the IDE stuff, when you run a script and encounter an error, you see a stack trace with line numbers of your lua script that broke things, this is usually enough for me. To debug more complicated stuff, I sometimes use zerobranestudio (zbs-torch is now only needed for qlua, the original zerobranestudio works for anything without qt).

    > fftw3 -good old fftw for the FFT

    You might then also like torch-signal (wraps fftw3 nicely and painlessly) http://soumith.ch/torch-signal/signal/

    • Scott Locklin said, on August 18, 2014 at 9:05 pm

      I was hoping nobody would notice this until I had it all figured out and working to my satisfaction. 😉
      Thanks for having a sense of humor about all my grousing.

      Your signal package looks very helpful, thanks.

      FWIIW, I would really like to add some approximate matrix techniques to wherever is appropriate, as I have a strong interest in these right now. These may help improve performance on neural net models with big matrices. I am not familiar enough with the NN literature to know if this has been thought of by anybody important, but I can see that the idea hasn’t made it to Torch7 at least. Since svd lives in unsup, should I work on that repo?

      Future ambition: embedding a good collection of Torch7 parts into another language (J or R). I suspect these languages might be easier to use for the data cleaning stage of things when used in production.

  2. asterix said, on March 13, 2015 at 9:31 pm

    Thanks for the information… Really useful, specially the lua-mode recommendation for visualizing th properly… Did you test the emacs debug mode> mobdebug ??… I would like to read if you have comments about that… Im trying to make it work but it has been useless so far…


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: