Skip to content

Altering OpenBlas Threads

July 26, 2012

It’s been some time since I posted anything. There are several strands in the notes, but this is the first issue to be resolved to completion. A while back I said I was having trouble changing the number of OpenBlas threads from within an R session. Before a session, I can use “export OPENBLAS_NUM_THREADS=x’ but from within a session, approaches such as ‘Sys.setenv(‘OPENBLAS_NUM_THREADS’=y) had no effect. I posted a question about this on r-sig-hpc but I did not receive a definitive answer. This kind of control is quite important – since sometimes implicit BLAS can slow down operations, it is desirable to have on line control over the number of threads to be evoked for certain functions.

I returned to this question recently, having noticed that Xianyi and co at OpenBlas seem to have made improvements for the SandyBridge processors. I reinstalled OpenBlas (note: this discussion pertains to a source installation of OpenBlas with their recommended settings and installation directories etc). On foot of this I reattempted to use their set_openblas_num_threads(int) function and, with some help from Xianyi, got it working in C++ with armadillo, and then created a wrapper for it that is callable from R.

There is a second, possibly surprising improvement, provided by set_openblas_num_threads(int). When I use “export OPENBLAS_NUM_THREADS=x”, implicit parallelization employs 4 threads, for all x >= 4, on my 4 dual-core system. However with set_openblas_num_threads, I can utilize all 8. I am not sure why this is the case, but this seems to be a useful side consequence of the wrapper function.

I have made a mini-package of the function (1 R file, 1 .c file, etc.) and also post below an inline version which can be called from the R REPL. Note that this package works for linux/ubuntu only – the openblas_set_num_threads(int) function for Windows OpenBlas is currently only a dummy, according to their site, but I assume a full function might be forthcoming. Thus, while my package is currently extremely small, any such additions to OPENBLAS’ capacities will be accommodated in due course. Also, further features may be added if a need arises. Please tell me if you have any ideas. Note the extern declaration below may not be strictly necessary – it gives an implicit declaration warning on my machine, but seems to work fine.

require(inline)
openblas.set.num.threads <- cfunction( signature(ipt="integer"),
      body = 'openblas_set_num_threads(*ipt);',
      otherdefs = c ('extern void openblas_set_num_threads(int);'),
      libargs = c ('-L/opt/openblas/lib -lopenblas'),
      language = "C",
      convention = ".C"
      )

As far as I can make out thus far, the function alters the number of threads for that session alone and thus e.g., if 2 sessions of R are opened independently from different terminals, then they each retain and maintain their own thread settings. This leads immediately to the question as to how these settings work with parallelization. It seems to be the case that makePSOCKCluster() and makeForkCluster() differ here, in that a ForkCluster inherits its parent’s variables, and hence, if openblas.set.num.threads(x) is set before the cluster is created, then the cluster’s cores inherit the x value and are effected by the new settings. (If however openblas.num.threads is set after a cluster is created then this seems to have no effect on the cluster, as one would expect.) How to use this feature to good effect is another matter however, since on preliminary cases, this generally appears to degrade performance, but more on this to follow.

The mini-package is here .

Package installation. There is an edited out PKG_LIBS value in the Makevars file in src/, for path opt/openblas. Either decompress, uncomment and recompress, if this is the right path, or send the linker argument to the INSTALL, e.g. like this:

R CMD INSTALL OpenBlasThreads –configure-args=’PKG_LIBS=-L/path/to/openblas/lib -lopenblas’

From → Uncategorized

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: