This post is a sequel to the previous article on how to use the old Fortran code to solve optimization problems in C++ applications. This time we consider the L-BFGS-B algorithm for solving smooth and box-constrained optimization problems of the form $$ \begin{align*} \min_{x}\quad & f(x)\\ \text{subject to}\quad & l\le x\le u, \end{align*} $$ where $l$ and $u$ are simple bounds for $x\in\mathbb{R}^n$, and can take $-\infty$ and $+\infty$ values.
L-BFGS is a well-known and widely-used optimization algorithm for smooth and unconstrained optimization problems. It was originally implemented in Fortran, and also has some more recent implementations including libLBFGS and my own LBFGS++.
The Fortran code was written more than 30 years ago, and looks a bit exotic from today’s perspective. However, it is still one of the most stable and mature implementations of the L-BFGS algorithm, and is typically used as a baseline in testing and benchmarking.
Motivation Deep learning frameworks such as PyTorch and Tensorflow provide excellent auto-differentiation support for matrices and vectors. They have included many built-in functions and operators that can be combined together to create complicated yet auto-differentiable functions. However, in some cases we prefer to manually define the gradient of a function, instead of relying on automatic differentiation; yet we still allow this function to be embedded into a larger program, which has end-to-end auto-differentiation support.
This article is mainly used to record my own experience on installing the Solus operating system to a new machine, during which I encountered troubles using the regular method. It may not be very helpful for general users, but if you had similar experience as mine, then the method introduced here is one possible solution.
I typically install three operating systems (OS) side by side on my machine for daily use: Windows, Manjaro Linux, and Solus.
Per the suggestion by @robmaz, RSpectra::svds() now has two new parameters center and scale, to support implicit centering and scaling of matrices in partial SVD. The minimum version for this new feature is RSpectra >= 0.16-0.
These two parameters are very useful for principal component analysis (PCA) based on the covariance or correlation matrix, without actually forming them. Below we simulate a random data matrix, and use both R’s built-in prcomp() and the svds() function in RSpectra to compute PCA.
A few days ago a friend asked me the following question: how to efficiently extract some specific lines from a large text file, possibily compressed by Gzip? He mentioned that he tried some R functions such as read.table(skip = ...), but found that reading the data was too slow. Hence he was looking for some alternative ways to extracting the data.
This is a common task in preprocessing large data sets, since in data exploration, very often we want to peek at a small subset of the whole data to gain some insights.
It has been one year since my last article, and here is a quick post indicating that my blog is not down. Instead, it has a new look thanks to blogdown. Yes, pun intended. :-)
blogdown, mostly written by Yihui, is an R package that can help you rapidly create a static blog or website. The package name has nothing to do with the status of a website (as in “the server is down”), but rather follows the convention of other Markdown-based packages such as rmarkdown and bookdown.