# recosystem: Recommender System Using Parallel Matrix Factorization

A Quick View of Recommender System The main task of recommender system is to predict unknown entries in the rating matrix based on observed values, as is shown in the table below: Each cell with number in it is the rating given by some user on a specific item, while those marked with question marks are unknown ratings that need to be predicted. In some other literatures, this problem may be named collaborative filtering, matrix completion, matrix recovery, etc.

# Large scale eigenvalue decomposition and SVD with rARPACK

In January 2016, I was honored to receive an “Honorable Mention” of the John Chambers Award 2016. This article was written for R-bloggers, whose builder, Tal Galili, kindly invited me to write an introduction to the rARPACK package. A Short Story of rARPACK Eigenvalue decomposition is a commonly used technique in numerous statistical problems. For example, principal component analysis (PCA) basically conducts eigenvalue decomposition on the sample covariance of a data matrix: the eigenvalues are the component variances, and eigenvectors are the variable loadings.

# An overview of linear algebra libraries in Scala/Java

This semester I’m taking a course in big data computing using Scala/Spark, and we are asked to finish a course project related to big data analysis. Since statistical modeling heavily relies on linear algebra, I investigated some existing libraries in Scala/Java that deal with matrix and linear algebra algorithms. 1. Set-up Scala/Java libraries are usually distributed as *.jar files. To use them in Scala, we can create a directory to hold them and set up the environment variable to let Scala know about this path.

# A Conversation with Hadley Wickham

Dr. Hadley Wickham is the Chief Scientist of RStudio and Assistant Professor of Statistics at Rice University. He is the developer of the famous R package ggplot2 for data visualization and the author of many other widely used packages like plyr and reshape2. On Sep 13, 2013 he gave a talk at Department of Statistics, Purdue University, and later I (Yixuan) had a conversation with him (Hadley), talking about his own experience and interest on data visualization, data tidying, R programming and other related topics.

# Is Normal normal?

The rumor says that Normal distribution is everything. It will take a long long time to talk about the Normal distribution thoroughly. However, today I will focus on a (seemingly) simple question, as is stated below: If $X$ and $Y$ are univariate Normal random variables, will $X+Y$ also be Normal? What’s your reaction towards this question? Well, at least for me, when I saw it I said “Oh, it’s stupid.

# Handwriting recognition using R

This title is a bit exaggerating since handwriting recognition is an advanced topic in machine learning involving complex techniques and algorithms. In this blog I’ll show you a simple demo illustrating how to recognize a single number (0 ~ 9) using R. The overall process is that, you draw a number in a graphics device in R using your mouse, and then the program will “guess” what you have input. It is just for FUN.

# How to run regression on large datasets in R

It’s well known that R is a memory based software, meaning that datasets must be copied into memory before being manipulated. For small or medium scale datasets, this doesn’t cause any troubles. However, when you need to deal with larger ones, for instance, financial time series or log data from the Internet, the consumption of memory is always a nuisance. Just to give a simple illustration, you can put in the following code into R to allocate a matrix named x and a vector named y.

#### Yixuan's Blog

Statistics, Programming, and …?