A Tutorial on R

Sessions to held on Wednesdays at 12:30 in Kielo starting 13.4.2011.

Introduction

Disclaimer: I'm just someone who just started learning R less than a year ago, so do not expect everything in this tutorial to be correct or extremely accurate. I'm stating things the way I think they are or the way I believe it is useful to think they are.

Aim of the tutorial

Starting from some fundamentals, learn R on a generic level. R has got so many different packages for many different approaches that it would be very difficult to cover all of them comprehensively. So I want to give you a nice start into learning R for whatever purpose you may need it for.

A short list of some basic R functions

Some basic R functions

function name	functionality
c	"combine" elements to a vector
array	creates arrays
data.frame	creates data.frames
list	creates lists
mean	mean of the elements of the input vector
sd	standard deviation of the elements of the input vector
cor	correlation
quantile	quantiles
read.table	reads delimited text files
write.table	writes delimited text files
save	save R objects
load	load R objects
apply, tapply, lapply	functions for applying functions over set margins, apply is for arrays, tapply for data.frames and lapply for lists
merge	merges two data.frames, equivalent of SQL joins
order	sorts data by given factors, output is an indexing vector
library	loads packages

Chapter 1 - the Basics of the R Language

• Basic syntax
• The creatures of R: vectors, lists, factors, data.frames, arrays
• Data types

R is an object oriented programming language.

The R console is a command line interpreter, typing out a command e.g. 2+2 returns 4. Mathematical operators (+,-,*,/,^) are used in their mathematical sense and the order of execution is the mathematical one. The operators can also be used in function form, e.g. '+'(1,1). More operators here. Brackets () can be used to control which expressions are evaluated first, e.g. (1+1)/2.

Variables and functions are saved in memory using either an arrow (<- or ->) or =, e.g. var1 <- 1. It is preferable to use the arrow operator.

Functions are used as function(parameter1, parameter2, parameter6 = value), e.g. mean(rnorm(10, sd = 2)). Checking syntax and other details for a function is easy by using ?function, e.g. ?mean. Lists of functions can be found using library(help="package"), where package is the container/library of functions of interest. E.g. library(help="base") or library(help="stats"). A semicolon (;) can be used to separate statements on the same line. E.g. a <- 2;a.

The basic structure most objects in R is a vector. Vectors can be atomic (contain only values), or recursive (vector of objects, basically). A vector in R is basically an ordered set of values or objects. Selecting (subscripting) the n:th element is done by vector[n] or vector[[n]]. The double bracket is used when subscripting from recursive vectors (lists and its subtypes) to extract the stored objects themselves rather than a recursive vector of the selected elements.

There are 3 basic data types: text (class = character), numbers (class = numeric) and logical. Data types in R consist of these basic data types and their more elaborate derivatives. E.g. a factor is a character vector (a vector that consists of textual elements) stored as a numeric vector, where each number represents a unique element of the character vector. The unique elements are stored as a levels attribute of the factor object. Integers are a special case of the numeric class, they are handled as numeric except in storage; normally all numbers are numeric, a special case is the 'a:b' operator which produces a vector of integers from a to b. An atomic vector can only contain values of a single data type.

Attributes are used to simulate more complex data structures. An atomic vector can be given a dimensions attribute dim (which is a numeric vector containing the lengths of the dimensions), to turn it into an array (a matrix is an array with length(dim) = 2). Dimensions can be given names in the dimnames attribute which is a list of named character vectors. Because arrays and matrices are atomic vectors by nature they can only contain values of one data type.

+ Show code- Hide code a <- 1:27;dim(a) <- c(3,3,3);dimnames(a) <- list(dim1=1:3, dim2=1:3, dim3=1:3);a b <- array(1:27, dim = c(3,3,3), dimnames = list(dim1=1:3, dim2=1:3, dim3=1:3));b

Recursive vectors; lists and data.frames (which are lists with elements of equal length atomic vectors, character vectors are by default converted to factors) can have values of different data types since they consist of different objects. The data.frame is perhaps the most common object type in R. It resembles the basic rectangular table format.

+ Show code- Hide code a <- 1:9; b <- factor(rep(c("a","b","c"),3)); d <- factor(rep(c("q","w","e"), each = 3)) df <- data.frame(a,b,d);df df2 <- data.frame(a = 18:10, b = factor(rep(c("a","b","c"),3)), d = factor(rep(c("q","w","e"), each = 3)));df2 l <- list(df, df2, k = 1:2);l l[[1]] l[[3]] l[3] l[[1]][5,1] l[[1]][5,] l[[1]][,1]

In general arrays are produced and used when data is summarized (summed or averaged over some marginals), for computational purposes the data.frame is superior.

More info on objects: http://cran.r-project.org/doc/manuals/R-lang.html#Objects

Classes in R refer to either the data type of a simple atomic vector, or the object type of a more complex object. Functions in R may have different methods for handling inputs of different classes and this may sometimes confuse newcomers; e.g. some functions take a factor input as only a numeric vector instead of a character vector. Many functions try to coerce their input into the format they can operate with by using functions like as.character and as.numeric.

+ Show code- Hide code class(1);class(1:4);class("a");class(TRUE);class(as.factor("a","a","b")) 1:4

More info on classes and other attributes: http://cran.r-project.org/doc/manuals/R-lang.html#Attributes

Many R functions are vectorized, meaning that a function can take one or more vectors as input to produce a vector as output. I.e. 1:5 + 10:6 produces 11 11 11 11 11. If the vectors are of different length, the shorter one is usually recycled to match the length of the longest vector. E.g. 1:2 + rep(4,5) produces 5 6 5 6 5 with a warning calling attention to the arguments different lengths. Vectorized operations are incredibly fast and should be used whenever possible in place of for, while or other loops.

Chapter 2 - Getting your data into R

• Importing and exporting data from/to files
• Working with data: subscription, merge, apply, reshape, conversion between data.frame and array
• OpasnetBaseUtils?

Example data: File:ArkS280.csv. Be careful when converting your Excel sheets to .csv, Microsoft Office is an idiotic piece of software and it sometimes writes empty cells in places where you have once edited something. To fix this open the exported .csv in the Open Office equivalent of Excel and overwrite the previous file.

The easiest way to get data in and out of R is through delimited text files (.txt or .csv). The read.table function reads files specified by a path to a local file or an url. read.table follows the following syntax read.table(file, header, sep, quote, dec, fill, strip.white, ...) (actually there are more arguments but they're not all so relevent, check ?read.table for yourself), where file is a character string specifying the file by path or url e.g. "M:/test.txt" (Note that you have to use forward slash, backslash is the escape character); header is either TRUE or FALSE depending on whether the first line in the file is a header, default is false; sep is the cell separator, default is "\t" meaning tab, csv files usually on an European locale use ";", while the global standard is ","; quote is the quote character used in the file, default is "\"'" (escaped " followed by '); dec is the decimal separator, default is "."; fill determines whether uneven rows are filled with extra empty cells, default is FALSE, hence by default an error will be produced when the file has uneven rows; strip.white removes extra white space from empty cells and strings' leading and tailing edges, default is FALSE. There is a csv wrapper (read.csv) for the function which changes the default of sep to ",", and csv2 (read.csv2) which changes sep to ";" and dec to ",". I would recommend always using the read.table while changing the arguments, since the wrappers don't accept some of the other arguments. The write.table function uses the following basic syntax: write.table(x, file, ... , sep, dec, row.names, na.string), where x is the object to be written; file, sep and dec are the same as for read.table; row.names specifies whether to write row-names into the file, default is TRUE; na.string is the string to be used in missing cells.

Example:

test <- read.table("M:/R koulutus/arkS280.csv", sep = ";", dec = ",", header = TRUE)
test

The output is in the data.frame format. We can select row(s) and column(s) by using subscription. The data.frame is the most flexible format when it comes to data exploration and subscription. Since data.frames are essentially lists, we can use list1[[x]] to select a column x, where x can be a numerical vector (it can be longer than one, experiment with nested lists if you're interested). If the list is named (data.frames always are) we can select its elements by using list1[[x]], where x is a character vector of length one, or list1$col1, where col1 is simply the name of the list element (column in a data.frame). After selecting an object from a list we can subscript from it again, e.g. list1[[1]][1] returns the first element of the first object stored in list1, list1$col1[1] is similar; list1[[1]][[1]]... can be used for nested lists. A data.frame is special in that it is also subscribeable as a two dimensional array: df1[x,y] returns the x:th value of the row y, both can be vectors of any length and of any basic data type (either numeric, character or logical). Either of the x and y can be left blank so that a vector is returned; if a vertical slice is extracted, the result is an atomic vector if only one column was selected (same as selecting the object from the list); if a horizontal slice is extracted, the result is a data.frame. Arrays with more dimensions can be subscribed from in similar fashion e.g. arr1[x,y,z,...], where x, y, z and so on can be vectors of any basic data type.

To better utilize subscription you should learn about logical operators. <, >, <=, >=, ==, and !=, are pretty self-explanatory. More advanced ones include the and (&) and or (|). %in% is also pretty useful. To obtain a inversion use (statement)==FALSE or !(statement). These operators are vectorized, however the expression on the right side must be of length one (comparing all values of a vector on the left side to a single value/expression is allowed, but element by element comparison is not allowed). grep can be used to find regular expressions. All logical operators return a logical vector.

Example:

test <- read.table("M:/R koulutus/arkS280.csv", sep = ";", dec = ",", header = TRUE)
test[test$Suklaa0>=5,]
test[test$Ryhmä=="ip"]

R has got very powerful data manipulation facilities. Data in data.frames usually consist of a few factors and numeric vectors. The factors are usually indices to the data, e.g. in a population data there could be indices for Age, Year and Place. Unique cells in the data would be identified by a unique combination of these factors. This format can be used very similarly to an array. We could sum or take a mean over the levels of specified factors using tapply (data.frame variant of apply). xtabs creates a contingency table from a data.frame with cross-classifying factors, this is similar to an array with some extras. table is similar to xtabs but simpler and only does counts. reshape is a function that transforms data.frames with a single numeric vector into a data.frame with multiple numeric vectors specified by one or more indexing factors and vice versa. merge is a function that merges two data.frames by finding matching (indexing) vectors, any extra vectors in either data are carried over to the resulting data.frame. To simplify variable selection, components of data.frames can be attached to the general namespace, i.e. attach(data) would enable calling the component vectors of data directly: e.g. vec1 instead of data[,"vec1"].

There are many packages for database connections. One for use with the Opasnet Base is the OpasnetBaseUtils, which uses the RODBC package for the actual connection.

A comprehensive guide to importing and exporting data in R can be found on http://cran.r-project.org/doc/manuals/R-data.html.

Chapter 3

• The magic
• Packages
• Fancy plots

R is completely modular, i.e. all functions in R come in packages (libraries). The basic installation of R comes with some 10 packages, which define most of R's basic functionalities. Installing new packages is easy through a top bar menu in the R GUI. Alternatively if you know what you're doing you can use the install.package function directly. Only the some basic packages are loaded into memory during R startup, though those settings can be altered. Specific packages can be loaded using library(packagename).

A useful package for plotting gorgeous graphs is ggplot2. Information on it can be found here.

Chapter 4

• Modeling
• Probability distributions

Chapter Z

• BRUGS (Open BUGS on R)
• ff (on disk objects)
• ...

Cool tricks with R

Multiplying data and adding depth to a time dimension (e.g. minutes) using string selections and regular expressions (completely vectorized)

Time format "1.1.2011 00:00:00", the length of this string varies.

+ Show code- Hide code # First we'll create some random data data <- data.frame(Time=paste("1.1.2011 ", c(paste(0, 0:9, sep = ""), as.character(10:23)), ":00:00", sep = ""), Conc = rnorm(24,10,2)) data # Then multiply the number of rows by 60 data <- data[rep(1:nrow(data), each = 60),] #select from data 60 of each row, overwrite data rownames(data) <- 1:nrow(data) #make rownames sensible data # Change minutes to "00" to "59", currently repeated "00" temp <- as.character(data[,"Time"]) svec <- regexpr(":00:", temp) # gives the positions of ":00:" in the strings, output is a vector so the strings may be of different length e.g. "11.1.2011 00:00:00" vs "1.1.2011 00:00:00" substring(temp,svec+1,svec+2) <- c(paste(0, 0:9, sep = ""), as.character(10:59)) # left side selects parts of the strings (vectorized) based on the svec -- # and the right side substitutes a vector of strings to replace the selection data[,"Time"] <- temp data

Applying a function in a data.frame with multiple rows with values

Example data from Chapter 2.

test <- read.table("M:/R koulutus/arkS280.csv", sep = ";", dec = ",", header = TRUE)

# The basic case where we want to take mean of a set of observations, indexed by something
tapply(test[,"Paino2"], test[,c("Annos", "Ryhmä")], mean) # mean of Paino2 by Annos and Ryhmä

# Tricky version where we take means of several sets of observations indexed by some indices
testf <- function(X, INDEX2, FUN2) tapply(X, INDEX2, FUN2) # define a custom function that does the above to a set of observations X
lapply(test[,7:15], testf, INDEX2 = test[,c("Annos", "Ryhmä")], FUN2 = mean) # apply our test function to all selected columns

# Could also be done with a loop
output <- list() # define a variable so it can be used inside the loop
for (i in 7:15) { # loop for i so that it takes the values given in a vector, 7:15 in this case
  output[[length(output)+1]] <- tapply(test[,i], test[,c("Annos", "Ryhmä")], mean) # apply the above to column defined by i and put into the output list
}
names(output) <- colnames(test)[7:15] # give names to the list objects from the column names of the original table
output # this is identical to the output from the tricky version. Both methods actually use loops so the main difference here is the syntax.

HELP

• How to merge two veeery big datasets together by R? Pauliina is going to workout it in here:

Temperature and population in Europe. Codes are very heavy and free to rewrite.

• If I want to attach new column with rownumbers, how can I do it (What is the code for that)?
  • Would it not suffice to use the rownames attribute? e.g. rownames(test) <- 1:nrow(test)
  • Yeah, I'll managed to do that with this code:

rownames(data) = c(1:nrow(data))#names rows by numbers of the row
rownames(data) = rownames(data, prefix = "ID")#name "ID" for the new column<\pre>