Skip to main content

4  Indexing Vectors and Data Frames

NoteLearning objectives

By the end of this chapter you should be able to:

  1. Index a vector by position, by a logical condition, and by name.
  2. Write comparisons and combine them with the logical operators &, |, and !.
  3. Create a data frame and pull out its rows, columns, and values.
  4. Read a table of data from a CSV file into R.

So far you have created values and stored them in variables. Most of the work of data analysis, though, is not creating data but picking out the part of it you need: the counties with incomes above the state median, the flights that left LaGuardia in June, the census tracts inside one city. Selecting part of a vector or table is called indexing, and it is the subject of this chapter.

We start with vectors, which you met in the last chapter, and three ways to index them: by position, by a logical condition, and by name. Then we stack vectors side by side into a data frame, R’s table, and finish by reading a table from a file.

TipWarm-up: what’s wrong with this code?
18 <- myAge

Find the bug before you read on. (The answer is at the end of Section 4.2.)

4.1 A vector to work with

Most of the examples in this chapter use one vector, called vect, which holds ten numbers between 1 and 1000:

Each element of a vector has a position, called its index. Table 4.1 shows each value above its index. Notice that R starts counting at 1, not 0 as some other languages do: the first element is at index 1.

Table 4.1: The values in vect and their index positions.
Value 1 2 5 10 25 50 100 200 500 1000
Index 1 2 3 4 5 6 7 8 9 10

To index a vector, put square brackets after its name. What you put inside the brackets decides which elements come back, and there are three choices:

Table 4.2: Three ways to index a vector.
Index by Example Returns
Position (number) vect[c(1, 3)] The elements at positions 1 and 3
Logical condition vect[vect > 10] The elements where the condition is TRUE
Name vect["Fozzie"] The element labeled "Fozzie"

The live code boxes in this chapter share one R session, so run the box above first to create vect. As you read, try to predict each answer before you run the code.

4.2 Indexing by position

4.2.1 One element

Put a single number in the brackets to get the value at that position:

vect[1] is 1, the value in position 1. What is vect[3]? Check Table 4.1, then run it:

The answer is 5, the value at position 3. A common mistake is to answer 3, which is the index rather than the value. Keep the two apart: the index says where, and the value is what is there.

An indexed value behaves like any other number, so you can do arithmetic with it. vect[3] is 5 and vect[4] is 10, so:

4.2.2 Leaving an element out

A negative index drops an element and returns all the others:

This returns nine values: everything except the 5 in position 3. It does not change vect itself. Indexing always makes a new vector and leaves the original alone, unless you store the result back in the same name with <-.

4.2.3 Several elements

To get several elements at once, index with a vector of positions:

This returns 1, 5, 25, 100, and 500, the values in positions 1, 3, 5, 7, and 9. The values come back in the order you ask for them, so vect[c(9, 1)] returns 500 and then 1.

For a run of positions in a row, the colon operator : makes the sequence for you. 6:8 is shorthand for c(6, 7, 8):

NoteWarm-up answer

18 <- myAge has the assignment arrow pointing the wrong way. It asks R to store myAge in the number 18, which isn’t allowed, so R stops with the error invalid (do_set) left-hand side to assignment. The name goes on the left of <- and the value on the right: myAge <- 18.

4.3 Logical operations

4.3.1 Comparisons return TRUE or FALSE

TRUE and FALSE are special values in R, called logical or Boolean values (after the mathematician George Boole). Every comparison operator returns one of them (Table 4.3).

Table 4.3: Comparison operators in R.
Operator Meaning Example Result
> greater than 5 > 5 FALSE
< less than 5 < 5 FALSE
>= greater than or equal to 5 >= 5 TRUE
<= less than or equal to 5 <= 5 TRUE
== equal to 5 == 5 TRUE
!= not equal to 5 != 5 FALSE

Notice that “equal to” is written with two equals signs, ==. A single = does something different: it sets an argument inside a function call, as in round(x, digits = 1). Mixing them up is one of the most common errors in R, and you will see it again when you start filtering data.

4.3.2 A comparison checks every element

Compare a whole vector with a number, and R checks every element, returning one TRUE or FALSE for each. The result is a logical vector the same length as the original:

The first four values (1, 2, 5, and 10) are not greater than 10, so they are FALSE; the other six are TRUE (Table 4.4).

Table 4.4: The result of vect > 10 for each element (T is TRUE, F is FALSE).
Value 1 2 5 10 25 50 100 200 500 1000
vect > 10 F F F F T T T T T T

4.3.3 Indexing with a condition

Now put that logical vector inside the brackets. R keeps the elements in the TRUE positions and drops the rest:

The result is 25, 50, 100, 200, 500, and 1000: every value greater than 10. Read vect[vect > 10] out loud as “the elements of vect where vect is greater than 10.” This is the single most useful idea in the chapter. Selecting the counties whose income is above the median, or the flights that left late, works exactly the same way.

4.3.4 Logical vectors are recycled

What if the logical vector is shorter than the vector you are indexing? R recycles it, repeating it until it is long enough. Here c(TRUE, FALSE) becomes TRUE, FALSE, TRUE, FALSE, ..., ten values in all:

The result is every other value, starting with the first: 1, 5, 25, 100, and 500. Recycling is handy now and then, but it can also hide mistakes, so make sure a logical index is the same length as the vector unless you mean it to be shorter.

4.3.5 Combining conditions: AND, OR, and NOT

Often you need two conditions at once: values greater than 10 and less than 500, say. R has three logical operators for combining TRUE and FALSE values.

AND (&) is TRUE only when both sides are TRUE. Is the statement “the sun rises in the west AND humans breathe oxygen” true? No: the first part is false, so the whole statement is false.

OR (|) is TRUE when either side is TRUE (or both). “The sun rises in the west OR humans breathe oxygen” is true, because the second part is. On most keyboards the vertical bar | is on the backslash key, above Enter.

Table 4.5 gives every combination. These are called truth tables.

Table 4.5: Truth tables for AND, written a & b in R, and OR, written with a vertical bar.
a b a AND b a OR b
TRUE TRUE TRUE TRUE
TRUE FALSE FALSE TRUE
FALSE TRUE FALSE TRUE
FALSE FALSE FALSE FALSE

With AND, only TRUE & TRUE gives TRUE. With OR, only FALSE | FALSE gives FALSE.

Now back to the question: which values of vect are greater than 10 and less than 500?

The answer is 25, 50, 100, and 200. 500 is left out because the test is strictly less than 500; use <= to include it. Now try the same conditions joined by OR:

This returns all ten values, because every number is either greater than 10 or less than 500. It’s a useful warning: a careless OR can end up filtering out nothing at all. When a filter returns more than you expected, check whether you meant &.

NOT (!) flips a logical value: !TRUE is FALSE, and !FALSE is TRUE. Combined with = it gives the “not equal to” operator !=, which is handy for removing particular values:

Only the element equal to 2 is dropped. You can also put ! in front of a whole condition in parentheses: vect[!(vect > 10)] returns the values that are not greater than 10.

4.4 Indexing by name

4.4.1 Naming the elements

The third way to index uses labels. The function names() attaches a name to each element of a vector. The vector of names must be the same length as the vector of data:

The values haven’t changed; each one now just has a label printed above it (Table 4.6).

Table 4.6: The values in vect, with their index positions and names.
Value 1 2 5 10 25 50 100 200 500 1000
Index 1 2 3 4 5 6 7 8 9 10
Name Kermit Piggy Gonzo Fozzie Animal DrTeeth Scooter Rowlf Chef Jim

4.4.2 Selecting by name

Put a name, in quotation marks, inside the brackets:

vect["Fozzie"] returns 10, because Fozzie is the fourth name and position 4 holds 10. vect[c("Kermit", "Scooter")] returns 1 and 100, each with its name attached. Use straight quotation marks ("), not the curly ones a word processor inserts, or R won’t recognize them.

Names make code easier to read, and they keep working if the order of the elements changes. vect["Fozzie"] still finds Fozzie’s value even if someone sorts the vector; vect[4] would quietly return whatever has moved into position 4. This is the same reason you will refer to the columns of a data frame by name.

4.4.3 How long is a vector?

The function length() returns the number of elements in a vector:

vect has 10 elements. How would you get the last element of a vector, whatever its length? You could type vect[10], but that only works while vect has exactly ten elements. Since length() gives the position of the last element, use it as the index:

This returns 1000 (with its name, Jim) for a vector of any length.

4.5 Data frames

4.5.1 From one dimension to two

A vector is one-dimensional: a single row of values. Stand several vectors of the same length side by side, and you get a two-dimensional table. In R, that table is a data frame, and it is how R stores almost every data set you will work with, including, later in the course, spatial data.

A data frame is organized into rows and columns (Table 4.7):

  • Each column is a variable: something measured or recorded, such as a name or an exam score.
  • Each row is an observation: one thing that was measured, such as a student, a county, or a flight.
  • All the values in a column must have the same data type. Each column is a vector, and, as you saw in the last chapter, a vector holds just one type. Different columns can have different types.
Table 4.7: A small data frame. Each row is a student and each column is a variable.
name exam1 exam2 bonus
Eliza “A” 98.1 TRUE
Shonda “A-” 85.4 TRUE
Me’shelle “A” 74.3 FALSE
Jack “A-” 92.2 TRUE

Here name and exam1 are character columns, exam2 is numeric, and bonus is logical. Suppose a number, such as 75.4, crept into the exam1 column of letter grades. R would not keep a number in a character column: it would coerce the 75.4 into the text "75.4". The same thing happens in reverse, and causes more trouble: a single letter or symbol in a column of numbers, such as "N/A" in exam2, turns the whole column into text, and then you can no longer average it.

4.5.2 Creating a data frame

The function data.frame() builds a data frame from vectors. Each argument becomes a column: the name before = is the column name, and the vector after = holds its values. Every vector must be the same length, here four:

TipThink about it

Here is another data frame:

df <- data.frame(
  x = 1:5,
  y = c("a", "b", "c", "d", "e")
)

What are its column names? Is df a good name for a data frame? Are x and y good names for its columns? Neither tells a reader anything about the data. Prefer descriptive names, such as grades or nc_counties, and add a comment saying what the data frame holds and where it came from.

4.5.3 Looking at a data frame

A few functions tell you what is in a data frame without printing all of it, which matters once a data frame has thousands of rows:

In RStudio, View(grades) also opens the data frame in a spreadsheet-like tab, and clicking a data frame’s name in the Environment pane does the same.

4.5.4 Indexing a data frame

The same bracket indexing works on data frames, with one change: a data frame has two dimensions, so the brackets take two indexes, separated by a comma. The row comes first, then the column: grades[row, column].

Leaving a position blank means “all of them,” so grades[2, ] is the whole second row and grades[, "exam2"] is the whole exam2 column. Don’t forget the comma: without it, R treats the number as a column index, not a row.

Because pulling out one column is so common, R has a shortcut, the dollar sign. grades$exam2 returns the exam2 column as a vector, which you can then use like any other vector:

Everything from earlier in the chapter now applies to rows. Put a condition in the row position, and you get the rows where it is TRUE:

The first line returns the two students who scored over 90 on exam 2. The second returns the names of the students who earned the bonus and an A on exam 1. In the next chapter you will meet filter(), the tidyverse’s way to do the same thing with less typing.

4.6 Reading tabular data

You will rarely type a data set in by hand. Usually it arrives as a file, and the most common format is a CSV file (comma-separated values): plain text in which each line is a row and commas separate the values. The first line usually holds the column names, called the header. Here is the grades table as a CSV file:

name,exam1,exam2,bonus
Eliza,A,98.1,TRUE
Shonda,A-,85.4,TRUE
Me'shelle,A,74.3,FALSE
Jack,A-,92.2,TRUE

Base R has two functions for reading text files like this into a data frame.

read.table() is the general-purpose workhorse. It reads any table of text in which the values are separated by some character, called the delimiter, and it has many options, including:

  • header: is the first row the column names (TRUE or FALSE)?
  • sep: the delimiter, such as "," for commas or "\t" for tabs.
  • row.names and col.names: labels for the rows and columns.
scores <- read.table("scores.txt", header = TRUE, sep = "\t")

read.csv() is read.table() with its defaults set for CSV files, sep = "," and header = TRUE, so usually all it needs is the file name:

scores <- read.csv("scores.csv")

Its sibling read.csv2() reads the kind of CSV file common in much of Europe, which uses a comma as the decimal mark (3,14) and a semicolon to separate values.

Remember from Section 3.4 that R looks for the file relative to the folder your .Rmd file is saved in. If the CSV file is in the same folder, the file name alone is enough; if it is in a data subfolder, write read.csv("data/scores.csv"). Most “cannot open file” errors are file path errors.

The box below can’t read a file from your computer, because it runs inside your web browser. Instead, it writes the CSV file above to a temporary file first, and then reads it back:

read.csv() worked out each column’s type by itself: character for name and exam1, numeric for exam2, and logical for bonus. Always check the types after reading a file. If a column you expected to be numeric comes in as character, something in it, such as a comma in "12,000" or a "N/A", isn’t a number.

The tidyverse has its own, faster function for reading CSV files, read_csv() (with an underscore), from the readr package. It works the same way, and you will see it in later chapters.

4.7 Check your understanding

For questions 1–5, vect <- c(1, 2, 5, 10, 25, 50, 100, 200, 500, 1000).

  1. What does vect[4] return?

  2. What does vect[2] + vect[5] return?

  3. How many values does vect[-1] return?

  4. What does vect[vect >= 100 & vect < 1000] return?

  5. Which line returns the last element of vect, whatever its length?

  6. 5 = 5 tests whether 5 is equal to 5.

  7. What does TRUE | FALSE return?

  8. In a data frame, each row is an and each column is a variable.

  9. Which function reads a comma-separated file into a data frame?

Work out vect >= 100 and vect < 1000 separately, then keep only the values where both are TRUE. Is 1000 less than 1000?

In R, a single = sets an argument, as in round(x, digits = 1). Testing for equality takes two.

4.8 Summary

  • Indexing selects part of a vector or table, with square brackets. R counts positions from 1.
  • By position: vect[3] is the value at position 3; vect[-3] drops it; vect[c(1, 3)] and vect[6:8] return several values. The colon operator : makes a sequence.
  • Comparison operators (>, <, >=, <=, ==, !=) return TRUE or FALSE for every element. Equality is ==; a single = sets an argument.
  • By condition: vect[vect > 10] keeps the elements where the condition is TRUE. Combine conditions with & (AND: both must be true), | (OR: either can be true), and ! (NOT). Short logical vectors are recycled.
  • By name: names() labels the elements, and vect["Fozzie"] selects by label. length() counts elements, and vect[length(vect)] is the last one.
  • A data frame is a table of equal-length vectors: columns are variables, rows are observations, and each column holds one data type. Make one with data.frame(), and index it with df[row, column] or df$column.
  • Read CSV files with read.csv() (or read.table() for other delimiters), and check the column types after you read.

4.9 Exercises

  1. Predict, then check. For vect <- c(1, 2, 5, 10, 25, 50, 100, 200, 500, 1000), write down what each line returns before you run it. Then run them and check.

    vect[c(2, 4, 6)]
    vect[-(1:3)]
    vect[vect < 10 | vect > 200]
    vect[!(vect > 5)]
    vect[c(FALSE, FALSE, TRUE)]
  2. Temperatures. Store a week of daily high temperatures in a vector, and name each element with a day of the week. Then return (a) Wednesday’s temperature, (b) the temperatures above 80, (c) the days on which it was between 70 and 80, and (d) the last day’s temperature, using length().

  3. Count with logicals. sum(vect > 10) returns 6. Why? (Hint: R counts TRUE as 1 and FALSE as 0.) Use the same idea to count how many values in vect are between 10 and 500, inclusive.

  4. Build a data frame. Make a data frame of five North Carolina cities with three columns: the city’s name, its approximate population, and whether it is a county seat. Use descriptive names and add a comment. Then return (a) the population column with $, (b) the third row, and (c) the names of the cities with more than 100,000 people.

  5. Type trouble. Create the data frame below and run str() on it. Why can’t you take the mean of pop? Fix the data so that you can.

    towns <- data.frame(
      town = c("Carrboro", "Hillsborough", "Pittsboro"),
      pop  = c("21,295", "9,660", "4,537")
    )
  6. Read a CSV file. In a spreadsheet program, make a small table with a header row and at least three columns, and save it as a CSV file in your geog215 folder. Read it into R with read.csv(), and use str() to check that each column has the type you expected.

4.10 Further reading

  • R for Data Science (2nd ed.), https://r4ds.hadley.nz. The chapter “Logical vectors” covers comparisons, &, |, and ! in more depth, and “Data import” covers reading CSV files with readr.
  • Hands-On Programming with R by Garrett Grolemund, free at https://rstudio-education.github.io/hopr/. Its chapter “R Notation” is a gentle, thorough guide to indexing vectors and data frames.
  • Advanced R (2nd ed.) by Hadley Wickham, https://adv-r.hadley.nz. The chapter “Subsetting” is the complete reference, for when you are ready for it.