Managing Data with R

I enjoyed teaching R workshops for many years, but I have retired from teaching them. I leave the workshop pages up just to let people know.

Before you can analyze data, it must be in the right form. Getting it into that form is often where we spend most of our time. This two-day workshop shows how to perform the most commonly used data management tasks in R. We will cover how to use R’s most popular add-on packages (dplyr, stringr, lubridate, tidyr, broom, compare, sqldf, etc.) and compare them to R’s older built-in functions.

Most of our time will be spent working through examples that you may run simultaneously on your computer. You will see both the instructor’s screen and yours, side-by-side, as we run the examples and discuss the output. However, the handouts include each step and its output, so feel free to skip the computing; it’s easy to just relax and take notes.

Most of the examples come from the highly-regarded books by the instructor, R for SAS and SPSS Users and R for Stata Users. That makes it easy to review what we did later with full explanations, or to learn more about a particular subject by extending an example which you have already seen.

The workshops are available on-site or via webinar.

The on-site workshops are the most thorough since direct face-to-face interaction is the most flexible. The instructor presents a topic for around twenty minutes. Then we switch to exercises, which are already open in another tabbed window. The exercises contain hints that show the general structure of the solution that you adapt to get the final solution. The complete solutions are in a third tabbed window, so if you get stuck the answers are a click away. There is plenty of time to handle in-depth questions on any of the topics covered, and the discussion often veers off into a broad range of interesting areas. The usual schedule for an on-site workshop is here.

The webinar version is particularly easy to work into a busy schedule. It’s offered in two half-day sessions with a day or two skipped in between to give participants a chance to do the exercises on their own and catch up on other work. There is time for questions on the lecture topics (live) and the exercises (via email). The lecture is recorded and available for review for 30 days.

For further details or to arrange a site visit, contact the instructor, Bob Muenchen, at muenchen.bob@gmail.com.

Prerequisites

Attendees should know basic R programming, including how to read data files and call functions.

Learning Outcomes

When finished, participants will be able to prepare most data sets for analysis.

Presenter

Robert A. Muenchen is the author of R for SAS and SPSS Users and, with Joseph M. Hilbe, R for Stata Users. He is also the creator of r4stats.com, a popular web site devoted to analyzing trends in analytics software and helping people learn the R language. Bob is an ASA Accredited Professional Statistician™ with 33 years of experience and is currently the manager of OIT Research Computing Support (formerly the Statistical Consulting Center) at the University of Tennessee. He has taught workshops on research computing topics for more than 500 organizations and currently offers training in partnership with DataCamp.com, Revolution Analytics, RStudio, New Horizon’s Computer Learning Centers, and Xerox Learning Services. Bob has written or coauthored over 70 articles published in scientific journals and conference proceedings, and has provided guidance on more than 1,000 graduate theses and dissertations.

Bob has served on the advisory boards of SAS Institute, SPSS Inc., StatAce OOD, Intuitics, the Statistical Graphics Corporation and PC Week Magazine. His suggested improvements have been incorporated into SAS, SPSS, JMP, STATGRAPHICS and several R packages. His research interests include statistical computing, data graphics and visualization, text analytics, and data mining.

Computer Requirements

On-site training is best done in a computer lab with a projector and, for large rooms, a PA system. The interactive video version requires only a web browser and an Internet connection fast enough to display video.

Course programs, data, and exercises will be sent a week before the workshop. The instructions include installing R, which you can download R for free here: http://www.r-project.org/. We will also use RStudio, which you can download for free here: http://RStudio.com. If you already know a different R editor, that’s fine too.

Course Outline 

1. INTRODUCTION
1.1 Topics
1.2 Preparing Your Computer
1.3 Note to System Administrators

2. INTRODUCTION TO THE TIDYVERSE
2.1 Tidyverse Packages
2.2 Tibble Creation
2.3 Tibbles Improve Printing
2.4 Other Tibble Advantages
2.5 Tibble Disadvantages
2.6 Tibble Conversions
2.7 The dplyr Package’s Verbs
2.8 dplyr Input & Output
2.9 Practice Time

3. CHOOSING VARIABLES AND OBSERVATIONS
3.1 Using Subscripts
3.2 Using dplyr Functions
3.3 Variations on select
3.4 Dropping Variables
3.5 Table of Logical Comparisons
3.6 Practice Time

4. COMBINING PROGRAMMING STEPS
4.1 Nesting Only
4.2 Piping
4.3 Saving Results for Re-use
4.4 Piping Details
4.5 Piping to a Specific Argument
4.6 Think About Your Steps
4.7 Summary
4.8 Practice Time

5. COPYING & DELETING OBJECTS
5.1 Copying Objects
5.2 Copying Variables
5.3 Removing/Dropping/Deleting Variables
5.4 Removing/Dropping/Deleting Entire Objects
5.5 Practice Time

6. RENAMING DATA SETS, VARIABLES, & ROWS
6.1 Renaming Objects
6.2 Renaming Big Objects
6.3 Renaming Variables with dplyr
6.4 Renaming All Variables Using “names”
6.5 Copying Names From Another Data Frame
6.6 Renaming a Block of Names, Step 1
6.7 Renaming a Block of Names, Step 2
6.8 Renaming Thousands of Variables
6.9 Choosing Best Variable Renaming Method
6.10 Renaming Rows
6.11 The tibble Approach to Row Names
6.12 Practice Time

7. TRANSFORMING VARIABLES
7.1 Prepare the Workspace
7.2 Using Classic Dollar Format
7.3 An Easier Way: mutate
7.4 mutate & transmute Details
7.5 Row-Specific Functions
7.6 The Base apply Function
7.7 apply Function Details
7.8 Many Variables, One Transformation
7.9 mutate_at Details
7.10 Table of Transformations
7.11 Practice Time

8. CONDITIONAL TRANSFORMATIONS
8.1 Prepare the Workspace
8.2 The ifelse Function
8.3 Recode Using ifelse
8.4 Recoding Many Variables with ifelse
8.5 The car::Recode Function
8.6 Recode Many Variables
8.7 Integers vs. Double Precision
8.8 Practice Time

9. SUMMARIZING VARIABLES
9.1 Prepare the Workspace
9.2 The “summarise” Function
9.3 summarise Details
9.4 Many R Functions Require Vectors
9.5 dplyr::summarise_at Function
9.6 summarise_at Details
9.7 Built-In Summary Functions
9.8 dplyr Summary Functions
9.9 dplyr Summary Combination Functions
9.10 dplyr Sequence Functions
9.11 dplyr Rank Functions
9.12 Comparison of mutate and summarise
9.13 Practice Time

10. GROUP-BY CALCULATIONS
10.1 Prepare the Workspace
10.2 The group_by Function
10.3 Printing Grouped Data
10.4 Review of mutate
10.5 mutate By Group
10.6 Summarisation By Group
10.7 summarise By Group
10.8 summarise_at By Group
10.9 Group By Next Level
10.10 Group By Next Level…Again!
10.11 Un-Grouping
10.12 Practice Time

11. GROUP-BY ANALYSIS WITH OUTPUT MANAGEMENT
11.1 Prepare the Workspace
11.2 R’s Built-in Approach
11.3 Recall How t.test Works
11.4 broom Package Cleans it Up
11.5 Simple Analysis with group_by
11.6 dplyr’s do Function
11.7 broom’s Functions
11.8 Model-Level Regression by Group
11.9 Coefficient-Level Regression by Group
11.10 Observation-Level Regression By Group
11.11 Advanced Features
11.12 Practice Time

12. SORTING DATA
12.1 Prepare the Workspace
12.2 R’s Various Ways to Sort
12.3 When Sorting is Needed in R
12.4 Data Not Sorted by Workshop
12.5 dplyr::arrange Sorts Data Frames
12.6 desc Does Descending Order
12.7 Sorting by Two Variables
12.8 R’s built-in sort Function
12.9 R’s order Function
12.10 Using order to Sort Data Frames
12.11 rev Function Reverses order
12.12 order by Two Variables
12.13 How Location Affects Sorting
12.14 Practice Time

13. SELECTING FIRST OR LAST OBSERVATION PER GROUP
13.1 Prepare the Workspace
13.2 When to Search for These Observations
13.3 When it’s Not Needed
13.4 dplyr’s slice Function
13.5 Finding Min/Max Observation Using Sorting
13.6 Finding Min/Max Observation Using filter
13.7 Finding Min/Max Observation Using Ranks
13.8 dplyr Ranking Functions
13.9 Practice Time

14. STACKING DATA SETS
14.1 Prepare the Workspace
14.2 Creating a Data Frame to Stack
14.3 Creating a 2nd Data Frame to Stack
14.4 Stacking with dplyr::bind_rows
14.5 R’s Built-in rbind
14.6 R’s Built-in union
14.7 Practice Time

15. FINDING AND REMOVING DUPLICATE OBSERVATIONS
15.1 Prepare the Workspace
15.2 Create Some Duplicates
15.3 Locating Duplicates
15.4 Generate Duplicate Report
15.5 Removing Duplicates
15.6 Checking Subsets of Variables
15.7 Practice Time

16. MERGING / JOINING DATA FRAMES
16.1 Prepare the Workspace
16.2 Creating a Data Frame to Join
16.3 Creating a 2nd Data Frame to Join
16.4 Join by Common Variables
16.5 Joining by Different Variables
16.6 Types of Joins
16.7 Practice Time

17. RESHAPING DATA FRAMES
17.1 Prepare the Workspace
17.2 Transposing Rows and Columns
17.3 Example Wide Data Structure
17.4 Advantages of Wide Data
17.5 The Long Data Structure
17.6 Advantages of Long Data
17.7 Reshaping Options in R
17.8 Gathering Wide to Long
17.9 Wide to Long Details
17.10 Spreading Long to Wide
17.11 Extracting Numeric Values
17.12 Practice Time

18. COMPARING OBJECTS
18.1 Prepare the Workspace
18.2 Comparing Vectors
18.3 Comparing Data Frames
18.4 Mixing Up a Data Frame
18.5 Three Ways to Compare
18.6 The compare Package
18.7 Visual Comparison
18.8 The compareDF Package
18.9 Practice Time

19. CHARACTER STRING MANIPULATIONS
19.1 Prepare the Workspace
19.2 The stringr Package
19.3 Regular Expression References
19.4 Generating Numeric Variable Names
19.5 Impact of Trailing Blanks
19.6 Trimming Blanks
19.7 Setting Case
19.8 Splitting at a Column
19.9 Splitting at a Blank
19.10 Extracting Vectors
19.11 Replacing Strings
19.12 Combining Strings
19.13 Finding: One Sub-string
19.14 Finding: Multiple Sub-strings
19.15 Finding: with Regular Expressions
19.16 Finding: with Table Lookups
19.17 The stringi Package
19.18 Practice Time

20. DATE & TIME MANIPULATIONS
20.1 Prepare the Workspace
20.2 Converting Strings to Dates
20.3 Subtracting Dates
20.4 The difftime Function
20.5 Converting Time Differences to Numeric
20.6 Measuring Time Until Today
20.7 Extracting Years, Weeks, Months
20.8 Extracting Days
20.9 Choosing Observations by Date
20.10 Dealing with 2-Digit Years
20.11 Date-Time References
20.12 Practice Time

21. USING SQL WITHIN R
21.1 Prepare the Workspace
21.2 The sqldf Package
21.3 Printing a Data Frame
21.4 Choosing and Sorting
21.5 Aggregating by Gender
21.6 Key Syntax Differences
21.7 Practice Time

22. CONCLUSION
22.1 Brief Review
22.2 Providing Feedback
22.3 Future Support
22.4 Question Time

Here is a slide show of previous workshops.