I’ve Been Replaced by an Analytics Robot

It was only a few years ago when the N.Y. Times declared my job “sexy”.  My old job title of statistician had sounded dull and stodgy, but then it became filled with exciting jargon: I’m a data scientist doing predictive analytics with (occasionally) big data. Three hot buzzwords in a single job description! However, in recent years, the powerful technology that has made my job so buzzworthy has me contemplating the future of the field. Computer programs that automatically generate complex models are becoming commonplace. Rob Hyndman’s forecast package for R, SAS Institite’s Forecast Studio, and IBM’s SPSS Forecasting offer the ability to generate forecasts that used to require years of training to develop. Similar tools are now available for other types of models as well.

Countless other careers have been eliminated due to new technology. The United States previously had over 70% of the population employed in farming and fewer than 2% are farmers today. Things change, people move on to other careers. The KDnuggests web site recently asked its readers, “When will most expert-level Predictive Analytics/Data Science tasks – currently done by human Data Scientists – be automated?” Fifty-one percent of the respondents – most of them data scientists themselves – estimated that this would happen within 10 years. Not all the respondents had such a dismal view though; 19% said that this would never happen.

My brain being analyzed by the machine that replaced my brain!
My brain being analyzed by the machine that replaced my brain! (Photograpy by Mike O’Neil)

If you had asked me in 1980 what would be the very last part of my job to be eliminated through automation, I probably would have said: brain wave analysis. It had far more steps involved than any other type of work I did. We were measuring the electrical activity of many parts of the brain, at many frequencies, thousands of times per second. An analysis that simply compared two groups would take many weeks of full-time work. Surprisingly, this was the first part of my job to be eliminated. However, our statistical consulting team supports many different departments, so I didn’t really notice when work stopped arriving from the EEG Lab. Years later I got a call from the new lab director offering to introduce me to my replacement: a “robot” named LORETA.

When I visited the lab, I was outfitted with the usual “bathing cap” full of electrodes. EEG paste (essentially K-Y jelly) was squirted into a hole in each electrode to ensure a good contact and the machine began recording my brain waves. I used bio-feedback to generate alpha waves which made a car go around a track in a simple video game. Your brain creates alpha waves when you get into a very relaxed, meditative state. Moments after I finished, LORETA had already analyzed my brain waves. “She” had done several weeks of analysis in just a few moments.

So that part of my career ended years ago, but I didn’t really notice it at the time. I was too busy using the time LORETA freed up to learn image analysis using ImageJ, text mining using WordStat and SAS Text Miner, and an endless variety of tasks using the amazing
R language. I’ve never had a moment when there wasn’t plenty of interesting new work to do.

There’s another aspect to my field that’s easy to overlook. When I began my career, 90% of the time was spent “battling” computers. They were incredibly difficult to operate. Today someone may send you a data file and you’ll be able to see the data moments after receiving it. In 1980 data arrived on tapes, and every computer manufacturer used a different tape format, each in numerous incompatible variations. Unless you had a copy of the program that created a tape, it might take days of tedious programming just to get the data off of it. Even asking the computer to run a program required error-prone Job Control Language. So from that perspective, easier-to-use computing technology has already eliminated 90% of what my job used to be. It wasn’t the interesting part of the job, so it was a change for the better.

Will the burgeoning field of data science eventually put itself out of business by developing a LORETA for every problem that needs to be solved? Will we just be letting our Star-Trek-class computers and robots do our work for us while we lounge around self-actualizing? Perhaps some day, but I doubt it will happen any time soon!

I invite you to follow me here or at http://twitter.com/BobMuenchen. If you’re interested in learning R, DataCamp.com offers my 16-hour interactive workshop, R for SAS, SPSS and Stata Users for $25. That’s a monthly fee, but it definitely won’t take you a month to take it!  For students & academics, it’s $9. I also do R training on-site.

12 thoughts on “I’ve Been Replaced by an Analytics Robot”

    1. Hi Thomas,

      Thanks, I’ve certainly enjoyed your comments on the subject of automated analytics. I look forward to reading more of your thoughts in the future.

      Cheers,
      Bob

    1. Hi Thomas,

      Thanks, I’ve certainly enjoyed your comments on the subject of automated analytics. I look forward to reading more of your thoughts in the future.

      Cheers,
      Bob

  1. Data analysis is becoming more like research. Would that be a true statement? Meaning that every job or analysis is unique or else it gets automated. I can see more of this in bioinformatics too. And I welcome it. If your “clients” do more of their own analysis it leaves more room to develop something completely new. There is always more to do, since data analysis never ends – it just gets more refined!

    1. Hi Pekka,

      That’s an interesting idea. Our statistical consulting team works with people from around 250 university departments, so the data we see is very diverse. While that can result in a wide range of steps during analysis, many of them do form a fairly standard, albeit complex, pattern. There are many efforts underway to automate those steps, such as The Automatic Statistican. I’ll be very interested to see how quickly they progress.

      Cheers,
      Bob

  2. Data analysis is becoming more like research. Would that be a true statement? Meaning that every job or analysis is unique or else it gets automated. I can see more of this in bioinformatics too. And I welcome it. If your “clients” do more of their own analysis it leaves more room to develop something completely new. There is always more to do, since data analysis never ends – it just gets more refined!

    1. Hi Pekka,

      That’s an interesting idea. Our statistical consulting team works with people from around 250 university departments, so the data we see is very diverse. While that can result in a wide range of steps during analysis, many of them do form a fairly standard, albeit complex, pattern. There are many efforts underway to automate those steps, such as The Automatic Statistican. I’ll be very interested to see how quickly they progress.

      Cheers,
      Bob

Leave a Reply to pkohonenCancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.