cancel
Showing results for 
Search instead for 
Did you mean: 

Fact or Fiction: Data Scientists Will Become Obsolete

ushukla
DataRobot Employee
DataRobot Employee
2 0 826

The Big Question

A few weeks ago, I was supporting a DataRobot demo with one of my customers. A few dozen new users had just completed the overall walkthrough of our core AutoML capabilities— preparing data, creating and assessing models, and learning how to apply predictions—when one participant hit the chat box with a particularly heady question: 

“So if DataRobot can do all this, what’s the point of even having a data scientist?”

Though we weren’t in the same room, I’m sure the questioner felt the same glimmer of worry that must have plagued folk legend John Henry1 and his fellow railroad workers. Is the steam hammer coming for my job? How long can I hold out before it finally bests me? And what was the point in working to become a top-notch data scientist if all my value can be automated away? 

Workers always welcome technology that helps them… until it replaces themWorkers always welcome technology that helps them… until it replaces them

It’s an understandable concern! DataRobot does many things that—until recently—only data scientists could do; at the same time, it’s democratizing the power of machine learning for a much wider set of analysts. Ultimately though, I don’t believe data scientists need worry about being displaced by the machine, at least not for a long time to come. 

First and foremost, it’s important to dispel the myth that AI will take over all of our work. To quote DataRobot’s own VP of AI Strategy Colin Priest, “AI won’t replace humans. AI will create jobs—and contrary to what you might think, those jobs won’t just be for computer geeks.2” Critically, the AI applications with the highest potential in the foreseeable future are those that automate the tasks that are easy for computers to do and time-intensive for humans—things like rule-based data processing, mathematics, and highly repetitive tasks.

What Should Data Scientists Do? 

Unfortunately, I understand why data scientists are so worried about AI replacement: they’ve been stuck doing menial work as a huge part of their job! A plurality (nearly 40%!) of participants in my session said they were spending 60–80% of their time on data preparation—all the work required to gather, clean, link, and validate data prior to any actual analysis. Seeing a suite of tools from DataRobot including Data Prep, automatic outlier detection, and feature engineering that will reduce the time required for these tasks tenfold would obviously ring alarm bells—“this is my job!”

Fortunately, there is plenty that data scientists can do with the efficiency gains with these tools. This suite of tools begins to push beyond traditional narrow AI into Augmented Intelligence, a new paradigm that, simply put, involves computers doing the things computers do well to allow humans to do the things humans do even better—with greater speed, confidence, and flexibility. 

Data scientists will always be needed to do three things that AI simply can’t do

  • Humans ask the right questions. Data scientists know that with a massive amount of data, anyone can start to find correlations, or build models to explain certain outcomes. However, it’s always critical to start with a clear question in mind. The types of work that organizations should ask data scientists to do should always be clearly-defined problems that directly inform an operational action or decision for the company. And since these companies, their customers, their partners, and their competitors are all humans, these priorities must be set by humans. 
  • Humans know that answers are subjective. No model is ever going to be a simple, unified rule that applies in all situations and for all problems. The old aphorism “all models are wrong; some models are useful” doesn’t even begin to scratch the surface of the subtle, intuitive work that data scientists do to unpack the value of a (wrong but useful) model. As Benn Stancil succinctly notes, an analyst’s job “isn’t to do the math right so that we can figure out which answer is in the back of the book; it’s to determine which version, out of a subjective set of options, helps us best run a business.” 
  • Humans tell stories. It’s one thing to generate a really robust model that can reliably predict an outcome. It’s another altogether to paint a picture to ensure that an audience of laypeople can understand the predictions. Humans are incredibly good at storytelling; it is in some ways the critical invention behind our cultures and civilizations.3 Data scientists just happen to be able to spin yarns with data features as protagonists and villains, teaching morals with models. 

Storytelling isn’t easy -- but humans are very good at it.Storytelling isn’t easy -- but humans are very good at it.

Designing a Springboard for Data Science

My hope is that data scientists who are reading this far have been nodding along, seeing the value that can be unlocked for their own productivity, passion, and peace of mind with the right support. Ultimately, however, reaping the benefits of AI and Augmented Intelligence isn’t just up to the data science community. Instead, it requires (very human) alignment on how best to leverage these capabilities across organizations: 

  • Executives need to be clear about what core strategic questions require analysis (not just reporting) for optimal decision making. Not every question needs to be answered with machine learning (though it might be more than you think!). It takes time and energy to build intuition around which questions are best answered by data scientists. 
  • Data science leaders and technical experts need to be capable of distilling these questions into answerable data science questions. An executive might have a clear vision and goal for an organization: how will an analytics team report on it? What metrics are concrete enough to be clearly tied to desired outcomes, and what variables might influence those metrics? What output must a model create to inform a decision? These interstitial individuals are critical to the smooth adoption of AI and help build a fully AI-driven enterprise
  • Data scientists themselves need to be considerate of the real-world human and operational outcomes that their insights will inform. Again, the hope for the future is that by reducing the burden of repetitive,low-value work for analytics team members, data scientists will be able to do what they do best: solve tough problems, and share the stories around those problems with the people that matter. 

Better together!Better together!

What do you think?

As a Data Scientist, are you feeling confident that you’ll only be better positioned with Augmented Intelligence? As a Business Analyst or developer, do you think Augmented Intelligence will aid you in machine learning, or are you still worried that jobs will be replaced? I’m looking forward to seeing your comments and sharing ideas with you! 


References

1 For those unfamiliar with the folktale, see https://virginiahistory.org/learn/historical-media/steel-drivin-man-john-henry-untold-story-american...

2 https://www.datarobot.com/blog/humans-and-ai-organizational-change/

3 https://time.com/5043166/storytelling-evolution/

Announcements
New to DataRobot? Check out all the resources to help you get going quickly! See the quick index for Knowledge Base Resources and quick index for Learning Sessions to find links to some great learning content.

Need a Tip?
DataRobot experts are putting together some helpful DataRobot usage tips for the platform, trial, features, etc. You can find these easily in the Tip of the Day board (under Read). Let us know if you've found a good one or have a good one to add!