Josh Wills - Statistician, Former Director of Data Engineering at Slack #51
Biography
Josh Wills is a former engineer and Statistician on Slack’s Search, Learning, and Intelligence Team helping to build the company’s production search and machine learning infrastructure. He’s a recovering manager, having most recently built and led Slack’s data engineering team and before that the data science and engineering teams at Cloudera and Google.
Josh is a member of the Apache Software Foundation, the founder of the Apache Crunch project, and a co-author of O’Reilly’s Advanced Analytics with Spark. In May of 2012, he tweeted a pithy definition of a data scientist as someone who is better at statistics than any software engineer and better at software engineering than any statistician, and his Twitter mentions have never been the same.
Click Here –> For more information about tech careers
Episode Summary
“Statistics in school, at least when I was there, was very much focused on the problem. ‘How do we extract as much information as we can from a relatively small subset of data.’ Statistics at Google and Facebook was much more concerned with ‘we are drowning in data, we have an overwhelming amount of data. How do we not delude ourselves?’”
“If you want to be a mentee, there’s no work that you’re too good to do. Like the best learning average opportunities of my life came from situations where I was more than willing to roll up my sleeves and do things that are not sexy and are not cool.”
“I didn’t go back to engineering and I never stopped engineering. I was always engineering while I was doing this management stuff on the side. Recruiting and hiring, I enjoy. And I very much enjoy mentoring people. But I do not personally care about or tolerate sitting through a sprint review or doing a budget planning meeting or, god help me, sitting through a promotion process.”
—Josh Wills
In this episode we’ll cover:
- What are the differences in studying statistics in school versus working as a statistician at Google or Facebook?
- Why Josh chose to be a high-level individual contributor (IC) over management at Slack?
- How Josh worked a full-time job at IBM while getting his master’s degree.
Key Milestones
Selected Links from the episode:
The Manager’s Path - by Camille Fournier
Ep. 4 Goldman Sachs Saved My Tech Career, with Camille Fournier
Software 2.0 by Andrej Karpathy
Mike Gualtieri – Tech Analyst at Forrester (#38)
How to Take Smart Notes - by Sönke Ahrens
Josh’s famous tweet
Josh’s talk about rebuilding Slack search
Josh’s talk ‘how to play well with others’ about mistakes he has made during his career
