Simon Willnauer - Cofounder of Elastic & Berlin Buzzwords (#35)
Biography
Simon has led Elastic core development for over five years and co-founded the Berlin Buzzwords conference on search, store, and scalability.
Simon is one of the main developers of Apache Lucene and a Lucene committer. He is also a PMC member and Apache Software Foundation member.
Click Here –> For more information about tech careers
Episode Summary
“The big common denominator between concurrency problems and other big problems is that they usually are not solved in front of the computer”
—Simon Willnauer
In this episode we’ll cover:
- The founding stories of Elasticsearch and Berlin Buzzwords!
- What is the future of search? Why is Java limited in its capabilities?
- Why the most challenging computer problems are best solved AWAY from the computer. Simon has had breakthroughs while running!
Key Milestones
Grant Ingersoll (00:19):
Welcome everyone to the Develomentor podcast, your source for interviews and content on careers and technology. I’m your host Grant Ingersoll. For those of you who are new to the show, we have two really simple goals. We want to showcase interesting people in tech across a variety of roles and we also want to highlight the different paths they took to get to that point in their career. To learn more about what we do or to hear other episodes, please visit our website, develomentor.com you know, one of the best parts of this show is that I get to interview people I’ve known for a long time and hear their inside story. Today’s guest is someone I’ve known from the Apache Lucene community for a good number of years where we’ve long worked to make that project what it is today. He’s worked for a number of years as a software engineer in search and security before going on to co-found elastic, the company behind elastic search. Even though our two companies that we cofounded often compete in the business world, we’ve stayed friends and shared a few beers along the way whilst swapping stories from the trenches of search, please welcome to the show. Simon Willnauer. Simon, great to have you here. Thank you Graham. Thank you. Thanks for introducing me. Yeah, thanks so much for joining me in. Why don’t we just start off by having you, you know, fill in the details on that introduction, spend some time on, on your career and how you got to where you are now.
Simon Willnauer (01:43):
Sure, sure. Yeah, so like taking a couple of steps back, I learned programming or got into computers very late. I think I wrote my first line of code when I was 22. I started to study computer science and two years later I happened to become a lucene committer because it was one of the projects that I was very excited about. So I was excited about open source. I found that a pretty cool idea, which was about 2005, 2006. And at that time I worked in, as you said, in a security computer security company. And we were looking at code most of the time from the other side in terms of reverse engineering and we had a couple of search problems and I used Lucene to solve some problems in that space. Not in, not necessarily what we do today with lucene and security. Like logging and security analytics. It was more on a micro level and from then on, yeah, I stick to it. It was it was a great community. There were a lot of people that I learned a lot from. I think that that’s also when we met. Yeah, that’s right. It’s more than 10 years ago, which is crazy by itself.
Simon Willnauer (02:58):
And after that I left security space after writing my teases in, got more into the the senior search space and natural language processing, all these kind of things. Worked as a freelancer. We work together on a couple of occasions as well back in the days and yeah. And in 2010 I started to kind of work full time Mmm. On lucene and two years later, chai, Ori, Stephen and I started to form a company around elastic search because it was a good product. That time was right. There was a lot of interesting need and development in the market for search in general. And yeah, I mean the rest is history company is, has been, has been climbing a steep curve and uyeah it’s great.
Grant Ingersoll (03:58):
And here you are today. Yeah. Yeah. I want to, I want to go back to that, that little bit of discovery of open source you know, I think one of the best things I recommend people often, you know, ask me career advise, et cetera. And you know as I mentioned to you before the show, my son is going off into computer science and you know, one of the things I always tell people is get involved in open source, right? Find a, find a good project and find a way to contribute. It doesn’t necessarily mean have to mean code. Doesn’t have to mean you go do the whole thing. You can just start, walk me through a little bit about that, those early days or how, you know, it sounds like it was kind of serendipitous that you found lucene, you were working on this other area, how to search problems, said, Hey, what’s out there, but walk me through kind of your open source journey and what it’s meant for your career.
Simon Willnauer (04:48):
Yeah, I mean it did to be honest. I didn’t know much about open source. I mean I understood, okay, Linux is open source and, and there was some people working on it and it’s great that it’s free. But the real beauty of open source just became obvious a little bit later in time. For me, from my perspective as a software engineer, it became obvious that I can work on whatever I want and I don’t need to work in a company that works on what I want to work like that is, that was one of the biggest drivers to me is I was in a security company and it was, it was interesting, I learned a lot of things about computers, but my real interest became a search at some point, but I didn’t need to work in a company that necessarily works in search.
Simon Willnauer (05:36):
And that is what open source allows me to do. Right. You can just work on whatever you want and get involved into it and work on the problems that you were interested in. I wasn’t the great natural language processing engineer, but I turned into somebody knows concurrency when you will and there was, there was space in, in the search world or in lucene for this and also learning a lot from other people about natural language processing. So yeah, that was one of the main drivers for me that I can, I can work on what I’m interested in. So my wife always was telling me back into this, you’re, you’re living with the, when you’re back towards us. Because it was sitting in front of my computer all the time and nobody was understanding why I was doing that. And I like, it’s interesting that that is one of the biggest drivers I guess. Yeah.
Grant Ingersoll (06:38):
Yeah. That ability to kind of create your own little world is both really amazing and scary at times because sometimes you can just get sucked down into this rabbit hole. I think your wife and my wife would get along just fine. Go back a little bit. Cause you said you wrote your first line of code at 22. Right. And you know, and I think for many digital natives these days, that’s where like wow, wait, I wrote mine at 10. For me personally, I think I was probably 18 I didn’t really get into coding until I got to college. And a friend of mine said, Hey grant, I think you would like this programming class. I’m kind of curious, like, you know, what were you doing before and how did, what was that moment of, Hey, I’m going to go try this code thing.
Simon Willnauer (07:23):
Yeah. [inaudible] it’s, it’s a good question. So I worked as a mechanic before I went to university and there was a little bit of programming involved in like computer numeric control machines and then like drilling and stuff like that. But it’s very, that was very basic as you could compare it to the assembly more than anything else I’m doing today. So I think I had this, maybe you call it mechanical sympathy for the, for the things and how, how I can move them around. At the end of the day, I think I’m a pretty creative hat, but when you look at my drawings, it’s not gonna cut it. And when computer science is one of the things where I personally thought I can be as creative as I want, because the sky’s the limit at some point, right? I mean, you learn the limits of a programming language and things like that, but you can be creative as creative as you want. And that made it kinda very interesting for me to get into that space. But also my, my entry point to that was very similar to yours. I started to study photography related stuff and things like that. And then we had this class in computer science and then I stopped doing all of [inaudible] things and just went into computer science, I guess stuff that that’s, that’s way cooler.
Grant Ingersoll (08:40):
Yeah, no, for sure. Yeah. I went to be, I was going to be an economist or an actuary. I mean I had a love for math right from early on. You know, I didn’t really think that careers for mathematicians were basically like economist or physicist or one of these other applied systems. Cause I had no, no interest in being the, you know, the, the math professor. So it’s super interesting like digging in. I mean I think one of the things of this show is that we often have these moments in our life, these people who introduce us to things and, and much of the show is about mentorship. But I’m wondering, you know, if you could highlight some of those moments, you know, people who have been particularly influential for you, if you want to give them a shout out, great. But where have the mentors been for you along the way? How have they helped you? And then I, you know, you and I have talked more recently, it seems you’ve moved into that partially into that role of mentorship. You know, walk me through a little bit of who are Simon’s mentors and what have been the impact.
Simon Willnauer (09:43):
Yeah, so that’s, it’s, it’s interesting. I think one of my biggest mentors is definitely Mike McCandless. The two of us were becoming Lucene committers at the same time, roughly. I think that was 2006. And Mike is, I call him the person who is the infinite patience, like so much patients. And he’s like, he listens to every idea. It doesn’t matter how crazy it is and then ask these really good questions, but is still at the same time he’s super up for crazy things. I like trying out something completely new. That was one of the main drivers for me. I don’t think I necessarily learned a lot of how to code from Mike, but I learned a lot how to think from him. And there were a lot of other people that forced me on improving my coding skills along the way. One of them was definitely Robert Murer is also a lucene committer. Learned a lot from him in terms of like strictness and how you write code and what makes a piece of code of good pieces of code.
Simon Willnauer (10:49):
Interesting enough, I think at some point I turned into this mentorship role as well, but it’s bi-directional. Like for instance, one of my coworkers, Leon Gronk, who became a lucene commiter, I still remember at the time that I met him face to face when Robert and I told him that he’ll become a lucene committer and he was 24 at that point. Now he’s in his mid thirties. And you know, I think I was a mentor for him at some point, but now he’s also a mentor for me. So it’s like a, it’s this, like people develop people, people increase their skills and you can learn from, from them. That’s one of the biggest learnings that I had is that you never, never forget. You can still be a mentee while, while you mentor
Grant Ingersoll (11:35):
Yeah, I mean I think that’s one of the best things about being a mentor is it is never a one way street. I mean, I think, you know, as you, as you get more experience in your career and you move into some of that mentorship, it is, is valuable to you. You’re, you’re learning different things for sure. But you’re still learning. You mentioned you threw out the word concurrency there. And for those listeners who are aren’t familiar, this is basically easy, you know, some really low level work around, you know, how do you make computers do multiple things at the same time? Right? And, and you know, I think Simon, you’ve spent a lot of your career on some of those really difficult low level engineering problems in search. You know, things like concurrency. That’s the hard stuff. Right? And so I’m kinda curious. So like how do you approach doing that inner loop stuff? How did, how did you prepare yourself? How do you like work through those kinds of problems where you know at the end of the day what you’re trying to do is juggle a whole bunch of different things at one time. Right. So how do you approach those kinds of problems? How do you dig in on them? How do you solve them?
Simon Willnauer (12:39):
Oh, that’s a good question. I have to differentiate a little bit. I think I’m approaching concurrency problems and a little bit different way than like architectural problems. Especially for the concurrency one. Yeah, I spend, I solved 99% of the problem not in front of the computer, but you need to be able to build a mental model of how your software works in order to figure out a concurrency, especially when it’s shared to concurrency like we do in Java, but that’s maybe too technical. But the big common denominator between concurrency problems and other problems to solve is that they usually are not solved in front of the computer. Like it’s this moment when you realize a problem and you sit and feel near your computer and you don’t have a good idea or sometimes you have a good idea and you try it out and cuts it. But most of the time that doesn’t work. And then I try to take a step back and for me it’s, it’s exercise. Like I do something else and my brain keeps on running in the background. That’s why I also tried to tell my coworker, it’s like you wouldn’t, you step away from a problem and you, I don’t know, you run your swim, you, I dunno, watch a movie. Whatever works for you. It’s work time, right? Because your brain does work there.
Grant Ingersoll (13:56):
Yeah, that’s, wow. I’m having this moment because I’ve so long felt that way. I can’t tell you the number of problems I’ve solved on the computer while out running or biking or I can even remember back in college, it’d be in the middle of hockey practice and the solution to some math problem I was working on would come to me. So the brain works in truly mysterious ways. So that’s fascinating that that you would have had that same experience. The other benefit, right, is like you can then bill hours while you’re while you’re working out, right. We won’t tell anybody that part right now.
Simon Willnauer (14:37):
Is this recorded? Oh, never mind.
Grant Ingersoll (14:42):
I love it. I love it. Whenever I have founders on, I just love to hear that founding story because there’s so often fascinating and there are so many compelling pieces to it and you know, elastic is like you said, but on this crazy growth curve and, and really help people who, who maybe weren’t familiar with search, get familiar with search. I’m curious if you know, to the extent you can, like what’s your founding story, what’s, what’s your little part of that seed that became elastic?
Simon Willnauer (15:15):
I think it all started with founding something else before that. In 2009, I attended, maybe it was 2008. I don’t know. I attended an Apache con in the U S together with a couple of other people from Berlin and we really wanted to have something like Apache con in Europe. And back then it wasn’t like it is today where originally every weekend there’s another conference on technology. But back then there was basically nothing except the Apache con. Mmm. And the Apache foundation off the foundation didn’t want to do another ApacheCon in Europe. And we were super sad about this. So we looked into helping and running the show, which didn’t happen for a couple of other reasons, but we found it another conference called Berlin buzzwords and Berlin buzzwords, first edition. I think it was the second talk in the morning. It was about like search store and scale, Lucene, solar, couple of other things like no SQL things and stuff like that.
Simon Willnauer (16:22):
The second talk in the morning, I promise you I was already wearing my fourth tee shirt because like running a conference is a tough job. There was a talk about the software called elastic search by a guy called Shay Bennon. And I was sitting in the front row and I was like, what is this? Like why is he not using solar? Why is he doing this? This doesn’t make sense. And then I talked to him and Shay is a super bright guy. I learned a ton from him and I think he learned a lot from me as well became partners two years later on elastic search. Especially because I’m, I was, I was very interested in getting more into the higher levels of search rather the low levels. Lucene is a library. Right. It’s like there’s not a lot of people go into that level. Solar Elasticsearch took a lot of the places where Lucene was used and getting into the higher spaces was, was important. Why did that not work in solar? But it’s probably a different story told on different day .in one sentence. I think that though, the way solar is architecture is not the way I would have done it and elastic search is more along those lines and it’s a taste. It’s not a, a statement on quality. It’s just different.
Grant Ingersoll (17:48):
You know, the classic thing of, you know, this is one of the beauties of open source, right? As you get to sit and watch how other people do things and then you can say, Oh, well I’ll do it this way. And then there’s many ways like these projects become mentors for each other, even though they’re, you know, quote unquote competing, right, of, you know, Hey, I wouldn’t do it that way. And then solar says the same thing. And then along comes some new project that says, Oh, I want to do this completely different than both of these. And, and, and like you said, it’s, it’s taste, not correctness. Right? It’s, it’s an opinion. Right. Yeah, of course. That’s really cool actually. So you, you talked a little bit about buzzwords, you know, and, and that, that conference in its own right, has become a thing, right? I mean, it’s one of my favorite conferences to go to. And you said running a conference. I was on my fourth tee-shirt by the second hour. You know, share a little bit like what goes into running a conference. We’ve actually had some prior guests. We had Todd Lewis on earlier in the season who it is a conference producer and he’s talked a lot about what goes into that. I’m curious as to kind of what was your experience in putting on a conference?
Simon Willnauer (18:59):
Yeah, so luckily we found a producer for the conference, so I think they were occurring a majority of the loads and running the conference, including Isabelle, who was by far like we were three founders of the conference, but Isabel was carrying the most of the load, that’s for sure. So also over the years being involved in content and like reviewing proposals, bringing in ideas and things like that. And, and also getting sponsors on board. I mean, there’s so many moving parts. It’s like when the conference is usually in June, mid June, early June, like the conference preparation starts two months later. Yeah. That’s a year, a year worth of work. I mean, in the meanwhile, there’s like a list of things that you need to do, get the conference going, what do we need? How much do we need a bit? These kinds of things.
Simon Willnauer (19:54):
But in the first, like when they did us the first time, it’s a massive guess like, do we even break even? Right? Like what, what did we want to do? We didn’t want to make the tickets very expensive because we wanted to attract open source developers and all of them don’t get paid for that, for that ticket or don’t even get paid for the trip and things like that. We wanted to also enable students to come and do the students don’t have 1500 euros to pay for a conference. So how can we make this going with with 250 euros along those lines for tickets? There’s so much thought you have to put into a conference like this and it’s not getting easier, right? It’s like back then, like things like code of conduct wasn’t a thing we probably should have, but it wasn’t. I’m sure we didn’t have one, but we have one now, but it becomes very, very complex. There’s so many things like you building, you building a little society for three days, you need to make sure that it’s going and running. And everybody’s happy with the food. Another one is, and restrictions. Then you have, you have different people from different religions, from different countries, from different backgrounds. You want to build a place where everybody feels free to speak up and discuss things. And that’s very, very difficult.
Grant Ingersoll (21:19):
Yeah, that’s, that’s so well you, you hit on all the things there, right. With a conference, I’m often amazed that they even ever get put on, given how much financial risk one has to take on to put them on. But I mean, it speaks to, I think your, what you ended up there with, which is you want to create a space where people can come together and share ideas and meet. And that at the end of the day is, is super powerful. Right?
Simon Willnauer (21:47):
Yeah. I mean, I think that producers, and like I’m, I’m not involved for, for a while now in, in the conference itself I’ve been to every of them and which is great and you’re seeing the same people over and over again coming back to it. So I think we’ve done something right here. Yeah, for sure. Yeah. Or it got to a very nice space and size. It’s not too crowded. There’s a very interesting crowd from all different companies. And this is also an, I meet all the Lucene committers there and they work in all different companies. They work at Apple, they work at lucid, they’re working on elastic, they work in Bloomberg. They’re work for libraries in Denmark. Yeah. And what have you. And it’s, it’s great to meet all these people together. It pays back a lot.
Grant Ingersoll (22:33):
Yeah, for sure. And I’m on the program committee, so this will be a plug for Berlin buzzwords as well. I mean, the other just really quick thing on Berlin buzzwords. This is simply like Berlin in June is absolutely amazing. So you know, here’s the little shout out for just like, you know, even if you didn’t learn anything at all at the conference, which you will, you will learn something there. Just being in Berlin and June is an experience everyone should have. So I want to ship.
Simon Willnauer (23:04):
We had we had a lot of really good weather. I think there was only one conference where we wasn’t warm and like, believe it or not, the other day my daughter came to me, she’s nine and she’s like, Hey daddy, there a lot of videos of you in the, in the internet. So what’s some, but there’s one where you not wearing shoes on stage, what’s going on? And I was like, it was one of the buzzwords. Where’s so warm? Like is insane.
Grant Ingersoll (23:34):
Yeah. Fantastic. And what is shift gears a little bit, Simon. Again, to the extent you’re comfortable. I mean this is, this is delving into, you know, some of the more challenging parts of anybody’s career, and I think, you know, one of the things I I like to do in the show is I think many times we project out that, you know, career paths are this, this amazing, you know, everything’s upward up into the right and, you know, to the extent comfortable. I mean, you know, what’s some things that you’ve had to overcome in your career? Like some of those key inflection points that you look back, you’re like, wow, I really struggled there, but I’m glad I got through it. And again, you know, I want to make sure you feel safe and, and talking about that, but just delve in a little bit if you will, of some of those key inflection points where you said, Hey, I gotta go do this or I’m struggling to get through this decision.
Simon Willnauer (24:22):
I mean, when, when we started elastic, I kind of had to switch gears going from lucene into Elasticsearch development, which was kind of driven by making a decision of founding a company. And then also like not, not spending too much time on lucene and more time on the elasticsearch, getting to know the software. Obviously I haven’t written it from the scratch, but at some point we shared so much the work, the, I took over the, the development for elastic search itself. That was a decision that I made very happily because there was, it was a new challenge as you said, and it’s still related to it, to lucene back then. Over the years, like you get, you get used to it, you could use to challenges where young people come in and say, Oh, it’s so great to work on all these search problems, but after doing it for 15 years, it kinda gets like, yeah, I’ve seen that before.
Simon Willnauer (25:27):
Yeah. Yeah. And in, in this this was always a moment for me to think about like, do I need to change something here? And it always depends on yourself. Like if you, if you feel happy to be in a mentor position and be the one that gets asked for things, but then somebody else does it why would you want to change jobs at that point? Right. That has changed for me. I’m doing less myself, but trying to get other people to do more and that’s, that’s exciting. Even though it’s hard because I had to learn that if somebody else does it, somebody else does it differently. That is tough game sometimes. That’s okay. Right. I mean, it’s still a valid solution. It’s just not the one that I would have done. And, and that that’s that is one of the things that I didn’t know I had the time and it just occurred to me that it is the way because I stick to something a little longer than probably somebody else or would it stick to it.
Simon Willnauer (26:33):
And I think my biggest advice for careers and also for relationships maybe you know, it’s like don’t drop it on the floor toured. Like it’s, it usually pays off if you stay in it a little longer and then you can make a conscious decision and even though like you’re looking at it from a different perspective. I’m not saying there is some kind of harassment going on or something like really wrong in the job. Of course you need do, that’s all what I’m saying. Like what I, what I’m saying is this working as a software engineer, it was like it was just seasonal. A job like there is, there’s a very highs where are you very productive and very creative and everything’s awesome and you shipping good features and you making progress. But then there’s also going, it’s going down incident is like you’re not productive, you’re not shipping features, all these things.
Simon Willnauer (27:29):
This is all boring. I’ve seen all this before. Why am I doing this? But you know, there’s, there’s, there’s an upcoming after that, then I think you need to learn to how to deal with these ups and downs. If that is, that is kind of a, it applies to everything at breast job to relationships, to sport, to training. I think patience is something that I learned over the years. Like pull myself out of the situation at all. Right. This is not so exciting that I want to stay in here, but let’s, let’s look, let’s put a horizon there, see what it is in three to five months then if it’s still,
Grant Ingersoll (28:09):
Yeah, I think that’s a really key piece there. And you hit on something really interesting that I know I struggled with, which was, you know, as a founder, like I, I think as a founder in particular, although it does apply other parts in your career, but you know, as your company grows, you effectively have to reinvent yourself along the way, right? Cause in the early days you kind of do everything and then you start to hire somebody who, who does a little part of what you do and then you realize, Oh wow, they, they do that better because they get to focus on it. Right? And so you’re constantly in this mode of effectively firing yourself. It was interesting, a good friend of mine, you work 35 years for the same company and he has this funny line. He’s like, I’ve worked here 35 years and I’ve been fired five times. Meaning like he’s, he’s changed jobs, I think five or six times in that 35 year career there. So it’s, it’s a fascinating to think about. You can, you know, like what you’re getting at is you can reinvent yourself even within the same company. You don’t have to necessarily move on, you just have to be open minded to trying something new. So I think you hit on some really key pieces for people as they begin their career to think about.
Simon Willnauer (29:23):
Absolutely. And also as a, especially when you look at it from a founding perspective, like when you found a company, I can tell it is in hindsight, it’s like I didn’t know, your job isn’t to make yourself obsolete if you can lean back. And so like, all right, I’m going to not work here for six and nothing is going to change. That’s success.
Grant Ingersoll (29:47):
What a good succinct way to say that, Simon.
Simon Willnauer (29:52):
It’s annoying sometimes. Sorry know it’s very annoying to like, you start to feel like aw these guys don’t need me anymore. There’s so many folks in my team that do my job now and that’s, but that’s great. Right. Being needed is absolute success and I can appreciate it now.
Grant Ingersoll (30:12):
Yeah. But it does take a little hit on the ego at first. I know I, I struggled with that too. But ultimately, yeah, that’s that. And in many ways [inaudible] what is ultimately allows you in the company to be success. And I actually had a good friend of mine had to tell me that like Grant, you know, like these people are helping you succeed, like, you know, let it go. That was a pretty pivotal moment there, you know. So speaking of success and search, I mean, you know, let’s shift gears and kind of look at the here and now and forward and what do you see, you know, you had this break early on, got into search and I think, you know, like myself, this has been a fascinating career. Just really digging in on how do we help computers understand content, you know, help them understand language. What do you see as kind of the here and now and starting to look forward challenges in the search space and the natural language processing space?
Simon Willnauer (31:08):
[Inaudible] That’s a good question. I think there’s a couple of challenges especially forward. We’ve seen and that’s unnecessarily down to every solution. I think that those things written in Java and Java programming language has a couple of things that prevented us from making progress in the search space. Like just the language capabilities is not being where they could be. And that’s something that we all also from elastic search perspective, we started to look into contributing to the JDK, bringing it more forward, bringing it, you know, getting, getting the the functions and intrinsics and to make better progress. Like as something as simple as like the functions on a, on a memory. I want to be able to tell the operating system, how are we accessing this data and if that would make a massive difference, we would have more capabilities in the programming language. Because I used to, I’m still convinced that a managed programming language, like Java is the right thing for new Lucene.
Simon Willnauer (32:17):
And once in awhile somebody comes up and says like, let’s rewrite Lucene in X. And I’m like, yeah, X is not our problem. It’s not our problem. The problem is that we’re stretching the limits of our ecosystem, the algorithms of the programming, language of the execution speed. And all of these kind of right directions. We cannot waste the time of rewriting something Lucene in a different programming language, the ecosystem’s so big it wouldn’t be worth it. From that perspective. We also using algorithms that are really, really old. That’s right. It’s not necessarily that they’re bad, but most of the research and you know that as good as I am has been done in the eighties nineties and early two thousands no?
Simon Willnauer (33:01):
How long did it take to move from the breakfast space model to being 25 for us, the challenge as I personally see for searches is. How can we efficiently incorporate something like new developments in artificial intelligence and machine learning into something that we can still understand. The beauty of existing search solutions is that it’s relatively straight forward to tell why some boat something is better than something else in terms of scoring. All right. Once we, once we move into the machine learning space, that pours a lot. And that is one of, one of the big issues that I personally see is that we need to work on is less on the making something significantly faster. I mean, it’s always good to have better execution, speeding, less computers from that perspective, but it’s the way to understand search. Just I think one of the biggest challenges because the search is everywhere these days. Every website has it in one way or the other. Not everybody scores, but as soon as you get scoring involved, it’s an enormously tricky and you either have a lot of domain knowledge to understand what’s going on.
Grant Ingersoll (34:12):
Yeah, for sure. I often tell people, I mean, I think we are moving into the era of probabilistic data stores and that is effectively what a search engine is, right? This ability to say instead of data being in or out of a set, based off of a relationship that you have this notion of probability attached to it and search engines have been built for that. And that is the future I think of all applications. Right. That’s, that’s really awesome Simon. I mean I think thanks for sharing so much deep knowledge here on, on these things. You already answered one of my questions, which is what is your best advice for somebody’s career? So let me skip through that and I’m always curious. I love to learn from other people and one of the best ways you can do that I think is true reading through podcasts, et cetera, order or some books or podcasts that have been particularly helpful for you over the years, you know, on on learning and programming, learning how to be better at your job, et cetera.
Simon Willnauer (35:12):
I tried to understand better how computers work in general, like from from a lower levels perspective and that helped me along the way. I’m making higher levels decisions and I think that’s something that that applies in general. If you want to really make efficient decisions on a high level, you need to understand the lower levels, not in every detail, but you know, try to make your into things like the, you know, how does, how does memory work and how can I officially access it? Like how do I need to do lay out data? And then the, the other thing is like I started to read papers and I started to judge papers. Mmm. In a way of like how practical are they for me, how practical is it to actually implement it? Then how crazy is the code going to look like if I do that? Because this is something that I see a lot of people jumping into too quickly. Aw this is a great solution that’s do it rather than like prototyping and seeing like how is it actually look like if we implement that and what are the implications?
Grant Ingersoll (36:25):
Well that makes sense. I mean, I think for me, one of the, you mentioned concurrency earlier and I, I know one of the books that really helped me was Brian gets his concurrency and in John Green books, you know, and I had worked on distributed and concurrent systems for a long time, but like that book just laid it all out in such a clean and clear way and I, you know, so maybe, yeah, it’s good to, good to hear that you’ve, you found that one helpful as well Simon. It’s been so great to have you on so many interesting pieces in there. You know, founding story like open source conference organizer, where can our listeners follow you and learn more about you if they want to dig in a bit more on your career or perhaps someday become a mentee as well.
Simon Willnauer (37:11):
I use Twitter regularly. I have a very cryptic Twitter name. Let me go and figure it out.
Grant Ingersoll (37:19):
Yeah, we can link that up in the show notes. But so Twitter is the main place for to connect with you. Twitter is a good one.
Simon Willnauer (37:29):
The Apache mailing, this is it. Once in a while. Github is a good place to I’m still involved into coding. I pay attention to it, not to everything. It’s sometimes it’s hard for me to see how big something like elastic search has become just elastic search elastic, the company that’s also insane, but they, the elastic search it’s so big I can, I can not drink from the fire hose. Its too much!
Grant Ingersoll (37:57):
Yeah. Well if you think about the sum total of impact on the software world and the broader business world of lucene, this little gem of a library that this, this guy Doug cutting way back when wrote as a way to learn Java. It is just absolutely insane. So yeah,
Simon Willnauer (38:21):
I don’t think a lot of people even understand that. How much came even out of the scene. That’s like how dude was part of this scene, right? There was a notch in the to student file system back then and resolve the same. Luene was the top level project for all these things.
Simon Willnauer (38:36):
Yeah. There’s so many spinoffs side of in companies and jobs and careers and solutions is crazy. Yeah. Like I said, it goes back to find an open source project to you like, right. Hey Simon, just just wrapping up. I want to thank you again so much for joining us today. I really appreciate you taking the time to join us to share your story with our listeners, so so thank you.
Simon Willnauer (38:58):
Thank you as well. It was great chatting and it’s always nice to look and take a look back. Yeah, for sure. I smile. I smiled on a couple occasions.
Grant Ingersoll (39:09):
That’s nice. I’m glad. At the end of the day I could, I could make you smile and for our listeners, as always, if you like the show, we’d love for you to subscribe to our podcast on iTunes or Google play or Spotify or whatever your favorite podcast app is, a pretty sure we’re on all of them.
Grant Ingersoll (39:26):
You can also visit us at develomentor.com to hear older episodes as well as find other content on careers in technology. You can of course find the show notes for this episode there as well. Most importantly, if you like the show, please tell your friends. Referrals are really the key of any podcast. We need you to spread the word. We would love to hear guests that you would like to have on this show. And of course, we love feedback. So leave us a review, leave us a rating. And if you also want to get in touch, feel free to drop us an email. If you think, do you know somebody who’d be a great guest or you’d be a great guest? Reach out to us at podcast@ develomentor.com otherwise, we thank you for joining us. Once again.
Additional Resources
Selected Links from the episode:
Java Concurrency in Practice - by Brian Goetz
