The Use Case Podcast: Storytelling About Karat With Jeffrey Spector

Music:  00:02

Welcome to RecruitingDaily’s Use Case Podcast, a show dedicated to the storytelling that happens or should happen when practitioners purchase technology. Each episode is designed to inspire new ways and ideas to make your business better as we speak with the brightest minds in recruitment and HR tech. That’s what we do. Here’s your host, William Tincup.

William:  00:24

Ladies and gentlemen, this is William Tincup and you’re listening to the Use Case Podcast. Today we have Jeff on from Karat. That’s with a K, K-A-R-A-T. And we’re really learning about the business case for the use case, or sometimes people use the cost benefit analysis, or ROI, things like that for wise prospects ultimately become customers of Karat. So can’t wait to learn a little bit about Karat, and before we do so let’s just get into introductions. Jeff, do us a favor and introduce both yourself and Karat.

Jeffrey:  01:03

Great. Well, it’s a pleasure to be here. Thank you so much for having me. I’m Jeffrey Spector, President and co-founder of Karat. Karat is the world leader in technical interviewing and we’ve created the Interviewing Cloud, which is an always on, scalable and consistent human plus tech solution for conducting engineering interviews on behalf of clients worldwide. Just by way of background, I started my career in China at a wireless technology startup. I was employee five, it scaled up like crazy. We grew to a couple 100, went public on the NASDAQ and we dealt with a lot of the same issues that Karat’s designed to solve. We were hiring up a lot of engineers in a very rapid amount of time, and really had no idea what we were doing.

Jeffrey:  01:46

Then I had to [inaudible 00:01:47] through finance for a little bit, and I got really passionate about where business and social good intersected. And so I joined the Gates Foundation. I was there for about six years. The last four, I was Melinda Gates’ chief of staff, which was an incredible opportunity. And one of the areas that I really got passionate about was their postsecondary education work in the United States. And we were really looking at how people got jobs and how broken that process was. That candidates were being ghosted. They never got any feedback from interviews or hiring processes, and really just didn’t have a straight line to getting a job, even though they were putting in the effort and the resources. And so I had experience on both the client side and the candidate side, and that led to forming Karat with my co-founder Mo.

Jeffrey:  02:33

And just stepping back, the problem that we saw is that every company in the world is turning to software. No more true than now in the pandemic. And there’s this multi-decade shortage of software engineers, and demand is outpacing supply. And because there’s not enough engineers, there’s also not enough engineers to interview other engineers, nor are there enough engineers who know how to interview well, because most people are not hired or promoted for that skillset, they’re hired because they’re good engineers, not because they’re good interviewers of other people. And really that happens because it’s nobody’s full-time job to make interviews smarter or better. And so we made it our job.

William:  03:11

I love this on so many levels. Let’s just start with the basics. What do we get right and wrong about technical interviews? Let’s just get the audience up to speed with current state of technical interviews.

Jeffrey:  03:29

Well, I think in general what people don’t necessarily do right, let’s start there, is that there are a lot of times they’re assessing for things that don’t matter in the process. That it’s not actually relevant for the job at hand. So we always emphasize, you really first have to identify the competencies that really matter. And then the actual craft of interviewing. The interviewer tends to be the most influential piece of the interview, which makes sense, right? It’s the person who’s actually guiding that process. And so how you give hints, how you encourage people, they all matter to the performance of the candidate. And so I think what’s happening today is that a lot of times you’re not actually assessing for the true quality of the candidate, but it could be that interviewers are coming in there with an agenda. They might want to prove themselves to be smarter than the other side. They may have biases, conscious and unconscious, that they’re entering into that interview with [inaudible 00:04:19] result in the best hiring outcome for their companies.

William:  04:25

First of all, I agree with everything you just said. Let’s parse that a little bit. So skills like, let’s just take something that people would see probably every day, DevOps. How do you make that easier for people, either that are technical or even that are not technical recruiters, hiring majors, sources, et cetera. How do you bring them over to understand what is and isn’t DevOps?

Jeffrey:  04:57

Yeah. I mean, I think this is where it can mean different things at every company. DevOps at one company can mean dramatically different things than it could at another company. And so a lot of what you’re trying to do is actually ask the question of, what actually are the competencies that you’re hiring for? And part of what Karat does is actually play a bridge role between talent acquisition and engineering. So a lot of times our main customer is the engineering leadership. And they’re actually coming to us and saying, “Hey, we want to actually hire for these competencies.” And how do you actually translate that back to the recruiting organization? So, a lot of what we do is, one, we’ll just sit down and understand from our clients, “What do you mean by this?”

Jeffrey:  05:38

Let’s take the job descriptions, break it down, what are the actual competencies you hire for? And then we actually integrate with all the applicant tracking systems and we can see what they’re hiring for over time. So a lot of times what happens is you say, “Let’s just take a backend engineer.” You say you don’t care about time and space complexity, but then half the reason why people are getting rejected on the onsite loop is time and space complexity. So then we’ll say, “Okay, either we need to fix the part of your onsite process where you’re assessing for that, or we need to bring it forward into the initial engineering screen so that it’s an efficient process.”

William:  06:12

So just so the audience understand. When you’re saying assessing, are you saying testing? Again, the breadth and depth of Python. So I’ve had a year of Python experience, but I worked on 15 different projects, or I’ve worked on Python for a year on one project. When you assess for that or obviously when you help your customers assess for that, how do they know what the breadth and depth of just those competencies are?

Jeffrey:  06:49

Yeah. It depends on what you’re actually going after, but you can do things like have a section of your interview that really is digging into their past projects. And then it’s what are you actually looking for in that? Is it the difficulty of the project? Is it their leadership on that project? Is it specific skill sets you’re looking for? We tend to believe also that the work samples are the best way of actually proving out whether or not someone has a demonstrated skill. A lot of our interviews are pair programming, there’re things where people are fingers to keyboard, actually showing you what they can do. And that levels the playing field as well, where you’re really assessing for where are they on a spectrum of quality in terms of that competency, and you’re not factoring in other outside forces as well.

William:  07:37

Thank you for explaining that. So when we talk about competencies from the engineering perspective and they list the things that are important to them, how do we prioritize that, or how should we prioritize it, or how does Karat help them prioritize? That laundry list, they put down 15 things. Well, trying to find all 15 of those at depth might be relatively difficult and that’s a bit sarcastic, but how do you get them to focus in on the most important competencies?

Jeffrey:  08:14

Yeah. You’re actually hitting on a really important point, also for inclusivity, because there’s all this research out there that shows that the more requirements you have on a job, the less diverse and underrepresented talent will actually apply to your job. So narrowing that down is not just good for the interviewing process, it’s also good for your sourcing and recruiting as well. It is a process that we actually go through with our clients, where in general it would be, what is it that you’re actually looking for in there? And then you do have to actually talk through, because you aren’t going to be able to assess everything. It’s helpful that the upfront interview, and you ask a question of like, is this a screen? And really what we are is a human to human interview powered by technology.

Jeffrey:  08:58

And we really believe that hiring is a fundamentally human affair and does require the interplay and feedback between people, between engineers. And you can learn a lot by that interplay. But what we’re trying to do is actually, because it’s only 45 minutes or an hour, you can only cover so much. So we’ll talk through, if you had to choose between these three or four competencies, which one would rise up and why? Again, a lot of what Karat does, is review the data on an ongoing basis and surface up what competencies are being assessed for versus which ones you’ve said you wanted to assess for, and then we keep adjusting and aligning to make sure that’s efficient.

William:  09:38

Do y’all have a play or a role in job descriptions and things that you get either on the career page or not?

Jeffrey:  09:47

So it’s not a major focus of what we do. We do actually digest all the job descriptions. And then we will actually compare them. To your DevOps example, we will digest your job description, and then say, “What are the terms? And what are the things that you’re emphasizing on your job description relative to all of our other clients on their job descriptions?” And that actually helps kick off that alignment process, because if you’re leaning into security more than someone else, we can then say, “Why is that?” And we dig into that specific competency area, “Why is it important to your organization? Why should it be in the interview process more so than something else we could use that valuable time for?” So that benchmarking helps us to really nail the efficiency of your process.

William:  10:30

How much have y’all seen is comp being a part of this assemblage of competencies that one puts together that y’all validate and consistently validate for folks? There’s prices next to that. Each of those competencies is just like Lego bricks, there’s comp that’s tied to that. How do you help, or do you play in the assistance of making sure people understand what things cost?

Jeffrey:  11:08

Yeah. I think really what we help highlight is where are you efficient in your hiring process? It’s interesting because Karat is a consistent part of your process. In the world that exists today, every part of the process, from the upfront interview to the onsite loop, to the recruiter screen, tends to be inconsistent. Every conversation can vary in a big way, but because Karat is doing a very standardized interview at the front end, it allows us to shed a light on the rest of the entire process. So we can say, “How high quality is your source relative to market? What is your onsite to offer ratio for this level of candidate?” And also, “Are you closing this level of candidates?” So really what will help identify is, maybe you’ve told us that your close rates were really high, but then when we look, it was people who didn’t perform very well on the upfront assessment.

Jeffrey:  11:58

So now it’s actually, you’re closing, but are you closing the candidates you want to close? And maybe it’s lower at the top tier. And so we can really highlight, if you’re not actually closing the people that are doing well on Karat, comp would be one thing that you’d want to look at. And we do help think through, “What is your comp package?” It’s not a major focus of Karat, but we will do an analytics on that as well, to help think through, what is the right amount? Because even as you think about compensation, the amount of time spent in the interview loop is enormous. Your most valuable asset right now is your engineering time and your people’s time. And so, if you can increase close rates by five or 10% by offering a sign on bonus, that’s probably worth it because it’s actually going to save you money and get you to your hiring goals faster.

William:  12:49

So for the audience, technical interview turn-ons and turn-offs. What do engineers abhor in the interview process, we’ll go through processing questions and just all of that, and things that they love. Just things that you can just tell that you’re going to win talent. If you go through these types of process to ask these types of questions, walk the talent through this type of experience, and you’re going to lose if you do these things.

Jeffrey:  13:27

Yeah. So I’d say a couple things. One, we’re in a candidate’s market. I think that’s pretty indisputable. And it’s only getting more in favor of the candidate over time. And so what does a candidate want? They want to move fast. They want to move quickly. They have other opportunities. There are other offers. And our data shows for example, first of all, they want flexibility also in when they can interview. So we offer interviews 24/7, we see about 50% of people interviewing on nights and weekends. They don’t want to be on your schedule, they want to be on their schedule. So you’re just thinking of the candidate as a customer, as opposed to somebody who’s just going through your paces, is really important. We also know speed matters. So what’s interesting about speed, is our data shows about 15% of our candidates interview within two to three days of their invite to Karat. And the ones who are taking longer, it’s their choice that they’re taking longer. But what’s interesting is those candidates are twice as likely to get hired by the company than people who interview even one week out.

Jeffrey:  14:29

Because they are the people who potentially have other offers. They’re the most prepared for the interview. They’re the most excited about your company. So you need to build a process that allows those candidates to move at the pace they want to move through the process. Another thing that candidates really love, is how do you de-stress the process? And how do you actually ensure that they are best equipped to succeed? So one would be being very transparent about what you’re going to interview them for. I don’t think a lot of people share really specific details on what you’re going to be interviewing them for. And then also doing things, and we suggest this to every company, but early in our history, candidates would come back to us and there’s an ability for them to write feedback after every interview and say, “God, I could have done better. That wasn’t my best showing. Can I have another shot?” So we created something called a redo interview, which is a no questions asked second interview for people that they can just take another interview with us if they feel they weren’t doing their best.

Jeffrey:  15:27

And what’s fascinating about that is that it increases [inaudible 00:15:31] by about 20%. So we have about 20% of candidates that take us up on that offer. And the efficiency of the process for the redo interviews is almost identical to the initial interview. On top of that, there’s really two types of talent that are helped by that. One is senior talent, so talent who may be out of interviewing practice and just need to remind themselves of how to interview again. And the second is underrepresented talent, who are less familiar with the interviewing process. So for them, the ability to go through it once, then go through it again, we see about 60% of underrepresented talent do better in their second interview, which is a disproportionate amount than non underrepresented talent.

William:  16:13

Questions that technical talent ask, we’ll do some table stakes, but also maybe even some of the more innovative things that you see in the marketplace. Years ago I did a study on Millennials, and at the time they were asking what’s next? There’s more or less a internal mobility. How are you going to reward me or praise me, which was total rewards. And how are you going to make me better, which was basically training and development. And again, that’s stated, but with engineering talent, I’m sure that there’s routine questions that they have when they’re talking with recruiters and hiring managers.

Jeffrey:  16:58

Yep. Yeah. And you’re asking what are those questions?

William:  17:03

Indeed.

Jeffrey:  17:04

Yeah. I don’t think it’s far afield from what you just said. Most people, engineers right now, to your question about comp, they’re getting comped at a pretty incredible rate. So compensation isn’t necessarily the number one factor that they’re looking for. It’s still an important factor obviously. But they are looking for what problems am I going to be solving? When I get there, isn’t there an exciting problem that I am really interested in solving? And then what are the growth opportunities when I get there? One of the interesting things about interviewing is, it’s treated today like a transaction that just goes and dies when the interview ends. One of the things we’ve been thinking about a lot is, how do we turn it into an asset that lives on and has value after the fact? So even the fact of when you first do an interview, you’re usually getting interviewed by six, seven people, maybe five to seven people. They’ve spent seven, eight hours assessing you.

Jeffrey:  18:00

And then when you start your job, nobody talks to you about what the assessment was. You had literally seven or eight of your colleagues who spent all this time thinking about where your strengths and opportunities for growth were. And nobody actually comes back to you to tell you that and sit you down. And so we always are like, the first day of the work or somewhere in the first month, you should sit down with your manager, and walk through that. So you at least know as much as other people at the company about what you’re going to be doing. So stuff like that, where if you can indicate that you care. If you even share some of that feedback in an interview for example, of where somebody’s strong and not, that gives them an impression that you care about their development and that you’re going to care about their growth path. And that really, really helps in closing candidates.

William:  18:46

So now, or even being pushed into the future, do you see Karat being used as a internal mobility tool and/or even an outplacement tool as well?

Jeffrey:  19:00

Yeah. I mean, I think for us, we clearly are thinking about how do we help in the entire developer life cycle over time. I think first we’ve tackled the assessment and interviewing piece, and there’s still a lot of innovation there that we can get into, things like hiring for potential, giving feedback to candidates, there’s a lot of innovation. We’re seeing a lot of the same candidates come through Karat for multiple clients. I think one of our candidates has come through 10 times for 10 different companies. And so what do we do with that? Candidates who do well, can we expedite them in the process? Why do they have to keep demonstrating the same skill sets over and over and over again? That’s just inefficient.

Jeffrey:  19:46

Candidates who are on their path to doing better. That same candidate who had done 10 interviews, what we’re seeing is that the first four didn’t do well, but then the last six has steadily grown. And so this hiring for potential is a real huge area of innovation where we can not just look at an interview as a point in time, but actually a series of interviews that people are getting better and better. So there’s a lot of innovation in the interviewing and assessment space. We do want to be in the world where we’re helping engineers, how they get a job, how they can assess for a job, and then how do they grow within their career.

William:  20:24

Yeah. First of all, I think it’s going to be important for them. So it’s not just a market opportunity for Karat, but I think it’s important for that talent to have a concierge that just moves along with them through this process and helps them both at their company and with other companies. And again, that’s that relationship that just carries across. Your customers are going to pull you there, so you don’t have to-

Jeffrey:  20:50

[crosstalk 00:20:50]. They are for sure.

William:  20:52

You don’t have to give it a whole lot of thought. They’re going to be asking you if you can do this, you’re going to say, “Well. Yeah, we can.”

Jeffrey:  21:00

And we are getting those requests already. For any startup, there’s the balance of how much do you move beyond what you’re doing right now, and really perfect that, and scale it? And how much do you enter into new products and new opportunity sets? And so we’re constantly debating that internally. But there’s so much room up and down the process to help out, especially because once you understand a candidate, and you can keep obviously growing that understanding over time, but once you understand a candidate and you understand the company, there’s real magic in sitting in the middle of that process.

William:  21:35

That’s true. That’s true. I love that you are recycling and getting through analytics or getting the insight into the separation between what they say and what they do, so that you can feed that back to them and then they can rightsize that. So I love that. Two fun questions, big questions, but fun, is let’s take the audience into the demo for just a second. When someone on your team, or yourself, shows Karat to someone that’s not seen it before, what do they fall in love with?

Jeffrey:  22:15

Yeah. Really it’s when they start actually seeing the interview. Because in our platform, interviews are recorded and things. So the minute they get that first interview, there’s hundreds of little things that we’ve done to optimize what that interview experience has been. And so when you go in and you actually watch an interview, and over time they’re not watching the interviews, they trust the results and we gain that trust and then you move really quickly. But for the first interview or two, it’s really that magic, the interplay between the two engineers. Here’s a great example, and one of the things we’re looking more deeply into is, how do you inject humor into an interviewing process?

Jeffrey:  23:01

So we’re working with the authors of a book called Humor, Seriously, which basically says that when you add levity to it, what otherwise would be a stressful situation, you get better outcomes, or true performance and things. So we’ve noticed that our interview engineers who do, and it doesn’t even need to be a comedian type of joke, but it’s a, “Hey.” Just humanity in the interview of, “What’d you have for breakfast this morning? Are you okay?” They’re not asking far afield questions, but just to warm people up and get them in the process when they’re struggling. I think one of our interview engineers said something like a candidate whenever they ask, “Do you mind if I think about this?” He comes back and says, “Yeah. We even encourage thinking in our interviews.” Something like that, where you’re just bringing levity to the situation.

William:  23:52

[crosstalk 00:23:52]. Because it’s raw with anxiety on all sides. Seriously, sources, recruiters, hiring managers, candidates, executive, everyone’s got anxiety.

Jeffrey:  24:00

A 100%. It’s a very stressful situation. There are huge consequences for both sides. An interview is an incredibly important moment in time and candidates, and interviewers to your point, deserve respect in that process. And they deserve it to be a professionalized experience where every part of that process is optimized. When we started this company, it’s amazing that there’s billions of interviews going on in the world, and nobody has really taken aim at it to improve that process. So that’s the moment I think in the demo. Some of our clients actually put their own engineers through our process. That’s pretty fun. They’ll just say, “Hey, I want to see what this is all about.” That’s a pretty eye-opening experience, because it’s not just, “I’m watching now. I’m actually experiencing it.” And generally those turn out really well. So that’s fun too.

William:  24:50

I love that. And again, I think lessening some of the anxiety, some of that stress, and getting people because it’s fit. Ultimately what Karat’s helping people do, is find the right fit, and that fit’s changing. It’s a moving target for everybody, for the engineers and for the candidates, for everybody. So you’re just helping people get that fit and get there fast. I love that. Favorite customer story? And again, you’re going to have thousands of these. And no names, we don’t have to go into that type of stuff, but just something. If you have to go to last week because there’s something that you’ve looked at because you co-founded the company, you created it with an idea, and then all of a sudden your customers are using it. And then all of a sudden you wake up one day and they’re using it in a different way or more innovative something. And you’re like, “Well, that’s cool. I didn’t quite think of that.” What’s your favorite story?

Jeffrey:  25:54

Yeah. Actually I have two here, if I can.

William:  25:56

Oh yeah.

Jeffrey:  25:58

Actually one of our biggest customers, we’ve been noticing that they’ve used Karat to help with leveling. So it was not the intended use case. Right now you’re interviewing for a role. What’s interesting about Karat is that we can rescore interviews based on different things you care about, so weightings and bars and different competencies.

William:  26:24

[crosstalk 00:26:24]. Oh cool.

Jeffrey:  26:24

If you’re a higher level candidate, let’s just take very simple example of, I care about my junior engineers’ coding ability. But I care about my senior engineers’ system design, architecture, understanding that distributed systems, things like that. Well, if you’re interviewing for both those things in an interview, you can just rescore the same interview and say, “We’re interviewing for a level four, but they were actually a level three.” They were doing this outside of our system because they knew how some of the system works. And then we actually just started facilitating that. But we get huge demand now for leveling. “Can we have one interview that would just tell me this person is a level three, four, five?” Because to your point back around compensation and other things, misleveling someone is probably a pretty big area why you might lose out on candidates. So that was interesting that they were taking our ability of that standardized interview and ability to rescore things and pushing forward. The second one I’d say is I love when our clients lean into the redo opportunity and they really emphasize it. Every candidate, again, leaves feedback after their interview.

Jeffrey:  27:35

And we have a channel on Slack that the whole company can see. And it’s a little bit of a dopamine hit because candidates write in and they talk about their experience and their interview engineer and things like that. But we had a neuroatypical candidate who came in, who had been really affected by the pandemic. And actually had been really stressed out in the interview, and had a panic attack in the interview. And the interviewer is this person who’s experienced and walked her through the interview at the end of it to help say, “Hey, it’s fine. You were doing great up until that point. Why don’t you take advantage of the redo? You can just come back another day.” And she ended up coming back, she wrote in, and just first thanking the interview engineer for being so supportive in that moment. And then she came back a week or two later, and crushed her interview, had no issues and came through. And they hired her and she’s doing great at the company at one of our clients.

Jeffrey:  28:31

Everybody’s going to have a bad day, everybody has a lot going on. You can’t afford to miss people in your hiring process that could otherwise be awesome. So I know you were talking about fit, which is really important, but also you are probably missing out on so many people. The fact that companies don’t offer a redo, especially when it’s proven, at least from our data now, that it’s just as effective. Literally you’re getting the same hiring efficiency without sourcing again, and it’s a better candidate experience. Why is every company not doing this? I just think that it was one of those stories that we all celebrated, one of our Karat’s values, is celebration. And so it was a story where it was just like, “Wow, this is what we’re all about.”

William:  29:14

I love that. I love both stories. The second one really speaks to me because again, what we’ve been through and are going through with COVID, and talking about mental health in a more positive way and not demystifying and also destigmatizing some of it, I think it’s important. I thought the interviewer did a fascinating job of not freaking out, A, but also just going, “You know what, you’re doing fine. And you know what, it’s a mulligan.” Who hasn’t had a bad question or a bad answer? And that goes both ways. That’s what’s interesting about it. I know we’re thinking about it from the candidate experience perspective, but as an interviewer, you also have bad days or maybe a bad question or get off to a bad start. So let me get a mulligan. So I love that and Jeffrey, I absolutely love what y’all are building and have built. And it’s needed, required, necessary, and I just love what you’re doing. So, thank you.

Jeffrey:  30:27

Well, thank you so much. And I really appreciate you having me on here.

William:  30:30

Absolutely. And thanks for everyone listening for the Use Case Podcast. Until next time.

Music:  30:34

You’ve been listening to RecruitingDaily’s Use Case Podcast. Be sure to subscribe on your favorite platform and hit us up at recruitingdaily.com.