Storytelling about Codility with Jason Medley
Welcome to the Use Case Podcast, episode 99. This week we have storytelling about Codility with Jason Medley. During this episode, Jason and I talk about how practitioners make the business case or the use case for purchasing Codility.
Jason is the Chief People Officer at Codility and is an expert in all things culture and talent. His passion for helping talent acquisition teams reach their goals through strategy really comes through during the podcast.
Give the show a listen and please let me know what you think.
Show length: 30 minutes
Enjoy the podcast?
Ladies and gentlemen, this is William Tincup, and you’re listening to the Use Case Podcast. Today, we have Jason on from Codility and we’re going to be learning all about his firm. So let’s just jump right into it because it’s always better to get to the guest as fast as possible.
Jason, would you do us a favor and introduce yourself and your firm?
Yeah. Thanks so much, William, and I’m excited to be here. This is going to be a fun conversation for sure. So as William said, I’m Jason Medley and I am the chief people officer for Codility. I’ll talk a little bit about Codility in a second, but my background, I really grew up in recruiting, worked for fast-growth startups, have been very, very hands-on in building teams and building engineering teams, and so naturally I was really excited about the opportunity to join Codility.
Codility is essentially a technical assessment platform for engineers in the recruiting process. So how we think about ourselves is we think that we’re really helping teams to spend less time on interviewing unqualified talent and the right amount of time on interviewing qualified talent, essentially opening up more engineering capacity for engineering teams to really solve the world’s problems.
So we were very much founded by an engineer who believed that he was spending way too much time interviewing and really needed to be hands-on within actually engineering like so many engineering teams do and so that’s where Codility came about. The way that we really solved this problem within our platform is that we have three key products. We have what we call CodeCheck, CodeLive, and Code Challenges.
CodeCheck, essentially high-level, is really a library of tasks that tests languages and frameworks, so the real-life skills of engineers. It’s able to benchmark those results against thousands of other engineers who have taken them and it’s able to even benchmark against a team’s own engineers to see where candidates fall.
We also have what we call CodeLive, which is our vertical whiteboard. So anyone out there who is doing engineering hiring, you have done some form of whiteboarding in the pandemic, you’ve had a go virtual, whether you’re doing that in Zoom or some other place, we offer a really great experience that’s really high touch and is really a great experience for engineers to go through.
Then the last product that we offer is what we call Code Challenges. So this is really top of funnel attraction. We partner with a lot of our customers to help them fill top of funnels through competitive code challenges that brings in hundreds of developers and really helps them fill immediate needs, but future roles as well.
So you’ve been on both sides of this, right? So you’ve seen both the recruiting side, the HR side, and now you work for a vendor partner that helps people with the efficiency of, again, you can spend your time as you probably have done in the past of really falling in love with a candidate and then only to find out that they’re not as qualified as maybe they thought they were, or maybe you thought they were. So you’ve seen this. You’ve been on this merry-go-round a couple of times. You’ve seen this happen.
I think one of the things, the hardest for people that are non-technical, and I’d throw myself in that category, by the way, is to understand the breadth and depth of technical knowledge. So when you see something JScript, or pick your favorite acronym or development language, as a non-technical person, you don’t know how wide that knowledge is or how deep that knowledge is or a combination thereof. I’m sure you’ve had to deal with this as you recruited talent as well, and I’m sure your customers are struggling with how broad or deep is one’s knowledge about whatever that they’re looking for.
Again, I love that the way that y’all go about this with CheckLive and Challenges. You’ve got different ways to get at this. Give us an idea of how your customers, how they get to that breadth and depth, like, okay, we need just need to verify?
Yeah, yeah. No, no, great question. So if you think about why people have to use this product in the first place, so we really have two buyers, so we have the talent acquisition side of the house and we have the engineering management side of the house, and anyone who’s been in recruiting, you know firsthand that you always have candidates in the past where they may not have the sexiest of backgrounds, meaning they may not come from a Google or a Facebook and be able to skip the line with hiring managers, but you know because you’ve spoken with so many engineers in the past that you know something’s different about them and you really want to give them a chance, but it’s really hard to do that, right?
So when you’re able to put them through CodeCheck, you’re able to really go in, you’re able to assign a task that’s specific to the skills that they’re actually going to be using on the job. So this is frameworks, this is languages, these are anywhere from 20 minutes to sometimes 80-minute assessments, depending on how senior or how depth the role is, but it’s able to go in and really check real time their skills. These tasks are very much we have a whole team of task creators who are engineers, very senior engineers, who build out these tasks, who test them. We’re always testing this out in the market where we’re taking signals from the engineers who are actually taking this to understand how fair they think it is and we’re constantly iterating and making sure that we’re being very protective in what we’re putting out there.
So from an engineering side, from an engineering management side of the house, one of the things for both engineering leaders and TA leaders, or even HR leaders, is that you don’t always really know how good your engineering team is at assessing technical talent. There’s always that fear in the background that, especially to your point, William, if you’re not technical, so I’m not technical, I have a high-level knowledge, but I can’t go and assess someone’s technical skills on my own, right? I can’t do a whiteboarding session and come out with a really good decision on my own. So literally we’re able to give assessments that benchmark them globally with thousands of people who have taken these assessments and it’s able to really build confidence that you’re making really smart moves and smart decisions and you’re able to get them through the process significantly faster.
The other thing that I would just call out and what I love and what honestly really attracted me to Codility in the position that I’m at is that I even think about it a little bit further than that. What we’re really doing is we’re starting to level the playing field for engineers. So if we go back to the example of, hey, everyone not necessarily has this sexy technical background, but you know what? Maybe I went to a bootcamp, maybe I’m a returning parent, but I have some mad skills. It’s really helping everyone to enter the playing field at the same level and have the same fighting chance. I think, especially in a remote world where all of a sudden so many of our customers in Codility as well, where we’ve looked up and we go, hey, we can hire someone anywhere in the world today, how do we do that efficiently, effectively, and with quality, and offering a great candidate experience? I think we’re able to really answer that.
I love that. Well, while we’re on it, how are your customers? I mean, first of all, you also as people operations and people officer, you’ve got this call to also think about diversity, inclusion, belonging, equity, and equality. You’re thinking about this, it’s keeping you up at night as well, but what have you seen from your customers in terms of how Codility helps them push whatever initiatives that they’re trying to push with DNI?
Yeah. No, I absolutely love this. We work, William, with everyone from SMB to large enterprise organizations. So we work with the Microsofts, the AMEX, SpaceX, Dolbys, Amazons of the world, and you’re right, everyone deals with the same problem, right? No matter how big you are, no matter how well known you are, how much money you have, attracting and securing diverse hires and creating cultures where they thrive is so many of our customer’s missions today.
I think how Codility is able to solve this directly, or how we’re able to help mitigate this, is really leveling the playing field, right? So we know this, and I know that you know this, that recruiting in so many ways, it’s just biased. We’re humans, it’s hard to overcome the bias that’s in the decision-making around recruiting. This is where really great technology can come in and help us with that.
I don’t know if my big dream one day is that the entire recruiting process, we can eliminate bias altogether but we’re still dealing with humans, but to say, you know what? Regardless of someone’s background, everyone’s going through the same structured technical interview process as efficiently and as quickly as possible and we’re able to get those benchmarks and get those signals.
What we’ve seen directly from our customers is that they’ve, and I’m in a really unique role because I’m extremely blessed and that I’m a buyer, but I am also external facing within the company, so I’m constantly hearing the challenges that are happening and I’m hearing the value that the product adds. What I hear is and what they’re telling me from recruiters to heads of TA to engineer to engineering managers is, “Jason, we’ve honestly hired people that we never would have looked at before. If we were just to look at their resume and the companies that they had worked at before, their backgrounds and experiences, we honestly wouldn’t have prioritized them. But the fact that we were able to assess their technical skills at the top of the funnel, we were able to get a really clear indicator of how strong they are, and we’ve seen diversity in experiences and in backgrounds that we haven’t ever seen at scale before.”
What’s so unique about that now, again, is because just today I was talking to Amazon, I think they have 50,000 new roles they had to hire for this year, I think 15,000 of those are engineering alone, I believe if I have that right, and they’re reaching further and farther than they ever have. Being able to think outside the box to look at those non-traditional backgrounds, but also have confidence that they have the technical skills, is huge, and we’re able to help companies like that do that.
I love that. Again, I love the fact that both you’re facing this both as a team, but you’re also helping clients with this as well. I think what’s interesting about COVID and work from home and being forced to work remotely, it’s opened up that aperture for both candidates and recruiters. You can live in Paris and work for a company in Topeka and vice versa, or you can have your recruiter in Topeka, and you can recruit somebody that lives in Paris. So it’s fascinating and it also can open up that aperture of what’s available in terms of talent because we used to kind of think of local. If you’re in San Francisco, you needed to hire someone because they had to come into the box. So it’s fascinating.
I want to go back to where you started because I think there’s some really interesting things to play with there. Because again, based on how you’re founded, but also what you do for your customers, it’s all about efficiency in the sense of spending the right amount of time with the right candidate so that you squeeze out the waste instead of chasing the wrong people or spending a lot of team interview time, et cetera, on the wrong candidates. And vice versa, if it’s the wrong candidate, it’s also wasting their time so it’s inefficient for candidates as well.
Could you talk to us a little bit more about how you squeeze out some of that inefficiency?
Yeah. Yeah. So I think a lot of times just the really obvious one is that most companies don’t know an engineer’s true technical ability until they’re able to go live with them and see them code in real-time. No matter how strong a technical recruiter is, they just can’t replace an actual engineer in watching someone code live, right? So more often than not, just like interviewing for any role, you spend a lot of wasted hours on candidates who aren’t qualified for the role. They’re not necessarily bad, they just may not be qualified for what you need.
Particularly for engineering teams and organizations, every company now as a company, the engineering capacity is a real problem, and our vision is all around unlocking and unleashing engineering capacity for organizations. So what we see is that immediately when people start using Codility that they’re able to significantly cut the amount of hours spent on total interviewing from an engineering perspective because they’re really taking care of a huge chunk of the technical assessing upfront.
Now. They still want to do a live whiteboarding session, but in a lot of organizations that do not use a product like Codility, they’re usually going through several technical interviews, and those technical interviews get spread out over a matter of weeks. In a lot of our organizations, what usually happens when they start using us is they want to build the model based off of what they’re currently doing and they quickly realize, oh my gosh, we can actually do this in two screens versus four to six screens. So what that means is that your time to hire can drastically be decreased if you’re willing to move quickly within the platform.
So that’s a really realistic way. I think another way, if I’m being honest, is just I’ve been part of organizations, we all have, where particularly with engineering, you get to the final, you’re debating, you may be in a debrief, and you’re not 100% sure. There always a risk in hiring and you’re not quite sure, and sometimes the hiring managers aren’t quite sure in their own engineer’s ability to test, and the benchmarks really, really come in handy. So they give you the courage to push ahead to make that offer, to move significantly faster, and you’re not waiting.
We see this especially with our SMB companies, where typically they want to benchmark one candidate against 10 other candidates for one or two open positions and they’re wasting time. They’re actually able to see how they perform against thousands of other engineers who’ve taken these same assessments and they’re able to even benchmark against their own engineers on their team. So they know if they’re coming in and they’re a technical add or if they’re not a technical add and they’re able to move and make those decisions and they become faster.
You know, one of the things I love about this is engineers are different than, let’s say, marketing professionals. They don’t like to waste time. Not that marketing folks do, but they’re hard enough to recruit, they’re hard enough to find, A, they’re hard enough to recruit, and they don’t like their time wasted. I think they’re looking for a fit just like everybody else, so I think they don’t mind challenges, they don’t mind tests, they don’t mind these things because it helps them also figure out is this a good fit for me.
So I think one of the things I love about what you all are doing is that you’re squeezing that inefficiency out for recruiters and hiring managers, of course, spending the right time with the right candidates, but also, you’re making it efficient for candidates as well so that they have a good experience while they’re going through this. Like, “Listen, we don’t want to waste your time. If you’re not a good fit or if this challenge that we have is not something that’s appealing to you, then let’s part as friends, but let’s part quickly as friends.” I think that speaks specifically to technical people.
Yeah. You know what? We measure this as well, and it’s really important to us because we know that engineers, I always joke, especially from a people ops perspective, I always say if I want real feedback on anything I’m rolling out, I’m going to go to an engineer first because they’re going to give it to me straight.
Brace yourself, it might be.
Yeah. Yeah. Yeah. It’s so true, and I love that because they’re so focused and they focus in on the details, that’s how their brains work, and it’s how they’re trained. So engineers are naturally skeptical at times, right? So we know that when you ask them to take an assessment that they’re already a little bit of a skeptic, they’re really judging it hardcore, they’re really thinking about it. So we measure, we actually ask, we want to know their overall candidate experience, for sure, but also do they think that this assessment was fair based on what they understand the company is looking for, right?
Oh, that’s cool. That’s smart.
But also knowing that there’s some bias there, too, because if they’re not scoring well, they could blame the [crosstalk 00:18:52].
Yeah, yeah, yeah, yeah, yeah, yeah.
We know that.
Yeah, of course.
But the scores from a candidate experience on average were 86%, and from a fairness, were 80% from the very candidates who are taking it, right? So I feel those numbers are really, really strong and really, really high and we take those as signals when we work with our customers as well.
So one of the really cool things that I think is a great differentiator about Codility is that we’ve always really excelled on the customer success side of things and the customer support side. So our product can be used right out of the box with on-demand training and we’re able to do that, but we find, especially on the TA side, and even for that matter on the engineering side, that like all of hiring, a lot of times they need some handholding, they need some guidance, they need some best practices, and we’re able to really dig in and help them with that with an amazing customer support team who can walk them through and help them thinking about where are the gaps in your overall experience, you know? How can you speed things up? How can you offer this overall better experience?
I love that piece about Codility, as well, because I know that we can have the best platform in the world, but if the overall experience isn’t great, then we’re going to be out of business. So measuring that is really critical to our success.
Well, one of the best compliments you could receive is someone that didn’t score well, but yet still thinks the assessment’s fair. Right?
Yeah. Yeah. You know what? They learn from this, right?
Our big vision around this, we’re not done building, right?
Oh, it’s never done. Yeah.
Yeah. Where a lot of companies I think are focusing to the left of the candidate funnel, meaning they’re over-indexing on sourcing, and I’ve seen that’s definitely a need, there’s definitely a want and a desire there, but seen so many companies fail on the sourcing side. What we see is that skill gaps are real, so helping candidates understand where are your skill gaps, helping organizations understand where are their skill gaps, and creating tools and assessments around that and building partnerships of how they can level up is something that’s extremely important for us.
Because again, we want to bring more engineering capacity to organizations. That’s our vision. So you’re going to continue to see us build more products down funnel and north of onboarding and once candidates are actually onboard, as well, as employees and how we can continue to support them and beef up their skills.
I love it. So where in the workflow for HR and TA work, where do you all see yourselves in the overall workflows? You talked about sourcing, recruitment, marketing, obviously the ATS plays in there, there’s all kinds of different technology that plays in here, but where do your clients put you in their workflow?
Yeah. So I think it’s a little different depending on sometimes the size of the organization.
Not even necessarily the size, but the size of your hiring plan, right?
Right, right, right.
So organizations who may be doing a lot of university recruiting or who just do not have a top of a funnel problem, which isn’t a lot of companies today if we’re being honest, but they look really top of the funnel, right? So those organizations would go, okay, we have so much inbound, or we’ve now opened up to all geos, and we need to figure out how to do this with quality. They’re really thinking about us at top of the funnel to really make sure that no one’s getting to the interview stage until they’ve done a CodeCheck and we know that they rank in the top 10%.
So we see that a lot. Other organizations, and still some of these same organizations, really dive into the optimization piece in CodeLive. So they want to make sure that the experience is great for the candidate, they want to make sure that it’s getting to the heart of what they’re trying to assess as quickly as possible by using the technology that is really engineering-friendly and really, really easy to use.
So that’s where we, today, that’s where we see the bulk of our companies really over-indexing is on CodeLive. Because again, as we’ve gone remote, how do you take that whiteboarding session that happens in so many meeting rooms live, how do you do that really, really well? The reality is Zoom’s just not going to cut it. That’s where that feature works really, really well.
Now what we’re getting a lot more of demand around as well is top of funnel attraction. So that’s where Code Challenges really help. So again, every company, our largest enterprise companies from financial services, they were having to all of a sudden just be way more progressive than they ever thought and were having to catch up and do things and change their employer-branding strategies, they’re all asking the same question: Hey, how can we attract more talent? Although we’re not an employer-branding organization, through Code Challenges, we’re able to do a lot of that.
There’s a company, you may be familiar with it, it’s called Avature, they’re a CRM in recruiting.
Yeah, of course.
We did a sponsored CodeChallenge with them and they attracted 700 new candidates who participated in multiple roles and then pipelined for future roles. Right? So that’s a great example of meeting engineers where they want to be in that competitive challenging space and also being able to build your pipelines at the same time and really advertise your brand in a really great way.
I love that. Okay, because you’ve worked both sides of this, currently work both sides, as your role, you buy HR tech and you have an HR technology stack, including all the stuff that goes on in recruiting, so what are some of your favorite buy questions when you’re evaluating a new ATS? We don’t need to name names, but what are some of your favorite questions that you like to ask the technology vendors?
Oh my gosh, this is so great. So one is, first of all, my first one always within HR tech is integrations, right? I mean, your different technologies need to speak to each other. If not, at any given time you’re going to have 50 tabs open and it’s a nightmare on the people ops side of things. So that’s a huge one and it’s usually my first one.
Organizations, different partners, are at different stages of integrations, and I recognize that as well. Some truly have a roadmap built out. So what a lot of organizations will tell you is, no, we don’t have this integration yet, but it’s on our roadmap. So that’s like the classic one. You buy and then they never build the integration. So I always want to go a step further and go where on your product roadmap is this? Exactly what are we talking about? Can you prove this out? Right?
I will even sometimes, because the question that I’m asking, for example, if I’m a Greenhouse user and I want to know if a product integrates with Greenhouse and they tell me that it’s in the works, it’s coming soon, I’ll even just ask Greenhouse, hey, how close are we on this integration, to make sure that happens. So integrations are huge.
I think for products like Codility, the other thing, and especially we do a lot of hiring in Europe as well, and I think things around fraud protection or really fraud detection for a platform like Codility is really important. Just for Codility, I mean, that really sets us apart. We’ve invested heavily there because I think it’s really important. But that’s something from a GDPR perspective that’s really important.
Candidate experience, right? I want to know what other users, and especially if candidates touch this product, what candidates’ experiences have been with that.
Then my other question is just renewal rate. I want to know are customers renewing? Are they buying into the hype or are they actually renewing?
That’s an indicator.
Are they [crosstalk 00:27:11], are they increasing spend? Those things are what really, really matter to me.
Then I think probably the last one is reporting. I will tell you, I think that every company out there could probably do a better job with the reporting functions. There are some companies that have been coming out in recent years who are add-ons who can help with HR-type reporting, but at least some functionality that doesn’t have me digging for hours or that at least I can integrate with other metrics is really key for me.
So last question, and this is really back to the product lines CheckLive and Challenges. I could see those being in different categories for folks, some assessment testing and maybe even talent pooling or talent community pipeline stuff. Do customers, do they buy all three? Do they buy two of the three? Can they buy one?
Such a smart question. So today, Code Challenges we can do separately. Today, it’s all integrated in our platform. However, because it’s a smart question and it’s something that we’re getting asked, we’re actually on the backend, we’re literally rebuilding our own code and infrastructure so that we can sell those independently. Also for our future roadmap and what we know that we’re building, we’re going to sell all of our products individually or all at discounted rates as a whole.
That’s so smart Because it lets people buy how they want to buy. If they are just really interested in the Live piece and that’s where they feel like they have the biggest problem, okay, fantastic. You know? Great.
Exactly, right? I think that, and my hope is, and even as I’ve joined Codility and have really been using the product and understanding it more intimately, I see how all these pieces fit together in a real harmonious way, but I understand that some buyers just need to have an introductory element to it and that’s why we’re hyper-focused on that from a product standpoint.
I love it. Brother, I could talk to you forever. This has been so much fun. We’ll have to schedule some more time because I want to get into some more of the buy-side stuff next time we talk because I think you’ve bought some tech, you probably have some good stories there. But Jason, thank you so much for the time today, and thanks for coming on the Use Case podcast.
Yeah, thanks so much, William. I really appreciate it. It was a lot of fun.
Alrighty. Thanks for everyone that listens to the Use Case Podcast. Until next time.
William is the President & Editor-at-Large of RecruitingDaily. At the intersection of HR and technology, he’s a writer, speaker, advisor, consultant, investor, storyteller & teacher. He's been writing about HR and Recruiting related issues for longer than he cares to disclose. William serves on the Board of Advisors / Board of Directors for 20+ HR technology startups. William is a graduate of the University of Alabama at Birmingham with a BA in Art History. He also earned an MA in American Indian Studies from the University of Arizona and an MBA from Case Western Reserve University.