The Use Case Podcast: Storytelling about HackerRank with Vivek Ravisankar

William:  00:25
Ladies and gentlemen, this is William Tincup and you were listening to Use Case Podcast. Today we have Vivek on from HackerRank. We’re going to be learning all about his firm. So let’s just jump right into it. Vivek, thanks for coming on the show and I look forward to learning about HackerRank. I’ve interacted with your brand, what seems like, for years now, but I think it’s just going to be good for us to talk and for you to explain things and really explain HackerRank. So would you do us a favor, me a favor, and introduce both yourself and introduce HackerRank?

Vivek:  01:00
Yeah, no, absolutely. Thank you for inviting me to the podcast. I’m Vivek, I’m one of the founders and CEO of HackerRank. At HackerRank our mission is to accelerate the world’s innovation. We believe we can do that by showcasing skilled developers across the globe to companies and enabling them to hire them, upskill, and mobilize them internally all based on skills. We’ve actually transformed the hiring process to be based on skills over pedigree for over 2,500 companies across the globe, across various industries. We built a large developer community, which is now over 11 million to maybe coming up on 12 million developers who come to our site actively to practice challenges and prepare themselves for jobs. We’re about 250 people, and like most companies, we model first. So that’s the journey so far,

William:  01:47
I love it. So with testing, take us into how… We’ll deal with the non-technical people, but myself, when I would hire a technologist, an engineer, I could never figure out the breadth and depth of their knowledge. Which, I know testing does that, right? So you’ll have people that are a beginner, or an intermediate-advanced, et cetera. How do you create tests and then how do you validate those tests and kind of keep those tests live? Or, how do you revise those tests as well?

Vivek:  02:25
I think firstly, the software and developers is a very interesting field because technologies keep changing. Five years back you could claim that you are a JavaScript developer if you knew basic vanilla JavaScript, but today that’s outdated. You have to have a pretty strong knowledge on a particular framework, whether that is React or Vue or any other frameworks that are going to keep coming up. And then there are also new kinds of roles that just get created. 10 years back there wasn’t really a role called a data scientist or a data engineer, and now it’s the hardest of all. Who knows what’s going to come up in the next five years?

Vivek:  03:04
Existing roles have a continuous evolve to technology, new technologies that keep coming up and there are new roles also being created. So it’s a very fascinating field and we need to be constantly ahead of the curve in terms of ability for us to foresee what’s going to come and be able to build an infrastructure that can actually support new technologies, new frameworks, new roles that keep coming up. We have a dedicated team it’s called the Skills Platform Team. And all that they do is to make sure that we are able to support the state-of-the-art roles’ skills, and be in a position to be able to support newer skills as fast as possible.

William:  03:44
What’s the hottest testing? For those recruiters and HR folks that are listening. What’s the hottest testing that’s going on right now? What are you getting hit most with from a testing perspective?

Vivek:  03:57
It varies a lot. It’s hard for me to exactly say, “This is the hottest.” But because there are a lot of large companies who use, for example, Java as their programming language. And so there are a lot of people that they’re continuing to hire Java developers, but you wouldn’t really call Java as the hottest language right now. But there’s just so much legacy code out there with large companies. Who are also, by the way, innovating and things, so they are hiring Java developers. And then on the other hand, you see this unbelievable rise to cloud, which seems obvious for a lot of folks, if you were a cloud-first kind of company. It’s even ironic you even say that now. Everything is just by default. You’re a cloud first. But there are lots of things that are still in on-prem lots of pieces of the infrastructure are still on-prem that are moving to cloud.

Vivek:  04:47
And as you can see the growth rates of AWS, Azure, and Google Cloud, just staggering growth rates at the scale! Billions of ARR! The need for site reliability engineers, Dev Ops, and others is very hot. And then on the other side, as more and more companies become software companies, more and more industries are fundamentally transforming into a software industry. And COVID has really accelerated that. I’ve had multiple doctor visits, hopefully things are good knock on wood, but I’ve never visited the hospital in the last year. It’s all through video and things along those lines. So you wouldn’t have expected this two years back, and maybe people are forecasting such a thing would happen maybe 10 years from now, 15 years from now, but nobody thought that they were happening a year from now.

Vivek:  05:36
Then you have the whole notion of multiple industries transforming into a software company. Which means they’re all going through the process of making sure that the data, and privacy, and security is very, critical. That has given rise to a lot of cybersecurity as a very hot field. So these things are just continuously evolving and changing. And then there’s, of course, the whole notion of we translate it from web 1.0 to web 2.0, and now there’s a web 3.0, which is a fundamentally different architecture that’s based on blockchain. So people are building new technologies, or sharing files, storing file, writing code on Solidity as a language for coding it on a PDM blockchain. There is a whole new set of roles and skills that are being created. It’s kind of crazy right now. Good kind of crazy. And of course, machine learning.

Vivek:  06:32
So it’s all over. In some ways, it’s very interesting. You could argue, “Was it easier to build product 10 years back?” Because you didn’t have to worry about, “Oh, is it mobile compatible? Is it iPad compatible? What kind of machine learning, what kind of data should I actually take? What kind of privacy should I look into? What kind of machine learning should I be able to apply to?” Maybe a lot of them didn’t actually have to think through all of those things. It’s as simple as like, “Okay, here’s a cloud application that I’m going to put on the web.” But now there are just so many places, so many types of distribution channels that you can get, so many things that you can continue to do and evolve. It probably has gotten a little more complex to actually build an app, but it’s also more fascinating. It has higher use cases that you can solve.

William:  07:24
It’s wonderful as you’ve simultaneously been building all the tests and helping people really understand where they are. Both for the candidates, but also for recruiters, et cetera. You’ve built a community, a wonderful community that can help you with some of these things. So if you can’t see around a corner, you’ve got 12 million people that you can lean on and they can help you see around the corner. I think that that’s a wonderful side part of what you built, what your company has built. Again if it was just testing, great, but that’s total value add. But you’ve also got a wonderful community that you can lean on, so kudos to that. I know you get asked this question probably a bunch, but where do you advise recruiters, global heads of talent, et cetera? Where should they put testing in their funnel? So they’re going to hire a Dev Ops person. They got a job description, et cetera. They’re going to use Indeed, okay, or their careers page, whatever. Where should they put testing into their model?

Vivek:  08:39
At a very high level, the standard answer is, “You should put up your screening challenges right up front in the funnel.” But there are lots to consider. I think it varies if you are going to hire a university grad when you’re going to visit a school, or right now all universities are being done remotely online. That has a different kind of a setup in terms of the tests that you create, the duration, how you position it. Then you have called them early talent professionals, people who are maybe somewhere about like a year to three or four years of work experience. Then again, the stage of the funnel is similar, but the way that you positioned it, the type of questions that… Would you actually have a call with them prior to actually sending them a challenge? Things actually vary. Then you have a mid to senior-level call it… I’m just using work experience as a proxy five to [inaudible 00:09:28] years of work experience. And then you have senior architects and others to do it. I think the simplest answer is it has to be in the earlier stages of the funnel.

Vivek:  09:39
But the type of questions, how you position it, how you actually use it in the later rounds… Like a lot of [inaudible 00:09:46] for university grads, you use that as a screening filter, but as you go senior, you use it as a way to continue the conversation forward in the interview process. Those things vary a lot, depending on experience levels.

William:  09:58
Yeah. I can see a use for using some of those real, not to say easy, but some of the basic screening to start with to really understand, “Okay. If you’re not even basically qualified, we shouldn’t spend a lot of time and money and energy on you.” But down the road, you’re going to want to know breadth and depth. You’re really going to want to know what their skills actually are. So I can kind of see a use case for that as well. Maybe the answer, as you’ve already said, is depending on the experience level, but also making sure maybe there’s a couple of different ways to look at testing. There’s going really deep with somebody that’s further in the process, and somebody that’s not further in the process maybe going light just to screen people in and screen people out.

Vivek:  10:52
Yep. Absolutely.

William:  10:54
Let me ask you about diversity inclusion and the world that you operate in. What questions are you fielding now? Because skills testing is a great equalizer, right? When you take the test it doesn’t matter if you’re a male, female, black, white, this, that, the other. It’s just, do you have the skills? Period, end of story. But that being said, I’m sure that people have asked you DnI related questions over the last year or so. What are you learning about the intersection between DnI and skills testing?

Vivek:  11:37
First thing, just zooming out. I think it’s important that companies treat DnI as a very integral part of their company building process, as opposed to just checking the box because it’s the latest buzzword. You understand that there are lots of studies for this. That having a very strong, diverse team, helps you to have a much better perspective of the world when it comes to building products. It’s actually going to help you innovate better. Help you understand the different types of people. And right now, any product that you put on the internet is by default global. Except of course if you’re trying to start a marketplace kind of an app. Maybe you’ll start small and then you continue to expand. But otherwise, most products that you put out there are global, and being very conscious of how different people will react to it and getting inputs from is very critical.

Vivek:  12:40
So, that’s one. One is to fundamentally understand that diversity can actually help your company make better decisions, and actually help you ship and build better products. The second one, I would say, is diversity and inclusion is often, unfortunately, interchangeably used with hiring black professionals or maybe women. That is definitely a dimension of having a diverse team, but there are lots of other dimensions. Putting together a team with experienced professionals and Gen Z is also a diverse team. Putting a team together where somebody who has actually worked in a large enterprise company, and somebody who’s only worked in startups, is also a level of diversity in terms of dimension. And of course, race is an important piece, but gender is an important piece of the dimension. So think about diversity in a lot of other dimensions, and not just reduce it to, “Okay, how many black professionals do we have? How many women do we have? Let’s go ahead and check that box and be done with it.”

Vivek:  13:48
So those two are very important principles. First, to understand that it should be a core fabric of how we think about building an organization, building a company. And second is, it goes beyond just the race and gender. Now once you have those two things, at least a mindset of how you operate, now comes into figuring out how you orchestrate your processes within the organization to support or to align with those principles. We play a pretty important role in the hiring piece of it, where resumes by default have introduced a tremendous amount of bias in your system, in your recruiting system. Because you’re just, by default, attracted the people like you, and there’s a pattern match. Our bodies is just made up of just energy fields and chemicals.

Vivek:  14:40
When your brain detects a pattern match, it secretes a chemical endorphin, which gives you the high. “Wow! This is amazing.” And then you feel good. So you tend to do the pattern and your body craves for these endorphins to actually be released. Every time you have a pattern match, that’s kind of what happens. So whenever you’re going to look at resumes and others, you’re by default going to navigate to resumes or people or names or schools that you already are familiar with and you actually know. Which introduces a huge amount of bias, reducing the diversity that you can potentially form in the team. And so the way that we think about it is, “How do you build a recruiting process that can focus on skills to as much of a degree as possible?”

Vivek:  15:34
Which means before even the hiring manager gives a thumbs up, the hiring managers shouldn’t even know the name of the person, the school, or the background, or nothing at all. Except for, “Hey, look at the code, and look at the skills.” And then of course, at some point in time during an interview process, you would have to know all of these things. But at least it reduces a tremendous amount of bias at the top of the funnel. Which, by the way, plays a very critical part. Which, is probably the biggest thing that you can shave off to improve the ability to form a diverse state. And there are lots and lots of studies here. I forgot, maybe it is the San Francisco Opera, the way that they try to build out their team, the music team is to have blind auditions.

Vivek:  16:25
So, you have a set of judges who are just like staring at a stage with the screen closed and there’s somebody behind the screen who’s actually playing the music and then you actually give them a thumbs up or thumbs down, depending on just hearing the music. Turns out, it’s actually working really well because it’s one of the best orchestras. So there are lots and lots of examples. Daniel Kahneman talks about this in his book Thinking Fast and Slow about how to build a structured process that he actually used to recruit Israeli army, to build the Israeli army. It is still used today! Which is a much more clearer rubric, as opposed to figuring out, “Hey, does this person actually look good? Or is charismatic or not?” So there’s lots and lots of evidence to prove how structured interview process with the blind audition angle can just help mitigate bias and help you build a strong diversity.

William:  17:26
First of all, fantastic answer and examples. I’ve been talking to a lot of HR leaders and global heads of talent where they’re talking a little bit more about potentiality. Which is a bit new, or newer, and aspirational skills. So you live in a world of what you’ve done and what you obviously can test. How are you interacting with folks when they’re looking for candidates, maybe they don’t have the skills? Or maybe they have a certain level of the skills, but they’re looking for the potentiality of that particular candidate or employee. How are you having those conversations for that?

Vivek:  18:20
Everyone has potential. Everyone needs to be factored in as an important part in the decision-making process. There isn’t a cookie-cutter answer to it because the way that I think about building teams is, “What are the risks for your organization? How can you peel off those risks? And what kind of DNA are the kind of people that you need to bring into your organization who can actually help you peel off those risks?” Then it becomes a question of, “What is the must-have, and what’s a nice to have?” When it comes to figuring out who you need to hire. I look for a lot of proven evidence for the must-haves requirements, which map pretty closely one-to-one to the kind of risks that you’re going to go ahead and solve. And then there are nice-to-haves, which are important for them putting in the role, but maybe you don’t have a lot of evidence that this person has done this before or done this part really well before.

Vivek:  19:25
And that’s when you probably are going to bet on, “Hey, can this person actually learn and do a better job? Or does this person have the penchant to do a better job on it?” So it varies a lot on the type of role and what are your must-haves, what are your risks that you’re trying to peel off. How do you judge potential? It’s a little subjective and qualitative. I think of the ability to judge potential mostly in terms of two dimensions. One is, what has been the rate of growth for the particular individual over their career. And the second one is, how self-aware are they? Which is, there are just too many things to learn in the world.

Vivek:  20:13
People actually sometimes even take the wrong lessons from a success that they’ve had, and there are just too many things to learn. So, are you constantly updating your mental model of how you view the world? And that doesn’t mean that you should lack conviction, but it’s more of a question of, “What are the inputs that you’re seeking, and how are you constantly updating your mental models of the world?” When you have a very strong rate of growth, and you’re very self-aware and conscious that you don’t know everything and you’re going to constantly learn and update. That’s a very good sign. You can actually ask them. Which is, “Hey, what is a belief that you’ve held that you changed or modified after you learned a few things about it?” And if the answer is, “Nothing.” Or the answer is, “Oh, there was the belief that I had like 10 years back that I changed.” Then clearly you’re probably not self-aware because the world 10 years back is not the same right now. So surely you should have updated your beliefs.

William:  21:19
Love that. What a great answer. So this question is going to be a kind of twofold. On one level, we’re all a little self-involved on some level. Maybe we have an inflated sense of worth in some regards. Maybe that’s a human condition. The other is people that just don’t take tests well. How do you… You’ve got to coach people through all kinds of different scenarios. How do you all deal with on one side, maybe a candidate takes the test and they just don’t believe, they don’t like the results. They just don’t like the results. They don’t think it adequately demonstrates what they’re capable of. So you got that, which could be an inflated sense of worth. Or, they’re just not great at taking tests. And again, your clients are going to be using this as well. So how do you all… Say, “Combat.” It’s not that, it’s, how do you have a conversation with a candidate around those things?

Vivek:  22:28
A few answers. Or, few things to note. There is no perfect hiring process. Nobody has cracked that. And I don’t think there is a principle for you to go ahead and crack that. Because hiring involves people, and people are just random in the sense that you just change, and it’s very hard to deal with. There’s no perfect hiring process. And a lot of times, which is very interesting and a somewhat counter-intuitive lesson for me, is you as a candidate actually aren’t really very clear on what you want. There are lots of hot companies, lots of things that are growing quickly, interesting technology, good manager, good culture, but you’re not really entirely sure on what exactly you want. In fact, the interview process feels more like a discovery, an internal discovery for yourself to figure out what you like and what you don’t like. As opposed to having a very clear understanding of, “This is my criteria.”

Vivek:  23:31
That’s totally fine, but I’m just saying that’s my observation. So with those two things in place, our job is to try our best, to reduce bias, and give everyone a fair shot at the process. We’re by no means perfect, and we’re continuing to get better at it. In fact, there is a team whose only job is Develop a Love. That’s literally the name of the team. And we measure this index, called DLI or Develop a Love Index. And all they do is to figure out how you can make the interface, the type of questions, the type of assessments, the way that you position, the way that we actually give you better messages if any. Making you feel at home, integrating with your own custom IDE. There’s just so many things. It’s almost like one of those things that’s a never-ending thing, because like I mentioned, there’s always new technology.

Vivek:  24:22
There’s always the new infrastructure for you to go ahead and code. And we’re just constantly working to make sure that we can help developers feel at home. Where the goal is, “Hey developer, let’s help you showcase your skills and also in the process help you discover a lot of this is the kind of problems that you would like to work on.” Because like I mentioned, a lot of times you’re not very clear on what you want before the interview process. So, that’s what I would say. That’s what we’re trying to do.

William:  24:56
Two real quick questions. One is your take on skills testing. We’ve focused a lot of the conversation on the talent attraction side, so recruiting. Obviously, skills testing has a place deeper into the organization with employees and internal mobility. Where have you all been, some of your clients, where have they taken you? In terms of skill testing? Not just in TA, but post-hire?

Vivek:  25:25
Yeah, it is a very hot space. Developer mobility, upskilling, is a very important piece of the equation. Our strategy, which we call it Developer Life Cycle Management, fits right into it. Which is, like the name suggests, building products that span across the entire life cycle journey of a developer. We think about it in a pre-hire phase, enabling developers to prepare and apply for jobs, the hiring phase, and the post-hire phase. I don’t think anyone has clearly cracked that yet for developers. In the sense that I know that a lot of Ad Tech companies who are just growing like crazy, but I also don’t know how much of the Ad Tech spend that a company does is more of a posturing than, “Hey, we care about your growth. And if you don’t buy this particular license for this particular Ad Tech software, it’s going to look badly on us.” Versus, “Do people actually use it, and are they actually continuing to grow?”

Vivek:  26:29
I’m just speaking for developers, I’m not [crosstalk 00:26:30] organization. So I don’t think so. I don’t think people have actually really cracked that. And there’s also a different kind of a notion among developers where if you want to learn something, I would much rather tinker around with certain things, than watch an hour YouTube tutorials, or any kind of tutorial for example. It’s probably very helpful if you’re trying to be a manager and helping you figure out, “What can you do better? How do you manage people? How do you deal with conflicts?” And things on those lines. You can watch videos, but I’m not sure how much it’s applicable for an individual developer.

Vivek:  27:04
So I think there is a giant white space. We’re actually building a very cool product in that space. We’ve been working with three customers over the last six to nine months, and been receiving really good feedback. I think it’s going to be another breakthrough that we’re going to make in this space, just like how we’ve made it in the skill assessment at hiring phase. I think there’s going to be another one in the post-hire. So we’re going to watch-

William:  27:29
I do too. I like it because, again, once what you’ve done in the front end, it goes with them. They’re learning new things. They’re on the job, they’re learning new things, new skills. And so, there’s going to be job openings that happen. You’re going to want a mechanism in place to be able to understand what they’ve learned. So on the way out, last thing. I know that this is just because I know you all do a really good job of this. But people would ask in terms of cheating. People take tests, right? So you have a percentage of the population that would try to gain that in some way or another. What type of things have you all done in terms of making sure that the recruiters and HR can trust the results.

Vivek:  28:28
We are doing a lot. Again, there are multiple ways of plagiarizing. You can plagiarize the code, copy code, you can have somebody else take it on your behalf, impersonation. So, there are probably other ways of it. We’re constantly figuring out like how we can give enough signals to the employer when we detect anomalies. But at the same time, not putting a lot of burden on the developer. Because take it to an extreme, you can ask everybody to come to an office and take the assessment with a browser that’s actually locked that does nothing but HackerRank. That’s taken to an extreme where you get the most secure environment, but it’s just a tremendous amount of burden. The other extreme is you just give them a simple link and you can do whatever they want. So we’re definitely figuring out the right balance in terms of making it developer-friendly, yet giving the confidence to the employer that this is a legit test.

William:  29:27
Love it. Brother, I could talk to you forever. Thank you so much for taking out the time and explaining HackerRank to us.

Vivek:  29:36
Awesome, thank you so much, William. Have a good day.

William:  29:39
Absolutely.