Use Case Podcast – Storytelling about Criteria with Josh Millet

Music:    Welcome to Recruiting Daily’s Use Case Podcast, a show dedicated to the storytelling that happens or should happen when practitioners purchase technology. Each episode is designed to inspire new ways and ideas to make your business better as we speak with the brightest minds in recruitment and HR tech. That’s what we do. Here’s your host, William Tincup.

William Tincup:    Ladies. Gentlemen is William Tincup and you are listening to the Use Case Podcast. Today we have Josh on from Criteria, where we’re learning about the business case and the use case, the cost benefit analysis for why his customers and his prospects choose Criteria. So without any further ado, let’s get to know Josh and Criteria. Josh, would you introduce both yourself and Criteria?

Josh:    Thanks William. Really happy to be on with you. So my name’s Josh Millet, I’m the founder and CEO of Criteria. We are a SaaS business based in Los Angeles, California, and we’re essentially an assessment platform. We call it a talent decisions platform, and the core of it is around assessments. And we help our customers make better hiring decisions. So we have a pretty broad portfolio of assessments, and we’ve recently started expanding beyond assessments as well to building structured interviewing tools and we acquired a video interviewing company not too long ago.

But the core of the platform is still around assessments. So we have a very broad portfolio of tools that our customers use, including aptitude assessments, behavioral and personality assessments, EQ assessments. We acquired a company in Australia that has a great game-based EQ assessment and skills as well as risk assessments. So a very broad portfolio of tools that our customers use typically very early in their hiring funnel to help make more informed hiring decisions and to drive efficiencies in their process. So most of our customers are using the assessments which tend to be on the shorter side as a really early data point in helping them make more data driven and science based talent decisions.

William Tincup:    I was going to ask you about that, where in the funnel do your customers tend to… Do they put these things a little bit later on or after they have 1,000 people apply? Do they push them further out or is it job to job? Or even talent acquisition leader is talent acquisition leader? Where do they like to put assessments?

Josh:    It’s really interesting mix. I would say you see differences across industries and in different environments, obviously the labor market at this juncture and history is really unusual, moving very quickly. But in general, about 70 to 80% of our customers use what I would describe squarely as top of funnel. And that can mean either they use it at the same time as they’re gathering a resume, so they actually embed a link to [crosstalk 00:    03:    11]. And get the assessment at the same time. Or it’s also quite common for them to collect resumes from a LinkedIn job poster, wherever they’re posting their jobs. And then within 48 hours, if you meet basic qualifications, they’ll send out the assessment invite at that point. Vast majority is top of funnel and that’s the best practice that we recommend. And so that’s the typical use case.

William Tincup:    Have your customers yet pushed you into internal mobility and assessing, I would say, the old days of PrivaZer and SHL where you have 50,000 assessments for basically everything? People would use some of those assessments later down the road for leadership development or succession or whatever just to assess for people’s capabilities and skills and aptitude and passion and all the other stuff. Tell us a story… I could see why they would do it, but I just wanted to find out is that some thing happening?

Josh:    Yeah. The talent mobility is a really interesting area, especially, and I think that comes to the fore especially in the labor scarcity market that we’re in now as well, because people don’t want a good people to leave if there’s a better role for them in the organization. So historically, William, we’re a 15 year old business, and the first 10 years we were very focused on the small and medium size market, and talent mobility is not something that’s incredibly important for that segment.

William Tincup:    Good point. Three employees. Good point.

Josh:    Or even a couple hundred employees.

William Tincup:    Oh, I know.

Josh:    But it is as something that enterprises are thinking about all the time. We actually have a board member, [inaudible 00:    05:    04], who’s the head of talent acquisition at SoFi and she’s a big proponent of talent mobility. And so in the last couple years, we have been pursuing that area. And the assessments that we provide that most of our customers use in pre-hire context are really helping if you think about what companies are doing with those assessments, we think they’re building a multidimensional person first data model where they started creating all this data around an individual, around their skills and behavioral traits and all these things. And they use that to make a pre-hire decision. But then after, let’s say, they do hire that person, they then start acquiring all this other post-hire data, like data outcomes, which we’re beginning to gather through our software as well.

And so that same IP can easily be repurposed to use for talent mobility purposes. If you have someone with a set of characteristics that is a good match for another job within your org, it seems silly not to use that data, especially if they’re a churn risk or a turnover risk because they’re not quite happy in their current role. So from a career development and talent mobility perspective, the biggest enterprises in our customer base is in the last five years we really have started building an enterprise business. It’s about 40 to 50% of our revenue now. Today is from enterprise and that just wasn’t the case five years ago. So a lot of our customers are beginning to use the data from our assessments to inform talent mobility. And it’s an area that we’re going to be really active in thinking about in our product to help enable that in the next couple years. It’s central to our roadmap, building out that functionality.

William Tincup:    That just seems smart, again, with the labor market, as you said, and also it’s one thing to assess somebody when they first come in, they’ve been with you for seven years, turns out some things have changed and just to having some other data points, I think would just be smart. So I understand the enterprise side of things, are there any industries that, y’all are dominant in, that you really just thrive in, or you industry agnostic? What’s the story there?

Josh:    We’re pretty industry agnostic. And I know, William, you know the assessment space pretty well. And some of the verticals, some of the industries that are, in general, biggest for assessments, we’re actually underweighted in. So assessments are really big in a lot of hourly fields. [crosstalk 00:    07:    41]

William Tincup:    I was thinking of PeopleAnswers when you first said that.

Josh:    Exactly. And you look at Outmatch, now Harvard and other businesses have a really big presence in service industries, retail, hospitality, restaurants. So those tend be higher turnover industries where the ROI of incrementally better hires is pretty dramatic just from a retention standpoint. And we certainly have customers in those areas, but I would say we’re a little bit underweighted in those areas. And our two biggest industries that we’re active in… And we track about 17 industries overall so we really are vertical agnostic.

Basically anyone who’s having any kind of hiring pain can be our customer. But our biggest two industries are technology and healthcare, in that order. So a lot of big software and other SaaS businesses actually use our services, a lot of venture backed and PE backed high growth software companies. And a lot of medical, hospitals, regional clinics, things like that, use it a lot mostly for their non-clinical staff and support staff. So those are the two biggest industries for us, but, again, we have an incredibly diverse customer base both in terms of the industry makeup, as well as the size mix that we have.

William Tincup:    Well, I think that’s smart on a lot of levels because you let those other folks that really focus on those industries, that’s fantastic, but also that’s their portfolio. And if those industries, for whatever reason, go through some hardship, then that’s their bread and butter. So I like the 17 industries. I like having a little bit of a portfolio in case things get sideways.

Josh:    That’s right. Especially during COVID, some of those industries have been so impacted. And actually we had a board meeting yesterday and we were talking about this, that the different trends you see by industry, the differences from one industry to another in terms of the labor dynamic now is the greatest diversity I’ve ever seen. And you hear a lot of talk about the labor shortage, for example. And we see that really acutely and some of our customers in certain industries are really feeling that and others have more up and flow than they’ve ever had. So in January, for example, of this year, we’re going to administer almost a million assessments through the platform in a month. And that’s about 20 to 25% higher than any previous month. And you’d think you hear about the labor shortage in the press all the time, it’s really acutely impacting certain verticals and others have more applicant flow than ever as I said. So you get a really differential view depending on what segment of the economy you’re looking at.

William Tincup:    Do you think TA or HR folks, the practitioners, do you think they get nervous about assessments? Because you want the data so you can make a great hiring decision. So that’s [inaudible 00:    10:    51] but do you think they get nervous when there’s talent shortages about putting assessments into the process?

Josh:    It’s a really great point, William, and we definitely have that perception that we deal with when we’re talking about our service to [crosstalk 00:    11:    08], especially prospects. And I think in today’s talent environment, any tool that has a perception that it might slow things down or that it might introduce friction of any kind is going to have a hard time thriving. And more than ever before in this really competitive environment, velocity is important. You can’t have a 17 interview process anymore, unless you are a really, really well established brand. And so one of the things that our assessments do is help increase velocity rather than slow it down. So a lot of our customers have pretty crazy applicant to higher ratios, even in this economy. And if you look at how you get from, let’s say, 150 applicants for a customer service rep position down to whatever it is, eight phone screens, 12 phone screens, whatever the next step is, using objective data that’s a quantifiable and easy to digest is really the quickest way you can do that.

No matter how good your resume parsing technology is. One of the quickest ways is to gather data very much top of funnel, and then direct your energies to the portion of the applicant pool that is most likely to be a good match. And so we have a lot of data around how our customers in certain industries have decreased their time to hire and increased their velocity through the use of assessments, because it allows you to move more confidently. As you said, you have this data point or these data points. And in this kind of environment, again, I think what TA people need to think about is they are going to, by necessity, have to take more risk today. It might be in the old days you could really demand, okay, the person doesn’t meet nine of the 10 job requirements, I’m not looking at them.

You just can’t have that stance in a lot of fields today. So if you’re are going to be content with someone who maybe meets seven or six out of 10, then you need a way to de-risk that process because you are taking on more risks just by the nature of the dynamics in the labor market. So assessments can be a great way to de-risk the process by looking for talent signals that we know long term predicts success in roles. So I think it’s partly about giving TA people the confidence to move at speed when the ground under their feet is shifting pretty quickly these days.

William Tincup:    You still need the data. At the end of the day, you might be scared about friction or whatever, but you still need the data. So you still need to make the best hiring decision. And again, I love the way that you don’t increase friction, you just increase their ability to make great decisions. I’ve spent a lot of time with SIOP, I’ve spoken at SIOP a couple times, and I’ve spent a lot of right time around IO psychologists. I call them propeller heads. It’s a term of endearment for me to them, but they are completely propeller heads. But buying assessments, if you’re not an IO psychologist, so when y’all sell into people that have an IO team and all that stuff, it’s probably a much different conversation. If they don’t have that, what should they be asking you? What’s the questions that you should ask in terms of validity? You just said, we’re going to have a million people go through the process this month alone, which is a huge number. When they’re on the buy side, what are the questions that they should ask you about the different types of assessments?

Josh:    It’s a great question. And we have a lot of great propellor heads on our team. They’re really fueling a lot of our success. I think the best questions to ask are really about the quality of the science. And so you mentioned, customers or prospects should be asking about primarily how are the assessments validated, how have you shown that they predict certain outcomes and really drilling down on the science. Because, as you said, with enterprise you’ll get IO folks on the other side of the sale, and those are very high level conversations and those are very informed buyers. And when you don’t have IO in-house though, which is the vast majority of our customers still, you should be asking and really drilling down on the quality of the science.

One quick way to do that is looking at a vendor, do they have a credible IO presence? Is it a good part of their org chart? And if they’re not, that’s an immediate red flag. If they don’t have good propeller heads on board. And so I think that’s an important part of it. Asking about the science and some of the details around the validity studies they’ve done, asking for validity studies maybe in your industry or more specifically for the roles that you’re hiring for.

William Tincup:    Oh, that’s nice.

Josh:    The vast majority of our customers don’t do their own validity studies or we don’t do validity studies for them. We’ve done more than 200 with our customer base, but that’s still a very small portion of the customer base. So the vast majority of the customers in terms of the validity evidence they’re relying on is what we call transportable validity evidence, where if we’ve done a dozen studies with customer service reps who have generally similar job descriptions to a prospects customer service reps, they can really have very good confidence that’s reliable, transportable validity information and that would be legally defensible in court, if it ever came to that, which probably wouldn’t, but that’s a really good sign that the vendor is well situated to be a good partner.

William Tincup:    How often do you think about validity studies? So you had an assessment rolling and it’s a great assessment, people use it, et cetera, et cetera, et cetera. So you get more science, you get more data, et cetera. Do y’all have a okay, every 18 months, or do you have a mechanism where you run your own studies on your own assessments?

Josh:    That’s a great question. I think when we first got going 15 years ago, we were doing validity studies for anyone who asked and we generally weren’t charging for them.

William Tincup:    Of course.

Josh:    And over time we’ve created so much data. And so much of the value of using assessments is the contextual data behind it. So as you start to get at scale and five years ago, we were just administering a million assessments a year, now we’re going to administer 10 million this year. And so the value of that data is really incredible. And so in terms of our assessments, we still do them on demand. Sometimes they’re free, sometimes they’re not. And one of the things there is we’re always looking to broaden the use cases. So we have a position that we’ve never done a validity study before for, but it’s a position that we think might be, more common in the future, then we have an eagerness, our IO team has an eagerness to get their hands on the data.

But with our biggest assessments, we’re constantly re-validating them and optimizing them. And I think that’s a lot of where the technology and the assessment space is headed, is using machine learning and some of these other techniques to optimize for what we call talent signal per minute. And that’s essentially a measure of efficiency. If you can deliver an assessment that gives you the same quality of data in 12 minutes versus 30, then clearly the first one is preferable. So we devote a lot of energy to studying individual items and making sure that some of our assessments, for example, are adaptive so that they will be very efficient at getting to the right ability level estimate, for example. So there’s a lot of interesting uses of technology that we utilize to get to greater efficiency there.

William Tincup:    I love it. So questions that practitioners… We’ll go down that line of thinking for just a second. When they’re thinking about doing this, maybe again, at the enterprise level they might or might not have the IO team, but they’re thinking about this, what are buying questions? Because you’ve done this for a long time so you’ve heard all the different questions from practitioners. What are some of the ones that you just love to hear because you know that, okay, they might not understand the science, but they get it, they innately really understand what’s going on here? And on the opposite side of that, what are questions that just, I would say, drive you into a wall? That’s probably not the right wording, but just questions that you’d like to retire.

Josh:    That’s a great question. On the second batch, there’s probably a long list of those.

William Tincup:    How much does this cost? Alrighty.

Josh:    Well, that’s actually one of them. So I think in the first batch, in terms of questions that I think are great and that we love to hear is we realized a long time ago that not everyone in the world loves assessments, loves tests as much as we do at Criteria. I always remind our team, especially the go to market folks, that people are not really buying assessments from us, they’re buying outcomes. And so when we get questions around impact and outcomes, there’s two main problems that people come to us looking to optimize for, or solve for. And those are generally around turnover that certain industries that we mentioned earlier have that problem a lot.

And the other is just a higher batting average or a higher hiring success rate, so something around productivity. So if it’s sales, they want higher performing sales people who sell more, or if it’s engineers, they want engineers who will get better ratings from their managers and write better code. And so when we get into an outcome discussion, that’s a really good sign for the prospect becoming a customer because the ROI of assessments is so dramatic. And so when they start talking about turnover reduction or the productivity orf their team, we just have such great data around that, that that’s a sign that you’re getting a customer who’s educated and doesn’t think of an assessment as check the box exercise.

And to your second part of your question around the questions we don’t like getting, you were joking about cost, but when a customer is really focused on cost, there’s a lot of assessment providers out there that are providing a check the box service that has some value in terms of making sure that an employee or a potential hire has certain basic skills, but they’re not really going to be predicting long term outcomes very well. And what we’re about is really changing long term outcomes for hires and helping companies optimize that. So staffing agencies are a big consumer of assessments, they’re not a big part of our customer base at all. Because the types of assessments they use are oriented around having someone show up and not be an embarrassment. They know how to use Microsoft Excel.

And look, we have a Microsoft Excel test, that’s fine. It’s fine as far as it goes, but really the science of our assessments is what differentiates us. So a customer that’s asking about the unit cost or comparing only based on cost is not really understanding that they’re buying science and that all assessments are not created equal. And so if they have quite a commoditized view of what, for example, a test should cost, that’s an argument we’re going to sometimes lose unless we can bring them around to the notion that hey, this is a strategic tool for optimizing your people asset.

William Tincup:    And so two questions left, one of them super easy. I’m assuming, especially on the enterprise side, but even on the other markets, that you’re integrated with ATS’s?

Josh:    That’s right.

William Tincup:    Makes sense, because the workflow just where you are, it would make sense that your post application you’re in there for all the compliance data and so you’re probably integrated with everybody on that side. The last question I have there is your favorite customer innovation story. Again, you’ve been doing this for a long time, so you’ll have a bunch of them, but maybe something recent where you just… And no names, by the way, you don’t have to name the customer. Just something where you feel like people have really, really, really, really understood and got more than they bargained for in terms of the outcomes or something that they’ve done that’s innovative that you’re like, “Wow, we didn’t think of that. That’s cool.”

Josh:    Well, one of the biggest customer… Because we get a lot and I think this is true of a lot of companies, but a lot of our best ideas come from our customers. And when we started out, and this is going back right to the beginning, because we’ve always been a SaaS platform and we’ve always had the model of customers, they pay us an annual subscription fee based on the volume they’re hiring and the size of their company or the size of the business unit they’re using it for.

And so when we got started 14, 15 years ago, William, the predominant model for selling assessments was on a per unit basis. It was transactional. You bought a thousand tests, whatever. And we never used that model. We always sold a subscription based service and small businesses would pay one price, big enterprises would pay a lot more. And one of the earliest innovations that we built into the platform, which was directly a result of customer behaviors, we were already encouraging people to use the assessments as early as possible. But one of our customers came to us and said, “No, I want to use it earlier than you’re thinking. I want to be the very top of the funnel, so you need to build a functionality that allows me in one click to integrate it into a Craigslist post where that is the first thing I get at the same time or even before the resume or a LinkedIn post or [crosstalk 00:    27:    09].”

And that was really a big eye opener for us. And that over the years became the predominant mode of accessing our assessments, is by embedding them directly in a job post. And now we’re famous for it because we have this model of, I don’t know if famous is the right word, but we’re known as having this unlimited use model. And once you start saying to customers, “Okay, if you pay us whatever it is, 10, $20,000 a year, you can use an unlimited amount of assessments. We’re never going to charge you more for using more.”

Then it’s like the buffet in Vegas, people start using the product. People start eating differently and they start using, in this case, the product differently. And now a lot of our customers find that they get the most value by using it right at the top of the process. And that’s now a best practice that we recommend. Although there’s certain cases where you wouldn’t want to do that, but certainly it’s the most common mode of using and consuming our assessments now is very much top of funnel.

William Tincup:    I love that. And it came from customers?

Josh:    It came directly from customers, yeah. We now take credit for the idea, but…

William Tincup:    Of course. You have to, at this stage the statute of limitations is [crosstalk 00:    28:    28].

Josh:    That’s what leadership is about right, [inaudible 00:    28:    31] for other people’s.

William Tincup:    So Josh, thank you so much for carving out time for us today. Thank you for being on the Use Case Podcast.

Josh:    Oh thank you for having me, William. It was really fun.

William Tincup:    Absolutely. And thanks for everyone listening to the Use Case Podcast. Until next time.

Music:    You’ve been listening to Recruiting Daily’s Use Case Podcast. Be sure to subscribe on your favorite platform and hit us up at recruitingdaily.com.