I’m here to talk to you about some of the lessons we’ve learned at Pymetrics about deploying AI in a regulated landscape. The problem Pymetrics focuses on is how do we find the right person for the right job. Everybody who goes through hiring has to solve this problem, and there is a variety of ways to solve it, but most of them are pretty poor.

Just to give one example, to call out a major tech firm that posted an ad recently, they tried to define the core feature of the job and really wanted somebody who was excellent at a skill called Kubernetes administration. They asked for 12 years of experience in this skill. But they were asking for expertise that doesn’t exist because this technology is only six years old. This sort of thing seems like a typo or a one-off but is actually pretty pervasive in the hiring realm.

Oftentimes, the things that you try to get when you look for the right person for the right job are impossible or impossible to define. At Pymetrics, we’re trying to solve that problem of scale using a data-driven approach.

The first thing Pymetrics did is we came at the problem of hiring. How do you find the right person for the right job in a way that’s different than a lot of other folks have? Instead of looking at resumes or skill sets, we took the hypothesis that many people could be a good fit for one job or another based on attributes of their personality or behavior that were measurable, and from that we could make decisions. We developed a set of core assessments that measure behavioral aptitude such as the effort or emotional style that somebody brings to a job, skills such as numerical reasoning, and communication, which is key to our day-to-day life.

It’s really important in the hiring landscape that we focus on ensuring that everybody has equal access to opportunity in the job context.

These three metrics of prediction, fairness, and explainability, we can then quantify in various ways, which we do for all of our clients. We quantify things like the efficiency that we bring to a client’s hiring life cycle. If they need to hire 1,000 or 10,000 people, how much time can a solution like Pymetrics save? We can quantify things like diversity. How much diversity do we bring into the company that can help offset some of the historical marginalization of underrepresented groups? We can also quantify things like the performance of people on the job, how effective they are and what the kind of candidate and client experience is over time. Over the last five or six years, we’ve developed a whole suite of metrics that we measure for everything. We have a proprietary, scalable, and really valuable data set that we use to drive our hiring decisions. We have a core of machine-learning algorithms that we use to make predictions and recommend who should be hired for a particular job. And lastly, we have really well-validated ways of demonstrating business value. So, that’s the nutshell of what Pymetrics does.

I want to bridge out from there to give three short lessons from this journey that we’ve learned so far. The first lesson is that, in addition to simply providing value, the fairness of the platform as a whole across many clients is a performance criterion for any analytics company. This is going to be true for everybody working in the analytics or AI space going forward. If it hasn’t hit your industry yet, it will hit your industry soon.

But a really important thing for us is that, unlike many of the analytics driven companies you see today, Pymetrics actually started out with regulations surrounding the decisions that we were recommending. The Equal Employment Opportunity Commission has developed standards for hiring assessments that say that a hiring assessment can only be deployed if it meets certain fairness criteria. If, for instance, a hiring solution is recommending that a certain fraction of white men like me get hired, then it must also recommend other demographics be hired at approximately the same rate within about an 80% margin.

Many of the tests that have been used historically such as cognitive testing, which is used broadly across a variety of industries in the hiring context, or even human resume reviews, actually fail this test. Cognitive testing has been reported in the literature to pass about 40% as many Black men as white men, which is just unacceptable in the hiring landscape and in the shape of the society we’d like to live in together.

There is a lot of missing trust in this field, and with good reason.

As the analytics landscape evolves throughout the broader business context, the real value comes from developing new data sets and new ways to demonstrate business value to a client.

I talked about Pymetrics’ approach to data collection. We use our specific cognitive assessments, our approach to machine learning, we have our particular special sauce as the core analytics engine of Pymetrics, and the way we quantify business value. One really important thing to note is that machine learning can be meaningfully commoditized. As the analytics landscape evolves throughout the broader business context, the real value comes from developing new data sets and new ways to demonstrate business value to a client.

But there is a fourth step that is coming and that we’ve learned a lot from at Pymetrics by starting in the regulatory context, which is the most important step in the analytics life cycle. It is integration into society. This is not just a legal framework about government regulations, but also about broader metrics that build trust as we deploy tools like these. Honestly, this phase is just beginning, and I think it’s going to be one of the most interesting and fruitful ways in which folks can engage in this. With that, I will give my big thanks to the organizers of this conference. It’s been a pleasure to be here. Thank you very much.