A team of researchers, including two from the Naveen Jindal School of Management, has found a way to improve the hiring process for the sales profession by using artificial intelligence.
Dr. Howard Dover and Dr. Khai Chiong, both from the Jindal School’s Marketing Area, and their co-authors won the Best Paper on AI & Marketing Award 2024 from the American Marketing Association for “Can AI and AI-Hybrids Detect Persuasion Skills? Salesforce Hiring with Conversational Video Interviews.” Their co-authors are Dr. Ishita Chakraborty from the University of Wisconsin at Madison and Dr. K. Sudhir from Yale University. The article appears in the Jan.-Feb. 2025 issue of Marketing Science, one of the peer-reviewed academic journals tracked in The UTD Top 100 Business School Research Rankings™.
The study looks at how AI and AI-human hybrid models analyze video interviews and evaluate persuasion skills by analyzing words, tone and body language during interviews of candidates for sales positions. It concluded that AI-human hybrid models are the most cost-effective method, followed by AI alone. In contrast, human-only interviews are the least cost-effective in large-scale entry-level hiring.
Chiong, an assistant professor in JSOM’s Marketing Area, said because business-to-business sales is an important employment sector for Jindal School students and there is high demand for skilled professionals in the B2B sales domain, he was inspired to look into AI and AI-human hybrid models for sales hiring.
“The Center has a lot of sales role-play video data that made it possible to study this domain,” he said. “We want to know how AI can be used to analyze conversational interview videos for cost-effective yet high-quality sales recruitment and training. Moreover, can human input improve on AI? When we started this project back in 2019, modelings of video data were still nascent, and we had to develop and learn new methodologies along the way.”
Chiong said the research team feeds text, audio, and visual input into the AI model so that it can learn the importance of aspects of verbal and non-verbal communication between the buyer and seller such as hand gestures, body postures, interactivity, and the content of the conversation. The AI model uses these features to measure persuasion skills.
The human-AI hybrid model works better than AI-only models, Chiong said, because humans can capture nuanced aspects like confidence and nervousness.
“Our current version of AI may find it hard to capture,” he said. “For example, humans seem to differentiate whether hand movements reflect innate enthusiasm or impatience and nervousness. Also, humans can capture these additional nuances from just the early stages of the interview.”
Chiong said that AI models, on the other hand, especially the recent development of Large Language Models, are quite effective in transcribing the video to text accurately and capturing a salesperson’s qualities such as active listening, confidence, politeness, and the ability to be collaborative.
Current AI hiring models tend to focus on hard skills, which are specific, measurable, and often tied to keywords that match job descriptions. These are easily detected in résumés. Dover, a clinical professor in the Jindal School and director of its Center for Professional Sales, said that many companies use role-play — simulated scenarios during the hiring process to assess skills, decision-making ability, and behavior in real-time — to make hiring decisions.
“The industry tends to lean on role plays to assess sales talent,” he said. “The power here is in how we can use AI tools to score and predict hireability.”
Chiong said persuasion skills are important in many jobs, not just in the sales profession.
“These soft skills cannot be easily screened through résumés,” he said. “This is the reason why the use of conversational videos to score interviews and predict abilities is the highlight of our study.”
Since AI models can inherit bias from training data or in the way it is programmed, which can lead to unfair or discriminatory hiring outcomes, the team took steps to minimize this risk.
“Our training sample was scored by a large panel of judges who were seasoned industry professionals,” Chiong said. “This panel is balanced in terms of gender. We also use statistical methods to remove potential confounding influences and biases from any particular judges or sellers. Our study would be applicable to jobs with elements of persuasion, which describe many types of jobs and industries such as law, marketing, non-profit fundraising, as well as many managerial positions,” he said.
The study suggests that having even just one human evaluator early in the hiring process can help optimize costs and increase accuracy.
“Given the need to reduce costs, I do see this being valuable in industry,” Dover said.
Chiong said the team has received inquiries from companies, which would indicate interest in adopting the study’s framework.
“Our study also echoes prevailing sentiments in the AI space that AI does not fully replace humans, but knowing how to augment AI with human collaboration will be important,” he said.
Dover said the team’s research stands out because it was an ambitious project that took several years to complete with several data collection cycles.
“We had the unique ability to recapture data for this project,” he said. “I think the unique use of AI and video models are key to why this paper was recognized.”
Chiong said B2B sales is an important but understudied area of research, as there are many challenges such as the lack of institutional knowledge and data about companies’ internal sales operations.
“This paper is being recognized for this unique contribution to the field,” he said. “The collaboration between UTD, Yale and Wisconsin has also uniquely positioned us to advance more studies in this area in the coming years. I think you will see a lot more research publications using Center for Professional Sales data. Also, we will be launching several other initiatives for the Sales and Sales Technology community.”
