Sophie Benbenek, TrueAccord’s Head of Data Science, discusses the early days of building our machine learning based engine, Heartbeat, and how it has evolved since. Hear about our approach to machine learning, how we move from heuristics to statistical models, and other anecdotes from the early days of TrueAccord.
TrueAccord’s machine learning based system handles millions of consumer interactions a month and is growing fast. In this podcast, hear our Head of Engineering Mike Higuera talk about scaling challenges, prioritizing work on bugs vs. features, and other pressing topics he’s had to deal with while building our system.
Innovative automation processes are finally gaining traction in debt collection, as companies increasingly distance themselves from costly and unmanageable call centers. And now, with an eye on continuous process improvement, a new focus on experimentation is enhancing the way these companies recover revenue and create a more effective user experience. Experimentation engines – whereby various collection scenarios and features are tested and evaluated based on real-time data – empower creative and customized contact and offer strategies that improve liquidation as well as customer satisfaction.
Typical Challenges for the Call Center Model
The traditional debt collection call center model faces multiple challenges. Because of their commission compensation model, collection agents often use aggressive tactics on the phone, pushing for an immediate lump sum payment, or a short-term installment option to speed payment. Even if the consumer picks up the phone at all (which in today’s smartphone culture is becoming far less likely), they feel pressured and may commit to a plan they simply can’t afford. The result is an installment plan that breaks, many times after the first payment, and consumers often charge back the phone payment because they felt antagonized about being pressured to begin with. The call center cost structure also cannot afford to support highly customized plans with irregular payment schedules, missing out on another segment of consumers. All of these add up to a significant disadvantage given today’s consumers and their financial needs.
Flip the System on It Head with a Machine Learning Based Approach
The modern approach to debt collection is omnichannel, digital-first, consumer-centric and leverages data and experimentation to determine the best course of action based on consumer preference and behavior.
TrueAccord’s system communicates with consumers automatically through a wide range of digital channels, including email, text and social channels. And because it’s digital-first and fully reactive to consumer behavior and preferences, it’s a far less aggressive, much more personalized collection environment that delivers superior results when competing with call centers. Historical data collected over several years, combined with machine learning algorithms that evaluate individual behavior and preferences, enables this highly targeted and personalized treatment. Two to three email interactions per week serve as a baseline, with added channels in support and reactive communications responding to consumer interactions when needed.
This approach is also highly collaborative, focused on educating consumers and treating them the way they want to be treated. When they’re ready to commit to a plan, they just view payment options online and choose the one that makes the most sense. The result is higher liquidation rates in the long run, higher payer rates, and higher consumer satisfaction that leads to fewer complaints.
Machine Learning Drives the Experimentation Engine
The most important asset in the TrueAccord model is the data collected and analyzed over time that enables us to accurately predict what messages people respond to, what payment offers work best, and for which type of consumer. This complex data-driven system is part of our DNA and entails a lot of moving parts that allow us to truly understand what resonates with each consumer.
The driving force behind the system’s ever evolving performance is an experimentation engine that allows us to test various scenarios to see how collection processes work and how they can be improved. Since digital-first channels are highly instrumented and offer real time tracking on our website, we can learn in short cycles and continuously improve. To launch an experiment, we establish a hypothesis we want to test, monitor what’s happening in the conversion funnel at each touchpoint, see how each product or plan is being used and where consumers are dropping off. Even when an experiment fails, we learn from the data and make future iterations in a continually improving system. We partner strategically with our clients to customize experiments for their product lines and make experimentation-based optimization an ongoing process.
A few sample experiments:
Aligning Payments to Income
The number one reason payment plans fail is consumers don’t have enough money on their card or in their bank account. Our hypothesis was that if you align debt payments with paydays, consumers are more likely to have funds available, and payment plan breakage is reduced. The experiment tested three scenarios: one as a control, one defaulting to payments on Fridays and one where consumers used a date-picker to align with their actual payday. After testing and analysis, we found that the date-picker approach worked best, lowering breakage without negatively impacting conversion.
Self-service Payment Experiences Reduce Costs and Breakage
Consumers with debt often can’t always predict when they’ll be paid or how much. Our hypothesis was that by allowing them to self-service their payment plans and make modifications along the way (based on changes in their lives), we would reduce the need for interaction and improve the customer experience while reducing breakage. This experiment was also a success, reducing breakage rate, and also lowering call rates because before its launch, consumers had to call to change their plan. By making the desired functionality readily available, we were able to increase payment plan success rate and save agent time.
Even Failures Are a Learning Experience
One hypothesis we tested was that customers that dropped off our radar after not choosing a plan could be enticed to sign up for a new plan if offered longer payment plans. After sending texts and emails based on their behavior, we found that new sign ups simply didn’t materialize by just offering longer payment plans with referring to the consumer’s specific life situation. The offers had a high open and click rates, but not sign ups. This indicated that we were on the right track but needed to iterate and come up with an alternative solution.
An experimentation engine allows every company to test their own hypotheses to see if their customized solutions work or not. A digital-first, highly instrumented experience allows us to run dozens of experiments concurrently, learning from each experiment so we can progressively improve our experience and results. Even when experiments fails, they unearth insights that can be used to improve performance next time as part of follow on experiments. In the world of debt collection, testing and continuous improvement means better results in the long run.
Our CEO, Ohad Samet’s, recorded a podcast with Lend Academy discussing the positive impact technology is creating in the collections space and the need for more innovation. Will discuss TrueAccord’s unique approach to debt collection using data-driven, digital communications to create deeply personalized consumer experiences. The podcast also covers the current state of the collections industry and where it’s likely headed as regulatory pressure, consumer preferences and compliance requirements converge. Will cover how TrueAccord is using machine learning to deliver deeply personalized and engaging experiences for consumers while achieving higher recovery rates across various debt types.
Tune in and learn:
- The state of the debt collection industry today and where it’s headed
- How the use of machine learning is personalizing the debt collections experience for greater conversions
- Why code-driven compliance outperforms traditional collections practices by reducing risk to organizations
- How understanding consumers’ preferences for easy, self-service options with flexibility empowers more consumers to pay off their debt and get on a path to financial health
If you’re rather read the transcript, download it here.
Experimentation in the movies sometimes gets a bad rap – you think of mad scientists blowing up labs or aliens arriving to probe unsuspecting humans or accidental AI monsters. It leaves the imagination to form an image of experimenters as cold-hearted, calculating and removed from reality. Real world experimentation is typically much more mundane, but the stereotypes often linger. This is unfortunate. The primary goal of experimentation (if you’re not a mad scientist) is: Does this thing work like I think it does? Does this feature deliver the results or benefits it is supposed to? If not why? This makes it an extremely powerful tool for designing products that work and are actually good for customers.
At TrueAccord we believe that experimentation is an integral part of designing a product that fulfills our mission to “reinvent the debt collections space by delivering great customer experiences that empower consumers to regain control of their financial health and help them to better manage their financial future.” Whenever possible we launch experiments, not outright features. This strategy has three main and essential benefits:
Tests our instincts are right or our models are functional
Allows us to gain valuable insights into who our customers are and what they need
Mitigates potential negative effects
Test Our Instincts: How do you ensure your team is actually moving the product forward? Only investing energy in features and experiences that will create an effective and positive debt collection experience? Experimentation. The TrueAccord team is full of clever people with clever ideas, but we know it’s important not to found our product on untested hunches. By testing our instincts before taking another step in the same direction, we make sure we invest energy where it matters and wait to develop our knowledge base before proceeding in directions we clearly do not yet understand.
Customer Insights: Understanding why your product works is often more important than understanding if it works. The real benefits of an experimentation infrastructure are in its ability to provide diversified and descriptive data as well as the emphasis on stopping to take a look. At TrueAccord we know it’s essential to understand if we’re looking at the problem the right way and if not what we’ve missed: Do we understand our customers’ needs?
We launched a new “better” email format that we rolled out as a variation across a spread of existing email content. After a 3 month run, we asserted that it was indeed performing significantly better in terms of both average open and click rate. This was surprising. We hadn’t changed anything that should have affected opens.
New base template content saw an open rate increase of ~10%! First Email: New base template and Second Email: Control
Upon further investigation, we realized that the new format unintentionally changed the email preview from displaying the start of our email content to consistently showing a formally-worded disclaimer! We then launched another experiment to ensure our findings were correct.
Mitigates Negative Effects: It’s easy in any industry to get blindsided by simple outcome metrics, especially in debt collection where the end objective is repayment. At TrueAccord we would consider it a failure if our product worked, but it worked for the wrong reasons – if our collections system converted, but didn’t provide a good experience for the consumer. Experimentation is our first wall of defense against treading down this path.
After researching existing accounts, we realized there was a need for more self-service tools in payment plan management. We developed a new payment plan account page and rolled out an experiment that automatically redirected some customers to this page any time they viewed the website while their plan was active.
We found that this did decrease payment plan breakage and increase liquidation, but because our system was set up to detect other types of impact we discovered it also increased outreach to our engagement team in the category of “Website Help”. Consumers were confused as to why they were not landing on the pages they expected upon navigating to our website. We had the right idea, but our implementation was not ideal for the consumer.
Experiment vs Control: % of inbound engagement team communication by category (total # of inbound communications was approx. the same)
Experimentation is not foolproof, getting these benefits comes from having an infrastructure that allows you to assess if what you built is useful and, if designed correctly, understand why. Indeed, through experimentation, we’ve grown our product to function effectively over diverse areas of debt and over the past few months alone improved the number of people who complete their plans by almost 4%, with a few simple experiments. Every small change compounds, and at TrueAccord’s scale, this means many more people who pay without experiencing any disruption. ! Check back soon for how we designed an experimentation structure that allows us to reap the benefits described above and fuel our collections product forward.
Yesterday, in an article on InsideARM.com, Tim Bauer, the President of InsideARM, described a somber state of affairs:
The TCPA and the 2015 FCC Rules interpreting the act have effectively eliminated the use of technology to efficiently call cell phones. Land line usage is dropping like an anchor. The CFPB is on the brink of announcing proposed debt collection rules that are likely to reduce the number of call attempts that can be made. Now, add this latest call blocking technology and the industry is challenged again.
This is a strong statement from a prominent thought leader in the debt collection industry. Mr. Bauer pointed out many efforts by different regulatory agencies and how they impact call centers: “anecdotal reports of right party connects down by 15-30%”, as the FCC includes debt collection calls as an “unwanted call” category in it’s “robocall” blocking initiatives.
At TrueAccord, we agree. The industry has been seeing tremendous pressure on its ability to call consumers efficiently, not only because of regulatory pressure – this pressure is driven by consumer preference, and the fact that consumers often opt to not pick up the phone, not to mention opening a letter. As strong advocates for technology in debt collection, with our CEO now part of the CFPB’s Consumer Advisory Board, we will continue to support forward thinkers such as Mr. Bauer and others who call for the use of new technologies in debt collection. It is the consumer friendly, smart, and efficient approach for the 21st century, and we strongly encourage our peers in this industry to begin adopting and utilizing these channels in preparation for the CFPB’s expected Notice of Proposed Rulemaking, expected later this year.
Debt collection is a highly litigated activity. Compliance personnel and systems budgets are crowding out other investments. It’s appropriate: debt collectors and creditors are often hit by class action lawsuits and government action, leading to huge fines and settlements. Reducing risk is their primary priority. When examined closely, though, the traditional debt collection model attracts numerous compliance issues. The legacy approach is being replaced by machine learning and digital first systems. These code-controlled systems offer predictable, scalable, and auditable operations that, coupled with best in class user experience, significantly reduce the risk of litigation and regulatory action.
The traditional model invites regulatory scrutiny and lawsuits
Collectors often cite compliance concerns as impediment to adopting new technologies. Lawyers are concerned about TCPA exposure from text messaging, consent requirements for emails, and FDCPA violations when using social media. These concerns are unfounded: text messages can be safely delivered if consent and revocation are properly documented, the CFPB saw no need for consent to email (as reflected by a growing body of opinions, as well as its 2016 rule outline), and social media can be used with restraint. While dragging their feet on evaluating new technologies, compliance departments embrace and perpetuate much bigger risks: the prevalent use of human labor, over reliance on phone calls, and the outdated, fragmented interfaces used by collectors.
Humans are the weakest link in the compliance chain
Traditional wisdom says that only people collect from people. That claim is demonstrably false. People are subject to biases and acting emotionally when interacting with debtors – which is why machine learning based systems collect better than humans. People may be tired, angry, or distracted. They can be baited into violating the FDCPA by a ill-meaning debtor. The prevalent commission-based compensation model, a broken and outdated model for collections, puts them in odds with debtors whenever they interact. Human beings just cannot do error-free work, no matter how trained or experienced they are.
Keeping appropriate staffing levels is another challenge for collection teams. Large market participants report 75-100% annual turnover rates (per the CFPB’s operational survey), requiring constant hiring of collection staff. Training and overseeing these new people is a daunting task, especially with the ever changing case law and legislative landscape in the collection space. Providing an efficient and fully compliant collection experience while relying on new and untrained collectors is almost impossible.
Phone calls are a dying communication method
Consumer preference is shifting away from phone calls, but phone call compliance would have been difficult even if that wasn’t the case. Calls are a compliance liability due to their frequency, their real-time nature, and the overall regulatory sentiment towards them.
Collection calls must be frequent to reach consumers. On days when an agent works an account, they may attempt to contact the consumer 4-6 times, often as frequently as 10 times per day. Consumers aren’t picking up the phone, so agents need to make more call attempts to try and reach them. While most states, and the FDCPA, don’t limit call frequency, high frequency of calls often leads to complaints and lawsuits alleging harassment. Collector take this huge risk because calling is the only tool they understand.
Collection calls are also real-time. No matter how elaborate call scripts are and how experienced collectors may be, it is impossible to completely control the development of any individual call. Voice analytics software is limited, unable to identify most baiting and escalation issues. Real time monitoring of all calls by supervisors is financially implausible. Collection agencies are forced to settle for the best training possible, clear escalation paths for collectors whose calls go badly, and hoping for the best. Realistically, when making a large volume of calls, every day will have some potential violation.
Finally, regulation has been working against phone calls for the past few years. The FCC’s ruling limiting the use of ATDS has been devastating, and expecting it to be completely undone by the new commissioner is a pipe dream – government is not debt collectors’ friend. States like West Virginia and Massachusetts have enacted call frequency limitations, and the CFPB’s new rule outline includes a 6-times-per-week limit on call attempts. All signs point to a future where phone calls cannot plausibly be the main channel for collecting debt with any semblance of compliance.
Code driven compliance is here, and it’s a big step forward
Code driven compliance gives us complete control on what actions can be triggered by our system. It’s one of the components in Heartbeat, our machine learning-based, digital first collection platform. Heartbeat is a leap forward in debt collection, and its compliance advantages are many: from better user experience to perfect auditability.
Best in class user experience in debt collection is a compliance advantage
Many if not most of debt collection lawsuits hang on a technicality. A word is arguably missing or written in a debatable way. It’s unclear whether 8 calls or 9 calls constitute harassment. Often, consumers don’t resort to lawyers because they know for a fact they have been wronged – it is often not clear that they have been – but because their experience with the collector has been bad enough to push them to seek defence or retribution. Great user experience is therefore not only a way to improve the creditor’s brand perception and returns, but also a way to reduce the rate of complaints and lawsuits. TrueAccord’s Heartbeat system attempts to contact consumers an average of 3 times per week, compared to 4-6 times a day for traditional agencies. That, paired with best in class web and mobile experience and a helpful customer service department, significantly reduces consumers’ desire to sue for, or complain about, ambiguous technicalities.
Consumers get a consolidated account page showing all their options
Since more than 90% of Heartbeat’s interactions with the consumer do not involve a human collector, human beings are only needed for a fraction of the work. TrueAccord is able to hire skilled workers and pay them a living wage, with no commission component. Knowing that they will earn a good salary working for a technology startup reduces any incentive our team members would have had to fight with or harass consumers. That, in turn, contributes to great user experience and reduces compliance risk.
Pre-approved content and an integrated system eliminate human error
Human error is the biggest challenge for compliance departments. Collectors today need to navigate multiple systems to call, negotiate with, and collect payments from consumers. Updating the results of a call is often a complex process, requiring yet another system. Many requests to unsubscribe numbers, cease and desist communications, or simply to provide debt verification are lost and lead to complaints. This fragmented process is extremely tedious and time consuming, and inherently flawed. Letting collectors write their own emails and text messages is too much risk – something that will surely lead to violations on a daily basis.
TrueAccord’s content approval console
Heartbeat takes a code controlled approach to communications. Every outgoing communication is pre-written, then reviewed and pre-approved by TrueAccord’s legal team. Every email, text, web page and letter have to pass TrueAccord’s content guidelines driven by law, policy and procedures, including required disclosures and forbidding certain words and phrases in subject lines, or in the body of communications. Our clients’ legal and content team are also involved in commenting on our procedures as well as specific content items, to make sure we fit each company’s risk tolerance. Heartbeat will only send text messages to numbers that it knows it has express consent to text, and that have gone through an ownership check within a defined time period. Even when collectors respond to inbound consumer emails, they use pre-written replies that then direct Heartbeat how to proceed in serving the consumer. The decision to proactively communicate is strictly based on Heartbeat logic, not on collector whims; collectors cannot decide to contact consumers whenever they see fit.
After contacting consumers, the system monitors their response. Consumers can easily opt out of communications, by replying to a text message or by clicking a link in every email that lets them easily unsubscribe from future email communications. Every email and every payment page contain a link that lets consumers ask for debt verification via a few simple online steps instead of a cumbersome and mail-based process. Every interaction is designed to give consumers an opportunity to ask for more information or limit communications to their preferred channel. Though easy to dismiss as an invitation for abuse, these options increase consumer engagement and result in overall better collections – while significantly reducing complaints about continued communications and missing documentation. These two categories have consistently been the top reasons for filing CFPB disputes ever since its dispute portal was made public.
The compliance firewall: enforcing compliance at scale
Human collectors are expected to remember dozens, maybe hundreds of compliance laws and regulations as well as creditor-imposed rules. It’s an impossible task, greatly simplified by Heartbeat’s Compliance Firewall. Since it controls all contact decisions by code, Heartbeat can enforce its compliance policy at scale on every interaction without needing to train human collectors. Contact timing or frequency, matching content to the right stage in a consumer’s process or preventing the use of unsubscribed contact methods, even making sure that a consumer doesn’t get a payment offer that the creditor didn’t approve – all are controlled by the Compliance Checker. Any attempted action outside of its well defined policy is dropped. Since it’s code controlled, it cannot forget to check the time and call a consumer after 9pm or before 8am.
The Compliance Firewall also allows updates to policies and procedures. Every new update can be implemented with accuracy within days, once the appropriate code is written. By taking judgement away from the collector and subjecting all contact decisions to a data-based, code-controlled system, Heartbeat makes the optimal decision for consumer experience and driving payments, without harassing the consumer or violating the myriad of restrictions that govern debt collection.
The easiest system to audit
Compliance requires tight monitoring, and creditors audit a large sample of collection activities by their vendors. With so many voice calls, even if they are all recorded, complete and accurate audits are impossible. Auditors need to sample cases and hope to find the right patterns, or employ a large and expensive team for sufficient coverage. Heartbeat eliminates almost 95% of phone calls (typically attempting to reach the consumer 3-5 times over a 90 day period), instead focusing on written communication. Back and forth written interactions are easier to capture, store, and search. The system also saves consumers’ browsing pattern on the website and their interactions with the content they receive. It’s easy to track consumer behavior and how the system responded to it, as well as why it made a specific decision. Code controlled compliance means that decisions are easy to replicate and trace back in case they’re questioned.
A readout from TrueAccord’s event-based audit trail
TrueAccord’s system also has an audit interface for creditor audits. Compliance staff can easily search for accounts and review all collection activity – including recorded calls, emails, and every other contact. It’s a much easier approach to compliance and controls than an unwieldy excel file or PDFs dropped in an FTP folder. TrueAccord’s data retention and tracking of consumer behavior provide a fuller snapshot of Heartbeat’s collection decisions and how consumers reacted to them.
Code driven compliance is the future
We examined the inherent risks in traditional collection activities and how sticking to the phone as the leading collection tool in a call center environment creates more risks than rewards. Then, we dove into how code controlled compliance offers predictable, pre-approved, and consistent collection strategies that are easy to audit and understand. The coming years will see more and more creditors and collectors move to these machine learning based systems, as they demonstrate dominance in returns and compliance. It’s time for risk averse compliance departments to realize that they are putting businesses at risk by sticking to their phone-based roots, and look beyond tradition. A whole world of mature, stable and trustworthy technologies awaits.
TrueAccord serves major issuers, debt buyers and lenders across the US. We compete with traditional collection agencies and beat them: TrueAccord collects more than 1.5 times the competition in a typical 90 day placement period. We use a machine learning based system, HeartBeat, that replaces the traditional call-heavy model with digital first communications that compliment consumer behavior and mode of communication, but that makes our liquidation curves look different from traditional agencies.
Traditional liquidation curves
Traditional liquidation curves typically shoot up in the first 30-45 days, followed by a plateau around 60-80 day, with a possible bump towards the end of the placement window. This pattern is driven by several factors.
Routine: agents receive fresh accounts and are eager to call them. They fire up dialers and quickly reach consumers who can either pay or be lightly pressured to pay. After a few weeks of calls, agents are tired of calling the same consumers. They heard what they think are excuses, have driven all the easy payments they could drive, and are ready for new accounts. “Old” accounts, as old as 30 days, get a worse treatment. Collection managers know this, and try to trick collectors into thinking they got fresh accounts by pulling the accounts out from the system and re-entering them. This rarely works. Collectors lose focus and with it, performance.
Net present value: settlements are better than payment plans for collectors – they mean more money now, versus a payment plan that may fail, and require reminders and additional work by the collector. Collectors opt for more settlements earlier, if they can get the consumer on the line. Under the pressure of a call, the consumer may commit to a payment plan. In this case the collector prefers as high a monthly payment as possible, since they assume the payment plan will fail early. The consumers, struggling with irregular cash flow and a large payment they shouldn’t have committed to, fail payment plans at a staggering rate: as much as 50% of payment plans fail.
Remorse: consumers who agree to settlements or plans often feel remorse after getting off the call, and tend to charge back on payments they made. Chargeback rates in the debt collection industry are so high (rates as high as 2% are not rare), that most payment providers won’t work with collection companies.
The initial bump in liquidation is often enough to beat other phone based agencies. Since all agencies use the same methods, a slight advantage in selecting the right accounts to call first can get an agency ahead of its unsophisticated peers.
The TrueAccord liquidation curve
In contrast to the traditional curve, TrueAccord’s liquidation curve is somewhat linear. It often starts lower than the traditional agency, but continues to rise through the placement period until it crosses and exceeds its competitors. That inflection point can happen as late as day 80 (before the algorithms have been tuned, early in a pilot) and as early as day 15 (once the algorithms have learned how to handle a new product). The difference is driven by several factors.
Data driven treatment at scale: TrueAccord’s system is machine learning based and digital first. Since it starts with an email, it can initiate contact with all consumers easily, without having to call them often – and consumers are much more likely to respond to digital communications than to a phone call: while Right Party Contact rates often hover around 4-5%, email open rates on TrueAccord’s platform reach 65-70% and click through rates reach 30-35%. Once it sends its first email, it uses real time tracking of consumer responses to tailor its next steps. The system relies on hundreds of millions of historic contact attempts to optimize its contact strategy. If the consumer doesn’t reply, the system can automatically switch between channels (from email to text, call, letter, and so on) to reach the consumer. It also uses data to figure out what time of the day to contact the consumer that will yield the best response rates, call centers are limited to just making phone calls, which often consumers ignore because they are busy or just don’t pick up calls from unknown numbers. Since a machine doesn’t get bored, it continues contact attempts (3 a week on average) until it is told to stop. Targeted, consistent communications at scale mean that more consumers will interact with our system compared to a call center.
Optimizing for liquidation: a data driven system can use historical data to understand what best fits consumer needs and leads to better liquidation. It doesn’t need to push for early settlements because its automation lets it serve each consumer according to their needs – making custom tailored plans viable. Consumers get easier payment terms that fit their needs, and end up paying more. We convinced several of our clients to move from a default payment plan length of 6 months to 12 months. Contrary to call center based intuition, these longer plans get more consumers to sign up and don’t cannibalize settlements, in turn leading to an increase in liquidation. The machine learning system can service these plans at scale and reduce failure rates: TrueAccord payment plans complete as much as 89% of the time (as low as 11% breakage). By the time payment plans for traditional collectors fail their second payment, TrueAccord’s liquidation rates start soaring.
Best in class user experience: consumers don’t like phone calls or letters. They prefer 24/7, personalized, easy to use services – and collections aren’t any different. Using our system they can customize and sign up for settlements, payment plans, or ask for debt verification. Having access to their account information, and a sense of control over payment options, consumers don’t feel pressured or remorseful after paying. TrueAccord’s chargeback rates are next to not existent.
Machine learning based debt collection is different in many ways that benefit creditors and consumers. Our liquidation curve tells the story of how our system behaves differently than call center based collections – serving consumers at scale, using their preferred communication channel, and while tailoring payment solutions that work for them.
Contributors: Vladimir Iglovikov, Sophie Benbenek, and Richard Yeung
It is Wednesday afternoon and the Data Science team at TrueAccord is arguing vociferously. The white board is covered in unintelligible hand writing and fancy looking diagrams. We’re in the middle of a heated debate about something the collections industry has had a fairly developed playbook on for decades: how to use the phone for collections.
Why are we so passionately discussing something so basic? As it turns out, phone is a deceptively deep topic when you are re-inventing recoveries and placing phone in the context of a multi-channel strategy.
Solving Attribution of Impact
The complexity of phone within a multi-channel strategy is revealed when you ask a simple question: “What was the impact of this phone call to Bob?”
In a world with only one channel, this question is easy. We call a thousand people and measure what percentage of them pay. But in a multi-channel setting where these people are also getting emails, SMS and letters, there is an attribution problem. If Bob pays after the phone call, we do not know if he would have paid without the phone call.
To complicate matters further, our experiments have shown that phone has two components of impact:
- The direct effect — the payments that happen on the call.
- The halo effect — the remaining impact of phone; for example seeing a missed call from us and going back to an email from us to click and pay.
To solve the attribution problem and capture both components of impact, we define the concept of incremental benefit as:
Intuitively, the incremental benefit of a phone call is the additional expected value from that customer due to the phone call. For example, assume Bob has a 5% chance of paying his $100 debt. If we know that by calling him, the probability of him paying increases to 7%, then the incremental benefit is $2 (100 * (0.07 – 0.05)).
How we calculate incremental benefit
Consider the incremental benefit equation in the last section. It requires us to predict the probability of Bob paying for each scenario where we call him and do not call him.
Hence we created models that predict the probability of a customer paying. These models take as inputs everything we know about the customer, including:
- Debt features: debt amount, days since charge-off, client, prior agencies worked, etc
- Behavioral features: entire email history, entire pageview history, interactions with agents, phone history, etc
- Temporal features: time of the day, day of the week, day of the month, etc
The output of the model is the probability of payment by the customer given all of this information. We then have the same model output two predictions: probability of payment with the current event history, and probability of payment if we add one more outbound phone call to the event history.
Back to our example of Bob, the model would output the probabilities of 7% and 5% chance of paying with and without an additional phone call respectively.
Optimal Call Allocation
The last step of the problem is choosing who to call, and when. The topic of timing optimization deserves its own write-up, so we will close with discussing who we call.
Without loss of generality, assume that we would only ever call a customer once. The diagram below has the percentage of customers called on the x-axis. And the y-axis is in dollars with 2 curves:
- Incremental Benefit — this curve shows the marginal incremental benefit of calling the customer with the next highest IB
- Avg cost — this horizontal curve shows the average cost of an outbound call
There are two very interesting points to discuss:
- Profit max — calling everyone to the left of the intersection of incremental benefit and avg cost is the allocation that maximizes profit. Every one of these calls brings in more revenue than cost.
- Conversion max — notice that incremental benefit dips below zero. This is especially true when you remove the assumption that we only call each customer once. The point that maximizes conversion for the client is to call everyone to the left of where incremental benefit intersects with zero.
Our default strategy is to call all customers to the left of the profit maximizing intercept. Interestingly, an intuitive investigation of the types of customers selected reveals customers at two extremes: we end up calling both very high value customers that have shown a lot of intent to pay (e.g. dropped off from signup after selecting a payment plan) and customers where email has been ineffectual (e.g. keeps opening emails with no clicks or no email opens.)
The world has become increasingly digital, and a multi-channel strategy is the right response. Bringing the traditional tool of phone, as just one channel within this strategy, forced us to rethink a lot of assumptions and see where the problem led us. We began by replacing the traditional “propensity to pay” phone metric with incremental benefit, found ways to predict this value, and implemented a phone allocation strategy that maximizes profits for the business.