Couldn’t attend Transform 2022? Check out all the top sessions in our on-demand library now! Look here.
Any organization that adopts artificial intelligence (AI) and machine learning (ML) in their business will want to use these powerful technologies to tackle thorny problems. For the New York TimesOne of the biggest challenges is striking a balance between meeting the latest target of 15 million digital subscribers by 2027 while getting more people to read articles online.
Today, the multimedia giant is digging into that complex cause-and-effect relationship using a causal machine learning model called the Dynamic Meter, which aims to make its paywall smarter. According to Chris Wiggins, chief data scientist at the New York TimesFor the past three or four years, the company has worked to scientifically understand their user journey in general and how the paywall works.
In 2011, when the Times began focusing on digital subscriptions, “metered” access was designed so that non-subscribers could read the same fixed number of articles each month before hitting a paywall that required a subscription. That enabled the company to acquire subscribers while also allowing readers to explore a range of offers before subscribing.
Machine learning for better decision making
Now, however, the Dynamic Meter can set personalized meter limits – that is, by providing the model with data-driven user insights – the causal machine learning model can be prescriptive and determine the right number of free articles that each user should get so that they have enough interest in the New York Times to subscribe to continue reading.
MetaBeat will bring together thought leaders to offer advice on how metaverse technology will change the way all industries communicate and do business October 4 in San Francisco, CA.
According to an blog post written by Rohit Supekar, a data scientist at the New York Times’ algorithmic targeting team, at the top of the site’s subscription funnel are unregistered users. At a certain meter limit, they are presented with a registration wall that blocks access and asks them to create an account. This gives them access to more free content and a registration ID allows the company to better understand their operations. Once registered users reach a different meter limit, they will be presented with a paywall with a subscription offer. The Dynamic Meter model learns from all this logged user data and determines the appropriate meter limit to optimize for specific key performance indicators (KPIs).
The idea, Wiggins said, is to establish a long-term relationship with readers. “It’s a much slower problem that people deal with for weeks or months,” he said. “Then at some point you ask them to become a subscriber and see if you did it right.”
Causal AI helps understand what might have happened
The most difficult challenge in building the causal machine learning model was setting up the robust data pipeline to understand the user activity of more than 130 million registered users on the New York Times‘ said Supekar.
The main technical advancement driving the Dynamic Meter is around causal AI, a machine learning method where you want to build models that can predict what would have happened.
“We’re really trying to understand cause and effect,” he explained.
If a given user were given a different number of free articles, what is the probability that they would subscribe or read a certain number of articles? This is a complicated question, he explained, because in reality they can only observe one of these outcomes.
“If we give someone 100 free articles, we have to guess what would have happened if they had received 50 articles,” he said. “These kinds of questions fall into the realm of causal AI.”
supekar’s blog post explained that it is clear how the causal machine learning model works by conducting a randomized control study, where certain groups of people are given different numbers of free articles and the model can learn based on this data. As the meter limit for registered users increases, the engagement measured by the average number of page views increases. But it also leads to a reduction in subscription conversions as fewer users hit the paywall. The dynamic meter needs to optimize and balance a trade-off between conversion engagement.
“For a specific user who received 100 free articles, we can determine what would have happened if they had received 50 articles, because we can compare them with other registered users who received 50 articles,” Supekar says. This is an example of why causal AI has become popular because “There are many business decisions, which in our case have a lot of impact on revenue, where we would like to understand the relationship between what happened and what would have happened, he explained. “That’s where causal AI really picked up steam.”
Machine learning requires understanding and ethics
Wiggins added that with so many organizations bringing AI into their business for automated decision-making, they really want to understand what’s going to happen.
“It’s different from machine learning at the service of insights, where you do a classification problem once and maybe study that as a model, but you don’t actually put the ML into production to make decisions for you,” he said. Instead, for a company that wants AI to make real decisions, they want to understand what’s going on. “You don’t want it to be a black box model,” he noted.
Supekar added that his team is aware of algorithmic ethics when it comes to the Dynamic Meter model. “Our exclusive first-party data is only about people’s engagement with the Times content, and we don’t include demographic or psychographics,” he said.
The Future of the New York Times Paywall
As for the future of the New York TimesPaywall, Supekar said he’s excited about exploring the science on the negative aspects of introducing paywalls into the media business.
“We know that if you show paywalls we get a lot of subscribers, but we’re also interested in how a paywall affects the habits of some readers and how likely they are to return in the future, even months or years later. ” he said. “We want to maintain a healthy audience so they can potentially become subscribers, but also serve our product mission to increase readership.”
The subscription business model has inherent challenges like these, Wiggins added.
“You don’t have those challenges if your business model is about clicking,” he said. “We think about how our design choices now affect whether someone will remain a subscriber in three months or three years. It is a complex science.”
The mission of VentureBeat is a digital city square for tech decision makers to gain knowledge about transformative business technology and transactions. Discover our briefings.