By Chris Peterson and Jason Gershowitz
We’re here at the #ODR2014 conference. ODR2014 is the annual meeting of the Online Dispute Resolution Forum, an international assembly of lawyers, mediators, technologists, and others who care about technology and dispute resolution.
The livestream is here and embedded below:
First up is Colin Rule, a fellow at NCTDR, former head of dispute resolution at eBay, founder/COO of Modria, and the primary organizer of this conference. He’ll be talking about best practices in for resolving customer issues through the web, primarily through the lens of his company Modria (slides).
Fundamentally, today is for thinking about customers. Colin asks how when we (meaning dispute resolution experts) can best help customers when they encounter difficulties, and best provide a dispute resolution experience that matters for that person. As disputes are resolved, trust increases and so does customer loyalty. Persistent data show that “the problem resolution experience is a key driver of customer loyalty,” Colin says.
Colin shares some statistics from a study conducted by buyers and sellers on eBay and PayPal. Looked at three months before and after a test month for both cohorts and divided into two cohorts: one who had a dispute during that month, and one group which did not.
If you just look at satisfaction, it’s highly correlated with outcome (did you get the product?). The counterintuitive finding, however, was that the continued activity rate for buyers who filed a dispute was higher than those who did not. On average, the buyer who amicably resolved a dispute with a seller was significantly higher than those who had no dispute at all (by about 9%)! This held true even in cases where the buyer was found at fault and didn’t get the product or the refund. The difference was such that it meant hundreds of millions of revenue for eBay and Paypal.
The only group which reactivated significantly less were buyers where the resolution of the dispute took a long time, whether or not they won. Buyers would rather lose a dispute quickly than win it after a really long time. If a company can show the customer that the business / organization can be trusted and move quickly, the relationship is enhanced and customer loyalty increases. You can achieve more with an effective customer dispute resolution experience than you can through other vehicles for customer engagement.
Modria, which was spun out from the eBay / PayPal code that Colin helped design, is meant to help make these kinds of amicable dispute resolution processes more accessible to businesses, governments, and NGOs.
Colin then introduces Amanda Long, Director General of Consumers International, who will be talking about “Increasing Trust through Resolving Issues.” (slides)
Amanda jokes that she feels at home in SF because she was always a disruptive child. She joined Consumers International six months ago with the aim to help businesses build trust and transform what a business can do by changing how you engage with the consumer. She finds Amanda finds this exciting: Amanda loved history as a child, and always got ticked off when things were done that she couldn’t be the first to do. She sees the collaborative economy as a ‘wild opportunity’ to improve business-consumer relations.
The focus of Amanda’s session is on consumer protection in the collaborative economy. Consumer protection will help, in itself, ensure the future of this innovation – providing space for it to continue to flourish and move forward.
Amanda has come here from London, where she recently experienced the #Uberwars where cabbies struck by driving slow to protest Uber. This is a dilemma which has played out across the world (including here in SF), and regulators are trying to figure out whether to side with incumbents or innovators, etc. From a consumer protection perspective, Amanda believes that the binary is the problem itself. The question is not whether incumbents or innovators, but to put the consumer first, and make sure they win every time. You need to enable innovation while also ensuring that consumer protections built for and around the incumbents persist.
It’s a straightforward solution, but hard to implement, Amanda admits. However, she believes there is a common cause among the people present in the audience here today, and the promise is to make sure that the folks here can engage with each other to maintain or improve these practices.
Consumers International’s oldest and most famous product is Consumer Reports. She quotes President Kennedy, who said in 1962 that “the march of technology…has increased the difficulties of the consumer along with his opportunities; and it has outmoded many of the old laws and regulations and made new legislation necessary.” So this problem isn’t new, and in fact Consumers International was born as a result of a Congressional initiative inaugurated by this speech. Interestingly, 1962 is also the year MIT computer scientist J.C.R Licklider was appointed head of of the Information Processing Techniques Office at DARPA, a position from which he would spearhead a number of transformational networked technologies.
Amanda reminds us that Internet access isn’t widespread. Example of Nigerian conference on consumer protection which struggled with its own technical infrastructure (CP: as many American academic conferences still do too!) Where access is widespread, however, the internet can be transformational. The channels, the timing, consumer interaction, buying power, all changes the meaning of what a consumer can even be. This creates an opportunity for the organizations that are providing the services to offload what Company A was responsible for to the provider / producer. The consumer is now engaging with these transactions.
Amanda pulls up a slide from the New York Times in April 2014 with the hed “Then, a few years ago, Silicon Valley had a collective insight: Better to ask forgiveness than permission.” Amanda takes issue with this characterisation, however there is a risk that that perception takes root – and that is a real problem. This is amplified by the siren voice of investor interest.
There are a growing number of standoffs around the globe (like #Uberwars) The tension is between the collaborative economy companies who are trying to get their products out there and the regulators and authorities who are under pressure to control the services, which may be portrayed negatively (unsafe, exploitative, tax dodging, etc). This is not just true in the states or UK, but has happened in Barcelona, Brussels, and many other countries where they are emerging. The fraught political relationship means that sometimes the same politicians wooing companies for economic reasons might bash them for regulatory reasons the next day.
Amanda says her biggest worry are kneejerk, reflexive decisions being made either way. There’s an opportunity to think more strategically to make sure that innovation can flourish while consumers are protected, but the problem is the pressure under the high rate of change characteristic of contemporary tech sector. If we get it wrong we’ll strangle innovation or endanger consumers. Particularly with the collaborative economy – a fast moving, amorphous subset of the sector where this is currently felt most keenly.
Reputation vehicles – like reviews, transparency mechanisms, and services like Modria – allow consumers to avoid bad experiences while also providing a platform to remove providers who are not working out. There are also intermediary organizations thinking about how they can bring consumers together to increase purchasing power for say energy or water prices. These platforms create and police a virtual marketplace. If the consumer is put first, and you actually pay attention to a quick response and focusing on a consumer need when it arises, potentially as frontline regulators those platforms can be very effective in providing consumer protection.
The way forward: Let’s collaborate! Amanda noted in her childhood she wondered what her opportunity would be to shape and change the world we live in. She emphasized that there is a real potential for this to be ground to a halt. As consumers, as advocates, there is huge opportunity for consumers in what is being offered here. The opportunity for empowerment is only just becoming clear, and the potential is very exciting. Since 1962 Consumer International has fought hard, for legislation to protect consumers, but the collaborative economy is a new world and new tools are need. For this to truly be a collaborative economy, we need to think outside of our comfort zone while remaining true to the customer. The consumer movement and ODR have much in common – pioneers of radical transparency, believers in the power of networks, fueled by consumer interest dreaming of perpetual advancement of consumer empowerment. Amanda concluded that we should disrupt together, to ensure the future of innovation for consumers, businesses, and online dispute resolution.
Next up is Brian Burke, Director of Global Feedback/Reputation at eBay, a longtime colleague of Colin’s (slides). Brian jokes that he’s been at eBay for 15 years which just makes him feel old. He’s worked in several sectors across eBay but now oversees all of of its subsidiaries, with a recent focus on seller trust.
Brian poses a fundamental question: why leave a review or a rating for a company? He mentions a fishing trip he took with his son. After some uncertainty, they had a great experience, which created a noneconomic incentive for Brian to leave a positive Yelp review so that others could have a similarly positive experience.
Brian articulates five elements of a reputational system: risk assessment, incentives, seller risk, rewards, and loyalty. These each flow to each other and together help shape the entire user experience. The system needs to create reputation signals, which buyers want from sellers, and which eBay wants from buyers. So eBay collects and amplifies from both by creating the system and mediating both ends of the transaction.
Context matters when considering reputation. A customer who wants a great attorney vs a customer who wants a great vacation host may need different types of information about different types of skills, and whether or not they are a great attorney may or may not matter if what they’re selling is a used laptop, not legal services. Brian argues that the higher the risk the greater the need for a reputational system. Buying an $8 iPhone case is low risk; renting a luxury yacht for a week is high risk.
Brian reports eBay has so much information on all participants in its system that the challenge is figuring how to make it intelligible to its users. What factors do you weight by how much, what is most helpful to the users (on both sides of the transaction), but eBay has found that recency really seems to matter to build trust and improve self-reported satisfaction.
It is not without its challenges, however. From 2004-2008, 8x more sellers left ‘retaliatory’ feedback for buyers who left negative reviews, which eBay felt damaged the trust users had in its reputation system. So eBay dropped seller’s ability to leave negative feedback for buyers. As a result, sellers began adapting their behavior, improving their experience so that buyers (who were now asymmetrically empowered) would be more likely to be satisfied. The tradeoff is that the signal isn’t as clean for future buyers and sellers.
Brian references The Sound of Silence in Online Feedback: Estimating Trading Risks in the Presence of Reporting Bias, a paper which investigates the ‘silent majority’ of users who neither leave nor receive feedback. Makes it hard to be certain you fully understand your marketplace because you know there are things you don’t know.
Are there other ways to figure this out? Brian brings up eBay’s efforts in the ‘Big Data’ space. Do you even need to ask your users, or can you infer, from certain signals, what the user’s experience was like, and optimize the system (for eBay’s revenues)? While some of these methods may add insight, Brian argues that reputation does more than just provide feedback as such: it also conveys quality, sets expectations, and helps build trust necessary for users to want to use your platform.
So how does dispute resolution interact with reputation? Because reputation matters (sellers with higher reputation not only sell more often, but have higher starting bids for equivalent items), it becomes something to be disputed, and these disputes need to be solved accurately, fairly, and quickly. Self-help resolution between buyers and sellers can reduce cost of managing the system because adjudication by customer support is comparatively slow and expensive. Brian concludes by summarizing eBay’s lessons learned about why reputation is important and how to manage the disputes about it which will inevitably arise.