Today we launched our most important and ambitious project: privacyscore.com. In it we are applying analytic methods to measure privacy risk across more than a thousand websites. We’re doing it to help solve a big problem for web users: how to understand the privacy risks they take every day online. Through privacyscores we’re also giving publishers and advertising companies important insights (and incentives) that they need to manage their own privacy practices, which in turn will benefit the people they serve. More than anything we’ve done before, privacyscore embodies the PrivacyChoice mission to “make privacy easier.”
What is a privacyscore?
A privacyscore for a website is an estimate, on a scale of 0 (worst) to 100 (best), of the privacy risk associated with giving the site your personal and behavioral information.
“Privacy risk” means the chance that your data may be used in ways you don’t expect. A privacyscore measures this risk through nine different factors, each weighted based on importance. The factors include how the site promises to protect personal user data, as well as the privacy qualifications of tracking companies who collect data, weighted based on their share of the tracking events on the site. (The factors and default weights are explained in detail on each privacyscore report and in the privacyscore FAQ.) At launch, the average privacyscore across all sites monitored was 71 of 100 possible points; within site categories, average privacyscores range from 65 for shopping sites to 80 for travel sites. We’re tracking these numbers on a daily basis; if we succeed, they will move up and to the right.
What does it mean for web users?
It’s obvious that long-form privacy policies have largely failed in informing web users about privacy risk. Convinced that there has to be a better way, we’ve experimented with icons and we’ve tested short summaries, each of which add value in their own ways. But again and again users have told us: There’s nothing simpler than a number. When you think of privacy risk in terms of numbers, you can know instantly if there’s cause for concern. With numbers, it’s much easier to compare privacy risks between different sites. When you use privacyscore through a browser add-on, privacy awareness becomes part of your regular web routine. You can identify risks before you provide personal data. You can also know at a glance whether a site is being responsible in how it shares data with advertisers. Privacyscore becomes your online privacy guide.
With privacyscores, web users also have the opportunity to contribute. The heart of our system is the mapping we maintain between web sites and the ecosystem of ad and data companies collecting cross-site data. When add-on users check the box to share tracking activity with our system, they extend the breadth and depth of our coverage. Soon we will also open up the tools we have built for the rapid review and classification of website privacy policies; privacy fans (and websites themselves, we expect) will help us scale privacyscore coverage to tens of thousands of sites.
What does it mean for websites and advertisers?
The launch of privacyscores marks the beginning of a new discipline we call “Privacy Analytics,” applying quantitative techniques to the measurement and management of privacy risk. For publishers and advertisers, this means for the first time they have metrics to guide the decisions they make affecting user privacy. We’re inviting publishers to join our Privacy Analytics beta to have access to a set of more advanced measurements and tools to manage privacy risk on their own websites.
Based on early feedback, web publishers are ready. In previewing privacyscores with websites and tracking companies, almost immediately they ask, “How can I make my privacyscore higher?” and “What are the privacyscores of my competitors?” You can’t manage what you can’t measure. With privacyscores web sites and ad companies have the metrics they need to manage privacy risk on behalf of their users.
Most publishers have been surprised by our estimate of privacy risk on their sites. Few sites monitor which third parties are collecting data, and even fewer focus on the privacy qualifications of those companies. The privacyscore report brings these facts into clear focus, but with more than just a list of trackers seen. By weighting the privacyscore based on prevalence, and accounting for differences in privacy qualifications among companies, publishers get a representative assessment of privacy risk and clear priorities for improvement.
What are the limitations of privacyscores?
A privacyscore is by no means a perfect measurement of privacy risk. Users have varied expectations about privacy. While our factors capture well-known risk factors, not everyone will agree with our picks, nor with our default weightings. We can’t account for the risk that actual data practices may not comply with stated privacy policies. Our framework also doesn’t reflect the risks in consensual information sharing (like in social networks) where notices may not always be complete nor well understood. Nor does our framework as yet comprehend the unique ways in which different types of tracking companies use and share data amongst each other. These shortcomings are part of the work ahead of us.
Where do we go from here?
We’re actively gathering feedback from our users. Our first deliverable based on that feedback is to activate privacyscore customization, so users can provide their own weights to apply throughout the experience. Privacyscores will become part of our API set, and will be used by other companies to power their own privacy applications. The algorithm also will evolve to include additional factors, which are likely to differ based on the nature of the site. The number of websites with privacyscores will grow through crowdsourcing. We will develop privacyscoring for mobile websites and apps, and they will reflect uniquely mobile privacy risks. We are already discussing with other researchers how to use privacyscore data for analysis that can be useful to policymakers and self-regulatory efforts.
We expect to continue to spend a lot of time responding to input from websites, ad companies and industry organizations, which will no doubt include some concerns. In this process we pledge to be as open and objective as we can be, and to quickly correct any mistakes or misconceptions (which are inevitable). We’re confident that this dialogue can only make the service better, and that there’s plenty of common ground when we stay focused on doing right by web users.
A note of thanks
This project could not have been realized without the great work and insights of over a dozen coders, designers and analysts, as well as scores of testers and supporters (at Nextspace and beyond). We’re inspired by these people and deeply grateful for their contributions.