The films we watch, the products we buy and the subjects that interest us can tell others a lot about who we are — information that, rightly, we might wish to keep to ourselves. The internet, to which we entrust so much of this information, works because of our faith in the confidentiality of the behaviours we exhibit and in a shared sense of responsibility. Work from three teams, recognised today in the Microsoft-sponsored Privacy Enhancing Technology Awards, suggests that more can be done to ensure that people can be confident that their privacy will be protected online.

Arvind Narayanan and Vitaly Shmatikov, researchers at the University of Texas, began looking into large, publicly available data sets that were cleansed of names or other personally identifiable information. They very quickly discovered a major privacy risk — anonymised data sets could be used to re-identify individuals using efficient algorithms. They took the theory and tested it in reality, examining anonymised movie ratings and dates of rating for 500,000 users published by a major online movie rental service. Narayanan and Shmatikov found that they could identify a user's ratings record with only five to ten educated guesses relating to some of those ratings. Narayanan and Shmatikov proceeded to develop a theory that shows how this applies to other data sets such as an online store's purchasing records.

Their work, which earned them the overall 2008 PET award, shows the danger in releasing apparently anonymised data without better methods to ensure that it can't be compromised. Their research will be invaluable in promoting and informing the development of ways to release data with provable privacy guarantees.

Chair of the 2008 judging committee Dr. Matthew Wright, assistant professor of computer science and engineering at the University of Texas at Arlington, recognised the team's contribution to improved privacy. “This year's PET award finalists are representatives of the breadth of the privacy technology field. The winners have discovered a technique that would identify an internet user's anonymised data, such as preferences and purchasing records, having the potential to create a critical privacy breach. This type of research will help organisations understand that, to protect their users' privacy, it is not sufficient to simply remove all obvious identifiers.”

The Privacy Enhancing Technologies Symposium (PETS) was launched seven years ago as a specialised conference focused on anonymous communication, but has extended its scope to address a wide range of challenging technical problems that need to be overcome to define and protect individuals' privacy. The PET Award is given to researchers who have made an exceptional contribution to the theory, design, implementation or deployment of privacy-enhancing technology. The winners are judged by a committee of leading privacy researchers from around the world. Microsoft has funded the annual €3,000 prize since the award's inception in 2003, but leaves decision-making in the hands of the symposium's prominent academic contributors.

“Microsoft believes it is necessary to bring together academic researchers and the privacy community to find original and innovative ways to protect people's personal information online,” said Caspar Bowden, chief privacy advisor for Microsoft Europe, Middle East and Africa. “Our sponsorship of the PET Award and Symposium is intended to promote excellence in this area of growing importance.”

Runners-up Steven J. Murdoch and Piotr Zieliński of Cambridge University also uncovered possible dangers to our online anonymity. Their paper discusses and analyses, for the first time, the possibility of surveillance at internet exchanges (IXes). High volumes of traffic pass through these exchanges when travelling from one network to another, making them an ideal point from which to gather surveillance data. Murdoch and Zieliński first showed that a single IX could observe a large fraction of traffic on the experimental Tor network, a distributed network of relays that bounces traffic around the internet to facilitate anonymous access to information. Despite the fact that the amount of data was overwhelming, Murdoch and Zielinski's study looked into how much they could learn about users from only a snapshot of the surveillance data gathered.

Using techniques that are realistic with today's network technology, they showed that this method of looking at a small sample of data was surprisingly effective and could uncover a lot of information about Tor users. This research is notable because it could change the way researchers think of the security of network privacy systems, and is likely to be influential in future research about internet surveillance.

Fellow finalists from the Department of Computer Science at Brown University in the US were recognised for a different privacy aspect — maintaining anonymity while ensuring accountability. Mira Belenkiy, Melissa Chase, C. Chris Erway, John Jannotti, Alptekin Küpçü, Anna Lysyanskaya and Erich Rachlin focused on peer-to-peer (P2P) systems, such as the popular BitTorrent file-sharing service. These types of services rely on every user's computer providing service as well as receiving services from others. The decentralised nature of peer-to-peer systems makes it difficult to ensure that all computers are providing their fair share of service.

Although privacy and accountability may seem contradictory, the team has found a way to enable this by using advanced cryptography techniques, which they also developed. The proposed approach is analogous to using electronic cash (e-cash) for payment for online services. E-cash systems are designed to be anonymous like real cash, but can also be engineered with mechanisms to inhibit cheating. Here, the currency of e-cash is not monetary value, but levels of trust within a P2P network. The authors explored the economic issues that would be involved in a real deployment in a system like BitTorrent, and evaluated the performance of the system in both simulation and testing of a prototype implementation. The team used computer equipment that is considered basic by today's standards, and concludes that provably secure, anonymous and scalable P2P systems are within reach.

The awards ceremony is part of the annual PET Symposium where prominent academics from around the world gather to discuss recent advances and new perspectives in information privacy on the internet, in data-sharing, and in other technologies such as location tracking of mobile devices. Microsoft supports the PET Symposium by providing a fund for travel stipends to graduate students in the field who want to attend but lack the financial means. The recipients of the stipends are decided independently by the PET organisation committee.

Dr Wright added, “The PET symposium brings together the people working to make IT compatible with our basic value of privacy. As more aspects of our daily lives are conducted on the internet, from banking to dating to politics, it's more important than ever to develop and evaluate new approaches for protecting privacy.”