Thursday 28-09-2017 23:07

I read with interest your explanation of your new project on measuring emotional experiences. It is exciting to be part of the birth of a new technology, and the wonder of innovation is clear in your AURA project which will translate sensed emotions into light. I think this will provide new opportunities to investigate processes of human emotion, especially for the ‘quantified self’ community already engaged in measuring and tracking their own experience of the world.

I question, however whether tracking one’s changing emotional state as one experiences media, or anything in fact, is part of a ‘customer journey’. This is not just about sensing, but about investigating the border between software and wetware – technology that aims to connect to and enhance the human brain.

It is interesting to its corporate sponsors because it promises new forms of access not to ‘the customer’ but to people, in all our idiosyncrasy and physicality. Those forms of access are not necessarily more accurate than asking people what they think, but they will be more seamless and frictionless, blending into our lives and becoming something we are rather than something we do.

You ask whether consumers need to choose between their privacy on the one hand and the comfort of personalized services on the other. I think this question may distract attention from a more central one: can we separate our existence as consumers from our existence as citizens, partners, workers, parents? Our emotions are an essential bridge between ourselves and others, and what we show or hold back determines the kinds of relationships we can form, and who we can be in relation to our social world.

The language of choice may not be the right language here: your project uses only volunteers, but is it clear what they are volunteering? Your technology has a 70-per-cent accuracy, according to test subjects. But there is profound disagreement amongst brain specialists as to what we measure when we study emotions.

William James, one of the founders of psychology, argued that our experience of emotions actually results from their physical expression: we feel sad because we cry and we feel happy because we smile, not the other way around. If this is true, the sensors you are developing will have better access to the biological content of our emotions than we will, which has implications for – among other things – our freedom to form our own identities and to experience ourselves.

I am reminded of a project of Facebook’s that was recently discussed in the media. The company’s lab is attempting to produce a brain-computer speech-to-text interface, which could enable people to post on social media directly from the speech centre of their brains - whatever this means, since there is no scientific consensus that there is such a thing as a "speech centre".

The company’s research director claims this cannot invade people’s privacy because it merely decodes words they have already decided to share by sending them to this posited speech centre. Interestingly, the firm will not confirm that people’s thoughts, once captured, will not be used to create advertising revenue.

You ask what is needed to establish trust in such a system. This is a good question, because if trust is needed the problem is not solved. This is one of a myriad initiatives where people are being asked to trust that commercial actors, if given power over them, will not exploit it for commercial purposes. Yet this is tech and media companies’ only function. If their brief was to nurture our autonomy and personhood, they would be parents, priests or primary school teachers.

The one fundamental rule about new technologies is that they are subject to function creep: they will be used for other purposes than their originators intended or even imagined. A system such as this can measure many protected classes of information, such as children’s response to advertisements, or adults’ sexual arousal during media consumption.

These sources of information are potentially far more marketable than the forms of response the technology is currently being developed to measure. How will the boundary be set and enforced between what may and may not be measured, when a technology like this could potentially be pre-loaded in every entertainment device? Now that entertainment devices include our phones, tablets and laptops, as well as televisions and film screens, how are we to decide when we want to be watched and assessed?

Monitoring technologies produce data, and data’s main characteristic is that it becomes more valuable over time. Its tendency is to replicate, to leak, and to reveal. I am not sure we should trust commercial actors whose actions we cannot verify, because trust without verification is religious faith.

Yours,

Linnet Taylor
TILT (Tilburg Institute for Law, Technology and Society)

Zondag 08-10-2017 21:56

Your response is a useful one. It has made me consider what we mean when we talk about trust, and how the word becomes stretched across very different contexts and processes. You ask, under what conditions would I recommend people provide their data to a system that can sense their response to media content, given the technical guarantee that no company or other party would actually be able to see the data, even if they wanted to.

This is, of course, a difficult question. People should be free to adopt any technology that they find useful, necessary, interesting, stimulating. And it is likely that this sensing system will be judged all of these things. Let us be honest here, though – it is not a citizen collective that has asked us to write this exchange of letters.

We are exchanging thoughts about the future activities of media corporations, at the request of a media corporation. If the technology were going to be used exclusively in a highly bounded context where the data produced could not be shared, sold or reused in any way, I am not sure we would have been asked to have this conversation.

I think the reason we have been asked to exchange ideas is because there are huge implications to a technology that purports to allow the user to view people’s emotional processes. This technology has the potential to help media providers shape content into personal filter bubbles, like our timelines on social media.

These bubbles have their own advantages and problems. There has been much recent analysis, for example, of how the populist parties coming to power around the world have benefited hugely from digital filter bubbles where people access personalised content that aligns strongly with their own views.

It is indeed important that such a system should be used in accordance with the law. But data protection law, in this case, is a necessary but insufficient safeguard against the misuse of data. The real issue here is volume. Most people living today are producing vast quantities of digital data every moment they are engaged with the world.

These data are stored, kept, used, and eventually anonymised – at which point data protection law ceases to apply, because how can privacy relate to anonymised data? Yet the system you are developing demonstrates exactly how. It is another technology of many that will potentially make profiling easier. It will show providers our weak points, the characteristics that make it possible to sell to us – and it can do this even if we do not use it.

An example: someone wishes to live without a filter bubble and does not consent to any personalisation. But all the other data they emit in the course of their everyday life generate a commercial profile of them which is highly detailed and available on the open market. The features which make them sensitive to some types of content and not others are identifiable: they have children, they like strawberries, they are suffering domestic violence, they are made happy by images of cats. A jumble of many thousands of data points like these constitute our digital profiles.

But it is not only our characteristics. It is those of people around us, or like us. Knowledge about the attributes of users of a system such as yours (whose response to content can be directly measured) can be cross-referenced with the attributes of those who do not use it. Once this happens, it becomes possible to infer that my heart will beat harder when I watch one movie than when I watch another; that I will choose to go on watching that provider’s content; that my attention will be available for sale in a particular place at a particular time.

In this way, consent and privacy become meaningless if there are enough data points about us all: new technologies that pinpoint our behaviour, feelings and susceptibilities are valuable not for their immediate uses but as an addition to the long-term stockpile of data on all of us – and especially useful with regard to those who do not choose personalisation and are therefore harder to pinpoint and predict.

This is why I am sceptical about invoking ‘trust’ as something that can be generated by making sure individual applications of a particular technology comply with data protection law. Data protection is a cousin to privacy, but it is not at all the same thing. We may guard data without guarding privacy, and we may certainly trust that our data is being handled compliantly with the law, while also having reservations about the bigger picture.

Things that are perfectly permissible under data protection law, yet are also unfair include charging different people different prices for the same goods online; following users’ activities across devices to understand precisely what makes them respond to advertisements, and a company passing on our personal data to unlimited subsidiary companies. Law is no panacea, nor can it be relied upon to predict what will happen next.

I do not cite these things to argue that you should stop developing affective computing technologies for commercial use. I use them to suggest two fundamental realities: first, that we are no longer capable of understanding the long-term or collective implications of the data we emit, and second, that our consent is not meaningful in that context.

Having made my argument for these two problems, and how they relate to your work, I can pose a question in return: how can we, as developers and users of data analytics technologies, collaborate to look beyond legal compliance to the many possible futures of those technologies, and create ways to shape those futures?

Yours with best regards,

Linnet Taylor