Transcript:
Why am I here
I teach at the Vrije Universiteit of Amsterdam on privacy and data protection. I like to explain to my students the importance of human rights by reading a part of the preamble of the UDHR, about the importance of human rights and why we should protect them through the rule of law in order to stop men from being compelled to have recourse to rebellion against tyranny and oppression.
After I read this to them, some start yawning involuntarily. Occasionally, one of them might even fall asleep. And the older I get, the more it bothers me that we think that our human rights are self-evident. Once I gave a lecture to more than a hundred students and when I asked them if they ever felt government treated them in an arbitrary manner, only one person raised his hand and then started talking about a parking ticket he once received. We take human rights for granted because we do not feel how it is not to have them. Enjoying these rights is like enjoying your limbs, it feels natural, but not special in any way. But it might be like this Dutch love song about a man who doesn’t know what he misses, but only when his lover is not there, only then he knows what he truly misses.
And just like you should not take your lover for granted, so too taking our rights for granted might ultimately lead to a break-up. Having rights in treaties, charters, and constitutions is important, but if they are not invoked by us, the people they ought to protect, they become a dead letter. Human rights can be viewed as codified norms and usually when norms are visibly violated this leads to a backlash from society. When the civil rights marches organized by dr. Martin Luther King were met with extreme violence by the US police forces, press was there to film it. When it reached the news the public outrage helped the civil rights movement to gain momentum and push for changes of the law. There is, however, one very important element in this story: the injustice was visible.
A warning from the past
These marches took place in the fifties and sixties when computers weighing over 900 kg and costing 500 000 dollars were considered ‘small and affordable’. Only two decades later microprocessor computers were introduced. A US Committee on Automated Personal Data Systems issued a report in 1973 in which it expressed concerns over the growing use of computers. And whilst they recognized their advantages for bureaucracy, they also foresaw that their introduction would cause a major shift in the power relation between citizens and the state.
They warned that record-keeping practices could lead to ‘a national dossier system to maintain information on every citizen from cradle to grave’ basically making everybody a prisoner of her or his recorded past.’ They adopted a quote of the Russian noble prize winner of literature, Alexander Solzhenitsyn, which illustrated their fear. Solzhenitsyn wrote that government personal data records results in hundreds of invisible threads radiating from every person and if these would materialize society would come to a grinding halt. Even though they are not material, each person aware of his threads would develop a respect for the people who manipulate them.
This quote is actually out of place. Solzhenitsyn lived in Stalinist Russia under the constant threat of the secret police keeping records. He clearly associated record keeping with an oppressive state. We in the West do not see our states as oppressive. We still trust the state. In Western democracies, data-driven record-keeping practices are mainstream, the critical debates take place in the fringes of the media, but the public at large doesn’t really seem to mind. Maybe in our subconscious, some fears might slumber, but the current tone of the debate is data is good, it brings comfort, security and efficiency. We truly want to believe we have nothing to fear, even resorting to clichés like: ‘I have nothing to hide, so why should I have anything to fear.’
What is currently happening
Let me pose a counterquestion: how do you know you have nothing to fear if you don’t even see what is taking place? Saying you have nothing to fear is just a rhetoric trick to ease your mind in the face of ensuing uncertainty.
The state increasingly resorts to data-driven practices on the basis of risk profiling, secret black lists, public-private fishing expeditions, and so on. Large quantities of data are processed to identify the risk a person poses for violating the law. The logic of the state of traditional law enforcement is turned upside down: instead of the state only using its powers against those who are suspects, the state creates its suspects on an automated bases. This basically boils down to mass surveillance against certain groups, neighbourhoods, cities, or even an entire country.
In the Netherlands, we employed a system named SyRI. No, I don’t mean the assistant of your smartphone started snitching on you to the police. SyRI was an acronym for System Risk Indication. This was a system adopted in a law in 2014 and allowed public authorities to make risk profiles of all people living in a certain neighbourhood, by bringing their data together and analysing the data through the use of an algorithm. The idea behind this was to identify people who were at a high risk of performing ‘fraud’. SyRI was operating without the awareness of the people who were subjected to this mass surveillance. I also work for a NGO which was part of a coalition that challenged this system in court, successfully. The court came to the conclusion that people need to be aware of these risk profiling practices and the risk models used to classify them, so they can challenge this process and the decisions taken on the basis of it. Although the state chose not to appeal this decision of the court, a day after the appeals deadline the Dutch authorities proposed a law which goes much further. In the new proposal, there is no transparency towards citizens and private-public data processing partnerships can make blacklists in which incriminating dossiers of citizens are euphemistically dubbed ‘information products’. The Platform for Civil Rights started calling this ‘Super SyRI’.
So even when a court draws a line, this doesn’t stop the government from bulldozing over it. The law is increasingly used to facilitate the data-driven state and weaken the protection of human rights standards. Legislators are working hard to change the laws in order to facilitate the gradual expansion of informational power. In the process, they’re forgetting the basic principles that we build our societies on. Democracy, as Karl Popper said, is about limiting power. The shift to data-driven governance is one in which the power of the state is continuously and gradually increased. THE point of departure of constitutional democracy is institutionalised distrust towards the state. The state should be transparent and accountable to its citizens. Citizens ought to have a private sphere that the state should respect. The shift to data-driven governance has resulted in institutionalized distrust of citizens instead. Citizens are made increasingly transparent by an invisible data processing machinery of the state. We cannot challenge what we cannot observe.
The Netherlands were rocked by the child benefits affair. Here child care is very expensive. Even when you have a well-paid job you still receive money from the Tax Service to pay for child care. Receiving this money makes you vulnerable in relation to the state. At some point the Tax Service started demanding benefits back from families, claiming they were not entitled to them. These were large amounts of money that could and did bankrupt families. Thousands of these families didn’t do anything wrong. This leaves the impression that the decisions on their case were not based on finding any actual wrongdoing, but instead them fitting a certain profile. In a recent interview, one of the victims said that she could never imagine this happening in the Netherlands. But she also issued a warning to the viewer when she said: if this can happen to me, it can also happen to you.
And this is not all. People who had debts to the Tax Service above a certain threshold were automatically placed on a blacklist and got the label ‘fraudster’. They were never informed about this. Some third parties, like municipalities, were informed about parents getting this label. Because parents were considered fraudsters they were refused a favourable arrangement for paying their debts. This meant they didn’t have enough money to provide for the minimum necessities to maintain their family. Cases are now in the media of people whose children have been taken away by child protection services because they allegedly couldn’t take care of them anymore. All the while the parents did not have a clue why this was happening to them. This is not about privacy. It is about the fundamental safety of individuals and families. Through data processing and analysis people can be investigated, found guilty, and sanctioned, without them ever being told about it. Like they are convicted by a secret court.
If we do not know about the systems employed it is also impossible to challenge their legality. This will become even more problematic in the future now that the EU is working on a legislative proposal for artificial intelligence in which predictive policing, a practice which data scientists refer to as ‘AI snake oil’, is introduced as a legitimate practice. Basically, we’re setting up a legal framework in which the child benefits affair will just be considered a failed pilot.
What can we do
After depressing you for ten minutes straight, now you probably expect me to present the silver bullet. Sorry to disappoint you. Raising awareness is the first step towards structurally engaging with this problem of invisible injustice. We need to realise that it is our civil duty to distrust the state. The people affected by the benefits affair all shared the feeling of shame, being seen by their friends and family as guilty and some of them ending-up in social isolation. We need to open up to the perspective that the data-driven state makes innocent victims and be open to help the people who get in trouble. Help them to get through their predicament and enable them to exercise their rights. We as lawyers have a responsibility to engage with the law in a way that is inclusive and in which everybody enjoys legal protection against the state. We as lawyers should not use our knowledge and skills to profit from laws or interpretations thereof which are only instrumental to the power of the state. We need to expose the illegal practices of the state taking place outside of our view, blow the whistle on them, take them to court and hold the state accountable. Everybody can help by taking an interest and supporting the work of NGOs which expose some of the state practices. Finally, we should all remember that ultimately this is not about human rights, but about the human lives they ought to protect.
Tijmen Wisman
