The tyranny of the algorithm yet again...

I was reminded yet again, over the weekend, how easy it is to get branded persona non grata in the age of the judgmental algorithm.

A search for "Isis Close" via any popular search engine throws up a significant collection of such streets, primarily in the Thames Valley region, in places like Long Hanborough, Aylesbury, Oxford, Abingdon and Putney, amongst others.

That these place names exist won't be a surprise to anyone familiar with English limnology - the study of rivers and inland waters. As Wikipedia helpfully tells us, "The Isis is the name given to the part of the River Thames above Iffley Lock which flows through the university city of Oxford". In at least one local primary school I'm familiar with, the classes are called Windrush, Cherwell, Isis and Thames.

Unfortunately for those who live in an Isis Close, Street, Road or other equivalent, the label Isis has been appropriated by or conferred upon (by the media) a bunch of murderous extremists in the Middle East. So the word "Isis" has become somewhat toxic in the West.

Now PayPal has decided that they are not prepared to facilitate payments for goods to be delivered to an address which includes the word "Isis".

An Isis street resident ran into some unexpected difficulties when attempting to purchase a small quantity of haberdashery on the internet with the aid of a PayPal account. The transaction would not process. In puzzlement she eventually got irritated enough to brave the 24/7 customer support telephone tag labyrinth. The short version of the response from the eventual real person she managed to get through to was that PayPal have blacklisted addresses which include the name "Isis". They will not process payments for goods to be delivered to an Isis related address, whatever state of privileged respectability the residents of such properties may have earned or inherited in their lifetimes to this point.

Who knows if the "avoid Isis" algorithm was added by a low level techie, a policy decision within PayPal driven by risk averse lawyers or some other process. Whatever the process, the result is that people with an Isis address have been tagged with a "do not touch" label on the internet.

We rarely understand that the poor, the mentally or physically disabled, ethnic and religious minorities, and every other collective marginalised group that privileged society discriminates against already live in a dystopian world. A world full of legal, environmental, economic and societal strictures and attitudes that make life difficult. We rarely understand, either, how easy it is, in a world full of big data, wielded by commerce and governments alike, to be cast out from our world of privilege to that of the marginalised.

So what are residents of "Isis" addresses to do. Interestingly enough, PayPal are not so wedded to their dissociation from "Isis" that they are banning this particular brand of humanity from holding PayPal accounts. The marginalised are still potentially profitable fodder. The organisation is merely concerned with not facilitating transactions which result in goods or services being delivered to those addresses. So Isis addressees could

• have their orders delivered to an alternative address for collection
• apply to the local council to have the name of their street changed
• move house

The first is a serious inconvenience. The latter two options rather extreme and even then, given the tendency of unwelcome labels to hang around people on the internet and in big databases, there's no guarantee that a change of street name or address, drastic as they are, would entirely rid the people concerned of the tyranny of the algorithm.

Being unable to buy a sewing kit on the internet might seem like a minor middle class inconvenience but metadata, like details of someone's address, matters. It's only two years since the former US National Security Agency and CIA chief, General Michael Hayden, went on record with his "We kill people based on metadata" comment. Metadata is used to categorise and discriminate. We have no control over what commercial organisations or government or other economic actors and indeed criminals do with it.

With the UK Investigatory Powers Bill heading for the statute books, and the government taking little or no notice of the serious criticism it has received, this state of affairs is not primed to improve any time soon. Could I remind the reader of this simple engineer's perspective of the meaning of s78 alone in this large and complex law - it looks a bit like this:

I leave you to decide whether the government and its institutions will be able or willing to use this unimaginably gigantic collection of data in the public interest and without doing too much collateral damage to the sifted and categorised populace along the way.

Update: A stark example of where the tyranny of the algorithm does real damage - software used across the US to predict future criminals and it’s biased against black people.

John Oliver, Privacy International & Ryan Gallagher on mass surveillance

In the light of The Intercept's latest story on the Snowden documents, could I recommend revisiting John Oliver on government surveillance plus his Snowden interview...



 ... and Privacy International's short videos on communications surveillance, big data, data protection, metadata and privacy

VE Day, freedom, justice and technology policy

An edited version of the following piece is available at The Conversation.

On the 70th anniversary of VE Day, it became clear that 36.9% of those who turned out, in the UK general election the previous day, voted for the Conservative party. With the turnout being 66.1%, this means that 24.3% of those eligible to vote gave the Tories an unexpected slim majority of seats in Parliament.

What does this mean for technology policy over the next five years, now that the Tories have been joyfully unleashed from their Liberal Democrat coalition partners? Well the big technology issues will continue to be things like communications infrastructure, surveillance, big data, human rights, encryption, security and intellectual property.

Communications infrastructure

Any progressive modern government indulging in rhetoric about being world leaders in technology needs to invest their money where their mouths are. Forget petty little projects like HS2, forget austerity and pour gigantic pots of the green stuff into building broadband infrastructure to every single corner of this little island. I'm not talking about promising households "up to 2Mbps" or even "up to 20Mbps" but a neutral network that delivers speeds upwards of one terabit per second to every household by 2020.

If we harbour serious ambitions to be technology leaders, then give every member of the population access to technology and robust neutral high speed networks. Then get out of the way and watch the human, constructive, social, technological, economic, commercial magic in that technological playpen.

Surveillance

Sadly, much of what passes for technology policy is likely to continue to be driven by the Home Office. Home Secretary Theresa May's obsession with the thoroughly discredited Communications Data Bill aka Snoopers' charter, is a top priority. Though the BBC reports (from 07:50) that the Snoopers charter is being handed to Mrs May's best mate, Michael Gove, in his new Justice Secretary role.  Whichever of them takes it on (and Home Office & Justice ministry insiders still believe it is the Home Office's responsibility) the government mean to reinforce and expand information systems and laws facilitating mass surveillance.

No amount of explanations that mass surveillance is dangerous and doesn't work ever gets through to the largely technologically illiterate MPs in Parliament - proportionately few have any scientific or technology training or background. Government ministers tend to avoid understanding something, when their jobs, and future tilts at the Tory leadership depend, on them not understanding it.

Effective law enforcement and intelligence work is complex, messy and difficult. Unfortunately Home and Justice Secretaries, under the gaze of the 24/7 news media, don't have the luxury of managing complex, messy and difficult. They need clear, simple and immediate actions and apparent solutions. 

Mrs May is concerned to be perceived as tough on crime and terrorism, repeatedly declaring a determination to give the security services everything they need - a phrase which, unlike the approval rating it gets in the UK, sends a shiver of fear down the spines of most continental Europeans, whose countries have experienced histories of totalitarian governance. Law enforcement and security services need to use modern digital technologies intelligently in their work and through targeted data preservation regimes – not the mass surveillance regime they are currently operating – engage in technological surveillance of individuals about whom they have reasonable cause to harbour suspicion. That is not, however, the same as building and enabling the legal operation of an infrastructure of mass surveillance.

The Intelligence and Security Committee of Parliament recently reported that "bulk collection" of communications data is acceptable because most of the data is only ever "seen" by computers. By that measure Mrs May & Mr Gove should install a top of the range CCTV camera in every room in every building in the country. They'll only use the footage if it becomes necessary, you see.

Yet not once, in the 14 years since the dreadful 9/11 attacks, on either side of the Atlantic, have mass surveillance magic terrorist catching machines ever been publicly,* credibly and specifically shown to identify a terrorist suspect pre-emptively.

Big data

Successive UK governments for a long time have not had a great record at managing large tranches of electronic data. On the promise of major medical breakthroughs both partners in the previous coalition government bought into the notion of big data in healthcare. The Tories are keen on privatising the health service. Making medical data available on a wide scale to medical researchers and industry is considered a no brainer, in the newly populated corridors of power.

The now dismantled Nu Labour National Programme for IT in the NHS was arguably the largest ever government IT disaster. But the coalition were and now the new Conservative government are keen to press forward with privacy-destroying management of NHS information systems and the further centralisation of medical records, in the ill-conceived care.data programme.

Government can't lose by claiming they are spending more on the NHS, even if it is on information systems that won't work; and in practice cause untold havoc in the long term. 

There is a serious question about facilitating constructive, enlightening, ethical, socially useful research into healthcare big data whilst protecting medical confidentiality. But it should not be done by letting government circumvent human rights law guarantees to privacy. It needs to be governed by a set of principles such as those set out in the Nuffield Bioethics Council report on the ethical use of data.

• treat people as individuals worthy of respect and not as industrial raw material - the principle of respect for persons
• abide by all relevant laws including human rights laws - the principle of respect for established human rights
• tell people what you are doing with their data and consult them properly and regularly - the principle of participation of those with morally relevant interests
• account for what you've done with the data and tell people if things go wrong - the principle of accounting for decisions

Human Rights Act
It's a little sad that 70 years on from VE day that 24.3% of eligible voters of the great British public apparently support the abolition of the Human Rights Act. In an information age, information and, by proxy, technology policy becomes human rights policy. The new government want to replace the Human Rights Act with a proper British Bill of Rights for proper, deserving, British people and not be dictated to by those unelected European judges. The Tories' proposals for the Bill of Rights suggest it would include all of the rights currently protected by the European Convention on Human Rights (and, therefore, the Human Rights Act) but judicial protection would be denied to those rights judged to be “trivial”. Just as a matter of interest, which of -

• Obligation to respect Human Rights
• Right to life
• Prohibition of torture
• Prohibition of slavery and forced labour
• Right to liberty and security
• Right to a fair trial
• No punishment without law
• Right to respect for private and family life
• Freedom of thought, conscience and religion
• Freedom of expression
• Freedom of assembly and association
• Right to an effective remedy
• Prohibition of discrimination
• Prohibition of abuse of rights

- would you consider trivial? If you'd like a lesson on what any of these mean in practice may I recommend a couple of minutes at day at the recently launched Rights Info site. Pay particular attention to the human rights myths section - every one of these myths has been trotted out by the mainstream media and governing politicians of varying flavours in the past ten years. Theresa May featured the pet cat story at her Tory conference speech in 2011. Michael Gove has apparently been given charge of the Bill of Rights job.

Plans to ban encryption

Much mockery has been poured by the tech community on David Cameron's plans and the FBI's desire to ban encryption and provide mandatory back doors to government. Basically it won't work as a crime/terrorism prevention measure. And it definitely would not be cost effective. Banning Transport Layer Security (TLS) and its predecessor, Secure Sockets Layer (SSL) would be very costly. It would lead to further mass data silos, useless for preventing nefarious actors engaging in attacks.

Think of it as every householder delivering copies of their door keys to the local police station, just in case the police would like to search any house without breaking the door down. No self-respecting crime syndicate is going to systematically break into multiple houses when they can break into the police station first to get all the keys; or encourage or coerce an insider to provide them with copies of the keys.

The rest

I could go on – there are a host of really important technology related issues to play out in the next 5 years, including: 

• Web blocking & age verification – think of the children and the poor copyright conglomerates
• Counter Terrorism & Security Act 2015 "prevent" duties – about to spawn a veritable bureaucratic industry
• EU data protection measures promised by the end 2015 – that’ll be fun
• Intellectual property – a complex black hole that determines control of information

- but you, dear reader, have already been inordinately patient. 

So I will close with a question.

70 years on from VE day, given the surveillance society we have built since the turn of the century and intend to expand and reinforce over the next 5 years, can we really be seen to be honouring the sacrifice of those who died to win our freedoms and a lasting peace, founded on justice and good will; or, as John Naughton so eloquently puts it, does our  indolence constitute “a shocking case study of what complacent ignorance can do to a democracy”?

* Updated for clarification to add the word "publicly".

Generations

The mindset of FBI director, James Comey, appears to be that of someone who believes the lives of all citizens should be on permanent display for government inspection and approval. Privacy, remember, is only for criminals. Apart from the fact that this won't be the first or last time that Apple or other commercial enterprise overstates the potential efficacy of its product features in any particular context, Mr Comey's belief that he has the right, nay, duty to berate the company for its modest implementation of privacy enhancing technology is not one I can share.

It's not a big logical leap for a society that normalises mass collection, processing, analysis and storage of communications (metadata and content, even if the distinction is no longer clear) to consider that those who would wish to opt out, or those that would help them do so, should be considered anomalous and suspicious.

It's not a big logical leap to say we'll collect all the information but don't worry it's only "seen" by the computer, not real people, so it's not real surveillance; and it will only be looked at for evidence of wrongdoing by bad guys... or those linked to bad guys... or those linked to those linked to bad guys...

It's not a big logical leap to say we're not collecting enough information.

Everything will be easier or better or more efficient if only we collect more.

Big data is the future of commerce and government.

We already have mass telephone and internet interception, let's install CCTV cameras in all homes. Oh yes, people already have webcams attached to their computers and we have already collected millions of users' Yahoo webcam images. But that's only in the rooms with computers and the cameras don't cover all corners of the rooms. And, after all, the footage will only be collected and "seen" by computers, not real people, so it's not real surveillance.

And we don't have to record the audio. At least at first. So we don't know what people are actually saying to each other. So it's only metadata. We know where they are and who they are talking to and for how long but not what they are actually saying to each other.  If we surreptitiously use lip reading programs that's only to detect serious criminality.  Ah, you know what, we have to protect people so it doesn't matter if we record the audio too. It will only be 'seen' by computers.

And if you have nothing to hide you have nothing to worry about.

It's not a big logical leap to say, you know all that information in all those giant databases? We should be using it not just to catch terrorists but to catch

• serious criminals
• drug dealers
• child abusers

It's not a big logical leap to say we shouldn't just limit it to these serious criminals. We need to apply use of this information to illegal immigrants, benefit scroungers, criminal youths, burglars, petty criminals, drunkards, louts, vandals, offensive people, inner city sink estates with unruly families who may cause trouble, ethnic minority communities which may have a link to a religion extreme forms of which may be cited as an excuse for murder and mayhem, Romanians (if UKIP ever get a say), the poor, disabled or sick or elderly...

It's not a big logical leap to say we need to use this information to improve public services -

• the NHS
• education
• social welfare
• policing and criminal justice
• intelligence
• defence
• foreign affairs
• economy

So people with a HSCIC database determined genetic predisposition towards contracting certain kinds of cancers may be required to take out private health insurance just in case they become a disproportionate drain on the NHS. Or kids with mental health or special educational or social needs be excluded from good schools so they don't disrupt the normal kids.

Even the most dedicated of public servants, and I have the privilege to know a lot of them, heavily under-resourced and under relentless pressure for outcomes, from management or politicians of varying levels and competence, succumb, with the best of intentions, to the temptation of mission or function creep.

It not a big logical leap to convince ourselves it is ok to use data or facilities gathered or created for one purpose for an additional 'useful' or convenient or target facilitating purpose.

Dedicated people in all walks of life bend/break/ignore/re-interpret/renew the rules because everyone else is doing it, so it's not a big deal.

But this combination of the normalisation of mass surveillance, function creep and (sometimes) well intentioned rule making and rule breaking create the conditions for the poisonous snakes in suits of the world to thrive. And it only takes a handful of them in the wrong places to cause widespread misery and abuse of human rights.

The sad thing is we can and sometimes do design, build, operate and use communications systems, computers and big data in socially, economically and legally enlightened ways, in the public good, when we do so intelligently and ethically. But we're often failing to do so. The seductive attractions of convenience, instant gratification and the ease and power with which all personal data can be collected, processed and stored (the economic agents doing the collection can figure out how to exploit/use/monetize it later), beat intelligence, ethics and the public good, every time.

I was chatting with my elder teen late one evening this week about some of this. I expressed a concern that he and his kids would be asking me, in my dotage, what the hell I thought I was doing when we were building the communications infrastructure of a surveillance state. After all, all that is required for evil to prosper is for good people to do nothing.

"Dad," says he, "I can guarantee that's one question I'll never have to ask you."

I'm wasn't sure whether to be flattered that he over-estimates the efficacy of my efforts in this landscape, sad that he may change his mind, concerned that I've burdened him with my worries or optimistic that he and his generation are smarter than me and mine; and they will put those smarts to good effect in building a more equitable and enlightened world.