It's time that 'metadata' met an end

It's time that 'metadata' met an end

Summary: Mandatory data retention is back on the political agenda, and Australian law enforcement agencies are presenting a new round of ambit claims. Watch out, meanings are being twisted.


"I think the journalism profession should push back on the use of the term 'metadata' by surveillance agencies. It's data. It's private," tweeted high-profile network engineer Mark Newton last Friday. Those who use the term are maintaining a fiction, he added; namely, the notion that some kinds of data about an individual's communication and online activities are less deserving than others, and don't need to be protected from unwarranted prying by police and spooks by the requirement for, erm, a warrant.

Newton is right. So here's my contribution to that push-back.

The word "metadata" is supposed to refer to any data associated with a communication, other than the "content" of the communication itself. This distinction is intended to parallel the distinction made with telephone calls, where police need a warrant to access the conversation itself through a "lawful intercept" (or, as Americans call it, a "wiretap"), but not to access any information about the call that was recorded by the telco — such as the time the call was made, its duration, and the number called.

That distinction is down an accident of technological history. Listening to a telephone call requires an intrusive act in real time, and it has to be organised in advance or the conversation is lost. The other information was being recorded for billing purposes, and kept long enough to resolve any customer billing disputes. Providing that information to the police was seen as no big deal.

Things are different on the internet. Email, for example, continues to exist even after it's been sent. The same goes for chat logs and file transfers. Routing information exists within the communication itself — think of email headers. And while many activities are logged, those logs are kept to investigate technical faults, not for billing — so they can be thrown out much sooner.

Mandatory data retention is simply the idea that all of that log data be kept, possibly for years, on the off chance that it might, perhaps, maybe one day be useful for investigating a crime — not just in our own country, but in any country that's signatory to the Council of Europe Convention on Cybercrime.

In 2011, attorneys-general from the Quintet nations — the law-officer counterpart of the "Five Eyes" intelligence-gathering nations of the US, the UK, Australia, Canada, and New Zealand — agreed to persuade the whole planet to adopt the convention. Data retention is at the heart of that treaty.

Australia's favourite attorney-general, Senator George Brandis, is pushing the data retention barrow because that's the plan globally — or at least amongst the English-speaking nations that think they run the joint. He'll push it even harder than his predecessors, because he intends to bring a "strong national security focus" to his office, as he said soon after the election.

Data retention supporters argue that because metadata isn't the content of the communication, it doesn't invade people's privacy to anywhere near the same extent — just as with telephone calls. In December, Brandis backed Prime Minister Tony Abbott's characterisation of metadata as "essentially billing data". It's just a few innocuous numbers.

Attorney-General Brandis is wrong. The police want this data precisely because it can reveal so much. Otherwise, why would they want to have it? And it's not even used for billing.

Attorney-General Brandis is clearly either ignorant or wilfully disingenuous. Doesn't he know that the entire commercial economy of the internet is built on the ability to construct or infer all manner of detailed information about people's personal lives by aggregating and data mining the myriad digital footprints they leave behind?

Researchers at Stanford University, for example, found that they could predict people's medical conditions, gun ownership, hobbies, relationships, and religious views simply by looking at the metadata.

"There's a participant in our study who had an early morning call with someone we were able to identify as her sister. And then, a couple of days later, had some calls with the local Planned Parenthood organisation, and then a couple of weeks after had some more calls, and then about a month after had a final call," Jonathan Mayer, the graduate student running the research, told ABC Radio.

"I think it raises the plausible inference that that participant had an abortion, and that in and of itself, even if it's not accurate, should give rise to some privacy concerns."

Another example? "We had a participant who in short order had calls with a lumber yard and a locksmith and a hydroponics dealer and a bong shop. Again, don't need a PhD in computer science to have some sense of what could be going on there."

Data retention is one of the most important political issues relating to our use of the internet now, and as far into the future as you care to imagine, I said on a Patch Monday podcast in October 2012, the last time it was on the political agenda.

Well, it's back on the political agenda right now, because the Senate is reviewing the Telecommunications Interception and Access Act. Police services are starting to issue a new round of ambit claims. The Northern Territory Police even want everyone's web browser history.

In law enforcement, as in every other part of society, the internet is changing information flows — and because information is power, that's triggering a power struggle. Cops and spooks need enough power to do their jobs effectively, sure, but not so much that it intrudes on people's quiet enjoyment of life, or leads to oppression.

Since Attorney-General Brandis is a history buff, he'll probably remember what 17th-century French clergyman and statesman Cardinal Richelieu said, or is supposed to have said. "If one would give me six lines written by the hand of the most honest man, I would find something in them to have him hanged." Under these 21st-century data retention proposals, Richelieu would have 6 terabytes of big data, plus the data mining tools to help construct the incriminating narrative. Is that what we want?

We'll need an intelligent and informed debate to find the new power balance. We won't get there if Brandis, and others of his ilk, continue the metadata fiction, whether they be fools or knaves. It's all data. It's all private.

Topics: Privacy, Government AU, Australia


Stilgherrian is a freelance journalist, commentator and podcaster interested in big-picture internet issues, especially security, cybercrime and hoovering up bulldust.

He studied computing science and linguistics before a wide-ranging media career and a stint at running an IT business. He can write iptables firewall rules, set a rabbit trap, clear a jam in an IBM model 026 card punch and mix a mean whiskey sour.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • Are you suggesting

    that we stop collecting ANY metadata? Or that the definition of metadata currently used is too broad?
    luke mayson
  • A Snowdenite speaks

    Stilgherrian wrote "Researchers at Stanford University, for example, found that they could predict people's medical conditions, gun ownership, hobbies, relationships, and religious views simply by looking at the metadata."

    Your entire article dealt with the abuse of that data by governments, but not by corporations. Why is that?

    Will Snowdenites please answer two questions?

    1) If the Internet is 100% secure, how will we catch child porn purveyors? And if you think this is not an important issue, do you consider children to be mere collateral damage of your desire for complete anonymity?

    2) Why is government spying unacceptable and corporate spying (Google, Facebook, Acxiom, Experian, etc.) acceptable? An answer of "because corporations cannot put you in jail" is insufficient because corporations can ruin your credit, allow your identity to be stolen, allow your property to be foreclosed for no reason, track your every movement via your emails, texts, and phone/tablet network card and then publish that data which would useful for home robbers, etc., with these events possibly leading to where you are unemployable except at low-wage companies like Target and Walmart.
    • Responses

      Prelude: I'm not for or against Snowden. It's nice to know what the NSA is doing, but I don't think we can do much about it. Who's gonna force them to implement changes? They deal in secrecy. They could agree to change, but remain the same. And how long would it take for us to find out a second time?

      Regardless, I'll try to answer your questions.

      1. I don't think Stilgherrian wants a complete end to data collection. I think his argument is that data collection without a warrant under the guise that it's "metadata" is abusive. I.E. He argues that the current definition of metadata is too broad. Note that he calls for a debate. An important subject of that debate would no doubt be devising strategies to catch cybercriminals, including child pornographers.

      2. By signing up for Google's or Facebook's services, you agree to give them your data. I, for one, did not see a sign up form for the NSA's data collection. As for Experian, they are a credit monitoring agency. Their very business requires them to have your data. Saying they spy on you is like saying the USPS spies on you because they know who you're mailing stuff to. It's an integral part of their business. As for Acxiom, I don't know enough about them, but it's easy enough to protect yourself: restricting cookies, Adblocker Plus, Disconnect, and Noscript should do the trick.
    • It's called sticking to the topic

      "Your entire article dealt with the abuse of that data by governments, but not by corporations. Why is that?", you ask.

      Well, that's because the theme of the article is the misuse of the term 'metadata' in the political debate surrounding the push for mandatory data retention for law enforcement purposes.

      The use of personal data by corporations is a different topic — one I've written about in the past and will doubtless write about again in the future.
    • Big Brother vs Big Business

      > Why is government spying unacceptable and corporate spying (Google, Facebook, Acxiom, Experian, etc.) acceptable?

      because one I have a choice to take part in, the other I don't. .... is a short answer.

      a slightly longer one is: because we're American, and it's built into the fabric of our patriotism to challenge the government.... and, well, businesses too if we so desire.

      In the end, will changes in laws stop them? No, not at all. Both government and businesses will just go underground about how they're doing it, be more discreet, hide it off shore but such that they can still access the data. ...but, is there really something wrong with questioning it?
  • 2 points to remember

    Any and all data on you can and will be used against you by the government, regardless of whether you are in a court of law or not.

    The government can be guaranteed to abuse their power.
    • ...the key word...

      the key word there being 'against' you... Check out the YouTube video: "Dont Talk to Police" (spelled exactly that way... without the quotations of course).
  • The real issue is...

    A senior national security executive called it a turn key totalitarianism. I'm afraid it is more a question of when than if.
  • I've said it before and..

    I'll say it again....

    Absolute power corrupts absolutely !

    Bureaucrats and bureaucracies salivate over more power to control citizens lives.

    It's the nature of the beast. They can't help themselves when it comes to making life miserable for those who can't retaliate ! .. the average citizen who doesn't have the resources to fight back.
    Bureaucrats have bottomless pockets when it comes to spending money to obtain just about anything they need to control others. The NSA's huge spend on a new computer centre is just one example.

    Brandis is a devious bureaucrat who is either very clever at clocking his real intent or he is technologically naive !

    I don't care who the bureaucrat is, give him the capability to snoop & he will, especially if it gives him more power.
    ie: DSD, AFP, ASIO, and especially 5 eyes & the NSA..the list of secret bureaucracies is never ending, who, essentially have NO oversight !.

    I don't question the need for these QUANGOS to have the power to investigate terrorist activity, to protect us, however it has to be with the controls & consent of a court of law & rules clearly defined & set down by government rule..

    Giving these organisations unlimited unsupervised powers, which is what they really want, is totally unacceptable. They will abuse it ! It borders on fascism !
  • A mis spelling .. sorry !

    Brandis is clever at 'clocking' was meant to be 'cloaking' his behaviour..