Creating Technology for Social Change

Forbidden Research liveblog: Against the law: countering lawful abuses of digital surveillance

With bunnie huang, Author, Hacking the Xbox: An Introduction to Reverse Engineering and Edward Snowden. Liveblog by Sam Klein, Erhardt Graeff, and myself.

Introduction and overview from Snowden

This is my first time giving an academic talk, and I think it’s the first time a US exile is presenting research at a US academic institution. One of the great things about Cory’s talk is that we don’t talk enough about how laws are a weak guarentee of outcome. theft, murder, etc still happen.

I’m Edward Snowden, I’m director of the Freedom of the Press Foundation. Some years ago I told the truth about a matter of public importance. Some years ago a warrant was issued for my arrest. I’m no longer allowed to travel freely. I’d like to thank MIT for organizing ths conference and the opportunity to speak to everyone in the room today. For journalists in the audience, that’s not a small thing; they deserve credit for living up to that commitment to knowledge. No one is perfect, everyone makes mistakes, but that is quite a risk. This may be the first time an American exile has been able to present research at an American university. That’s [already] enough reason to have this talk at a forbidden research conference.

The guiding theme of many of the talks today is that law is no substitute for conscience. Our investigation covers lawful abuse. What is that? It seems it might be a contradiction in terms. When I talked to someone on Twitter, immediately they said ‘lawful abuse – it’s not a contradiction!’ But if you think about it for a moment it might seem more clear. The legality of a think is after all quite distinct from the morality of it. I claim no special expertise for any of this, but having worked for both the NSA he CIA I know about about lawful abuses. After all, mass surveillance was thought to be constitutional… yet it was later found by the courts to be different, after more than a decade. A lawful abuse, I would define as “an immoral or unethical activity protected under a shell of law”.

What about things that are more recent? Mass surveillance is closest to my own experience, but let’s set that aside. What about torture? the Bush administration decided that this could be indefinitely [legalized]. What about internment? Extra judicial killing, far from any war zone, often by drones? The [targets] may be criminals, or armed combatants — in many cases, but not all. The fact that these things are changing, often in secret, without anyone’s consent, should be concerning.

Such abuses aren’t limited strictly to national security. We don’t want to this to be about politics between doves and hawks.
Segregation.
Slavery.
Genocide.
These have all been perpetuated under frameworks that said they were lawful as long as you abide by regulations.

Lawful abuse surveillance might be more difficult to spot:

  • A restriction on who and how you can love someone,
  • An intentional tax loophole, or
  • Discrimination.

Lawful abuse: so we’ve defined the term. [Willow is thinking about an anarchist zine about D&D called “Lawful Ain’t Good” and how there are only 8 (not 9) alignments.!]

Combined with legal frameworks, our daily activities produce an endless wealth of records which can and are being used to harm individuals, including those who have themselves done no wrong. If you have a phone in your pocket that’s turned on, a long-lived record of your movements has been created. As a result of how the network functions, your devices are constantly shouting into the air, via radio signals, a unique identity that validates you to the phone company. This is not only saved by the phone company, but can be observed as it travels, by independent, even more dangerous third parties.

Due to proliferation of an ancient 3d-party-doctrine style interpretation of law, even the most predatory and unethical data collection regimes are [usually] entirely legal. So if you have a device, you have a dossier. They may not be reading or using it, but it’s out there.

Why should we care? Even if there are these comprehensive records of your private activities: where you are, who you went with, how long you were there, who you meet with, what you purchased – any electronic activity records…?
I can think of 1,070 reasons why it matters. According to figures of the committee to protect journalists, more than 1070 journalists or media workers have been killed or gone missing since January 2005. This might not be as intuitive as you expect… we’ve had a number of wars going on, those could be combat deaths. But: murder is a more common cause of death, and politics was a more common newsbeat [to be targeted] than war correspondence.

Why is this? Because one good journalist in the right place and time can change history. They can move the needle in the context of an election. They can influence the outcome of a war. This makes journalists a target, and increasingly the tools of their trade are being used against them: technology is beginning to betray us not just as individuals but as classes of workers, including those putting a lot on the line in the public interest – especially those who rely on communication as part of their daily work.

And journalists are being targeted specifically based on those communications. A single mistake can have a lot of impact; it can result in detention. For example, David Miranda (related to reporting on Snowden) had his materials seized by the British government, after they intercepted his communications about plans to travel.

It can also result in far worse than that. In Syria, Assad began surveillance the city of Homs, to the extent that all foreign journalists were forced to flee. The government stopped accrediting journalists, and they were being beaten, harassed, disappeared. Only a few remained, including a few who specifically headed there to document abuses being visited upon the population.

Typically in such circumstances , a journalist wouldn’t file reports until after they had left the conflict area, to avoid reprisals. But what happens when you can’t wait? When there are things a government is sort of arguing aren’t happening, but are happening? At the time they denied they were targeting civilians; they were claimed to be enemy combatants. These lawful abuses of activities happen in many places. You say surely this isn’t lawful! By international law you are right; by any interpretation of the Universal Declaration of Human Rights, it’s not lawful. But domestic laws are a hell of a thing… China, Russia, North Korea, Syria have courts. They have lawyers and general counsels, who create policy and frameworks to justify whatever it is the institutions of power want to do.

In Homs, the Syrian government was lying in a way that affected international relations: they justified the offensive, but there was a reporter there [Marie Colvin] infiltrating the city. She crawled in through a tunnel in the dark, climbing stone walls, not speaking to avoid being fired upon. She said this [the government’s claim] was not the case. She filed live report despite the fact they worried about reprisal. She spoke four times to government agencies on a single day. [quote from Colvin’s report – “there are only civilian houses here”], the building she was in was later precisely targeted, and she was killed.

This might sound like just another war story. But the next day, the makeshift media center she was working out of, was repeatedly and precisely shelled. She died, as did a French journalist. The photographer she worked with was wounded. It wasn’t until a while after that we found, based on intelligence collection, that the Syrian Army had given the order to target journalists. How did they discover her? Know where to aim? According to reporting this week: her family has filed a suit against the Syrian government, claiming the audio frequencies of her communications were intercepted by the army (using direction-finding capabilities). Then they walked artillery fire towards the makeshift media center. They had a spotter somewhere in the city helping. By the time the second shell hit, they know they were in trouble… She was caught by a shell and killed.

There’s a question here among policy officials: was this legal, how do we remediate these threats when they happen, when do policies fail? This is an argument that the Syrian government says the event was misunderstood—these were terrorist attacks, or they were lawful.

But does it matter, if it was lawful or not [by national law]? [Perhaps we should ask:] Was it moral? Can we put safeguards in place for future journalists? What about journalists who have to meet with a source in a denied area? They don’t want their phone to be shouting indications of their movements.

This is the area of our research.

We also wanted to investigate: Can we use devices, that are so frequently used against us, as a canary to detect these new efforts to monitor us? (ex: malware attacks, to compromise the phone)

For example, there was an Argentine prosecutor [Alberto Nisman] who was killed. They discovered malware on his phone. It did not match the OS, so it was not responsible in that case, but it was clearly an attempt has been made to compromise devices and use them against him. This same attack was used on other lawyers and journalists in Latin America.

If we can start using our devices as a canary to know when phones have been compromised, and can get that to a targeted class of individuals—journalists or human rights workers—so they know they are acting in unexpected ways. We can affect the risk calculation of the offending actors. The NSA is very nervous about getting caught red-handed. They don’t want to be known to target these groups, journalists and lawyers. They have only done this rarely; it’s not their meat and potatoes [but it has happened].

But if we can find out when it happens, we can start to change the risk calculation. If we can create a clear record of activites. In the cases so far, impunity was the most frequent outcome. Perhaps, we can start affecting the cost of carrying out lawful abuse of digital surveillance.

Let’s go to the technical side and talk about what we’ve done. [to bunnie]

bunnie tells us about the technical parts

There are a lot of smart people working to turn phones into cyber fortresses. But smartphones are a large, complicated attack surface. Can you trust the gatekeeper and UI? If you read things about airplane mode after ios8, it doesn’t turn off GPS. It’s constantly on without any indicator on the phone. So you can turn on bluetooth or wifi mode… but The little icon makes you still think you’re not sending or receiving signals. Can we have a CCTV on our own phone? Technical goal is to be sure the cellular model, WiFi, GPS, etc. Trying to secure this against a state-level adversary is difficult. Turn over the phone and look on the back, and you have a surface that’s simpler, with only two notable features: antennae. A choke point for things going in or out. If you want to ensure your phone isn’t sending signals, you can turn on airplane mode.

Technique: “Direct introspection”
Principles:

  1. OS and inspectable, you don’t have to trust us.
  2. partitioned execution environment for introspection. (in case phone was compromised, don’t ask it to self-eval)
  3. proper operation field-verifiable,
  4. hard to trigger false positives (like walking by a strong wifi emitter),
  5. hard to trigger false negatives Vendor can put holes in a wall that you once thought was intact.
  6. be undetectable: avoid leaving a signature that’s easy to profile (that you’re introspecting)
  7. intuitive interface 🙂 Shouldn’t have to be a cryto person to use it.
  8. final solution should be usable every day; not hard to do while traveling in and out of protected areas.

With that in mind, I went to shenzhen and started buying a bunch of bits and bobs. Are there any viable signals to introspect? We found signals strongly correlated w/ activation of the radio. even firmware updates would have a hard time bypassing that. Candidate wires/signals: configuring antenna switches, configuring power amps, baseband to comms, wlan to comms, reseting pci bus, bluetooth to comms, gps quality sync.

Next steps:

  • Develop hardware. Build circuit to monitor signals. “Battery case” add-on to existing iPhone 6
  • Extend technique. Other makes and models of phones. Filesystem and OS integrity using disk introspection.

Closing

See more: htps://goog.lg/y0Fslu and pubpub.org/pub/direct-radio-introspection

This was my first acad collab; having bunnie as your first collaborator is amazing. He is one of those individuals whose competence gives people impostor syndrome. So, I’ll do my best. thank you very much.