Many ideas and norms once considered unthinkable, like test tube babies and gay marriage, have now become everyday norms. It’s impossible to imagine life without them. For society to evolve, however, we must always be challenging our norms as well as the rules and laws that reflect them. Our institutions must lead in a way that harnesses this questioning into a driver for positive change. This session looks at how institutions can become “disobedience robust” — cultivating the ability to question themselves and accept questioning from others.
Moderated by Joi Ito, Director, MIT Media Lab with panelists
Liz George, MIT Alum Class of 2008
bunnie huang, Author, Hacking the Xbox: An Introduction to Reverse Engineering
Karrie Karahalios, Assistant Professor, Siebel Center for Computer Science, University of Illinois
All panelists are former MIT students (although Joi says he come in the backdoor:). Before this event, Joi interviewed lots of administrators at MIT including John DiFava. And everyone said that they had never met a student who was a bad person. And DiFava spent his career chasing bad guys with the MA State Police before coming to MIT.
Karrie remembers coming to MIT for rush week her first year. She took an “orange tour” and loved it, and it seemed like it was sanctioned by the university. Her House Master encouraged the students to win the East Campus lockpicking contest. And students were constantly hacking things in the dorms like phones and washing machines.
A discipline committee was set up around 2003. The police stopped arresting people at all, but started picking up more people and sending them to this discipline committee. So people who would have been let off altogether were getting in trouble.
The trouble now is that students have this fear of getting caught.
Should they not?
Then it’s hard to abide by that last point of the hacker ethic: avoid getting caught, but if you are, cooperate fully.
I find in some places if everything is legal, it’s not nearly as interesting. Is that a big part of it or not?
Sure it is. [but that doesn’t mean “risk of imprisonment”]“Just so you know, statue of limitations is 7 years.”
There’s a thing, ‘if you see something broken, report it to Physical Plant’, we would note these things if we were up on the roof and tell them. That almost helped build as sene of community across the groups. I did get caught at the ML, I was picking a lock here, I wanted access to one of the tool rooms? They described me as “tall kid, dark hair, asian” in an email to ML. I could have avoided fessing up, but I did tell someone in my lab. They got reasonably mad at me, saying just ask for permission, it’s not hard, and showing me how. That was a scolding, not disciplinary action. I’m a little cynical about MIT’s stance – Liz said MIT likes to own successes and disavow failures [JI – that’s broadly true you know, in the world] But a university is a good place to catch people when they stumble. Especially when they just did something unfortunate [not hurting others]
The ethics associated with hacking. Breaking the rules mindfully. MIT hacker secret sheet #forbiddenML pic.twitter.com/V2ctRuvZiV
— andres lombana b. (@vVvA) July 21, 2016
Hacking the xbox led me to discover the problems with the DMCA which led me down this whole long path to where I am today [w today’s announcement]. My advisor Tom Knight introduced me to the general counsel, I was all excited; they met me with a sealed envelope on the table; they said I just want you to know we don’t want to touch this or have anything to do with it; you did this on your own time with your own funding, good luck.
I wanted to know, why is the institute disavowing me? I didn’t even disclose what was going on, I just wanted their help to do disclosure responsibly. They wouldn’t even hep with that. Fortunately Hal A and Tom K helped me find an amicable solution (via the EFF).
Joi: Ethan mentioned TidBit, where we had a student working on a project. It’s funny b/c MIT’s counsel represents MIT. There’s some things they can and can’t do. We ended up setting up a law clinic – I’ll give credit to the GC’s office and Provost for this fully-funded and pro-bono clinic at BU. So any problems a student has can go to a BU clinic; in your case they could have directed you to that. This is recent, within the last several months. One of the problems has been the liability concern that something happens and the inst. gets sued by the parents, and lawyers are lawyers. We’ve been trying to set up ways for people to talk to the admins. Legal support is key. Hoping that will help in the future.
There are certain things the institute should sanction, but there are certain things we can’t. On the research side, there are things that affect the whole institute. Some kids don’t know what the repercussions are. I went to a lot of cases I heard about to hear their side of the story. If some of these things came out, it would hurt privacy of the student. It can be better. But the trust gets developed when you have communication. Need informal communication because formal puts you at risk in other ways. Secret backchannel. So the hackers will often tell the police so they’re not caught off guard. No formal acknowledgement. People based and don’t persist over time. 4 year turnover. Met about what could be done. The policy in the handbook was drafted, students said “given all the things that have happened, you can’t ignore it.” You can’t say “look how amazing this place is” and then not support people when they get in trouble. People had to meet with a lot of people about what could be said legally. What was acceptable to write in there? The trust went up a lot after that.
What are the principals we put into a disobedience prize? Principles, playfulness, creativity, social benefit. Large scale collaboration. Swipe all cards in all doors as a way to add noise into a database. CSAIL and Media Lab run our own networks. Here we retain as little information as possible. No cameras here because it creeps us out, can be taken over. We have a lot of theft here, we know we wouldn’t catch the professionals anyway. Research in how we increase security without sacrificing privacy.
What aren’t you allowed to do? Purposefully destruct something for the sake of being destructive (if you’ve evaluated that it’s necessary, go for it).
Relationship between an institution and those performing civil disobedience is difficult. People who get attacked by people on the other side of the argument. Building an institution where disobedience doesn’t have grave consequences. It doesn’t have to be completely safe, but reasonably safe. Tenure, if you piss off the people in your department you don’t get it. How do you do things which speak truth to power while still being on people’s good side.
Tenure was put in place to allow people to be disobedient during McCarthyism. But maybe you’re past that phase in your life by then, you’re like 45. High school Japanese kids working in a lab, being picked on by grad students. Academia shouldn’t be about that, it should be the new people encouraged to question authority. “Does scientific research advance one funeral at a time?” – Max Planck Spoiler: yes [according to a recent MIT paper (PDF) which you should read].
These people learn most of their things on the internet; but without access to journals, they have a hard time applying it. We can go to first principles and questioning these things. That’s what I want to start with: questioning these things [publishing, bad laws]
Q+A
Q: do you think the administration today is too strict? A: when I was a student, yes. students aren’t doing this for the thrill of the illegal; but curiosity, exploring the inaccessible, doing a great piece of engaging engineering.
Maria Zuber: We now have a list of amazing, disobedient folk… a list we have to protect.
How is our Head of Research about this? I’m so glad we’re great at being self-reflexive. I think it went well. You’re listening to all this and some of it is shocking or discomforting, but the MIT administration talks about this stuff all the time. There’s a whole range of opinions. The administration gets a full spectrum of points of view. The ones we heard here [today] are not in the center of the bell curve on where the campus falls in a lot of matters. If there’s a question of how far we can push something, we can always have a conversation about it at least. Try to find the right balance. Try to explain why when we can’t do something.
One thing we are talking about is: is it OK to take an action that affects a broad population? IRB violations could shut down research across the institution. When things come to me, I have to think about the balance of people wanting to do something that moves their work in a positive direction (for them) but may affect others.
Can I ask a Q about that? ITAR rules for instance prvent you from bringing IR sensors into certain countries. Shoud an institution like MIT be pushing back against these rules with the govt? Saying these are rules that are iportant? MIT as an institution has a lot of clout.
A: on that particular issue, I have in fact been pushing back, personally. I’m a space scientist, so I care a lot also about thse cameras. We do that. but in the meantime we [may?] have to also obey the rules so people can continue to do their work.
Joi friend at Twitter and now White House. Would go to police events and just hand out his business card. So instead of busting things, we’d just have people calling him to fix it first. [This is so reminiscent of disaster response I can’t even]. When it happens in public there are egos on the line, too.
(Cory) The opposite of disobedience isn’t obedience, it’s compliance. it’s if you don’t immediately comply with an order, you risk summary execution.
Need to challenge laws in order to progress the nation.
(Sam) There are simple laws and painful laws. How do we make sure the institute isn’t standing in for excessive response, even when it happens outside that bubble?
(Joi) There are laws which are supported by commercial interests (DRM, CFAA, SoPA, PIPA). The Media Lab has made statements against those, and we have a lot of money from Hollywood (none of which went away). Courage to stand up to those that are backing you. We’re in a privildeged position to do that. Then there are laws that limit academic and scientific progress. There’s a way to try to talk to the authorities. There are broad thinkers there.
(Liz) those of us with power need to take those risks, support our folk when they cause problems.
(Kendra) Institutional trust and these back channels, I feel that insitutional trust is not something everyone has. What is MIT doing to help students know they have support? If you are a young black man, being arrested has different connotations.
(Joi) I’m thinking about this a lot lately, and it’s not just for us. I just try to talk with and connect with as many folk as possible
(Karrie) That you have this clinic is a huge signal that you want to support your students. I’d love a phone number for those students. I’d love to be able to promote this sort of structure at my university. We don’t have access to this sort of clinic.
(Joi) if we can figure it out, I hope others will also follow.
() This has been fantastic and it’s appreciated. What about research that would piss off your liberal collegues? It’s close to an ideological monoculture. Many of these entrenched things are not useful. Can we have another Forbidden Research conference which pokes holes in liberal assumptions?
(Joi) DARPA study on race and bias etc. The Uni had to be sued to release the study.
Steward Brand, claiming the final question
What’s most interest to me is the elegant hack, minmax, lazy hack. So fiendishly clever, subtle and undetectable. And yet has a great big effect. Do you have any examples of that?
(liz): The insription in Lobby 7 says what the institute was founded for: ‘for the furthering of science and technology and agriculture and commerce’ 20yrs ago some hackers made an incredible reproduction, but replaced the last two words with ‘entertainment and hacking’. Anyone passing by thought it was the original; and it was there for a really long time until a tour guide noticed it, and said “Here are the famous founding principals of MIT, and read it outloud.”
Joi: there’s a piece of artwork, metal and black, in front of the Green building. it’s a bunch of pieces of metal. some kids would leave pieces of metal, and people would think it was broken and weld it back on… it’s a good story, don’t know if it’s true.