I started participating in the GA Tally project to help refine the code base and get it ready for a more serious launch. The idea has always been simple: make it easy for folks to visualize gender disparity at events, in lists, and in the media. The service is built in NodeJS and uses Google Charts to generate pie graphs based on the input of the user.

For my part I helped clean up the interfaces, add some core functionality, and bring it to the iOS and Android app stores. I’m also helping the Gender Avenger team think through their technology strategy.

Papers, Posts, and Press

• Gender Avenger site lets users spotlight gender inequality (Milwaukee Journal Sentinel)
• Where My Ladies At? Gender Avenger Tracks Inequality At SXSW And Beyond (MTV News)
• New tool aims for gender parity at conferences (MSNBC)
• Gender Avenger Wants You to Report All-Male SXSW Panels With Its Simple App (Good Magazine)

Technologies

• NodeJS

Testimonials

At first I was skeptical that the problem of gender inequality could be universally solved in one swoop with a pie chart app. I couldn’t have been more wrong!
– Apple Store Reviewer

You may have heard of closed captions before: they’re the little words that show up at the bottom of TVs in airports, bars, or maybe at home when you mute the TV. Opened Captions scrapes those captions and regurgitates them into something called a web socket (web sockets take data and pass it along to anything willing to listen.) The service can take in caption data directly from a USB, or from another opened captions server.

This project came from my Knight-Mozilla fellowship at the Boston Globe during the 2012 elections. We were brainstorming innovative ways to cover the presidential debates, and quickly realized that the best of them required a way to know what was being said in real time. I got to work looking for solutions and, finding none, decided to make my own. A few days later and Opened Captions was born.

Although we didn’t end up building any official Boston Globe coverage, I did manage to throw together a wonderful project called DRUNK-SAPN. The application streamed the debates and told the world to drink on key phrases. Most importantly, the transcript took shots along with the players. You can view the transcript here.

Papers, Posts, and Press

• Introducing Opened Captions (slifty.com)
• Opened Captions: Turning the spoken words on TV screens into streams of hackable data (Nieman Lab)
• Introducing Opened Captions: A SocketIO API for live TV closed captions (Source)
• Opened Captions (Slideshare)

Technologies

• Node.js
• Socket.io

Testimonials

h*hic*Tey brougttth*hic* su binder*hic*s ffflul fo wmoen.
– Mitt Romney

CivOmega makes it easy to convert government data and APIs into resources for people who didn’t spend half their lives learning to program. For instance I made it possible to ask the question “What bills are about [X]” where X can be any phrase you want. Behind the scenes CivOmega knows how to go out and talk to the appropriate datasets at which point light shines down from the heavens and angels sing to you about congress.

Lots of data has been published, but it’s hard for non-technical constituents to benefit from it. Now, thanks to CivOmega, when a new dataset is released to the developer community, someone could spend a few extra hours to unlocking all sorts of new questions and answers through CivOmega.

The system is built in Python, and the first version of this was created during a 24 hour hackathon at the MIT Civic Media Conference in 2013. Since then it has been rewritten with a more attention to code quality and extensibility thanks to a grant from the Sunlight Foundation.

A CivOmega module is made up of three parts:

1. Question patterns. These patterns are used to guide a user to questions that the system knows through the main CivOmega interface. When a user asks a question that fits that pattern, CivOmega creates a structured object and passes it to the module that knows the answer. (Example: { “politician”: “John McCain”, “issue”: “Campaign Finance” })
2. Mapping Logic. Custom logic takes the structured data extracted from a pattern by CivOmega and uses it to look up answers. (Example: using the Sunlight Foundation’s congress API to get a list of all bills about Campaign Finance that were supported by John McCain.)
3. Answer Templates. The result of that search is packaged and sent to an HTML / JavaScript / CSS template, which renders the answer in whatever format is most appropriate (as determined by the module author).

Papers, Posts, and Press

• Introducing CivOmega: An Effort to Democratize Government Data (slifty.com)
• The alpha of CivOmega: A hack-day tool to parse civic data and tell you more about Beyoncé’s travels (Nieman Lab)
• Projects from the OpenNews-MIT JHack Day (Source)
• CivOmega Wants Your Open Data APIs to Help Fuel Search Engine (Programmable Web)
• OpenGov Voices: CivOmega: Making Government Data Useful to Everyone (Sunlight Foundation)

Technologies

• Python
• jQuery

Testimonials

Oh good, answers to questions I’ll never ask.
– The American Public

NewsJack was built as a class project for Sasha Costanza Chock‘s “Introduction to Civic Media.” It enables détournement using web technologies. For those of you who don’t speak French / Chinese / Whatever that is, it means “turning expressions of the capitalist system and its media culture against itself.” It’s a very specific form of satire that takes subversive messages and wraps them in a skin that you are used to seeing (this might mean brand, it might mean medium, it might mean something completely different). Good détournement forces the viewer to question their world and their expectations.

The specific inspiration behind NewsJack is the Yes Men’s New York Times Special Edition. This fake paper was actually printed and handed out on the streets of New York City in 2009. It looked like a real copy of the Times, but had headlines like “IRAQ WAR ENDS” and “Maximum Wage Law Succeeds.” Imagining picking up a newspaper that you believed was the New York Times and seeing that type of headline. For a moment, until you realized what was going on, it might change the way you see your world.

That’s the experience that I ultimately wanted to enable online, where content is far harder to modify and spread for anyone who doesn’t know how to code. It is built using a modified version of a Mozilla tool called Hackasaurus. The original code was designed to help people learn HTML (the core building block of the Internet). I stripped out all that hippie foo foo learning crap and left the essence: code that makes it possible to edit a website by pointing and clicking.

Once you’re done you can hit publish, and a copy of the page you just modified is uploaded to some server out in the universe. You get a URL to share around, and suddenly your remix is alive and kicking.

You might say to yourself: “… how is this legal?” to which I would respond by picking up a copy of the first amendment, making a paper airplane out of it, and throwing it at your head. Of course, that didn’t stop the New York Times from sending a cease and desist on the first day we launched the site! I suppose that’s a story for another time.

Papers, Posts, and Press

• NewsJack launches to let you remix, edit news websites (Poynter)
• All The News You’d Love to See: NewsJack Launches To Allow You To Remix & Spoof The Media (BostInno)
• Jack This Site: NewsJack Lets Regular Users Remix Websites (Media Bistro)
• Workshopping NewsJack with Press Pass TV (C4CM)

Technologies

• jQuery
• MySQL
• PHP

Testimonials

I have a feeling our lawyers will be particularly interested in this project.
– Every Mainstream Media Source

The Onion is an amazing satirical newspaper, which I suggest you read if you don’t already. It features headlines such as Michele Bachmann Figures Why Not, Introduces Homosexual-Beheading Bill or New Obesity Drug Delicious. In case you couldn’t tell, these headlines are fake.

But not with The Ononeon. The Ononeon is updated every day with new headlines that look like they should be from The Onion, but sadly are not. Like China Censors the word Censorship, or Phillies fan covers face with shirt; proceeds to drink beer through shirt.

I built the site with Matt Stempeck because he was about to lose control of the URL and we wanted an excuse for him to pay for another year of it. Plus we had four hours of free time. To this day it is by far our most popular project.

It works by hijacking the top rated headlines curated by the /r/nottheonion, a community that finds headlines and news articles that look like they should come from the satirical paper. For each headline our script runs an automatic google image search and saves the top image. After that takes a cloned copy of The Onion website’s layout and design and replaces all the old fake headlines with our new real ones.

The beauty of all this from an academic perspective is that it uses a blend of human curation and algorithmic search to create a pretty damn effective satirical experience. From all other perspectives it is purely hilarious.

Papers, Posts, and Press

• Stranger Than Fiction: Meet Two MIT Grads Who Built The Onion’s Non-Fiction Dopplegänger (Cultivated Wit)
• The Ononeon: Your One Stop Real-Life Onion Stories Shop (Gawker)
• “Real” Onion parody rips off Reddit’s funniest headlines (Daily Dot)
• The On1on Is Like The Onion, But Real (Animal)
• Front page of Hacker News

Technologies

• PHP
• Reddit

Testimonials

Kick them out of school
– Steve Hannah, Onion CEO

Truth Goggles was my master’s thesis at the MIT Media Lab. This means that I spent a lot of time exploring the many challenges behind an idea like this, and attempting to solve at least some of them. There are three gigantic hurdles to jump:

1. Fact Database – What should be used as ground truth? How do you identify it? Is there enough of it? I ended up realizing that there aren’t many universally believed truths, and I want the system to be accessible to a diverse audience. It needs to contain as many well-explained and thoughtful verdicts as possible. I decided that for the first round a truth source needs to do things: have a reputation of neutrality (i.e.both sides call them biased or neither side calls them biased), and explain the reason behind their verdicts.
2. Paraphrase Detection – There are thousands of ways to say the same thing, how do you identify known fact-checks with slightly different phrasing? This is a challenging problem because it mans computers need to understand language. Luckily there are a lot of smart people exploring this space, so I can use existing tools to get part of the way there.
3. Human Brains – Assume we have a perfect system that is able to identify fact checked phrases 100% of the time. Would you trust it? Would you use it? What if it told you that you were wrong, would it change your mind? A system that isn’t usable isn’t worth building.

I focused on this third problem (human brains) because I like thinking about people more than algorithms. I tried to design the system to make it easy to swap out better algorithms and data sets down the line, but my experimentation revolved around the user experience.

The result of my thesis was a prototype and a gigantic document. The front end is written in JavaScript and jQuery. It scrapes the page and sends it to a credibility API written in PHP. That API checks against known instances of paraphrases and also sends the text to a “fuzzy matching” API that is currently written in Python.

The prototype can be used anywhere online, but it doesn’t do any intelligent paraphrase detection at this point so chances are it won’t be useful in most places. The study results were promising, and indicated that credibility layers could very well help people think more carefully and in more nuanced ways.

There is still a lot to do for Truth Goggles, and the project is not dead! It’s also clear that people love the idea of an automated bullshit detector.

Papers, Posts, and Press

This got a huge amount of coverage, here are some highlights.

• The first piece by Nieman Lab
• El Tiempo
• CBC Interview
• La Vanguardia
• The Register
• Tech Crunch
• NPR Interview
• The second piece by Nieman Lab
• Boing Boing
• New Scientist (Volume 215, Issue 2882, 15 September 2012, Pages 44–47)
• Wired UK (November 2012)
• Introducing Truth Goggles
• Achievement Unlocked: Thesis
• Truth Goggles Study Results

Technologies

• jQuery
• MySQL
• Python
• PHP

Testimonials

Oh shit.
– Politicians

Long Description

The Media Lab gets a lot of visitors. People bring in friends and family, sponsors send representatives, and others just wander in off of the street. The Glass Ingrastructure is designed to help anyone quickly and effectively explore the kind of work being done in the building without having to bother us. Basically, grad students really just don’t like talking to people if they can avoid it.

The system is made up of about 30 giant touch screens, wired to mac minis and RFID readers. They are located throughout the Media Lab’s old and new buildings, and display maps, projects, and other applications.

When I arrived the Glass Infrastructure had a fairly linear interface. Each screen displayed a rotating list of nearby projects based on where it was placed. Our goal was to redesign the entire thing, building on the systems already set up. We had a pretty large team too (about nine people) working on various parts of the product.

We wanted to reflect the relationship between people and projects without overwhelming the user. There also needed to be a way for people carrying RFID tags to “favorite” different projects, and view a list of their favorites on the bottom of the screen. After weeks of discussion and thrown out designs we came up with the molecule interface.

I spent most of my time working on client side middleware, middle learning a bit more about jQuery at the same time. The backend was a RESTful API that worked with the database in addition to some fancy Natural Language Processing (NLP) tech which automatically found project relationships based on their descriptions.

The design that we came up with is still in use today, and the Glass Infrastructure has become a core part of the visitor experience at the Media Lab. We have also set up a few screens in other parts of the world (Spain, for instance) so that sponsor companies can have a window into the work being done here.

Technologies

• jQuery
• CSS3

Papers, Posts and Press

• The Glass Infrastructure: Using Common Sense to Create a Dynamic, Place-Based Social Information System (Havasi, 2011)

Testimonials

You know what would be better than asking me? Going over and prodding that screen over there.
– Media Lab Grad Students

Long Description

ATTN-SPAN was my final project in a future-of-TV course called Social Television. My goal was to figure out a way to make C-SPAN not completely suck. Fun fact: this was almost my thesis project.

The plan was to take C-SPAN, processes it, tag the moments of video with as much metadata as possible, and use those moments to create a compelling consumption experience. Where would the data come from? Three places:

• Closed Captioning – C-SPAN is closed captioned. The captions are full of typos, but they can be used for general analysis (e.g. guessing what topics are being discussed).
• Video Processing – OCR (Optical Character Recognition) is a process that takes an image and looks for words. On C-SPAN, text consistently appears in the lower left hand corner of the screen to state names of politicians, bills, states, and political parties. Face recognition is another possible technique.
• Audio Processing – there are a few algorithms that can detect different voices in a conversation. They might not be able to tell you what is being said, but you can at least get a sense of who is talking.

From these streams I can do things like find all content that was spoken by YOUR senator. It might not sound that interesting but it actually does make the content significantly more relevant.

I made a quick prototype using an existing project called Metavid. This service handled most of the above processing for me, so I could focus on creating a fast hack to show what it would be like to have an episode of only your senator. I eventually hoped to do all the heavy lifting myself (rather than rely on a third party).

The prototype was a reasonable success, although it didn’t end up being insanely compelling. This was partially due to implementation and partially due to the fact that Metavid started acting unstable and was slow to process new footage.

Had I continued to work on ATTN-SPAN I would have designed an API so that newsrooms request custom generated episodes. Those episodes would be displayed along-side their political coverage. Imagine if you could read about new policies and see exactly what ~~your~~ senator said on the floor about the topic, right there, embedded in the article. This concept of automatically filtered primary source content almost made its way into Truth Goggles. It still might some day.

Technologies

• PHP
• MySQL
• jQuery

Posts and Press

• My learning lab proposal

Testimonials

I had no idea that C-SPAN was so interesting! Oh wait no this is still really boring.
– The American Public

Long Description

This project was a collaborative effort by myself, Julia Ma, and our undergraduate researcher Nat Atnafu. It started off as a course project for “Eccescopy” taught by Ken Perlin, which was about social augmented reality. Our argument was that looking into the past is totally social, it’s just time lapsed!

The system is made up of a web server (written in PHP and MySQL), where you can upload and annotate panoramas, and an iPad client (written using OpenFrameworks) that accesses and renders those panoramas. The user must calibrate the panorama by orienting the iPad so that the space being displayed matches the location of that space.

After calibration, the iPad tracks rotation and updates the portion of the panorama that appears on screen. This creates the illusion that the iPad acts as a looking glass into the past. You can see people and objects that used to exist in the space in front of you standing right there.

Although Ghost panoramas are never perfectly aligned, it gets “close enough” for your eye to believe it. We can get away with this because you aren’t able to focus on the iPad screen and the real-world background at the same time. The fact that the scale might not be quite right or that the image is offset by a few degrees won’t actually disrupt the experience.

It also turns out that this approach can be used to create other AR experiences. Normally you need to figure out where the camera is actually pointing with very high levels of accuracy if you want to create a compelling effect. By using an existing panorama you don’t need to be pixel perfect in the real world any longer, you can just use the pixels from the photo and be done with it.

We imagined several use cases for Ghosts. The original intent was to make it easy to capture and re-live community or personal events. If you took a panorama of a wedding, or of a block party, you could use Ghosts of the Past years later to remember what the event was like. This is pretty much supported in full.

You could also use the AR potential of Ghosts to provide more information, or to show hidden objects. We worked with Jim Vrabel, a local historian, to create a few panoramas with “information points.” Users could then stand in special locations in downtown Boston and learn more about the buildings by pointing their iPad at the right spot.

You can also manipulate the panorama itself using photo editing tools to create some interesting effects. We worked with the MIT Museum to create a few panoramas that spliced in old photos of artifacts working in their original space. As you viewed the panorama in the museum, you would see the objects sitting in the lab with scientists working around them.

There are some obvious next steps. The first is geofencing using the iPad’s GPS, so that only nearby panoramas can be picked, and so that people can be guided to stand in the optimal spot for viewing. The second is auto calibration using the iPad’s compass, so that images are oriented properly without any effort on the part of the user.

Technologies

• C++ / OpenFrameworks
• PHP
• MySQL

Testimonials

Hasn’t this been done already?
– My Advisors

This was my final project in How to Make Almost Anything. The idea was originally an artistic/troll project: I wanted to create a modern day printing press. You would line up the letters, spread the ink, and press it onto paper. The message would go on the paper as you would expect, but the type blocks would secretly be digital, and the message would also go directly onto Twitter.

I decided to change course because, while that application was a nice statement about legacy media, it seemed that abstracting the interaction might be more useful. It turned out that group members had talked about the idea of connecting more things to the internet and using Twitter as a way for those things to communicate long before I had arrived.

I wanted to be able to swap out sensors, so I designed three types of circuit:

• A hub which did the communication with the Python that sent out tweets.
• Chain-able receivers which could be connected together, allowing for any number of sensors to be added.
• Sensor modules which could be plugged into receivers. These would be responsible for measuring whatever the sensor measured and triggering the hub to send tweets.

I made four sensor modules. I started with a simple four-state button that you could set to hold a static value (e.g. “happy, sad, bored, excited”). Then I moved onto using a light sensor, a temperature sensor, and then I wrapped up with a proximity sensor. This kit was enough to warrant a mascot (the pet rock I had created as an earlier project) and tape the proximity sensor on his head. He talked about all sorts of things, like when people walked by, or when he felt lonely, or when it got too dark.

Behind the googley eyes, sensors would trigger events when their readings changed significantly. After an event trigger, the hub would then ask all of the modules to share their messages to the world, and those messages would be passed to a Python script over USB. The Python would then send it to Twitter.

Honestly, I spent most of my time learning to make the circuits (and make them modular), so I wasn’t able to implement the full vision. That vision would have included wireless functionality, direct communication with Twitter (bypassing python), a more robust set of logical operators (e.g. “count the number of people that entered a room”), additional sensors, and far more pithy quotes.

Basically what I wanted it to look like was Twine, which came a year later from the two Info Eco alums who had originally explored this space.

Posts and Press

• My How to Make Almost Anything blog post

Technologies

• C
• Milling Machines (to create the traces)
• Sensors, solder, and microcontrollers
• Python

Testimonials

What is this, some kind of party? Don’t you know the neighbors are trying to sleep?
– Your living room