You may want to skip this boring post and just check out the site.

Over the past 24 hours I worked with an amazing team to start building a Siri for government. Well, Wolfram Alpha is more like it, but you probably have a better sense of what Siri is. The site is called CivOmega and it allows you to ask any question you want about civics. The system will do its best to get you an answer.

I can’t speak for the team, but I’ll let you know why I proposed this idea at a hackathon about open data. I’ll even use big letters:

Open Data Sucks

People have talked about making government data more accessible for approximately 500 years. The hope is that if you can find data about the way your government operates, you can shed light on interesting patterns and stories. It’s all about transparency and accountability. It’s a beautiful concept. It’s wonderful for society.

But actually data is pretty crappy. It’s dirty and boring: just a bunch of numbers and rows and tables. This kind of stuff doesn’t usually tell you much without a lot of very laborious prodding and exploration. Don’t believe me? Fine. Go find out for yourself. If you managed to get anything interesting out of that link then you have too much time on your hands.

The ONLY thing that civic data has going for it is that programmers tend to build cool hacks using it. I guess every once in a while you get a groundbreaking piece of journalism out of it too but I’ll ignore that for the sake of argument.

It’s Also Elitist

Here’s another problem: programmers have awesome, special tools to access data. These tools are called “Application Programming Interfaces” (also known as an APIs). An API is just a standard way for computers to ask each other for information.

A human version of this plays out every time you go to a restaurant and order from a menu. You look at the list of what you can ask for, you ask for what you want, and eventually you either get your food or you get impatient and start throwing your silverware at other patrons.

In my analogy the food is data and you and the chef are computers. The waiter is the API and the menu is the documentation. I guess the restaurant is the Internet and the restaurant’s manager is the NSA or something. The silverware don’t really fit in.

The point is that the COOL stuff happens because of these APIs. Too bad nobody real knows what the hell an API is or how they could possibly go about using it. Don’t believe me? Go find out for yourself. If the stuff on that page gave you access to data then you’re a nerd.

If nerds and people who have too much time on their hands are the only ones who can use government data then it won’t change the world. Plus, why should those people get to decide what is and isn’t important?

Humanizing Government Data

And so we come back to CivOmega. This is an attempt to give people with normal, human questions the ability to benefit from the data that so many have worked their asses off to expose. It makes it possible for a human to interact with an API in the same way they might interact with their waiter: by asking questions. Users can type in questions about the government and it attempts to provide answers.

It is built on a programming language called Python and the way it works is pretty simple. A programmer who understands an API can write some code that knows how to answer certain question patterns. For instance I made it possible to ask the question “What bills are about [X]” where X can be any phrase you want. If you ask that, CivOmega will talk to the appropriate APIs to get you the answer you want. Then it will tell you what it learned.

The beauty of this setup is that any other programmer can spend a few minutes teaching the system to answer new kinds of questions. For instance maybe someone knows about an environmental dataset and wants you to be able to ask questions about natural disasters (how many forest fires happened in California last year?). That person could easily unlock that resource.

If you’re a developer, go take a look at the repository and consider adding a module. If you are a master of NLP please get in touch with me so we can improve the way people ask questions. If you don’t know what either of those sentences meant, please just go check out the site.

He goes on to explain that he is a journalist writing about Tor. He also tells us that he bets that the CIA and the Italian Secret Service are going to have moles there. What he obviously meant to say was “I work for the CIA and I’ve been watching you now for quite some time.”

It’s possible that he didn’t actually work for the CIA. His name and photo checked out under the website he claimed to write for. It was probably just a one-time job. Even if this isn’t true, even if a network of government spies didn’t track my position across Europe just to meet us in a restaurant, his comment set the tone for my weekend in Florence.

Tor is serious business.

What the hell is Tor?

Did I mention Tor yet?

Tor is a program that makes you anonymous. This means that, for better or for worse, the big brothers, neighborhood hackers, and ad agencies of the world can’t tell what you are doing on the Internet without going through a lot of effort and expense.

Is that too abstract? Here are some illustrative statements. *Taps the microphone*

• A Tor user walks into a bar, the bartender says “who are you?”
• How many Tor users does it take to screw in a light bulb? Only a few, but you’ll never know who did it.
• I used Tor last night and now my wife says that she doesn’t even know who I am any more.

I’ll be here all night.

If you use Tor you become Spartacus. Tor takes everything you do, makes it look exactly like what everyone else is doing, and gets random computers on their network to do the talking for you. Ta-da! Now it is practically impossible to pin an action on you.

The Original Need

I bet you wouldn’t have guessed that this idea was invented by The U.S. Navy. You would have? Oh.

Put on your paper sailor hat and I’ll explain. Imagine you are the king of the Navy and you’re going to war with your fleet of a thousand brand new Navy cars (I don’t really know how the Navy works). Being king, you are in the most important car of all because you’re calling the shots. You don’t want the enemy to know which vehicle is yours. You also don’t want them to know who is receiving orders because that could give away your tactics.

“I know,” you say, “I’ll encrypt everything so that they can’t see the content. Then they won’t be able to tell that my broadcasts are more important than others.”

Unfortunately for you, the enemy has fancy technology. They can’t decrypt messages but they are able to track where everything comes from and where it is going. They can’t tell what you’re saying, but they have all they need.

After about 5 minutes you think you’re doing well. Half of the enemy cars are already on fire! Yours explodes. “How did they do that?” you say in the afterlife. “Easy,” responds god, “they were able to see that your car was sending out the most messages. They knew exactly where you were.” Then he slaps you with a piece of linguini and drifts away.

To prevent this from ever happening again the Navy decided to invent the concept of an “Onion Network” (not to be confused with The Onion Network). Now instead of having packets go directly from point A to point B, each one randomly hops around the fleet first. Because of encryption, the enemy can’t tell the difference between a new message and a “hop” message — they all look the same. It’s like running an invisible sprinkler in a thunderstorm.

Suddenly nobody but the sender and the recipient can figure out the end points of a message chain. Even the middle men (the ones doing the hops) don’t know the path. Each piece of the hop — each “layer” of the message — is encrypted with a different key, so the only thing a relay knows is who gave them the package and where it should go next.

Onions have layers too, that’s why this setup is called an Onion Network. Get it? It’s like Shrek!

Trolls use the Internet, Ogres use Tor. (Illustration by Anne Buckwalter)

What’s it Good For?

Tor has applications in the real world. You can buy drugs and guns, share illegal pictures, and hire assassins. Oh wait, I’m just describing Tor’s reputation (more on that later). Seriously, there are a lot of important situations where people have moral and compelling reasons to want anonymity.

Here are a few:

• Protecting witnesses and victims of domestic abuse. Anyone who wants to be able to access the internet without being discovered by a third party can use Tor to defend against their stalkers.
• If you don’t like being tracked by your government, Internet Service Providers, or search engines.
• Providing truly anonymous tips. There are times when people need or want to share information against the wishes of powerful and potentially dangerous forces (e.g. mafias, governments, or corporations).
• Safely bypassing censorship. If you live in Syria, China, or The United States of RIAA/MPAA, you might use Tor to access content from the outside world more safely.

These kinds of reasons explain why organizations with very good reputations, like the Knight Foundation, are devoting resources to Tor.

The Dark Web

What I’ve just described is a spin on the way people access normal information online. If you point Tor Browser to Google you will see the same old Google, it’s just that now Google doesn’t know who you are. That’s powerful enough, but there’s more: Tor also lets you see hidden content on the Internet.

Using Tor is like entering a cheat code into real life and playing the lost levels. It is the digital equivalent of platform 9 and 3/4. This secret section of the Internet is possible because Tor users can serve content anonymously too.

If you don’t know much about how the Internet works, believe me when I say that if a web site’s location is hidden it becomes essentially impossible to access. It would be like trying to visit someone’s house without knowing anything about where they live — not even the country. Tor gives you a blindfold and leads you there. You still don’t know where the house is, but at least you can visit.

Anonymous sites are accessed through something called an “onion address,” which is made up of a series of random letters and numbers. For instance, this is a “clean” version of Tor’s wikipedia: 3suaolltfj2xjksb.onion. Feel free to try clicking the link, it won’t work (Unless, of course, you are using the Tor browser).

Note: even if that link worked you wouldn’t see any terrible images. However, you need to use your brain before you start actually clicking around if you don’t want to get really upset.

That random looking string is used to find the server within the Tor network. Because the addresses don’t point to a real address on the Internet, there is no way to fully access this content without Tor. There are services you can use to get there without using Tor, but you lose all benefits of anonymity and content is often censored.

Onion addresses are the most fascinating part of Tor, albeit the most potentially disturbing. Rest assured that they don’t all lead to child porn, guns, and drugs. For example there is a secret version of Twitter, a bunch of blogs, a search engine, and an email service. There is even a secret version of 4chan (called Torchan), which I won’t link to because that one does lead to child porn and drugs.

These types of content networks—ones that are served on top of the normal web so that you need special programs to reach them—are known as the Dark Web. Not necessarily because the content is darker (it is), but because it is hidden from view and can’t really be searched and scraped as reliably.

Implications of The Dark Web

Most uses for Tor become more potent with onion addresses. Anonymous servers are just as protected from higher powers as anonymous users. If Amazon suddenly started selling illegal drugs they would get in trouble. If a Tor marketplace started selling illegal drugs, the law would have to figure out a way to find them first.

This power applies to legitimate uses as well. If a government official wanted to contact The Boston Globe with a corruption leak, he or she could use Tor to create a gmail account anonymously. The government could then subpoena Google, and Google might be willing to give away the information they have. They won’t know much, but now things like account access patterns and full email logs would be fair game.

If the official had used Tormail then even Google wouldn’t know what happened. The government would have no course of action because there would be no service provider to ask. Every journalist in the world should be able to agree that there is no good reason for a watchdog to trust the organizations they are watching. Why should you trust in corporations and governments to keep sources safe?

Tor has a reputation because it has a lot of criminal content, but the social good that it supports is just so important (criminals will always be criminals). I’m working on a game called Torwolf to simulate a few situations where Tor would be effective (if you have played Werewolf or Mafia, you can start to imagine what the game will be like). In the mean time, read up on Tor if you’re curious. Better yet, go try it out.

IMPORTANT EDIT: while Tor is much better than nothing, it is neither foolproof nor perfect. If maintaining anonymity could be a matter of life / death / imprisonment, then you need to know more than what I could fit into the scope of this overview. This paper is a good starting point, but seriously, spend some time researching on your own.

ATTN-SPAN Intro.

Part 2: Prototype and Development Plan

The Good News: I created a proof of concept prototype of the ATTN-SPAN platform powered by the Metavid project.

The Bad News: Metavid is having a lot of stability issues right now, so you probably won’t be able to use my prototype. I made a screen cast just in case.

Relying on a 3rd party for the most important aspect of an application is a major risk; one that I must mitigate. This brings me to my first batch of design work: the content scraper.

Scraping, Slicing, and Scrubbing C-SPAN

How do you get from a TV channel to a rich video archive and how do you get there automatically? The goal is to convert C-SPAN into a series of overlapping video segments that are identified in terms of state, politician, topic, party, action, and legislative item. Some of this is straightforward and some of it might be impossible, but here’s an overview of the planned nuts and bolts:

1. DirecTV offers TV content in a format that is easy to record digitally and VLC is a free tool that can do that recording. Combine the two and we can download C-SPAN streams into individual files that are primed and ready for analysis.
2. Once a video file is in our clutches we can use VLC once again to separate out the video from the Closed Captioning transcript.
3. Now we have a transcript and a raw video file. Next we register all of this information (in a database) so that we can look it all up later, and then convert the video file in to streaming-friendly formats and store it alongside the original recording.
4. C-SPAN consistently shows a graphic on the bottom of the screen that says who is talking, their state, their party, and what is being debated. By using a technique called Optical Character Recognition (OCR) we can pull this text out of the video image. Once pulled, we can add that to our database so that we can access all of this information for any moment in the video.
5. At this point we have most of the information we need, but there is still room for fine tuning. We can use audio levels and the closed captioning transcripts to try to identify moments of inactivity, normal dialogue, and heated dialogue.

These steps are enough to split up and categorize C-SPAN footage into an organized video database, but there are still more ways to flag special moments in the footage. For example, we may want to identify changes in speaker emotion in order to give our algorithms the ability to craft more engaging episodes. This is possible through the work of Affective Computing group at the MIT Media Lab, a group which has developed several tools that perform emotional analysis using facial recognition.

We may also want to identify specific legislative action (e.g. “calling a vote”). This could be accomplished by looking for key words in the transcript (e.g. “call a vote”) and possibly through common patterns in the audio signal (maybe there are identifiable sounds, such as a gavel hitting the table). Both of these concepts require additional research.

Creating a Profile and Constructing an Episode

If video events are the building blocks then viewer interests are the glue. The creation of a personalized episode requires two things: A user account, and a context. The user account provides general information like where you live, what issues you have identified as important, and (if you are willing to connect with Twitter or Facebook) what issues your circles have been discussing lately.

The context comes from time and cyberspace. Every night, after congress closes their gates, your profile is used to create a short, rich video experience designed to contain as much relevant content from that day as possible. At this point you might get an email begging you to watch, or maybe you log in on your own because you are addicted to badges and points and you want as much ATTN-SPAN karma as you can get.

There is another way to access this content though, and that is through the web sites you visit anyway. Imagine if you could read an article about the National Debt on the New York Times (or in a chain email) and actually see quotes from your own senators in the report. What if you could supplement the national report with a video widget that lets you browse what your house members had to say when they controlled the floor during the debt debates.

From a technical perspective this isn’t that far fetched. Truth Goggles, one of my other projects, is a bookmarklet that will analyze the web page you are viewing, fact check it, and rewrite the content to highlight truths and lies. This impossible feat is fairly similar to what I’m proposing here.

Adding Rich Information

Once an episode is pieced together we can look up the information surrounding the video to know who is talking and what they are talking about. What else can be added and how do we get it? Existing APIs offer some good options:

• Contact Information – Thanks to the Sunlight Labs Congress API it is possible to get the contact information for any member of congress on the fly. Thanks to VOIP services it is possible to create web-based hooks to call those people with the click of a button.
• Campaign Contributions – The New York Times offers a Campaign Finance API which can help you understand where the person on screen gets his or her money.
• Voting Records – The New York Times also offers a Congress API that will make it possible to know vote outcomes from related bills as well as information about the active speaker’s voting records.
• Truth and Lie Identification – My Truth Goggles project can be easily adapted to work with snippets from video transcripts. This will allow ATTN-SPAN to take advantage of fact checking services like PolitiFact and NewsTrust.

This is a good start, but I would also like to show links to related news coverage and create socially driven events based on community sentiment (for instance to track moments that caused people to get upset or happy). This won’t come for free, but it should be accessible given the right interface design.

Part 3: A Note to the Newsies

So that’s the idea and the plan. What’s the value?

It seems plausible that ATTN-SPAN, a system that analyzes primary source footage and pulls out any content that is related to a particular beat could be useful as a reporters tool, but what about your subscribers? ATTN-SPAN can augment an individual article so that it hits everybody close to home. Suddenly one article becomes as effective as two dozen. Moving past text, for larger organizations with a significant amount video footage ATTN-SPAN can be tweaked to use your programming instead of (or in addition to) C-SPAN.

At this point I have to warn you that this is not the first nor will it be the last project to work with C-SPAN. A 2003 demo out of the Media Lab used C-SPAN as one of several sources of information in a platform aimed to provide citizens with Total Government Awareness. Metavid, the platform I used in my initial prototype, already makes C-SPAN more accessible by enabling searches and filters. The list surely goes on.

So why is this a more powerful project? Well, the real goal of ATTN-SPAN isn’t to get more people watching C-SPAN. In fact I tricked you: this project isn’t about government awareness at all. It’s actually part of an effort to make indisputable fact (“blunt reality” and “primary source footage”) a more prominent part of the media experience without requiring additional effort from the audience. Newsrooms do an amazing job of reporting events and providing insight, but for deeper stories there simply isn’t enough time or money to cover everybody’s niche without going beyond the average person’s attention span.

Thus ends my pitch.

The code for both prototypes mentioned in this post can be found on github: ATTN-SPAN and Truth Goggles. Please forgive any dirty hacks. I would be thrilled if anybody wants to offer suggestions or even collaborate. On that note, please get in touch on Twitter @slifty.