Data Anxiety

Tempus fugit

“I yearn for the days when finances will be done electronically by password, so there’ll be no incentive for street robbery.”

WWWTXT (1980–94): ☯83FEB

Letter from an FCC commissioner to jounalists, fini

I am not the country’s only concerned citizen. During my 10 years at the FCC, I took part in scores of town hall meetings and community forums all across America to tell people what I saw happening and to learn more about their personal experiences…In some places these meetings would attract attention; in others they would go unnoticed.

Anomalous press coverage disparities

If a city or town’s media was under consolidated control—with a large, far-away company owning the major broadcast and newspaper outlets…the coverage would be somewhere between slim and none…Most of these town hall meetings went on for hours…with hundreds of citizens…there would be an open microphone, so everyone could speak; a U.S. senator or congressperson would often be on the program, sometimes even hosting it.

Following the hearings, I would rush…to flip on the TV looking for coverage. Occasionally there would be a mention; more often it was silence in Big Media Land. But if I was visiting a town where independent media still existed and locally employed journalists were on the beat, there would be advance notice that a meeting was going to happen; there was often live TV coverage; and the event would be reported in detail…on the front page of the local paper.

Media ownership

I am not a conspiracy theorist by nature. While still a Commissioner, I went one day to visit the editorial page editor of a major newspaper. I had noticed an editorial chastising the excesses of big oil companies. I told the editor I was there to urge the paper to run a similar critique about the excesses of big media. The response I got was a negative shake of the head…the editor had complete freedom to cover any issue—except one. That issue was media ownership. I nearly fell through the floor at this stark admission.

Legislators will find the doors to corrective national legislation already barred by many states—or, more accurately, barred by ALEC. But when private-sector business plans diminish the infrastructure upon which we rely for our news and information, it is a story that needs to be told. When infotainment supplants hard news, shouted opinion displaces fact, and whole swathes of this land of diversity go uncovered, it is a story that needs to be told. When government policy aids and abets the transformation, it is a story that needs to be told.

More after the jump »

Letter from an FCC commissioner, continued

Frankly, I was expecting change for the better after the 2008 presidential election and the coming of a Democratic majority to the FCC. After all, Senator Barack Obama had expressed his opposition to the pace of media industry consolidation and had affirmed his intention that public interest considerations should drive FCC decision-making…if only performance had lived up to promise.

So it happened that in the very first year of the new administration, cable giant Comcast came knocking at the Commission door seeking approval to purchase majority control of the already huge and powerful NBC-Universal media complex. The proposal was daunting in both its breadth and depth. The merged entity would include media and telecom; broadcast and broadband; distribution and content; traditional and new media. I cast the lone dissenting vote. Allowing one mega-corporation to wield gateway and content power over TV, cable and broadband in markets around the country, I said, dooms consumer-friendly competition, curbs the diversity of voices that a diverse nation must hear, and confers power that no one company should wield.

Comcast’s power grab was not the end of it—not even close…And TV stations are hotter-than-ever commodities in the wake of the Supreme Court’s infamous Citizens United decision that freed up billions of super PAC and dark money dollars to purchase TV advertising and fattened the coffers of TV outlets.

“But wait,” you may be thinking. “Won’t the new media of the Internet cure the downsides of consolidation? Too bad about the shrinkage of news and journalism in newspapers, radio and TV, but they were headed into the ash-can of history anyways.” 

More after the jump »

A Letter to Journalists from a Former FCC Commissioner »

'“What a totally awesome job this is going to be,” I thought as I sat down at my desk. “I’ll be dealing with edge-of-the-envelope issues that are transforming the planet; I’ll meet the visionaries and innovators who are making it happen…And broadband, the savants told us, would bring the revolutionary wonders of the Internet to every home and hamlet. The new media of the Internet would complement the traditional media of newspapers, radio, TV and cable, ushering in a golden age of communications. News and information journalism would flourish, and America’s civic dialogue…would be nourished as never before.

My expectations were short-lived. It turned out that the FCC I was joining had an altogether different agenda. One of the first requests that I received from my new Chairman was to support a merger between two media companies…my waking hours would be spent listening to big media types tell me how their latest proposal to gobble up more properties would translate into enormous “efficiencies” and “economies of scale” to produce more and better news…everywhere I looked, I saw newsrooms being shuttered or drastically downsized, reporters getting the axe, and investigative journalism clinging to the slenderest of threads.’

In order to maximize profits and to finance their costly media transactions, the merged companies were under the financial gun to cut costs. The first place they looked to cut was, and is, the newsroom. Instead of expanding news and creating opportunities for journalists like you, they cut the muscle out of deep-dive reporting and disinvested in you and your future.

Then another light bulb went on: The public policy the FCC was making was a major force refashioning our media ecosystem. It wasn’t just the excesses of a Wall Street bazaar run wild. It wasn’t just private sector business plans wreaking all this havoc. It was proactive government policy-making. Government—my own agency—was the willing, indeed eager, accomplice in diminishing our news and disfiguring our journalism. The regulatory agency where I worked was actually making things worse. 

You need to know this story. 

To be continued…

About the Hype Cycle »

nosql:

via Steve Francia on the NoSQL hype cycle[1]:

Technology trigger

As Facebook, Twitter and others saw them as solutions to their massive scalability problem (and because they were using relational databases for things they shouldn’t have) people began to see NoSQL as a panacea.

Peak of inflated expectations

Unfortunately knowing when to use the technology requires actual experience with it, which never seems to catch up to the hype engine quickly enough, so the technology transforms into a panacea, i.e. better at everything and ready to displace all predecessors.

Trough of Disillusionment

Current technologies exist because they do something well. When a new technology emerges it will likely be good at a different thing, meaning the two will co-exist.

hype cycle

Generalizable! This is a much-loved favorite.

Welcome to Tumblr, ODNI General Counsel Robert Litt! We are delighted that you have decided to join us. Your Tumblr HTML and CSS choices are very appropriate, although I would personally prefer a heavier typeface or more contrast, for the sake of readability.
On the off chance that you or the minions notice, I have a few suggestions. I have taken the liberty of excerpting from your speech, then sharing my thoughts. There’s a lot to be said for responsive reading, in the style of the Hebrew Union prayer book, like I used back home in New Mexico. Okay, here we go!
icontherecord:

Remarks of ODNI General Counsel Robert Litt at American University Washington College of Law Freedom of Information Day Celebration, March 17, 2004
President Obama, who has noted our “profound national commitment to ensuring an open government,” called upon the entire government to comply with both the letter and the spirit of the Freedom of Information Act.  Today I would like to talk to you about the challenges of reconciling that commitment with the secrecy necessary to conduct effective intelligence operations in defense of our national interests… We’ve set up a tumblr website – not that I really understand what a tumblr is – on which we post for the public what we have released. The President has directed that this transparency continue, and it will. This is something that I am personally involved in and personally committed to…

Stop reiterating your commitment to transparency. You are setting yourself up for failure and diminished public confidence. “Open” has become the subject of derision. 
There are many exemplary sources of government data, in human and machine digestible format. The Department of Defense candidly offers two decades of earmark history. The Federal Reserve maintains a host of numeric, geographical and historical data portals for public perusal and download: FRED, GeoFRED, ALFRED.
Open data market: Saturated with excess supply
I have an account on Socrata, the Gov 2.0 open data platform. Socrata is a CIA In-Q-Tel funded venture. It’s great, works really well! There’s fascinating data there, for our citizen journalists clamoring for transparency:
the data set of presidential appointments, linkable to historical political campaign contributions by party, candidate, donor and filing state. 
outlier searches, e.g. why does such-and-such bureau only employ attorneys, with average compensation of $185,000 per annum? What is all this foreign exchange activity going on at Dept. of the REDACTED? 
Wow, the annual budget for the National Endowment for the Arts was cut by nearly 40% in 1994, and again in 2014!
However, hardly a soul uses it. Data usage statistics are available. They are grim. A fine young data scientist, Thomas Levine, does periodic reports, and his findings confirm this. There are only 38 active citizen users of Socrata open data (Thomas’s post even includes bio’s with photographs of the top 3 most active users). All others are U.S. federal government and state agency staff, Socrata employees or bots. The bots all belong to recovery.gov or data.gov, with one exception, Kenya Open Data Bot.
You are TOO transparent!
Well, you’re transparent about the wrong things.

In particular, I think that greater transparency about our processes could help us cope with one of the principal failings I see in the current public discussion, which is the failure to distinguish among what the Intelligence Community can do technically, what it can do legally and what it actually does do.

You mentioned that the recent flood of FOIA requests were costing the NSA a small fortune. Maybe you could get some of the funding for Socrata reallocated to your FOIA group? It is atavistic, but it seems to be what the public truly wants.
Don’t be overwhelmed by technology
Consult with well-informed individuals, the sort that have training and experience. None of Condoleeza Rice’s and Hillary Clinton’s young Turks such as Jared Cohen or the insufferable ioerror. And stop listening to Harvard about the wonders of privatization! They’ll tell you to give everything to contractors like Booz Allen, USIS and Amazon.com.
Encryption

There have been press reports that NSA has engaged in a concerted effort to break encryption. Without commenting on the accuracy of any particular story, isn’t that exactly what intelligence agencies have historically done and what they are supposed to do?

Yes.

We know that our enemies use encryption and other techniques precisely to avoid surveillance; NSA’s job is to figure out how to break those techniques…But saying that NSA can break encryption is different from saying that they routinely spy on encrypted conversations of ordinary Americans or foreigners. They don’t…

The encryption breaking is problematic because it was in large part developed under the aegis of NIST. NIST is part of the U.S. Department of Commerce. That encryption is used globally, in government and commercial applications. Trust has been undermined. You must address this. It is vital.

And this leads to the other question I want to raise about transparency, which is, Who is responsible for deciding what should be secret and what should be public? In my opinion, we cannot survive as a nation if we let those decisions be made by individuals who do not have adequate insight into the implications of their actions…etcetera

You have many wise, compassionate and yes, friendly, experts available e.g. the Naval War College, FBI, Carnegie Mellon and Cornell Universities, IBM, RSA, Verisign and MITRE. Many of the recent Department of Homeland Security hires have distinctively political appointee-type curriculum vitae. I’d steer clear; same for technocrats. Technocrats are rarely technical, most are just good talkers!

Welcome to Tumblr, ODNI General Counsel Robert Litt! We are delighted that you have decided to join us. Your Tumblr HTML and CSS choices are very appropriate, although I would personally prefer a heavier typeface or more contrast, for the sake of readability.

On the off chance that you or the minions notice, I have a few suggestions. I have taken the liberty of excerpting from your speech, then sharing my thoughts. There’s a lot to be said for responsive reading, in the style of the Hebrew Union prayer book, like I used back home in New Mexico. Okay, here we go!

icontherecord:

Remarks of ODNI General Counsel Robert Litt at American University Washington College of Law Freedom of Information Day Celebration, March 17, 2004

President Obama, who has noted our “profound national commitment to ensuring an open government,” called upon the entire government to comply with both the letter and the spirit of the Freedom of Information Act.  Today I would like to talk to you about the challenges of reconciling that commitment with the secrecy necessary to conduct effective intelligence operations in defense of our national interests… We’ve set up a tumblr website – not that I really understand what a tumblr is – on which we post for the public what we have released. The President has directed that this transparency continue, and it will. This is something that I am personally involved in and personally committed to…

Stop reiterating your commitment to transparency. You are setting yourself up for failure and diminished public confidence. “Open” has become the subject of derision.

There are many exemplary sources of government data, in human and machine digestible format. The Department of Defense candidly offers two decades of earmark history. The Federal Reserve maintains a host of numeric, geographical and historical data portals for public perusal and download: FRED, GeoFRED, ALFRED.

Open data market: Saturated with excess supply

I have an account on Socrata, the Gov 2.0 open data platform. Socrata is a CIA In-Q-Tel funded venture. It’s great, works really well! There’s fascinating data there, for our citizen journalists clamoring for transparency:

  • the data set of presidential appointments, linkable to historical political campaign contributions by party, candidate, donor and filing state.
  • outlier searches, e.g. why does such-and-such bureau only employ attorneys, with average compensation of $185,000 per annum? What is all this foreign exchange activity going on at Dept. of the REDACTED?
  • Wow, the annual budget for the National Endowment for the Arts was cut by nearly 40% in 1994, and again in 2014!

However, hardly a soul uses it. Data usage statistics are available. They are grim. A fine young data scientist, Thomas Levine, does periodic reports, and his findings confirm this. There are only 38 active citizen users of Socrata open data (Thomas’s post even includes bio’s with photographs of the top 3 most active users). All others are U.S. federal government and state agency staff, Socrata employees or bots. The bots all belong to recovery.gov or data.gov, with one exception, Kenya Open Data Bot.

You are TOO transparent!

Well, you’re transparent about the wrong things.

In particular, I think that greater transparency about our processes could help us cope with one of the principal failings I see in the current public discussion, which is the failure to distinguish among what the Intelligence Community can do technically, what it can do legally and what it actually does do.

You mentioned that the recent flood of FOIA requests were costing the NSA a small fortune. Maybe you could get some of the funding for Socrata reallocated to your FOIA group? It is atavistic, but it seems to be what the public truly wants.

Don’t be overwhelmed by technology

Consult with well-informed individuals, the sort that have training and experience. None of Condoleeza Rice’s and Hillary Clinton’s young Turks such as Jared Cohen or the insufferable ioerror. And stop listening to Harvard about the wonders of privatization! They’ll tell you to give everything to contractors like Booz Allen, USIS and Amazon.com.

Encryption

There have been press reports that NSA has engaged in a concerted effort to break encryption. Without commenting on the accuracy of any particular story, isn’t that exactly what intelligence agencies have historically done and what they are supposed to do?

Yes.

We know that our enemies use encryption and other techniques precisely to avoid surveillance; NSA’s job is to figure out how to break those techniques…But saying that NSA can break encryption is different from saying that they routinely spy on encrypted conversations of ordinary Americans or foreigners. They don’t…

The encryption breaking is problematic because it was in large part developed under the aegis of NIST. NIST is part of the U.S. Department of Commerce. That encryption is used globally, in government and commercial applications. Trust has been undermined. You must address this. It is vital.

And this leads to the other question I want to raise about transparency, which is, Who is responsible for deciding what should be secret and what should be public? In my opinion, we cannot survive as a nation if we let those decisions be made by individuals who do not have adequate insight into the implications of their actions…etcetera

You have many wise, compassionate and yes, friendly, experts available e.g. the Naval War College, FBI, Carnegie Mellon and Cornell Universities, IBM, RSA, Verisign and MITRE. Many of the recent Department of Homeland Security hires have distinctively political appointee-type curriculum vitae. I’d steer clear; same for technocrats. Technocrats are rarely technical, most are just good talkers!