The Programmable City

The federal government has revamped its data portal in an effort to encourage the development of innovative apps that make use of the publicly accessible federal government data on topics ranging from housing to air quality.

"Open Data is Canada's new natural resource," said Treasury Board President Tony Clement in a statement, as the government officially relaunched its data.gc.ca portal Tuesday afternoon in Toronto. ...

A few years ago, New York City set up a “geek squad” tasked with analyzing vast amounts of data to improve the way the city runs. 

It used information from utility companies to identify which buildings lost power after a storm; data from sprinkler systems to find those most vulnerable to fire; geospatial data from sewers to figure out which restaurants were clogging pipes with illegally dumped grease.

New York is at the forefront of the “Big Data Revolution,” which could usher in one of the greatest societal changes of our lifetime. Vast amounts of digital information are being put to creative, problem-solving use. Google can help predict flu outbreaks by mining data about where people are looking up symptoms. Real-time information from commuters’ smart phones can help urban planners design better transportation systems. The number of UPS packages can detect an upswing in the economy.

Exclusive: phones were monitored and fake internet cafes set up to gather information from allies in London in 2009 .. Foreign politicians and officials who took part in two G20 summit meetings in London in 2009 had their computers monitored and their phone calls intercepted on the instructions of their British government hosts, according to documents seen by the Guardian. Some delegates were tricked into using internet cafes which had been set up by British intelligence agencies to read their email traffic. ...

Glenn Greenwald: Addressing many of the issues arising from last week's NSA stories

This is more or less a repeat of my course at ASU earlier this year and although I peppered this with a lot more comment and discussion in terms of the actual presentations to graduates at Tel-Aviv University (TAU), most of the lectures are the same.

The meaning of the N.S.A.’s scandalous data-gathering programs—and of the Big Data mindset they represent. 

The idea of a government monitoring and sifting through virtually all electronic communications was once a dystopian, sci-fi fantasy. Within the past two decades, it became a technical possibility. In the past several years, it became a suspicion. Now, it may be at least close to the truth ....

How the U.S. Uses Technology to Mine More Data More Quickly.

A revolution in software technology has transformed the National Security Agency by providing automated and instantaneous analysis of volumes of digital information.

Secret PRISM program gives intelligence agency access to web and email of Google, Facebook and Apple customers

Policy paper

Published 18 June 2013

See climate change, deforestation and urban sprawl unfold as Earth evolves over 30 years. 

Type in the location of a place to see how the land-use has changed over time.

It's hard to escape the trap of Big Data these days. In the current issue of The Week, for example, the cover headline reads: "Trapped in Big Data: The debate over surveillance, privacy and national security." The National Security Agency (NSA) revelations have brought into relief just how much personal data is routinely mined not only by federal, state and local governments, but also by corporations. The seemingly limitless records of our e-mails, phone calls, purchases, travels and current locations are used to pitch products and services. If we are to believe the NSA, these data have also been used to foil terrorist plots.

The NSA revelations have unleashed a spirited political debate about the costs and benefits of Big Data collection and analysis. NSA defenders, including President Obama, suggest that the state must tread "lightly" on our rights to privacy in order to keep us safe. Critics suggest that government thirst for Big Data is the first step toward an authoritarian state in America.

Beyond the important debates about privacy and the constitutionality of NSA snooping, there is one issue that has received less attention -- the usefulness of Big Data. In a recent Ethnography Matters blog, cultural sociologist Tricia Wang, who writes about the human dimensions of technology, describes the use and misuse of Big Data ...

Instead of focusing upon the concept of big data, organisations should concentrate on the intelligence data can offer. 

I'm no fan of big data. In fact, you could say I'm a big data sceptic. That may surprise some people given my current roles, but I'm not alone in having this view.

Writing recently in trade journal Data IQ, Terry Hunt – a modern godfather of data and one of the men behind Tesco Clubcard – had this to say about it: "I don't think you have to be a privacy militant to find this hyper-excitement disturbing, even distasteful … Big in business doesn't really have much of a reputational record. It generally signifies excesses of corporate hubris and the irresponsible exertion of power."

In 2008, a ruling by a surveillance court said to be against Yahoo discouraged technology firms from fighting data requests from the government.

In a secret court in Washington, Yahoo’s top lawyers made their case. The government had sought help in spying on certain foreign users, without a warrant, and Yahoo had refused, saying the broad requests were unconstitutional.

The judges disagreed. That left Yahoo two choices: Hand over the data or break the law....

The NSA's new $1.7bn facility in the heart of Mormon country has the potential to snoop on US citizens for decades to come

Most of the discussion around the revelations about the data collection activities of the NSA has been about the threat to our civil rights and the potential damage abroad to U.S. political and business interests. Relatively little has been said, however, about the wisdom of collecting all phone call records and lots of other data in the fight against terrorism or other threats to the United States. Faith in the power (especially the predictive power) of more data is of course a central tenet of the religion of big data and it looks like the NSA has been a willing convert. But not everybody agrees it’s the most effective course of action.

The top-secret Boundless Informant tool details and maps by country the voluminous amount of information it collects from computer and telephone networks

In Silicon Valley, dismay at revelations about government monitoring has led some to call for disclosure of companies’ involvement.

Europe has a travel problem. High-speed railways, budget airlines, and a single currency have made border crossing easier than ever before. For Americans accustomed to long flights and few railways, the European model can seem like a breath of fresh air. Actually booking a trip, on the other hand, can be surprisingly cumbersome.

Despite (and, in some ways, because of) Europe's tight connectivity, there isn't an easy way for travelers to compare and combine different modes of travel. A 2012 EU regulation on data sharing has certainly made railway booking more seamless, but comparing train rates to plane fares — or even combining train, plane, and public transit into one itinerary — has thus far remained a difficult and disjointed task. For web users, this typically involves hidden costs, repeated Google searches, and multiple browser tabs.

Two startups in Berlin are hoping to change that....

Those of us who believe that democratic governments have a central role to play in multi-stakeholder cyberspace governance have received in the past few weeks a bracing reminder of both the hazards of this ideal in practice and the importance of broad-based civil society mobilization. Democratic states, while not sufficient for effective internet governance, are necessary parties because no other institutions have yet emerged that combine as well as they do the inclusivity, legitimacy and resources to help manage the internet effectively in the broad public interest. However, when such states violate the democratic principles that they espouse and are built on, they seriously undermine their legitimacy as well as the viability of the internet governance project overall. ...