Public Datasets -...
Follow
Find tag "opendata"
8.5K views | +0 today
Public Datasets - Open Data -
Your new post is loading...
Your new post is loading...
Scooped by luiy
Scoop.it!

The #datascience ecosystem, part 3: Data applications | #openSource #tools

The #datascience ecosystem, part 3: Data applications | #openSource #tools | Public Datasets - Open Data - | Scoop.it
The third part in a series on the data science ecosystem looks at the applications that turn data into insights or models.
luiy's insight:
Open-source tools

Probably because this category has the most ongoing research, there is quite a rich collection of open-source modeling and insights tools. R is an essential tool for most data scientists and works both as a programming language and an interactive environment for exploring data. Octave is a free, open-source port of matlab that works very well. Julia is becoming increasingly popular for technical computing. Stanford has an NLP library that has tools for most standard language processing tasks. Scikit, a machine learning package for python, is becoming very powerful and has implementations of most standard modeling and machine learning algorithms.


In the end, data application tools are what make data scientists incredibly valuable to any organization. They're the exact thing that allows a data scientist to make powerful suggestions, uncover hidden trends and provide tangible value. But these tools simply don't work unless you have good data and unless you enrich, blend and clean that data.
more...
No comment yet.
Scooped by luiy
Scoop.it!

@DataFrance : Plateforme de visualisation de données ouvertes | #opendata #dataviz

@DataFrance : Plateforme de visualisation de données ouvertes | #opendata #dataviz | Public Datasets - Open Data - | Scoop.it
Éducation, économie, transports publics, etc. Visualisez facilement et partout en France plus de 50 jeux de données sur notre carte interactive.

 

Nous voulons faciliter l’accès à l'open data

Nous avons pour ambition de rendre accessible les données ouvertes au plus grand nombre via une interface simple de consultation. DataFrance s’inscrit dans le mouvement open data et veut contribuer à promouvoir une plus large ouverture des données dans l’intérêt de tout public.

luiy's insight:

Selection de données:

 

- Logement vacants à Paris

- Accesibilité aux médecins généralistes en Gironde

- Vols de voitures en France

- Lycées par taux de reussite au bac à Marseille

- Risque de séisme en France

- Musées de France à Lyon

more...
No comment yet.
Scooped by luiy
Scoop.it!

Crimes de guerre et décryptage de données : nouvelles révélations de Snowden | #NSA #opendata

Crimes de guerre et décryptage de données : nouvelles révélations de Snowden | #NSA #opendata | Public Datasets - Open Data - | Scoop.it
De nouvelles révélations d'Edward Snowden mettent en lumière les agissement des pays de l'Otan en Afghanistan et le décryptage de données par la NSA américaine.
luiy's insight:

(De Hambourg) C’était un passage attendu du 31c3, le 31ème Chaos Communication Congress qui se déroule en ce moment à Hambourg : dimanche soir, devant 3 500 personnes, la journaliste Laura Poitras et le hacker Jacob Appelbaum ont révélé des documents d’Edward Snowden jusque-là inconnus du public.

 

Simultanément, plusieurs articles contenant plusieurs dizaines de ces documents ont été publiés (en anglais et en allemand) sur le site de l’hebdomadaire allemand Der Spiegel.

 

Des révélations touchant à des domaines très variés, allant de la guerre en Afghanistan aux capacités qu’a la NSA de décrypter les données circulant sur le Web.

 

http://www.spiegel.de/media/media-35508.pdf

 

more...
No comment yet.
Scooped by luiy
Scoop.it!

The First Interactive Network and Graph Data #Repository with Interactive Graph Analytics and Visualization | #opendata #SNA

The First Interactive Network and Graph Data #Repository with Interactive Graph Analytics and Visualization | #opendata #SNA | Public Datasets - Open Data - | Scoop.it
The First Interactive Network Data Repository with Real-time Interactive Visualization and Analytics
luiy's insight:

Network Data Repository. Exploratory Analysis & Visualization.

 

A network and graph data repository containing hundreds of real-world networks and benchmark datasets. This large comprehensive collection of network graph data is useful for making significant research findings as well as benchmark data sets for machine learning and network science. All data sets are easily downloaded into a standard consistent format. We also have built a multi-level interactive graph analytics engine that allows for visualizing the structure of networks as well as many global graph statistics and local node level properties. 

more...
No comment yet.
Scooped by luiy
Scoop.it!

Europeana Labs: 30 million metadata records linking to millions of openly licensed media objects | #opendata #datasets

Europeana Labs: 30 million metadata records linking to millions of openly licensed media objects | #opendata #datasets | Public Datasets - Open Data - | Scoop.it
luiy's insight:

Data

 

Our database contains over 30 million metadata records linking to millions of openly licensed media objects - books, photos, art, artefacts, audio clips and more. We'll be featuring some of our very best content here.

 

Europeana Labs combines rights-cleared images, videos, audio and text files with technical expertise, tools, services and business knowledge.

more...
No comment yet.
Scooped by luiy
Scoop.it!

Seven Ways to Create a Storymap | #opendata #maps #ddj

Seven Ways to Create a Storymap | #opendata #maps #ddj | Public Datasets - Open Data - | Scoop.it
Evidence is Power
luiy's insight:

The above examples describe a wide range of geographical and geotemporal storytelling models, often based around quite simple data files containing information about individual events. Many of the tools make a strong use of image files as pat of the display. it may be interesting to complete a more detailed review that describes the exact data models used by each of the techniques, with a view to identifying a generic data model that could be used by each of the different models, or transformed into the distinct data representations supported by each of the separate tools.


- See more at: http://schoolofdata.org/2014/08/25/seven-ways-to-create-a-storymap/#sthash.tWi68hgm.dpuf

more...
No comment yet.
Scooped by luiy
Scoop.it!

The #OpenData Research network | #opengov

The #OpenData Research network | #opengov | Public Datasets - Open Data - | Scoop.it
luiy's insight:

The Open Data Research network.

 

Governments, civil society organisations and companies across the world are actively engaging with open data: publishing and using datasets to promote innovation, development and democratic change.

 

The Open Data Research network has been established to connect researchers from across the world working to explore the implementation and impact of open data initiatives. It is a joint project of IDRC and the Web Foundation, and is seeking to develop wider partnerships over the coming year.

 

The network currently hosts the 'Exploring the Emerging Impacts of Open Data in Development Countries (ODDC)' programme.

more...
No comment yet.
Scooped by luiy
Scoop.it!

Digital Ecologies Research Partnership | #openResearch #opendata

Digital Ecologies Research Partnership | #openResearch #opendata | Public Datasets - Open Data - | Scoop.it
The Digital Ecologies Research Partnership (DERP) is a joint initiative to promote academic inquiry into social dynamics of the web.
luiy's insight:

Launched in 2014, the Digital Ecologies Research Partnership (DERP) is a joint initiative by an alliance of community websites to promote open, publicly accessible, and ethical academic inquiry into the vibrant social dynamics of the web.

 

DERP seeks to solve two problems in the academic research space:

 

First, it is difficult for academic researchers to easily obtain data for their work beyond the confines of the largest social media platforms. DERP is a single point of contact for researchers to get in touch with relevant team members across a range of different community sites. We envision that this will lower the friction to investigating these sites in more depth, and broaden the scope of research happening within the academic community.

 

Second, it remains difficult to conduct good cross-platform analyses in academic research. By bringing a number community of sites together under a single cooperative effort, we intend to lower the friction of doing so, as well as better enable the sites themselves to coordinate with one another on supporting researchers.

 

DERP focuses on providing public data to academic researchers while facilitating an active online research community of Fellows. DERP will only support research that respects user privacy, responsibly uses data, and meets IRB approval. All research supported by DERP will be released openly and made publicly available. Partner platforms may also have additional guidelines and privacy commitments that apply to the research they support.

more...
No comment yet.
Scooped by luiy
Scoop.it!

#Opendata, crowdsourcing, and #sharing economy tech take on new roles in disasters

#Opendata, crowdsourcing, and #sharing economy tech take on new roles in disasters | Public Datasets - Open Data - | Scoop.it
Sharing economy services and crowdsourced tech can help aggregate and distribute aid, housing, energy, and transportation to disaster survivors. Airbnb is one company that's stepping up.
luiy's insight:

In a disaster, figuring out what technology you have still works, how long it will work, and who knows how to use it is precious information. The idea of "disaster tech" is a vocative of that notion: what works when nothing else does? Last week, at a forum convened by the White House to share commitments from the private sector and to demonstrate data-driven innovation to helping people and disasters, I was reminded of that need.

 

In a sign of the times, the categories in the program included the "sharing economy" and survivor support, crowdsourcing, open data, and public alerts. Behind the buzzwords, what matters most about any disaster technology is whether it works (from being interoperable with existing systems to being accessible to users), and whether it improves upon existing systems used by first responders and aid workers. If sharing economy services aggregate and distribute the demands of survivors to aid, excess housing, energy, or transport capacity, they can help people in need. These outcomes aren't theoretical.

more...
No comment yet.
Scooped by luiy
Scoop.it!

Mapping Data: A guide for making #geodata visualizations | #ddj #methods #tools

Mapping Data: A guide for making #geodata visualizations | #ddj #methods #tools | Public Datasets - Open Data - | Scoop.it
luiy's insight:

As an add-on to our presentation we produced two more things, that some of you out there mind find helpful too:

 

- Mappable Toolset: The number of tools to process data, make maps, interactive visualizations etc. is continuously growing. While we love new tools, this leads to a situation that makes it quite hard to keep an overview of which tools are good for a certain tasks, where to find them and how much they cost. To keep track of the tools we've used so far and as a guide for others we thus collected our toolset. Have a look at it here:English version, German version.


-  Mappable Cheat-Sheet: Making maps and other visualizations with a geospatial component is certainly not a trivial tasks. There are many pitfalls, take alone spatial reference systems as an example, that might completely mess up your visualization if you don't handle them correctly. We thus created a checklist for making geodata visualizations in (data-driven) journalism. You can find it here: English version, German version.

more...
No comment yet.
Scooped by luiy
Scoop.it!

#OpenCorporates : How #complex are corporate structures? | #Opendata #dataviz

#OpenCorporates : How #complex are corporate structures? | #Opendata #dataviz | Public Datasets - Open Data - | Scoop.it
luiy's insight:

How complex are international corporate structures?

If you want to understand how complex multinational companies are, consider this:

 

In Hong Kong, there's a company called Goldman Sachs Structured Products (Asia) Limited. It's controlled by another company called Goldman Sachs (Asia) Finance, registered in Mauritius.

 

That's controlled by a company in Hong Kong, which is controlled by a company in New York, which is controlled by a company in Delaware, and that company is controlled by another company in Delaware called GS Holdings (Delaware) L.L.C. 

 

Which itself is a subsidiary of the only Goldman you're likely to have heard of, The Goldman Sachs Group in New York City.

That's only one of hundreds of such chains. All told, Goldman Sachs consists of more than 4000 separate corporate entities all over the world, some of which are around ten layers of control below the New York HQ.

 

Of those companies approximately a third are registered in nations that might be described as tax havens.Indeed, in the world of Goldman Sachs, the Cayman Islands are bigger than South America, and Mauritius is bigger than Africa.

These are maps of the top five banking companies in the US, and are based on publicy available data from the Federal Reserve. Read more about our data on the link at the top left.

more...
No comment yet.
Scooped by luiy
Scoop.it!

Open #Development Toolkit | #opendata #tools #dataviz

Open #Development Toolkit | #opendata #tools #dataviz | Public Datasets - Open Data - | Scoop.it
Open Development Toolkit
luiy's insight:

I want to visualise aid data

I want to understand aid flows into my country / from my country / from a certain donor /globallyI want to find aid data

 

What is ‘open development’?


Open Development sits at the intersection of the ‘open’ movement, and international development. This could take the form of looking at how open data can affect decisions made within international development; open access to research materials; or opening up the ways we work, for example by being more inclusive, to name just a few examples. If you want to find out more about open development, join the Open Development mailing list, where a wide range of topics within Open Development are discussed. It’s open to everyone to join!

 

more...
No comment yet.
Rescooped by luiy from Open Data
Scoop.it!

Released: The #OpenData Era in #Health and Social Care

Released: The #OpenData Era in #Health and Social Care | Public Datasets - Open Data - | Scoop.it

Download the Full Report NHS England and The GovLab at New York University have jointly created a blueprint – The Open Data Era in Health and Social Care – to accelerate the use of open data in healthcare settings.


Via Yves Mulkers
luiy's insight:

The Value of Open Data: The blueprint suggests a framework to review the potential for open data in:

 

• Holding healthcare organizations and providers accountable for treatment outcomes.


• Enabling patients to make informed choices from among the healthcare options available to them.


• Improving the efficiency and cost-effectiveness of delivering healthcare.


• Improving treatment outcomes.


• Educating patients and their families and make healthcare institutions more responsive.


• Fueling new healthcare companies and innovation.

 

Need for Evidence: For all the recognition of open data’s potential, there is an urgent need for more research and actionable evidence to help guide investments and priorities. This blueprint represents a framework for a conversation on how NHS England can develop an evidence-based program to guide the investments and further research into the benefits of open data.

more...
No comment yet.
Scooped by luiy
Scoop.it!

The First #Ecological Land Units #Map of the World | #opendata #dataviz

The First #Ecological Land Units #Map of the World | #opendata #dataviz | Public Datasets - Open Data - | Scoop.it
luiy's insight:

This map as well as the data layers used to create it can be explored in a new story map that introduces ecological land units. The data is available in the form of services and can enrich any GIS effort.

 

The collaborative partnership between Esri and USGS resulted in a dynamic online map representing the world’s ecological diversity at unprecedented detail and authority. This work leveraged quantitative methods, geographic science, and big data produced by government agencies and the scientific community.  To create this map the data were processed in Esri’s ArcGIS cloud computing environment. This map provides new knowledge and understanding of geographic patterns and relationships by distinguishing the geography of the planets’ ecosystems.

- See more at: http://blogs.esri.com/esri/esri-insider/2014/12/09/the-first-detailed-ecological-land-unitsmap-in-the-world/#sthash.Fm99R5lg.dpuf

more...
No comment yet.
Scooped by luiy
Scoop.it!

Is Cape Town's new #opendata portal any good? | #dataviz

Is Cape Town's new #opendata portal any good? | #dataviz | Public Datasets - Open Data - | Scoop.it
Code4SA's Adi Eyal takes a look at Africa's first open data city.
luiy's insight:

 ... the City chose to build their portal in-house rather than using one of the existing platforms. The UK open data portal for instance uses an open source platform CKAN. A commercial offering called Socrata is also available if you feel like you need the security of using a closed source solution. The benefit of using an existing platform is that you’re benefiting from the wisdom of those who came before you. It is also often much cheaper than doing it on your own. Our own modest data portal uses Socrata. As a developer, for me the killer feature here is that every dataset comes packaged with a pre-built API and documentation. It also allows non-technical users to create graphs and maps without having to download the data and opening up the spreadsheets. The City’s website in contrast, is not much more than a collection of links.

more...
No comment yet.
Scooped by luiy
Scoop.it!

Project #BigData. Expanding on Project C to look at a different use case | #datascience #opendata

Project #BigData. Expanding on Project C to look at a different use case | #datascience #opendata | Public Datasets - Open Data - | Scoop.it
luiy's insight:

Project Big Data is an interactive tool which enables you to visualize and explore the funding patterns of over 600 companies in the Big Data ecosystem! It is based on the work I did for Project C (which you see and can read about here). The list of companies and their classification into categories is based on a dozen published sources and rough text analytics of the Crunchbase database. Crunchbase is a curated crowed sourced database of over 285k companies.

 

As for the data, there are 645 public & private companies in the data set. From Teradata and IBM to Actuate & Zoomdata. I began by harvesting data from Crunchbase by using their free API w/ Python. As of September, Crunchbase had 1250 funding events for 410 of the companies on my list. I've grouped these companies into 18 categories, allowing you to compare peers as well as trends across categories. Some of the categories are broken down further. For example, the tool allows you to differentiate between cloud-based and on premise solutions or SQL vs. NoSQL databases. I gathered additional data from a variety of sources. For example, LinkedIn was used to find the number of employees.

 

 

OPENACCESS Workbook: Project Big Data v1.0 

https://public.tableausoftware.com/download/workbooks/ProjectBigDatav1_0?format=html

 

more...
No comment yet.
Scooped by luiy
Scoop.it!

All the Open #Datasets from New York City Visualized in a Single View | #opendata #dataviz

All the Open #Datasets from New York City Visualized in a Single View | #opendata #dataviz | Public Datasets - Open Data - | Scoop.it
luiy's insight:

"Visualizing NYC's Open Data" [chriswhong.com] by self-proclaimed urbanist, map maker and data junkie Chris Wong provides a single view of the more than 1,100 open datasets made available by New York City.

 

The visualization of the "dataset of datasets" consists of a force-directed graph, of which the nodes are colored according to whether the according dataset is a table, chart, map, a file or a user-created view (colored blue).

 

The graph acts as an alternative portal to explore the available data, while demonstrating its scale and diversity.

 

more...
No comment yet.
Scooped by luiy
Scoop.it!

Data Repositories - Mother's Milk for Data Scientists | #datasets #opendata

Data Repositories - Mother's Milk for Data Scientists | #datasets #opendata | Public Datasets - Open Data - | Scoop.it
Mothers are life givers, giving the milk of life. While there are so very few analogies so apropos, data is often considered the Mother's Milk of Corporate Valuation. So, as a data scientist, we sh...
luiy's insight:

Here are a few repositories from KDnuggets that are worth taking a look at:

more...
No comment yet.
Scooped by luiy
Scoop.it!

#OpenData Barometer Data | #opengov #dataviz

#OpenData Barometer Data | #opengov #dataviz | Public Datasets - Open Data - | Scoop.it
luiy's insight:

The Open Data Barometer takes a multidimensional look at the current adoption level of open data policy and practice around the world. Three main categories are considered as part of the barometer:

 

- Readiness - identifies how far a country has in place the political, social and economic foundations for realising the potential benefits of open data. The Barometer covers the readiness of government, entrepreneurs and business, and citizen and civil society.

 

- Implementation - identifies the extent to which government has published a range of key datasets to support innovation, accountability and more improved social policy. The barometer covers 14 datasets split across three clusters to capture datasets commonly used for: securing government accountability; improving social policy; and enabling innovation and economic activity.

 

- Emerging impacts - identifies the extent to which open data has been seen to lead to positive political, social and environment, and economic change. The Barometer looks for political impacts – including transparency & accountability, and improved government efficiency and effectiveness; economic impacts – through supporting start-up entrepreneurs and existing businesses; and social impacts – including environmental impacts, and contributing to greater inclusion for marginalised groups in society.

 

These factors have been combined onto a Radar chart, this represents the countries barometer.

more...
No comment yet.
Scooped by luiy
Scoop.it!

Quantifying Memory: Mapping the #GDELT data in #R (and some Russian protests, too) | #opendata #datascience

Quantifying Memory: Mapping the #GDELT data in #R (and some Russian protests, too) | #opendata #datascience | Public Datasets - Open Data - | Scoop.it
luiy's insight:

The Guardian recently published an article linking to a database of 250 million events. Sounds too good to be true, but as I'm writing a PhD on recent Russian memory events, I was excited to try it out. I downloaded the data, generously made available by Kalev Leetaru of the University of Illinois, and got going. It's a large 650mb zip file (4.6gb uncompressed!), and this is apparently the abbreviated version. Consequently this early stage of the analysis was dominated by eager anticipation, as the Cambridge University internet did its thing.

 

Meanwhile I headed over to David Masad's writeup on getting started with GDELT in python

more...
No comment yet.
Scooped by luiy
Scoop.it!

import.io + Open Refine + Google Fusion Tables = Magic! | #SNA #ddj

import.io + Open Refine + Google Fusion Tables = Magic! | #SNA #ddj | Public Datasets - Open Data - | Scoop.it
luiy's insight:

The University of Ottawa Library holds an employee training week every year, giving colleagues the opportunity to share experiences, skills, and insights with one another. I jumped on this opportunity to showcase import.io as a means of creating datasets from website content. The tutorial I developed demonstrated how to create a dataset from the City of Ottawa’s open data catalogue. It’s a really simple example to get users familiar with the functionality of import.io, an easy way to scrape web content via a simple interface and without having to code. In this post I will also demo how to use Open Refine to clean the data captured by import.io and how to visualize it using Google Fusion Tables.

more...
No comment yet.
Rescooped by luiy from Data Visualization & Open data
Scoop.it!

10 Great Places to Find #Datasets for Infographics | #opendata

10 Great Places to Find #Datasets for Infographics | #opendata | Public Datasets - Open Data - | Scoop.it

“Creating an infographic is an excellent way to break down complex information and statistics into an easy-to-follow visual that is designed with your target audience in mind. Infographics have grown in popularity because they are easy to share and a simple tactic for promoting a business. Perhaps you’ve considered incorporating infographics into your content marketing…”


Via massimo facchinetti, Jesse Soininen
luiy's insight:

1. Government Websites
2. Google’s Public Data Directory 
3. Reddit.com 
4. Infochimp 
5. UNData 
6. Visual.ly
7. DataMarket
8. Number Of
9. Gallup Polls
10. Get the Data

more...
No comment yet.
Scooped by luiy
Scoop.it!

The Open Database Of The #Corporate World | #opendata #economy

The Open Database Of The #Corporate World | #opendata #economy | Public Datasets - Open Data - | Scoop.it
Free and Open Company Data on millions of companies and corporations in over 20 countries, including UK, Spain, US, ...
luiy's insight:

What is OpenCorporates?

 

OpenCorporates aims to do a straightforward (though big) thing: have a URL for every company in the world.

 

Is that all?

Well, no useful though that would be, we're also gradually importing government data relating to companies, and trying to match it to specific companies

 

Why do this?

Few parts of the corporate world are limited to a single country, and so the world needs a way of bringing the information together in a single place, and more than that, a place that's accessible to anyone, not just those who subscribe to proprietary datasets. See also the OpenCorporates Principles

 

There are quite a few countries you're missing

We've grown from 3 territories and a few million companies to over 75 jurisdictions and 55 million companies, and are working with the open data community to add more each week.

 

How can we get hold of the data?

We have a new API service, as well as our highly popular Google Refine reconciliation service (seedocumentation), and this allows access to the information as JSON or XML. If you need data in bulk, either for academic research work, for another cool open data project, or commercially, drop us an email atinfo@opencorporates.com.

more...
No comment yet.
Scooped by luiy
Scoop.it!

ckan - The #OpenSource data #portal software | #opendata #Opengob

ckan - The #OpenSource data #portal software | #opendata #Opengob | Public Datasets - Open Data - | Scoop.it
The open source data portal software
luiy's insight:

CKAN is open source, free software. This means that you can use it without any license fees, but more importantly, when you choose CKAN for your catalog you are also ensuring that you retain all rights to the data and metadata you enter, giving you freedom to move it elsewhere or manipulate it with your own tools without restriction.


There are lots of different open source licenses (you can find them at http://opensource.org) – CKAN is licensed specifically under the terms of the Affero GNU GPL v3.0. One of the strengths of the open source model is in the communities that form around free software products. The CKAN community is no different, and is arguably one of the strongest open data communities in the world. Together, the CKAN community has a wealth of knowledge and expertise that other people using the CKAN software can draw on. The Open Knowledge Foundation draw on and contribute to this rich resource to help us drive CKAN product development. - See more at: http://ckan.org/developers/about-ckan/#sthash.nY6V2GU7.dpuf

 

- See more at: http://ckan.org/#sthash.LMq0zg2B.dpuf

more...
No comment yet.
Scooped by luiy
Scoop.it!

CitySDK: #toolkit for the development of digital services within cities | #opendata #smartcities

CitySDK: #toolkit for the development of digital services within cities | #opendata #smartcities | Public Datasets - Open Data - | Scoop.it
luiy's insight:

CitySDK is creating a toolkit for the development of digital services within cities. The toolkit comprises of open and interoperable digital service interfaces as well as processes, guidelines and usability standards. CitySDK enables a more efficient utilisation of the expertise and know-how of developer communities to be applied in city service development.

 

Apps and tools for CitySDK are developed in cooperation with the Code for Europe fellows (see www.codeforeurope.net)

more...
No comment yet.