Public Datasets - Open Data -
9.9K views | +0 today
Public Datasets - Open Data -
Your new post is loading...
Your new post is loading...
Scooped by luiy!

Twitter Data Mining Round Up | #python #ddj #openaccess

Twitter Data Mining Round Up | #python #ddj #openaccess | Public Datasets - Open Data - |
luiy's insight:

Since the release of Mining the Social Web, 2E in late October of last year, I have mostly focused on creating supplemental content that focused on Twitter data. This seemed like a natural starting point given that the first chapter of the book is a gentle introduction to data mining with Twitter’s API coupled with the inherent openness of accessing and analyzing Twitter data (in comparison to other data sources that are a little more restrictive.) Twitter’s IPO late last year also focused the spotlight a bit on Twitter, which provided some good opportunities to opine on Twitter’s underlying data model that can be interpreted as an interest graph.

No comment yet.
Rescooped by luiy from Data is big!

Mining of Massive Datasets | #datascience #freebook

Mining of Massive Datasets | #datascience #freebook | Public Datasets - Open Data - |

Via ukituki
luiy's insight:

Preface and Table of Content

Chapter 1. Data Mining

Chapter 2. Map-Reduce and the New Software Stack

Chapter 3. Finding Similar Items

Chapter 4. Mining Data Streams

Chapter 5. Link Analysis

Chapter 6. Frequent Itemsets

Chapter 7. Clustering

Chapter 8. Advertising on the Web

Chapter 9. Recommendation Systems

Chapter 10. Mining Social-Network Graphs

Chapter 11. Dimensionality Reduction

Chapter 12. Large-Scale Machine Learning


Download Full Book :

ukituki's curator insight, August 28, 2014 6:22 PM

The book is based on Stanford Computer Science course CS246: Mining Massive Datasets (and CS345A: Data Mining).

Scooped by luiy!

Cleaning Data with OpenRefine I #datacleaning #clustering #openTools

Cleaning Data with OpenRefine I #datacleaning #clustering #openTools | Public Datasets - Open Data - |
luiy's insight:

Don’t take your data at face value. That is the key message of this tutorial which focuses on how scholars can diagnose and act upon the accuracy of data. In this lesson, you will learn the principles and practice of data cleaning, as well as how OpenRefine can be used to perform four essential tasks that will help you to clean your data:


Remove duplicate recordsSeparate multiple values contained in the same fieldAnalyse the distribution of values throughout a data setGroup together different representations of the same reality 

These steps are illustrated with the help of a series of exercises based on a collection of metadata from the Powerhouse museum, demonstrating how (semi-)automated methods can help you correct the errors in your data.

No comment yet.
Scooped by luiy!

Hyphe #Crawler | #Medialab Tools | #dataviz #datamining #SNA_indatcom

Hyphe #Crawler | #Medialab Tools | #dataviz #datamining #SNA_indatcom | Public Datasets - Open Data - |
luiy's insight:

Hyphe does not manage different corpora or users at the moment. All the data is stored as a single corpus summarized here.


We used: 

- Web Interface:  Domino.js, Sigma.js, Bootstrap, jQuery,Modernizr, Initializr 

- Crawl & storage serverAPI:  Lucene, Scrapy, Twisted,JsonRPC, MongoDB, Thrift

No comment yet.
Scooped by luiy!

#DataMining: Practical Machine Learning #Tools and Techniques | #Weka #datascience #openaccess

#DataMining: Practical Machine Learning #Tools and Techniques | #Weka #datascience #openaccess | Public Datasets - Open Data - |
luiy's insight:

Teaching material


Slides for Chapters 1-5 of the 3rd edition can be found here.

Slides for Chapters 6-8 of the 3rd edition can be found here


These archives contain .pdf files as well as .odp files in Open Document Format that were generated using OpenOffice 2.0. Note that there are several free office programs now that can read .odp files. There is also a plug-in for Word made by Sun for reading this format. Corresponding information is on this Wikipedia page.

No comment yet.
Rescooped by luiy from Big Data, IoT and other stuffs!

A Programmer's Guide to #DataMining I #OpenBook #DataScience

A Programmer's Guide to #DataMining I #OpenBook #DataScience | Public Datasets - Open Data - |

Via Joaquín Herrero Pintado, Toni Sánchez
luiy's insight:

Table of Contents


This book’s contents are freely available as PDF files. When you click on a chapter title below, you will be taken to a webpage for that chapter. The page contains links for a PDF of that chapter and for any sample Python code and data that chapter requires. Please let me know if you see an error in the book, if some part of the book is confusing, or if you have some other comment. I will use these to revise the chapters.


Chapter 1: Introduction


Finding out what data mining is and what problems it solves. What will you be able to do when you finish this book.


Chapter 2: Get Started with Recommendation Systems


Introduction to social filtering. Basic distance measures including Manhattan distance, Euclidean distance, and Minkowski distance. Pearson Correlation Coefficient. Implementing a basic algorithm in Python.


Chapter 3: Implicit ratings and item-based filtering


A discussion of the types of user ratings we can use. Users can explicitly give ratings (thumbs up, thumbs down, 5 stars, or whatever) or they can rate products implicitly–if they buy an mp3 from Amazon, we can view that purchase as a ‘like’ rating.

Chapter 4: Classification


In  previous chapters we used  people’s ratings of products to make recommendations. Now we turn to using attributes of the products themselves to make recommendations. This approach is used by Pandora among others.


Chapter 5: Further Explorations in Classification


A discussion on how to evaluate classifiers including 10-fold cross-validation, leave-one-out, and the Kappa statistic. The k Nearest Neighbor algorithm is also introduced.


Chapter 6: Naïve Bayes


An exploration of Naïve Bayes classification methods. Dealing with numerical data using probability density functions.


Chapter 7: Naïve Bayes and unstructured text


This chapter explores how we can use Naïve Bayes to classify unstructured text. Can we classify twitter posts about a movie as to whether the post was a positive review or a negative one? (new version coming November 2013)

Intriguing Networks's curator insight, December 8, 2013 5:48 PM

Cheers thanks for this handy for all budding DH students

Scooped by luiy!

Apache #Mahout: Scalable #MachineLearning and #DataMining I #bigdata

Apache #Mahout: Scalable #MachineLearning and #DataMining I #bigdata | Public Datasets - Open Data - |
luiy's insight:
Mahout currently hasCollaborative FilteringUser and Item based recommendersK-Means, Fuzzy K-Means clusteringMean Shift clusteringDirichlet process clusteringLatent Dirichlet AllocationSingular value decompositionParallel Frequent Pattern miningComplementary Naive Bayes classifierRandom forest decision tree based classifierHigh performance java collections (previously colt collections)A vibrant communityand many more cool stuff to come by this summer thanks to Google summer of code
No comment yet.