Big Data and Hadoop

By combining data from both real and virtual worlds, we can now understand behavior at a previously unimaginable scale.

When we use data to uncover the workplace behaviors that make people effective, happy, creative, experts, leaders, followers, early adopters, and so on, we are using “people analytics.”

PernixData, a San Jose, Calif.-based storage software provider, is gearing up for the software-defined storage race. The startup is leveraging server-side flash in the hopes that the technology will give it a leg up on its competitors in the traditional enterprise information technology market.

The community team at Revolution Analytics has just updated this list of resources to learn about R on the Web. Included is this list of the top 3 resources for absolute beginners getting started with R:

Netflix CEO thought he could do a better job at developing a recommendation algorithm than his engineers. He failed – and the episode shaped the way the company has looked at data ever since.

Noam Ross recently shared a very useful guide to speeding up your R code. 

Get a bigger computer (for example, renting an instance on the Amazon cloud for a few cents an hour)Use parallel programming techniquesUsing the R byte-compilerProfiling and benchmarking your codeUsing high-performance packages (like xts, for time series)And lastly, rewriting your code to use more efficient constructs

One other tip that can have some great performance benefits is linking R to parallel BLAS libraries (Revolution R does this by default). For more details on how to speed up your R code read Noam's excellent guide, linked below.

Noam Ross: FasteR! HigheR! StrongeR! - A Guide to Speeding Up R Code for Busy People

Logistics giant Linfox is embarking on a big data-crunching exercise that will give its control centres the ability to predict hazards and help drivers navigate around them. 

The company is using a SAP HANA in-memory analytics database engine to crawl about 12 million real-time records generated by telematics equipment on a subset of its 5000-plus truck fleet.

As someone who trained as a statistician, I've always struggled with that title. I love the rigor and insight that Statistics brings to data analysis, but let's face it: Statistics — the name — has always had a bit of a branding problem.

GraphLab-the-company wants to capitalize on the success of GraphLab-the-open-source-project by building a commercial product for applying advanced machine-learning to massive graph datasets, referring to its platform as a “Hadoop but for graphs” on a high level. The company promises to continue actively supporting the open-source project.

I've been impressed in recent months by the number and quality of free datascience/machine learning books available online. I don't mean free as in some guy paid for a PDF version of an O'Reilly book and then posted it online for others to use/steal, but I mean genuine published books with a free online version sanctioned by the publisher. That is, "the publisher has graciously agreed to allow a full, free version of my book to be available on this site."

Although the science of prediction continues to improve, the work of making predictions in criminal justice is plagued by persistent shortcomings. Some stem from unfamiliarity with scientific strategies or an over-reliance on timeworn — but unreliable — prediction habits. If prediction in criminal justice is to take full advantage of the strength of these new tools, practitioners, analysts, researchers and others must avoid some commonplace mistakes and pitfalls in how they make predictions.