Corporate Challenge of Big Data
8.3K views | +0 today
Follow
Corporate Challenge of Big Data
Leverage Big Data for Commercial success
Curated by Ian Sykes
Your new post is loading...
Your new post is loading...
Scooped by Ian Sykes
Scoop.it!

Spark Reliability and Performance through Monitoring | MapR

Spark Reliability and Performance through Monitoring | MapR | Corporate Challenge of Big Data | Scoop.it
As more organizations begin to deploy Spark in their production clusters, the need for fine-grained monitoring tools becomes paramount.
No comment yet.
Scooped by Ian Sykes
Scoop.it!

Google's Data Architecture and What it Takes to Work at Scale - The New Stack

Google's Data Architecture and What it Takes to Work at Scale - The New Stack | Corporate Challenge of Big Data | Scoop.it
In his presentation, “What does it take to make Google work at scale?” Schwarzkopf discusses the architecture behind those 139 microseconds between submitting a search request in the Google input bar, and the pages of ads-and-search results that are returned.
No comment yet.
Scooped by Ian Sykes
Scoop.it!

Machine learning is already all around you.

Machine learning is already all around you. | Corporate Challenge of Big Data | Scoop.it

Last year, a Spotify intern named Sander Dieleman applied a powerful machine learning technology called “deep learning” to their database, allowing the program to learn to analyze music. The neural network automatically — using nothing but raw audio data — came to recognize distinctive patterns in the music.

No comment yet.
Scooped by Ian Sykes
Scoop.it!

Location, Location, Location - The Secret of Airbnb’s Pricing Algorithm

Location, Location, Location - The Secret of Airbnb’s Pricing Algorithm | Corporate Challenge of Big Data | Scoop.it

The sharing economy needs machine intelligence to set prices...With knowledge about the success of its tips, our system began adjusting the weights it gives to the different characteristics about a listing—the “signals” it is getting about a particular property. We started out with some assumptions, such as that geographic location is hugely important but that usually the presence of a hot tub is less so. 


Dan Hill, product lead at Airbnb, wrote the lodging-rental website’s pricing algorithm. 

Scooped by Ian Sykes
Scoop.it!

How to Leverage Machine Learning via Predictive APIs

How to Leverage Machine Learning via Predictive APIs | Corporate Challenge of Big Data | Scoop.it
Based on Louis Dorard's talk at APIcon UK, this article looks at the various business opportunities that have arisen due to predictive APIs and how organizations can leverage them.
No comment yet.
Scooped by Ian Sykes
Scoop.it!

Machine Learning Tools – An Overview

Machine Learning Tools – An Overview | Corporate Challenge of Big Data | Scoop.it

When it comes to training computers to act without being explicitly programmed there exist an abundance of tools from the field of Machine Learning. Academics and industry professionals use these tools for building a number of applications from Speech Recognition to Cancer Detection in MRI scans. 

No comment yet.
Scooped by Ian Sykes
Scoop.it!

Why the only thing better than big data is bigger data

Why the only thing better than big data is bigger data | Corporate Challenge of Big Data | Scoop.it

A recent study of companies concentrated not on the mathematics of how companies use sparse data, but the economic benefit that accrues to the companies that do. After controlling for other factors that could account for increased performance, the researchers behind that study found companies using more big data were more productive than their peers, and those using less big data were less productive.

No comment yet.
Scooped by Ian Sykes
Scoop.it!

Artificial Intelligence - QuickTake

Artificial Intelligence - QuickTake | Corporate Challenge of Big Data | Scoop.it

It’s the stuff of sci-fi movies and dystopian, end-of-humanity nightmares — and now, of mind-numbingly dull white-collar work. After decades of premature promises, artificial intelligence is finding its way into all sorts of businesses. Its arrival has been low-key. 

No comment yet.
Scooped by Ian Sykes
Scoop.it!

Confused Sbout Spark? Six Facts You Need to Know

Confused Sbout Spark? Six Facts You Need to Know | Corporate Challenge of Big Data | Scoop.it

A scalable data processing engine, most likely... [to] replace MapReduce.


Rajiv Bhat, senior vice president of data sciences and marketplace at InMobi, told the Economic Times. “With Hadoop, it would take us six-seven months to develop a machine learning model. Now, we can do about four models a day.”

No comment yet.
Scooped by Ian Sykes
Scoop.it!

“Analytics” is Overrated – Top 5 Reasons Why

“Analytics” is Overrated – Top 5 Reasons Why | Corporate Challenge of Big Data | Scoop.it

By promoting the rise of the data scientist and creating mystique around sophisticated data mining and algorithmic modelling, we risk building a wall between the analytics team and the business team. 

No comment yet.
Scooped by Ian Sykes
Scoop.it!

Google scolds businesses for citing security as a reason not to use cloud

Google scolds businesses for citing security as a reason not to use cloud | Corporate Challenge of Big Data | Scoop.it
Search giant reckons on-premise data is more likely to be breached than information stored in the cloud
No comment yet.
Scooped by Ian Sykes
Scoop.it!

Harvard medical professor: Big data and analytics help cure cancer | ZDNet

Harvard medical professor: Big data and analytics help cure cancer | ZDNet | Corporate Challenge of Big Data | Scoop.it
One of the world's top healthcare CIOs explains how data and predictive analytics can benefit patients and improve health care delivery.
No comment yet.
Scooped by Ian Sykes
Scoop.it!

Big data, the cloud way

Big data, the cloud way | Corporate Challenge of Big Data | Scoop.it

Big data the cloud way means being more productive when building applications, with faster and better insights, without having to worry about the underlying infrastructure. More specifically, it includes:

NoOps:  the platform handles such tasks and optimizations for you.Cost effectiveness: You manage your costs by dialing up or down the number of queries and the latency of your processing based on your cost/benefit analysis. Safe and easy collaboration: There’s one version of the data – which you control – and authorized users can access it (at no cost to you) without affecting the performance of your jobs.
No comment yet.
Scooped by Ian Sykes
Scoop.it!

Fujitsu sidesteps data scientists with a move toward tuned machine learning

Fujitsu sidesteps data scientists with a move toward tuned machine learning | Corporate Challenge of Big Data | Scoop.it

The Fujitsu Labs researchers then ran tests using 8 x 12-core servers and a 50 million record dataset. It claims that “existing techniques would take roughly one week to develop a predictive model with 96 per cent accuracy.“

With its preliminary machine learning algorithm selection run, its “technique reached that level in slightly more than two hours.” Going from one week to two hours is one heck of a speed up.

No comment yet.
Scooped by Ian Sykes
Scoop.it!

Get ready for machines that can pay each other.

Get ready for machines that can pay each other. | Corporate Challenge of Big Data | Scoop.it

Imagine a world where commercial agreements are executed by digital smart contracts. Where the contract and payments are actually tied together by mathematical rules of the blockchain.

A commercial term like “Net-30” that obligates a business to pay its invoice in 30 days, could be automatically executed on the day it’s due. Or an agreement to pay on delivery could track that the product actually was received and signed off on as in good order, before payment is released.

Escrow systems, third-party agents, and intermediary networks could be rethought to be far more efficient.

Or perhaps even more exciting is the ability for business-to-business blockchains to enable digital native payment networks that can instantly settle funds directly between parties.

No comment yet.
Scooped by Ian Sykes
Scoop.it!

The Internet of Things: Five critical questions | McKinsey & Company

The Internet of Things: Five critical questions | McKinsey & Company | Corporate Challenge of Big Data | Scoop.it
How do industry experts view the development of the Internet of Things, and what would they change? A McKinsey Global Institute article.
No comment yet.
Scooped by Ian Sykes
Scoop.it!

Human-level control through deep reinforcement learning : Nature : Nature Publishing Group

Human-level control through deep reinforcement learning : Nature : Nature Publishing Group | Corporate Challenge of Big Data | Scoop.it

We tested this agent on the challenging domain of classic Atari 2600 games12. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters.

No comment yet.
Scooped by Ian Sykes
Scoop.it!

The Next Wave Of Enterprise Software Powered By Machine Learning

The Next Wave Of Enterprise Software Powered By Machine Learning | Corporate Challenge of Big Data | Scoop.it

The possibilities fuelled by machine learning are endless.

No comment yet.
Scooped by Ian Sykes
Scoop.it!

Cloudera Creates Ibis Project to Tie Python to Hadoop Using APIs

Cloudera Creates Ibis Project to Tie Python to Hadoop Using APIs | Corporate Challenge of Big Data | Scoop.it
The latest big data project from Cloudera has the potential to make big data analytics directly available to Python application developers.
No comment yet.
Scooped by Ian Sykes
Scoop.it!

The Industrial Awakening: The Internet of Heavier Things

The Industrial Awakening: The Internet of Heavier Things | Corporate Challenge of Big Data | Scoop.it
Interest in the Internet of Things has focused on the connected home, with attention to newly designed thermostats, lights, and security systems. Yet the connected home is just the first wave of a greater transition toward “smart, connected products” throughout the economy. The next wave, focusing on what is known as the Industrial Internet, may have an even more transformative effect on our lives.
No comment yet.
Scooped by Ian Sykes
Scoop.it!

Array Databases: The Next Big Thing in Data Analytics?

Array Databases: The Next Big Thing in Data Analytics? | Corporate Challenge of Big Data | Scoop.it
There's no lack of database choices when it comes to building your next big data analytics platform. Relational data stores, column-oriented databases, in-memory data grids, graph databases, scale-out NewSQL systems, and Hadoop will all get time in the sun. But according to database pioneer Mike Stonebraker, none of these hold a candle to array databases when it comes to running complex analytics on big data sets.
No comment yet.
Scooped by Ian Sykes
Scoop.it!

Spark and Hadoop: Taking big data to the next level | The Big Data Hub

Spark and Hadoop: Taking big data to the next level | The Big Data Hub | Corporate Challenge of Big Data | Scoop.it
Spark is one of the key enablers of Hadoop’s journey from the Trough of Disillusionment to the Plateau of Productivity. Together, Spark and Hadoop are taking big data to the next level.
No comment yet.
Scooped by Ian Sykes
Scoop.it!

Switching user database on a running system

Switching user database on a running system | Corporate Challenge of Big Data | Scoop.it
Introduction All Spotify users are now stored in a Cassandra database instead of Postgres. The final switch was made on May 11th, and we, the team responsible for user login at Spotify, would like to tell you a little bit…
No comment yet.
Scooped by Ian Sykes
Scoop.it!

CogniMem Technologies Inc. • Applications: Video Surveillance

THE APPLICATION

Sort fishes on a boat before they enter the filleting machine at 360 ppm.Accept, Reject or Recycle based on the species, the fish conditions and its orientation in the metal pocket
Ian Sykess insight:

Real World example of Machine Learning in the field. The hostile environment of the Trawler at sea. Means the right fish at the right size are caught and the rest are let go.

No comment yet.
Scooped by Ian Sykes
Scoop.it!

Data preparation: A necessary step, an annoying hurdle?

Data preparation: A necessary step, an annoying hurdle? | Corporate Challenge of Big Data | Scoop.it
When discussing data analytics, professionals working with the technology often refer to the process of collecting raw information and turning it into "actionable intelligence" - a phrase my colleagues and I find to be overused.
No comment yet.