EEDSP
Follow
Find tag "storm"
15.0K views | +10 today
EEDSP
Digital Signal Processing, Data Analytics, Big Data, HPC, Deep Learning, GPGPU, Distributed and Parallel Computing
Curated by Shiwon Cho
Your new post is loading...
Your new post is loading...
Scooped by Shiwon Cho
Scoop.it!

Apache Storm and Kafka Together: A Real-time Data Refinery

Apache Storm and Kafka Together: A Real-time Data Refinery | EEDSP | Scoop.it
This blog focuses on data processing engines Apache Storm and Apache Kafka, explains a multi-stage event processing pipeline, mentions use cases, and describes Storm topologies.
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Apache Storm Graduates to a Top-Level Project

Apache Storm Graduates to a Top-Level Project | EEDSP | Scoop.it
Apache Storm is a distributed, fault tolerant, and scalable platform for processing streaming data, supporting real-time analytics and machine learning.

On September 17, the Apache Software Foundation (ASF) voted to graduate Apache Storm to a top-level project (TLP). This represents a major step forward for the project and represents the momentum built by a broad community of developers from not only Hortonworks, but also Yahoo!, Alibaba, Twitter, Microsoft and many other companies.

 

http://storm.apache.org

 

more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

The Future of Apache Storm: Secure, Highly-Available, Multi-Tenant

The Future of Apache Storm: Secure, Highly-Available, Multi-Tenant | EEDSP | Scoop.it
YARN and Apache Storm: A Powerful Combination YARN changed the game for all data access engines in Apache Hadoop. As part of Hadoop 2, YARN took the resource management capabilities that were in MapReduce and packaged them for use by new engines. Now Apache Storm is one of those data-processing engines that can run alongside …
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Storm, Neo4j and Python: Real-Time Stream Computation on Graphs | Javalobby

Storm, Neo4j and Python: Real-Time Stream Computation on Graphs | Javalobby | EEDSP | Scoop.it
For a look at real-time stream computation with Storm, Neo4j, and Python, check out this talk from Sonal Raj at PyCon India 2013. The talk covers a lot of...
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Yahoo! Open Sources Storm on Hadoop

Apache Hadoop is de facto standard for Big Data storage and batch processing, while Tweeter Storm is quickly becoming a standard for large-scale event processing implementations. Unfortunately, up until recently, Storm and Hadoop required two physically different clusters for their implementation. Last week Yahoo! announced open sourcing Storm running on a Hadoop cluster.

more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

MapR Apache Hadoop MapReduce Software Download | MapR Technologies

MapR Apache Hadoop MapReduce Software Download | MapR Technologies | EEDSP | Scoop.it

Hadoop users were excited to see the real-time Hadoop analytics demonstration at the Strata Conference in Santa Clara.  By streaming the #strataconf twitter hashtag directly into a cluster during the conference, MapR displayed two real-time tag clouds showing a word bubble with the most frequently used words in conference tweets and a user name cloud of top tweeters.  Watching the information change proved mesmerizing for some.

 
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Storm and Hadoop: Convergence of Big-Data and Low-Latency Processing · YDN Blog

Storm and Hadoop: Convergence of Big-Data and Low-Latency Processing · YDN Blog | EEDSP | Scoop.it

At Yahoo!, Hadoop plays a central role in providing personalized experiences for our users and creating value for our advertisers. To serve Yahoo!’s emerging business needs, the Cloud Engineering Group is working on a next generation platform that enables the convergence of big-data and low-latency processing.

more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Process real-time big data with Twitter Storm

Storm is an open source, big-data processing system that differs from other systems in that it's intended for distributed real-time processing and is language independent.
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Introducing Storm – the “Hadoop of realtime” « BigData

Introducing Storm – the “Hadoop of realtime” « BigData | EEDSP | Scoop.it

Storm is a distributed realtime computation system, in this page we’ll review a functional example explaining the basic concepts

more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Apache Storm 0.9 basic training - Verisign

Apache Storm 0.9 basic training (130 slides) covering: 1. Introducing Storm: history, Storm adoption in the industry, why Storm 2. Storm core concepts: topolog…
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Apache Storm Design Pattern—Micro Batching

Apache Storm Design Pattern—Micro Batching | EEDSP | Scoop.it
As part of Apache Storm design patterns' series blog, this first blog explores three options for micro-batching using Apache Storm's core APIs.
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Apache Storm 0.9.2 Released with Netty Transport Improvements and New APIs

Apache Storm 0.9.2 Released with Netty Transport Improvements and New APIs | EEDSP | Scoop.it
The Apache Storm community releases Apache Storm 0.9.2 with new set of APIs and improvements to Nett's transport layer.
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Twitter open sources Storm-Hadoop hybrid called Summingbird

Twitter open sources Storm-Hadoop hybrid called Summingbird | EEDSP | Scoop.it
Twitter has open sourced a “streaming MapReduce” system called Summingbird that makes Hadoop and Storm play nicer together so applications that require both batch and stream processing can do their jobs with as little complexity as possible.
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

A Hadoop Alternative: Building a real-time data pipeline with Storm

A Hadoop Alternative: Building a real-time data pipeline with Storm | EEDSP | Scoop.it
With the tremendous growth of the online advertising industry, ad networks have to deal with a humongous amount of data to process. For years, Hadoop has been the de-facto technology used to aggreg...
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Near Real-time Processing Over Hadoop and HBase | Cerner Engineering Health

Near Real-time Processing Over Hadoop and HBase | Cerner Engineering Health | EEDSP | Scoop.it

These significant differences mean different processing infrastructures. Nathan Marz described this well in his How to Beat the CAP Theorem post. The result is a system that uses complementary technologies: stream-based processing with Storm and batch processing with Hadoop.

Interestingly, HBase sits at a juncture between realtime and batch processing models. It offers aspects of batch processing; computation can be moved to the data via direct MapReduce support. It also supports realtime patterns with random access and fas

more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Implementing Real-Time Trending Topics With A Distributed Rolling Count Algorithm in Storm - Michael G. Noll

Implementing Real-Time Trending Topics With A Distributed Rolling Count Algorithm in Storm - Michael G. Noll | EEDSP | Scoop.it
How to implement a distributed rolling count (aka sliding window) algorithm in Storm
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Storm, distributed and fault-tolerant realtime computation

Storm, distributed and fault-tolerant realtime computation | EEDSP | Scoop.it
Storm is a distributed and fault-tolerant realtime computation system. Similar to how Hadoop provides a set of general primitives for doing batch processing, Storm provides a set of general primitives for doing realtime computation.
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Storm 0.8.0: The Most Significant Release of Storm Yet

In Nathan Marz’s words Storm1 0.8.0 is “a major step forward in the evolution of the project”:
• Executors: Storm 0.8.0 has a new model where a worker is a process and an executor is a thread.
•...
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Storm: The Real-Time Layer - GlueCon 2012

Created by Nathan Marz at Twitter, Storm promises to help companies augment their batch-based big data processing systems with real-time computation.
more...
No comment yet.