EEDSP
Follow
Find tag "twitter"
12.0K views | +12 today
EEDSP
Digital Signal Processing, Data Analytics, Big Data, HPC, Deep Learning, GPGPU
Curated by Shiwon Cho
Your new post is loading...
Your new post is loading...
Scooped by Shiwon Cho
Scoop.it!

Twitter open sources Storm-Hadoop hybrid called Summingbird

Twitter open sources Storm-Hadoop hybrid called Summingbird | EEDSP | Scoop.it
Twitter has open sourced a “streaming MapReduce” system called Summingbird that makes Hadoop and Storm play nicer together so applications that require both batch and stream processing can do their jobs with as little complexity as possible.
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

MapR Apache Hadoop MapReduce Software Download | MapR Technologies

MapR Apache Hadoop MapReduce Software Download | MapR Technologies | EEDSP | Scoop.it

Hadoop users were excited to see the real-time Hadoop analytics demonstration at the Strata Conference in Santa Clara.  By streaming the #strataconf twitter hashtag directly into a cluster during the conference, MapR displayed two real-time tag clouds showing a word bubble with the most frequently used words in conference tweets and a user name cloud of top tweeters.  Watching the information change proved mesmerizing for some.

 
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Twitter and Google offer case studies in spanning distributed systems

Twitter and Google offer case studies in spanning distributed systems | EEDSP | Scoop.it
As our compute infrastructure becomes more distributed it's much harder to keep everything synced.
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Twitter Engineering: Trident: a high-level abstraction for realtime computation

Twitter Engineering: Trident: a high-level abstraction for realtime computation | EEDSP | Scoop.it

Trident is a new high-level abstraction for doing realtime computing on top of Twitter Storm, available in Storm 0.8.0 (released today). It allows you to seamlessly mix high throughput (millions of messages per second), stateful stream processing with low latency distributed querying. If you're familiar with high level batch processing tools like Pig or Cascading, the concepts of Trident will be very familiar - Trident has joins, aggregations, grouping, functions, and filters. In addition to these, Trident adds primitives for doing stateful, incremental processing on top of any database or persistence store. Trident has consistent, exactly-once semantics, so it is easy to reason about Trident topologies.

more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

High Scalability - High Scalability - The Architecture Twitter Uses to Deal with 150M Active Users, 300K QPS, a 22 MB/S Firehose, and Send Tweets in Under 5 Seconds

High Scalability - High Scalability - The Architecture Twitter Uses to Deal with 150M Active Users, 300K QPS, a 22 MB/S Firehose, and Send Tweets in Under 5 Seconds | EEDSP | Scoop.it
Toy solutions solving Twitter’s “problems” are a favorite scalabi...
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Twitter Engineering: Introducing Flight: a web application framework

Twitter Engineering: Introducing Flight: a web application framework | EEDSP | Scoop.it

Flight is distinct from existing frameworks in that it doesn't prescribe or provide any particular approach to rendering or providing data to a web application. It's agnostic on how requests are routed, which templating language you use, or even if you render your HTML on the client or the server. While some web frameworks encourage developers to arrange their code around a prescribed model layer, Flight is organized around the existing DOM model with functionality mapped directly to DOM nodes.

more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Process real-time big data with Twitter Storm

Storm is an open source, big-data processing system that differs from other systems in that it's intended for distributed real-time processing and is language independent.
more...
No comment yet.