EEDSP
17.8K views | +5 today
Follow
EEDSP
Digital Signal Processing, Data Analytics, Big Data, HPC, Deep Learning, GPGPU, Distributed and Parallel Computing
Curated by Shiwon Cho
Your new post is loading...
Your new post is loading...
Scooped by Shiwon Cho
Scoop.it!

Pelikan Cache

Pelikan is a framework for building cache services. It is part of Twitter's unified cache project.
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

How Twitter Users Can Generate Better Ideas | MIT Sloan Management Review

How Twitter Users Can Generate Better Ideas | MIT Sloan Management Review | EEDSP | Scoop.it
There's a link between the amount of diversity in employees’ Twitter networks and the quality of their ideas.
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Twitter open sources Storm-Hadoop hybrid called Summingbird

Twitter open sources Storm-Hadoop hybrid called Summingbird | EEDSP | Scoop.it
Twitter has open sourced a “streaming MapReduce” system called Summingbird that makes Hadoop and Storm play nicer together so applications that require both batch and stream processing can do their jobs with as little complexity as possible.
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

MapR Apache Hadoop MapReduce Software Download | MapR Technologies

MapR Apache Hadoop MapReduce Software Download | MapR Technologies | EEDSP | Scoop.it

Hadoop users were excited to see the real-time Hadoop analytics demonstration at the Strata Conference in Santa Clara.  By streaming the #strataconf twitter hashtag directly into a cluster during the conference, MapR displayed two real-time tag clouds showing a word bubble with the most frequently used words in conference tweets and a user name cloud of top tweeters.  Watching the information change proved mesmerizing for some.

 
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Twitter and Google offer case studies in spanning distributed systems

Twitter and Google offer case studies in spanning distributed systems | EEDSP | Scoop.it
As our compute infrastructure becomes more distributed it's much harder to keep everything synced.
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Twitter Engineering: Trident: a high-level abstraction for realtime computation

Twitter Engineering: Trident: a high-level abstraction for realtime computation | EEDSP | Scoop.it

Trident is a new high-level abstraction for doing realtime computing on top of Twitter Storm, available in Storm 0.8.0 (released today). It allows you to seamlessly mix high throughput (millions of messages per second), stateful stream processing with low latency distributed querying. If you're familiar with high level batch processing tools like Pig or Cascading, the concepts of Trident will be very familiar - Trident has joins, aggregations, grouping, functions, and filters. In addition to these, Trident adds primitives for doing stateful, incremental processing on top of any database or persistence store. Trident has consistent, exactly-once semantics, so it is easy to reason about Trident topologies.

more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Real-time Analytics at Massive Scale with Pinot | LinkedIn Engineering

Real-time Analytics at Massive Scale with Pinot | LinkedIn Engineering | EEDSP | Scoop.it

Pinot is the infrastructure that powers 18 member facing analytics products and more than 15 internal analytics products.

more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Analysing the Twitter Mentions Network

Analysing the Twitter Mentions Network | EEDSP | Scoop.it
By Douglas Ashton, Consultant One of the big successes of data analytics is the cultural change in how business decisions are being made. There is now wide spread acceptance of the role that data science has to play in decision making.
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Introducing practical and robust anomaly detection in a time series | Twitter Blogs

Introducing practical and robust anomaly detection in a time series | Twitter Blogs | EEDSP | Scoop.it

Both last year and this year, we saw a spike in the number of photos uploaded to Twitter on Christmas Eve, Christmas and New Year’s Eve (in other words, an anomaly occurred in the corresponding time series). Today, we’re announcing AnomalyDetection, our open-source R package that automatically detects anomalies like these in big data in a practical and robust way.

more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

High Scalability - High Scalability - The Architecture Twitter Uses to Deal with 150M Active Users, 300K QPS, a 22 MB/S Firehose, and Send Tweets in Under 5 Seconds

High Scalability - High Scalability - The Architecture Twitter Uses to Deal with 150M Active Users, 300K QPS, a 22 MB/S Firehose, and Send Tweets in Under 5 Seconds | EEDSP | Scoop.it
Toy solutions solving Twitter’s “problems” are a favorite scalabi...
more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Twitter Engineering: Introducing Flight: a web application framework

Twitter Engineering: Introducing Flight: a web application framework | EEDSP | Scoop.it

Flight is distinct from existing frameworks in that it doesn't prescribe or provide any particular approach to rendering or providing data to a web application. It's agnostic on how requests are routed, which templating language you use, or even if you render your HTML on the client or the server. While some web frameworks encourage developers to arrange their code around a prescribed model layer, Flight is organized around the existing DOM model with functionality mapped directly to DOM nodes.

more...
No comment yet.
Scooped by Shiwon Cho
Scoop.it!

Process real-time big data with Twitter Storm

Storm is an open source, big-data processing system that differs from other systems in that it's intended for distributed real-time processing and is language independent.
more...
No comment yet.