Large-scale Incremental Processing
17.5K views | +0 today
Follow
 
Scooped by Jaeboo Jeong
onto Large-scale Incremental Processing
Scoop.it!

Eventual Consistency in Concurrent Data Structures - belliottsmith

Eventual Consistency in Concurrent Data Structures - belliottsmith | Large-scale Incremental Processing | Scoop.it
In true modern fashion, I have reached the conclusion that the world needs to hear from me, and that I need to butcher buzzwords into places they don’t belong (this isn’t really about eventual consistency, at least in the distributed systems sense).
more...
No comment yet.
Large-scale Incremental Processing
Your new post is loading...
Your new post is loading...
Scooped by Jaeboo Jeong
Scoop.it!

Incremental consistency guarantees for replicated objects

Incremental consistency guarantees for replicated objects Guerraoui et al., OSDI 2016 We know that there’s a price to be paid for strong consistency in terms of higher latencies and reduced t…
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

How big compute is powering the deep learning rocket ship

How big compute is powering the deep learning rocket ship | Large-scale Incremental Processing | Scoop.it
The O’Reilly Data Show Podcast: Greg Diamos on building computer systems for deep learning and AI.
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

Elassandra: Elasticsearch as a Cassandra Secondary Index (Rémi Trouville, Vincent Royer, Independent) | C* Summit 2016

Many companies use both elasticsearch and cassandra, typically in the form of logs or time series, but managing many softwares at a large scale can be quite ch…
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

Steven Strogatz on Teaching Eigenvectors and Eigenvalues

Explore the largest community of artists, bands, podcasters and creators of music & audio
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

Using Amazon Athena with Apache Zeppelin

Using Amazon Athena with Apache Zeppelin | Large-scale Incremental Processing | Scoop.it
Here is how we can use Amazon Athena as a backend for Apache Zeppelin. Amazon Athena is something like Presto as a service, which provides WebUI and JDBC interface. With this we can easily run…
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

Hello gRPC! (with ScalaPB) - codecentric AG Blog

gRPC is a modern RPC framework developed by Google. It picks up the traditional idea of RPC frameworks – call remote methods as easily as if they were local – while trying to avoid mistakes made by its predecessors and focusing on requirements of microservice-oriented systems. gRPC has been heavily utilized by Google for several... Read more
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

How-to: Fuzzy Name Indexing in Apache Hadoop with Rosette and Cloudera Search - Cloudera Engineering Blog

How-to: Fuzzy Name Indexing in Apache Hadoop with Rosette and Cloudera Search - Cloudera Engineering Blog | Large-scale Incremental Processing | Scoop.it
In this guide, learn how to use Cloudera Search with Basis Technology’s Rosette®  to perform fuzzy name searches in multiple languages and scripts.
Our thanks to Basis Technology team (Jeanne Le Garrec, Hannah MacKenzie-Margulies and Brian Sawyer) for supporting writing this how-to blog.
Cloudera Search, powered by Apache Solr brings full-text, interactive search, and scalable indexing to Apache Hadoop by marrying SolrCloud with HDFS, Apache HBase, Read More
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

5 Hadoop Trends to Watch in 2017

5 Hadoop Trends to Watch in 2017 | Large-scale Incremental Processing | Scoop.it
Where will the distributed computing system go in 2017? Here are five macro trends impacting Hadoop to keep an eye out this year.
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

Google Brain Residency Program - 7 months in and looking ahead

Google Brain Residency Program - 7 months in and looking ahead | Large-scale Incremental Processing | Scoop.it
Posted by Jeff Dean, Google Senior Fellow and Leslie Phillips, Google Brain Residency Program Manager “Beyond being incredibl
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

Processing Image Documents on MapR at Scale | MapR

Processing Image Documents on MapR at Scale | MapR | Large-scale Incremental Processing | Scoop.it
There has been a lot of research in document image processing over the past 20 years, but not much research has been done in terms of parallel processing. Some of the solutions proposed for parallel processing have been to create threads of execution for each image, or to use GNU Parallel.
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

Building an Amazon SQS Custom Origin for StreamSets Data Collector - StreamSets

Building an Amazon SQS Custom Origin for StreamSets Data Collector - StreamSets | Large-scale Incremental Processing | Scoop.it
As I explained in my recent tutorial, Creating a Custom Origin for StreamSets Data Collector, it’s straightforward to extend StreamSets Data Collector (SDC) to ingest data from pretty much any source. Yogesh Choudhary, a software engineer at consulting and services company Clairvoyant, just posted his own walkthrough of building a custom origin for Amazon Simple Queue Service
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

24/7 Spark Streaming on YARN in Production | inovex-Blog

24/7 Spark Streaming on YARN in Production | inovex-Blog | Large-scale Incremental Processing | Scoop.it
At a large client in the German food retailing industry, we have been running Spark Streaming on Apache Hadoop™ YARN in production for close to a year now. Overall, Spark Streaming has proved to be a flexible, robust and scalable streaming engine.
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

KIP-98 - Exactly Once Delivery and Transactional Messaging - Apache Kafka - Apache Software Foundation

KIP-98 - Exactly Once Delivery and Transactional Messaging - Apache Kafka - Apache Software Foundation | Large-scale Incremental Processing | Scoop.it
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

Infinit: Modern Storage Platform for Container Environments

Providing state to applications in Docker requires a backend storage component that is both scalable and resilient in order to cope with a variety of use cases…
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

Building a Continous Integration Pipeline with Docker

Building a Continous Integration Pipeline with Docker | Large-scale Incremental Processing | Scoop.it
Continuous integration is one of the most popular Docker use cases. Docker and its open APis make it easy for teams to build out automated pipelines that help them create and deploy applications from dev, test, staging and into production much faster.
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

[1701.02392] Reinforcement Learning via Recurrent Convolutional Neural Networks

more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

Adaptive logging: optimizing logging and recovery costs in distributed in-memory databases

Adaptive Logging: Optimizing logging and recovery costs in distributed In-memory databases Yao et al., SIGMOD 2016 This is a paper about the trade-offs between transaction throughput and database r…
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

Shasta: Interactive reporting at scale

Shasta: Interactive Reporting At Scale Manoharan et al., SIGMOD 2016 You have vast database schemas with hundreds of tables, applications that need to combine OLTP and OLAP functionality, queries t…
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

Apache Hadoop YARN: Yet another resource negotiator

Apache Hadoop YARN: Yet Another Resource Negotiator Vavilapalli et al., SoCC 2013 The opening section of Prof. Demirbas’ reading list is concerned with programming the datacenter, aka ‘…
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

Live In-Docker Debugging for Java with IntelliJ

In this tutorial, Docker Developer Relations Engineer Sophia Parafina shows you how to set-up a development environment with IntelliJ, including liv
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

SparkSQL, Ranger, and LLAP via Spark Thrift Server for BI scenarios to provide row, column level security, and masking - Hortonworks

SparkSQL, Ranger, and LLAP via Spark Thrift Server for BI scenarios to provide row, column level security, and masking - Hortonworks | Large-scale Incremental Processing | Scoop.it
Apache Spark has ignited an explosion of data exploration on very large data sets. Spark played a big role in making general purpose distributed compute ac
more...
No comment yet.
Scooped by Jaeboo Jeong
Scoop.it!

Docker Reference Architecture: Universal Control Plane 2.0 Service Discovery and Load Balancing

Docker Reference Architecture: Universal Control Plane 2.0 Service Discovery and Load Balancing | Large-scale Incremental Processing | Scoop.it
more...
No comment yet.