Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc)
21.3K views | +6 today
Follow
Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc)
Search Engines, Architecture, Information Retrieval , Solr , Lucene , ElasticSearch , Natural Language Processing, Rank, Relevance, etc
Your new post is loading...
Your new post is loading...
Suggested by Ahmed Besbes
Scoop.it!

Sentiment analysis on Twitter using word2vec and keras

Sentiment analysis on Twitter using word2vec and keras | Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc) | Scoop.it
The focus of this post is sentiment analysis. This is a Natural Language Processing (NLP) application I find challenging but enjoyable. It aims at identifying emotional states, reactions and subjective information. It tries to determine the attitude of a speaker with respect to some topic. If done automatically with high precision and on a large scale, sentiment analysis could be a goldmine for marketers or politicians who want to measure the public opinion through social networks. In this post I'll show you how I built a machine learning model that classifies tweets with respect to their polarity. Tweets are short and yet capture lots of subjective information. That's why we'll be playing with them. Some words for those who are ready to dive in the code: I'll be using python, gensim, the word2vec model and Keras.
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Elasticsearch API 101 - DZone Integration

Elasticsearch API 101 - DZone Integration | Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc) | Scoop.it
Here, we'll be looking at the main calls in the Elasticsearch REST API and examples of its use to help you get the most out of this highly useful API.
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

A Useful Elasticsearch Cheat Sheet in Times of Trouble

A Useful Elasticsearch Cheat Sheet in Times of Trouble | Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc) | Scoop.it
Instead of going through Elasticsearch’s documentation yet another time or looking at random Stack Overflow answers, just save this in your favorites!
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Text Retrieval and Search Engines | Coursera

Text Retrieval and Search Engines | Coursera | Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc) | Scoop.it
Text Retrieval and Search Engines from University of Illinois at Urbana-Champaign. Recent years have seen a dramatic growth of natural language text data, including web pages, news articles, scientific literature, emails, enterprise documents
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Tools for Monitoring Elasticsearch Performance

Tools for Monitoring Elasticsearch Performance | Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc) | Scoop.it
We invite you to take a moment of your day to acquaint yourself with the best monitoring tools for Elasticsearch performance.
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Location Relevance at Airbnb

Location Relevance at Airbnb | Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc) | Scoop.it
How data powers search and location relevance at airbnb
more...
No comment yet.
Rescooped by Carlos Sponchiado (Sponch) from BigData NoSql and Data Stuff
Scoop.it!

ElasticSearch Query: Performance Optimisation

ElasticSearch Query: Performance Optimisation | Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc) | Scoop.it

In one of my previous posts on elasticsearch, i shared my understanding of elasticsearch configurations and best practices. That was mostly from an indexing perspective. There are several tweaks one can use to optimise query performance as well. Improving querying time can be even more challenging than trying to improve indexing times. Lets see why querying is more of a challenge:

Queries can go on while index is getting updatedDifferent queries would need different strategies for optimisationsThere are far more configurations that impact query performance:Query syntax/clauses usedIndex schemaElasticsearch configurationsRAM, CPU, Network, IO

And there are times when you need to fire 2 or more queries in succession to get certain results back from ES. I have had one such scenario recently where i needed to fire 3 queries to ES and make sure that the response times where always less then a second. The 3 queries in question were related to each other in a sense that query 2 uses output of query 1 and query 3 uses output from query 2. For my use case, one of the queries was simple, while others two were more complex as they had aggregations, stats, filters etc.

As outlined above, there are several things that can prevent an optimal response time. Also, to safely say that  a desired response time has been achieved, one needs to test and test right. A poor testing method would lead to misleading performance statistics. Below are details of my testing methodology and tweaks that led to sub second response times for 3 queries.

ElasticSearch Cluster and Indexes5 Machines in the cluster5 Shards per index250 GB EBS volume on each machine to hold indexesIndexes are stored as compressedNo indexing takes place while testing (my use case asks for indexing in batch once a day)3 indexesIndex A: with 24+ million records (used in 1st query)All integer fields.4 fields.Index B: with 90+ million records (used in 2nd query)All integers3 fieldsIndex C: with 340K records (used in 3rd query)String, Integer and Date fieldsonly few fields used in querying.Different machine types:to hold ES indexes: m3.large to c3.4xlargeRAMDifferent sizes for tests, starting from 4GB to 15GB given to ES instance.
Via Alex Kantone
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Global Languages Support at Netflix - Testing Search Queries

Globalization at Netflix Having launched the Netflix service globally in January, we now support search in 190 countries.  We currentl
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Performance Considerations for Elasticsearch Indexing | Elastic

Performance Considerations for Elasticsearch Indexing | Elastic | Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc) | Scoop.it
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Autocomplete Using Elasticsearch - DZone Big Data

Autocomplete Using Elasticsearch - DZone Big Data | Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc) | Scoop.it
We take a look at how to implement autocomplete using Elasticsearch and nGrams in this post. Read on for more information.
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

A Complete And Easy Guide To Check Elasticsearch - DZone Big Data

A Complete And Easy Guide To Check Elasticsearch - DZone Big Data | Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc) | Scoop.it
As Ops/DevOps, we are usually more concerned about cluster health and the data inside. This article is a simple step-by-step guide to check Elasticsearch, cluster, nodes, shards, indices, documents, and more.
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Shay Banon - ElasticSearch: Big Data, Search, and Analytics

In this session we will explore elasticsearch, specifically, how to handle huge amount of data with it, how to effectively search it, and last, use facet
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Backstage Blog - Architecture behind our new Search and Explore experience - SoundCloud Developers

more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Better Search through Better Queries – Daniel Tunkelang – Medium

Better Search through Better Queries – Daniel Tunkelang – Medium | Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc) | Scoop.it
I spend a lot of time helping companies improve their search engines. Much of my work involves returning better results for their users’ search queries. But there’s a better strategy: improving the…
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

now publishers - Searching the Enterprise

Abstract

Search has become ubiquitous but that does not mean that search has been solved. Enterprise search, which is broadly speaking the use of information retrieval technology to find information within organisations, is a good example to illustrate this. It is an area that is of huge importance for businesses, yet has attracted relatively little academic interest. This monograph will explore the main issues involved in enterprise search both from a research as well as a practical point of view. We will first plot the landscape of enterprise search and its links to related areas. This will allow us to identify key features before we survey the field in more detail. Throughout the monograph we will discuss the topic as part of the wider information retrieval research field, and we use Web search as a common reference point as this is likely the search application area that the average reader is most familiar with.
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Top 15 Solr vs. Elasticsearch Differences

Top 15 Solr vs. Elasticsearch Differences | Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc) | Scoop.it
Solr vs. Elasticsearch. Elasticsearch vs. Solr.  Which one is better? How are they different? Which one should you use? Before we start, check out two useful Cheat Sheets to guide you through both …
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Merge Policy Internals in Solr - DZone Java

Merge Policy Internals in Solr - DZone Java | Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc) | Scoop.it
Last week, a colleague asked me a really simple question about segments merging in Solr. After discussing the answer for some minutes while playing aroun
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Search & Big Data Analytics in 2017: 5 Hot Topics

Search & Big Data Analytics in 2017: 5 Hot Topics | Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc) | Scoop.it
Search and big data analytics have evolved significantly over the last few years, and organizations are increasingly using these technologies to meet their mission-critical needs.   At the beginn
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Daria Sorokina - Amazon Search: The Joy of Ranking Products - MLconf SF 2016

Amazon Search: The Joy of Ranking Products: Amazon is one of the world’s largest e-commerce sites and Amazon Search powers the majority of Amazon’s sales
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Presentations Lucene / Solr Revolution 2016

Presentations Lucene / Solr Revolution 2016 | Text Retrieval and Search Engines Technologies ( Natural Language Processing, Solr, Lucene, Elasticsearch, etc) | Scoop.it
Read and download presentations by Lucidworks
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Q&A: Relevant Search with Elasticsearch and Solr

In their book "Relevant Search", Doug Turnbull and John Berryman focus on the challenge of providing search results by balancing the needs and intents of the user. Using Elasticsearch and Solr, relevance engineers can constantly tune the needs of the business vs. the needs of the user.
more...
No comment yet.
Scooped by Carlos Sponchiado (Sponch)
Scoop.it!

Elastic{ON} Videos | Elastic

The premier Elasticsearch, Logstash, and Kibana conference for the Elastic data platform.
more...
No comment yet.