Big Data Technolo...
Find tag "xml"
11.5K views | +94 today
Big Data Technology, Semantics and Analytics
Trends, success and applications for big data including the use of semantic technology
Curated by Tony Agresta
Your new post is loading...
Your new post is loading...
Scooped by Tony Agresta!

Nate Silver's big-data insights -- FCW

Nate Silver's big-data insights -- FCW | Big Data Technology, Semantics and Analytics |
In his latest book, statistician and predictive analytics expert Nate Silver describes his approach to forming forecasts out of data.
Tony Agresta's insight:

“Big data is not a cure-all, and it is inherently filled with noise and uncertainty, but it does have tremendous potential if people approach it the right way. ‘The world is not lacking for techniques, it's more about the right goals and right attitudes,’ Silver said.”  Having goals associated with big data analysis is a must.   Applying technology and techniques to achieve those goals is not far behind. 

Different approaches to analysis, some of which are presented in this article, complement one another and allow you to reach those goals faster. Let's take three classic approaches - dashboards, predictive models and data visualization – and the problem of fraud detection.  Let’s say our goals include improved fraud detection for incoming insurance claims and more efficient allocation of resources to investigate those claims.  If analysts can prioritize the workload for investigators, they can find fraud faster and reduce costs.

BI dashboards typically show key metrics which may lead the analyst to spot trends that they want to model using predictive analysis.   They also point analysts to independent data that may have some explanatory power in the model.   For example, a BI dashboard showing recent insurance claims by postal code may show a spike in certain areas which could lead to deeper analysis where geographic indicators (city, zip+4) are selected as attributes to predict fraudulent claims.   While knowing that the insurance claim has a higher likelihood of being fraudulent is important, understanding the ring of people linked to that claim is potentially more important. Are those people linked to other claims that have been investigated and found to be fraudulent?  Do these people share the same address?  Are they using the same doctor or pharmacy?  Have they worked together in the past?  

Data visualization allows you to explore those relationships and picks up where predictive models leave off.  In this case, all of the major types of analysis were used to achieve the goal of identifying suspicious claims and ultimately identifying a fraud ring.

Different approaches to analysis can complement one another.  Business Intelligence and dashboards provide one level of visibility.  They point the analyst to key trends and relationships that may require a model to be built.  Results of those models (scores or yes/no indictors) can be used with data discovery tools to understand relationships, identify patterns of behavior, show connections between seemingly disparate data and rapidly draw conclusions.   Identifying goals up front will allow analysts to formulate questions they want to ask of the data.  Using different types of analysis helps address challenges with big data. 

To learn more about how you achieve your goals using Enterprise NoSQL, you can go here:

No comment yet.
Rescooped by Tony Agresta from Big Data, IoT and other stuffs!

When Pirates Meet Advanced Analytics

When Pirates Meet Advanced Analytics | Big Data Technology, Semantics and Analytics |
Seagoing criminals are always changing tactics. To catch them, uncover their hidden patterns.

Via Toni Sánchez
Tony Agresta's comment, January 15, 2013 9:30 AM
This article is spot on - with so many different sources of data, identifying suspects can be challenging. The volume and variety of data stored in different systems makes the job of the intelligence officer almost impossible. But new approaches in big data technology allow data scientists to consolidate the data, including unstructured data. Tight integration with data visualization tools that talk to Enterprise NoSQL databases make it easier than ever to profile all of the data in support of identifying connections between people, events, locations and more. This approach has proven to work in some of the largest intelligence agencies in the world. It has been applied in local law enforcement, fraud detection, loss prevention, cyber security and social networking. Real time alerts make it possible to immediately notify analysts as big data streams into the NoSQL database meeting predefined conditions. But to do this you need an Enterprise approach to NoSQL one that is hardened, scales, meets security requirements and has disaster recovery and high availability as part of the foundational technology providing production grade implementations in less time.
Scooped by Tony Agresta!

The new reality for Business Intelligence and Big Data

The new reality for Business Intelligence and Big Data | Big Data Technology, Semantics and Analytics |
You know about Big Data and its potential, how it creates greater understanding of our world, reduces waste and misuse of resources, and dramatically increases efficiency.
Tony Agresta's insight:

Data discovery tools allow you to reveal hidden insights in your data when you don't know what to look for in advance.  These highly interactive tools allow you to visualize disparate data in various forms - charts, timelines, graphs, geo-spatial and tables – and explore relationships in data to uncover patterns that static dashboards cannot.  


With the explosion of big data, organizations are now using these tools with structured, semi-structured and unstructured data.  This approach allows them to consolidate data without having to build complex schemas, search the data instantly, deliver new content products dynamically and analyze all of their data in real time.  A transformational shift in data analysis is underway allowing organizations to do this with documents, e-mails, video and other sources.   Imagine if you could load data into Hadoop, enrich it, ingest the data into an enterprise NoSQL database in real-time, index everything for instant search & discovery and analyze that data using Tableau or Cognos.   As the only Enterprise NoSQL Database on the market, MarkLogic allows you to do just that.


You can learn more here:


No comment yet.