Big Data Technolo...
Find tag "bi"
10.2K views | +3 today
Big Data Technology, Semantics and Analytics
Trends, success and applications for big data including the use of semantic technology
Curated by Tony Agresta
Your new post is loading...
Your new post is loading...
Scooped by Tony Agresta!

Data Analysis and Unstructured Data: Expanding Business Intelligence (BI) by Thinking Outside of the Box -

Data Analysis and Unstructured Data: Expanding Business Intelligence (BI) by Thinking Outside of the Box - | Big Data Technology, Semantics and Analytics |
Tony Agresta's insight:

New forms of business intelligence incorporate both structured and unstructured data into your analysis.   Where does this apply today?  Customer service, intelligence analysis in government, fraud analysis in financial services, healthcare, consumer packaged goods, retail and other markets can benefit from this approach.  The open web provides organizations with limitless data containing valuable information on sentiment, people, events, employers, relationships and more.   The ability to extract meaning from unstructured sources combined with structured data yields new insights that can be used to improve decisions. 


Let's take a look at healthcare, for example.


In an article by Dennis Amorosano entitled "Unstructured data a common hurdle to achieving guidelines", Mr. Amorosano writes "... of the 1.2 billion clinical documents produced in the United States each year, approximately 60 percent contain valuable information trapped in unstructured documents that are unavailable for clinical use, quality measurement and data mining. These paper documents have until now been the natural byproduct of most hospital workflows, as healthcare is one of the most document-intensive industries."


Forbes published an article last year entitled "The Next Revolution in Healthcare"  ( in which the author points out that the best healthcare institutions in the world still rely heavily on calculating risk to patients using clinical data.  At the same time "the real tragedy is that the information needed to properly assess the patient’s risk and determine treatment is available in the clinician’s notes, but without the proper tools the knowledge remains unavailable and hence, unused."


The good news is that new analytic solutions are available that leverage both forms of data.   BI connectivity brings the power of familiar Business tools to your applications that include unstructured data. Some of the benefits to this approach include:


  • Combining BI and NoSQL provides capabilities not available using relational stores and EDWs - real-time analysis and extended query features.
  • BI tools layer on top of NoSQL databases that use sophisticated security models to protect sensitive data. Users see only the information for which they have permissions.
  • Analysts can learn faster using data discovery tools that allow for rapid investigation of both unstructured and structured data within the same application.  A more complete view of your analysis offers tremendous advantages in patient diagnosis, claims analysis and personalized care.


To learn more about how analytics technology is working with Enterprise NoSQL Databases ideally suited to ingest, store, search and analyze all types of data, you can visit this page:


No comment yet.
Scooped by Tony Agresta!

HSBC to Pay $1.92 Billion to Settle Charges of Money Laundering

HSBC to Pay $1.92 Billion to Settle Charges of Money Laundering | Big Data Technology, Semantics and Analytics |
The announcement of a settlement on Tuesday came after state and federal authorities decided against indicting the British bank in a money-laundering case.
Tony Agresta's insight:

If you read this, notice one of the last paragraphs - 


"HSBC has since moved to bolster its safeguards. The bank doubled its spending on compliance functions and revamped its oversight, according to a spokesman. In January, HSBC hired Stuart A. Levey as chief legal officer to come up with stricter internal standards to thwart the illegal flow of cash. Mr. Levey was formerly an under secretary at the Treasury Department who focused on terrorism and financial intelligence."


Big Data Analytics is one way to do this.   But HSBC may have fallen into the trap where they focus on one form of analysis to detect money laundering.  Predictive models used to identify transactions that may be fraud or money laundering can be a useful way to detect this type of activity.   But all models contain some amount of error.  When network analysis, geospatial analysis and temporal analysis are also applied, money laundering schemes can be revealed using data visualization that show unusual patterns of behavior, linkages between people and events, fund transfers that take place at odd times and more.   Most of these institutions need to combine descriptive reporting, alerts which are triggered when outlier transactions are ready for approval, predictive models and interactive data visualization including link analysis to explore hidden relationships in data.   Without this comprehensive approach, this problem will continue to occur.  The data is all there.  Now it needs to be integrated (including unstructured data in the form of notes) and analyzed using all of the major techniques.

WebMarketingStore's comment, May 2, 2014 1:59 AM
Staggering: $1.9b is the 'settlement' amount? How much might the damage have been, full-tilt?
Scooped by Tony Agresta!

Nate Silver's big-data insights -- FCW

Nate Silver's big-data insights -- FCW | Big Data Technology, Semantics and Analytics |
In his latest book, statistician and predictive analytics expert Nate Silver describes his approach to forming forecasts out of data.
Tony Agresta's insight:

“Big data is not a cure-all, and it is inherently filled with noise and uncertainty, but it does have tremendous potential if people approach it the right way. ‘The world is not lacking for techniques, it's more about the right goals and right attitudes,’ Silver said.”  Having goals associated with big data analysis is a must.   Applying technology and techniques to achieve those goals is not far behind. 

Different approaches to analysis, some of which are presented in this article, complement one another and allow you to reach those goals faster. Let's take three classic approaches - dashboards, predictive models and data visualization – and the problem of fraud detection.  Let’s say our goals include improved fraud detection for incoming insurance claims and more efficient allocation of resources to investigate those claims.  If analysts can prioritize the workload for investigators, they can find fraud faster and reduce costs.

BI dashboards typically show key metrics which may lead the analyst to spot trends that they want to model using predictive analysis.   They also point analysts to independent data that may have some explanatory power in the model.   For example, a BI dashboard showing recent insurance claims by postal code may show a spike in certain areas which could lead to deeper analysis where geographic indicators (city, zip+4) are selected as attributes to predict fraudulent claims.   While knowing that the insurance claim has a higher likelihood of being fraudulent is important, understanding the ring of people linked to that claim is potentially more important. Are those people linked to other claims that have been investigated and found to be fraudulent?  Do these people share the same address?  Are they using the same doctor or pharmacy?  Have they worked together in the past?  

Data visualization allows you to explore those relationships and picks up where predictive models leave off.  In this case, all of the major types of analysis were used to achieve the goal of identifying suspicious claims and ultimately identifying a fraud ring.

Different approaches to analysis can complement one another.  Business Intelligence and dashboards provide one level of visibility.  They point the analyst to key trends and relationships that may require a model to be built.  Results of those models (scores or yes/no indictors) can be used with data discovery tools to understand relationships, identify patterns of behavior, show connections between seemingly disparate data and rapidly draw conclusions.   Identifying goals up front will allow analysts to formulate questions they want to ask of the data.  Using different types of analysis helps address challenges with big data. 

To learn more about how you achieve your goals using Enterprise NoSQL, you can go here:

No comment yet.