Big Data Technolo...
Follow
Find
11.7K views | +94 today
Big Data Technology, Semantics and Analytics
Trends, success and applications for big data including the use of semantic technology
Curated by Tony Agresta
Your new post is loading...
Your new post is loading...
Scooped by Tony Agresta
Scoop.it!

Visualization startup Datahero opens its doors and delivers data analysis for the masses

Visualization startup Datahero opens its doors and delivers data analysis for the masses | Big Data Technology, Semantics and Analytics | Scoop.it
A year after launching, data-analysis-for-the-masses startup Datahero is finally opening its doors to the public. All in all, it delivers on its promise with a service that’s both intelligent and intuitive.
Tony Agresta's insight:

One of the primary challenges with data discovery and visualization technology is the process of exploring and uncovering meaningful insights in your data.  It takes time.  To the extent this can be automated by matching visualization types to the data in support of uncovering hidden meaning, users will find these tools more valuable.  It won't always be perfect but if the automated part of data visualization begins to focus your attention on a slice of the data, the analyst gains an advantage.   Consider it a starting point to continue the interactive side of data discovery which may include filtering the data, integrating other sources, broadcasting results from one chart to another or sharing results with other analysts that may provide additional insights.  It sounds like Chart Magic may have some of this built in. 

 


more...
Scooped by Tony Agresta
Scoop.it!

Can You Pass the ACID Test? - MarkLogic

Can You Pass the ACID Test? - MarkLogic | Big Data Technology, Semantics and Analytics | Scoop.it
Visit MarkLogic.com to read Can You Pass the ACID Test?.
more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

How to harness Big Data in motion | TechRepublic

How to harness Big Data in motion | TechRepublic | Big Data Technology, Semantics and Analytics | Scoop.it
Here are some of the technologies that companies need to look at if they want real-time big data analytics.
Tony Agresta's insight:

The premise of this article is very true.   Harnessing big data in motion is critical in many applications. Hadoop's parallel processing of large sets of data is not the answer today.   Relatively static data in the Hadoop Distributed File System processed in bulk does not meet the demands of mission critical real time applications in either government or commercial sectors. And the article goes on to indicate there are solutions to this problem.

 

But the article fails to reference MarkLogic who has been doing this for over a decade.   Today, in concert with Hadoop, data can be processed in bulk and streamed, in real time, into MarkLogic.   Alternatively, data can be ingested directly into MarkLogic.   Since indexes (many) are derived in real time and stored in both memory and on disk, the data is available instantly through enterprise search, business intelligence tools or customer applications.

 

Applications for big data in motion are diverse and include homeland defense, derivatives trading to assess and manage risk, access to content in support or new digital media products, real time analytics on cargo shipments and a host of customer relationship management applications including data streaming in from the open web.

 

Buyers should look for technology that is "enterprise grade", low risk to your organization, powerful enough to meet the growing data demands and customer expectations.  The ability to read, write, backup and recover big data within a computing architecture that provides real time, high availability should not be underestimated.

 

Adam Fowler does a great job at describing a very important aspect of Enterprise NoSQL here (ACID Transactions):


http://adamfowlerml.wordpress.com/2013/01/25/true-acid-compliance/


Adam also provides some research he collected on why users are choosing to use NoSQL databases  (and not). Notice that streaming data and data volumes are two of the top three reasons in support of NoSQL


http://adamfowlerml.wordpress.com/2013/01/04/why-use-a-nosql-database-and-why-not/







more...
No comment yet.
Rescooped by Tony Agresta from NoSQL Databases
Scoop.it!

Big Data Star System - Infographic showing players and relative impact of Big Data on their business

Big Data Star System - Infographic showing players and relative impact of Big Data on their business | Big Data Technology, Semantics and Analytics | Scoop.it

Click here to edit the title


Via Adrian Carr
more...
Adrian Carr's curator insight, May 16, 2013 5:29 AM

This is a great visualisation of the wikibon analysis of who is doing what in "big data".

It separatres software, hardware and services and shows relative size % of Big Data revenuw as a percentage of total.

Nice.

I love Infographics, definitely the way my small brain learns..

Scooped by Tony Agresta
Scoop.it!

MarkLogic 7 Vision: World-Class Triple Store and World-Beating Information Store - semanticweb.com

MarkLogic 7 Vision: World-Class Triple Store and World-Beating Information Store - semanticweb.com | Big Data Technology, Semantics and Analytics | Scoop.it
MarkLogic 7 Vision: World-Class Triple Store and World-Beating Information Store
more...
No comment yet.
Rescooped by Tony Agresta from NoSQL Databases
Scoop.it!

MarkLogic Search and Discovery Whiteboard

Understand the key challenges faced by enterprises today when delivering search and discovery applications to end users handling Big Data, and, discover how MarkLogic Enterprise NoSQL solutions overcome those challenges.

 


Via Dominic Spitz, Adrian Carr
more...
Adrian Carr's curator insight, May 10, 2013 11:26 AM

This is a great beginers guide to NoSQL in general and what Enterprise NoSQL means.

Short. Sweet and it starts from what we know...RDBMS !!

Scooped by Tony Agresta
Scoop.it!

The Many Uses for Searchable Databases: Video

The Many Uses for Searchable Databases: Video | Big Data Technology, Semantics and Analytics | Scoop.it
MarkLogic CEO Gary Bloom discusses the uses for searchable databases with Cory Johnson on Bloomberg Television's "Bloomberg West." (Source: Bloomberg)
more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Visa Says Big Data Identifies Billions of Dollars in Fraud

Visa Says Big Data Identifies Billions of Dollars in Fraud | Big Data Technology, Semantics and Analytics | Scoop.it
Visa’s chief enterprise risk officer, Ellen Richey, says “you see the criminal capability evolving on the technology side.” She gives CIO Journal an inside look at how the company has used Big Data to make its network more secure...
Tony Agresta's insight:



The approach Visa takes in identifying fraud is grounded in 16 different predictive models and allows for new independent variables to be added to the model.  This improves accuracy while alowing the models to be kept up to date.  Here's an excerpt from the WJS Article:

 

"The new analytic engine can study as many as 500 aspects of a transaction at once. That’s a sharp improvement from 2005, when the company’s previous analytic engine could study only 40 aspects at once. And instead of using just one analytic model, as it did in 2005, Visa now operates 16 models, covering different segments of its market, such as geographic regions."

 

The article also states that the analytics engine has the card number and not the personal information about the transaction - likley stored in a different system.  I wonder if Visa, at some point in the process, also takes the fraud transactions and analyzes them visually to identify connections and linkages based on address, other geographic identifiers, 3rd party data, employer data and more?  Are two or more of the fraud cases in some way connected?  Does this represent a ring of activity presening higher risk to merchants, customers and Visa?

 

The tools on the market to do this work are expanding.   The data used to analyze this activity (including unstructured data) is being stored in databases that allow for the visual analysis of big data.  Graph databases replete with underlying intelligence extracted from text that identify people, places and events can be used to extend the type of analysis that Visa is doing and prioritize investigations.   Through more efficient allocation of investigation resources, fraud prevention can jump to a higher level.


more...
luiy's curator insight, April 27, 2013 2:37 PM

“From the strategic point of view, we are achieving an amazing improvement, year over year, in our ability to detect fraud,” says Richey. “It’s not just our ability to analyze our transactions, but our ability to add new kinds of data, such as geo-location, to that analysis. With every new type of data, we increase the accuracy of our models. And from a strategic point of view we can think about taking and additional step change of fraud out of our system.”

In the future, Big Data will play a bigger role in authenticating users, reducing the need for the system to ask users for multiple proofs of their identify, according to Richey, and 90% or more of transactions will be processed without asking customers those extra questions, because algorithms that analyze their behavior and the context of the transaction will dispel doubts. “Data and authentication will come together,” Richey said.

The data-driven improvement in security accomplishes two strategic goals at once, according to Richey. It improves security itself, and it increases trust in the brand, which is critical for the growth and well-being of the business, because consumers won’t put up with a lot of credit-card fraud. “To my mind, that is the importance of the security improvements we are seeing,” she said. “Our investments in data and analysis are baseline to our ability to thrive and grow as a company.”

Rescooped by Tony Agresta from NoSQL Databases
Scoop.it!

Welcome to the Age of Big Drone Data | Danger Room | Wired.com

Welcome to the Age of Big Drone Data | Danger Room | Wired.com | Big Data Technology, Semantics and Analytics | Scoop.it
The Air Force can't deal with all of the footage its flying spy 'bots take. And with the fancy new cameras the drones are getting, the problem is about to get way, way worse.

Via Adrian Carr
more...
Adrian Carr's curator insight, April 25, 2013 9:53 AM

You are being watched !

and downloaded, indexed, sorted, serached, reported on and...hopefully exlcuded.

Big Drone Data will drive infrastructure upgrades for the USAF and doubtless many nations will follow suit.

This represents huge volumes of data, at speed - a challenge for the network but even more of a challenge for the analyst.

Scooped by Tony Agresta
Scoop.it!

Data Analysis and Unstructured Data: Expanding Business Intelligence (BI) by Thinking Outside of the Box - Zunia.org

Data Analysis and Unstructured Data: Expanding Business Intelligence (BI) by Thinking Outside of the Box - Zunia.org | Big Data Technology, Semantics and Analytics | Scoop.it
Tony Agresta's insight:

New forms of business intelligence incorporate both structured and unstructured data into your analysis.   Where does this apply today?  Customer service, intelligence analysis in government, fraud analysis in financial services, healthcare, consumer packaged goods, retail and other markets can benefit from this approach.  The open web provides organizations with limitless data containing valuable information on sentiment, people, events, employers, relationships and more.   The ability to extract meaning from unstructured sources combined with structured data yields new insights that can be used to improve decisions. 

 

Let's take a look at healthcare, for example.

 

In an article by Dennis Amorosano entitled "Unstructured data a common hurdle to achieving guidelines", Mr. Amorosano writes "... of the 1.2 billion clinical documents produced in the United States each year, approximately 60 percent contain valuable information trapped in unstructured documents that are unavailable for clinical use, quality measurement and data mining. These paper documents have until now been the natural byproduct of most hospital workflows, as healthcare is one of the most document-intensive industries."

 

Forbes published an article last year entitled "The Next Revolution in Healthcare"  (http://www.forbes.com/sites/singularity/2012/10/01/the-next-revolution-in-healthcare/) in which the author points out that the best healthcare institutions in the world still rely heavily on calculating risk to patients using clinical data.  At the same time "the real tragedy is that the information needed to properly assess the patient’s risk and determine treatment is available in the clinician’s notes, but without the proper tools the knowledge remains unavailable and hence, unused."

 

The good news is that new analytic solutions are available that leverage both forms of data.   BI connectivity brings the power of familiar Business tools to your applications that include unstructured data. Some of the benefits to this approach include:

 

  • Combining BI and NoSQL provides capabilities not available using relational stores and EDWs - real-time analysis and extended query features.
  • BI tools layer on top of NoSQL databases that use sophisticated security models to protect sensitive data. Users see only the information for which they have permissions.
  • Analysts can learn faster using data discovery tools that allow for rapid investigation of both unstructured and structured data within the same application.  A more complete view of your analysis offers tremendous advantages in patient diagnosis, claims analysis and personalized care.

 

To learn more about how analytics technology is working with Enterprise NoSQL Databases ideally suited to ingest, store, search and analyze all types of data, you can visit this page:

 

http://www.marklogic.com/what-is-marklogic/analytics-business-intelligence/

 

more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

ACID Transactions...Check! | MarkLogic

Visit MarkLogic.com to read ACID Transactions...Check!.
more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Google Uses Big Data to Fight Human Trafficking | CIO Today

Google Uses Big Data to Fight Human Trafficking | CIO Today | Big Data Technology, Semantics and Analytics | Scoop.it
Big Data is being used to obtain intelligence on virtually every transaction conceivable. Now, Google is taking Big Data analytics into a new arena -- the fight against human trafficking.
more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Big Data is Just the Beginning: Cisco Connected World Technology Report

Big Data is Just the Beginning: Cisco Connected World Technology Report | Big Data Technology, Semantics and Analytics | Scoop.it
This week, the third and final chapter of the most recent Cisco Connected World Technology Report was released. This global survey asked 1,800 IT professionals
Tony Agresta's insight:

As this article states, security around Big Data is essential.   Enterprise approaches to big data are secure to handle mission critical applications in government sectors.     This should be an important evaluation criteria when evaluating big data technology.   A quote from the Cisco report: 


"More than one in four respondents globally said that data security and risk management are a major concern. Protecting Big Data is a big challenge. Data resides everywhere; it is in the cloud, in mobile devices, and in social networks. Data comes from disparate sources in different shapes and forms. Multi-perimeter protection is key but securing data and protecting users’ privacy goes beyond the traditional IT view."


To learn more about MarkLogic, you can go here:

http://www.marklogic.com/what-is-marklogic/inside-marklogic/



more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

The human-safety side of big data

The human-safety side of big data | Big Data Technology, Semantics and Analytics | Scoop.it
How the Canadian operations of an international construction company used data analytics to drastically
more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Beyond Enterprise Search...

Searching through all your content is fine - until you get a mountain of it with similar content, differentiated only by context. Then you'll need to understand the meaning within the content. In t...
Tony Agresta's insight:

A  must read from Adam Fowler.

more...
No comment yet.
Rescooped by Tony Agresta from MarkLogic - Enterprise NoSQL Database
Scoop.it!

MarkLogic Enhances Enterprise NoSQL Database Applications & Analytic Solutions With Intel(R) Distribution for Apache Hadoop Software

MarkLogic Enhances Enterprise NoSQL Database Applications & Analytic Solutions With Intel(R) Distribution for Apache Hadoop Software | Big Data Technology, Semantics and Analytics | Scoop.it
SANTA CLARA, CA--(Marketwire - Feb 26, 2013) - Strata Conference - To ensure that organizations are able to securely make use of their growing

Via Dominic Spitz
Tony Agresta's insight:

With this announcement, Enterprise Hadoop meets Enterprise NoSQL including certification and testing of a known configuration, integrated support, enterprise management tools and performance & security enhancements.  To be specific, Apache Hadoop can take full advantage of the advanced processor capabilities such as AES-NI and SSE.   Energy efficient servers for scale-out architectures are enabled with Xeon E5.   Faster data ingest, export, perfromance and reliability support SSD big data usage models.    Most importantly, enterprise hadoop complements Marklogic in areas such as operational applications, real time decision making, distributed indexes and enterprise search.


more...
Dominic Spitz's curator insight, February 26, 2013 12:18 PM

To ensure that organizations are able to securely make use of their growing volumes of data MarkLogic Corporation today announced that it will deliver an enterprise-grade application and analytics software solution to unlock the power of Big Data based on the Intel® Distribution for Apache Hadoop software (Intel® Distribution). The Intel Distribution will work seamlessly with the MarkLogic Enterprise NoSQL (not only SQL) database to enable real-time transactional and analytic applications.


Read More At IBD: http://news.investors.com/newsfeed-marketwire/022613-141442300-marklogic-enhances-enterprise-nosql-database-applications-and-analytic-solutions-with-intelr-distribution-for-apache-hadoop-software.aspx#ixzz2M1i05XXF
Adam Fowler's curator insight, May 3, 2013 9:04 AM

Hadoop is increasingly important for cheap data archival and batch analysis. Read why using this with MarkLogic is of benefit.

Rescooped by Tony Agresta from Big Data "The Two Second Advantage"
Scoop.it!

Imagine what you could do if you had access to all your data.......

Imagine what you could do if you had access to all your data....... | Big Data Technology, Semantics and Analytics | Scoop.it

Via Marc McNeill
more...
Marc McNeill's curator insight, May 10, 2013 4:27 AM

Our customers are part of the new generation using the power of all their data, not just the data that sits in traditional relational databases. Watch this video and find out why.

Scooped by Tony Agresta
Scoop.it!

Facebook Is About To Launch A Huge Play In 'Big Data' Analytics

Facebook Is About To Launch A Huge Play In 'Big Data' Analytics | Big Data Technology, Semantics and Analytics | Scoop.it
Facebook has data on $1 trillion off-line purchases.
Tony Agresta's insight:

Enhancing your data with third party information has significant benefits.  For one, profiles of your customers or prospects become much more vivid.  In years past, enhancement was limited to adding demographic, socio-economic and lifestyle attributes for the geographic areas in which the customer lived.    Today, the detailed information specific to the information is extraordinary and growing all the time.  Targeted advertising becomes much more precise.  Using the enhanced data as inputs into predictive models can improve the accuracy of the models. 

Real time access to data helps both government and commercial organizations.  Access to public records data on personal property owned, places of residence and employment history are often used in investigations analysis.  This could lead to identifying related individuals comprising a network of activity. 


Big data includes more than just your internal data.   Third party data can play a significant role in your big data targeting strategy.  One word of caution - when adding third party data, look closely at whether or not the data is specific to the person or inferred about the person based on similar profiles or models.   Typically vendors have this for each data characteristic and will provide it at the macro level.  Understanding how the inferred data is added will be important as well.  

If you’re trying to decide how significant a role the enhanced data can play in your targeting strategy, you could always enhance a sample, test the results and see which of the variables has some explanatory power before you enhance massive amounts of data and burn through your budget.


more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

MarkLogic points up its Hadoop big-data future | ZDNet

MarkLogic points up its Hadoop big-data future | ZDNet | Big Data Technology, Semantics and Analytics | Scoop.it
Open-source distributed computing software Hadoop features strongly in the plans of NoSQL database company MarkLogic.
Tony Agresta's insight:

Ths article includes important information about the benefits of tiered storage in managing Big Data as well as MarkLogic's relationship with Intel.   Layering your data across different storage tiers provides flexibility, performance gains and cost management benefits.  Service levels can be optimized. 

more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

US patent office embraces big data

US patent office embraces big data | Big Data Technology, Semantics and Analytics | Scoop.it
For inventors and companies getting a patent or trademark the process can be long and difficult. Patent applications have their own language, and both users and application reviewers alike have to ...
Tony Agresta's insight:

A quote from the article:  “The patent office is trying to make patents more accessible, there is a large backlog of people applying, and a lot of that is because of the process,” explains Rick Miller, Director Federal, State & Local for MarkLogic in an interview with CivSource. “The key benefit to putting manuals online for patent applicants, is so they can learn if their application is patentable before going through the whole process.”

more...
No comment yet.
Rescooped by Tony Agresta from NoSQL Databases
Scoop.it!

Land Registry provides linked data - Land Registry for England and Wales

UK Government Agency responsible for the House Price Index, Property Prices, Property Ownership, Title Deeds, Title Plans, Title Registers.

Via Adrian Carr
more...
Adrian Carr's curator insight, April 25, 2013 8:52 AM

England & Wales Land registry make their data available to the public.

The Open data revolution continues a pace and this is clear evidence that there is substantial amounts of open data available which does not contain sensitive information.

 

The next stpe of the revoution is for companies to leverage this data, combine it with their own ip and make money from it.

 

When people ask what is "Big Data" all about...it is this. 

Adam Fowler's curator insight, May 3, 2013 7:20 AM

More UK Linked Data innovation!

Rescooped by Tony Agresta from NoSQL Databases
Scoop.it!

Gartner's basic business rules for capitalizing on big data

Gartner's basic business rules for capitalizing on big data | Big Data Technology, Semantics and Analytics | Scoop.it
A breakthrough like big data comes around only once in a blue moon. Here are some basic business rules on what executives need to do to capitalize on it.

Via Adrian Carr
more...
Adrian Carr's curator insight, April 25, 2013 10:00 AM

Doug Laney of Gartner continues to drive out the ghosts of "Big Data" and pin it's feet firmly to the ground (mixing metaphors ? - maybe).

A pragmatic approach to grounding objectives in business necessity, starting with internal data sources (not all Big Data projects start with collecting Tweets and blogs..) and then evolve.

 

I am turning into a big fan of Doug's.

Bryan Borda's curator insight, April 26, 2013 9:29 AM

Excellent insights from Gartner.

Scooped by Tony Agresta
Scoop.it!

Big Data: The New Natural Resource | The Big Data Hub

Big Data: The New Natural Resource | The Big Data Hub | Big Data Technology, Semantics and Analytics | Scoop.it
Infographic: Big data is the digital convergence of structured data found inside databases, and unstructured data flowing from new sources like social networks, mobile devices, sensors, RFID, smart meters and financial systems.
Tony Agresta's insight:

Interesting metrics on how fast Big Data is growing.

more...
Adrian Carr's curator insight, April 23, 2013 7:51 AM

Intresting infographic regarding the growth in Big Data

Scooped by Tony Agresta
Scoop.it!

It's the Shape, Not the Size - of the Data - that Matters

It's the Shape, Not the Size - of the Data - that Matters | Big Data Technology, Semantics and Analytics | Scoop.it
Community Blogs, comments and opinions by industry professionals
Tony Agresta's insight:

Great insights from Amir Halfon on Enterprise NoSQL and some of the challenges organizations face today with respect to relational databases.   Here's a quote from the blog post


"The answer lies within a different category of technology called Enterprise NoSQL, which has been designed and built with transactions and enterprise features from the ground up, just like relational databases. But unlike those, an Enterprise NoSQL database models the data as hierarchical trees rather than rows and columns. These trees are aggressively indexed in-memory as soon as the data is ingested, and then used for both element retrieval and full text search, unifying two concepts that have traditionally been separate - the database and the search engine."

more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Sequoia, Tenaya lead $25M funding of Big Data platform provider MarkLogic - Silicon Valley Business Journal

Sequoia, Tenaya lead $25M funding of Big Data platform provider MarkLogic - Silicon Valley Business Journal | Big Data Technology, Semantics and Analytics | Scoop.it
Big Data platform provider MarkLogic Corp. said on Wednesday it has raised $25 million...
Tony Agresta's insight:

Good news here for MarkLogic!

more...
Adam Fowler's curator insight, May 3, 2013 7:19 AM

Great news! Especially as were using some of it to expand the sales and pre-sales organisation that I'm part of!