Linked & Open Data
89 views | +0 today
Follow
 
Rescooped by Adrian Carr from Big Data Technology, Semantics and Analytics
onto Linked & Open Data
Scoop.it!

The Best Database for Hadoop - Optimize Cost, Performance and Data Availability


Via Tony Agresta
Adrian Carr's insight:

As Hadoop comes to the top of the Gartner Hype Cycle and people and organisations begin to realise the limitations (as well as strengths) of the technology, the timing is perfect for MarkLogic to launch software to fill the gap.

Hadoop is a file store system, not a database.  Which means to use it effectively, especially for Enterprise applications, there is a lot of functionality which is required on top.

This is where MarkLogic comes in.

MarkLogic IS the database with all the 'Enterprise' functionality you would expect (High Availability, DR, ACID Transactions etc).  With the launch of Tiered Storage, MarkLogic can archive data into Hadoop storage using commodity storage whilst keeping track of it and being able to retrieve it.

Yes - MarkLogic is a proprietary software package but history has shown that to be the route to most effcetive long term support and development versus toolkit/build projects with their expensive and eventually impractical support burden and new feature drought.

Take a look at MarkLogic.com and download a free developer version.

more...
Tony Agresta's curator insight, August 1, 2013 11:52 AM

Not all data is created equal.  Some is active and absolutely essential in solving real time problems.  And some may be needed in the future but, for now, can occupy fewer resources while keeping your costs down.   This presentation discusses how organizations can optimize cost, performance and availability using tiered storage with Hadoop and MarkLogic.  It demonstrates how you can have the best of both worlds - real time access to mission critical data AND the ability to immediately activate your long tail data stored in HDFS as needed. 


The benefits include less data movement, less ETL, the ability to index your data once, selectively mount data for real time usage and cost effective storage options. 


Today, more than ever, organizations need the flexibility to manage their data efficiently.  Why not store MarkLogic data directly in the Hadoop File System, apply Map Reduce to operate on that data through batch processing and then mount any portion of that data in MarkLogic for real time access?  Mixing real time and batch workloads allows you to manage your low density, active data in an enterprise environment with replication and high availability while also maintaining all other data in Hadoop.  This video is worth watching.


Your new post is loading...
Your new post is loading...
Scooped by Adrian Carr
Scoop.it!

Our World in Data

Our World in Data | Linked & Open Data | Scoop.it
Visualised in graphs I am presenting the long-term data on how we are changing our world. This is the Empirical View on How We Are Making Our World a Better Place.
Adrian Carr's insight:
A great collection of interactive graphs and data collected from numerous sources showing 'Our World in Data' - populations growth; mortality rates; GDP growth'; nuclear arms proliferation; impact of wars.
Hours of fun for the whole family.
more...
No comment yet.
Scooped by Adrian Carr
Scoop.it!

Linked Data: Best Thing to Happen to Semantic Web | MarkLogic

Visit MarkLogic.com to read Linked Data: Best Thing to Happen to Semantic Web.
Adrian Carr's insight:

Interest in Linked Data, and the Semantic Web in general, has grown consistently as organizations begin to understand how the Semantic Web technologies can be used to enhance services, enrich products, improve user journeys and streamline internal processes.

Philip Fennel of MarkLogic discusses this growth in his blog...

more...
No comment yet.
Rescooped by Adrian Carr from Big Data Technology, Semantics and Analytics
Scoop.it!

The Best Database for Hadoop - Optimize Cost, Performance and Data Availability


Via Tony Agresta
Adrian Carr's insight:

As Hadoop comes to the top of the Gartner Hype Cycle and people and organisations begin to realise the limitations (as well as strengths) of the technology, the timing is perfect for MarkLogic to launch software to fill the gap.

Hadoop is a file store system, not a database.  Which means to use it effectively, especially for Enterprise applications, there is a lot of functionality which is required on top.

This is where MarkLogic comes in.

MarkLogic IS the database with all the 'Enterprise' functionality you would expect (High Availability, DR, ACID Transactions etc).  With the launch of Tiered Storage, MarkLogic can archive data into Hadoop storage using commodity storage whilst keeping track of it and being able to retrieve it.

Yes - MarkLogic is a proprietary software package but history has shown that to be the route to most effcetive long term support and development versus toolkit/build projects with their expensive and eventually impractical support burden and new feature drought.

Take a look at MarkLogic.com and download a free developer version.

more...
Tony Agresta's curator insight, August 1, 2013 11:52 AM

Not all data is created equal.  Some is active and absolutely essential in solving real time problems.  And some may be needed in the future but, for now, can occupy fewer resources while keeping your costs down.   This presentation discusses how organizations can optimize cost, performance and availability using tiered storage with Hadoop and MarkLogic.  It demonstrates how you can have the best of both worlds - real time access to mission critical data AND the ability to immediately activate your long tail data stored in HDFS as needed. 


The benefits include less data movement, less ETL, the ability to index your data once, selectively mount data for real time usage and cost effective storage options. 


Today, more than ever, organizations need the flexibility to manage their data efficiently.  Why not store MarkLogic data directly in the Hadoop File System, apply Map Reduce to operate on that data through batch processing and then mount any portion of that data in MarkLogic for real time access?  Mixing real time and batch workloads allows you to manage your low density, active data in an enterprise environment with replication and high availability while also maintaining all other data in Hadoop.  This video is worth watching.


Scooped by Adrian Carr
Scoop.it!

Open public data can drive innovation and create new opportunities.

Open public data can drive innovation and create new opportunities. | Linked & Open Data | Scoop.it
More and more governments are opening up their public data sets for individuals and organisations to use and create new business from it.
Adrian Carr's insight:

EU and Dutch government activities to leverage the value of their data.

This leads Mark's article to focus on teh areas where this new business area are developing rapidly thanks to innovators such as Google and Amazon.

This is just the beginning of what will be a multi billion dollar business sector..

more...
No comment yet.
Scooped by Adrian Carr
Scoop.it!

Welcome to Data.ac.uk | Data.ac.uk

Adrian Carr's insight:

This new site intends to become the single point of contact for Linked Open data for academia.

more...
No comment yet.
Scooped by Adrian Carr
Scoop.it!

DataShine: Census

DataShine: Census | Linked & Open Data | Scoop.it
The Census 2011, mapped with context for England/Wales.
Adrian Carr's insight:

Datashine is a lovely interactive site based on 2011 Census data.

Great presentation with interactive tools to home in on all sorts of population characteristics by UK area.  A combination of Big Data and Analytics/Visualisation bringing citizen data to life.

more...
No comment yet.
Scooped by Adrian Carr
Scoop.it!

Worldwide Open Data Sites | DataRemixed

Worldwide Open Data Sites | DataRemixed | Linked & Open Data | Scoop.it
Adrian Carr's insight:

A great Tableau tool showing Open Data sites around the world.  A great looking, simple to use tool.  Nice work Tableau

more...
No comment yet.
Rescooped by Adrian Carr from Big Data Technology, Semantics and Analytics
Scoop.it!

The Age of Big Data - Predicting Crime After Shocks

 

 


Via Tony Agresta
more...
Tony Agresta's curator insight, August 7, 2013 9:25 AM

The BBC documentary follows people who mine Big Data, including the Los Angeles Police Department (LAPD) who uses data to predict crime.  It's proven that historical patterns can be used to predict future behavior.  With a database of over 13 million crimes spanning 80 years and real time continuous updates, the LAPD has applied mathematical algorithms and pattern recognition to identify crime hotspots.   Targeted police work has resulted in a 26% decrease in burglaries and a 12% decrease in property crimes. 

 

How does this work?  In the same way that earthquake aftershocks can be predicted, data miners analyzed historical crime statistics including location and timing.  They found patterns in the big data crime landscape.  By tracking the history, timing and location of crimes, they revealed that the probability another crime would occur in certain locales was higher.  They discovered patterns in the data.  In this case, the rate of crime and geospatial distribution of events were excellent predictors of future behavior including pinpointing small geograpic areas which they used to direct police resources. 

 

Today, these predictive aftershocks are becoming more accurate through the use of real time data feeds, alerts, geospatial analysis and temporal analysis.  Over 150 cities in the US are starting to apply these techniques allowing police officers to anticipate, focus, apprehend and therefore lower risk.   


Thanks to KD Nuggets for providing the link to the BBC video which is very well done.

Scooped by Adrian Carr
Scoop.it!

Open Data Policy by ACT Government

Adrian Carr's insight:

(Aussie) ACT Government Draft Open Data Policy.

A good step forward.

more...
No comment yet.
Scooped by Adrian Carr
Scoop.it!

Linked Open Data - What is it?

Linked Open Data - What is it? | Linked & Open Data | Scoop.it
Simple animation to explain what Linked Open Data is and why it's a good thing, both for users and for data providers. To find more information about Europeana's linked data pilot, visit data.europ...
Adrian Carr's insight:

This is a great intro to what Linked Open Data is and why it is important.

A very easy to follow video from the europeana.eu organisation which explains why this will be a huge game changer in the way the species learns and develops.

A bold statement - yes but too bold ?

You decide.

more...
Intriguing Networks's curator insight, July 17, 2013 2:43 AM

More connections and discovery so worth all business thinking about how to best incorporate Linked Data as part of Digital Plans.