Big Data Technolo...
Follow
Find
8.7K views | +0 today
Big Data Technology, Semantics and Analytics
Trends, success and applications for big data including the use of semantic technology
Curated by Tony Agresta
Your new post is loading...
Your new post is loading...
Scooped by Tony Agresta
Scoop.it!

How Big Data is Being Used to Help Raise the IQ of School Buildings « A Smarter Planet Blog

How Big Data is Being Used to Help Raise the IQ of School Buildings « A Smarter Planet Blog | Big Data Technology, Semantics and Analytics | Scoop.it
By Christopher Luongo More school systems across the U.S.
more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Financial Firms Overhaul Data Management Strategies

Financial Firms Overhaul Data Management Strategies | Big Data Technology, Semantics and Analytics | Scoop.it
As capital markets organizations focus on ways to reduce risk, data management is receiving a top-to-bottom makeover.
Tony Agresta's insight:

"In the current environment, smaller is sometimes better. For instance, firms are no longer spending money on gigantic warehouses and agreeing on a single data model that would fit all the data in one place," says Amir Halfon, CTO of financial services at MarkLogic, a provider of database technology. "It's that aspect that introduces the most cost. The traditional approach of a big data warehouse doesn't cut it anymore."

more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Personalized Learning Is Sweeping College Campuses, Courtesy of Big Data

Personalized Learning Is Sweeping College Campuses, Courtesy of Big Data | Big Data Technology, Semantics and Analytics | Scoop.it
Data mining and analytics teach educators about their students.
more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Consulting Services | MarkLogic

Consulting Services | MarkLogic | Big Data Technology, Semantics and Analytics | Scoop.it
Visit MarkLogic.com to read Consulting Services.
Tony Agresta's insight:

"Consulting Services team members are Big Data experts with vast experience solving challenges with data volume, variety, velocity, and complexity. They are working with the world’s most innovative organizations on projects with lives at stake and trillions of dollars on the line. Consulting Services is constantly evolving its methodology to capture best practices and lessons learned. This dedication to perfection enables MarkLogic customers to bring Big Data Applications to market faster and cheaper than their competitors."

more...
Scooped by Tony Agresta
Scoop.it!

Professional Sports & Big Data - Gaining the Competitive Edge via Analytics and Understanding

Professional Sports & Big Data - Gaining the Competitive Edge via Analytics and Understanding | Big Data Technology, Semantics and Analytics | Scoop.it
The NFL season has just ended and a new era of Big Data analytics is just beginning.
more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

DNA - The New Big Data Storage Solution of the 21st Century? | CloudTimes

DNA - The New Big Data Storage Solution of the 21st Century? | CloudTimes | Big Data Technology, Semantics and Analytics | Scoop.it
Scientists are constantly looking for alternative methods of storing information that will ensure long-term preservation of data at low cost and operating cost.
more...
No comment yet.
Rescooped by Tony Agresta from visual data
Scoop.it!

[INFOGRAPHIC] BIG DATA: What Your IT Team Wants You To Know

[INFOGRAPHIC] BIG DATA: What Your IT Team Wants You To Know | Big Data Technology, Semantics and Analytics | Scoop.it

The purpose of Big Data is to supply companies with actionable information on any variety of aspects. But this is proving to be far more difficult than it looks with over half of Big Data projects left uncompleted.


Two of the most often reported reasons for project failures are a lack of expertise in data analysis. Reports show that data processing, management and analysis are all difficult in any phase of the project, with IT teams citing each of those reasons more than 40% of the time.

However, failures in Big Data projects may not solely lie on faulty project management. In a recent survey, a staggering 80% of Big Data’s biggest challenges are from a lack of appropriate talent. The field’s relative infancy is making it hard to find the necessary staff to see projects through, resulting in underutilized data and missed project goals.

IT teams are quickly recognizing a chasm between executives and frontline staffers whose job it is to apply findings from Big Data. In the end,it may not be the anticipated cure-all for 21st century business management. It is only as good as good as the system that runs it.


Via Peter Azzopardi, Berend de Jonge, Lauren Moss
Tony Agresta's insight:

Very interesting infographic.  Why do they fail?  For all of the reasons above and then some...    Over 80% of the data being collected today is unstructured and not readily stored in relational database technology burdened by complex extract, transform and load.  There's also pre-existing data, sometimes referred to as "dark data" that includes documents which need to be included and made discoverable for a host of reasons - compliance and regulatory issues are one.   Log activity and e-mail traffic used to detect cyber threats and mitigate risk through analysis of file transfers is yet another set of data that requires immediate attention.

 

Social and mobile are clearly channels that need to be addressed as organizations continue to mine data from the open web in support of CRM, product alerts, real time advertising options and more.  

 

To accomplish all of this, organizations need a platform with enterprise hardened technology that can ingest all of these forms of data in real time, without having to write complex schemas.   Getting back to the point - What do most projects fail?   If companies attempt to do this with technology that is not reliable, not durable and does not leverage the skills of their existing development organization, the project will fail.  

 

We have seen this time and time again.   MarkLogic to the rescue.   With over 350 customers and 500 big data applications, our Enterprise NoSQL approach mitigates the risk.  Why?  Our technology stack includes connectors to Hadoop, integration with leading analytics tools using SQL, Java and Rest APIs, JSON support, real time data ingestion, the ability to handle any form of data, alerting, in database analytics functions, high availability, replication, security and a lot more.  

 

When you match this technology with a world-class services organization with proven implementation skills, we can guarantee your next Big Data project will work.  We have done it hundreds of times with the largest companies in the world and very, very big data.

 

www.marklogic.com



more...
Olivier Vandelaer's curator insight, January 30, 2013 2:45 AM

Looking at the infographic, it clearly reminds me about the start of "Enterprise Data Warehouse": failures by "Innacurate scope", "Technical Roadblocks" & "Siloed data and no collaboration". It looks so familiar.

Adrian Carr's curator insight, January 30, 2013 10:27 AM

This is a great infographic - it shows that whilst everyone is doing it (it being "Big Data" - whatever that is...), talent is rare, technology is hard to find and the projects never end.  A far cry from the speed with which companies such as the BBC deployed MarkLogic to serve all data for the sport websites through the Olympics.  Now that was big data, delivered by a talented team in a short space of time.

Rescooped by Tony Agresta from RetailFit
Scoop.it!

Technology And BIG Data Are Changing The Retail World - Forbes

Technology And BIG Data Are Changing The Retail World - Forbes | Big Data Technology, Semantics and Analytics | Scoop.it

Via Kun Le
Tony Agresta's insight:

The conclusions in this article are spot on.


  • Streamline data and data gathered for you by market researchers
  • Help you manage, tabulate, analyze, visualize and deliver data in interactive, web-based and mobile formats
  • Provide insight into your target market
  • Offer solutions to pressing business problems.


The only one I would add would emphasize the need to use ALL of your data including new forms of unstructured data created through social and mobile.  Streaming this data in real time allows retailers to react to advertising, sentiment, changes in buying behavior and more.

more...
Edward Chenard's curator insight, January 29, 2013 10:22 AM

And we are at the forefront of this.

Rescooped by Tony Agresta from MarkLogic - Enterprise NoSQL Database
Scoop.it!

What is MarkLogic?

What is MarkLogic? | Big Data Technology, Semantics and Analytics | Scoop.it

 

 


Via Dominic Spitz
more...
Dominic Spitz's curator insight, January 23, 2013 10:55 AM

→ Enterprise NoSQL Database Technology:

For more than a decade, MarkLogic has delivered a powerful and trusted enterprise-grade NoSQL database that enables organizations to turn all data into valuable and actionable information. Key features include ACID transactions, horizontal scaling, real-time indexing, high availability, disaster recovery, government-grade security, and more

→ Best Big Data Search:

Big Data is only valuable if you can find the information that is critical to your organization’s success. Searching across different data types — text, images, date/time, geospatial, and currencies — from multiple, disparate systems is difficult, and often requires weeks of data manipulation and normalization. Only when you have the power to search across all your data are you able to get the full value out of your information. With MarkLogic, you can easily add new information, index “as-is,” and get up to date sub-second search results.

→ Application Development:

 

Application development teams continually face challenges around costs, resources, time-to-market, feature completeness, and quality. Tools that promote rapid development and best practices are critical for meeting the stringent requirements of application initiatives. With the growing challenge of managing unstructured information and Big Data, development efficiency and effectiveness are more important than ever. You can quickly develop advanced database applications that search through terabytes of information to store and retrieve any type of data, using MarkLogic’s rich application development tools and APIs.

→ Analytics & Business Intelligence:

 

Big Data opens up the potential for predictive analysis, competitive and customer intelligence, sentiment analysis, social media surveillance, and so much more. However, not every Big Data technology platform supports real-time Business Intelligence

→ Get Inside the Brains of MarkLogic - MarkLogic Architecture:

Fast, agile, and scalable – that’s what makes MarkLogic the leading database for unstructured information. It fuses together database internals, search-style indexing, and application server behaviors into a unified system. It uses XML documents as its data model, and stores the documents within a transactional repository. It indexes the words and values from each of the loaded documents, as well as the document structure.

→ Real-time Your Hadoop:


Get the best of real-time Big Data Applications with the benefits of batch processing and archival storage by combining Hadoop with MarkLogic. The MarkLogic Connector for Hadoop allows you to easily move data between MarkLogic and Hadoop within applications, creating opportunities to identify new business insights

 

Rescooped by Tony Agresta from MarkLogic - Enterprise NoSQL Database
Scoop.it!

Data Virtualization Solutions | MarkLogic

Data Virtualization Solutions | MarkLogic | Big Data Technology, Semantics and Analytics | Scoop.it
MarkLogic’s enterprise NoSQL database overcomes the limitations of rigid RDBMS-based systems, enabling comprehensive data virtualization.

Via Dominic Spitz
more...
Dominic Spitz's curator insight, January 21, 2013 11:13 AM

Organizations with business critical information scattered across separately owned and managed silos need this data in one place. MarkLogic’s Enterprise NoSQL database overcomes the limitations of rigid RDBMS-based systems, enabling comprehensive data virtualization.

 

The growing volumes of information available today continue to put a heavy work load on data analysts. It is difficult and often too time consuming for analysts to separate the useful information from the rest. They risk drawing incorrect conclusions because they missed buried, relevant information. Or they risk missing a critical deadline or opportunity because it took so long to analyze the data.

 

MarkLogic is the Enterprise NoSQL database that manages all types of data at scale in real time. MarkLogic lets developers build the visualization capabilities that help analysts make quick, informed decisions with Big Data.

Rescooped by Tony Agresta from Big Data, IoT and other stuffs
Scoop.it!

How the Cloud and Big Data Are Changing Entertainment - Forbes

How the Cloud and Big Data Are Changing Entertainment - Forbes | Big Data Technology, Semantics and Analytics | Scoop.it
filmratings.com From Amazon to Wal-Mart, companies are carving up the fast-growing market for direct digital delivery of movies to the living room.

Via Toni Sánchez
Tony Agresta's insight:

"Whether a consumer is browsing the shelves of a traditional video rental store or choosing a movie from the cloud, discoverability is a huge user challenge.


That’s why many of the streaming services are investing in big-data analytics, to help give consumers suggestions of what movies they might want to watch."

more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Big Data, MarkLogic, and Tableau = Greatest Possible Insights

Big Data, MarkLogic, and Tableau = Greatest Possible Insights | Big Data Technology, Semantics and Analytics | Scoop.it
This presentation at the recent Tableau conference was so popular, we just had to repeat it! Fernando Mesa, Chief Technologist, Enterprise, will reprise the presentation in this special webcast.
Tony Agresta's insight:

User adoption of analytics technology is driven by ease of use and robust functionality the result of which are insights you can apply to drive performance. When combined with an Enterprise NoSQL approach that includes native search capabilities, data can be exposed to Tableau in a highly scalable, secure environment that also includes real time analysis, notifications, geospatial search and other angles of analysis. Loading data “as is” using a schema-less database can dramatically shorten the cycles from data ingestion to decisions. This premise is especially important in mission critical applications across healthcare, government, financial services and media markets, to name a few. Hadoop is now part of the MarkLogic platform allowing users to store the data and make it available to MarkLogic “on demand”. This staging area has many benefts including data enrichement and on demand delivery to MarkLogic (and therefore Tableau) for specific sets of data.  MarkLogic data is exposed to Tableau though an ODBC SQL query.  Enhanced data discovery includes all of  your data - including unstructured and semi-structured data in MarkLogic.

more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Build or Buy: Open Source or Enterprise Software | MarkLogic

Looking for a NoSQL database? Should you build or buy, use open source or Enterprise software?
Tony Agresta's insight:

A must read for anyone considering big data technology.  

more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

MarkLogic Simplifies Development of Enterprise-Ready Applications: Free Developer License for MarkLogic(R) Enterprise Edition Now Available

SAN CARLOS, CA, Feb 12 (Marketwire) -- MarkLogic Corporation, provider of the only enterprise NoSQL database,today announced the availability of a free Developer License forMarkLogic(R) Enterprise Edition.
Tony Agresta's insight:
From Gary Bloom: "By providing a free Developer License we enable developers to quicklydeliver reliable, scalable and secure information and analyticapplications that are production-ready," said Gary Bloom, CEO andPresident of MarkLogic. "Many of our customers first experimented withother free NoSQL products, but turned to MarkLogic when they recognizedthe need for search, security, support for ACID transactions and otherfeatures necessary for enterprise environments. Our goal is to eliminatethe cost barrier for developers and give them access to the bestenterprise NoSQL platform from the start."

Learn more here:
http://www.marklogic.com/press-releases/marklogic-simplifies-development-of-enterprise-ready-applications-free-developer-license-for-marklogic-enterprise-edition-now-available/


more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Playtime with MongoDB is Over. Upgrade to MarkLogic Enterprise NoSQL.

Visit MarkLogic.com to read Playtime is Over. Upgrade to Enterprise NoSQL..
Tony Agresta's insight:

Playtime is over.   It's time to upgrade to MarkLogic Enterprise NoSQL.


CIOs, heads of development and the developers themselves are waking up to the realization that when it is time to put these NoSQL applications into the production, there is a tremendous amount of work still needed to get them ready. If they go into production without security, ACID transactions, high availability, backup and recovery—it would be the equivalent of heading down a long and winding road in a toy car. Eventually, the breaks stop working and the wheels fall off.


But while the open-source enthusiasts have been enjoying their pizza and beer and playing with flavor-of-the-month technology, a new class of Enterprise NoSQL developer has emerged. They recognized the need for integrated search, certified security, clustering, replication, failover, alerting, full-text and geospatial indexing, and a suite of application development tools. And, they turned to the only Enterprise NoSQL database that provided those features: MarkLogic.


This new class of Enterprise NoSQL developer starts building out applications confident that they have all of the capabilities they will need in order to deploy their applications into production as soon as they’re done. They don’t have the drudgery of creating workarounds to ensure ACID transactions, building in security, integrating and maintaining 3rd party search engines, or building out frameworks or administration tools. All that work was already done by MarkLogic: who is trusted by enterprises like Citibank, Dow Jones, BBC and every three letter US agency.

more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Upcoming Events — Georgetown Law

Upcoming Events
Tony Agresta's insight:

This may be a good networking event for the MarkLogic public sector team.

more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Big Data “Hype” Coming To An End | SiliconANGLE

Big Data “Hype” Coming To An End | SiliconANGLE | Big Data Technology, Semantics and Analytics | Scoop.it
Tony Agresta's insight:

"Organizations have fascinating ideas, but they are disappointed with a difficulty in figuring out reliable solutions,” writes Sicular from The Gartner Group.

 

"Their disappointment applies to more advanced cases of sentiment analysis, which go beyond traditional vendor offerings.  Difficulties are also abundant when organizations work on new ideas, which depend on factors that have been traditionally outside of their industry competence, e.g. linking a variety of unstructured data sources.”

 

Today, organizations are coming to the realization that free or low cost open source technology to handle big data requires intense development cycles that burn costs and time.  Solving demanding challenges in these four areas has proven difficult:

 

  • Search & Discovery
  • Content Delivery
  • Analytics and Information Products
  • Data Consolidation

 

Organizations need to work with proven technology that's reliable and durable.  They need to work with technology that handles ACID transactions, enterprise security, high availability, replication, real time indexing and alerting - without having to right 10,000+ lines of code. 

 

Major financial institutions, healthcare payors, government agencies, media giants, energy companies, and state & local organizations have standardized on big data technology proven to increase developer productivity, create new revenue streams and address mission critical operations in a post 9-11 era. 

 

You can read about this success by visiting the MarkLogic Customers page or the MarkLogic Solutions page.









more...
Adrian Carr's curator insight, February 11, 2013 11:11 AM

IT does it again.  Build a technology up until we start to believe it will solve all world problems.  It generates huge "science projects" and then everything comes tumbling down.  Finally a voice of reason says...maybe we set expectations unrealistically...One more trough of disillussionment !

Rescooped by Tony Agresta from Big Data and Personalization
Scoop.it!

Predicting the Future By Mining Online News and Other Web Data | MIT Technology Review

Predicting the Future By Mining Online News and Other Web Data | MIT Technology Review | Big Data Technology, Semantics and Analytics | Scoop.it
Prototype software can give early warnings of disease or violence outbreaks by spotting clues in news reports.

Via Fred Thiel, Edward Chenard
more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Debunking NoSQL Database Myths | Data Informed

Debunking NoSQL Database Myths | Data Informed | Big Data Technology, Semantics and Analytics | Scoop.it
There are still myths about exactly what a NoSQL database can and can't do.
Tony Agresta's insight:

If you are interested in big data, please listen to the podcast with Adam Fowler.  He gets to the truth about big data technology. The fact is, enterprise NoSQL databses are available and proven to work. 

more...
Adrian Carr's curator insight, February 11, 2013 11:12 AM

Adam Fowler provides some good answers to what is realin the NoSQL space. The fact is, enterprise NoSQL databses are available and proven to work.  You just have to select carefully and be discerning abotu what is REAL and what is "on the Roadmap" i.e. Not real.

Scooped by Tony Agresta
Scoop.it!

Facebook Data Provides The Most Accurate NFL Fandom Map Ever Created

Facebook Data Provides The Most Accurate NFL Fandom Map Ever Created | Big Data Technology, Semantics and Analytics | Scoop.it
It's long been a dream of mankind to put together a map showing the geographic locations of NFL fanbases. The CommonCensus map tried to do it though polling;
Tony Agresta's insight:

49ers 27, Ravens 24.

more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

MarkLogic Server - Technology Preview: HDFS Storage

An introduction to the HDFS Storage feature available as a technology preview from MarkLogic.
Tony Agresta's insight:

Seamlessly combine the power of MapReduce with MarkLogic’s real-time, interactive analysis and indexing on a single, unified platform.


  • Get more power out of Hadoop. Hadoop and MarkLogic together can allow you to tackle problems that would be difficult or impossible to address by either technology alone.
  • Save money by leveraging common infrastructure. Using MarkLogic and Hadoop Distributed File System (HDFS) enables common batch-processing infrastructure to be used across many different projects and applications.
  • Enterprise-class support for Hadoop. Our partnership with Hortonworks provides a strong, supported platform for building enterprise-class Big Data Applications with Apache Hadoop.


more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Future Big Data: How Analytics Will Impact NFL | SiliconANGLE

Future Big Data: How Analytics Will Impact NFL | SiliconANGLE | Big Data Technology, Semantics and Analytics | Scoop.it
Tony Agresta's insight:

The well-known book and movie documenting the success of Billy Beane and the Oakland A's is probably the best example of using data to provide a completive advantage in sports. 

 

Analyzing player tendencies like pitch sequences, at-bats and defensive moves are interesting.  When you connect them to other players and teams, they become even more interesting and can lead to sets of rules that make up how you coach every detail.  Expectations and patterns taught to players provide them with guidelines on how to react and increase the odds of winning.

 

Could this be applied to football?  It seems like that's what watching film of past games is all about.  If a team could tag the plays with meaningful content about the outcome, the situation, the players on the field, the time, location, weather and then make it discoverable, coaches could identify patterns and tendencies that were previously undetected. 

 

There are probably some other applications of big data in football.  Analyzing new recruits by looking at unstructured data from the open web is one.  Real time twitter streams during game time linked in advertising is likely another.  

 

So who wins?   49ers, 27-24.  Sorry Ray Lewis. Just one ring for you.

more...
No comment yet.
Rescooped by Tony Agresta from Big Data Venture Capital
Scoop.it!

AllAnalytics - Seth Grimes - Scouting the Next Frontiers for 'Unstructured' Data

AllAnalytics - Seth Grimes - Scouting the Next Frontiers for 'Unstructured' Data | Big Data Technology, Semantics and Analytics | Scoop.it
Beyond text, lots of work remains in getting a handle on unstructured data.

Via M. Edward (Ed) Borasky, Bryan St.Laurent
more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

With MarkLogic Search Technology, Factiva Enables Standardized Search And Improved Experiences Across Dow Jones Digital Network, - semanticweb.com

With MarkLogic Search Technology, Factiva Enables Standardized Search And Improved Experiences Across Dow Jones Digital Network, - semanticweb.com | Big Data Technology, Semantics and Analytics | Scoop.it
With MarkLogic Search Technology, Factiva Enables Standardized Search And Improved Experiences Across Dow Jones Digital Network,
Tony Agresta's insight:

Let's not forget that enterprise search is often a critical component in most big data applications.  Users evaluating NoSQL technology should investigate the degree to which the technology supports search.

 

Here are some of the benefits that Dow Jones found in the MarkLogic Enterprise approach:

 

  • One powerful, unified search platform to service the search needs of both consumer and enterprise customers.
  • The enhancements they make will be scalable and efficiently accessible to everyone.
  • Support for dynamic taxonomy coding of content. For example, the same person  (Barack Obama and President Obama) or company (BP and British Petroleum) may be referred to and discovered without requiring the user to specify all variations of what they are seeking.
  • Improved ability to use "proximity of search" elements allowing users to search for words that may be in close proximity to others words thereby increasing the power and relevance of the search results.
  • Fast response to very precise search
  • Personalized search that customizes results using search history patterns and interests
  • Narrowing the search result set to improve direct access to relevant content within the structured or unstructured data, 
  • Time savings through easy detection of related sources of content

 

If this topic interests you, I would suggest looking at the Search capabilities in MarkLogic 6. You can find them here:  Search Developer Guide.   Some of the guide is technical but a lot of it summarizes the types of applications you can build with MarkLogic.  It will provide you with ideas on what you can include in your search applications.

 

 


more...
No comment yet.
Scooped by Tony Agresta
Scoop.it!

Convergence of technologies will spur big data projects in 2013 -- GCN

Convergence of technologies will spur big data projects in 2013 -- GCN | Big Data Technology, Semantics and Analytics | Scoop.it
Cloud computing, Hadoop, NoSQL databases and semantic technology are related layers that will allow agencies to derive more value from big data applications.
Tony Agresta's insight:

“This is the year that government agencies will see that Hadoop handles distribution of computing tasks well, but doesn’t give real-time, immediate response to the user that most of the applications actually require. Agencies will see that NoSQL databases come into play here, giving responses back to the user in real time"  Chris Biow, CTO Federal at MarkLogic

more...
No comment yet.