Complex Insight - Understanding our world
11.0K views | +0 today
Follow
Complex Insight  - Understanding our world
Latest news on complex systems in life sciences, engineering, education and government
Curated by ComplexInsight
Your new post is loading...
Your new post is loading...
Scooped by ComplexInsight
Scoop.it!

NASA - NASA Tournament Lab to Launch Big Data Challenge Series for U.S. Government Agencies

NASA - NASA Tournament Lab to Launch Big Data Challenge Series for U.S. Government Agencies | Complex Insight  - Understanding our world | Scoop.it

NASA, the National Science Foundation and the Department of Energy announced Wednesday the launch of the Big Data Challenge, a series of competitions hosted through the NASA Tournament Lab (NTL). The Big Data Challenge series will apply the process of open innovation to conceptualizing new and novel approaches to using "big data" information sets from various U.S. government agencies. This data comes from the fields of health, energy and Earth science. Competitors will be tasked with imagining analytical techniques and software tools that use big data from discrete government information domains. They will need to describe how the data may be shared as universal, cross-agency solutions that transcend the limitations of individual agencies. Click on the image or the title to learn more.

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

TechAmerica Foundation : TechAmerica Foundation Big Data Commission

TechAmerica Foundation : TechAmerica Foundation Big Data Commission | Complex Insight  - Understanding our world | Scoop.it

How do you use Big Data to make intelligent decisions? How will agencies effectively govern and secure huge volumes of information, while protecting privacy and civil liberties? And perhaps most importantly, what value will it really deliver to the US Government and the citizenry we serve?The Big Data Commission released its findings October 3rd to help answer these questions and many more. The report, “Demystifying Big Data: A Practical Guide to Transforming the Business of Government,” provides the government’s senior policy and decision makers with a comprehensive roadmap to using Big Data for better government. Click on the image or the title to open the page with  additional information and links to downloadable copy of the report.

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Who's Really Using Big Data

Who's Really Using Big Data | Complex Insight  - Understanding our world | Scoop.it

Interesting survey results on reality of big data adoption.A survey of executives at Fortune 1000 companies and large government agencies about where they stand on Big Data. Three high-level takeaways.First, the people we surveyed have high hopes for what they can get out of advanced analytics. Second, it's early days for most of them. They don't yet have the capabilities they need to exploit Big Data. Third, there are disconnects in the survey results — hints that the people inside individual organizations aren't aligned on some key issues.  Only 31% of IT respondents rank the "current role of Big Data" in their company as Planned or at Proof-of-Concept. CLick on the HBR image or the title for more info.

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Big Data on Heroku – Hadoop from Treasure Data

Big Data on Heroku – Hadoop from Treasure Data | Complex Insight  - Understanding our world | Scoop.it

Heroku is a cloud platform as a service (PaaS) owned by Salesforce.com. Heroku supports a range of programming languages for SaaS deployment and a long list of addons including – among others – RDBMS and NoSQL capabilities and Hadoop-based data warehouse developed by Treasure Data. This is a good introductory article on getting started with Heroku and TreasureData Hadoop map-reduce solution. Click on the image or the title to learn more...

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Apache Accumulo

The Apache Accumulo™ sorted, distributed key/value store is a robust, scalable, high performance data storage and retrieval system. Apache Accumulo is based on Google's BigTable design and is built on top of Apache Hadoop, Zookeeper, and Thrift. Apache Accumulo features a few novel improvements on the BigTable design in the form of cell-based access control and a server-side programming mechanism that can modify key/value pairs at various points in the data management process. Click on title to learn more..

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

PLoS Computational Biology: Which of Our Modeling Predictions Are Robust?

PLoS Computational Biology: Which of Our Modeling Predictions Are Robust? | Complex Insight  - Understanding our world | Scoop.it

An interesting paper from Rob. j. De Boer on modelling predictions. If you are involved in predictive analytics or systems modeling and simulation - this is worth taking the time to read.  In theoretical ecology it is well known that the steady state expressions of the variables in a food chain crucially depend on the parity of the length of the chain. This poses a major problem for modeling real food webs because it is difficult to establish their true number of trophic levels, with sometimes rare predators and often rampant pathogens. Similar problems arise in the modeling of chronic viral infections. This Perspective aims to open the discussion on this problem..  Click on the title to learn more...

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

R3 Map Reduce framework by heynemann

R3 Map Reduce framework by heynemann | Complex Insight  - Understanding our world | Scoop.it

r⊃3; is a map reduce engine written in python using a redis backend. It's purpose is to be simple and easy to set up. r⊃3; has only three concepts to grasp: input streams, mappers and reducers.

 

On first glance  - looks useful. Click on image or headline to learn more...

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

'Governments that embrace open data will also switch to open source' | Joinup

Definitely a trend occuring in Europe, be interesting to see how it takes root elsewhere. Public administrations that grasp the benefits of making publicly available their data will also increase their use of free and open source, experts on open data agree. Open data and open source face comparable threats: initial lack of support and a fear for the impact on the organisation. Click on title to learn more.

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Impact of Big Data – Geoffrey Moore Keynote

Impact of Big Data –  Geoffrey Moore Keynote | Complex Insight  - Understanding our world | Scoop.it

Geoffrey Moore's keynote at Hadoop Summit 2012. Moore is famous for the Crossing the Chasma and Into the Tornado models of technology adoption.  If you are tracking Hadoop and BigData markets, Moore's keynote at Hadoop market is worth watching. He concludes that BigData/Hadoop is still in the early market and not at the bowling alley stage. He concludes that the industry can tell it is crossing the chasm when it focus on use cases with domain expertise in the outcomes of the target market (.e. g optimization of Healthcare processing, or optimization of financial processing etc) as opposed to Batch analysis or Map reduce. Insightful and worth watching. click on the elephant or the title to learn more..

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

big data and the geoweb | ryan burns

big data and the geoweb | ryan burns | Complex Insight  - Understanding our world | Scoop.it

Insightful blog post from the – university of washington's Ryan Burns on the realtionship between BigData and GeoWeb and its implications.

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Google's compute engine versus Microsoft Azure

If you are looking at hosting costs for cloud computing it quickly gets really complicated to make comparisons. Great article on thenextweb.com compares Google's compute engine eith Microsoft Azure. If you need the details - read the article. Takeaway for linux hosting compute cycles are similarly priced - Google is slightly cheaper and offers a lot more RAM than Microsoft machines. So if you are doing Memory intensive compute and need cloud - Google Compute Engine is the way to go for now. Click on the title for more info.

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Jenna Burrell | Assistant Professor, School of Information, UC-Berkeley

Jenna Burrell | Assistant Professor, School of Information, UC-Berkeley | Complex Insight  - Understanding our world | Scoop.it

Jenna Burrel is a sociologist and assistant professor in the School of Information at UC-Berkeley. If you are interested in the intersection of socioeconmics, ethnography, big data and good research her blog is a must read.  Also her latest article “Technology Hype vs. Enduring Uses: A longitudinal study of Internet use among early adopters in an African city” is out this month in the wonderful open access journal First Monday. Click on the image or headline to learn more..

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Royal Society wants to see equal value in datasets and journal articles

Royal Society wants to see equal value in datasets and journal articles | Complex Insight  - Understanding our world | Scoop.it

A timley and important argument as Open Data underpins more science and decision making. Britain's Royal Societyargues that high-quality and publicly accessible datasets should be given as much credit as standard publications in the research excellence framework.Read more: http://www.insidehighered.com/news/2012/06/22/royal-society-wants-britain-see-equal-value-datasets-and-journal-articles#ixzz1yXsks2Jz

 

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Microsoft applies big data toward cataloging endangered species

Microsoft applies big data toward cataloging endangered species | Complex Insight  - Understanding our world | Scoop.it

Microsoft has created a new application using big data technicues to help identify current and future species extinction threats, allowing scientists with the International Union for Conservation of Nature (IUCN) to define conservation priorities on the Red List of Threatened Species. Click on the image or title to learn more.

more...
No comment yet.
Rescooped by ComplexInsight from Digital and smart cities
Scoop.it!

Big data for Smart cities – How do we go from Open Data to Big Data for Smart cities

Big data for Smart cities – How do we go from Open Data to Big Data for Smart cities | Complex Insight  - Understanding our world | Scoop.it

Snagged this from the excellent Digital Cities Scoop team. In many cases, Smart cities are linked to the idea of ‘data’. For instance, Open data and sensor data provide the foundation for making cities smarter by enabling new services and acting as a feedback loop for improving existing services. However, data itself does not constitute value. Data must go through a platform that enables value added services. Good article - click on the image or the title to learn more.


Via estratic, Patrick Genoud
more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Hadoop Won't Displace Data Management Systems: Cloudera

ALways nice to here vendor CEO's state sense. In this article its Cloudera's CEO Mike Olson making sense. According to Olson, big data platform Hadoop will complement, not replace, legacy systems,. "Almost without exception, when we see Hadoop in real customer deployments, it's stood up next to existing infrastructure that's aimed at existing business problems," said Olson. He added, "In my view, it's unlikely that a brand-new entrant to the market is going to displace (legacy) tools for established workloads." Click on the title to learn more.

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Building data science teams - O'Reilly Radar

Building data science teams - O'Reilly Radar | Complex Insight  - Understanding our world | Scoop.it

Great article from Oreilly on how to build data scienceteams that cam create great data products,  what skills and traits you need in your team and what type of environment (technical and working) needed to make it a success.  If like us you are involved with working with data science teams on a regular basis - either helping create, manage or consult with them - this is a good article to read and pass on. To learn more click on the title or the image.

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Pragmatic Programming Techniques: BIG Data Analytics Pipeline

Pragmatic Programming Techniques: BIG Data Analytics Pipeline | Complex Insight  - Understanding our world | Scoop.it

Good article by Ricky Ho on big data analytics architecture patterns. going from typical data processing pipeline through to basic large scale data analytics architecture and implrementation architectures. Worth reading - click on image or title for more information.

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Your Laptop Can Now Analyze Big Data - Technology Review

Your Laptop Can Now Analyze Big Data - Technology Review | Complex Insight  - Understanding our world | Scoop.it

New software makes it possible to do in minutes on a small computer what used to be done by large clusters of computers by using hardisks rather than in memory stores. Carlos Guestrin a computer scientist at Carnegie Mellon's Select Lab who led developmet of GraphChi with student Aapo Kyrola, says  "a Mac Mini running GraphChi can analyze Twitter's social graph from 2010—which contains 40 million users and 1.2 billion connections—in 59 minutes. The previous published result on this problem took 400 minutes using a cluster of about 1,000 computers".  Developments such as GraphChi wil enable researcher in computational biology could do large-scale computations on their PC; a developer working on a data-center algorithm can test it on their laptop before pushing it to the cloud. Tools like GraphChi will let many companies and startups solve all their graph-computing needs on a single machine. It's cost effective, and it drives innovation, too. Learn more about GraphChi here: http://graphlab.org/graphchi/ or click on the image or title to read the orginal Technology Review article.

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Utilities and Other Industries Not Ready for Big Data, Say New Oracle Reports

Utilities and Other Industries Not Ready for Big Data, Say New Oracle Reports | Complex Insight  - Understanding our world | Scoop.it

New evidence is emerging that organizations across many industries are simply not ready for the challenges of big data. Oracle has released two new reports that add hard numbers to mounting concerns about the problems big companies face when dealing with data. This week’s report from Oracle polled 333 North American executives to find out how their companies and organizations are handling the onslaught of big data, both in terms of setting up big data infrastructure and then properly analyzing the data once they have it gathered. The number one problem? Translating information into actionable intelligence, was repoted by 48% of respondents. I think this is inkeeping wit the earlier post on Geoffrey Moore's presentation on Big Data and where it wss in the technology adoption curve.

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Predictive data analytics in New York City - O'Reilly Radar

Predictive data analytics in New York City - O'Reilly Radar | Complex Insight  - Understanding our world | Scoop.it

A predictive data analytics team in the Mayor's Office of New York City has been quietly using data science to find patterns in regulatory data that can then be applied to law enforcement, public safety, public health and better allocation of resources. In a good interview Michael Flowers explains why applying data science to regulatory data is necessary to use city resources better. This is the other aspect of smart cities - better analytics and models informing policy and practise. Worth readig. Click on the image or headline to learn more

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

How to Mine Unstructured Data - Information Management Newsletters Article

How to Mine Unstructured Data - Information Management Newsletters Article | Complex Insight  - Understanding our world | Scoop.it

Mining Unstructured Data: Difficult, but Not Impossible. Interesting highlevel overview of steps to mine unstructured data for analysis.

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Open Data in Development Aid

This European Public Sector Information Platform topic report focuses on the question of how improved access to and analysis of data can help increase transparency, accountability and effectiveness to improve our understanding of how aid can be made more effective. Link to PDF file on web page. Click the headline to learn more...

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Google introduces Compute Engine, Google-scale Linux virtualization

From ArsTechnica: Google has announced a new cloud service Compute Engine, offering large-scale Linux virtualization on Google's infrastructure.

Google boasts that Compute Engine offers up to 50 percent more compute power per dollar than competing IaaS offerings. For applications with low bandwidth and I/O demands, Compute Engine can offer hundreds of thousands of cores; Google demonstrated a genetic application running on 600,000 cores. 

more...
No comment yet.
Scooped by ComplexInsight
Scoop.it!

Using semantic technology to turn big data into smart data

Using semantic technology to turn big data into smart data | Complex Insight  - Understanding our world | Scoop.it

 Second part of an interesting interview with David Saul, Chief Scientist at State Street Corp. He talks about the challenges and tradeoffs of getting a semantic database off the ground, and how semantic technology could produce really big benefits for many companies, thanks to his and others' efforts now under way to develop standards. Given the role of semantics in large scale data analysis - certainly worth a read. Learn more by clicking on the image or the headline.

more...
No comment yet.