My view on (High Performance) Scientific Computing
209 views | +0 today
Follow
Your new post is loading...
Your new post is loading...
Scooped by David Colignon
Scoop.it!

Playing with Shifter Part 2 – converted Docker containers inside Slurm | The Musings of Chris Samuel

Playing with Shifter Part 2 – converted Docker containers inside Slurm | The Musings of Chris Samuel | My view on (High Performance) Scientific Computing | Scoop.it
Playing with Shifter Part 2 - converted Docker containers inside #Slurm (with IB!)

https://t.co/rcG4WBldyj

#HPC via @chris_bloke
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

Reproducible bioinformatics pipelines using Make

Aim to commit “atomic” changes to your pipeline. This means you should usually run make test before committing your changes so that regressions don’t need to be fixed in subsequent commits.

more...
No comment yet.
Scooped by David Colignon
Scoop.it!

Big Dimensions, and What You Can Do About It

Big Dimensions, and What You Can Do About It | My view on (High Performance) Scientific Computing | Scoop.it

Data is abundant, data is big, and big is a problem. Let me start with an example. Let's say you have a list of movie titles and you want to learn their genre: romance, action, drama, etc.

more...
No comment yet.
Scooped by David Colignon
Scoop.it!

Big RAM is eating big data – Size of datasets used for analytics | Data Science Los Angeles

Big RAM is eating big data – Size of datasets used for analytics | Data Science Los Angeles | My view on (High Performance) Scientific Computing | Scoop.it

With so much hype about "big data" and the industry pushing for "big data" analytical tools for everyone, the question arises how many people have big data (for

more...
No comment yet.
Scooped by David Colignon
Scoop.it!

11 Lesser known facts about Data Science that everyone must know.

11 Lesser known facts about Data Science that everyone must know. | My view on (High Performance) Scientific Computing | Scoop.it
Are you a data scientist/analyst or planning to make a career in analytics? If yes, then you must not miss out on these 11 important facts about data science.
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

The Cloud Versus HPC Cluster Cost Conundrum - The Platform

The Cloud Versus HPC Cluster Cost Conundrum - The Platform | My view on (High Performance) Scientific Computing | Scoop.it
The build versus buy argument for high performance computing clusters has gathered steam lately, in part because some of the critical missing pieces both p
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

2015 Bio-IT Trends From the Trenches

Talk slides from my annual address at the Bio-IT World Expo & Conference where I cover trends, best practices and emerging pain points for life science focused…
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

Easy cluster parallelization with ZeroMQ | mdup

Easy cluster parallelization with ZeroMQ | mdup | My view on (High Performance) Scientific Computing | Scoop.it
Easy cluster parallelization with ZeroMQ | mdup
http://t.co/X24Sv3kXTP
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

ONELAB - ONELAB

ONELAB - ONELAB | My view on (High Performance) Scientific Computing | Scoop.it

ONELAB (Open Numerical Engineering LABoratory) is a lightweight interface to finite element software for engineering applications. It allows calling sequences of independent clients (e.g. mesh generators, finite element solvers and other related tools) and have them share parameters and modeling information.

Give it a try!
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

Best Practices for Scientific Computing

Best Practices for Scientific Computing | My view on (High Performance) Scientific Computing | Scoop.it
We describe a set of best practices for scientific software development, based on research and experience, that will improve scientists' productivity and the reliability of their software.
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

Compilers and More: Is Amdahl's Law Still Relevant? - HPCwire (blog)

Compilers and More: Is Amdahl's Law Still Relevant? - HPCwire (blog) | My view on (High Performance) Scientific Computing | Scoop.it
From time to time, you will read an article or hear a presentation that states that some new architectural feature or some new programming strategy will le
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

GNU Parallel script processing and execution - YouTube

GNU Parallel version 20100620 http://www.gnu.org/software/parallel/ is a shell tool for executing jobs in parallel locally or using remote machines. A job is...
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

Glenn K. Lockwood: Basics of I/O Benchmarking

Glenn K. Lockwood: Basics of I/O Benchmarking | My view on (High Performance) Scientific Computing | Scoop.it
Basics of benchmarking parallel file systems by @glennklockwood https://t.co/fPG4HKsMB8
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

Self-Documented Makefile

Self-Documented Makefile | My view on (High Performance) Scientific Computing | Scoop.it
Good projects are automated projects. But documenting all those Make targets in a README file doesn't make much sense, when you can make the Makefile auto-documented.
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

GNU Parallel, A Million Text Files And A Single Laptop

GNU Parallel, A Million Text Files And A Single Laptop | My view on (High Performance) Scientific Computing | Scoop.it

Although I make my living doing distributed computing, not every solution requires millions of dollars of hardware. Using GNU Parallel is often "enough".

more...
No comment yet.
Scooped by David Colignon
Scoop.it!

Technical Debt Could Leave Machine Learning Bankrupt

Technical Debt Could Leave Machine Learning Bankrupt | My view on (High Performance) Scientific Computing | Scoop.it
There has been a resounding uptick in attention around machine learning, but with relatively few large-scale systems in production (and even fewer public s
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

Is Parallel Programming Hard, And, If So, What Can You Do About It?

Is Parallel Programming Hard, And, If So, What Can You Do About It? http://t.co/91HDJl5SiK
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

The Price of Open-source Software - a Joint Response - insideHPC

The Price of Open-source Software - a Joint Response - insideHPC | My view on (High Performance) Scientific Computing | Scoop.it
Members of the HPC community discuss the the role of open source software in publicly funded research.
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

"What is Code?" is a 38000 word primer on programming that everyone should read

"What is Code?" is a 38000 word primer on programming that everyone should read | My view on (High Performance) Scientific Computing | Scoop.it
"What is Code?" is a 38,000 word piece written for Bloomberg Businessweek that sets out to answer its titular question in a clear yet comprehensive manner. Penned by programmer Paul Ford, th..
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

Argonne MPI Tutorials

Tomorrow, @argonne hosts a day-long public tutorial on parallel programming with MPI. Info/register here: http://t.co/NVyYNJnPDT
David Colignon's insight:

The slides are available

more...
No comment yet.
Scooped by David Colignon
Scoop.it!

Supercomputing Quantum Mechanics - insideHPC

Supercomputing Quantum Mechanics - insideHPC | My view on (High Performance) Scientific Computing | Scoop.it
Supercomputers in the USA, Europe, and Japan have proved indispensable for research into the effects of quantum mechanics at hugely different length scales.
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

Calculons… librement

Calculons… librement | My view on (High Performance) Scientific Computing | Scoop.it

Les mondes académique et industriel partagent un besoin grandissant de logiciels de calcul scientifique, dans des domaines aussi variés que l’ingénierie électromécanique et aéronautique, la chimie ou la biomédecine. Ces logiciels et les simulations qu’ils rendent possibles sont devenus indispensables à la conception de la plupart des produits et procédés innovants sur lesquels se base notre mode de vie moderne. Si le coût des licences de tels logiciels commerciaux est justifié pour les grandes entreprises qui en font un usage intensif, notre expérience des 15 dernières années montre qu’une utilisation plus occasionnelle peut difficilement en justifier le coût. Cet état de fait conduit nombre de petites et moyennes entreprises à ne pas recourir à ces outils, bridant significativement leur potentiel d’innovation. Et il a pour conséquence que bon nombre d’universités et d’écoles supérieures doivent se passer du formidable apport pédagogique que les moyens de simulation numérique modernes peuvent offrir à leur enseignement.

more...
No comment yet.
Scooped by David Colignon
Scoop.it!

On Eigenfaces: Creating ghost-like images from a set of faces.

On Eigenfaces: Creating ghost-like images from a set of faces. | My view on (High Performance) Scientific Computing | Scoop.it
On Eigenfaces: Creating ghost-like images from a set of faces.
http://t.co/cWFjrAh7Lg
more...
No comment yet.
Scooped by David Colignon
Scoop.it!

nucleotides · genome assembler benchmarking

continuous, objective and reproducible evaluation of genome assemblers using docker containers (I will presenting in the HPC track at #labtech2014.
more...
No comment yet.