'Biological' signals ripple through massive cluster management monster
Exclusive One of Google's most advanced data center systems behaves more like a living thing than a tightly controlled provisioning system. This has huge implications for how large clusters of IT resources are going to be managed in the future.
"Emergent" behaviors have been appearing in prototypes of Google's Omega cluster management and application scheduling technology since its inception, and similar behaviors are regularly glimpsed in its "Borg" predecessor, sources familiar with the matter confirmed to The Register.
Emergence is a property of large distributed systems. It can lead to unforeseen behavior arising out of sufficiently large groups of basic entities.
Just as biology emerges from the laws of chemistry; ants give rise to ant colonies; and intersections and traffic lights can bring about cascading traffic jams, so too do the ricocheting complications of vast fields of computers allow data centers to take on a life of their own.
The kind of emergent traits Google's Omega system displays means that the placement and prioritization of some workloads is not entirely predictable by Googlers. And that's a good thing.
"Systems at a certain complexity start demonstrating emergent behavior, and it can be hard to know what to do with it," says Google's cloud chief Peter Magnusson. "When you build these systems you get emergent behavior."