Collective genius: The secret to innovation

Google’s astonishing success in its first decade now seems to have been almost inevitable. But the company’s meteoric growth depended, in large part, on its ability to innovate and scale up its infrastructure at an unprecedented pace.

Mr Bill Coughran, as senior vice-president of engineering, led the group from 2003 to 2011. His 1,000-person organisation built Google’s “engine room” — the systems and equipment that allow us to use Google and its many services 24/7.

When he joined the company in 2003, it had already reinvented multiple times the way it handled Web search and data storage. His group was using Google File System (GFS) to store the massive amount of data required to support Google searches. Given the firm’s ferocious appetite for growth, he knew GFS — once a groundbreaking innovation — would have to be replaced within a couple of years. Building the next-generation system was the job of the systems infrastructure group. It had to create a new engine room in-house while simultaneously refining the current one.

Because this was Mr Coughran’s top priority, one might expect that he would focus first on developing a technical solution for Google’s storage problems and then lead his group through its implementation.

However, to him, there was a bigger problem, a perennial challenge that many leaders inevitably come to contemplate: How do I build an organisation capable of innovating continually over time? He knew the role of a leader of innovation was not to set a vision and motivate others to follow it; it was to create a community that is willing and able to generate new ideas.

THE LINK BETWEEN LEADERSHIP AND INNOVATION

Few companies have the resources of Google at their disposal, but most of them can relate to Mr Coughran’s fundamental challenge.

In 2005, we came together to study exceptional leaders of innovation. They are a diverse lot, but they all think about leadership in a similar way: Leading innovation cannot be about creating and selling a vision to people and then somehow inspiring them to execute it. Instead, leaders can draw out the slices of genius in each individual and assemble them into innovations that represent collective genius.

Innovation usually emerges when a diverse group of people collaborate to generate a wide-ranging portfolio of ideas, which they then refine and even evolve into new ideas through give-and-take and often-heated debates. Thus, collaboration should involve passionate disagreement.

Yet, the friction of clashing ideas may be hard to bear. Often, organisations try to discourage or minimise differences, but this only stifles the free flow of ideas and rich discussion that innovation needs. Leaders must manage this tension to create an environment supportive enough that people are willing to share their genius, but confrontational enough to improve ideas and spark new thinking.

Innovation also requires trial and error. Innovative groups act rather than plan their way forward, and the solutions that emerge are usually different from anything anyone had anticipated. Leaders of innovation create environments that strike the right balance between the need for improvisation and the realities of performance.

Finally, creating something novel and useful involves moving beyond “either-or” thinking to “both-and” thinking. Innovation requires integrating ideas to create a new and better option.

FOSTERING WILLINGNESS AND ABILITY TO INNOVATE

To build willingness, leaders must create communities that share a sense of purpose, values and rules of engagement.

Purpose. Purpose is not what a group does but who is in it or why it exists. It is about a collective identity. Purpose makes people willing to take the risks and do the hard work inherent in innovation.

Shared values. To form a community, members have to agree on what is important. By shaping the group’s priorities and choices, values influence individual and collective thought and action.

We found four that all truly innovative organisations embrace: Bold ambition, responsibility to the community, collaboration and learning.

Rules of engagement. Together with purpose and shared values, rules of engagement keep members focused on what is imperative, discourage unproductive behaviour and encourage activities that foster innovation.

Generally, rules fall into two categories. The first is how people interact, and those rules call for mutual trust, mutual respect and mutual influence. The second is how people think, and those rules call for everyone to question everything, be data-driven and to see the whole.

Innovation requires developing three organisational capabilities: For collaboration, organisations need creative abrasion, or the ability to generate ideas through discourse and debate; for discovery-driven learning, they need creative agility, or the ability to test and experiment through quick pursuit, reflection and adjustment; and for integrative decision-making, they need creative resolution, or the ability to make decisions that combine disparate and sometimes even opposing ideas.

As Mr Coughran began talking with his staff at Google about the need for a new storage system, two self-organising groups of engineers emerged: One wanted to add systems on top of GFS that would handle the new storage needs. This was the Big Table team. The other believed that Google’s new storage requirements were so different from those of search alone that GFS had to be replaced, not adapted. This was the Build from Scratch team.

Mr Coughran gave as much freedom as possible to his engineers, all the while “keeping the reins in enough so we didn’t degenerate into chaos”. He set certain clear expectations: That each team would move forward through rigorous testing of its ideas, and that members would respond to challenges and disagreement with objective data. He rarely had to say “Don’t do that” — words he believed would destroy talent and motivation. Nor did he answer questions directly, despite his expertise. “You want to challenge people to think for themselves,” he said.

After two years, Mr Coughran had to admit that Build from Scratch was not stable enough for the company’s needs and Big Table could not handle the growing array of Google apps, which included YouTube. However, he believed the Big Table approach was more viable in the short term.

Ultimately, Big Table’s storage stack was implemented throughout the company. But Mr Coughran also asked the two most senior engineers in the systems infrastructure group to work on a next-generation system that would eventually replace it. He invited the Build from Scratch team to join the effort, and indeed, some of the ideas developed by its members played key roles in the next-generation system.

Great leaders of innovation see their role not as take-charge direction setters but as creators of a context in which others make innovation happen. We should let them take roles that put their skills on display and provide them with the experiences and tools they need to unleash and harness the individual slices of genius around them. © 2014 Harvard Business School Publishing Corp

ABOUT THE AUTHORS: Linda Hill is a professor of business administration and faculty chair of the Leadership Initiative at Harvard Business School. Greg Brandeau is a former head of technology at Pixar. Emily Truelove is a researcher and doctoral candidate at the MIT Sloan School of Management. Kent Lineback has spent more than 25 years as a manager and an executive. They are the authors of Collective Genius: The Art And Practice Of Leading Innovation.