Our world is dominated by data; masses of data, big data, conflicting data, accessible data. 30 billion pieces of content are shared each month on Facebook alone and on the flip side it costs less than £600 to buy a disk drive that can store all of the world’s music. It has been estimated that big data has the potential to add 250 billion euros to Europe’s public administration; and there is indisputable evidence that those companies that can harness its power and read the hidden messages can gain significant economic benefit.

The ability of companies to unlock this potential hinges on hard to find, and recruit, data wizards who can make sense out of this hodgepodge and help companies get something useful to base decisions on.

So who are these magicians? How can a company find and attract this special talent?

“There is a significant shortage of analytical talent necessary to make the most of big data,” says Richard Stevenson, manager of our Technology Practice. “As one of the biggest players in the field, Hydrogen Group has a significant and exclusive pool of data scientists, still, when a candidate becomes available they are placed pretty much immediately,” Stevenson remarks.

To capture the full economic potential of big data, companies and policy makers will have to address the talent gap. New research by the McKinsey Global Institute (MGI) projects that by 2018, the US alone may face a 50 to 60 percent gap between supply and the requisite demand of deep analytic talent.

UK universities are beginning to address the problem by producing more data science-focused graduates and post-graduates. The University of Dundee, University College London and the University of Brighton are a examples where departments are working together so that computer science degrees include business, mathematics and problem-solving.

Meanwhile, hiring data scientists with advanced background in statistics and computer science, is for the time being proving difficult. “Hiring teams, instead of the mythical all-rounder, is a good way to get around the skills gap,” says Stevenson. “A single expert, would be prohibitively expensive, even if found. It is much better to determine the skills required and recruit a team to fulfil the project requirements,” he suggests.

To put things in perspective, even though data science requires specialised hard to find skills, it is attracting lots of smart professionals who recognise the potential of big data. Short terms there will be shortages, no doubt, but in the longer term, the skills gap will be bridged.​