Big Data: when too much information is never enough
In the era of Big Data, we are swimming in a sea of stats and numbers derived from website interactions, social media mentions, demographic information, location data and more.
When it comes to determining needs and trends in business, this stuff is worth its weight in gold.
So naturally, those who can make sense of the figures by interpreting the numbers and mapping the data to show what it means are hot property. In fact, the demand for data scientists has doubled over the last four years.
It’s hard to think of a sector that will never need the assistance of a talented data wrangler who can help uncover patterns, find the hidden stories and inform business decisions.
Collecting, managing and interpreting information is changing the way we make decisions across the board – from education and healthcare, to banking, manufacturing and the environment.
So what do you need to succeed in this fast-paced, pervasive field?
Hard and soft skills
Data scientists and analysts carry technical skills – mathematics, statistics and programming – but they’re valued just as much for their creativity, and in particular, how they can communicate complex ideas in ways that non-specialists can easily understand.
“Analytics and data science professionals must be highly creative problem-solvers,” RMIT University analytics expert Dr James Baglin says.
“They need to be good at reframing questions in terms of data, methodological in all they do, willing to follow the evidence wherever it leads them, and critical of their own solutions.
“You’re working in multidisciplinary teams and you’ll usually report directly to managers and stakeholders who don’t have the same technical background, so you also need an understanding of how to communicate and visually present technical information.”
There is a growing demand for qualified data experts – IBM predicts the demand for data scientists in the US alone will soar by 28 per cent by 2020.
As a consequence, there has been a surge in the numbers of people opting for postgraduate study in this field, where industry-connected research projects and internship opportunities can help them develop the right combination of technical and “soft” skills.
“Having the chance to apply what you learn by working with a business partner on a project is an invaluable way to flex your creativity and your communication and teamwork skills,” Baglin says.
“These skills can’t easily be taught in a lecture, they need to be practised and fostered, and the best way to do this is through real-life experiences.
“Working with Sustainability Victoria or Austin Health, as our students have done – it’s these kinds of experiences that help you develop as a well-rounded professional."
Data scientists and analysts need to know how to communicate technical information in ways that makes sense to colleagues from different fields.
Same same, but different
The two main areas of focus in this field are analytics and data science, and while they are closely related, a few key differences mean that one will suit some people more than the other.
Analytics emphasises the statistical, mathematical and technological skills required to make informed business decisions. It cuts across all industries and sectors, big and small, and involves retrieving, preparing, analysing and modelling complex real-world data.
Data analysts can be found evaluating the impact of air, water, and soil pollution; estimating the unemployment rate of a country; measuring consumer demand for products and services; studying the causes, patterns and effects of health conditions and disease within a population; or monitoring use of transportation systems.
While data science also incorporates statistical methods, it puts a greater emphasis on the specialised computational skills required to manage and analyse big data accumulating from sources such as social media, sensors, mobiles and transactions.
Data scientists work in big tech companies, smaller start-ups and everything in between. Here you’ll find them mining data sets in order to glean new insights and create new products or platforms.
Big data presents many challenges and data scientists focus on developing the capability to derive insight and opportunity from the vast repositories of information that organisations collect.
It’s a mighty task – little wonder Harvard Business Review has called data science the “sexiest job of the 21st century”.
So if you love the idea of getting deep into data, what’s the best avenue for you?
“If you are business focused and your dream job is to use data to drive business decision-making and problem-solving, your future career plans should focus on analytics,” Baglin says.
“If you love data, computers, programming and using computational methods to analyse data and find insight, developing the skills to be a data science professional would be a better fit.”
First published 23 August 2017.