Is your business being crushed by big data? Don’t worry, help is at hand

Is your business being crushed by big data? Don’t worry, help is at hand

By Colin Bryce

From being the preserve of actuaries and statisticians a generation ago, suddenly we have all become big data handlers.

Even the smallest of businesses and organisations are now expected to handle and organise flows of information from multiple channels every day and, for many, it can feel overwhelming.

The reality is that it’s only going to intensify. By 2025, it is estimated that we will be generating 175 zettabytes – or 175 trillion gigabytes – of data globally every year.

Since one gigabyte is equal to one thousand million (109) bytes, 175 trillion gigabytes is expressed as 175,000,000,000,000,000,000,000 or 17.521 bytes, representing a fivefold increase in data generation since 2018, and 180 times more than was generated 20 years ago.

To put that figure into some kind of perspective, the Apollo 11 moon landing was achieved using around 4,000 bytes of computing power. A modern smartphone typically uses four gigabytes of random-access memory (RAM).

Big-data technology is transforming almost every industry by creating actionable information and, because of its effectiveness in helping to generate new business, even small companies, usually slow to adopt new technologies, are coming on board, helping them to leverage up 15% more sales, according to a recent study.

An increasing amount of data is already generated by artificial intelligence (AI), to map the performance and output of the growing number of digital things in our lives.

As 5G communications become more widely adopted, these will soon include driverless transport, Internet of Things (IoT) devices including sensors in our bodies, homes, factories, and cities, as well as high-resolution content for 360 video and augmented reality.

Companies and organisations that are early adopters of big data analysis tools are more likely to benefit from reduced costs – including by optimising pricing strategies – increased operational efficiency and an improved ability to identify weaknesses and failures in their operations.

They will also be better able to design new products and services, conduct 360-degree customer reviews and identify and prevent fraud.

As a provider of Google management tools and services, the main bread and butter of our business has, until now, been platforms that help our clients handle documents, email files, video calls and other communication tools.

We estimate that, within two years the bulk of our work will be helping those same businesses to cope with the enormous flood of data they can expect to receive.

All companies have data; most have a lot of data and quite often the way it is organised is chaotic or not fully thought through. There’s not much governance and thought currently give to how that data is collected and organised and shared.

Our first step with new clients is to help them collect data in a consistent way – which means it’s easier to make sense of – and then to feed it through into what are called data lakes. We effectively pipe all of their data, from different sources, into one place, so that it can be more easily interrogated.

Depending on the business, data can come from a range of different sources. Here are just some of the different types of intelligence that is routinely gleaned about its customers.

• Personal data: Includes information such as their gender, occupation, age, and social class as well as non-personal material such as their IP address, web browser cookies and IDs of devices including laptops and mobile phones.
• Engagement data: This helps businesses to understand how their customers interact with their website, mobile phone apps, text messages, social media pages, emails, paid ads, and other customer service routes.
• Behavioural data: Provides companies with transactional details, including purchase histories, product usage information, such as repeat custom as well as qualitative data, such as mouse movement information.
• Attitudinal data: Encompasses metrics on items such as consumer satisfaction, buying criteria, product desirability and more.

Giving all that data structure and coherence helps businesses to better understand what it means, to visualise it and to create dashboards, charts and graphs so that non-experts are able to see and understand what is happening with their customers.

Providing them with live intelligence and insights into customers habits and behaviour, means they can make better informed, data-driven decisions in a useful way, rather than flying blind.

Some businesses that have yet to perform any data analyses often worry that they don’t have enough information to analyse. Most have more information than they can handle and, as the business grows, big-data systems become more relevant.

Few business owners can expect to gain any meaningful insight by reviewing data manually; they need the right tools and methods provided for them and explained by a specialist.

If they don’t have the budget for a full-time analyst, hiring a consultant who can point them in the right direction is a useful alternative.

As well as creating statistical summaries of their data analyses, an expert will also help them to understand what’s causing the patterns in their data. Business owners and senior managers will also need to know what may happen in the future and technologies such as predictive and prescriptive analytics can help them to achieve that.

Using real-time analytics can provide them with critical insights, in real time, on an executive dashboard. Dashboards permit them to access information, on-demand, via a smartphone, laptop, or other connected device and to share the information with colleagues.

Most businesses have spreadsheets, databases and customer-relationship management tools that are full of valuable information and can be used in combination with commercially available data sets and easy-to-use, Google data management tools.

After they have analysed their data, they will need to have it presented in a way that non-technical staff members can comprehend and use to make informed decisions. Again, Google has a range of data visualisation tools that will work with their current technologies.

It’s generally accepted that Big Query is the best tool for gathering data and they also have arguably the best data visualisation tool in Looker. It’s more than a visualisation tool, it also takes care of governance and sharing of data.

Everyone has their strengths in different parts of the cloud but I think everyone would accept that Google has the lead in data.

Change management is something that we have long standing expertise in and we have noticed that the techniques we have developed over a decade in helping people move to Google Workspace translate perfectly in helping them, to make sense of their data challenges, such as communicating with them, giving them a clear picture of what the pathway is going to be, training people and upskilling them and improving their knowledge.

As we all strive to become big data handlers, it’s comforting to know that there are tools available to help lighten the load.

Colin Bryce is Managing Director of Cobry, a Glasgow-based digital transformation company and Google Cloud partner.