After having built a team of 40 analysts to solve fundamental as well as critical problems faced by digital businesses, if there is one thing I can say with utmost confidence, it is that a majority of an analyst’s time is wasted when they go about their business without the right infrastructure support. We stopped being that company three years back, started enabling our analysts to be a lot more productive, and as a result, have reaped the benefits. The article below talks about how you can make your analysts productive as well.
Leaders in growing businesses often rely on a data analyst to look into the business performance and come up with insights that when implemented, can help the company grow. They might also hire analysts just to provide them with information on what is happening in the business. It is therefore natural that an analyst’s time has to be optimized for creating value.
So, how can business owners and divisional leaders get the best performance from their analyst team? Let us first begin by clarifying the nomenclature of the few data-related roles we have in the industry.
Data Engineer
A resource that is adept at building data pipelines i.e mechanism to move data from one system to a centralized location, a data warehouse or a data lake for example, so that data analysts and data scientists can use that data to do their magic.
Data Analyst
A resource skilled in the exploration of data using a programming language like SQL and is good at communicating the output of analysis to the business in terms the business users understand. A data analyst is generally also good at using data visualization or business intelligence tools to communicate data stories to the business.
Data Scientist
A data scientist is someone who is a good programmer and can write statistical, ML or AI-based models that addressed the business problem at hand.
In larger companies, each department may have their own data analyst teams assisting decision-makers in these departments. Often times analysis is done in an excel and results presented in a powerpoint.
Breaking down an analyst’s workload can reveal some interesting findings of the quality of time spent and the value they are able to add as opposed to the true value they can add to the business.
In companies where there is minimal to no support of the data engineering team, analysts will have to double up as the data engineer. The reason is simple; analysts need data to do their job. Imagine an analyst having to perform various analyses of data from 15 different systems. Number sound too high? Well, this is the reality in an e-commerce world, for instance. Between marketing, online sales, marketplaces, retail outlets, analytics tools like Google Analytics, customer support, live chat, marketing automation, shipping, warehouse management, and CRM there is an army of applications that are used to run the business.
If you look at the nature of this data, it is varied as well. Data is stored in
- CSV or EDI files
- Relational and Non-relational databases
- SaaS applications
- Salesforce, Zendesk, Zoho CRM, Google Analytics, etc
- Advertising platforms
- Google Ads, Bing Ads, Criteo, Taboola, Amazon Advertising, etc
The picture below provides a high-level view of the journey the data goes through prior to landing in the hands of the business decision-maker. The analyst is sitting between the data sources to the left, and the PowerPoint presentation or the analysis output to the right.
Let’s just focus on the manual data pull portion of the process and understand why it is manual.
To achieve automation of data extraction, an analyst would have to do the following.
- Understand where the data is stored
- Read the documentation to figure out how to pull data from these systems
- Write custom code in Python or R or some other language to extract data from these systems.
- Ensure that the limitations enforced by the source systems are accounted for
- Maintaining the code as the source system APIs, file formats, or databases change
Or
- Run reports in their source systems
- Download those reports to their desktop
- And prepare the data for analysis in excel.
Guess what path gets chosen? Of course the second one – every single time. Because asking an analyst to do a data engineer’s job is easy, but doing it is extremely hard and something that the analysts are typically not equipped to do well.
In the absence of a data engineering team and a data modeling team, an analyst has to take on those responsibilities. As a natural consequence, analysts spend most (>60%) of their time on retrieving data from the various systems manually and preparing the data for analysis. This leaves little time for the analyst to perform the analysis for which they were hired in the first place. In the absence of automation of data extracts/data pulls and data models, it becomes impossible for the analysts to be productive and eventually lose motivation after doing repetitive jobs day in and day out. More analysts get hired because less work is getting done and the cycle continues.
With the advent of cloud computing, it is now possible to completely automate the data pull without the analysts having to write a line of code. By leveraging Daton and a cloud data warehouse like Oracle Autonomous DB, Google Bigquery, Snowflake or AWS Redshift, analysts can now focus on just building automated data models and the analysis of data.
Replacing the manual data pull with an automated solution like Daton frees up analysts’ time which they can use to perform the tasks they were originally hired for.
Analyst workload before Daton:
- 40% Gathering
- 30% Preparing
- 30% Analysing
Analyst workload after Daton
- 0% Gathering
- 20% Preparing
- 80% Analysing
Analysts at Saras are ultra-productive and a big role in that is because Daton takes care of the redundant yet critical data engineering work leaving the analysts to focus on what they do best – Analysis.
We firmly believe that every company can benefit from leveraging a cloud data pipeline and enabling their analysts to deliver growth for the business. Have your analysts sign up for a free trial today and get back a major portion of your analyst time so that they can focus on analysis and insights.
