Welcome back. This is the fourth blog of this roadmap series I've been writing to help people get better clarity and direction. This one is going to be about everything in analytics.
I value all of this equally. I've worked in almost all these aspects—data science, building agentic systems, applied AI, applied ML. Now I'm transitioning towards making really good research contributions in ML science and AI. But analytics? It remains foundational to everything.
Let's keep it simple. Data analytics, product analytics, growth data science, growth analytics, analytics engineering—I'd say it's all the same. All of it shares one core thing, and since I come from an instrumentation and control engineering background, let me frame it that way.
Analytics as Instrumentation
In instrumentation, everything is about bringing signals to the master control system so it understands what's happening around the system. We use sensors. We use DAC (data acquisition cards) to get data from those sensors. We do calibrations, noise reduction, Kalman filtering—all to get better signals, better state estimates.
Analytics is exactly this: bringing in signals and making sense of them.
The signals can come from anywhere—that's just data from different sources. If it comes from a product, you call it product analytics. If it comes from retail or business operations, you have business analytics or business analysts. The usual term is "data analyst," but "business analyst" adds more context—they analyze signals from business operations to help functional and technical teams take action.
As simple as that.
The DIPP Framework and Where People Actually Operate
People always mention DIPP: Descriptive, Inquisitive, Predictive, Prescriptive analytics.
But in reality, most people work up to the inquisitive part. Predictive and prescriptive work usually falls to people with titles like data scientist, machine learning engineer, or ML scientist. If your company does it differently, that's not an issue—I'm not drawing hard boundaries. I'm just giving you an accurate picture of how companies actually operate.
Analytics in the AI Era
Let's talk about how analytics and AI go hand in hand now, and how AI is being used for analytics.
Here's the thing: you cannot dump millions of rows into any AI system and magically get sense out of it. Previously, everyone wrote SQL. When I was working at Mu Sigma, people were so good at SQL and Excel. I wasn't at that level because I never had the opportunity to do that ad hoc analysis—daily product reports, customer information sheets, audit reports. I was fully on the machine learning and optimization side.
But those colleagues? They knew almost every Excel shortcut. They knew SQL optimization—cascaded queries, correlated queries. Everyone had their queries saved, ready to modify and run. It was impressive.
Now AI helps write these queries automatically. But ultimately, analysts are the ones who decide where to look, what data to examine, what new columns or metrics to create, what KPIs to track, and how to view them.
The Problem with Raw Numbers
Numbers can be misleading—even cryptic.
Take Annual Recurring Revenue: $1,000 ARR. Sounds straightforward, but will it actually recur next year? It can go up or down due to cannibalization, competition, a hundred other factors. That single number doesn't tell you the whole story.
To make sense of specific business characteristics or user behavior, we need specific metrics: click-through rate, customer lifetime value, monthly active users, dwelling time. When I was working on product analytics, we developed something called "stickiness"—coming up with a meaningful metric is itself significant work.
All the data gets dumped from systems, but systems don't record KPIs directly. That would confuse people because not all KPIs are needed everywhere. So there's substantial engineering effort involved—this is what people now call analytics engineering: building the data analytics pipeline.
Where the Work Starts
The work starts from the data warehouse where tables are properly arranged. That's enterprise-level data management—it's already set up. Analytics engineers, data analysts, and business analysts work from these warehouses and data sources.
These sources can come from many places: Delta Lake, Snowflake, AWS (there's a warehouse service—I forget the exact name), Google BigQuery.
When I was at Mu Sigma in 2019, people used Pentaho and Informatica. Over time, I've seen things shift toward Snowflake, Databricks, BigQuery, Azure Data Factory. But everything supports SQL. People know the business, they know the data dictionary, they know where everything lives.
The key is writing effective queries because queries take time and cost money. You write effective queries, get your data snapshot, calculate metrics, and build dashboards.
Why Analytics Engineering Is Growing
The term "analytics engineering" has become prominent, especially in emerging companies and across the US. The reason: analytics has become more software-oriented.
Many companies don't need unique views of data. They need standard things: monthly sales, weekly sales, how sales have changed, which channels contribute traffic, where users come from. This is typical for product companies.
For standardized needs, you want a software engineering approach—streamlined data warehouse connections, easy data fetching, version-controlled tables and views, calculated metrics, proper reporting. This gets connected through tools like DBT. Tableau has everything built-in for streamlining.
For large-scale operations where companies want to stay minimal, they use Airflow to run queries against BigQuery or Spark, then visualize in Power BI or Tableau.
The Two Essential Elements
In business analytics—whether you're a business analyst or analytics engineer—you need two elements:
First: Strong understanding of the business or operations you're dealing with.
This is the real important thing. Without it, you can't make sense of expectations from business teams.
Second: Familiarity with the tools.
The contribution here isn't innovative or scientific in the research sense—we're not winning algorithmic prizes. It's about workflow. How you build that workflow.
The Critical Differentiator: Analytical Thinking
If you want to become really good, there's a third thing: analytical thinking.
Sometimes business teams don't know how to look at data. They don't know what metrics they need or what observability would help them. They might be confused about whether to look at gross sales or foot traffic. These confusions happen.
That's where you, as an analytical thinker, clear the chaos. You say: "This situation needs this way of looking at trends and patterns."
Two Dimensions of Analytics Work
I see the work existing across two dimensions:
Dimension One: Software-Engineered Analytics Workflows
This is the descriptive analytics part. You know exactly what metrics to calculate, how dashboards should look, who receives reports, where data gets fetched. Everything is predefined and systematized.
Dimension Two: Experiments and Hypothesis Testing
This is fundamentally different from calculating KPIs. Those KPIs might reveal a problem—say, customers aren't renewing subscriptions, or they're not buying more. The metrics reveal the problem.
But going from problem to clarity requires testing multiple hypotheses. Is it a bug? Is onboarding time causing issues? Is inventory not supporting demand? Are there delivery delays?
These are hypotheses that the analytics professional must form. And they must test them too.
This is a treasure hunt. Looking for evidence is not the same as calculating metrics.
Building Statistical Rigor
Based on your curiosity and involvement, you use different methodologies. If you want to level up, focus on statistical inference and quantitative approaches.
For example:
- Monte Carlo sampling to validate patterns
- Simple linear regression models for additional assurance
- Proper hypothesis testing that leads to statistical evidence
As you get interested, start with basic frequentist analytics. Don't waste time fighting about "I'm Bayesian" or "I'm frequentist"—that's not your job. Stick with frequentist approaches. Start with high school statistics: dispersion, data distributions, variance analysis, hypothesis testing, statistical inference.
The Misconception Problem
I sometimes feel frustrated that influencers and tutorials reduce statistics to if-else conditions. "If p-value is less than 0.05, reject the hypothesis. If greater, accept it."
That's not how it works. There are nuances.
Another misconception: "If you have less than 30 data points, use t-test. If more than 30, use z-test."
Times have changed. Data acquisition isn't expensive anymore. These rules came from centuries ago when data collection itself was difficult. They were developed for clinical, healthcare, and biological contexts where you genuinely had tiny samples—Francis Galton's era, Fisher's t-test period.
Now we have millions of rows. We used to say "big data"—now nothing feels big anymore. We just have data. In these situations, use statistical tests with more clarity and understanding.
Spend a month studying common misconceptions in data science courses, analytics tutorials, and statistical blogs. Even understanding hypothesis testing has many misconceptions that most content glosses over.
Recommended Reading
For Rigorous Statistics
All of Statistics by Larry Wasserman. I always suggest this. His lectures and details are really, really good.
For Intuition Building
The Art of Statistics if you're more of a populist reader—good for getting intuition, less technical depth.
For Thinking Like an Econometrician
Keep reading habits that expose you to how econometricians approach problems. Joshua Angrist, the Nobel laureate who studied labor markets and causal inference, has said "metrics is the original data science"—econometrics is just different forms of statistical and linear modeling.
Focus on: statistical testing, linear models, how to understand and interpret results, and the problems with p-value hacking. If you're interested, explore e-values—they're gaining attention.
Building Your Leverage
Here's how to think about this: all analytics professionals know business. Let's say you're a beginner—you know Excel and SQL. What else can you learn? How can you think through problems? How can you give insights that have statistical evidence behind them?
That's building leverage.
Scalable Analytics Skills
Most scalability problems have been solved by existing tools. But if you're joining a startup, you can't always expect polished data interfaces and infrastructure in place.
For leverage, I'd say: learn DBT, learn PySpark.
This gives you a Swiss Army knife. You combine scalable analytics systems with statistical thinking and analytical thinking. You can move between descriptive and inquisitive analytics, even lean toward predictive to an extent.
Nobody wants to stay in the same job forever. These are the opportunities for growth.
AI for Analytics: The Real Opportunity
What do I mean by AI for analytics? I'm not talking about using Claude to generate SQL codes—you'll figure that out on your own.
Here's the bigger picture: most unstructured business data is text and images. Customer reviews, customer feedback, complaints, support tickets.
If you've already geared yourself up to write Airflow, DBT, or Spark workflows, it's just a matter of time before you write a few prompts, use APIs, and build AI into those workflows. You can integrate this into your Airflow workflow, Astronomer workflow, or Databricks notebooks.
Then you're not just analyzing numbers, metrics, and dollar figures. You're also understanding:
- What operations teams wrote as complaints
- What rejections or refusals occurred
- What issues customers expressed
- What patterns emerge from feedback
If you work at a marketing company, think about social listening—monitoring what people say about your brand across social media.
AI models, even open-source ones, have completely taken over the burden of data extraction. Someone writes something positive or negative? You classify it and bucket it for analysis. You extract golden signals from comments, write-ups, images, all the rantings.
A Project Idea
For your personal projects, think about data fusion: combining structured and unstructured data, using relevant methods for each, merging them, coming up with insights, identifying doubts, and proceeding further. It becomes a loop.
A Note for Analytics Leadership
I'd suggest that analytics leaders think about this: not everything has to go to completely separate teams. An analyst working close to business and operations can easily do these extractions, run inference, perform statistical validations, and create a nice presentation.
Communication: Really, Really Important
Anyone calling themselves an analytics professional—communication is really, really, really important.
If you're a machine learning engineer, you might only talk to your manager or the person guiding you. But irrespective of your level, you have to communicate with different functions in the company and team.
Communication isn't just English. It's data storytelling.
Books I Always Recommend
- Storytelling with Data by Cole Nussbaumer Knaflic
- Information is Beautiful by David McCandless
These are excellent reference materials for adopting a visual and narrative style.
Bonus content: Read data journalism. The New York Times has exceptional visualizations and plots. Study how they present complex information.
Additional Reads
- The Little Book of Data
- How Data Happened
These give you perspective: you're not doing a simple job. You're a critical piece of the organization. From you, clarity starts for the company.
Your Position in the AI Era
Maybe people say data engineers are the ones doing the heavy lifting. But analytics as a collective organization matters equally. Data analysts and analytics engineers are crucial—and they gain more power in the AI era.
You don't need to worry about debugging SQL code or fighting with queries anymore. Think broader. Look for those golden signals. Figure out the blind spots so you can help businesses see what they're missing.
If you have leadership thinking or strategic thinking, you can lean toward:
- Analytics and Strategy
- Analytics and Customer Success
- Analytics and Marketing Success
The Bigger Game
I'd say analytics is genuinely beginner-friendly. It's a very good stepping stone into a bigger industry.
But as I mentioned—keep building your leverage so you can play a bigger game:
- Analytics + Strategy
- Analytics + Policy
- Analytics + Data Journalism
- Analytics + Advocacy
Nobody stays in the same job forever. The question is what doors you're opening today.
Key Takeaways
- Analytics is instrumentation — bringing in signals and making sense of them
- Most work happens in descriptive and inquisitive analytics — predictive/prescriptive typically falls to data scientists
- AI helps but doesn't replace analytical thinking — you still decide what to look at and how
- Analytics engineering is growing — software-oriented approaches to standardized analytics needs
- Two essential elements — business understanding and tool familiarity
- Analytical thinking is the differentiator — helping teams see what metrics they actually need
- Build statistical rigor — go beyond p-value thresholds, understand the nuances
- Learn DBT and PySpark for leverage — Swiss Army knife for scalable analytics
- AI for unstructured data is the opportunity — integrate text/image analysis into your workflows
- Communication and storytelling matter — you create clarity for the entire company
- Play a bigger game — Analytics + Strategy, Policy, Journalism, or Advocacy
This is Part 4 of my roadmap series. Keep building, keep exploring.