Product Analytics and Metrics Course with A/B Testing

Introduction

Digital product teams and public e-service owners collect millions of user events, yet many still cannot say whether a release improved retention, why users abandon onboarding or which feature is worth keeping. Tracking is inconsistent, dashboards show vanity counts and experiments end without a trustworthy verdict. This Core Concept course builds practical product analytics and metrics skills, from metric frameworks and event instrumentation to funnel, cohort and A/B testing analysis. Participants produce a Product Metrics Framework, Tracking Plan and Experiment Design for a case app.

Course Objectives

• Define a North Star metric, its input metrics and HEART signals for a digital product or public e-service
• Specify an event tracking plan with a consistent taxonomy, event properties and a data layer ready for instrumentation
• Analyse conversion funnels, cohort retention and feature adoption to locate where and why users drop off
• Measure activation and time-to-value and segment users by behaviour to target product improvements
• Design A/B tests with a clear hypothesis, calculated sample size, significance criteria and guardrail metrics, and read the results correctly
• Build a Product Metrics Framework, Tracking Plan and Experiment Design that a product team can run from

Target Audience

• Product management staff who own feature decisions and success measures for apps and web products
• Product and data analysis staff who build funnels, cohort tables and experiment readouts
• Growth staff responsible for activation, retention and conversion targets
• User experience research and design staff who combine behavioural data with usability evidence
• Digital service owners who run citizen-facing e-services and track their take-up and completion
• Engineering staff who implement event instrumentation and tag management

Course Outline

Day 1: Product Analytics Foundations and Current-State Review

• Product Analytics Stack: Event Analytics Platforms, Tag Managers and Customer Data Platforms
• Actionable Versus Vanity Metrics Diagnostic
• Product Data Maturity Assessment for Apps and Public E-Services
• Mixed-Method Insight: Pairing Surveys, Session Replays and Usability Findings with Event Data
• Current-State Metric Inventory and Tracking Gap Register

Day 2: Metric Frameworks and Tracking Architecture

• North Star Metric and Input Metric Tree Construction
• HEART Framework with Goals-Signals-Metrics Mapping
• AARRR Pirate Metrics Across the User Lifecycle
• Event Taxonomy and Naming Convention Rules for a Tracking Plan
• Data Layer Specification and Tag Management Instrumentation Workflow

Day 3: Funnel, Cohort and Engagement Analysis

• Conversion Funnel Build and Step Drop-Off Analysis
• Cohort Retention Tables and Retention Curve Interpretation
• Feature Adoption, Stickiness Ratio and Engagement Depth Analysis
• Behavioural Segmentation and Power User Identification
• Activation Milestones and Time-to-Value Measurement for Product-Led Growth

Day 4: Experimentation, Data Quality and Privacy Risks

• A/B Test Hypothesis Statements and Primary Metric Selection
• Sample Size, Minimum Detectable Effect and Statistical Significance Calculation
• Guardrail Metrics, CUPED Variance Reduction and Test Choice by Metric Type
• Experiment Pitfalls: Early Peeking, Sample Ratio Mismatch, Novelty Effects and Segment Reversals
• Consent Capture, Data Minimisation and Privacy-Safe Event Collection

Day 5: Modelling Build: Metrics Framework, Tracking Plan and Experiment

• Case App Brief: Diagnosing a Stalled Onboarding Funnel
• Metric Tree and HEART Scorecard Build for the Case App
• Tracking Plan Workbook with Event Properties and QA Checklist
• Experiment Design Document with Sample Size Workbook and Guardrails
• Product Dashboard Mock-Up and Metric Review Presentation

Skills You Will Gain

• Product Metric Design
• Event Instrumentation Planning
• Funnel Diagnostics
• Retention Cohort Analysis
• Behavioural Segmentation
• Experiment Design and Readout
• Product Dashboard Reporting
• Privacy-Aware Data Collection

Why Attend This Course

• Leave with a Product Metrics Framework, Tracking Plan and Experiment Design built and reviewed during the course
• Settle debates about a release or feature with funnel, cohort and experiment evidence instead of opinion
• Stop shipping tests that end inconclusive by sizing them properly before launch
• Give engineers a tracking specification they can implement without guesswork or rework

Conclusion

Product decisions improve when every team member knows which metric matters, how it is tracked and how a change will be tested. The course moves from the analytics stack and current-state review, through metric frameworks and tracking architecture, to funnel, cohort and engagement analysis, and then to experiment design, data quality and privacy. The final day brings this together in a modelling build that produces a Product Metrics Framework, Tracking Plan and Experiment Design ready for the next product review.