Who This Helps
Hey there, Junior Analyst. You just saw a key metric drop 15% overnight. Your stakeholder is asking why, and you need a clear answer fast. This is for anyone who wants to move from panic to pinpoint accuracy. It's a core skill from the Data Reliability Leadership course.
Mini Case
Sam's weekly active user report showed a sudden 12% dip. Instead of diving into every query, they first checked their data contract for the 'active_user' definition. Turns out, a backend service change had filtered out a user segment for 48 hours—it wasn't a real business drop. They diagnosed it in 25 minutes and saved a day of wild goose chases.
Do This Now (5 Steps)
- Pause the panic. When you see the drop, block 30 minutes on your calendar. No frantic digging yet.
- Grab your contract. Pull up the data contract or definition doc for the wobbly KPI. If you don't have one, note that as your first finding. (This is a key mission in Data Reliability Leadership: defining contracts for key metrics.)
- Check the source. Verify the raw data source for the last 7 days. Did an ingestion job fail? Was there a schema change?
- Isolate the change. Compare the metric's components. Did one specific segment, region, or platform drive 80% of the change?
- State your hypothesis. Write one sentence: "The drop is likely caused by X in source Y, impacting Z segment." Now you have a focused path to prove or disprove it.
Avoid These Traps
- Don't start by building three new dashboards. You're diagnosing, not decorating.
- Don't blame 'data quality' without specifics. Which table? Which field? Get granular.
- Don't loop in 10 people immediately. Do your focused 30-minute check first, then communicate.
- Don't ignore small, consistent dips. A 2% drop for 5 days is a bigger signal than a 15% one-day blip.
- Skipping the contract check is like fixing a car without opening the hood. You might get lucky, but you'll probably make it worse.
Your Win by Friday
By using this method, you'll ship one analysis this week where the recommendation is crystal clear because you know the exact root cause. You'll replace "the data looks weird" with "the nightly user sync job failed for two days, causing a 12% undercount—here's the fix." That's how you build trust and look like the calmest person in the room when numbers get shaky.