You know the report worked last month: Nobody changed the calculations, the workflow still runs without an error, and yet the number on the dashboard just looks wrong.
So you start tracing backward, and eventually you find what changed. A new category value arrived and never got mapped, one source came in late, or a join that behaved perfectly for months is suddenly producing extra rows.
If you’ve owned recurring analytics for a while, you know the drill. Finding the issue may take some digging, but the more frustrating part is realizing you’ve solved some version of the problem before — and the check that would have caught it still lives in your head.
You need more of that knowledge built into the process, so problems show up earlier instead of sending you scrambling back through the workflow right before the report goes out.
Start with what has to be true for this report
For reporting, data quality starts with what you’re asking the data to do. The same data set can work perfectly well for one analysis and be wrong for another, so a universal checklist of “good data” gets you only so far.
As Gartner notes, “data quality tools do not exist alone.” What matters is how well they support the analytics and reporting work you need to get done.
Take a monthly finance report, where your checks may be very concrete:
- Did all the expected source files arrive?
- Does the extract include the full reporting period?
- Do the totals reconcile to the control figure you expect?
- Did any new cost centers show up without a mapping?
- Is the data still at the expected granularity — for example, one row per invoice?
Move to a different workflow, and the checks change with it. In a customer segmentation workflow, duplicate entities or matching quality may matter much more. For an operational dashboard, the bigger concern might be whether today’s data showed up when you expected it to.
That’s why “clean data” isn’t a very useful finish line. A date can be formatted correctly and still belong to the wrong month, while two source tables can each look fine until the join between them starts creating duplicates.
You probably already have a data quality process
If you’ve been running the same report for a while, you’ve probably built up a collection of checks without ever sitting down and calling them a “data quality process.” Some live in the workflow, others happen in Excel, and a few are the things you remember because a source surprised you three reporting cycles ago.
Meanwhile, somebody else on your team may be watching for a completely different problem because they got burned by it before. Before long, the process is split between what’s built into the workflow and what people know to check on the side. As the data changes and new problems turn up, more checks get added.
You don’t need software to make every judgment for you. What helps is getting more of the checks you repeat into the process, so they’re still there when fresh data arrives.
Profiling helps you avoid making up rules in a vacuum
Say you’re trying to decide when missing values are enough to flag a field. A threshold means more if you know how the field normally behaves: A drop from 99.9% complete to 94% tells a very different story from a field that has always hovered around 90%.
You can run into the same thing with uniqueness. Before you require invoice ID to be unique, make sure the workflow still has one row per invoice at that stage; otherwise, you may end up with a check that fires so often people stop paying attention to it.
When you’re comparing platforms, see how quickly data profiling helps you understand the context behind a rule. Six months from now, you or another analyst should still be able to tell why a threshold exists instead of inheriting a number nobody remembers choosing.
The checks worth keeping are often the ones you already make
You probably have a mental list you run through before a report goes out. Maybe you always check that the current month is there, or you know there’s one join that deserves a second look because it has added rows before.
Those are the checks worth turning into repeatable data validation, especially when there’s a business reason behind them. “This field can’t be null” is useful, but “every active store needs a region before the sales report goes out” tells the next person why the check exists and what it’s protecting against.
A failed data quality check is only the beginning
Say it’s late afternoon, and your completeness check turns red. Good, you caught something — but now you need to work out what happened. Maybe a handful of records are missing; maybe it’s a big chunk of the data. Either way, you need to know whether the problem started in the source or somewhere in the workflow.
If all the platform gives you is “quality failed,” you’re still stuck doing the detective work. When you’re comparing platforms, pay attention to how quickly a failed check gets you to the records behind it. When the report is due in an hour, that path matters a lot more than a polished status screen.
Maybe 30 transactions came through without a valid mapping. If your next move is to export those records, fix them somewhere else, and bring them back into the reporting workflow, you’ve found the problem without making it much easier to deal with.
Sometimes that handoff makes sense because another team owns the decision. But when the business logic sits with your team, see how much of the investigation and follow-up you can handle without leaving the workflow.
What you do next depends on what failed
“Find the bad data” sounds straightforward until you’re looking at a real exception. Maybe a new product code shows up with no match in your mapping table. You can see which records are affected, but the platform can’t know whether that product belongs in Consumer, Enterprise, or a category the business created yesterday.
Other problems are easier because the decision has already been made. If a source sends “N.Y.” and “New York” interchangeably, and everyone has agreed on the standard, data cleansing can apply the same correction without making you revisit it every month.
That’s why “automatically fixes bad data” isn’t a very useful test when you’re comparing platforms. Some problems have a settled response; others still need your judgment. And if a critical source is missing, the right move may be to stop the report before it goes any further.
You’re still deciding what each kind of failure means. What you want from the platform is a way to make the decisions you’ve already settled easier to repeat.
Compare platforms with a workflow you already know
Feature grids can make very different data quality platforms look weirdly similar. Profiling, rules, monitoring, automation — most vendors can point to something in each column, but the list won’t tell you what Friday afternoon looks like when the numbers don’t reconcile.
Bring a real workflow into the evaluation instead, preferably one your team knows well and that has a little history behind it. A report that still needs a manual reconciliation, or has a step everyone knows to double-check, will tell you a lot more than pristine demo data. You get to see how the platform handles the kind of work your team deals with on a normal week, not just the example it was built to show off.
Test the rules your report really relies on
The checks you rely on usually go beyond basic field-level validation. Maybe you need to make sure every store that traded this month has a region. Or perhaps the issue you’ve learned to watch is a join that sometimes creates extra rows. Your rules should reflect the things that can throw the report off kilter, not just whether the fields pass a basic technical check.
Use the rules your team depends on in real life. If they’re awkward to build or hard for another analyst to understand, pay attention. Important checks have a habit of drifting back into spreadsheets or somebody’s pre-publish routine when the platform handles only the easy ones gracefully.
Clear, reusable checks make handoffs easier too. The analyst covering the workflow can see what to watch for without needing a half-hour download of everything the report “sometimes does.”
Break something and see how the platform behaves
A demo tells you a lot more when the data gets a little messy. Leave out an input you know the workflow needs, or throw in a value that won’t match the existing logic. If joins have caused trouble before, use a test case that creates the same kind of duplicate rows.
Then follow the problem through and see how quickly you can get from the failed check to the records behind it. You also want to know whether the workflow can respond differently depending on what went wrong — stopping for a missing source, for example, while sending an uncertain match for review.
Prepared demo data can make almost anything look smooth. A test case that misbehaves gives you a much better sense of what using the platform will feel like when the real data does the same thing.
Where data quality problems tend to show up in reporting
By the time you’re getting to data reporting, someone is about to use the output, which is when a quality problem gets a lot harder to ignore.
A dashboard can refresh perfectly even though the latest reporting period is missing because the visualization is doing exactly what it was told to do; the workflow just never checked whether the expected data showed up.
A join can catch you the same way. Both inputs may look fine on their own, but if the relationship between them changes, you can suddenly end up with extra rows downstream. If you expect one row per invoice, checking that right after the join tells you much more than simply confirming that both source tables looked clean.
Sometimes the data itself is perfectly valid, but your reporting logic hasn’t seen it before. Maybe a new sales channel appears with no place in the current reporting categories. The value isn’t wrong; it just needs a decision before the report is ready.
When a dedicated data quality or observability platform makes more sense
Sometimes you’re solving a bigger problem than whether one reporting workflow is ready to go. If your team needs to monitor quality across a large data estate or support monitoring that sits closer to data engineering, a dedicated data quality or data observability platform may be the better fit.
But if most of your pain starts after a check fails, look closely at what happens next. You want to know whether you can get to the affected records, deal with the issue, and keep the workflow moving without sending the data through another process.
Those are different needs. If your priority is broad monitoring across the enterprise, evaluate for that. If your team spends more time chasing down failed checks inside reporting workflows, pay more attention to how easily you can deal with the problem without breaking the workflow apart.
See what a connected analytics workflow looks like in practice
Take a weekly inventory report where one supplier starts sending a status value the workflow hasn’t seen before. The report may still run, but those records now need a decision before the output is ready.
If you’re testing a platform against a case like that, follow the unexpected value through the workflow. You should be able to get to the affected records without turning the investigation into a separate side process. And if the right treatment is already known, that logic can stay with the workflow so you don’t have to make the same fix again next week.
If the answer isn’t known yet, the workflow shouldn’t pretend otherwise. Those records may need review before the report moves on.
What matters here is the distance between spotting the problem and being able to do something useful about it. A connected workflow should make that path easier to follow without asking you to stitch the process back together along the way.
If you want to see how Alteryx One handles this kind of reporting workflow, explore Alteryx One.
Start with a report your team knows well
You probably don’t need to start with the biggest workflow in the company. Pick a recurring report your team knows well enough to name the parts that tend to cause trouble, then think about the checks that are there because something went wrong before.
Which checks still depend on somebody remembering them? Which fixes keep showing up again? If something serious broke tomorrow, would the workflow know enough to stop, or would it keep going until somebody noticed?
Those questions are a much better test of a data quality platform than a generic feature checklist. You’re looking for a way to make the checks your team already relies on easier to repeat — without trying to automate the judgment calls that still need a person.
If you want to put that approach to the test, start a free trial of Alteryx One with a recurring report your team already knows inside and out.