What a 19th Century Data Problem Can Teach Us About Scaling Analytics Today

PESSOAS   |   Alan Jacobson   |   12 de março de 2020

You might not know it, but your organization has a lot in common with the U.S. Census Bureau (USCB).

During the 1870 census, the USCB ran into a data problem. Its small staff needed to quickly and accurately tabulate millions of census cards by hand and paper. Rather than hire more people, they used a rudimentary version of what we refer to today as self-service analytics.

Their solution was a machine that made cards easier to count by grouping them in pairs of two. It worked perfectly, until the next census when the population of the U.S. increased nearly 25% from 40 million to 50 million people, and the census took nearly eight years to complete.

For the 1890 census, the USCB used a better machine. This new machine could automatically tabulate the results and display them in a visual format. Unfortunately, its inventor realized he had just created something profitable and subsequently raised the price.

Although most of this took place over 100 years ago, the problems the USCB experienced then are similar to challenges that organizations face today:

  • Keeping up with the rapid expansion of data
  • The ever-increasing cost of data tools
  • The growing reliance on data to make crucial organizational decisions

Consider This.

A recent survey by Harvard Business Review Analytic Services shows that 60% of respondents “use data analytics to contribute to business decisions” and 40% base their “business decisions solely on what the data says.”

All of this is to say, if you don’t have the right tools in place, you could be facing a long, uphill battle versus a steep downhill slide.

The Importance of Self-Service Analytics for Scale

Although they might not have realized it at the time, the tools used by the USCB were a precursor to data solutions people want today — scalable self-service data analytics. The machines used for the census were easy to use, didn’t require specialized training, could process multiple data inputs, and display the results in a visual form.

But many of the solutions that organizations use today don’t meet these wants. Nearly two out of every three (63%) organizations “rely on IT and analytics groups for the data analysis they need for decision making.” What’s more, 80% of organizations report needing to customize existing reports frequently.

The lack of good self-service analytics in your organization can not only create bottlenecks between your departments, it also hinders your ability to scale data analytics across your organization. You’re paying more for data that is often already outdated by weeks, if not months, by the time you get it.

Not Every Data Analytics Tool is the Same

The first machine that was created for the 1870 census had one purpose — to tabulate results faster and more accurately. It was great at addressing the problem at the time but not when the USCB needed to scale up as data increased or optimal for scaling across the organization because it still involved manual processes.

You probably feel the same about your tools, and you aren’t alone. Only 36% of those using data integration tools, and just 33% of those using spreadsheets to analyze data, are “highly satisfied with the tools’ ability to provide the information they need.”

Tools by percentage of use:

  • Spreadsheets — 88%
  • Data Visualization (e.g. Tableau, Qlik, PowerBI) — 52%
  • Data Integration (e.g. SQL, Talend, Informatica) — 33%
  • Data Preparation/Blending (e.g. SQL, Python, Perl code) — 30%
  • Advanced Analytics (e.g. SAS, SPSS, Matlab, ESRI) — 21%

Simply having a data analytics tool doesn’t position your organization to handle its data needs. In some cases, they can make it harder for everyone to do their job.

Most of today’s data analytics software require specialized training or education in data analytics or coding. Some of them make it hard to share and repeat reports. Instead of enabling everyone in an organization to share data, the software leads to data silos, misinformation, and disagreement about the results. It’s hindering you from scaling data analytics across your organization.

If your HR department wanted to look at ways to improve employee retention or increase employee training and compliance, could they do it themselves, or would they need to rely on people trained to use the software? What about your sales and marketing teams?

However, what if like the USCB, they could drag and drop data into a workflow and then reuse or modify that workflow as they received new data, run their reports faster, and have the data help them retain more employees and improve the company?

What Makes a Good Self-Service Analytics Solution?

A good self-service analytics platform should be easy and reliable enough for all departments to use. As we said above, one of the largest barriers to an organization successfully scaling analytics across their organization is the limitation that their software places on its employees.

A good self-service data analytics solution today should include features such as:

  • The ability to ingest and use more of the data.
  • Little-to-no training for managers, analysts, data scientists, and executives.
  • Reporting tools that are easy to share both internally and externally.
  • Data lineage or the ability to trace data.
  • Security and privacy options to maintain data privacy and adhere to compliance regulations.
  • Reliability and accuracy in descriptive, predictive, and prescriptive analytics.
  • Cross-departmental accessibility to analytics and reports.
  • Repeatability for running reports.
  • No coding required, even if it includes Python and R functionality.

More importantly, a self-service analytics solution should be easy to implement today. It shouldn’t take months to install and it shouldn’t require major upgrades to your hardware.

At Your Service.

Of those surveyed, nearly half (45%) reported using self-service analytics as the solution to their data problems. Of those who use it, 71% reported using it daily or weekly.

As you scale data analytics across your organization, you should have a platform that leads to:

  • Increased cross-departmental collaboration.
  • Improved efficiency.
  • Optimization of existing processes and workflows.
  • Implementing new or expanding current automation processes.
  • Reducing repetitive work.
  • Driving revenue and/or decreasing spending.
  • Elevating employee retention rates.
  • Adopting new technologies.

If you focus on these goals and the needs of your organization, you should be able to research and find the right platform for you. When that happens, scaling data analytics across your organization should not only be seamless, it should be easier than running a census.

Survey data used in this article are repurposed from an Alteryx sponsored study: “Scaling the Power of Analytics Throughout the Organization” by Harvard Business Review Analytic Services.


Key Takeaways

As more organizations rely on data and analytics for faster and more insightful decision making, self-service data analytics solutions provide a viable means of alleviating some of the most pressing problems.

Stay Put.


For a deeper dive, you can access the full article here.


Send your challenges or solutions to building an analytics competency across the enterprise @alteryx