Before:
RISE’s Company Growth Creates a Need for Better Data Quality Management
RISE’s growth was putting a strain on internal information systems that hadn’t been built to scale. Multiple state agencies relied on the information to verify worker training and certification, so maintaining data quality and trust was vital.
Staff productivity was drained as data team members struggled to resolve issues across the 12K worker records they received daily from different state and agency systems
Nathan Beil, RISE’s Director of IT Systems, knew that scaling for growth would require an overhaul of RISE’s data architecture, infrastructure, and tooling. Beil and his team’s goal was to create a master record and unique identifier for each worker record. But with no single source of data truth, along with state agencies and organizations using different systems, their goal proved to be too complex.
Duplicate records required a RISE team member to analyze the data, track down verification, and update the system. A lack of clear data ownership made this effort even more time-consuming.
One senior data manager was spending 20% of his time on operational tasks that should have been automated, yet he was the only one who could address them. The rest of the team was similarly bogged down.
Meanwhile, RISE continued to grow in leaps and bounds. With 12,000 records coming on a daily basis, it was impossible to manage the rapidly increasing volume of data. With available time at an all-time low, RISE turned to Data Clymer, a Spaulding Ridge company, for help.
Solution:
A Personalized Modern Data Stack Enables Scalability Without IT Restrictions
After connecting with RISE’s team, we recommended a modern data stack consisting of Snowflake as a cloud data platform, Matillion for data ingestion and transformation, and Sigma for analytics and data visualization.
RISE looked to us to build their personalized modern data stack. First, we implemented Matillion to ingest and migrate data from various sources to Snowflake’s cloud data platform, enabling powerful data transformations and accuracy. We then built custom connectors to RISE’s Learning Management System, empowering RISE’s team to create special connectors to allow the system to integrate with external data sources, strengthening integration capabilities.
With a new data source of truth in Snowflake, we proceeded to migrate the accurate data to Sigma, transforming the data into visual dashboards and reporting resources.
With a new modern data stack in place, RISE had the tools necessary to adapt to the growth of the company, eliminate tedious manual processes, and utilize their data’s new insights to make better decisions.