Reducing Data Modeling Creation Time by 33% – Pushpay

Before:

Complex Data Processes Leads to Inaccurate Insights

Pushpay used dbt as their data modeling software, and it was effective in generating visual reports. However, most of their team didn’t have training or experience with the platform. As a result, data models weren’t built according to best practices, creating slow data refresh times. Models sometimes took up to six hours to run, hindering financial reporting and marketing analytics.

Along with tedious manual processes, the team experienced roadblocks in the form of:

• Slow data delivery times hindered timely reporting.
• Month-end financial reports were delayed while waiting for data to be refreshed.
• Stale marketing data negatively impacted the demand generation strategy.
• Business owners and executives struggled to access up to date KPI reports.
• Lack of dbt expertise created time-intensive troubleshooting.

Although Pushpay’s data team understood SQL and other data engineering concepts, they needed additional expertise in dbt and modern data stack approaches. After connecting with the dbt team, they recommended that Data Clymer, a Spaulding Ridge company, implement a better solution.

Solution:

A Co-Developmental Approach Empowers Data Efficiency

We recommended an audit & health check of Pushpay’s current debt environment and integrations with Amazon Redshift & Tableau. We assessed how the platform was structured, the team’s capabilities, and their data architecture. From our findings, we proposed improvements:

• Recommendations to support improved time to value and data quality.
• Actionable ways to refactor existing code and align with dbt best practices.
• Action plan for enhancing Pushpay team’s skills.
• Strategy to implement short-term and long-term improvements to the dbt project and development processes.
• Topics for co-development time.

We began implementing our recommendations, focusing on a co-developmental approach with Pushpay’s team. The collaborative approach allowed Pushpay’s cloud data engineers to refactor existing code to adhere to dbt best practices with guided help from our end.

We conducted personalized group sessions to help their team get up to speed on dbt, enhancing their knowledge of the platform to prevent future bottlenecks. The new data stack integrated seamlessly into dbt, generating accurate reporting to enhance business outcomes.