Faire connects local retailers with boutique brands and wholesalers across the globe, allowing them to easily find the right products for their shop. Over the last five years, Faire's mission and platform have resonated with brands and retailers growing to over 100,000 brands in more than 100 countries, including 700,000 retailers in North America and Europe alone.
Rapid growth complicated access to self-serve data and impacted data quality
Faire started as a platform to connect boutique brands and small retailers, but they have quickly evolved beyond a wholesale marketplace. They are now a major growth partner for retail brands, managing digital e-commerce storefronts, order fulfillment, customer relationships, and transactions.
But along with their success came difficulties. As the platform expanded from a few brands to 100,000, and from a few retailers to 700,000 globally, this rapid growth led to a data explosion, complicating the machine learning models and data queries.
“Sometime around 2021, our data and workload volume had grown enough that week-to-week changes in peak capacity meant frequent cluster halts that would require downtime. Much of the business data analytics and engineering initiatives like democratizing data access and expanding data capabilities were all blocked,” explained Ben Thompson, Staff Analytics Engineer at Faire.
Faire’s previous data warehouse couldn't keep up with Faire's accelerated growth, halting all their critical business processes. This is where Snowflake stepped in, offering scalable solutions and efficient peak load handling.
After the transition, the improvements were clear: Snowflake not only eradicated downtime for cluster scaling but also reduced the average runtime for BI queries by 75%.
Now on the new data platform, Faire needed to rethink their ETL design and their “core data” model to truly enable data democratization across the business.
The data team had a new mission to accommodate the company's growth – especially as the company was continuing to launch new features to the customers. Everyone at Faire needed access to data at any time to make informed decisions, take calculated bets, and quantify values without barriers to access or understanding. To achieve this, the data team needed to balance scalability, self-serve access, and data quality.
“There was friction in getting into the data, and using it for day-to-day work was a big inefficiency for us,” Ben explained. “Often we'd have slightly different permutations of metrics or attributes within our tables and it would take a while for users to learn which one should be used in which scenario. Or worse, they would just pick one at random and often get the wrong value.”
Faire recognized a need for a data catalog that would coalesce knowledge on hundreds of columns and make it discoverable for everyone at the company.
Select Star allows us to engage users early with exactly how each upstream change impacts their downstream workflows. This has built a lot of trust in the team and helps iron out thorny issues quickly.
Senior Staff Analytics Engineer at Faire
Intuitive UI/UX and automatic popularity rankings bring immediate data insights
Faire’s data team evaluated several open-source tools and vendor products. Select Star was chosen for two main reasons:
With Select Star as the single source of truth, Faire managed to harmonize usage across tables.
Taking on the task of synthesizing usage in dozens of tables with hundreds of columns was a very daunting task, but Select Star's Platform made it possible for us.
Using Select Star, the data team was able to build their core data model:
By quickly identifying interconnected columns, Faire added new columns to their data tables without having to worry about the usual change management issues when new data models are introduced.
“Select Star helped us answer all of these questions about each table and also about who to talk to,” Ben said.
More self-service data users and empowered internal teams
By incorporating Select Star into their daily workflows, Faire has successfully democratized access to core data, fostering a culture of transparency and informed decision-making. Making connections between tables, dashboards, and columns to specific users or teams enabled user-focused insights and facilitated early feedback.
All this led to the following results:
“As stewards of the company's data, we will continue to invest in these advanced features of Select Star, including linking tables, not just core ones, to their associated Airflow and SQL files in the code base, making it simple to audit pipelines and understand, as a user, how the data you're using is generated and where it comes from,” explained Ben.
When Faire initially integrated Select Star, the expectation was to primarily use it for documentation and search. However, as the platform became more integrated into their overall processes, it became a vital asset for understanding and navigating Faire's internal data platform.
The journey with Select Star has set Faire on a path of continuous excellence in data management and utilization. Moving forward, Faire plans to continue leveraging Select Star's capabilities to further streamline data management processes.
Want to learn more about how other companies are leveraging advanced data management tools like Select Star? Talk to us.