Airbnb/Apache Superset and Shopify

We first discussed Superset here. The post was about first impressions, potential, pros and cons vs more mature solutions like Tableau, I suggest to read that post too as there is a lot of information and thoughts about Superset there. But I wanted to test this tool a bit more, so I thought why not trying something out with Shopify?

Shopify is an ecommerce cloud application and as such:

  • It generates lots of data
  • It needs to be real time (you don’t want to wait one week to see all the orders booked in your store)

I also think Shopify store owners want to dig into their data, so a solution that poses no limits to exploration such as Superset would be a great fit.

There are many reporting solutions specific to Spotify, but what if one has data in multiple apps? What if you have also inventory on Amazon FBA? Or Facebook ads to analyze? With Superset you could have all this data at your fingertips. I could also provide canned analytic dashboards that are best practice in the industry while still allowing users to explore when and where needed.

The final result

Below you can find a screenshot of the final result in Superset. It looks pretty good already while I did not spend much time building it.

You can check the demo yourself at superset.assemblinganalytics.com (user and password “demo”).

enter image description here

The above updates every 30 or so minutes.

The solution

The initial idea was to put together a self-service setup with no code or IT required. I managed most of that, this is the breakdown of the approach:

  • I use an online integration tool to pull data near realtime from Shopify (Stichdata or Blendo fit well here)
  • I use staging tables to park the data from the integration tool into my data warehouse
  • I build views on top of the staging tables
  • I pull data from views into Superset (easy, Superset connects natively to Postgres)

Sure, the setup above is quite simplistic and does not take into account complex use cases or big data or any speed optimization, but it works and it is very quick to put together. You can see the overall architecture in the screenshot below.

enter image description here

Next steps

This architecture works and it is effective. If you want a place to have realtime canned reports and the possibility to do a deeper exploration of the data, this solution is powerful, not only for Shopify, but for any other cloud application you might be using (Salesforce and Close.io are supported too!). In terms of next steps, I want to add additional apps (close.io maybe?) and stress test Superset a bit more. By the way feel free to suggest app ideas in the comments below.

Leave a Reply

Your email address will not be published. Required fields are marked *