Shopify is turning out be a very scalable eCommerce platform. Its API is stable enough to handle large product catalogues and a lot of data! Clients often ask us to integrate large amounts of data. This data can be merged with metafields to provide more functionality or can be used to integrate product images. In either case it is essential that the catalogue is correctly and completely retrieved from the store. To maintain product information,one needs to decide if a product needs to be created or updated. In either case, it is essential to grab all the data from the store to perform look ups.
In this blog post I will discuss how you can use the API to your advantage and display all the products on your Shopify store, so that product look ups can be done within the code. This procedure not all applies to products, but also applies to images, variants and metafields.
Getting all the products from the shopify store
The following endpoint gets all the products from the Shopify store.
This seems simple enough, but this get has a maximum of 50 products. If your store contains more than 50 products, then you would have to add an extra parameter to the endpoint url.
This will return a maximum of 250 products from your store. What if you have more then 250? Can’t you just increase the limit to the total number of products in your catalogue to 1000? As simple as it sounds, it doesn’t quite work in that way… Shopify uses a technique called pagination, which basically divides the response to your request into separate memory chunks. This helps in overall performance of the system, by limiting the waiting time required to download the product data, when a request is made.
Obtaining the First 50 Products
The following endpoint can be used to get the first 50 the products in the first page.
The above endpoint, would return just 50 products. To get all the products, your code would have to iterate through all the pages. To get an upper bound on the total number of pages, you would have use a bit of Calculus.
Gotya! Just kidding!
All you need to do is a bit of math to figure out how many requests the function would have to make, based on the page size (the total number of products per page). You would also need obtain a tally of all the products in the sore. The following endpoint will help you get that:
Once, you have the count of the total number of products, getting all the products is fairly easy. For example, if the store has 1000 products, with a page size limit of 50, then the total number of pages will equate to 1000/50 = 20 pages.
To get all the products, the function would have to loop through all 20 pages:
Increasing the limit to 250
The limit can also be increased to 250, in which case the same request can be made (1000/250) = 4 times to get all the products from the store. The choice of deciding between the page size and the total number of pages, is obviously left up to the developer and the resource at his/her disposal.