Mini Tutorial: How to Set Up Bulk Imports in Shopify

Shopify’s REST api is really well structured to handle large imports of data. I will explain how to set up bulk imports in Shopify by updating product variants in bulk and how to respect Shopify’s call limit so that your integration application runs smoothly.

In my first Shopify tutorial I blogged on how to push a single product. The tutorial explained how to create a new shopify product and discussed various tools you can use to visualize the API’s response. This tutorial will use some of the concepts of REST# that I discussed before. For a quick refresher, visit Mini Tutorial on Uploading Products in Shopify with REST and C#.

updating variant products

Like most systems you need to work your way around it to get what needs to be done. One notable issue that I ran into when updating a shopify product was updating variant products.

The API has a PUT request method that helps you update a product in Shopify


PUT /admin/products/632910392.json
{
  "product": {
    "id": 632910392,
    "title": "New product title"
  }
}

This method doesn’t really work for updating variant products. For example, a naive approach would be to update a variant by adding it the JSON string and update it as a Shopify product. At least, that is how I approached it initially. Unfortunately, this method would not update your variants. Instead, Shopify will try to create a new variant, with a default option for you. Unless, the new variant is not in the store already, the variant won’t be created and the API would trigger an error informing you that the option of the variant is already created.

{"errors":{"variants":["is invalid"],"base":["Options are not unique"]}}

This response does make sense. As previously discussed, your product can vary by size and colour. So in the latter case, your variant options would be size and colour. Shopify prevents the user from creating Options that have already been created and hence the error above is triggered.

To tend to this issue, you would have to update each variant product individually. If the variant exists in your store update it, otherwise create a new variant. The following code snippets will help update and add a new variant.
If you don’t understand some methods and objects in the methods check my last post where I showed how to create a Shopify Product class using the fields of a Shopify product. I am sure it will help. The following code uses REST# to make requests to your Shopify store.


//update variant
private ShopifyProduct UpdateProduct(ShopifyProduct shopifyProduct, string productId)
{
   var serialize = SimpleJson.SimpleJson.SerializeObject(shopifyProduct);
   serialize = "{" + "\"product\":" + serialize + "}";
   var client = new RestClient("Store URL");
   var product = new ShopifyProduct();

   var resource = string.Format("/admin/products/{0}.json", productId);
   var request = new RestRequest(resource, Method.PUT);
   request.AddHeader("X-Shopify-Access-Token", "Your store key");
   request.AddParameter("application/json; charset=utf-8", serialize, ParameterType.RequestBody);
   request.RequestFormat = DataFormat.Json;
   var restResponse = client.Execute(request);

   if (restResponse.StatusCode.ToString().ToLowerInvariant().Equals("ok"))
   {
     var deserialize = new JavaScriptSerializer();
     product = deserialize.Deserialize(restResponse.Content).product;
   }
   return product;
}


//add new variant    
private Variant AddVariant(Variant variantProduct, string productId)
{
   var variant= SimpleJson.SimpleJson.SerializeObject(variantProduct);
   variant = "{" + "\"variant\":" + variant + "}";

   var product = new Variant();
   var resource = string.Format("/admin/products/{0}/variants.json",productId);
   var client = new RestClient("Store URL");
   var request = new RestRequest(resource, Method.POST);
   request.AddHeader("X-Shopify-Access-Token", "Your store key");
   request.AddParameter("application/json; charset=utf-8", variant, ParameterType.RequestBody);
   request.RequestFormat = DataFormat.Json;
   var restResponse = client.Execute(request);

   if (restResponse.StatusCode.ToString().ToLowerInvariant().Equals("ok"))
   {
     var deserialize = new JavaScriptSerializer();
     product = deserialize.Deserialize<VariantRoot>(restResponse.Content).variant;
   }
   
   return product;
}

Is that all I need to know to make bulk updates?

No! The latter method has its drawbacks. Updating and creating products individually takes a toll on your run time. Sending and receiving a response makes up for most of the overhead.Moreover, your application would have respect Shopify’s API call limit. Shopify runs a smart leaky bucket algorithm to make sure your application respects their API call limit.

Essentially, at a given time your application can only make 40 calls to their server. This means that if your integration exceeds that limit, Shopify would start throttling your requests. Your application should, on average, make 2 calls per second, anything over and you shall receive a nasty 429 “bucket overflow” error. You can learn more about the algorithm on Wikipedia. It is a pretty cool algorithm to keep data transmission in check.

Now what?

All is not lost. Where there’s will there’s a way. Whenever your application makes a request it returns a header identifying how many calls your application has made.

HEADERRESPONSE1

What you need to do is simply parse those numbers out of the string such that you have a fraction representing ratio of the total number of calls and the maximum call limit, which is 40 in this case. Once you know your current call count you can put your application to sleep. This will prevent your application from making requests for x seconds and help decrease your call rate significantly. This method does slow the application down but insures that your application doesn’t crash because of a 429.

Cool!

I know! You can let your code rip with these additions. These insights will definitely help your app with bulk imports using Shopify’s API. If you have a better approach to tackle these Shopify issues, feel free to share your insights!

Until next time, Happy Coding!!