Magento 1.13 was released not too long ago. Among many new features (including optimized indexing and improved caching) the most exciting feature for me is something I had always asked for – Bulk updates. We will show you how to do Bulk updates with Magento’s 1.13 Api.
Demac Media Integration Framework
When we first began, our Proprietary Integration Framework was once referred to as the Data Donkey. No longer must our donkey carry a single, small bucket of water to and fro between ERP product information and the Magento backend. The donkey now operates a forklift! Lifting skids of product information all at once, and the speed difference we’ve been noticing in our testing so far makes this analogy very apt. The speed difference is very noticeable and very, very nice! We no longer need to update product information one at a time.
This comes with considerations though. Integration is never just a “Let’s throw all this stuff in here and That’s it! Hurray!” And this rings even louder with our new power.
Server load is always the main concern when dealing with integration, and while before it was never a super big deal since we were pushing such a small load, now becomes something much bigger to look out for. If you want to push product updates for 10k+ products (which you absolutely can do) we need to make sure we push small enough bulks so the server doesn’t Seppuku. Yet, we need sizable chunks to make sure this update is even worth doing at all!
It’s a balancing act
It’s a balancing act that we’re still experimenting and optimizing with. Every server is different and can handle different loads in different ways so this will always be a balancing act for any project, but we’ll be able to discover basic guidelines based on server specs/stack set up/other misc. factors.
So with all the warning out of the way. It’s time to get on with the code.
The new api endpoint is catalogProductMultiUpdate(SessionID, string productIds, catalogProductCreateEntity products, string storeview, string identifier)
Looking at this endpoint, there is definitely still one big factor missing. You can still only push bulks for one specific storeview. If you have code that wildly pushes in data for all kinds of different storeviews the solution isn’t all that difficult. Just group by storeview! Otherwise it looks pretty standard, but there is one MASSIVE gotcha that’s going to make life very, very bad if you mess up.
productIds are the identifiers you’re pushing in (we typically use sku, default magento is to use product_id, although what value you’re using is defined by string identifier)
catalogProductCreateEntity products are your full product information objects. Exactly what you would use for single updates.
So here we have two completely separate arrays, that have to work in tandem to update product information properly.
So essentially, if these two arrays are mismatched. ALL OF YOUR PRODUCT INFORMATION WILL BE WRONG!!! Let me re-iterate: ALL OF YOUR PRODUCT INFORMATION WILL BE WRONG!!!
productId MUST MATCH TO products
productId MUST MATCH TO products
We’re still having internal discussions about the best way to ensure that mismatches do not ever happen. We’re considering structure types such as SortedLists or Dictionaries.
The overhead on making tons of requests to the API and tons of responses coming back is rather large and this is going to cut down HUGE on that. We’re still testing and making sure this does work as intended and works well, and seeing how far we can stretch the server before it affects site performance. Any issues or logic gaps on our end using this endpoint could be disastrous.
Preliminary tests show this is a very promising addition to the Magneto API.
With great power, comes great responsibility. This endpoint addition has us very excited, but we must tread carefully.