Let me just start by saying that this post is one of those weird ones where it probably serves more function as a “lookup” for a very fringe issue in the off chance we run into it again.

We’ve recently been doing some work with the book publishing industry ONIX file format, which for those who don’t know, is an XML schema for moving book product data between publishers, distributors and retailers.  When we were asked to bring the ONIX format into the AspDotNetStorefront eCommerce platform, we decided to build an integration service that used LINQ to XML to work with ONIX.  We’ve used LINQ to XML in the past and found that it’s speed to develop with was important for this job (the code is just really nice to write).

All was going well until we started testing the service with ONIX files that contained north of 20,000 records, roughly 75MB of XML in size per file.  The service started to chew up several GB of RAM while running, clearly the result of a memory link somewhere in the entire process. After smashing our heads against the wall I decided to grab a free trial of the Redgate ANTS Performance Profiler to see if they might shed some light as to why we were leaking memory from a rather simply set of functions.

The ANTS Profiler showed us that our major culprit was the XElement object that is part of LINQ to XML.  So we began googling around to see if anybody else was having similar problems.  There were hints here and there that some were experiencing this issue but nobody seemed to be solving the issue except for changing their method of XML manipulation to using traditional stream readers/writers.

Then, one of our senior developers (Dimitri) took a look at the code and saw that we weren’t calling RemoveAll() on our container XElement that held all the manipulated xml.  Before you comment, know that this XElement was contained in a separate class that was instantiated + destroyed every 50 records or so in an attempt to work in smaller more manageable data chunks.

So, we added the .RemoveAll() to our root XElement object and bingo, no more memory leaks!  The lesson here is that the XElement is a fully managed object that does not implement IDisposable, so there is no nice way to force it to remove from memory.  The .RemoveAll() method at least seems to destroy a lot of the memory footprint of this rather wierd beast.

Hope this helps someone else out!