Magento 1 – collection iteration optimization
In this tutorial I show you how to optimize collection iteration. If you use large collections, you may have discovered performance problems. Especially for low cost hosting systems, there may be drawbacks if you iterate through a big collection. In the worst case, you end up in a memory allocation error. I show you how to do it right.
Magento 1 – collection iteration optimization
The following code example is the suggested way to loop a collection:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | $products = Mage::getModel('catalog/product')->getCollection()->addAttributeToSelect('*'); $products->setPageSize(1000); $pages = $products->getLastPageNumber(); $currentPage = 1; do{ $products->setCurPage($currentPage); $products->load(); foreach ($products as $product){ //here you can do things with a single product } $products->clear(); echo "Finished page $currentPage of $pages \n"; $currentPage++; } while ($currentPage <= $pages); |
In the first line we get our collection of all products in store. This is basic. In the second line we set a page size limit. So each package will have not more than 1000 products. With getLastPageNumber() we get the number of all pages. So in a do while loop we loop every 1000 product collection till the number of pages. In this loop we initialize the current package by setting the current page number with setCurPage() and then we load() this collection. In a foreach loop we can do our code for each single product.
The load() function creates the memory consumption. So if you call it in the first line, we need memory for all products in store. If we do it in our do loop, we only need memory for 1000 products. If that still does not work, try to reduce page size.
This code works not only for product collections. You can also use it with order, customer or any other Magento object collection.
Memory problems?
Have you ever seen this error message?
1 | Fatal error: Allowed memory size of 128018523 bytes exhausted (tried to allocate 45 bytes) |
I suppose that nearly every PHP coder or Magento developer have seen this message already. It says, that your current PHP process uses already more than 128 MB RAM space and tries to allocate additional bytes. This causes an error and your PHP process dies. But why?
Theory
On each operating system, a process uses space in memory. For a PHP process, you need a bit memory for your script source code and a lot of additional memory for variables. On Linux systems you have a configuration file called php.ini where you can set the maximum usable memory for a single process. The value is called memory_limit. You can set memory_limit in three ways:
- php.ini
This may only be editable on a root or virtual server. This is the main configuration file for your PHP version. You set the limit with this line of code:1memory_limit = 512M ; - .htaccess
On most shared or managed hosts you have the option to redeclare php.ini values in you .htaccess file. This may be deactivated by your host, so is is not garanteed that this works:1php_value memory_limit 512M - php script
In your PHP script you can set this limit like:1ini_set('memory_limit', '512M');
The maximum allowed memory may be limited by your hoster. If the given value does not change anything on your error message, your only option is to optimize your script that it do not use that amount of memory.
Conclusion
You may discover memory limits with large collections. I showed you a good way to deal with it. You spit up the collection in some parts and loop only one part at a time. This collection iteration optimization works as expected and is very simple to implement.
