In my new Magento Code That Sucks course, we covered how to fix the N+1 query problem in Magento. There are many other solutions to this problem! We could pop an item onto a queue with RabbitMQ, or orchestrate a custom query in a few different ways. But one other solution to process a large dataset in a short amount of time is by using the Zepgram_MultiThreading module. This module allows you to define a callback within a custom processor for either a search criteria builder query, collection, array, store, or website. After writing your code, you'll go to the CLI and run the new "bin/magento thread:processor" command. Within that command you can specify a timeout, the number of iterations to run, any environment-specific variables, and whether or not to show a progress bar while executing the command. When the process is executed, child processes are created and run the callback function for each item within the resultset. Since each callback is executed in a separate process, it allows for much faster processing times over the built-in Magento methods. You could also by pass memory limit restrictions, because the mem limit will be reset when each child process is created. Get the module on GitHub at https://lnkd.in/gjUd-C_N
Sebastian P.
6mo
I have never used that yet, but maybe some solution (not everywhere) are also PHP native generators https://www.php.net/manual/en/language.generators.overview.php . But this thread module looks more user friendly and usefull for other purposes.
Sam Abrar
5mo
Great Insight on using Zepgram_MultiThreading to speed up Magento! The ability to bypass memory limits and process data faster Mark Shust.
Pradip Shah
6mo
Shanmugam PK you could use this
Founder at luroConnect
6moI can see a lot of use for this - we routinely work with developers advising things like pagination and making sub processes to execute a subset. However, a word of caution - see how many cpus and free memory you have. If using production resources you could run out of them and slow down or even bring down the frontend!