Not Enough Storage is Available to Process This Command Errors

We recently started having problems on one of our servers.  Processes would start dieing with “Not enough storage is available to process this command” errors. It turns out that this is a catch all error that Windows uses when it runs out of resources of almost any kind.  It rarely has anything to do with the amount of memory or disk installed on your system.  Some of the things that can causes the error to occur are:

  1. System paged pool is exhausted.
  2. System non-page pool in exhausted.
  3. Maximum IRP stack size has been exceeded.
  4. Any number of other things that I have not been able to find documented.

First, we need to define some terms.

System paged pool

The system paged pool is a section of memory set aside for use by the operating system kernel.  Memory in the pool can be swapped (paged) in and out of physical memory as needed.  The size of the pool is determined by the operating system and is fixed irrespective of the amount of physical memory in your system.  You can make suggestions about the maximum size of this pool by editing the registry but the operating system ultimately decides how big it should be.

System non-paged pool

The system non-paged pool is similar to the system paged pool except that memory in this pool cannot be swapped (paged) into and out of memory.  Like the paged pool, this size of the non-paged pool is fixed irrespective of the amount of physical memory in your system.  The size of this pool is determined by the operating system.

There is an excellent description of the system memory pools here.

IRP stack size

The IRP stack size is the number locations available in the I/O Request Packet stack.  It is sometimes necessary to increase the size of the IRP stack depending on the software that is installed on your machine. Certain device drivers may require more entries on the IRP stack than others.

If you are encountering this error due to either of the system pools being exhausted you will probably see entries in the system event log.  These entries will have an event ID of 2020 (unable to allocate from the paged pool) or 2019 (unable to allocate from the non-page pool).

If you are running out of pool space it may be that the operating system is either not allocating enough memory to the pool or is not trimming unneeded entries from the pool fast enough.  Both of these problems can be addressed (within limits) by changing certain entries in the registry.  All of these registry entries can be found in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Session Manager\Memory Management.  After changing any of the values, you must reboot for the change to take effect.

To increase the size of the paged pool change the PagedPoolSize value.  The default value of zero allows the operating system to pick the size that it thinks is best.  Entering a value will cause the operating system to use that as a maximum size for the paged pool.  You might end up with less than this depending on your hardware architecture, the amount of physical memory installed, etc.  Entering 192000000 would set the maximum paged pool size to 192MB.  Entering 0xffffffff tells the operating system to use the maximum possible value.  On Windows 2003 that is 650MB.  For Windows 2000 and XP it is 470MB.

No matter how big your paged pool is you might run into problems if the system gets sufficiently busy because, under extreme load, the operating system may not trim unneeded entries from the paged pool quickly enough.  By default, Windows starts trimming the paged pool when usage reaches 80%.  You can change this  behaviour by changing the PoolUsageMaximum value.  Try setting this to 60% to force the operating system to start trimming sooner.  If that isn’t enough to solve your problem you can try smaller values.

If changing these values doesn’t fix your problem you need to start monitoring your system pool usage.  You can find the amount of memory being used by the paged and non-paged pools by looking on the Performance tab in Task Manager.  The values will be shown in the Kernel Memory section of the window.  If either of the pools appears to be constantly increasing in size then you have probably either got a buggy driver or an application that is leaking handles (handles are allocated from the paged pool).

Buggy drivers are extremely difficult to track down.  In my experience it is usually a printer driver causing the problem.   I just start de-installing drivers, starting with the most recently installed working backwards, until the problem goes away.

However, it is quite easy to find out if an application is leaking handles.   Start Task Manager and select the Processes tab.  Use the View | Select Columns dialog to enabled the Handle Count column.  Then click on the Handle Count column header to sort by the number of handles used.  Just look for the applications with the largest number of handles.  There is no hard and fast rule to determine how many handles are too many but anything over 5,000 should be suspect.  Monitor any applications over this value to see if the number of handles keeps increasing.  If you find one with a constantly increasing handle count, try killing the process and see if your paged pool usage decreases and stays low.  If it does, you have found the culprit and it is time to raise hell with the application developer.

One problem with Task Manager is that it only tells how much pool memory you are using, it doesn’t tell you how pool memory is available.  Fortunately, there is a tool that (with a bit of coaxing) will tell what your pool limits are.  You need to get your hands on a tool called Process Explorer which is available here.   Once you have Process Explorer installed you need to get a copy of the Windows Debugging Tools which are available here. Once you have the debugging tools installed you need to tell Process Explorer where to find them.  Select Options | Configure Symbols from the menu.  Browse to find DbgHelp.dll.  It will be in the folder where you installed the debugging tools.  Then enter SRV*C:\websymbols* as the Symbols Path.  Stop and restart Process Explorer.  Then select View | System Information from the menu.  The current pool usage and the pool limits are now shown in the Kernel Memory section of the Window.

If system pool space doesn’t seem to be a problem for you then you need to try increasing the IRP Stack Size.  This is changed in the registry in the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanserver\parameters key.  You need to change the IRPStackSize value.  The default value for this parameter is 11 so you will want to set it to value greater than that.  You can pick a value anywhere between 11 and 50.  There is no way of knowing what a good value for your system might be, it is largely a matter of trail and error.  Try starting with a value of 25 and go from there.

This is far from an exhaustive treatment of this complex subject but hopefully this will give you a place to start troubleshooting.

Leave a Reply