Monday, March 9, 2009

Trying to break Python memory consumption

I've been trying to find consistent method to raise a MemoryError exception in Python and have so far been unsuccessful. I'm mostly interested in real-world usage scenarios such has executing huge number arithmetic. Here is what my latest test looks like.
#Example; Massive memory consumption.

if __name__=="__main__":
list_obj=[]
cnt=0
while cnt<10**24:
list_obj.append(cnt**2)
cnt+=1
Here, we are simply appending progressively larger integers to a list. Executing this on my Ubuntu laptop results a huge increase in memory consumption. Because we are constructing a massive list object, this would make sense. However, a memory error isn't raised for me. Figuring that my laptop has enough memory to not let any kind of memory mishap take place for awhile, I fired-up a virtual machine with much less memory capabilities and executed the same program. The Python process ends up being killed. No MemoryError exception. What gives?

If I want to handle memory errors in Python, how can I deal with it if the process is terminated before I get a chance to?