top button
Flag Notify
Site Registration

How to go about debugging memory leak problems in Python?

0 votes
390 views

I've written a program using Twisted that uses SqlAlchemy to access a database using threads.deferToThread(...) and SqlAlchemy's scoped_session(...). This program runs for a long time, but leaks memory slowly to the point of needing to be restarted. I don't know that the SqlAlchemy/threads thing is the problem, but thought I'd make you aware of it.

Anyway, my real question is how to go about debugging memory leak problems in Python, particularly for a long running server process written with Twisted. I'm not sure how to use heapy or guppy, and objgraph doesn't tell me enough to locate the problem. If anyone as any suggestions or pointers it would be very much appreciated!

posted Jun 13, 2013 by anonymous

Share this question
Facebook Share Button Twitter Share Button LinkedIn Share Button

1 Answer

+2 votes
 
Best answer

Analysing memory leaks is really difficult: huge amounts of data is involved and usually, it is almost impossible to determine which of the mentioned objects are leaked and which are rightfully in use.
In addition, long running Python processes usually have degrading memory use - due to memory fragmentation. There is nothing you can do against this.

Therefore: if the leak seems to be small, it may be much more advicable to restart your process periodically (during times where a restart does not hurt much) rather than try to find (and fix) the leaks. Only when the leak is large enough that it would force you to too frequent restarts, a deeper analysis may be advicable (large leaks are easier to locate as well).

I have analysed memory leaks several times for Zope applications.

Zope helped me much by its "Top Refcount" functionality. This uses the fact that a class/type instance (in many cases) holds a reference to the corresponding class/type instance (it seems not to work for all elementary types). Thus looking at the refcount of the class/type gives you an indication how many instances of this class/type are around. Zope presents this information sorted by number. Then you send requests against Zope and reexamine the information:

You get something like:

 Class June 13, 2013 8:18 am June 13, 2013 8:18 am Delta
...ApplicationManager 15 22 +7
...DebugManager 9 12 +3 

In the case above, my requests have created 7 additional "ApplicationManager" and 3 additional "DebugManager" instances.

If Zope objects for which this functionality works leak, then this is a powerful tool to detect those object classes.

You could implement something similar for your server.

As mentioned, the approach does not work for (many; all?) elementary Python types. Thus, if the leak involves only those instances, it cannot be detected this way.

Memory leaks are often introduced by C extensions - and do not involve Python objects (but leak C level memory). Those, too, cannot be analysed by Python level approaches.

answer Jun 13, 2013 by anonymous
Similar Questions
+1 vote

The change in integer division seems to be the most insidious source of silent errors in porting code from python2 - since it changes the behavior or valid code silently.

I wish the interpreter had an instrumented mode to detect and report such problems.

+2 votes

I have a pile of C code that I wrote that I want to interface to via the ctypes module ( https://docs.python.org/3/library/ctypes.html ).

The C code uses the Boehm-Demers-Weiser garbage collector ( http://www.hboehm.info/gc/ ) for all of its memory management. What I want to know is, who owns allocated memory? That is, if my C code allocates memory via GC_MALLOC() (the standard call for allocating memory in the garbage collector), and I access some object via ctypes in python, will the python garbage collector assume that it owns it and attempt to dispose of it when it goes out of scope?

Ideally, the memory is owned by the side that created it, with the other side simply referencing it, but I want to be sure before I invest a lot of time interfacing the two sides together.

0 votes

Previously, we found that our python scripts consume too much memory. So I use python's resource module to restrict RLIMIT_AS's soft limit and hard limit to 200M.
On my RHEL5.3, it works OK. But on CentOS 6.2 + python2.6.6, it reports memory error(exceeding 200M). And I tested with a very small script, and result is out of my expect, it still use too much memory on my CentOS 6.2 python:
I could understand that 64 bit machines will occupy more virtual memory than that on 32 bit, because the length of some types are not the same. But I don't know why they differs so greatly(6M to 180M), Or is this only caused by that python2.6 on CentOS 6.2's memory allocation is different from python's default one? Could you kindly give me some clues?

...