Posts

Showing posts from June, 2013

Memory reclaiming in Python

Running GC-based languages on embedded systems always give a challenge to limit the physical memory amount taken by the processes. Python scripting is obviously a good example what would happen if you use long-running processes and which problems you could face. Let me show my research and the way I've used to fix the memory consumption.

First, a trivial example which is used in the Internet, and which is actually wrong and doesn't show the problem:

import gc
import os
iterations = 1000000
pid = os.getpid()


def rss():
    with open('/proc/%d/status' % pid, 'r') as f:
        for line in f:
            if 'VmRSS' in line:
                return line


def main():
    print 'Before allocating ', rss(),

    l = []
    for i in xrange(iterations):
        l.append({})

    print 'After allocating  ', rss(),

    # Ignore optimizations, just try to free whatever possible

    # First kill
    for i in xrange(iterations):
        l[i] = None

    # Second kill
    l = …