F = T ∇ Sτ

Running GC-based languages on embedded systems always give a challenge to limit the physical memory amount taken by the processes. Python scripting is obviously a good example what would happen if you use long-running processes and which problems you could face. Let me show my research and the way I've used to fix the memory consumption. First, a trivial example which is used in the Internet, and which is actually wrong and doesn't show the problem: import gc import os iterations = 1000000 pid = os.getpid() def rss():     with open('/proc/%d/status' % pid, 'r') as f:         for line in f:             if 'VmRSS' in line:                 return line def main():     print 'Before allocating ', rss(),     l = []     for i in xrange(iterations):         l.append({})     print 'After allocating  ', rss()...

Embedded system development always poses certain challenges and one of them is the threading support. Including POSIX threads (pthread) is not an option sometimes, but fortunately Linux kernel allows to make LWP (light-weight processes) which are similar to threads. The key function here is the  clone  function. fork internally uses clone too, but clone allows to create child processes with the different settings including  CLONE_VM (share the memory between parent and children processes) which is essential for threading. Without further ado let me give an example, and I'll describe key pieces below: //Linux light-weight processes usage example #define _GNU_SOURCE #include <fcntl.h> #include <sched.h> #include <stdarg.h> #include <stdio.h> #include <stdlib.h> #include <sys/syscall.h> #include <sys/types.h> #include <sys/wait.h> // Default thread number #define THREAD_NUM 5 int message(const char *f...

Another comparison between: standard formatting; more advanced standard string.template ; Mako Genshi Jinja2 Here the code I used for measuring: #!/usr/bin/env python import sys NAME = 'name' def render1(): template = "<p>Hello %s!</p>" return template % NAME def render2(): from string import Template template = Template("<p>Hello ${name}!</p>") return template.substitute(dict(name=NAME)) def render3(): from mako.template import Template template = Template("<p>Hello ${name}!</p>") return template.render(name=NAME) def render4(): from genshi.template import MarkupTemplate tmpl = MarkupTemplate('<p>Hello $name!</p>') stream = tmpl.generate(name=NAME) return stream.render('xhtml') def render5(): from jinja2 import Template template = Template('<p>Hello ...

Memory consumption is slightly specific to my area of software development now, but I did some research recently and maybe these results can be useful for others. Of course, I know that precious comparison is very difficult to carry out, but actually I needed only overall picture. And let me admit that results are pretty interesting and even frustrated (at least for me). As a basis I took so-starving project, and measured initial RSS (Resident Set Size) of the each process (local development webservers). Platform: x86_64 Linux (latest Ubuntu with all updates). As a reference, here is the RSS of the interpreters in interactive console mode: Interpreter Version RSS (kB) stackless python 2.6.4 3916 ruby (via irb) 1.8.7 4664 python 2.7.1 5624 php 5.3.5 6924 v8 (via node.js shell) 2.5.9.9 8796 One more reference - the most simplest WSGI app ( example  in the Python documentation). It's RSS: 7336 Kb , so I assume it's almost impossible to consume ...

One of the biggest challenges with embedding platforms is the limitation related to file sizes. Here is the hint how to make executables smaller - strip them and pack them: strip is a tool from GNU binutils , it discards symbols. Usually the platform toolchain has one. upx is an excellent executable packer. Can be downloaded as binary or sources from the UPX sf.net site . As an example, I'll show you the packing of Python 2.6.7 binary: $ ls -s --block-size=KB python 6493kB python $ strip -s python $ ls -s --block-size=KB python 1696kB python $ upx --best python                        Ultimate Packer for eXecutables                           Copyright (C) 1996 - 2010 UPX 3.05        Markus Oberhumer, Laszlo Molnar & John Reiser   Apr 27th 2010         File size         Rati...

A little hint for Python developers who use it for embedded or unconventional platforms (like Cray supercomputers if you're lucky): it can be compiled and used without any dynamic libraries. I've got the problem with stripped libc.so - some Python shared object (like _socket.so ) try to use it, but can't find anything because it's stripped. The only choice I had is using Python without these shared objects. Fortunately, Python support it out of the box. After configuring it, you can use Modules/Setup file to set up which modules have to be compiled within the Python binary: The build process works like this:  1. Build all modules that are declared as static in Modules/Setup,     combine them into libpythonxy.a, combine that into python.  2. Build all modules that are listed as shared in Modules/Setup.  3. Invoke setup.py. That builds all modules that     a) are not builtin, and     b) are not listed in Modules/Setup, and   ...

Just want to share some useful links about embedding Python to your C-based application: the main article:  Embedding Python in Another Application additional article that shows peculiarities of multithreading, sockets and shared memory: Embedding Python in C/C++ ( Part1 , Part2 ) Cython (or Pyrex ) can be used to reduce handwritten code for Python interoperability: A quick Cython introduction If you have doubts about Python size, there are some minimal implementations: see Embedded Python article. Let me quote tinypy : tinypy is a minimalist implementation of python in 64k of code ... What more could you possibly want?? a pony? However, I highly recommend to use classic CPython implementation (basically because it has a huge number of contributors and supporters, and has an excellent documentation). It can be stripped up to 1-2 megabytes depending of your requirements.