F = T ∇ Sτ

One of my personal project has a requirement to have scripting support, and while the research of the options is still in progress, I’d like to share my preliminary results. The initial requirements for the scripting language were having the smallest footprint, but good functionality. That has given a start to the Embedded Languages Footprint project. The requirements are quite specific though (please see below), hence the most popular languages like Lua and Python are not considered suitable, but were included for the reference. For now, the most promising languages are Chibi-Scheme and Wasm3, but it’s still on-going research and final choice will be made later. The major requirements: C/C++ only (to ensure the best portability); No extra compilation dependencies (to build with the bare devkits); Actively maintained (to compile without its source code changes); Strong typing (to reduce the number of possible errors); Permissive license for both...

Running GC-based languages on embedded systems always give a challenge to limit the physical memory amount taken by the processes. Python scripting is obviously a good example what would happen if you use long-running processes and which problems you could face. Let me show my research and the way I've used to fix the memory consumption. First, a trivial example which is used in the Internet, and which is actually wrong and doesn't show the problem: import gc import os iterations = 1000000 pid = os.getpid() def rss():     with open('/proc/%d/status' % pid, 'r') as f:         for line in f:             if 'VmRSS' in line:                 return line def main():     print 'Before allocating ', rss(),     l = []     for i in xrange(iterations):         l.append({})     print 'After allocating  ', rss()...

A little note from my debugging experience. Split function works differently and I would say unexpectedly for empty string in different programming languages, and it can cause difficult to find bugs (especially if you use a lot of languages simultaneously). I've created a table with the popular programming languages: Language Split without parameters Split with parameter Python ''.split()=[] ''.split(',')=[''] Ruby ''.split()=[] ''.split(',')=[] JavaScript ''.split()=[''] ''.split(',')=[''] PHP N/A explode(',', '')=array(0=>'') Java N/A "".split(",")={""} C# "".Split()={""} "".Split(',')={""} As you can see sometimes it returns empty array, but sometimes an array with the one empty element. So please be careful with the split operation :)

Another comparison between: standard formatting; more advanced standard string.template ; Mako Genshi Jinja2 Here the code I used for measuring: #!/usr/bin/env python import sys NAME = 'name' def render1(): template = "<p>Hello %s!</p>" return template % NAME def render2(): from string import Template template = Template("<p>Hello ${name}!</p>") return template.substitute(dict(name=NAME)) def render3(): from mako.template import Template template = Template("<p>Hello ${name}!</p>") return template.render(name=NAME) def render4(): from genshi.template import MarkupTemplate tmpl = MarkupTemplate('<p>Hello $name!</p>') stream = tmpl.generate(name=NAME) return stream.render('xhtml') def render5(): from jinja2 import Template template = Template('<p>Hello ...

Memory consumption is slightly specific to my area of software development now, but I did some research recently and maybe these results can be useful for others. Of course, I know that precious comparison is very difficult to carry out, but actually I needed only overall picture. And let me admit that results are pretty interesting and even frustrated (at least for me). As a basis I took so-starving project, and measured initial RSS (Resident Set Size) of the each process (local development webservers). Platform: x86_64 Linux (latest Ubuntu with all updates). As a reference, here is the RSS of the interpreters in interactive console mode: Interpreter Version RSS (kB) stackless python 2.6.4 3916 ruby (via irb) 1.8.7 4664 python 2.7.1 5624 php 5.3.5 6924 v8 (via node.js shell) 2.5.9.9 8796 One more reference - the most simplest WSGI app ( example  in the Python documentation). It's RSS: 7336 Kb , so I assume it's almost impossible to consume ...

One of the biggest challenges with embedding platforms is the limitation related to file sizes. Here is the hint how to make executables smaller - strip them and pack them: strip is a tool from GNU binutils , it discards symbols. Usually the platform toolchain has one. upx is an excellent executable packer. Can be downloaded as binary or sources from the UPX sf.net site . As an example, I'll show you the packing of Python 2.6.7 binary: $ ls -s --block-size=KB python 6493kB python $ strip -s python $ ls -s --block-size=KB python 1696kB python $ upx --best python                        Ultimate Packer for eXecutables                           Copyright (C) 1996 - 2010 UPX 3.05        Markus Oberhumer, Laszlo Molnar & John Reiser   Apr 27th 2010         File size         Rati...

A little hint for Python developers who use it for embedded or unconventional platforms (like Cray supercomputers if you're lucky): it can be compiled and used without any dynamic libraries. I've got the problem with stripped libc.so - some Python shared object (like _socket.so ) try to use it, but can't find anything because it's stripped. The only choice I had is using Python without these shared objects. Fortunately, Python support it out of the box. After configuring it, you can use Modules/Setup file to set up which modules have to be compiled within the Python binary: The build process works like this:  1. Build all modules that are declared as static in Modules/Setup,     combine them into libpythonxy.a, combine that into python.  2. Build all modules that are listed as shared in Modules/Setup.  3. Invoke setup.py. That builds all modules that     a) are not builtin, and     b) are not listed in Modules/Setup, and   ...

Just want to share some useful links about embedding Python to your C-based application: the main article:  Embedding Python in Another Application additional article that shows peculiarities of multithreading, sockets and shared memory: Embedding Python in C/C++ ( Part1 , Part2 ) Cython (or Pyrex ) can be used to reduce handwritten code for Python interoperability: A quick Cython introduction If you have doubts about Python size, there are some minimal implementations: see Embedded Python article. Let me quote tinypy : tinypy is a minimalist implementation of python in 64k of code ... What more could you possibly want?? a pony? However, I highly recommend to use classic CPython implementation (basically because it has a huge number of contributors and supporters, and has an excellent documentation). It can be stripped up to 1-2 megabytes depending of your requirements.

Review Board is a powerful web-based code review tool that offers developers an easy way to handle code reviews. It scales well from small projects to large companies and offers a variety of tools to take much of the stress and time out of the code review process. Review Board is written in the Python programming language and makes use of the Django web framework. Installation   Install auxiliary packages if needed and all its dependencies: $ sudo apt-get install python-setuptools $ sudo apt-get install python-svn $ sudo apt-get install python-subversion $ sudo apt-get install apache2 $ sudo apt-get install libapache2-mod-python $ sudo apt-get install git Clone the ReviewBoard package and install it: $ git clone git://github.com/reviewboard/reviewboard.git $ cd reviewboard $ sudo python setup.py develop Also install post-review tool: $ sudo easy_install -U RBTools Set up the required site for the ReviewBoard (for Apache/SQLite backend, otherwise - see Cr...

I've found a good wiki-article about programming languages faults and want to share a link:  http://wiki.theory.org/YourLanguageSucks . The "winner" is PHP as usual, but Python is also noticed (as well as Ruby :) ). I agree with almost everything there, but in my own eyes Python is still the best choice for programming. Nevertheless, I also want to point out some other recent links related to PHP (I really shouldn't but just can't help doing it): PHP Sucks PHP Must Die What are the horrors of PHP? What factors during the development of PHP contributed to it being such a poorly designed language? And a quote from the interview with Rasmus Lerdorf  (the creator of PHP): I don't know how to stop it, there was never any intent to write a programming language [...] I have absolutely no idea how to write a programming language, I just kept adding the next logical step on the way.

There is a common mistake about "If everybody use it, so I also have to use it - millions of people can't be wrong". Apparently, they can, and huge codebase, support and knowledge mean nothing, otherwise we would still use Fortran, Cobol, Basic and other almost died monsters. Also there is an another common mistake about "If big corporation use it, I also have to use it". It's very doubtful, almost always decision are made in hurry and/or by wrong people and/or without serious consideration. And after some period of time, it's difficult to nullify previous decision because it would require huge efforts. Good example - Facebook. Let me quote the presentation  HipHop for PHP Tech Tasting : PHP is problematic for Facebook: High CPU usage High memory usage Reuse of PHP logic in other systems Extensions are hard to write for most PHP developers But huge codebase, strange affection towards PHP (in what universe "loose typing and universal ar...

UPDATE (07/18/17): The original article was written in 2011 and pretty much outdated, I've updated the numbers and conclusions. I've got a question about which programming language is preferable for the website development for embedded systems (with limited resources). Here is my small investigation in a table form. Please note that the question was about only these 3 programming languages - there are better candidates for the embedded systems now (for example, Rust). The Memory and Performance overhead numbers are based on the n-body benchmark and calculated as relatives to "C gcc #4" measurements. Python 3 Node.js PHP Memory Overhead x7.62 x29.04 x8.53 Type System Strong Typing Weak Typing Weak Typing Vulnerabilities 220 1411 5626 Performance Overhead x78.31 x2.82 x30.12 Documentation Excellent Excellent Average C Bindings Excellent Average Poor Code Readability With PEP8 it can be almost perfect ESLint enforces very good style PEAR Coding Stan...

The short answer - yes . The reasons are below: There are well-knows systems (like YouTube) that shows Python suitability: YouTube Architecture The most performance problems are related with not a language speed, but with communication and databases speed. Let me show the example from my own experience: I've worked on project that process a huge amount of data (about inserting 100-200k records a day, and selecting from about 100-200 million records 10-20 times per second). The bottleneck was a database, not the language speed. All indexes were pretty complex, and selecting can take up to 30-90 seconds which was inappropriate. We had to change architecture: use data pool, caches, AMQP (with RabbitMQ server), and after that we don't know any problems at all with performance. Right tools means everything. For example, if you have to serve enormous web-requests, just use the suitable webserver like Tornado . Especially it is useful for serving Comet -based web applications. A...

The CodeExample renders a code example box that supports syntax highlighting. It support three types of examples: simple, correct, and incorrect. I've made the major changes to the plugin and now it is fully compatible with Trac 0.11 and Trac 0.12. Changelog from version 0.3: Support for multiply repositories (Trac 0.12 and upper) has been added. Collapsing/expanding code blocks have been implemented. Ability to change title has been added. Options using has been reimplemented. Two examples (the first gets source from repository, the second contains the code inside): {{{ #!CodeExample ## type = good ## title = GPGData sample code ## repo = MacGPGME ## path=GPGData.m ## regex="static void releaseCallback\(.*" ## lines=4 #!objective-c }}} {{{ #!CodeExample ## type = bad #!haskell fibs = 0 : 1 : [ a + b | a <- fibs | b <- tail fibs ] }}} These will be rendered as: See details on trac-hacks CodeExampleMacro...

In the article "Using Python command line" I've described some executable modules and now want to give more details about the most interesting ones. I do not pretend to cover everything, just want to point out some directions where you can go if you find this useful. SimpleHTTPServer The executable module which I use if there is no possibility to send files from one computer to another by other methods (ssh, smb, ftp etc). It is not so fast, but can be very useful sometimes. The usage is very simple - just change directory to the required one, and start SimpleHTTPServer: calendar Very simple module - just shows the year calendar: urllib Very simple download manager. Useful if you want to download a file or a page via command line, but don't have wget utility (for example, in Windows). Now you don't even need to open IE for FF downloading: $ python -m urllib "http://download.mozilla.org/?product=firefox-3.6.6&os=win&lang=en-US" > firefox_se...

Recently I had a task to write a command file for Windows that run a simple Python commands via python –c command . The manual says that command may contain multiple statements separated by newlines. Unfortunately, Windows console doesn’t allow it (and Ctrl-T trick doesn’t help) so I had to find a workaround. The intuition and language reference helped me to find out that the simple statements can be separated by semicolons like in C/C++ (however, these languages allow to separate all the commands, not the only simple ones). Let me give an example. Imagine that accidentally I want to know the error message corresponding to the error code 9. Of course, I can read manuals and documentation, but there is a more simple way: $ python -c "import os;print os.strerror(9)" Bad file descriptor But this "trick" works only with simple statements. It is not allowed to mix simple and compound statements, so the following command will not work: python -c "import os;...

Updated list of the projects I'm working on: Mail Dispatcher (role: admin, platform: Python/PyGTK) - A tool for dispatching (basically, deleting) email messages on POP3 server via Plain or SSL connection with advanced filtering capabilities. DHCP Explorer (role: admin, platform: Python) - A console cross-platform tool for locating all available DHCP servers. sdict2db (role: admin, platform: C#/WinForms) - Parser for SDict-based format dictionaries with ability to save in a SQL server (with creating table, index and filling data) and in a text file (SDict text format). DNN Dictionary (role: admin, platform: C#/DotNetNuke) - A DotNetNuke module for translating texts using SDict-based (http://sdict.ru/en/) dictionaries. It works using AJAX mechanism so there is no pages reloading during translating. Geek Workspace (role: admin, platform: Python/PyQt4) - Geek Workspace is a desktop environment aimed for developers, scientists, engineers and students. The key point is a progr...

GStreamer is a library for constructing graphs of media-handling components. The applications it supports range from simple Ogg/Vorbis playback, audio/video streaming to complex audio (mixing) and video (non-linear editing) processing. GStreamer is released under the LGPL, so it can be used in commercial applications. The power of GStreamer is in its module infrastructure - you're working with certain "blocks", combine it with "pipes" and got results without any coding (via gst-launch utility). Of course, after playing with gst-launch you can write an application in any language you prefer (GStreamer supports various bindings and due to fact that GStreamer is written in C, I do not see any problems to bind it with an application that is developed in any serious programming language). I do not want to explain GStream basics, but move forward to the blog topic - creating videoconference application. If you want certain tutorial, please take a look to the P...