Posts

Showing posts with the label linux

LinuxSys Probe Utility

I’ve published a new small GNU/Linux system probe utility to: read cgroups information (cpusets and memory) read /proc information probe CPU affinity and memory allocation The detailed description and usage could be found on the aforementioned link. Let me here describe a couple of use-cases as an intruduction. Probe CPU affinity and memory allocation Setting CPU affinity and allocating memory could silently fail (e.g. if cgroups control them). The utility uses those failures to find the boundaries: docker run -it --rm -v ` pwd ` :/opt \ --cpuset-cpus = "2-4" --memory = "100m" alpine \ /opt/linuxsys-probe -d 10MiB -r probe probe.cpu::affinity [0] = [*2, 3, 4] probe.cpu::affinity [1] = [*2, 3, 4] probe.cpu::affinity [2] = [*2] probe.cpu::affinity [3] = [*3] probe.cpu::affinity [4] = [*4] probe.cpu::affinity [5] = [*2, 3, 4] probe.cpu::affinity [6] = [2, *3, 4] probe.cpu::affinity [7] = [2, 3, *4] probe.cpu::affinity ...

Running arbitrary containers in LinuxKit

Debugging issues with LinuxKit images could be a serious challenge in some cases. Unfortunately, the documentation is not full enough to cover all the caveats, and in this article I'm going to show the general principle how to deal with arbitrary containers and run the commands in them.

Using LinuxKit on real hardware

LinuxKit is a promising toolkit for building secure, portable and lean operating systems for containers. One of its advantages is having a predictable container set (in other words, installed packages in the system), hence providing a homomorphic environment for QA and benchmarking tests. However, the project is fairly new, and users may have problems setting and running it up, especially on the real hardware. The documentation and packages are under way, but it's already possible to get it working.

wxWidgets and OpenSSL

Securing connections is always a good idea and OpenSSL provides a number of handy ways to do it. Blocking OpenSSL API is the most easiest to use and it could provide a working solution in no time, however it doesn't work with wxWidgets library so smooth as one may hope. I spent some time to find the reason why my application doesn't work as intended (plain sockets work just fine though) and had to do a little research on that matter. Regardless of the flags you provide to wxSocketServer it always creates a socket on the new connection with that ioctl call (UnblockAndRegisterWithEventLoop function in include/wx/unix/private/sockunix.h): ioctl(m_fd, FIONBIO, &trueArg); Basically, it just makes the socket non-blocking. I don't know is it a bug or feature, but we have it now and have to deal with it. Naturally all blocking OpenSSL API doesn't work and we have to develop the code using non-blocking behaviour. However, it's pretty easy to use if we emulate th...

Memory reclaiming in Python

Running GC-based languages on embedded systems always give a challenge to limit the physical memory amount taken by the processes. Python scripting is obviously a good example what would happen if you use long-running processes and which problems you could face. Let me show my research and the way I've used to fix the memory consumption. First, a trivial example which is used in the Internet, and which is actually wrong and doesn't show the problem: import gc import os iterations = 1000000 pid = os.getpid() def rss():     with open('/proc/%d/status' % pid, 'r') as f:         for line in f:             if 'VmRSS' in line:                 return line def main():     print 'Before allocating ', rss(),     l = []     for i in xrange(iterations):         l.append({})     print 'After allocating  ', rss()...

Minimalistic Linux threading

Embedded system development always poses certain challenges and one of them is the threading support. Including POSIX threads (pthread) is not an option sometimes, but fortunately Linux kernel allows to make LWP (light-weight processes) which are similar to threads. The key function here is the  clone  function. fork internally uses clone too, but clone allows to create child processes with the different settings including  CLONE_VM (share the memory between parent and children processes) which is essential for threading. Without further ado let me give an example, and I'll describe key pieces below: //Linux light-weight processes usage example #define _GNU_SOURCE #include <fcntl.h> #include <sched.h> #include <stdarg.h> #include <stdio.h> #include <stdlib.h> #include <sys/syscall.h> #include <sys/types.h> #include <sys/wait.h> // Default thread number #define THREAD_NUM 5 int message(const char *f...

Mini HOWTO: Getting file names in Zip-archives using Bash

Image
I'm gathering stats about my archives, and one of these is getting all the file names in them. There are some challenges about it, so let me show the required commands. Getting file names from the one archive: unzip -l /path/to/zip-file | tail -n +4 | head -n -2 | cut -c31- Executing pipelined commands in xargs: xargs -I {} -i sh -c 'command1 | command2 | ... | commandN' For my case I've used the expression: find . -iname "*.zip" -print0 | xargs -0 -n1 -I {} -i sh -c 'unzip -l {} | tail -n +4 | head -n -2 | cut -c31-' | sort | uniq -c Yeah, yeah, black magic, gotcha Good luck!

C modules unit-testing in Linux

Image
In spite of its age, C programming language is still very popular, especially for developing system or low-level software like drivers, compilers, virtual machines etc. And as any software, it have to be tested. Let me show brief introduction in unit-testing for C modules. There are many unit-testing frameworks for C, and one of the most well-known is cmockery . But I'll show the usage of much more simpler "framework" - FCTX . The main advantage of it is that it consists of just one header file, so it can be easily used for test tasks, small projects and examples. For calculating code coverage I use gcov / lcov tools. Gcov is included in GCC, so you don't have to install it. Lcov is a graphical front-end for Gcov and should be installed from the repository: $ sudo apt-get install lcov As a sample code for testing I'll use a simple hash function from Robert Sedgwicks Algorithms in C book: #include "hash.h" unsigned int RSHash(char* str, uns...

Brief introduction to Metasploit

Image
As a part of increasing IT-infrastructure security, penetration testing is one of the most valuable tools. Of course, system updates, using firewalls, IDS/IPS, right ACL and other methods are very efficient, but you can't be 100% assured that everything is fine. Security is a battle between defenders and attackers, and usually attackers are one step ahead in this battle. To be a good security professional, you have to know how attackers work, which tools and methods they are using, you have to be an attacker (of course, white-hat) - embrace Dark Side, but not be dominated by it and stay with Light Side. So, the Metasploit is one of must-be-known tool for every security professional: Metasploit provides useful information and tools for penetration testers, security researchers, and IDS signature developers. This project was created to provide information on exploit techniques and to create a functional knowledgebase for exploit developers and security professionals. The t...