[System] Use buildroot to create an embedded distribution

Get the buildroot git repository:

git clone git://git.buildroot.net/buildroot

Follow blindly the README sacred words.

make list-defconfigs

Let's start with a defconfig of the vexpress arm machine

make qemu_arm_vexpress_defconfig

The versatile defconfig doesn't support device tree, and it's a pain in the ass to make it work, moreover the vexpress board support is better in qemu.

launch the build:

make

Build an awesome card castle.
Come back when it's finished.

Read the readme.txt at board/qemu/arm_vexpress
Do what you are told to do.

qemu-system-arm -M vexpress-a9 -kernel output/images/zImage -drive file=output/images/rootfs.ext2,if=sd -append "console=ttyAMA0,115200 root=/dev/mmcblk0" -serial stdio -net nic,model=lan9118 -net user

Thanks the creator of buildroot in your mind because this commandline is difficult to find on first try. Tweak it to make it work for you. You can add the -nographic option if you don't want the framebuffer to show.

That's it you've created a complete and minimal embedded system. Now you will want to customize it, change the kernel options to use the device tree for example or add the openssh server package to your rootfs. Explore your options using the make menuconfig  command.

[Trivia] Visualize your git history with gource

Find the source here:
https://github.com/acaudwell/Gource

The compilation is pretty straight forward:

./autogen
./configure
# fix the dependencies which are missing, probably sdl2.
make -j `nproc`
make install

That's it.

Go in your git repository (or svn, bazar?, whatever)

gource

You can now see your project grow up, get his first girlfriend, move out of your computer, buy a new repository at github city, etc.

A snapshot of the buildroot project

[qemu] Create a complete system image without booting in the emulator

Create an empty disk image:

qemu-img create -f qcow2 debian_bootstrap.qcow2 10G

Use the nbd kernel module (Network Block Device) to create and acceed partitions on the qcow2 disk image.

modinfo nbd

filename:       /lib/modules/3.18.4-1-ARCH/kernel/drivers/block/nbd.ko.gz
license:        GPL
description:    Network Block Device
depends:      
intree:         Y
vermagic:       3.18.4-1-ARCH SMP preempt mod_unload modversions
parm:           nbds_max:number of network block devices to initialize (default: 16) (int)
parm:           max_part:number of partitions per device (default: 0) (int)
parm:           debugflags:flags for controlling debug output (int)

sudo modprobe nbd max_part=16 

Associate the qcow2 disk image to the /dev/nbd0 disk node.

sudo qemu-nbd -c /dev/nbd0 debian_bootstrap.qcow2 

Create your partitions:

sudo parted /dev/nbd0

Or to get a swap + rootfs partition scheme, you can use:

sfdisk /dev/nbd0 -D -uM << EOF
,512,82
;
EOF

If you used parted with gpt partitions, you need to set up the first partition as a 2MB partition with the bios_grub flag set If no new disk file nbd0p1 shows up, you can usethe following command to force then to appear.

 partx -a /dev/nbd0

Format your partitions with the filesystems of your choice. (You can also do this in parted if you wish)

mkswap /dev/nbd0p1
mkfs.ext4 /dev/nbd0p2

Mount your root file system, you are almost ready to bootstrap a new debian in your qcow2 disk image.

mount /dev/nbd0p2 /mnt/

That's it, debootstrap this little rootfs ! (Choose the closest ftp to you to speed up the download)

debootstrap --include=less,locales-all,vim,sudo,openssh-server stable /mnt http://ftp.us.debian.org/debian

chroot in your new debian rootfs:

mount --bind /dev/ /mnt/dev
LANG=C chroot /mnt/ /bin/bash
mount -t proc none /proc
mount -t sysfs none /sys

You might need to set the path correctly if your host distribution doesn't use the same binaries path

PATH=$PATH:/bin:/sbin:/usr/sbin:/usr/bin

Install the latest kernel and grub-pc (beware, you want to install grub2 and not grub legacy)

apt-get install linux-image-amd64 grub-pc

When installing grub, choose the /dev/nbp0p2, you wouldn't want to install it on your own disk.

# To be sure grub is installed ^^
grub-install /dev/nbd0
update-grub

Set a passwd for root

passwd root

exit the chroot

umount /proc/ /sys/ /dev/
exit

# Some say you can fix the grub installation with those commands. Didn't work for me.
grub-install /dev/nbd0 --root-directory=/mnt --modules="biosdisk part_msdos" 
sed -e 's/nbd0p2/sda2/g' boot/grub/grub.cfg > boot/grub/grub.cfg
# I think if you want to have a working grub, just write the configuration yourself, 
# I fix this by hand at the first boot.

If you have other partition than the rootfs, you will need to edit your fstab.conf accordingly.
You are finished, clean your mess.

umount /mnt
qemu-nbd -d /dev/nbd0 

Start your VM You might get the grub console instead of a beautiful grub menu because the grub configuration is still broken! To launch linux you can do:

ls
#identify the rootfs partition
ls (hd0,gpt2)
ls (hd0,gpt2)/
#You should see your filesystem here, names can change
set root = (hd0,gpt2)
linux /boot/vmlinux-... root=/dev/sda2 # very important, specify your rootfs partition here
initrd boot/initrd-...
boot

Launch your newly created VM ! update-grub inside to fix the grub, this will eventually fix your grub.

[qemu] Create a complete subnet of virtual machines

Let's create a virtual network with qemu/kvm and vde :)

First, create a virtual switch where you will plug your virtual machines.

vde_switch -s /tmp/switch1.ctl -m 666 --daemon --mgmt /tmp/switch1.mgmt 

The -m option is here to set the flags so as to let everyone use the emulated switch. This is useful if you declare a switch with a link to a tap device.

Without the --daemon option, the vde_switch doesn't go in background and provides us a shell to configure its internals. This shell doesn't support history and readline commodities, so a better option is to use the --daemon mode coupled with the --mgmt option, this provide a socket where the virtual switch will listen for configuration input.

 
vdeterm /tmp/switch1.mgmt 

This command will give you a fully fledged shell. (You know you deserve it)

Finally the -s option is the most important as it is the interface you have to use to connect something to this virtual switch.

Congratulation, there is now an emulated empty switch running on your computer !

You can connect a vm to this switch with the following command:

qemu-system-x86_64 -drive file=your_image.qcow2,if=virtio --enable-kvm -net nic,model=virtio,macaddr=52:54:00:12:34:57 -net vde,sock=/tmp/switch1.ctl -cpu host -smp 4 -m 1G -vga qxl

The important thing is the -net vde option here.
Your vm should have registered to the vde_switch.

You can find out by using the command port/print in the vdeterm. You should get something like that:

 



The VMs you plug like this will be able to communicate with each other.

To enable the VMs to have an internet access, you can set up a NAT. This can be
done using slirp.

slirpvde -s /tmp/switch1.ctl --dhcp

That's it, it handles DNS, dhcp, name it, it does it :)
However, with this setup, you won't be able to access the VMs from the host, if you try and add a tap interface to the switch, it will get a dhcp and dns setup, it will get a default route rule. This will conflict with the nat and the ip forwarding won't work because the path are not correct.

You can prevent that by not using the dhcp server at all whith the tap interface. You can also use the tap interface as gateway for the VMs and enable the ip forwarding and masquerade nat rules instead of using the slirp tool (I will describe this in another post)

Sources: 

• http://wiki.v2.cs.unibo.it/wiki/index.php/VDE_Basic_Networking
• https://wiki.archlinux.org/index.php/QEMU#Networking_with_VDE2
• http://wiki.qemu.org/Documentation/Networkin
• https://xkcd.com/350/

[Python] colored ascii art with pyfiglet

Your scripts are boring? Keep them interesting with some big ascii art characters!

The following code is an example of what can be created with just a few line of code.

The colorama module enable us to have the same behaviour as the posix auto coloring mode. If we are targeting a tty, then the color will show up, but if the actual stdout isn't directed to a tty, then the color mode is switch off.

import sys
from colorama import init
init(strip=not sys.stdout.isatty())
from termcolor import cprint
from pyfiglet import figlet_format

text="Awesome Character Ascii Art"
cprint(figlet_format(text, font="standard"), "blue") 

The standard font is the default, but there are many fonts you can try, like the starwars font for example. Here is the result of the former piece of code:



The width of your terminal is not autodetected by pyfiglet and it default to 80 columns.

If you want the newline support, you will need my version of pyfiglet (You can find it here) but it should be merged in the mainline pyfiglet in a few days/weeks. (EDIT: It's done, you can get it with pip install pyfiglet now)

If you want to have a width corresponding to the terminal your are using, you can use https://gist.github.com/jtriley/1108174.

You can then feed figlet_format with the newly calculated width:

cprint(figlet_format(text, font="standard", width=terminal_width), "blue")

Pyfiglet is a port of the figlet utility in python (versions 2.6 to 3.4 are supported). In fact it should behave the same way.

Have fun with your scripts ;)

[Trivia] Mount a Windows 7 share repository with mount

Trying to access a shared repository with the commandline without specifying the version 2.0 will result in a I/O failure.

sudo mount -t cifs //ip_adress/partage-path mount-point -o user=youruser,vers=2.0

[Trivia] Debugging Qemu

Intro

First time I tried debugging Qemu, I had some problems with signals, serials, etc.

Debug

configure

Configure qemu to use debug
./configure --enable-debug --enable-fdt --target-list=arm-softmmu

 

signals

add the following lines to your .gdbinit:
handle SIGUSR1 SIGUSR2 noprint nostop 

Qemu uses those signals for timeout and internal stuff, we don't need to track them

 

serials

Then I had problems with serial lines in gdb, Reading the serial worked well but I couldn't write to it...

My workaround is to call:

socat -d -d pty pty 

You get two pty.
Call screen on the second one to be able to write to the serial port. 

References

[0] http://lnotestoself.blogspot.fr/2014/01/arm-debugging-with-qemu-and-gdb.html