Category Archives: Linux

Installing Gentoo on a Raspberry PI

Install ARM Toolchain

$ emerge -av crossdev
$ crossdev -S -v -t armv6j-hardfloat-linux-gnueabi

Get Raspberry Kernel Sources

$ git clone --depth=1 https://github.com/raspberrypi/linux.git
$ wget http://xecdesign.com/downloads/linux-qemu/linux-arm.patch
$ patch -p1 -d linux/ < linux-arm.patch

Configure the Kernel for QEMU

$ cd linux
$ make ARCH=arm menuconfig
$ make ARCH=arm
$ make ARCH=arm INSTALL_MOD_PATH=../modules modules_install
$ cp arch/arm/boot/zImage ../raspi-kernel

This is my .config
and the corresponding zImage and Modules: kernel_raspi_qemu.tar

Prepare “SD Card”

Create a sparse Image (this will be the SD Card)
$ dd if=/dev/null of=gentoo-raspi.img bs=1G seek=8 # 8GiB Image Partition Image
$ cfdisk gentoo-raspi.img
#first partition: boot, vfat ~80MiB (Partition Type: 0x0C)
#second: Gentoo Root (Partition Type: 0x83)

Mount Image as NBD

$ modprobe nbd max_part=4
$ qemu-nbd --connect=/dev/nbd0 -n /path/to/gentoo-raspi.img

$ emerge -av dosfstools
$ mkfs.vfat -n boot /dev/nbd0p1
$ mkfs.ext4 -j -L Gentoo -E lazy_itable_init=0,lazy_journal_init=0 /dev/nbd0p2
$ mount /dev/nbd0p2 /mnt/raspi
$ cd /mnt/raspi
$ curl http://distfiles.gentoo.org/releases/arm/autobuilds/current-stage3-armv6j_hardfp/stage3-armv6j_hardfp-20130207.tar.bz2 | tar -xvjp

Prepare Base System


$ dd if=/dev/zero of=/mnt/raspi/var/swap bs=1M count=512
$ chmod 0600 /mnt/raspi/var/swap
$ mkswap /mnt/raspi/var/swap

$ cd /mnt/raspi/etc
$ vi fstab


Insert the following lines:
LABEL=Gentoo / ext4 noatime 0 1
LABEL=BOOT /boot vfat noatime,defaults 0 0
/var/swap swap swap defaults 0 0

Set Root Password
$ openssl passwd -1 raspi
$ vi shadow

first line: root:$1$pDGeoXQj$0zq3sa5b9o9rJ8CpcIpPR/:10770:0:::::

Set Hostname & Keymap & timezone
$ vi conf.d/hostname
$ vi conf.d/keymaps
$ ln -sf ../usr/share/zoneinfo/Europe/Berlin localtime

Configure locale.gen
$ vi /etc/locale.gen

Copy Kernel Modules
$ cp -av /path/to/modules/lib /mnt/raspi

Install Portage
$ curl http://distfiles.gentoo.org/snapshots/portage-latest.tar.bz2 | tar -xvjp -C /mnt/raspi/usr

Set correct make.profile
$ cd /mnt/raspi/etc/portage rm make.profile
$ ln -sf ../../usr/portage/profiles/default/linux/arm/13.0 make.profile

Disconnect NBD
$ umount /mnt/raspi
$ qemu-nbd --disconnect /dev/nbd0

Boot your Raspberry on QEMU

$ qemu-system-arm -kernel zImage -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1 console=ttyAMA0" -hda gentoo-raspi.img
Login: root:raspi
Set Date (Raspi has no hardware Clock 🙁 )
$ date --set="20130808 14:00"
$ rc-update del hwclock default

Configure Network
$ ifconfig eth0 10.0.2.15 #USE THIS IP!
$ route add default gw 10.0.2.2
$ echo nameserver 8.8.8.8 > /etc/resolv.conf #just use google's dns

Get dhcp Client and configure network
$ emerge dhcpcd
$ dhcpcd

Configure eth0 persistently
$ cd /etc/init.d
$ ln -s net.lo net.eth0
$ rc-update add net.eth0 boot

$ cd /etc/conf.d/
$ nano net

add the line: config_eth0="dhcp"

You WANT distcc.
emerge -av distcc #Run on both, host and guest

Edit make.conf
$ echo FEATURES="distcc" >> /etc/portage/make.conf
$ echo MAKEOPTS="-j6" >> /etc/portage/make.conf

Set DistCC Hosts
$ cd /etc/distcc
$ nano hosts

Now compile everything you want

Emerge boot stuff and kernel
$ mkdir /etc/portage/package.keywords
$ cd /etc/portage/package.keywords
$ echo sys-boot/raspberrypi-firmware ** ~arm >> raspi
$ echo sys-kernel/raspberrypi-sources ** ~arm >> raspi
$ echo media-libs/raspberrypi-userland ** ~arm >> raspi

$ mount /boot
$ emerge -av raspberrypi-firmware raspberrypi-sources raspberrypi-userland

Compile the kernel
$ cd /usr/src/linux
$ make menuconfig

$ make CC="distcc armv6j-hardfloat-linux-gnueabi-gcc" modules -j5
$ make modules_install
$ make CC="distcc armv6j-hardfloat-linux-gnueabi-gcc" -j5
$ cp arch/arm/boot/zImage /boot/kernel.img

This is my precompiled Kernel and Modules / Firmware:
.config
kernel-3.6.11-raspi.tar.gz

You also want to use SSH:
$ rc-update add sshd default

And as we do have no RTC:
$ rc-update add ntp-client default

And a syslogger:
$ emerge -av syslog-ng
$ rc-update add syslog-ng boot

Here's my image:
It needs at least 2.8GiB of space (2.3GiB without /var/swap)

As .tar.gz archive:
VFAT Boot Partition: Raspberry-Boot-20130816.tar.gz
Gentoo Installation: Raspberry-Stage4-20130816.tar.gz

As full 6GiB raw image which is bootable out of the box:
Raspberry-Gentoo-6GiB-Image-20130816.img.gz

You can use dd to flash it on a SD Card
$ gunzip Raspberry-Gentoo-6GiB-Image-20130816.img.gz
$ dd if=Raspberry-Gentoo-6GiB-Image-20130816.img of=/dev/yourraspisdcard

Have fun with Gentoo on your Raspi 🙂

Update 2014-06-03:

Here's another nice tutorial without any cross-compilation

Linux’s strange memory allocation strategy

As I learned today, Linux has a real strange memory allocation strategy. Have a look at this piece of code:

#include <stdio.h>
#include <stdlib.h>

int main(void) {
  while(1) {
    char *foo = (char*)malloc(1024);
    if(foo == NULL) {
      printf("Couldn't alloc\n");
      fflush(stdout);
      return 0;
    }
  }
  return 0;
}

According to the malloc reference it should return NULL if it is not able to allocate memory:

“If the function failed to allocate the requested block of memory, a null pointer is returned.”

So I thought my process would write Couldn’t alloc to stdout and exit in a proper way. But it didn’t act that way. When my system ran out of memory the process got killed by the kernel with signal SIGKILL. So why does it act like this? Wikipedia writes about Out of memory:

“A process which exceeds its per-process limit and then attempts to allocate further memory – with malloc(), for example – will return failure. A well-behaved application should handle this situation gracefully; however, many do not. An attempt to allocate memory without checking the result is known as an “unchecked malloc”.”

Well… Yes… Of course, you always should check malloc() if it returned NULL, but under normal conditions it never will return NULL because of Linux’s memory overcommitting. By default Linux has an optimistic memory allocation strategy. When allocating memory, malloc() will return a pointer. The space on the heap which is needed for that pointer gets allocated at the first read/write operation to that address. In my opinion, this is a really strange behaviour, because when memory gets allocated, it will actually be used. Here’s a short example to see the effects of that strange behaviour:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define MALLOC_SIZE (1024*1024*10)

int main(void) {
  int i = 0;
  while(1) {
    char *foo = (char*)malloc(MALLOC_SIZE);
    //memset(foo, 0xaa, MALLOC_SIZE);
    printf("Allocation No. %d\n", i++);
    if(foo == NULL) {
      printf("Couldn't alloc\n");
      fflush(stdout);
      return 0;
    }
  }
  return 0;
}

With the memset line commented out it results to:

ralf@Pegasus:C$ ./test|tail
Allocation No. 1841101
Allocation No. 1841102
Allocation No. 1841103
Allocation No. 1841104
Allocation No. 1841105
Allocation No. 1841106
Allocation No. 1841107
Allocation No. 1841108
Allocation No. 1841109
Allocation Nzsh: killed     ./test |
zsh: done       tail

And with the memset commented in it results to:

ralf@Pegasus:C$ ./test|tail
Allocation No. 1275
Allocation No. 1276
Allocation No. 1277
Allocation No. 1278
Allocation No. 1279
Allocation No. 1280
Allocation No. 1281
Allocation No. 1282
Allocation No. 1283
Allocazsh: killed     ./test |
zsh: done       tail

This is really funny. You can allocate tons of space you don’t have. That’s creepy. But this behaviour can be switched by

echo 2 > /proc/sys/vm/overcommit_memory

“Since 2.5.30 the values are: 0 (default): as before: guess about how much overcommitment is reasonable, 1: never refuse any malloc(), 2: be precise about the overcommit – never commit a virtual address space larger than swap space plus a fraction overcommit_ratio of the physical memory. Here /proc/sys/vm/overcommit_ratio (by default 50) is another user-settable parameter. It is possible to set overcommit_ratio to values larger than 100. (See also Documentation/vm/overcommit-accounting.)” Source

Then, both results (with and without memset) lead to:

ralf@Pegasus:C$ ./test|tail
Allocation No. 433
Allocation No. 434
Allocation No. 435
Allocation No. 436
Allocation No. 437
Allocation No. 438
Allocation No. 439
Allocation No. 440
Allocation No. 441
Couldn't alloc

And this is exactly what I originally expected to be the normal way. Without this bypass, malloc will never return NULL and a process will not be able to shut down properly if the system runs out of memory.

Here‘s a nice anecdote on that topic.