Installing KVM and Cloudstack 4.6 agent on Raspberry Pi 2

 

This document covers the installation of KVM and Cloudstack 4.6 agent on Raspberry Pi 2. This document assumes that you have basic familiarity with Linux operating systems in general and also some familiarity with Ubuntu.
The system preparation steps are very similar to “Install Cloudstack Management server”. They are repeated here just for completion. If you have more than “basic” familiarity with Linux systems, you probably would know which steps are to be skipped or modified.
Please note that this document is NOT finished yet as I’m still trying to add this host to Cloudstack management server. This is a work in progress.
When all trouble shootings are done and the host is successfully is added to the management server, I will update this page respectively.

Preparing the hypervisor image:

In this section (A) we prepare an SD card image to be used for the provisioning of the new hypervisors, and in the next section (B) we use this image to provision the first hypervisor.

1. Download Ubutnu 14.04 LTS (Trusty Tahr) image and write to the micro SD card. You need a micro SD card with minimum 8 GB space.
Download the image here:
https://wiki.ubuntu.com/ARM/RaspberryPi

Alternatively you can download the image from claspi.org (mirror):
http://www.claspi.org/downloads/images/2015-04-06-ubuntu-trusty.zip

Extract the zip file and follow the instructions below to write the image on the micro SD card:
Linux
Mac OS
Windows

You need a HDMI cable and keyboard connected to Raspberry Pi at this stage. Login to Raspberry Pi by:
Username: ubuntu
Password: ubuntu

 

2. Install OpenSSH

$sudo apt-get install openssh-server

This command takes about 2 minutes to complete.
From now on you can connect to the ip address of this box using an SSH terminal application (e.g. putty)

 

3. Install OpenNtpd:

$sudo apt-get install openntpd

 

4. Resize the flash partition to the maximum size of the micro SD card:

Determine the storage devices: (we see in the result below that /dev/mmcblk0 is the storage device)

$ll /dev/mm*
brw-rw---- 1 root disk 179, 0 Jan  1 00:00 /dev/mmcblk0
brw-rw---- 1 root disk 179, 1 Jan  1 00:00 /dev/mmcblk0p1
brw-rw---- 1 root disk 179, 2 Jan  1 00:00 /dev/mmcblk0p2

Print the partition table:

$ sudo parted /dev/mmcblk0
GNU Parted 2.3
Using /dev/mmcblk0
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) unit chs
(parted) print
Model: SD 00000 (sd/mmc)
Disk /dev/mmcblk0: 1947,203,47
Sector size (logical/physical): 512B/512B
BIOS cylinder,head,sector geometry: 1947,255,63.  Each cylinder is 8225kB.
Partition Table: msdos

Number  Start    End         Type     File system  Flags
 1      0,32,32  8,73,0      primary  fat16        boot, lba
 2      8,73,1   228,114,13  primary  ext4

(parted)

The root partition starts at 8, 73,1 and should end at 1947,203,47.
Important: this is just an example, depending on the size of your micro SD cards, these geometry may vary. Do not copy paste command below, you need to modify the geometry in your commands first. Continue by removing partition 2:

(parted) rm 2
Warning: Partition /dev/mmcblk0p2 is being used. Are you sure you want to continue?
Yes/No? Y
Error: Partition(s) 2 on /dev/mmcblk0 have been written, but we have been unable to
inform the kernel of the change, probably because it/they are in use. As a result,
the old partition(s) will remain in use. You should reboot now before making further
changes.
Ignore/Cancel? I
(parted) print
Model: SD 00000 (sd/mmc)
Disk /dev/mmcblk0: 1947,203,47
Sector size (logical/physical): 512B/512B
BIOS cylinder,head,sector geometry: 1947,255,63. Each cylinder is 8225kB.
Partition Table: msdos

Number Start End Type File system Flags
 1 0,32,32 8,73,0 primary fat16 boot, lba

Now we need to create partition 2 again:

(parted) mkpart primary 8,73,1 1947,203,47 
(do not copy paste this command. 
You need to change the start and end cylinder of the partition based on the info above) 

(parted) print
Model: SD 00000 (sd/mmc)
Disk /dev/mmcblk0: 1947,203,47
Sector size (logical/physical): 512B/512B
BIOS cylinder,head,sector geometry: 1947,255,63.  Each cylinder is 8225kB.
Partition Table: msdos

Number  Start    End          Type     File system  Flags
 1      0,32,32  8,73,0       primary  fat16        boot, lba
 2      8,73,1   1947,203,47  primary  ext4

(parted) quit

Reboot:

$sudo reboot

Extend the root partition to the max size:

$sudo resize2fs /dev/mmcblk0p2

Check the patition size:

$df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p2   15G  1.5G   13G  11% /
devtmpfs        458M  4.0K  458M   1% /dev
none            4.0K     0  4.0K   0% /sys/fs/cgroup
none             93M  256K   93M   1% /run
none            5.0M  8.0K  5.0M   1% /run/lock
none            462M     0  462M   0% /run/shm
none            100M     0  100M   0% /run/user
/dev/mmcblk0p1   64M   17M   48M  27% /boot/firmware

Now we see that /dev/mmcblk0p2 (root) has the size of 15G (in this example we have a 16GB micro SD card)

 

5. Add a swap file of 1GB:
Follow these commands:

$sudo fallocate -l 1G /swapfile
$sudo chmod 600 /swapfile
$sudo mkswap /swapfile
Setting up swapspace version 1, size = 1048572 KiB 
no label, UUID=...
$sudo swapon /swapfile

Edit fstab:

$sudo nano /etc/fstab

Add the following line to fstab to make the swap file permanent:

/swapfile   none    swap    sw    0   0

Save and close.

 

6. Configure your Raspi to use the claspi.org APT repository

Before being able to install cloudstack-management from claspi.org repository you need to add the repo to your source list.

$sudo nano /etc/apt/sources.list.d/cloudstack.list

This file (cloudstack.list) is new and empty. Add the following line to the beginning of the file:

deb http://www.claspi.org/cloudstack/repo/binary ./

Save and close.

Alternatively you can download the edited cloudstack.list file from here, by the following command:

$sudo wget -P /etc/apt/sources.list.d http://www.claspi.org/downloads/files/cloudstack.list
$sudo apt-get update

 

 

7. Update Linux kernel with HYP support:

The Raspberry Pi kernel used here for enabling HYP support is based on the following article:
http://blog.flexvdi.com/2015/03/17/enabling-kvm-virtualization-on-the-raspberry-pi-2/
The same kernel (thanks to Sergio L.Pascual) is provided in this website (as a mirror) with no modification yet. You can either choose to download the kernel image (kernel7.img) from the link above, or from claspi.org repo by the provided commands.

First check the current kernel version:

$uname -r
3.18.0-23-rpi2

Use the dmesg command to see if the kernel is supporting HYP:

$sudo dmesg | grep CPU

In the boot log, look for a line that shows something like this:

[0.061264] CPU: All CPU(s) started in SVC mode.

This means the HYP support is currently NOT enabled in the kernel.

Update linux kernel and headers to 3.18.8-v7+ by using the rpi-update tool:

$sudo apt-get install curl
$sudo apt-get install binutils
$sudo curl -L --output /usr/bin/rpi-update https://raw.githubusercontent.com/Hexxeh/rpi-update/master/rpi-update && sudo chmod +x /usr/bin/rpi-update
$sudo rpi-update 8f6196d9e3c96915d8

Backup the current kernel and replace it with the kernel with KVM (virtualization) support:

$sudo mv /boot/firmware/kernel7.img /boot/firmware/kernel7.img.bak

Download the kernel with HYP support and save to /boot/firmware directory:

$sudo wget http://www.claspi.org/downloads/kernel/hyp/kernel7.img -O /boot/firmware/kernel7.img
...
Length: 4202530 (4.0M) [application/octet-stream]
Saving to: '/boot/firmware/kernel7.img'

Add the kernel_old=1 option to your boot config.txt:

$sudo echo "kernel_old=1" >> /boot/config.txt

Isolate a CPU core:

$sudo nano /boot/cmdline.txt

And add isolcpus=3 to the end of the line:

dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootwait isolcpus=3

Save, close and reboot.
After reboot test the kernel for the correct version and supporting virtualization:

$uname -r
3.18.8-v7+
$sudo dmesg | grep CPU
...
[ 0.157974] Brought up 4 CPUs
[ 0.158148] CPU: All CPU(s) started in HYP mode.
[ 0.158178] CPU: Virtualization extensions available.

 

8. Patch, compile and install the latest version of KVM-Qemu:

$sudo apt-get build-dep qemu
$sudo apt-get install libpixman-1-dev

These commands takes about 5 minutes to complete.
Create a directory for the sources, download QEMU 2.2 and uncompress it:

$mkdir srcs
$cd srcs
$wget http://wiki.qemu-project.org/download/qemu-2.2.0.tar.bz2
$tar xf qemu-2.2.0.tar.bz2

Apply the CPU affinity patch and enable KVM:

~/srcs$ wget http://www.claspi.org/downloads/files/qemu-cpu-affinity.patch
$cd qemu-2.2.0
$patch -p1 < ../qemu-cpu-affinity.patch
$sudo ./configure --enable-kvm --target-list=arm-softmmu

Make and install kvm-qemu:

~/srcs/qemu-2.2.0$ make

Compiling qemu on Raspberry Pi takes ages to complete, so you need to be patient. That’s why we are making an image to (hopefully) have to do this only once.
After compiling is complete, install kvm-qemu:

$sudo make install

 

9. Install Java Open JDK 7:
Cloudstack agent needs OpenJDK 7. Install it by:

$sudo apt-get install openjdk-7-jdk

The installation takes about 5 minutes to complete.

 

10. After the above preparation steps are complete, using one of the tools mentioned in step 1, create an image from the micro SD card.
We will use this image as the base image for all the hypervisors.

 

 

Provisioning hypervisors:

Now we can restore the SD card image to a new micro SD card (or use the existing one) and configure the first host:

1. Configure network with static IP address:
In this document and examples, 192.168.2.121 IP address is used for the first Cloudstack agent (hypervisor/host) in the cluster, assuming 192.168.2.254 is the gateway. Please change all the IP addresses in the commands if you run the setup with an alternative IP address/gateway. After installation of Cloudstack agent, changing the IP address is not possible.
We configure Cloudstack agent with standard Linux bridge. Other networking technologies like OpenVSwitch is not covered here yet. For future development I might be able to look at OpenVSwtich and update this page respectively.
Using “nano” as the text editor is not mandatory and feel free to use any EOTM (editor of the month) for editing your files.

$sudo nano /etc/network/interfaces
[sudo] password for ubuntu: (type "ubuntu" as password)

Edit eth0 interface as follows: (this configures eth0 and cloud vlans)

# interfaces(5) file used by ifup(8) and ifdown(8)
# Include files from /etc/network/interfaces.d:
source-directory /etc/network/interfaces.d

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auro eth0.100
iface eth0.100 inet static
address 192.168.2.121
netmask 255.255.255.0
gateway 192.168.2.254
network 192.168.2.0
broadcast 192.168.2.255
dns-nameservers 8.8.8.8 8.8.4.4

# Public network
auto cloudbr0
iface cloudbr0 inet manual
 bridge_ports eth0.200
 bridge_fd 5
 bridge_stp off
 bridge_maxwait 1

# Private network
auto cloudbr1
iface cloudbr1 inet manual
 bridge_ports eth0.300
 bridge_fd 5
 bridge_stp off
 bridge_maxwait 1

Save and close. (in nano: CTRL-x -> y -> Enter)
You can reboot Raspberry now and continue by SSH terminal instead of a direct console (ssh connect to 192.168.2.121)

$sudo reboot

Login to Raspi again and ping a website to check the internet connection and dns resolution:

$ping www.google.com

 

2. Configure hostname to fqdn:

$sudo nano /etc/hosts

Change 127.0.0.1 to a fully qualified domain name, it means change “ubuntu” to a name like “ubuntu.mydomain.local” (in this example we will lxc1.claspi.org as our fqdn name for the first host in the cluster)

127.0.0.1    lxc1.claspi.org

Edit /etc/hostname file with fqdn name

$sudo nano /etc/hostname

Change hostname from “ubuntu” to “lxc1.claspi.org”

lxc1.claspi.org

Save and close, then reboot Raspberry Pi for the change to take effect.

$sudo reboot

After reboot is complete, use hostname command to check the hostname:

$hostname --fqdn
>lxc1.claspi.org

 

3. Install cloudstack-agent:
The claspi.org repository also contains cloudstack-agent for Raspberry Pi 2. You can simply install the cloudstack-agent by using apt-get:

$sudo apt-get install cloudstack-agent

This command takes about 5 minutes to complete.
At this stage if you try to start the cloudstack-agent, it fails:

$sudo service cloudstack-agent start
...
 * Starting CloudStack Agent cloudstack-agent                                                                                                                                Cannot locate Java Home
cat: /var/run/cloudstack-agent.pid: No such file or directory
 * jsvc failed to start

 

4. Configure cloudstack-agent:
Configure agent to use JAVA for armf:

$sudo nano /etc/init.d/cloudstack-agent

Edit the following lines (what you need to edit is highlighted in bold):

...
# The first existing directory is used for JAVA_HOME 
(if JAVA_HOME is not defined in $DEFAULT)
JDK_DIRS="/usr/lib/jvm/java-1.7.0-openjdk-armhf /usr/lib/jvm/java-7-openjdk-amd64 
/usr/lib/jvm/java-7-openjdk-i386 /usr/lib/jvm/java-7-oracle 
/usr/lib/jvm/java-6-openjdk /u$
...
if start_daemon -p $PIDFILE $DAEMON -Xms256m -Xmx768m -cp "$CLASSPATH" 
-Djna.nosys=true -pidfile "$PIDFILE" -errfile SYSLOG $CLASS

Configure agent to use vmx and host-passthrough:

$sudo nano /etc/cloudstack/agent/agent.properties

Uncomment and edit the following parameters:

...
guest.cpu.mode=host-passthrough
guest.cpu.features=vmx
...

Configure libvirt to listen to TCP connections:

$sudo nano /etc/libvirt/libvirtd.conf

Uncomment and edit the following parameters:

...
listen_tls = 0
listen_tcp = 1
tcp_port = "16509"
auth_tcp = "none"
mdns_adv = 0

Also configure libvirt deamon to listen to TCP:

$sudo nano /etc/default/libvirt-bin

Uncomment and edit the following parameter (Add “-l” to the line)

...
libvirtd_opts="-d -l"
...

Restart libvirt:

$sudo service libvirt-bin restart

Start the cloudstack-agent:

$sudo service cloudstack-agent start

If all configuration is ok, the cloudstack-agent should start with no error and libvirt should listen to 192.168.2.121 port 16509.
The configuration of the cloudstack-agent is completed. The next step is to add this hypervisor the the cloudstack-management.


Leave a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>