Installing KVM and Cloudstack 4.6 agent on Raspberry Pi 2
This document covers the installation of KVM and Cloudstack 4.6 agent on Raspberry Pi 2. This document assumes that you have basic familiarity with Linux operating systems in general and also some familiarity with Ubuntu.
The system preparation steps are very similar to “Install Cloudstack Management server”. They are repeated here just for completion. If you have more than “basic” familiarity with Linux systems, you probably would know which steps are to be skipped or modified.
Please note that this document is NOT finished yet as I’m still trying to add this host to Cloudstack management server. This is a work in progress.
When all trouble shootings are done and the host is successfully is added to the management server, I will update this page respectively.
Preparing the hypervisor image:
In this section (A) we prepare an SD card image to be used for the provisioning of the new hypervisors, and in the next section (B) we use this image to provision the first hypervisor.
1. Download Ubutnu 14.04 LTS (Trusty Tahr) image and write to the micro SD card. You need a micro SD card with minimum 8 GB space.
Download the image here:
Alternatively you can download the image from claspi.org (mirror):
You need a HDMI cable and keyboard connected to Raspberry Pi at this stage. Login to Raspberry Pi by:
2. Install OpenSSH
$sudo apt-get install openssh-server
This command takes about 2 minutes to complete.
From now on you can connect to the ip address of this box using an SSH terminal application (e.g. putty)
3. Install OpenNtpd:
$sudo apt-get install openntpd
4. Resize the flash partition to the maximum size of the micro SD card:
Determine the storage devices: (we see in the result below that /dev/mmcblk0 is the storage device)
$ll /dev/mm* brw-rw---- 1 root disk 179, 0 Jan 1 00:00 /dev/mmcblk0 brw-rw---- 1 root disk 179, 1 Jan 1 00:00 /dev/mmcblk0p1 brw-rw---- 1 root disk 179, 2 Jan 1 00:00 /dev/mmcblk0p2
Print the partition table:
$ sudo parted /dev/mmcblk0 GNU Parted 2.3 Using /dev/mmcblk0 Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) unit chs (parted) print Model: SD 00000 (sd/mmc) Disk /dev/mmcblk0: 1947,203,47 Sector size (logical/physical): 512B/512B BIOS cylinder,head,sector geometry: 1947,255,63. Each cylinder is 8225kB. Partition Table: msdos Number Start End Type File system Flags 1 0,32,32 8,73,0 primary fat16 boot, lba 2 8,73,1 228,114,13 primary ext4 (parted)
The root partition starts at 8, 73,1 and should end at 1947,203,47.
Important: this is just an example, depending on the size of your micro SD cards, these geometry may vary. Do not copy paste command below, you need to modify the geometry in your commands first. Continue by removing partition 2:
(parted) rm 2 Warning: Partition /dev/mmcblk0p2 is being used. Are you sure you want to continue? Yes/No? Y Error: Partition(s) 2 on /dev/mmcblk0 have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use. As a result, the old partition(s) will remain in use. You should reboot now before making further changes. Ignore/Cancel? I (parted) print Model: SD 00000 (sd/mmc) Disk /dev/mmcblk0: 1947,203,47 Sector size (logical/physical): 512B/512B BIOS cylinder,head,sector geometry: 1947,255,63. Each cylinder is 8225kB. Partition Table: msdos Number Start End Type File system Flags 1 0,32,32 8,73,0 primary fat16 boot, lba
Now we need to create partition 2 again:
(parted) mkpart primary 8,73,1 1947,203,47 (do not copy paste this command. You need to change the start and end cylinder of the partition based on the info above) (parted) print Model: SD 00000 (sd/mmc) Disk /dev/mmcblk0: 1947,203,47 Sector size (logical/physical): 512B/512B BIOS cylinder,head,sector geometry: 1947,255,63. Each cylinder is 8225kB. Partition Table: msdos Number Start End Type File system Flags 1 0,32,32 8,73,0 primary fat16 boot, lba 2 8,73,1 1947,203,47 primary ext4 (parted) quit
Extend the root partition to the max size:
$sudo resize2fs /dev/mmcblk0p2
Check the patition size:
$df -h Filesystem Size Used Avail Use% Mounted on /dev/mmcblk0p2 15G 1.5G 13G 11% / devtmpfs 458M 4.0K 458M 1% /dev none 4.0K 0 4.0K 0% /sys/fs/cgroup none 93M 256K 93M 1% /run none 5.0M 8.0K 5.0M 1% /run/lock none 462M 0 462M 0% /run/shm none 100M 0 100M 0% /run/user /dev/mmcblk0p1 64M 17M 48M 27% /boot/firmware
Now we see that /dev/mmcblk0p2 (root) has the size of 15G (in this example we have a 16GB micro SD card)
5. Add a swap file of 1GB:
Follow these commands:
$sudo fallocate -l 1G /swapfile $sudo chmod 600 /swapfile $sudo mkswap /swapfile Setting up swapspace version 1, size = 1048572 KiB no label, UUID=... $sudo swapon /swapfile
$sudo nano /etc/fstab
Add the following line to fstab to make the swap file permanent:
/swapfile none swap sw 0 0
Save and close.
6. Configure your Raspi to use the claspi.org APT repository
Before being able to install cloudstack-management from claspi.org repository you need to add the repo to your source list.
$sudo nano /etc/apt/sources.list.d/cloudstack.list
This file (cloudstack.list) is new and empty. Add the following line to the beginning of the file:
deb http://www.claspi.org/cloudstack/repo/binary ./
Save and close.
Alternatively you can download the edited cloudstack.list file from here, by the following command:
$sudo wget -P /etc/apt/sources.list.d http://www.claspi.org/downloads/files/cloudstack.list $sudo apt-get update
7. Update Linux kernel with HYP support:
The Raspberry Pi kernel used here for enabling HYP support is based on the following article:
The same kernel (thanks to Sergio L.Pascual) is provided in this website (as a mirror) with no modification yet. You can either choose to download the kernel image (kernel7.img) from the link above, or from claspi.org repo by the provided commands.
First check the current kernel version:
$uname -r 3.18.0-23-rpi2
Use the dmesg command to see if the kernel is supporting HYP:
$sudo dmesg | grep CPU
In the boot log, look for a line that shows something like this:
[0.061264] CPU: All CPU(s) started in SVC mode.
This means the HYP support is currently NOT enabled in the kernel.
Update linux kernel and headers to 3.18.8-v7+ by using the rpi-update tool:
$sudo apt-get install curl $sudo apt-get install binutils $sudo curl -L --output /usr/bin/rpi-update https://raw.githubusercontent.com/Hexxeh/rpi-update/master/rpi-update && sudo chmod +x /usr/bin/rpi-update $sudo rpi-update 8f6196d9e3c96915d8
Backup the current kernel and replace it with the kernel with KVM (virtualization) support:
$sudo mv /boot/firmware/kernel7.img /boot/firmware/kernel7.img.bak
Download the kernel with HYP support and save to /boot/firmware directory:
$sudo wget http://www.claspi.org/downloads/kernel/hyp/kernel7.img -O /boot/firmware/kernel7.img ... Length: 4202530 (4.0M) [application/octet-stream] Saving to: '/boot/firmware/kernel7.img'
Add the kernel_old=1 option to your boot config.txt:
$sudo echo "kernel_old=1" >> /boot/config.txt
Isolate a CPU core:
$sudo nano /boot/cmdline.txt
And add isolcpus=3 to the end of the line:
dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootwait isolcpus=3
Save, close and reboot.
After reboot test the kernel for the correct version and supporting virtualization:
$uname -r 3.18.8-v7+ $sudo dmesg | grep CPU ... [ 0.157974] Brought up 4 CPUs [ 0.158148] CPU: All CPU(s) started in HYP mode. [ 0.158178] CPU: Virtualization extensions available.
8. Patch, compile and install the latest version of KVM-Qemu:
$sudo apt-get build-dep qemu $sudo apt-get install libpixman-1-dev
These commands takes about 5 minutes to complete.
Create a directory for the sources, download QEMU 2.2 and uncompress it:
$mkdir srcs $cd srcs $wget http://wiki.qemu-project.org/download/qemu-2.2.0.tar.bz2 $tar xf qemu-2.2.0.tar.bz2
Apply the CPU affinity patch and enable KVM:
~/srcs$ wget http://www.claspi.org/downloads/files/qemu-cpu-affinity.patch $cd qemu-2.2.0 $patch -p1 < ../qemu-cpu-affinity.patch $sudo ./configure --enable-kvm --target-list=arm-softmmu
Make and install kvm-qemu:
Compiling qemu on Raspberry Pi takes ages to complete, so you need to be patient. That’s why we are making an image to (hopefully) have to do this only once.
After compiling is complete, install kvm-qemu:
$sudo make install
9. Install Java Open JDK 7:
Cloudstack agent needs OpenJDK 7. Install it by:
$sudo apt-get install openjdk-7-jdk
The installation takes about 5 minutes to complete.
10. After the above preparation steps are complete, using one of the tools mentioned in step 1, create an image from the micro SD card.
We will use this image as the base image for all the hypervisors.
Now we can restore the SD card image to a new micro SD card (or use the existing one) and configure the first host:
1. Configure network with static IP address:
In this document and examples, 192.168.2.121 IP address is used for the first Cloudstack agent (hypervisor/host) in the cluster, assuming 192.168.2.254 is the gateway. Please change all the IP addresses in the commands if you run the setup with an alternative IP address/gateway. After installation of Cloudstack agent, changing the IP address is not possible.
We configure Cloudstack agent with standard Linux bridge. Other networking technologies like OpenVSwitch is not covered here yet. For future development I might be able to look at OpenVSwtich and update this page respectively.
Using “nano” as the text editor is not mandatory and feel free to use any EOTM (editor of the month) for editing your files.
$sudo nano /etc/network/interfaces [sudo] password for ubuntu: (type "ubuntu" as password)
Edit eth0 interface as follows: (this configures eth0 and cloud vlans)
# interfaces(5) file used by ifup(8) and ifdown(8) # Include files from /etc/network/interfaces.d: source-directory /etc/network/interfaces.d # The loopback network interface auto lo iface lo inet loopback # The primary network interface auro eth0.100 iface eth0.100 inet static address 192.168.2.121 netmask 255.255.255.0 gateway 192.168.2.254 network 192.168.2.0 broadcast 192.168.2.255 dns-nameservers 18.104.22.168 22.214.171.124 # Public network auto cloudbr0 iface cloudbr0 inet manual bridge_ports eth0.200 bridge_fd 5 bridge_stp off bridge_maxwait 1 # Private network auto cloudbr1 iface cloudbr1 inet manual bridge_ports eth0.300 bridge_fd 5 bridge_stp off bridge_maxwait 1
Save and close. (in nano: CTRL-x -> y -> Enter)
You can reboot Raspberry now and continue by SSH terminal instead of a direct console (ssh connect to 192.168.2.121)
Login to Raspi again and ping a website to check the internet connection and dns resolution:
2. Configure hostname to fqdn:
$sudo nano /etc/hosts
Change 127.0.0.1 to a fully qualified domain name, it means change “ubuntu” to a name like “ubuntu.mydomain.local” (in this example we will lxc1.claspi.org as our fqdn name for the first host in the cluster)
Edit /etc/hostname file with fqdn name
$sudo nano /etc/hostname
Change hostname from “ubuntu” to “lxc1.claspi.org”
Save and close, then reboot Raspberry Pi for the change to take effect.
After reboot is complete, use hostname command to check the hostname:
$hostname --fqdn >lxc1.claspi.org
3. Install cloudstack-agent:
The claspi.org repository also contains cloudstack-agent for Raspberry Pi 2. You can simply install the cloudstack-agent by using apt-get:
$sudo apt-get install cloudstack-agent
This command takes about 5 minutes to complete.
At this stage if you try to start the cloudstack-agent, it fails:
$sudo service cloudstack-agent start ... * Starting CloudStack Agent cloudstack-agent Cannot locate Java Home cat: /var/run/cloudstack-agent.pid: No such file or directory * jsvc failed to start
4. Configure cloudstack-agent:
Configure agent to use JAVA for armf:
$sudo nano /etc/init.d/cloudstack-agent
Edit the following lines (what you need to edit is highlighted in bold):
... # The first existing directory is used for JAVA_HOME (if JAVA_HOME is not defined in $DEFAULT) JDK_DIRS="/usr/lib/jvm/java-1.7.0-openjdk-armhf /usr/lib/jvm/java-7-openjdk-amd64 /usr/lib/jvm/java-7-openjdk-i386 /usr/lib/jvm/java-7-oracle /usr/lib/jvm/java-6-openjdk /u$ ... if start_daemon -p $PIDFILE $DAEMON -Xms256m -Xmx768m -cp "$CLASSPATH" -Djna.nosys=true -pidfile "$PIDFILE" -errfile SYSLOG $CLASS
Configure agent to use vmx and host-passthrough:
$sudo nano /etc/cloudstack/agent/agent.properties
Uncomment and edit the following parameters:
... guest.cpu.mode=host-passthrough guest.cpu.features=vmx ...
Configure libvirt to listen to TCP connections:
$sudo nano /etc/libvirt/libvirtd.conf
Uncomment and edit the following parameters:
... listen_tls = 0 listen_tcp = 1 tcp_port = "16509" auth_tcp = "none" mdns_adv = 0
Also configure libvirt deamon to listen to TCP:
$sudo nano /etc/default/libvirt-bin
Uncomment and edit the following parameter (Add “-l” to the line)
... libvirtd_opts="-d -l" ...
$sudo service libvirt-bin restart
Start the cloudstack-agent:
$sudo service cloudstack-agent start
If all configuration is ok, the cloudstack-agent should start with no error and libvirt should listen to 192.168.2.121 port 16509.
The configuration of the cloudstack-agent is completed. The next step is to add this hypervisor the the cloudstack-management.