System administration guidance (for the unmanaged cloud)

This article is dedicated to tenancy administrators who are responsible on provisioning virtual machines in the JASMIN unmanaged cloud service. It covers the following:

  • How to manage storage in VM
  • How to create a VM template 
  • How to manage core counts and RAM 

Storage management 

A virtual machine in a tenancy gets allocated a hard disk storage of 32 GB in size.  It is possible to add extra  storage to a virtual machine by following these two steps: Step 1 On the machines page for your tenancy, click the   button - this will launch a dialog where a new disk can be added:

Once the new disk has been successfully added, power the machine on. The new disk will be visible to the machine, but will not be usable. This can be verified using the  lsblkcommand:

$ lsblk
NAME                         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                            8:0    0    32G  0 disk
├─sda1                         8:1    0   243M  0 part /boot
├─sda2                         8:2    0     1K  0 part
└─sda5                         8:5    0  15.8G  0 part
  ├─ubuntu--vg-root (dm-0)   252:0    0  13.8G  0 lvm  /
  └─ubuntu--vg-swap_1 (dm-1) 252:1    0     2G  0 lvm  [SWAP]
sdb                            8:16   0    16G  0 disk
sr0                           11:0    1  1024M  0 rom<br>

Here, we can see that the operating system is recognising the new disk (  sdb), but there are no partitions or file systems associated with it. There are two options for making the disk usable:

  1. Add a new file system to the disk and mount it somewhere (e.g. /data)
  2. If the template has been built with the root (i.e. /) file system using LVM (as is the case for the official CentOS and Ubuntu templates), the new disk can be used to extend the root file system

The simplest option is the first, an example of which is shown here:

# Create a single partition spanning the whole disk
$ fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x598d636f.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p):
Using default response p
Partition number (1-4, default 1):
Using default value 1
First sector (2048-33554431, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-33554431, default 33554431):
Using default value 33554431

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

# Verify that the partition was created
$ lsblk /dev/sdb
NAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sdb      8:16   0  16G  0 disk
└─sdb1   8:17   0  16G  0 part

# Create a filesystem on the partition
$ mkfs.ext4 /dev/sdb1
mke2fs 1.42.9 (4-Feb-2014)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
1048576 inodes, 4194048 blocks
209702 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
128 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
    4096000

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

# Mount the filesystem
$ mkdir /data
$ mount /dev/sdb1 /data

# Verify that the filesystem is now available
$ lsblk
NAME                         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                            8:0    0    32G  0 disk
├─sda1                         8:1    0   243M  0 part /boot
├─sda2                         8:2    0     1K  0 part
└─sda5                         8:5    0  15.8G  0 part
  ├─ubuntu--vg-root (dm-0)   252:0    0  13.8G  0 lvm  /
  └─ubuntu--vg-swap_1 (dm-1) 252:1    0     2G  0 lvm  [SWAP]
sdb                            8:16   0    16G  0 disk
└─sdb1                         8:17   0    16G  0 part /data
sr0                           11:0    1  1024M  0 rom
$ df -h
Filesystem                   Size  Used Avail Use% Mounted on
udev                         990M  4.0K  990M   1% /dev
tmpfs                        201M  676K  200M   1% /run
/dev/mapper/ubuntu--vg-root   14G  1.5G   12G  12% /
none                         4.0K     0  4.0K   0% /sys/fs/cgroup
none                         5.0M     0  5.0M   0% /run/lock
none                        1001M     0 1001M   0% /run/shm
none                         100M     0  100M   0% /run/user
/dev/sda1                    236M   41M  183M  19% /boot
/dev/sdb1                     16G   44M   15G   1% /data

# Add a line to /etc/fstab to make the mount persistent (i.e. automatic mount on boot)
echo "/dev/sdb1  /data  ext4  defaults  0 0" >> /etc/fstab<br>

How to create a VM template

It is possible to take a running virtual machine and create a template from it that can then be used to provision new machines. However before doing this, you should consider whether it is really what you need. In the majority of cases, where what you really want is a repeatable deployment process for your service, it is often better to use a system for automated deployment such as   AnsibleChef or Puppet. These systems allow you to provision a new machine from a standard template (such as the standard CentOS or Ubuntu templates in the JASMIN Cloud) and run an automated process on the new machine to deploy your service. You also get many other nice benefits - for example, you can version control the files defining your deployment process, and you can easily take your deployment to a different platform or use the same process to deploy local test sandboxes using VirtualBox and Vagrant.

Another reason you may want to consider using an automated deployment system is that custom templates count towards the storage quota for your tenancy - quota that could otherwise be allocated to virtual machines! A template takes up at least as much space as the size of the disks attached to the machine it was created from. For example, if you create a template from a machine with a 100GB disk, the resulting template will consume at least 100GB of your storage quota. For this reason it is recommended to create templates from machines with as little storage attached as possible.

In order to create a reusable template, a machine must be prepared appropriately. Unfortunately, it is not possible for the portal to check whether a machine has been prepared correctly - the portal will still create a template from an un-prepared machine, but machines provisioned using the template may exhibit strange behaviour (e.g. unpredictable networking issues). Also, the process of creating a template destroys the source machine, so care should be taken.

To prepare a machine to become a reusable template, the only thing that is   required is to run the script at /root/vappclean.sh. This script does things that are necessary from a system point-of-view to ensure machines deployed from the template behave nicely, but also renders the machine virtually unusable for anything other than creating a template. Amongst other things, it will:

  • Clean out system caches (e.g. for the package manager)
  • Remove any log files, temp files or history files
  • Remove any SSH host keys - this ensures that new host keys are generated for any machines provisioned from the template
  • Remove the hardware ID from any network interfaces to allow the operating system to use the same interfaces for new virtual network cards

However, from an application point-of-view, there may be other things you want to do to create a fully reusable template from a machine. For example, if you are trying to create a reusable template for a web service, you may want to remove machine specific configurations such as SSL private keys before creating a template.

Once a machine has been fully prepared, navigate to the machines page for your tenancy and click the    button for the machine. This will show a form to collect a name and description for the template. Once the template has been created, it will be available to select in the catalogue for your tenancy.

Still need help? Contact Us Contact Us