Hands-on Lab: Advanced Topics


Overview

In the first Hands-on lab you learned Cloud basics: How to create a virtual machine (VM) template, how to create, start and stop a VM and how to perform some basic operations.

This lab will cover more advanced topics like autoscaling or the use of the Amazon EC2 compatible command line interface.

Objectives

In this hands-on lab, you will learn how to:

  • use the Amazon EC2-compatible command line interface to manage your resources,
  • create a persistent disk and installing own software,
  • enable OneGate for your virtual machines and to use it as a key-value-store
  • use OneFlow to create an elastic service that automatically scales up and down.

Prerequisits

The following is required to complete this hands-on lab:

  • LRZ user ID assigned to a LRZ Compute Cloud project,
  • a Linux/Unix workstation with a web browser, and
  • basic knowledge of how to use Sunstone and virtual machines, ideally by completing the first Hands-on lab.

Exercises

This hands-on lab includes the following exercises:

  1. Exercise 1: Configuring and using Amazon EC2 tools with the LRZ Compute Cloud
  2. Exercise 2: Create a persistent disk
  3. Exercise 3: Set up OneGate
  4. Exercise 4: Create an elastic application

Exercise 1: Configuring and using Amazon EC2 tools with the LRZ Compute Cloud

This exercise shows how to configure and use the Amazon EC2 command line tools for Linux to connect to the LRZ Compute Cloud and manage your virtual machines.

This Amazon-EC2-compatible interface allows you to manage your LRZ Compute Cloud resources with tools usually targeting the Amazon Cloud. Not every command of Amazon's interface is supported but a subset of only a subset of commands. However, usually the functionality of the supported commands is ample enough to let you get things done.

The REST endpoint address for the LRZ Cloud is:

https://www.cloud.mwn.de:22

Support for the EC2 interface is not activated by default but must be activated by each user in the Cloud Dashboard individually. It is important to know that you cannot simply use the username and password you need to enter the Dashboard but you need to create a password exclusively used for the EC2 interface.

  1. Activate support for the EC2 interface for your user account in the Dashboard. In the top menu click on the little triangle next to your username and click on settings:
    Username -> Settings
    to open the user's settings menu:
    User settings
  2. In the main settings window you can see the blue Update password button button. When you click on it you can specify a password that will be used when connecting to the LRZ compute cloud via EC2 tools. Create an easy to remember password. This password is exclusively used for the EC2 interface. Do not use the same password as for your Cloud login since it is stored unencrypted on your local disk!
    Update password
    After providing the new password 2 times click the Change button button to save this password. You can change it at any time later by coming back to this settings dialog.
  3. To access the Cloud via the command line tools you need to have the EC2 tools installed (you should find binaries starting with econe-). Please refer to our EC2 tutorial to find instructions to install this software on your local machine. If you are using our classroom VM all these tools are already in place.

    URL https://www.cloud.mwn.de:22
    Username Your username to log in to the LRZ Compute Cloud
    Password The password you defined in step 2 above
    1. Specify the local URL to be used in the environment variable EC2_URL. When using the Bash shell you can do that by typing:

      export EC2_URL=https://www.cloud.mwn.de:22

      Export EC2_URL

    2. Create a directory .one in your home directory:

      mkdir ~/.one
    3. Create a new file one_auth in this directory using your favorite editor. To create this file with the Nano editor, simply type:

      nano ~/.one/one_auth

      Nano
      This file must contain the credentials to log into the Cloud in the format <username>:<password>. This password is stored in clear text! It should look similar to this example:
      one_auth config file.
      Save changes and quit the editor. If you are using Nano, this is done by pressing CRTL-X Y <Enter>

  4. You you need to do some modifications to bridge the gap between the EC2 tools and OpenNebula.

    1. Templates need to provide instance type information describing which resources should be used when a template is instantiated.
      In the Cloud Dashboard, choose your template from the Templates section.
      Overview of the templates
    2. In the Templates menu, press the Update button.
    3. In the new view, click on the Other tab on the right-hand side of the menu.
      Template Menu
    4. On the Other page you can see a Custom Tags section at the bottom. Here you can define custom tags and assign any values to them. The instance type you need to provide is referred to as EC2_INSTANCE_TYPE. Every type of instance you enter here defines an amount of resources (the number of virtual CPUs, the amount of main memory).[^instance-types]
      Define a new custom tag called EC2_INSTANCE_TYPE and assign the value m1.small. Press the Add to save the tag.
      Update template: Other section
    5. You should see the custom tag in the list:
      List of custom tags
      Press the Update button to save your changed to the template.
  5. At this point you can start (and stop) instances in the LRZ Cloud using the EC2 command line tools. Each instance represents a virtual machine. You can show the instances available to you using:

    econe-describe-instances

    econe-describe-instances
    The column instanceId contains the ID of your virtual machines. If you want to start your VM with the ID 12065 (which is the one that has been created earlier in this lab) you can use the econe-start-instances command:

    econe-start-instances i-00012065

    econe-start-instances i-00012065

  6. After a short time you can display the list of your instances by typing again:

    econe-describe-instances

    econe-describe-instances

  7. You can shut down running instances using the econe-stop-instaces command and specify the ID of the instance to stop:

    econe-stop-instances i-00012065

    econe-stop-instances i-00012065

  8. To create a new instance (a new virtual machine) based on an existing template and an OS image you need to use the econe-run-instances command. You must specify an instance type and an image.

    1. To display all available images that can be used to start new instances, use

      econe-describe-images

      econe-describe-images
      Unfortunately this output is not very helpful but you can see how images are represented in the EC2 tools. If you have a look at the Cloud dashboard you can see the names of the images.
      List of images
      (Please note that this screenshot displays the 2nd page of the image list because we need the ID of the image called Debian_7: 772)

    2. To use the Debian_7 image to create a new virtual machine from using the template used for the VM you created earlier in this lab, you need to use the econe-run-instances command. You must specify an instance type to create a new instance:

      econe-run-instances -t m1.small ami-00000772

      econe-run-instances -t m1.small ami-00000772

    3. After some time you will see the newly created virtual machine/instance in the instances list:

      econe-describe-instances

      econe-describe-instances
      You can see this newly created machine also in the web interface.
      Sunstone VMs list

    4. If you are done, please shut down your unused VM! ;-)

      econe-stop-instances i-00012179

      econe-stop-instances

In this exercise you have learned how to acticate the EC2 functionality for your LRZ Compute Cloud account, how to set a password for using it and how to start, stop and create new instances using the command line tools.

The EC2 tools can be used for more things than starting or stopping virtual machines/instances. There are some important things you need to know when using the EC2 tools with the LRZ Compute Cloud (or an OpenNebula-based cloud in general). Please refer to our EC2 Tutorial to find more information about the details and further usage of the EC2 tools.

Exercise 2: Create a persistent disk

When you start from scratch and create a new virtual machine from an existing image provided by the LRZ, you will find just the operating system. Of course you can boot the VM, install software and undeploy the VM afterwards and you will have this software available whenever you boot this particular VM.

If you want to have multiple VMs with this software installed, you could simply repeat the installation process for every VM. However, a much better way would be to create a disk image which has the software installed. This image is then used to instantiate new VMs from.

This is an interesting scenario if you need several VMs configured in the same manner (e.g., when running a particular distributed simulation software on them) started either manually or automatically via OneFlow. OneFlow will be covered in a later exercise.

In this example you will learn how to create a persistent image, install software and then use it for starting multiple VMs.

In this example we will use an image provided by the LRZ and install own software to it using the package manager of the installed Linux system.

  1. As a first step you should clone an existing image. In Sunstone, go to the Virtual resources menu and click on the Images submenu. Search for all images having Debian in their name.
    Looking up Debian images
  2. Activate the checkbox next to the image called Debian_20G (ID: 135) and press the Clone button. A new formn will appead asking for the name of the new image. Use course-worker:
    Looking up Debian images
  3. After pressing the Clone button you will be returned to the list of images. This list will not change because it still displays all images having a Debian in their name. Delete the Debian string from the search box, press enter, and all images will become visible.
    Looking up Debian images
  4. Newly created images are non-persistent by default. If you want to make an image persistent you must set the Persistent flag. Fortunately, this flag can be set easily using Sunstone.
    Make sure that the checkboxes of all images but your newly created image course-worker are deactivated. Your list should somehow look like that:
    Looking up Debian images
    Then click on the More button next to the Clone button you used in Step 3 to open a dropdown menu. From this menu, choose Make persistent.
    Dropdown menu
  5. Unfortunately, you cannot see from the list if a newly cloned image that has not been used yet is persistent. To check if the flag is set, just click your image and you will find an entry named Persistent in the Information section of the image's details page. If the flag has been set correctly, it's value should be yes.
    Details of a persistent image

    • You can modify the Persistent flag directly from this information page. On the right side of the row with the Persistent flag click on the Edit button. Now, you can click on the field and choose Yes or No to decide whether or not an image should be persistent.
      Change the persistence flag
      Make sure that the value is Yes before you leave this screen.

    Click the Back to list button to return to the list of images.

  6. To use this newly created image go to the Templates menu and create a new template using the Plus button in the upper left corner. As Name use course-installation.
    General section

  7. In the horizontal menu click on the Storage icon.
    Storage section
    On the left side you can see a list of images called Disk 0, Disk 1, and Disk 2 that are attached to this template. Click on Disk 0 to change the Disk image for the first disk. A list appears which shows the images available to you. Simply click on course-worker to select it as image for Disk 0.
    Choosing new image for Disk 0
    Right below the list you can see which image has been selected for the particular Disk:
    Selected image

  8. Next you need to connect your virtual network interface card (NIC) to a network. Click on Network in the horizontal menu and make sure that Interface 0 is chosen in the list of NICs on the left. From the list of available networks choose MWN_access_105 (ID: 11).
    Choosing a network

  9. Then, click on Context, mark the check boxes besides Add SSH contextualization and Add OneGate token and make sure that your Public SSH key is copied into the appropriate field. (If you have any questions regarding the creation of an SSH key pair have a look at Exercise 2 from the basic lab.)
    Context section

  10. Click on the Create button to create your new template. You will be immediately returned to your list of templates and will find the newly created one on top of the list:
    Templates list

  11. Mark the checkbox next to your new image and click on the Instantiate button to create a new VM from this template. Choose course-installation as name, then click on the Instantiate button:
    Instantiate template

  12. Go to Virtual Machines. After a couple of seconds you will (hopefully) find your new VM in the state RUNNING. You should also make sure that the VM has an IP address assigned.
    List of your VMs

  13. Once your machine is running and the template is configured correctly you can find the IP address in the list of your VMs. You then can connect via SSH to your machine using private SSH key belonging to the public key that you specified in the Context section of the template. Using this key you can log in as root. Remember that you can specify the private key to use during the authentication process by using the -i flag:

    ssh -i .ssh/course root@10.155.208.119

    First SSH login
    If you log in for the first time make sure that you change the password for root using the passwd command.

    passwd
  14. Update the package repository by calling

    apt-get update

    apt-get-update Now your package repository is up to date. You should update the package lists every time before using the package repository of your Linux distribution.

  15. You can now use the package repository to install software packages that you need to do your work. There are some package we need later:

    apt-get install apache2 php5 curl unzip

    Answer Y (or just press enter) when apt asks you if you want to continue.
    apt-get-update
    Depending on the amount of data that needs to be fetched and the number of software packages you want to install, the installation process will take some time.

  16. To check if you can connect to your machine. Open your web browser and paste the IP of your virtual machine as the address. If your request timed out, your VM probably has some firewall rules in place prevent a user to call your web server from the outside. The iptables rules are located in the file /etc/iptables/rules.v4. Open this file with your favorite editor:
    iptables rules
    After line 9 insert a new line allowing incoming connections on port 80 (HTTP):

    -A INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT

    iptables rules

  17. To apply the new iptables rules, you need to reload them to your local firewall system.

    iptables-restore < /etc/iptables/rules.v4
  18. If you connect to your server now via HTTP (simply press the refresh button in your browser) you should receive a valid answer.
    Response from the web server

  19. Back on the SSH shell you need to download a software archive from our servers containing some sample code. Download and unpack the archive to your home directory:

    cd ~
    curl -O http://niels.userweb.mwn.de/course/2016/Software.zip
    unzip Software.zip

    Downloading and unpacking the archive

  20. The root directory of your web service is located at /var/www/html/. First, delete all files in this directory (there is a default index.html file you need to get out of the way). Then, copy all files from the Software directory to this directory. Finally, you need to adjust the file permissions. You can use the fsck.sh script that is also included in the archive.

    rm /var/www/html/*
    cp -v Software/* /var/www/html/
    cd /var/www/html/
    sh fsck.sh

    Copying the files

  21. If you visit your server again using your webbrowser (Reload! ;-)) you should see this:
    webpage

  22. Try to update the load by entering an integer number to the Set new value: field. Then, press the Save button. The Current load should reflect your changes.
    webpage

  23. Everything is set up now. But before logging out and using this script for other virtual machines, we recommend to remove all contents from the folder /var/lib/cloud/. This is a cache that is filled by OpenNebula. By deleting all contents of this directory OpenNebula is forced to run the complete VM initialization process again when a VM is booted from that image. To delete the files in this directory, execute

    rm -rf /var/lib/cloud/*

    rm -rf /var/lib/cloud/

  24. Log out of the VM using the the logout command. Alternatively, you can press <CTRL>-D that is a shortcut for that command.

  25. After installing and configuring everything as needed, we can undeploy the VM. In Sunstone, go to Virtual Machines, mark the checkbox next to your course-installation VM, press the Stop button and select Undeploy.
    Undeploy the VM

  26. Remember that your disk image is persistent. That means that only one VM at a time can access this image. If you try to intantiate a new VM from the template using this image while another VM based on this image is running, you will get an error message:
    Error while trying to use a persistent image a second time

  27. If you want to use it to instantiate several VMs at a time, you need to make this image non-persistent.

    1. Go to Virtual Machines, mark the checkbox next to your course-installation VM, press the Delete button and select Delete. Confirm with OK.Delete a VM
    2. Go to Templates, mark the checkbox next to your course-installation template and click the Delete button. Press the OK button to delete the template.
      Delete a template
    3. Go to Images, mark the checkbox next to the course-headnode image, click the More button and select Make non persistent.Make image non persistent

In this exercise you have learned how to make an image persistent and store files or install software in it. You also have learned how to make it non-persistent again to allow several VMs at a time to be instantiated from this single image.

Remarks on creating own datablock images

If you want to create persistent images, you do not necessarily want to clone them from existing ones. To create an empty image without any software installed (an empty disk), you can create a datablock images.

These images can be created from the Images submenu. If you are in the Images submenu you can press the Plus button to create a new image. In the form you can select the DATABLOCK type.

Choosing no file system in the new image dialog

If you create a new datablock, it is strongly recommended not to choose a file system in Sunstone. On the bottom if the form you will find some options regarding the file system in the Image location section.

If you create a file system in the web interface, OpenNebula will not only create the raw datablock for you but will create a file system itself. If you simply choose No file system in the dropdown menu, things will be considerably faster (depending on your datablock's size) and gives you more possibilities regarding the file system you want to use for this datablock.

Choosing no file system in the new image dialog

To create a filesystem yourself, you can attach the datablock to one of your VMs and use the mkfs util from the command line. Let's assume that your newly created datastore is the vdd device, you can simply create a new btrfs file system using the following command. Please note, that Sunstone does not give you the opportunity to choose btrfs as filesystem for new datablocks.

mkfs -t btrfs /dev/vdd

Afterwards you can use the mount command to mount this datablock to its mountpoint:

mount -t btrfs /dev/vdd /mnt

Exercise 3: Set up OneGate

In OpenNebula it is possible to define multi-tier applications, i.e., a collection of virtual machines that is seen and managed as a single entity (or service). Within a service, a user can define a hierarchy (which VM should start first), elasticity rules (how and when to scale up and down) and send cumulative commands to the member VMs.

The following 2 exercises will give you an impression of OpenNebula's load-depending auto-scaling functionality using the OpenNebula components OneGate and OneFlow.

  • OneGate is a metadata repository for VMs. Each VM can access it in an exclusive way by means of a unique token created at deployment time. It works as a key/value store, in the form <variable name> = <alphanumerical value>. If the VM is part of a service (a OneFlow application), then the service orchestrator can also access the store. In fact, OneGate is the mechanism used to exchange information between the VM(s) and OneFlow.
    Each VM has its own namespace, so variables with the same name but belonging to different VMs can co-exist. The definition of variables and the criteria to update them is up to the user, as well as the script(s), or any other mechanism, to perform the operation.
  • OneFlow allows users and administrators to define, execute and manage multi-tiered applications, or services composed of interconnected Virtual Machines with deployment dependencies between them.

To access the OneGate store each VM can access it by means of a unique token that is created at deployment time. You will later see that OneFlow can also access the OneGate store of the VMs it orchestrates.

In this exercise you will learn how to enable OneGate for a VM, and how to store and read data from OneGate. OneGate is not enabled by default. You need to enable it from the web interface.

  1. If you are not already logged in to Sunstone, you should do it now. After login you will find yourself at the overview page.
    Dashboard
  2. Update the template you have created in the preceding hands-on lab. Navigate to Virtual Resources -> Templates, press the New Template to create a new template.
  3. In the first tab (General), you need to specify the name for the template. Use elastic-worker.
    Create template for elastic worker
  4. Go to the Storage tab. From the list of disks choose Disk 0 and attach the image course-worker to it.
    Attach image to Disk 0
  5. Go to the Network tab and attach the network MWN_access_105 (ID: 11) to the template.
    Add network
  6. Go to the Context tab. Add the public SSH key you want to use for authentication to the Public Key text field. OneGate will also be actived in this menu. To activate OneGate, select the Add OneGate token checkbox. This triggers the generation of a context disk, which is attached to the VM as a CD-ROM to the first available channel of the IDE bus (i.e., the device hda, if no other CD-ROM images have been added by the user. Some distributions may set up the device /dev/cdrom or emulate the SCSI interface for the cdrom, so that the device is accessible as /dev/sr0 or /dev/sr1).
    Activate OneGate
  7. In the left menu click on Custom vars. Here you can set additional keys and values that should be uses in the contextualization process.
    Custom vars
    By activating OneGate in the preceding step OpenNebula will ensure that the CD-ROM is created. However, it will not be available to your virtual machine until you mount it. Usually, users need to do it manually. To trigger the automatic execution of commands when booting a VM you can also use contextualization. You can provide a script that will be executed right after the boot process of the VM. Unfortunately, you cannot simply upload the script but you need to base64-encode it.

    1. OPTIONAL: If you want to do it yourself, you can write the script yourself. You need to create a new file, let's say script.sh. Use your favorite editor to create this file on your local machine with the following contents:

      #!/bin/sh
      echo "/dev/cdrom /media/cdrom udf,iso9660 defaults 0 0" >>/etc/fstab
      mkdir -p /media/cdrom
      mount -r /media/cdrom
      rm -rf /var/lib/cloud/*

      Save it and exit your editor. In the first line you specify what interpreter should be used to run this script and in the last line you mount the CD-ROM device to /mnt.
      Shell script to mount the contextualization CD-ROM

    2. OPTIONAL: You now need to base64-encode your script. You can use base64 command to do that.

      cat script.sh | base64
    3. The result should be the following string:

      IyEvYmluL3NoCmVjaG8gIi9kZXYvY2Ryb20gL21lZGlhL2Nkcm9tIHVkZixpc285NjYwIGRlZmF1bHRzIDAgMCIgPj4vZXRjL2ZzdGFiCm1rZGlyIC1wIC9tZWRpYS9jZHJvbQptb3VudCAtciAvbWVkaWEvY2Ryb20Kcm0gLXJmIC92YXIvbGliL2Nsb3VkLyoK

      Shell script to mount the contextualization CD-ROM
      You need to store it as a new key called USER_DATA. In the top input field enter the name USER_DATA, in the lower input field enter the base64-string from above. Then press the Add to save it.
      Custom vars

    4. You need to add another key/value specifying how you encoded the USER_DATA field. Therefore, add the key USERDATA_ENCODING to the first input field and base64 in the lower input field. Again, press the Add to save it.
      Custom vars

    5. Make sure that both entries exist before you leave.Custom vars

  8. Make sure that you activate your changes by pressing the Update button.

  9. Generation of the OneGate token depends on the property TOKEN_PASSWORD in the user's settings. In order to check that, simply open the user's settings by means of the drop-down menu at the top of the page.
    Accessing the user's settings
    Scroll down to the Custom Attributes section and make sure that an entry named TOKEN_PASSWORD exists.
    User settings: Custom Attributes

    • If this key does not exist you can create it at any time. The TOKEN_PASSWORD will be used to sign the VM token. In fact the string should be nothing but the SHA1 sum of a password. If you need to create one, you can just type

      echo -n $(date) | sha1sum

      on a Linux terminal or

      echo -n $(date) | shasum

      on a Mac and copy the output string into the form. Actually, the string that has been hashed is not important, it will never be requested.
      Scroll down to the bottom and enter a new property with the name TOKEN_PASSWORD and the 40 characters alphanumerical string created using one of the commands above as value. Do not forget to click on the Add button to save it.
      User settings: Add token password

  10. Time to create a new virtual machine from the new template. Go to the template overview page, mark the checkbox next to the elastic-worker template, and click on Instantiate in the top row.
    Instantiate a new template

  11. Name the VM to create elastic-worker and click on the Instantiate button to create the new VM.
    Instantiate a new template

  12. After some time you will see the running VM in the Virtual Machines overview page of SunStone.
    Overview of existing VMs

  13. This page shows the IP address of the VM. Use this IP and the private SSH key created earlier to connect to the VM. In my case, the key is named course and stored in the .ssh directory in my home. Just type:

    ssh -i .ssh/course root@10.155.208.119

    Connect to the VM via SSH Please note that we do not change the password for user root here. This has nothing to do with forgetfulness... you already changed it when you created the image in persistent mode!

  14. The contextualisation disk contains all the info to access OneGate, namely the IP address and the token. Remember the base64-encoded script you entered in the Contextualization area of the template settings? This script mounts the disk automatically during the boot process. You should have a look if this script worked correctly by showing all active mounts having cdrom in their name.

    mount | grep "cdrom"

    Show all mounts with "cdrom" in its name

    • If you see no output the script has not worked correctly. You need to mount the disk manually. This can be done as follows:

      mount /dev/cdrom /media/cdrom

      You can list the contents of the disk by typing:

    ls /media/cdrom/

    Contents of the contextualization disk
    To get an overview of the contents of each file you can show them by typing

    cat /media/cdrom/context.sh

    or

    cat /media/cdrom/token.txt

    respectively.
    Contents of the contextualization files
    The file context.sh containts among other fhints the OneGate endpoint address to where data will be sent and the ID of the virtual machine. The file token.txt will contain the actual token.

  15. To communicate with OneGate you need to source context.sh and need to store the contents of token.txt to the environment variable ONEGATE_TOKEN. You can do that by typing:

    source /media/cdrom/context.sh
    ONEGATE_TOKEN=$(cat /media/cdrom/token.txt)

    You can check if the files were loaded correctly by viewing the values of $ONEGATE_ENDPOINT and $ONEGATE_TOKEN, respectively.

    echo $ONEGATE_ENDPOINT
    echo $ONEGATE_TOKEN

    Load files

  16. Let's test if the OneGate server responds to our very first request. Let us ask OneGate for some details about our virtual machine. We need to put the request together manually and use curl to send the data to the OneGate endpoint. The command line looks as follows:

    curl -X "GET" "${ONEGATE_ENDPOINT}/vm" --header "X-ONEGATE-TOKEN: $ONEGATE_TOKEN" --header "X-ONEGATE-VMID: $VMID"

    Shortly after you should receive an answer.
    OneGate answer
    The options used for curl are the following ones:

    • -X: the request type, namely GET (retrieve) and PUT (store) are significant in our case;
    • --header: the headers that are expected by OneGate on top of the HTML ones. Two headers are needed (the token and the VM identifier), so the option should be used two times.
  17. We mentioned that OneGate provides also the ability to store data. This can also be achieved by using curl and using the PUT request type. In the following example we will store the current load of the virtual machine to OneGate. The -d switch defines the data to store to OneGate. We use the uptime command to get the current load of the virtual machine and use awk to reformat the output. The complete command line looks like this:

    curl -X "PUT" "${ONEGATE_ENDPOINT}/vm" --header "X-ONEGATE-TOKEN: $ONEGATE_TOKEN" --header "X-ONEGATE-VMID: $VMID" -d "LOAD = $(uptime | awk '{ load=100*$NF; print load }')"

    OneGate answer
    As you see, you will see nothing. But you can repeat the GET command from above to check if a new key named LOAD will be returned.

    curl -X "GET" "${ONEGATE_ENDPOINT}/vm" --header "X-ONEGATE-TOKEN: $ONEGATE_TOKEN" --header "X-ONEGATE-VMID: $VMID"

    Second OneGate answer
    You can find that the new key LOAD and it's value will be returned from OneGate.

  18. You can also see the values stored in OneGate using the SunStone web interface. In the virtual machine list click on the virtual machine course-worker.
    VM overview
    On the details page scroll down to the Attributes section. Here you will also find the newly created key LOAD and its value.
    OneGate keys

In this exercise you have learned how to set up OneGate and perform basic requests to the OneGate server. These steps build the foundation for the next exercises and are necessary to use OneGate from your virtual machines.

Please remember that the namespace is strictly private. It can be accessed only via the token, which is unique for each VM, even if generated from the same template

OneFlow can access the namespace of all VMs. If the VMs are part of a service, then OneGate is the right tool to support elasticity rules

OneGate is not a way to exchange information between VMs, it is rather a way to communicate to the service orchestrator when the VMs are part of an elastic application.

Exercise 4: Create an elastic application

The next step in the creation of elastic cloud service based on several virtual machines that scale up and down automatically is to set up OneFlow.

In this exercise, you will learn how to deploy an application that comprises of a worker VM. Usually you will find multi-tier applications that comprise at least two typical roles: A frontend and a backend. However, to understand the auto-scaling mechanism it is suitable to work with a single role.

An elastic application is described by a service template you are going to create in this exercise. A template defines a couple of important things:

  • Roles: the VM's type or the VM's task. A role is associated to a VM template, that is, VMs with the same role are launched using the same template (picked from the list of templates accessible to the user).
  • Hierarchy: the dependencies between roles, i.e., which VMs should be started first.
  • Elasticity rule(s): the criteria(s) to start new VMs (scale up) or shrink the infrastructure (scale down).

  1. In the main menu you will find the OneFlow button as the last button on the left-hand side. When clicked it will expand and reveals 2 sub menus called Services and Templates.
    Main Menu
  2. The first step on the way to create an elastic application is to create a service template. Click on the Templates submenu on the lower left-hand side.
    Service templates list
  3. To create a new template press the + button on the upper left of this template list. A new form will open asking you for the name of the template and other things.
    New template form
  4. You need to choose a Name for your new service. Use elastic-app. If you like, you can add an optional description.
    Set the name of the service
  5. Scroll down to the Roles section. If you could not see it, you need click on Advanced Service Parameters. Here you need to set the Role Name to Worker and set the number of VMs to 1. As VM template select the elastic-worker template you created earlier.
    Set the number of VMs
  6. The next step is to define the elasticity rules. These rules are evaluated regularly and based on the evaluation your service can scale up or down.
    Let us employ the following strategy:

    • Scale up when a worker node has a load greater or equal than 5 three times in a row and evaluate this rule every 30 seconds. In other words: OpenNebula will scale up if a worker has overload for at least 90 seconds.
    • Scale down when a worker node has a load of 0 three times in a row and evaluation this rule every 30 seconds. After 90 seconds of no load the OpenNebzla will scale down your service.

    Scroll down to the Elasticity policies section.

    1. You need to set the number of Min VMs, Max VMs and the Cooldown period.

      • Min VMs: the minimum number of VMs that should be always present for this role (usually 1);
      • Max VMs: the maximum number of VMs allowed for this role. This is the maximum capacity;
      • Cooldown: after an elasticity policy is triggered, OneFlow stops evaluating the rule(s) for a certain amount of time, defined as Cooldown. By default it is 300 seconds (5 minutes), but it can be set here for all the rules specified for this role. The reason to have a cooldown interval is to temper the effect of transients, avoiding to trigger unnecessary actions and giving enough time to the control parameter to adjust itself to the new infrastructure (i.e., more VMs to share the load with). If a specific rule specifies its Cooldown period, then the general (this) and the default ones are overridden.

      Set Min VMs to 1, Max VMs to 3, and Cooldown to 30. These values might be a bit too small for real-world use-cases but they fit perfectly to this hands-on lab because they allow you to see OpenNebula scaling after few minutes.Setting *Min VMs*, *Max VMs*, and *Cooldown*

    2. Now you need to set the rules to be evaluated. You have 2 different possibilities:

      • Elasticity policies: they are event-triggered, a parameter is monitored and if a certain threshold is met, then OneFlow takes a certain action (deploy a new VM or shutdown an old one);
      • Scheduled policies: the action described here (deploy a new VM or shutdown an old one) is time-triggered, i.e., carried out at a certain moment or at regular intervals.

      In our case, the Elasticity policies are the correct ones, because we want to react on increased and decreased load values. We need to define 2 rules here, one for scaling up and another for scaling down.

      An elasticity rule comprises several fields:

      • Type: the action that should be carried out by OneFlow when this particular rule is triggered. This parameter works together with the Adjust box (detailed later on) since the latter provides the magnitude. In other words, an action consist of Type and Adjust. The user can choose among 3 different values for Type:
        • Change: add or remove the number of VMs specified in the Adjust box;
        • Cardinality: set the total number of VMs to what is specified in the Adjust box;
        • Percentage: add or remove VMs, interpreting Adjust as a percentage, to be applied to the current total number of VMs. The minimum adjustment that is applied is contained in the input box Min.
      • Adjust: the numerical parameter (or magnitude) that defines an action together with the Type. It has to be an integer and its meaning varies according to the value of Type:
        • Change: the number of VMs that should be added or removed. In case of removal, just specify a negative integer;
        • Cardinality: the total number of VMs that should be present as a result of the triggered rule. In this case only positive numbers are meaningful;
        • Percentage: the VMs that should be added or removed, expressed as a percentage of the current total number of VMs. It should be a positive integer number, less than or equal to 100. In case of removal, just specify a negative number. The minimum adjustment that is applied is contained in the input box Min.
      • Min: in case the action Type is Percentage, OneFlow deploys or shuts down at least this minimum number of VM. In case the percentage (with respect to the current total number of running VM for this service) contained in Adjust is below this threshold, the number is rounded up to Min;
      • Expression: the condition that triggers the current rule. It is simply an expression (>, <, ==, >=, <=) that involves a parameter and a numerical threshold. The parameter value should be updated by all the VMs belonging to this role via OneGate. OneFlow reads the same facility and it executes the rule if at least one of the VMs in this role satisfies the condition. The same rule is not evaluated anymore for a period of time equal to the Cooldown period (in the following order: the specific one for this rule, the general one for this service template, the default one). In our example, the expression that triggers the deployment of a new VM is LOAD >= 5, while the shut down of a VM happens when LOAD == 0;
      • #: the number of Periods during which the Expression should be satisfied in order to trigger the current rule;
      • Period: the time interval that is used by OneFlow to check the value of the Expression. It is a sort of sampling time. The rule is triggered if the Expression evaluates to true for a contiguous number of Periods equal to what specified in #.
      • Cooldown: as explained before, this is an inactivity period after a rule is triggered to temper the effect of transients. This value overwrites the default and what is specified at the beginning of the form for the current role. Of course it should be greater than the observation time window required to trigger the rule.

      After this long introduction to Elasticity policies we can formulate the 2 rules we need. From our strategy mentioned above, we need to specify the following 2 rules:
      Elasticity rules

      By default, there is only one line available to specify a rule. Just click on the Add another policy to add a new line.

    3. When finished, click on the Create button on the top to create your service template.
      OneFlow services list

    4. Now we can start our service. Mark the checkbox next to our newly created template and click on the Instantiate button.
      Instantiate a OneFlow service

    5. You need to specify a Name and a Number of instances of this service. Use course-service as name and let the Number of instances at 1.
      Enter a name

    6. If you go to Oneflow > Services you can see your available services. After some moments your new service will enter the state RUNNING.
      List of services

    7. Click on the service to see more details. In the main *Info windwow you see general information about your service.
      Service details

    8. The Roles tab is much more interesting because it provides information for VMs belonging to your different roles. In our case, we only have one Role. Click on the appropriate Role to get information for the VMs belonging to this role.

      Service details

      If you scroll down you will see all active VMs taking this role and the Elasticity policies. What it is interesting for us is the IP of the virtual machine.

      Service details

    9. Open your web browser and provide the IP of your worker VM. This will lead you to the webpage where you an specify the load for this particular VM. To test if our rule for upscaling evaluates correctly, enter 10 as load and press the Save.

      Settingt the load

    10. You can test it the value in OneGate was set correctly by clicking on the VM in the list. On the Info page that appears scroll to the bottom. If you find an Attribute called LOAD everything worked correctly.

      VM details

    11. Back in the Roles details of your services, you can see the current state of your rules at the bottom of the page.

      State of the rules

    12. After some time you can see the new VMs here that have been started by OpenNebula.

      State of the rules

    13. To scale down you can set the load of for your elastic service on the web page of the VM you opened earlier. Just set the load to 0 to trigger the downscale process.

    14. You can also scalue manually at every time. In the Roles details of your service, you can find a Scake button. Click on this button and you can set the cardinality (the number of running VMs) of the marked role(s) manually. Click on the Scale button and set the Cardinality to 1. Then press the Scale button to trigger the scaling process.

      Set cardinality manually

      Remember that OneFlow will scale up again automatically until you adjust the load via the webpage.

    15. After some time you can see in the VM list which VMs are marked for going down.

      VMs marked for going down

      Wait for few moments and the VMs will disappear from that list.

      List without the shut down VMs

    16. To stop your service, just click on the Shutdown on the top right when you are in the service details or go the the list of your services, mark the checkbox next to the service you want to stop and click on the Shutdown button.

      Shut down a service

That's it. In this exercise you created an elastic service, learned the ways how to scale up and down a service and how to stop a running service.

Remember that this last exercise showed a simplyfied workflow to introduce you to the basic actions. Real-world use-cases would probably contain serveral roles and a more complex mechanism to store real load data to OneGate. You can find some additional information on our tutorial page.

Summary

Congratulations for completing this hands-on lab covering advanced topics.

In this hands-on lab, you learned how to:

  • use the Amazon EC2-compatible command line interface to manage your virtual resources,
  • how to create persistent images and store configurations and software,
  • how to enable OneGate and store and read information from your virtual machines,
  • and how to create a basic elastic service based on OneFlow.

You should have gained a profound understanding of the LRZ Compute Cloud after completing the training and both hands-on labs. Probably your individual case is not exactly reflected by these exercises, but you should be aware of the fact that there might be several ways leading to the same result.

We encourage you to play around with the technology, have a look at our documentation and get in contact if you run into trouble.

Links/Further Reading


These materials are part of the LRZ Compute Cloud training course and are copyright by the Leibniz Supercomputing Centre. All rights reserved. Except where otherwise noted, these materials are licensed under the terms of the Apache License, Version 2.0. You may use it according to the license as is most appropriate for your project on a case-by-case basis. The terms of this license can be found in http://www.apache.org/licenses/LICENSE-2.0.