wiki:HowTo/ManageCustomImagesInstaGENI

Version 41 (modified by lnevers@bbn.com, 8 years ago) (diff)

--

InstaGENI Custom Images

Custom images are usually created when you have defined an experiment node that has all the features and applications that you intend to use again and again for additional experiments. This page capture examples for creating InstaGENI custom images using both the GENI Portal and Omni tools. For guidelines on managing and maintaining snapshots, see the How to manage Custom Images page.

In InstaGENI it is possible to create a custom images (aka snapshot) of a setup that you have reserved. When a snapshot is taken, a copy of that image is available to use in any InstaGENI rack. However, keep in mind that the /users directories are erased when a snapshot is taken. Also if any software is installed that adds a new user or group in the system (/etc/passwd, /etc/shadow, /etc/group), then these user or group changes are not saved.

For example, if you installed the SNMP Daemon package, a new user called 'snmpd' is added and the SNMP server is configured to use this userid to work correctly, but when a custom image is created and then used on a new node, the 'snmpd' user id and group is missing, which will cause the SNMP server to fail to run. As a possible workaround you can save off these newly added ids and groups and add a startup script in rc.local which would add them back in when the image is used later. If you need to save files in your image, you can create a directory on the / directory and save it inside that folder.

In addition to creating a / directory to save any files that may not be otherwise preserved, you can also choose to run an install script that puts this information back in place when the custom image is used again. See the How to create an Install Script page for more details. Following is an outline of features and expected behavior for custom images.

Custom Images with Omni

The Omni tool supports create, using, listing and deleting custom images. Each of these custom image functions are outlined in this section.

Creating a Custom Image

The Omni tool allows experimenters to create custom images by replicating any node in an active experiment. In this example a slice lntest is used which includes 2 hosts. The host1 image will be snapshotted with Omni.

  1. In order to capture the Image running on host1 we must determine the URN for the host1 device by getting a sliver status:
    $ omni.py sliverstatus -a gpo-ig lntest -o
    
  1. Look at the content of the sliver status output file (lntest-sliverstatus-instageni-gpolab-bbn-com.json) and search for host1 and find its sliver_id . Below is an excerpt of the sliver status output file that show information required:
 "attributes": {
 "exclusive": "false",
 "sliver_id": "urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+18223",         <--URN needed to create image
 "component_manager_id": "urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm",
 "client_id": "host1",                                                          <-- This is host1 to snapshot
 "component_id": "urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1"     
  1. You now have the device URN that is actively running as host1 in the slice lntest. With this information we can now have Omni create a custom image which we will name Icreatedthisimage using the slice lntest' and the URN that corresponds to host1:
    $ omni createimage -a gpo-ig lntest Icreatedthisimage -u urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+18223
    17:28:46 INFO     omni: Loading agg_nick_cache file '/home/lnevers/.gcf/agg_nick_cache'
    17:28:46 INFO     omni: Loading config file /home/lnevers/.gcf/omni_config
    17:28:46 INFO     omni: Using control framework portal
    17:28:46 INFO     omni: Substituting AM nickname gpo-ig with URL https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm
    17:28:47 INFO     omni: Slice urn:publicid:IDN+ch.geni.net:ln-prj+slice+lntest expires on 2013-11-04 21:21:50 UTC
    17:28:49 INFO     omni:  (PG log url - look here for details on any failures: https://boss.instageni.gpolab.bbn.com/spewlogfile.php3?logfile=4e800868c7546f75ec6d528ef879bc40)
    17:28:49 INFO     omni: Snapshotting disk on urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+18223 at https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, creating public image ['urn:publicid:IDN+instageni.gpolab.bbn.com+image+ch-geni-net:Icreatedthisimage', 'https://boss.instageni.gpolab.bbn.com/image_metadata.php?uuid=ef4340a8-4017-11e3-9226-029e26f15299']
    17:28:49 INFO     omni: ------------------------------------------------------------
    17:28:49 INFO     omni:  Completed createimage:
    Args: createimage lntest Icreatedthisimage
    
      Result Summary: Snapshotting disk on urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+18223 at https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, creating public image ['urn:publicid:IDN+instageni.gpolab.bbn.com+image+ch-geni-net:Icreatedthisimage', 'https://boss.instageni.gpolab.bbn.com/image_metadata.php?uuid=ef4340a8-4017-11e3-9226-029e26f15299']
    17:28:49 INFO     omni: ============================================================
    

Notice the Result Summary includes the information that you need to use the image: creating public image ['urn:publicid:IDN+instageni.gpolab.bbn.com+image+ch-geni-net:Icreatedthisimage', 'https://boss.instageni.gpolab.bbn.com/image_metadata.php?uuid=ef4340a8-4017-11e3-9226-029e26f15299']. The new custom image URN and URL can be used for other nodes to load the image.

Note The image is usually ready in less than 10 minutes, an email notification will be sent, but the notification tend to be handles as spam by email servers.
  1. Omni also includes a command to listimages , which lists your custom images, both public and private:
    $ omni listimages -a gpo-ig
    17:29:41 INFO     omni: Loading agg_nick_cache file '/home/lnevers/.gcf/agg_nick_cache'
    17:29:41 INFO     omni: Loading config file /home/lnevers/.gcf/omni_config
    17:29:41 INFO     omni: Using control framework portal
    17:29:41 INFO     omni: Substituting AM nickname gpo-ig with URL https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm
    17:29:42 INFO     omni: ListImages using creator_urn 'urn:publicid:IDN+ch.geni.net+user+lnevers'
    17:29:44 INFO     omni:  (PG log url - look here for details on any failures: https://boss.instageni.gpolab.bbn.com/spewlogfile.php3?logfile=fc27433f9d2b6f5dd0183dbf784bd1a4)
    17:29:44 INFO     omni: [
      {
        "url": "https://boss.instageni.gpolab.bbn.com/image_metadata.php?uuid=ef4340a8-4017-11e3-9226-029e26f15299",
        "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+image+ch-geni-net:Icreatedthisimage"
      }
    ]
    17:29:44 INFO     omni: ------------------------------------------------------------
    17:29:44 INFO     omni:  Completed listimages:
    Args: listimages
    
      Result Summary: Images created by urn:publicid:IDN+ch.geni.net+user+lnevers at https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0:
    [
      {
        "url": "https://boss.instageni.gpolab.bbn.com/image_metadata.php?uuid=ef4340a8-4017-11e3-9226-029e26f15299",
        "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+image+ch-geni-net:Icreatedthisimage"            <--The image we just created 
      }
    ]
    17:29:44 INFO     omni: ============================================================
    
  1. You are now ready to use the custom image in other slices. Following is an example node definition in a request RSpec that uses the image we just created:
      <node client_id="cust-host1" exclusive="false">
          <sliver_type name="emulab-openvz" />
          <disk_image name="urn:publicid:IDN+instageni.gpolab.bbn.com+image+ch-geni-net:Icreatedthisimage" url="https://boss.instageni.gpolab.bbn.com/image_metadata.php?uuid=ef4340a8-4017-11e3-9226-029e26f15299" />
          </sliver_type>
          <interface client_id="cust-host1:if0" />
      </node>
    

Using a Custom Image

First find out the Image URL and URN information. This can be done with the Omni listimages option at your rack aggregate:

$ omni listimages -a my_rack_nickname

Now that you have the image information, you can add it to your request RSpec. To add an custom disk image that can run in any rack, you should use both URN and URL in your RSpec:

   <disk_image name="urn:publicid:IDN+instageni.gpolab.bbn.com+image+ch-geni-net:Icreatedthisimage" 
    url="https://boss.instageni.gpolab.bbn.com/image_metadata.php?uuid=ef4340a8-4017-11e3-9226-029e26f15299" />

If you plan to use the custom image in the same rack as the original sliver used to generate the image, you only need to specify your custom image URN if you are using in the same rack:

   <disk_image name="urn:publicid:IDN+instageni.gpolab.bbn.com+image+ch-geni-net:Icreatedthisimage" />

If you plan to use the custom image in a different rack than the original sliver used to generate the image, you only need specify your custom image URL:

   <disk_image url="https://boss.instageni.gpolab.bbn.com/image_metadata.php?uuid=ef4340a8-4017-11e3-9226-029e26f15299"/>

That is all that is required to use your custom image!

Deleting a Custom Image

Deleting your custom image is done as follows with Omni.

  1. First determine the image urn by using the Omni listimages option:
    lnevers@sendaria:~/gcf-2.4.1/instarspec$ omni listimages -a gpo-ig
    17:29:41 INFO     omni: Loading agg_nick_cache file '/home/lnevers/.gcf/agg_nick_cache'
    17:29:41 INFO     omni: Loading config file /home/lnevers/.gcf/omni_config
    17:29:41 INFO     omni: Using control framework portal
    17:29:41 INFO     omni: Substituting AM nickname gpo-ig with URL https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm
    17:29:42 INFO     omni: ListImages using creator_urn 'urn:publicid:IDN+ch.geni.net+user+lnevers'
    17:29:44 INFO     omni:  (PG log url - look here for details on any failures: https://boss.instageni.gpolab.bbn.com/spewlogfile.php3?logfile=fc27433f9d2b6f5dd0183dbf784bd1a4)
    17:29:44 INFO     omni: [
      {
        "url": "https://boss.instageni.gpolab.bbn.com/image_metadata.php?uuid=ef4340a8-4017-11e3-9226-029e26f15299",
        "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+image+ch-geni-net:Icreatedthisimage"
      }
    ]
    17:29:44 INFO     omni: ------------------------------------------------------------
    17:29:44 INFO     omni:  Completed listimages:
    Args: listimages
    
      Result Summary: Images created by urn:publicid:IDN+ch.geni.net+user+lnevers at https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0:
    [
      {
        "url": "https://boss.instageni.gpolab.bbn.com/image_metadata.php?uuid=ef4340a8-4017-11e3-9226-029e26f15299",
        "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+image+ch-geni-net:Icreatedthisimage"  <--The image that we will delete 
      }
    ]
    
  1. Delete the image:
    $ omni.py deleteimage urn:publicid:IDN+instageni.gpolab.bbn.com+image+ch-geni-net:Icreatedthisimage -a gpo-ig
    10:30:54 INFO     omni: Loading agg_nick_cache file '/home/lnevers/.gcf/agg_nick_cache'
    10:30:54 INFO     omni: Loading config file /home/lnevers/.gcf/omni_config
    10:30:54 INFO     omni: Using control framework portal
    10:30:54 INFO     omni: Substituting AM nickname gpo-ig with URL https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0, URN urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm
    10:30:54 INFO     omni: DeleteImage using image_urn 'urn:publicid:IDN+instageni.gpolab.bbn.com+image+ch-geni-net:Icreatedthisimage'
    10:30:57 INFO     omni:  (PG log url - look here for details on any failures: https://boss.instageni.gpolab.bbn.com/spewlogfile.php3?logfile=c5d9fb81c216d701cb729a103b7adab5)
    10:30:57 INFO     omni: Deleted image urn:publicid:IDN+instageni.gpolab.bbn.com+image+ch-geni-net:Icreatedthisimage at https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0
    10:30:57 INFO     omni:  ------------------------------------------------------------
    10:30:57 INFO     omni:  Completed deleteimage:
    Args: deleteimage urn:publicid:IDN+instageni.gpolab.bbn.com+image+ch-geni-net:Icreatedthisimage
    
      Result Summary: Deleted image urn:publicid:IDN+instageni.gpolab.bbn.com+image+ch-geni-net:Icreatedthisimage at https://boss.instageni.gpolab.bbn.com:12369/protogeni/xmlrpc/am/2.0 
    10:30:57 INFO     omni:  ============================================================
    

Your image is now been removed!

Custom Images with the GENI Portal/Jacks

The GENI portal supports the creation and use of custom image, however it does not support deletions. This sections describe how to use custom images in the portal.

Creating a Custom Image

Assuming you are already logged into the GENI portal and are ready to create your custom image.

  1. Go to the "Slices" page and select your slice,
  2. Select the compute node that you want to save,
  3. Click on the "Snapshot" button.

You will then be prompted for the name of your custom image and whether you want it to be public (anybody with the image URN or URL can use it) or private (only you can use it).

Note Choose a custom image name that reflects its content. Also note that the image name MUST include alphanumeric characters only.

Using a Custom Image

To use a custom image in the GENI Portal, you simply requests a VM that uses that image, following these steps:

  1. Select the Slice panel and then click on Add Resources button.
  2. Drag and drop a node type to the canvas.
  3. Select your preferred site aggregate.
  4. Click on the node and then in the left-hand panel select the Disk Image pull-down and choose Other.
  5. In the URN slot paste your 'custom image URN' if the image is on the same rack, or past the URL if you will be loading the image from a remote rack.

You are now ready to use the image.

Deleting a Custom Image

Any custom images that are no longer used should be deleted. Currently it is only possible to delete a custom image with Omni tools, as described in the Delete Custom Image section above. There is no delete image function in the GENI Portal at this time.

Attachments (12)

Download all attachments as: .zip