Version 5 (modified by, 9 years ago) (diff)


GRAM Experimenters User Guide


GRAM (GENI Rack Aggregate Manager) is fully compliant (V3.0) GENI Aggregate Manager that presents rack resources to authenticated and authorized clients of the AM API.

GRAM supports GENI Rspecs V3.

A description of the AM API is beyond the scope of this document. For this see:

This document focuses on unique features of the GRAM AM, as well as features that are currently still under development.

GRAM Implementation of the GENI AM API V3

GRAM aggregates provide experimenters with two types of resources:

  • Virtual machines
    • GRAM virtual machines come in a variety of flavors where the different flavors represent virtual machines with different amounts of CPU and memory resources. Examples of virtual machine flavors include mi.small, m1.medium, etc. Experimenters can specify the flavor they desire as an attribute of the <sliver_type> element of the request rspec. A list of available flavors can be obtained by calling GetVersion on the GRAM aggregate.
    • GRAM does not support exclusive virtual machines i.e. experimenters cannot request a virtual machine be placed on a compute host all to itself. Request rspecs with the exclusive attribute of the <node> element set to true will be rejected.
  • Network links
    • Network links that connect virtual machines within a GRAM aggregate are bi-directional.
    • GRAM does not honor bandwidth specifications in request rspecs. Experimenters who need to know the throughput of their links will have to measure this themselves.
    • Only links that connect virtual machines on different compute hosts pass through the rack OpenFlow switch and consequently only the flows on these links can be controlled by OpenFlow controllers. See the "OpenFlow-controlled GRAM Topologies" section below for details on use of OpenFlow in GRAM slices.

The following are brief descriptions of the GENI AM API V3 methods implemented by GRAM.

GetVersion. Returns information about the GRAM AM.

ListResources. Lists the virtual machine types and disk images available at this aggregate.

Describe. Returns the status (allocation status, operational status, expiration time) of the specified slivers along with a manifest rspec for these slivers. To get the status of all the slivers in a slice and a complete slice manifest, specify the slice name instead of sliver urns.

Allocate. Allocates resources specified in the request rspec and returns a manifest rspec along with sliver status. Allocate may be called multiple times on the same slice to add resources to the slice. However a resource that has been allocated cannot be allocated again unless it is first deleted.

Provision. Provisions the specified slivers i.e. instantiates virtual machines and links for these slivers. If the slivers specified include links and virtual machines, the links are provisioned before the virtual machines. Provision may be called multiple times on the same slice to provisional additional resources. If a node is connected to one or more links, the links must be provisioned before the node. If a node is provisioned before one or more of the links to which it is connected, the provision of the node will succeed but it will not have an interface connected to the link and will therefore not be able to use this link. [This limitation is expected to go away when GRAM implements the Update method proposed for AM API V4.] To provision an entire slice, specify the slice name instead of sliver urns.

Renew. Changes the sliver expiration time to the specified value. If the specified time is past the expiration time on the slice credentials presented, the expiration time is set to the slice credentials expiration time.

Status. Returns the status (allocation status, operational status, expiration time) of the specified slivers. To get the status of all the slivers in a slice, specify the slice name instead of sliver urns.

PerformOperationalAction. GRAM does not implement this method. A call to this method will return with an error.

Delete. Deletes the specified slivers. f the slivers specified include links and virtual machines, the virtual machines are deleted before the links. If one or more slivers did not get deleted, the call will succeed but the return will include a message saying one or more slivers failed to delete. The operational status of the slivers that failed to delete will remain unchanged i.e. they will not be in the geni_stopping state. A link will fail to delete if there are one or more provisioned virtual machines still connected to the link. To delete all the slivers in a slice, specify the slice name instead of sliver urns.

Shutdown. Deletes all the slivers in the specified slice.

GRAM AM API V2 support

GRAM supports version 2 of the GENI AM API as well. There is a separate URL/port that presents a version 2 aggregate that translates V2 calls to V3 calls and then uses the V3 implementation.

OpenFlow-controlled GRAM Topologies

  • GRAM provides OpenFlow control to experimenters who provide a controller in their request rspec. Those that do not provide a controller will be provided with a standard L2-learning switch behavior
  • GRAM provides a single controller called VMOC (VLAN-based Multiplexed OpenFlow Controller) that multiplexes control traffic between multiple controllers and multiple switches based on VLAN tags associated with slices and switch DPID's. VMOC serves as a controller to switches and as a switch to controller. To debug a controller, one may want to shut down VMOC and have the OpenFlow switch speak exclusively to that controller (see the Administrator's guide for details).
  • If a slice includes an OpenFlow controller at a GRAM aggregate, the slice cannot have more virtual machines at that aggregate than the number of compute hosts at the aggregate. This is because network links connecting the virtual machines pass through the GRAM rack OpenFlow switch only if the virtual machines are on different compute hosts. In order for experimenter provided OpenFlow controllers to be able to control flows on links connecting the virtual machines in their slices, the virtual machines in a slice must be on different compute hosts. When a slice is provisioned, GRAM attempts to place the virtual machines in that slice on different compute hosts. If the slice includes an OpenFlow controller at that aggregate, an attempt to provision more virtual machines than compute hosts will fail. If the slice does not include an OpenFlow controller, there is no limit on the number of virtual machines that can be provisioned (subject to the availability of resources and to local site policies).

Rspec Extenstions

  • Users can request an externally-routable public IP address on a VM in one of two ways:

By using the emulab extension in the node specification. For example:

<node client_id="exp1-host1">

<sliver_type name="m1.small"> <disk_image description="" name="ubuntu-12.04" os="Linux" version="12"/> </sliver_type>

<interface client_id="exp1-host1:if0" > </interface>

<emulab:routable_control_ip xmlns:emulab=""/>


Or by as a node attribute. For example:

<node client_id="exp1-host1" component_manager_id="urn:publicid:IDN+geni:bbn-cam-ctrl-1:gcf:gram+authority+am" exclusive="false" external_ip="true" >

Note: The routable ip address will not show up in 'ifconfig', instead the user will see a management address. Traffic is NAT-ed between the management ip address and the routable address so nodes behave (from the outside) as if they have this address but they do not display it. Use 'describe' to determine the assigned routable IP address.

Custom Images

Users can create a snapshot of a VM for use in future slices. Images can be created and deleted using the 'performoerationaction' command and requires an options file. The command to create a custom image of a snapshot in a VM that exists in the slice is:

omni -a <am-nickname> -V3 performoperationalaction <slice name> create_snapshot --options <optionsfile>

The options file must be in the form of a python dictionary have a "vm_name" key which contains the name of an existing VM in the specified slice and "snapshot_name" which is the name of image to be created. An example optionsfile is shown below.

 "vm_name": "exp1-host1",
 "snapshot_name": "my-ubuntu"

The command to delete and image is:

omni -a <am-nickname> -V3 performoperationalaction <slice name> delete_snapshot --options <optionsfile>

The optionsfile is of the same format and must contain the field "snapshot_name". It must be called using an existing slice, though the slice need not contain any VMs with the image to be deleted.

A list of custom images is provided at the end of the advertisement rspec.

It is important to note that private images are not supported at this time. Any use can instantiate any custom image stored in GRAM. Private or proprietary data should not be stored open on custom images.

Each custom image must have a unique name.

Developing Features

This section describes features of GRAM that are still under development.

  • GMOC Integration. Integration of the GRAM rack with GMOC (GENI Meta-Operations Center) monitoring and reporting capabilities. This provides statistics to the GMOC about mappings of slices to sliver resources and use of aggregate resources by authorized users.
  • How to Create a Multi-Point VLAN On OpenGENI. A systematic way of doing this is still under development.