ProtoGENI Project Status Report

Period: 2Q 2009

I. Major accomplishments

A. Milestones achieved

While we have not finished any milestones this quarter, we have made significant progress towards may of them. This progress is detailed throughout this report.

B. Deliverables made

We have continued to make our software available under open source licenses (the GENI Public License and the GPL). This software includes not only our component manger, clearinghouse, etc., but also example scripts demonstrating and documenting the use of this software. (This supports the milestone "Support experiments on ProtoGENI cluster".)

We continue to release documentation and design decisions on our wiki at . Many design discussions take place on an open mailing list, linked to from the ProtoGENI wiki. Some important recent additions to the wiki:

  • Significant progress on our RSpec, including the first stages of extensibility, some examples, and modifications for our Manifest.
  • Updates to the Component Manager and Slice Authority APIs
  • New Slice Embedding Service API (see below for more details on this)
  • Details about our backbone deployment (see below for more details)

II. Description of work performed during last quarter

A. Activities and findings

A major focus of our activity this quarter has been on the deployment of the ProtoGENI backbone with Internet2. Details of our plans can be found on these pages:

We have acquired almost all of the hardware we need for the deployment, which has been a very difficult process. (This addresses the milestone "Integrate, install 3 nodes in I2".)

First, Dell is end-of-lifing the servers we had planned to use at the backbone sites, and the replacement does not suit our needs (it us unable to support the NetFPGA cards.) Thus, we have purchased (through other funding) the PCs for all backbone sites expected to be deployed over the next three years. We did extensive testing of these PCs to ensure that they would work in the configuration needed, and that the remote access capabilities (console and power control over IP) are sufficient for recovery in case of problems. We also re-worked our boot process to be more robust, since getting physical access will be difficult and costly. Our usual wide-area PCs boot from a single USB flash drive that is write-protected; this way, experimenters cannot maliciously or accidentally interfere with the boot process. However, we were concerned that this did not provide enough flexibility: if we ever needed to update the boot code, that would be very hard to do, since it would require a physical swap of the flash drive. However, we were not willing to make the boot device read-write, as it would not be possible to prevent it from being overwritten by experimenters (since we intend to give full control over some machines to them.) Our solution now involves two USB drives, one of which is write protected, and one of which is not. This way, we can normally boot from the read-write drive, but if there is some problem, we can boot from the read-only drive, which can be used to re-flash the read-write drive. These PCs are spec'ed to be very sliceable: they have eight cores, sixteen GB of RAM, four disks, and six network interfaces. (We will be able to meet the milestone "Control Plane integration of backbone nodes" shortly after these nodes are installed.)

Second, the nature of our relationship with HP changed (see the "project participants" section for more details). This delayed the purchase of the switches. A number of other issues conspired to delay the purchase, including getting organizational approval on both sides, some confusion over quantities, and the fact that the purchase was not done through usual channels. A piece of good news, however, is that due to the change in our relationship with HP, we were able to purchase equipment without being subject to F&A charges from the University. This allowed us to purchase the set of equipment included in our original proposal, rather than the reduced amount in our amended proposal due to budget cuts in the subcontract awarded. This will give us additional ports in some switches for connections to other projects, some of which we are already planning uses for. These switches have *almost* all arrived at Utah at this point; we are missing two power supplies, which are still on their way from HP. We have spares from another project that can be used if the ones on order do not arrive in time. Before shipping equipment for the first site, we have done extensive testing in our lab of the software for controlling the switches. (This testing ensures that we will be able to meet the milestone "L2 control in backbone nodes" shortly after installation.)

We moved from using UUIDs to identify resources in ProtoGENI to URNs. URNs offer the advantage of a hierarchical namespace, mirroring the hierarchical set of authorities we plan to deploy. This scheme was proposed by the GMOC, with refinements based on our feedback. Our current software supports both UUIDs and URNs for backwards compatibility, but UUIDs will be faded out over time.

(The following text describes important progress made under other funding.)

We have added support for native shared nodes to the Emulab and ProtoGENI software. This enables us to create VLAN-based virtual networks on sliced nodes. Specifically, we have added support for Linux-based virtual nodes using OpenVZ. OpenVZ is a Linux kernel modification that provides lightweight, container-based virtualization by contextualizing the kernel's operation on many namespaces and objects on a per-container basis. When using OpenVZ-based virtual nodes, Emulab provides the user with one OpenVZ container per logical node. OpenVZ is an excellent match for ProtoGENI since it virtualizes much of the network stack down to Layer 2, meaning that it can present virtual MAC layer devices to containers, allowing users to configure device and protocol properties fully above that layer (i.e., each container has its own private, configurable IPv4 routing table). Currently, we provide a Fedora 8 Linux file system inside each container. In the future, we will support multiple Linux distributions and user-customizable container "disk" imaging and loading like we provide for physical nodes.

While developing the OpenVZ support itself, we also explored and prototyped a standard API for configuring Emulab and ProtoGENI virtual nodes, called libvnode. Eventually, we believe Emulab should support more vitualization technologies, since each can expose a different point along the realism versus cost spectrum (i.e., heavyweight Xen or VMWare that provide a fully emulated machine, or lightweight process-level virtualization like Linux Vservers or OpenVZ that provide only a customizable user space environment). libvnode provides a single Emulab virtual node configuration API that can support multiple back-end virtualization technologies. We have used and modified the libvnode API to add preliminary Xen virtual node support to Emulab.

We have deployed a Slice Embedding Service: Emulab' resource mapping program, assign, has many of the key properties required for a GENI slice embedding service. A slice embedding service takes a description of the experimenter's desired virtual network and a description of the facility's physical resources, and then selects an appropriate set of components on which to instantiate the virtual topology. The assign program fills this role in Emulab, and we have extended it to become a prototype Slice Embedding Service (SES).

Our SES runs as a service, not tightly coupled to any of the federated facilities. It is an optional step in the process of creating a slice, taking two parameters in RSpec format: a description of the topology that the user has designed, and the set of resources available at some Component manager. The SES annotates the user's requested topology, filling in the identities of nodes and links that can be used to satisfy it. By offering the SES as a standalone service, we make it possible for multiple SESes to exist; some could specialize in embeddings for particular types of hardware or topologies. The user may also give a "partial solution": part of the topology could already be embedded by the user himself or by another SES. This enables the use of multiple embedding services, in serial, on a single request topology.

Currently, our SES supports only embedding to a single aggregate at a time. Future work on this service will enable it to map slices across multiple aggregates.

B. Project participants

University of Utah

The subcontract with Internet2 has been successfully signed, and we have been working closely with Internet2 staff (as well at the GPO) to plan and deploy our backbone nodes.

HP Labs

The exact nature of our relationship with HP has changed. Originally, HP was going to be a sub-contractor to us. This turned out to require some legal arrangements that didn't seem necessary for what was essentially a hardware purchase. Thus, at HP's suggestion, we purchased the equipment from them using a purchase order (at the steeply discounted price originally agreed to), as well as consulting services for HP representatives to attend GECs, participate in GENI-related activities, etc. This change was approved by the GPO.

C. Publications (individual and organizational)

D. Outreach activities

We participated in the second GENI-FIRE Workshop in Seattle, giving a talk about the current state of federation in ProtoGENI and the future possibilities for international federation.

We have kept in touch with several projects that proposed to join our control framework cluster under Solicitation 2.

Robert Ricci served on three program committees that received GENI-related papers:

  • The First ACM SIGCOMM Workshop on Virtualized Infrastructure Systems and Architectures (VISA), held in conjunction with SIGCOMM 2009
  • The 2009 workshop on Cyber Security Experiment and Test (CSET), held in conjunction with the USENIX security conference
  • The Fourth Workshop on Real Overlays and Distributed Systems (ROADS), to be held alongside SOSP 2009

E. Collaborations

We have continued to organize bi-weekly Cluster C conference calls, which have helped our cluster members to make progress together. In addition to the members assigned to our cluster, the Security Architecture project has been a frequent participant, and some calls have featured members of the GPO and other invited projects. (These collaborations are in support of the milestone "control plane integration of cluster-mates".)

Two new projects have joined our cluster this quarter, the Million Node GENI project from Washington and the Digital Object Repository from CNRI.

In addition to the existing members of our federation (Utah, Wisconsin, CMU, Kentucky), two new facilities have joined our federated clearinghouse:

Programmable Edge Node (UMass-Lowell)
BBN (test facility being set up by intern Evan Zhang)

We have also interacted with a number of other GPO-funded projects outside of our cluster, including:

SPP Overlay Nodes: Will be connected through the Internet2 donated wave, using equipment from our project.

GMOC: Have adopted a proposal by the GMOC for using URNs to identify users and resources in GENI. Ongoing discussions about the

Million-Node GENI: Many design discussion about how the Seattle abstractions are not perfectly matched to our APIs, and we are working on refining some of our design to encompass Seattle-like resources.

Mid-Atlantic Network: Have worked out connectivity at 10GB between our equipment in the Washington, DC POP and the MAX equipment in the same POP. Will be used to connect to Cluster B projects (such as SPP) as well as Cluster C projects.

Great Plains Network: Are talking about possible connectivity to our equipment, so that they can connect to SPP, MAX, and Cluster C projects

GUSH: Have given them accounts on Emulab to experiment with using GUSH on Emulab

We have held talks with our campus and regional networks, who have been very supportive of our efforts. The expected outcome of these talks is dedicated connectivity between the Utah Emulab facility and the donated Interet2 wave. Our campus is also being supportive of the BGPMux project, agreeing to provide a BGP feed for experimenter use.

We participated in two GPO-sponsored workshops: one about the RSpec and connections between substrates, in Chicago, and another about measurement hosted in Madison.

Robert Ricci has been named incoming co-Chair of the Control Framework Working Group.

We have continued to work with the GPO and EDJ Associates to plan GEC6 in Salt Lake City.

F. Other Contributions

Last modified 13 years ago Last modified on 08/12/09 16:44:56