GENI Desktop Project Status Report

Period: Solicitation 4 Year 2

I. Major accomplishments

The following highlights our accomplishments during the last year.

A. Milestones achieved

• Modified the GENI Desktop to support "Speaks-for" authentication being developed/supported by the control frameworks and other GENI tools/services.

• Incorporated user-driven feedback into the GENI Desktop to support new user-requested services and features.

• Developed new training materials that incorporate the changes made to the GENI Desktop.

• Integrated and leveraged existing tools and services (e.g., Jacks) into the GENI Desktop for managing topologies, experiments, and results.

• Collected user feedback regarding usability of the GENI Desktop and made major changes to improve its ease-of-use and aesthetics.

• Adapted the GENI Desktop archiving service for storing and retrieving experiment results and artifacts from iRoDs to support a new archival service with enhanced features.

• Enhanced the set of scriptable resource management, instrumentation, and monitoring available to experimenters and other tools.

• Enabled integration of the GENI Desktop with other experimenter tools.

• Created documentation and tutorial materials to reflect this latest version of the GENI Desktop.

B. Deliverables made

• We enhanced the GENI Desktop to use "Speaks-for" credentials for accessing resources from other GENI components on behalf of users.

• We implemented an initial version of the slice verification and configuration testing service.

• We developed code for super slice support in the GENI Desktop.

• We developed a completely new user interface for the GENI Desktop which we call "GENI Desktop Lite" that greatly improves the look-and-feel and ease-of-use of the GENI Desktop.

• We integrated the Jacks tool into the GENI Desktop, enabling users to create topologies and instantiate experiments (i.e., slices) using the Jacks tool. We also integrated the Adopt-A-GENI (AAG) tool into the GENI desktop.

• We developed and integrated a new archival service into the GENI Desktop that leverages VMs to hold, and later display, the archived experiment state in the same context as it was initially collected and viewed.

• We designed and implemented a GENI Desktop Command Line Interface (gdcli) that enables users to write scripts that control, manage, and measure the performance of their slices through the GENI Desktop.

• We demonstrated how the new gdcli can be used by other experimenter tools to integrate with the GENI Desktop.

• We developed online documentation for the new gdcli interface and gave a tutorial entitled "Monitoring and Controlling Experiments with GENI Desktop Scripts and Modules" at the GEC 23 conference.

II. Description of work performed during the last year

The following provides a description of the progress made during the last year.

A. Activities and findings

Our activities this past year have resulted in enhanced functionality, improved ease-of-use, better authentication/security, a redesigned user interface, and the ability to control the GENI Desktop programmatically throught scripts. In particular, we:

• Incorporated support for "Speaks-for" into the GENI Desktop

• Designed and implemented a slice verification service

• Designed and implemented a super slice abstraction

• Developed a new archival services based on Xen VMs to restore entire contexts

• Redesigned the look-and-feel of the GENI Desktop user interface to make it much easier to use

• Integrated Jacks and Adopt-A-GENI (AAG) tools into the GENI Desktop

• Designed and implemented a GENI Desktop Command Line Interface (gdcli) to enable script-based access to GENI Desktop functionality.

The following briefly describes our activities this past year. We beginning with our efforts to improve the authentication/authorization used to access the GENI Desktop, and then move on to describe new functionality added to the GENI Desktop, followed by a description of our efforts to enhance the look-and-feel of the GENI Desktop, and lastly our efforts to allow scripting of the GENI Desktop.

Authorization: Supporting "Speaks-for"

The "Speaks-for" credential allows trusted tools to act for, instead of acting as, an experimenter to perform certain actions, such as requesting resources from aggregates, accessing allocated resources, and installing software on experimental nodes. We enhanced the GENI Desktop to use "Speaks-for" credentials for accessing resources on behalf of users. Users no longer need to provide the private key to the GENI Desktop. We implemented an interface for the user to authorize the GENI Desktop to speak for her/him. A GENI Desktop-specific certificate is signed using the private key of the user. Because the whole process happens within the browser on the client side, the private key never leaves the user's machine. The "Speaks-for" credential allows the GENI Desktop to talk to aggregates and perform all necessary actions on behalf of the user.

New Functionality (1): Jacks, Archival, Verification and Super Slice Services

Another goal this past year has been to begin integrating support for other tools into the GENI Desktop. To demonstrate the ability to support other tools, we began by integrating the Jacks tool into the GENI Desktop. Users can now add resources to their slices by selecting Jacks in the GENI Desktop which will direct them to a GENI Desktop page that embeds the Jacks tool and allows them to allocate the resources (i.e., which uses the OMNI tool). RSPECs created by Jacks can be saved by the GENI Desktop for future use.

In addition, we integrated the Adopt-A-GENI (AAG) flow specification module into the GENI Desktop, allowing users to visually define OpenFlow paths across the topology that are then sent to the AAG module to be instantiated in the OpenFlow controller. Although the AAG functionality is logically a distinct service/tool, the messaging system between windows in the GENI Desktop made it possible to incorporate this new tool with relatively little effort. In addition, we were able to add a new AAG Controller node type to the Jacks wrapper, thereby integrating the AAG controller into the Jacks tool as well.

The existing archival service in the GENI Desktop leveraged the iRoDs storage service to store and later retrieve measurement data collected by the GENI Desktop. A key limitation of this service was the inability to easily (and quickly) access, view, and make sense of archived measurement data. To address this need, we developed a new archival service that not only archives the measurement data, but also archives the software and context used to display the data. Because the data and the environment needed to view the data are archived, users can quickly access an archive and view the saved data using the same tools available at the time the data was collected.

To support this new archival service, we implemented an archival server that not only captures the measurement data stored on the global node (where measurement data is collected), but it also captures the state of the drupal system used to display the data, including all web server (apache) and database (mysql) files. GENI Desktop users can request that an archive be made, which is then sent to the archive server. When a user visits the archive web page on the archive server, they can select from any of the archived snapshots. The archive server will dynamically launch a Xen VM, setup the apache, mysql, and Drupal state needed to view the measurement data, install the archived measurement data, create login credentials for the user, and share the credentials with the GD so the user is automatically logged into the archive VM. The result is that the user is presented with the same look-and-feel as if they had gone to the global node at the time the snapshot was taken.

We implemented the slice verification and configuration testing service as a module in GENI Desktop by taking advantage of the module builder function of the GENI Desktop. Based on the manifest of an experiment, the verification service analyzes the topology and performs tests about the interfaces of all nodes in the experiment. The initial version we implemented checks whether each interface is up and whether it is reachable from a ping test. The results are presented in a table showing the status of all the interfaces of all the nodes in the experiment. Later versions of the verification service included additional checks (particularly automated bandwidth checks) and also made it possible for users to write their own verification scripts to test for things of importance to their experiment.

Building a large experiment is a difficult task in GENI, partly because it is more likely to fail if we create an experiment with a lot of nodes. At the same time, we may have multiple related experiments and want to combine these relatively small experiments together to form a large experiment. We developed a new "super slice" service in the GENI Desktop to support this functionality. Users can use the GENI Desktop to create a super slice by combining multiple existing slices together. The GENI Desktop provides a GUI for users to display multiple slices at the same time and pick any pair of nodes from different slices to establish a link between them. The Super Slice service in the GENI Desktop currently can then automatically set up GRE tunnels between these selected pairs of nodes from different slices.

A New User Interface: GENI Desktop Lite

Over the years the number of features and capabilities offered by the GENI Desktop has continued to expand. Indeed, a key goal of the GENI Desktop was to provide users with a context for managing all aspects of their experiment from setup and deployment to monitoring and archiving of measurements and results. The downside to this expanded functionality is increased complexity using the tool. At the same time, the number of experimenters who are using the GENI Desktop to create, manage, monitor, and control their slices has been grown rapidly, due, in part, to users being exposed to the GENI Desktop as part of GENI tutorials, summer camps, demontrations and online documents and videos. Feedback from this user group indicated that the extensive functionality available in GENI Desktop made it difficult for new users to navigate and use.

To address this need for a tool that could be easily learned and used by new users, we completely redesigned the look-and-feel of the GENI Desktop to reduce complexity and make it simple to create, run, and monitor experiments. Our new "GENI Desktop (GD) Lite" interface is now the default intereface that users see when they log into the GENI Desktop. Users can still access the (original) advanced user interface if needed, but in most cases find that the GD Lite interface is sufficient. The Lite intereface is designed to take users through the lifecycle of an experiment. The Lite intereface starts by helping users create a slice, assigning resources to the slice, and then giving them access to a simplified version of the GENI Desktop topology view where they can log in to nodes, run their experiment, monitor basic traffic types, and archive results. Initial feedback on the new intereface has been extremely positive.

In addition to a major rewrite of the web code for the user interface, one of the key challenges that we had to address was automating the global node setup, initialization, and instrumentation. While these "backend" operations were clearly visible in the old user interface, they had to be hidden in the new interface. This meant that the GENI Desktop had to be able to add global nodes into the slice (one for each aggregate) on the user's behalf. This required working with the aggregates to support the GENI AM API calls needed to add resources to an existing slice. In addition, the GENI Desktop needed to be able to initialize and then instrumentize the slice in the background (i.e., while allowing the user to view and use the slice in the GENI Desktop). This required changes to the GENI Desktop to monitor the background initialization/instrumentation process and incrementally enable functionality as it became available. For example, while resources are being allocated the GENI Desktop can only display the topology and the status of the node initialization. As soon as the initilization completes, the file upload, ssh, and run command functionality become available in the user interface. Later when the instrumentation completes, functionality such as displaying basic traffic graphs or archiving measurement data become available in the user interface. In short, users are now taken directly to the GENI Desktop topology view, bypassing several setup steps required by the old user interface. Commonly needed functionality is then automatically added as it becomes available. As part of the new Lite interface, we also simplified the design of the web page(s) used to select a predefined RSPEC.

Programming the Desktop: The GENI Desktop CLI (gdcli)

The GENI Desktop greatly simplifies the task of instrumenting and monitoring a users' experiment (slice). However, users could only access the GENI Desktop via a web interface. In other words, there was not programmatic way for experimenters or tool developers to leverage the GENI Desktop functionality.

To address this need we designed a new interface to the GENI Desktop that could be used to programatically upload files, run commands, download measurement graphs, etc --- functions previously only possible via the GENI Desktop web interface. In particular, we developed an application that runs on Linux (or other Unix-based systems), Mac, and Window called the gdcli program that can be used to interact with the GENI Desktop. The gdcli program can be used to:

• Upload files to a select set of nodes
• Run a command on a select set of nodes
• Download a traffic measurement graph (as PNG or CSV) from a select set of nodes
• Download a normal file from a select set of nodes
• Get a list of slices
• Check the status of a slice
• Get the topology of a slice
• Validate the setup of a slice
• List the nodes in a slice
• List the links in a slice

The gdcli program can be called from any scripting language (e.g., python, perl, sh (bash), .BAT files, etc). As a result, users are able to write programs in their favorite scripting language that make calls to the GENI Desktop to upload/download files, download measurement graphs, run commands, etc.

There were several challenges that we had to address while implementing the gdcli scripting interface. First, we needed a way to make calls to the GENI Desktop server (e.g., to download a traffic graph, or run a command). To solve this problem we enhanced the GENI Desktop server to support HTTP posts that included parameters to the request specifying, for example, the list of traffic graphs to be downloaded (i.e., the nodes/links names and the types of graphs desired). We implemented a python backend server specifically designed to process the request, perform the action, and return the results. The python backend shares access with the previous GENI Desktop PHP code to the databases and files used by the GENI Desktop, thereby ensuring that the results returned by the gdcli are the same information as would be seen in the GENI Desktop web interface.

A second challenge was securing the access to, and communication with, the new python backend server. To ensure communication is secure, all communication occurs over a secure connection using https. The problem of authorization requires not only that the user authenticate themselves to the server, but that the server obtain a "speaks-for" certificate to act on behalf of the user. Because the existing speaks-for generation tools are designed for interactive web use, not scripting, we decided to require that users first authorize a speaks-for using the existing GENI Desktop web interface which can then be stored and used by the GENI Desktop (and our new backend server) until the speaks-for expires. However, this does not solve the authorization problem. To ensure the users has the right to issue commands to our python backend server, the web interface of the GENI Desktop also creates a secret key (say at the same time the user authorizes the speaks for) that the user must store on their local machine. The secret key is used when communicating with the python backend server to prove that the user has the right to invoke the requested operations on the GENI Desktop. In that sense, users can think of the gdcli secret key like an ssh key that must be present on their local machine in order to access the service.

A third issue involved handling the results/output of a gdcli request. The gdcli tool provide two mechanisms for handling the output from a request. The first, and most simple mechanism, concatenates all the output files/graphs and prints them to standard output, allowing users to redirect output to other programs or tools. The second way gdcli handles output is to deposit each graph, downloaded file, or output from a run command into a different file on the local machine. Files are automatically assigned names that describe their content (based on the slice, the aggregate, the node or link, and the type of graph). Because the naming convention is known to experimenters, they can easily write scripts that know what filenames to look for, and then feed those files to the appropriate program for processing (e.g., copying traffic graphs into a web directory to create a user-defined traffic mashup view).

B. Project participants

The following individuals are involved with the project in one way or another:

• Jim Griffioen - Project PI
• Zongming Fei - Project Co-PI
• Hussamuddin Nasir - Technician/Programmer
• Charles Carpenter - Technician/Programmer
• Xiongqi Wu - Ph.D. Student
• Jeremy Reed - Ph.D. Student

C. Publications (individual and organizational)

D. Outreach activities

• We gave a presentation about the GENI Desktop and its features during the Introduction to GENI Instrumentation & Measurement Tools portion of the Getting Started with GENI tutorial at GEC 22 and GEC 23.

• We gave a demo of the latest GENI Desktop features during the demo session at GEC 21, GEC 22, and GEC 23.

• We gave a tutorial entitled "Monitoring and Controlling Experiments with GENI Desktop Scripts and Modules" at the GEC 23 conference.

• We developed and posted online-documentation and online-tutorials that describe the new features of the GENI Desktop for users.

E. Collaborations

• Most of our collaborations have been between the GPO Portal team and the aggregate teams at Utah and RENCI.

F. Other Contributions