wiki:ShadowNet-Report-November-2011

Version 1 (modified by griff@netlab.uky.edu, 12 years ago) (diff)

--

GENI Shadownet Project Status Report

Period: Post GEC 12 Report

I. Major accomplishments

The following highlights our accomplishments during the last reporting period.

A. Milestones achieved

  • During the last reporting period we made significant progress toward (milestone S4.a): "Continue to develop and integrate the Shadownet virtualization control software with the measurement and instrumentation toolset INSTOOLS."
  • We also vastly improved our integration of the Shadownet control software with the ProtoGENI control framework (i.e., we made significant enhancements to our previously accomplished milestone S3.b).

B. Deliverables made

  • Routers are now available for allocation and can be included in users' experiments through the ProtoGENI Flack interface.

II. Description of work performed during last quarter

The following provides a description of the progress made during the last reporting period.

A. Activities and findings

Although we had shown interoperability between the Shadownet component manager (CM) and the ProtoGENI control framework by the time of our previous report, the Shadownet CM only offered the most basic services. One of the goals of our work over the last reporting period has been to do an even better job of integrating the Shadownet component manger (CM) with the ProtoGENI control framework in order to enable additional features and to better control the Shadownet resources from the Flack graphical user interface. As part of this, we worked closely with our colleagues from the ProtoGENI group to add the information that Flack needed in order to be Shadownet-aware. In particular, we modified the Shadownet CM to include information about available logical routers in the advertisement RSPEC so that Flack can know how many logical routers are available on a physical router. We also came up with a way to create links between Juniper routers using Flack. To do this, Flack treats logical Juniper routers as special nodes based on a modified version of the code use to support Planetlab nodes. It allows users to create links between these logical Juniper routers even when no interfaces are defined in the advertisement, while links between normal nodes can be created in Flack only if interfaces for them are defined in the advertisement RSPEC. On the Shadownet CM side, we modified the Shadownet CM to report back to the Slice Authority with the manifest after it creates a slice successfully so that Flack is able to determine when the logical router setup is complete. Because logical router setup and initialization can take a long time, we had to adjust the apache server's timeout to prevent XLMRPC calls from failing, but we are exploring other ways to both speed up the creation process and to more accurately report the creation status to Flack.

The major focus of our work this past period has been to integrate the Shadownet CM with the INSTOOLS measurement and instrumentation tools. We found that this was a non-trivial task for a number of reasons.

First, Juniper logical routers differ in many ways from the "PC router" resources available at the UK and Utah aggregates. For example, INSTOOLS relies on each (PC) router running an SNMP daemon to collect network statistics. However, Juniper logical routers do not have their own SNMP servers. Instead, there is one SNMP server for the entire physical Juniper router. The physical router's SNMP server records all the information from all the logical routers. In order to obtain the information for a specific logical router, the SNMP request to the physical SNMP server must be formulated in a particular way. Moreover, all information from all logical routers is available to any client that has access to the physical SNMP server. To ensure that a slice's MC only accesses the data for the logical routers that are part of the slice, we implemented a proxy server that checks permissions and then translates the request for logical router information into the appropriate calls to the physcial SNMP daemon. To ensure MC's cannot access the physical SNMP server, we configure the physical SNMP server so that it can only be accessed by our known set of proxies. When the MC of a slice wants to collect data from logical routers, it sends the requests to the proxy. Whenever the proxy gets an SNMP request, it checks a local database to determine whether the logical router is ready and whether the IP address for the MC node is permitted to access the routers. Only those authorized MCs will be able to get the SNMP request through the proxy to collect measurement information about the routers.

A second challenge was the fact that the Shadownet aggregate manager only has logical router resources to offer and no general purpose PCs. Consequently, there are no general purpose nodes in the Shadownet aggregate that can serve as MCs. The INSTOOLS software is designed around the concept of a local MC; a general purpose node allocated from the local aggregate. Each aggregate in the slice is given its own MC to localize data collection. Because Shadownet has no nodes to offer as MCs, we modified the INSTOOLS Service API calls at the Shadownet CM to "borrow" nodes from another aggregate (the ukgeni.cm) to use for its MCs. Although the MC is running in a different aggregate, it believes it is part of the Shadownet aggregate and is configured to monitor the logical routers in the Shadownet aggregate. Because the Shadownet CM looks and feels like other CMs (i.e., it pretends it has MC nodes to allocate just like other CMs), the normal ProtoGENI API calls can be used by INSTOOLS to create and initialize the MC.

In order to accommodate logical routers in ProtoGENI/INSTOOLS, we created a new attribute value to define a Juniper logical juniper in RSPECs and Manifests: (virtualization_type="juniper-lrouter"). Both the MC's drupal content management system and the portal in INSTOOLS have been modified to recognize these special node types as logical routers. Correspondingly, the login method to the node will use the logical router name instead of the user name. In addition, some normal PC measurements (such as CPU load, memory usage) are disabled because they are not well supported on the logical routers.

We demonstrated the new Shadownet CM at the GEC 12 conference. The demo showed how logical routers in the backbone of Internet 2 can be allocated and linked together using VLANs and then showed how the INSTOOLs software can provide live views of the traffic for each logical router.

During this last reporting period, we continued to support and operate the Juniper routers in the Internet 2 backbone, and have been using the routers in our testing and evaluation of the new Shadownet CM.

Finally, we have continued to have discussions with the PerfSONAR team to better understand how we can more tightly integrate the data collection from our logical routers with PerfSONAR.

B. Project participants

The following individuals are involved with the project in one way or another:

  • Jim Griffioen - Project PI (Kentucky)
  • Zongming Fei - Project Co-PI (Kentucky)
  • Kobus van der Merwe - Project Co-PI (AT\&T)
  • Eric Boyd - Subcontract Lead (Internet2)
  • Brian Cashman - Network Planning Manager (Internet2)
  • Lowell Pike - Network administrator (Kentucky)
  • Hussamuddin Nasir - Technician/Programmer (Kentucky)
  • Charles Carpenter - Researcher/Programmer (Kentucky)
  • Emmanouil Mavrogiorgis - Research Staff (AT\&T)

C. Publications (individual and organizational)

D. Outreach activities

We gave a Shadownet Demo at GEC 12 showing how logical routers can be allocated and initialized via the ProtoGENI Flack GUI.

E. Collaborations

Most of our collaborations have been with the Shadownet team. In particular, it involves participants from Kentucky, AT\&T, and Internet2, but we have also had several conversations with our ProtoGENI and perfSONAR colleagues.

F. Other Contributions