Changes between Initial Version and Version 1 of ShadowNet-Report-November-2011

Timestamp:

11/28/11 15:32:17 (12 years ago)

Author:

griff@netlab.uky.edu

Comment:

--

ShadowNet-Report-November-2011

@@ +1 @@
+[[PageOutline]]
+
+= GENI Shadownet Project Status Report = 
+
+Period: Post GEC 12 Report
+
+== I. Major accomplishments ==
+
+The following highlights our accomplishments
+during the last reporting period.
+
+=== A. Milestones achieved ===
+ * During the last reporting period we made significant progress toward 
+   (milestone S4.a):  "Continue to develop and integrate the Shadownet
+   virtualization control software with the measurement and instrumentation
+   toolset INSTOOLS."
+
+ * We also vastly improved our integration of the Shadownet control software
+   with the ProtoGENI control framework (i.e., we made significant enhancements
+   to our previously accomplished milestone S3.b).
+
+=== B. Deliverables made ===
+
+ * Routers are now available for allocation and can be included in 
+   users' experiments through the ProtoGENI Flack interface. 
+
+== II. Description of work performed during last quarter ==
+
+The following provides a description of the progress made during the last
+reporting period.
+
+
+=== A. Activities and findings ===
+
+
+Although we had shown interoperability between the Shadownet component
+manager (CM) and the ProtoGENI control framework by the time of our previous
+report, the Shadownet CM only offered the most basic services.  
+One of the goals of our work over the last reporting period has been to do an
+even better job of integrating the Shadownet component manger (CM) with the
+ProtoGENI control framework in order to enable additional features and to
+better control the Shadownet resources from the Flack graphical user
+interface.
+As part of this, we worked closely with our colleagues from the ProtoGENI group
+to add the information that Flack needed in order to be Shadownet-aware.
+In particular, we modified the Shadownet CM to include information about
+available logical routers in the advertisement RSPEC so that Flack can know
+how many logical routers are available on a physical router.  We also came up
+with a way to create links between Juniper routers using Flack.  To do this,
+Flack treats logical Juniper routers as special nodes based on a modified
+version of the code use to support Planetlab nodes.  It allows users to
+create links between these logical Juniper routers even when no interfaces
+are defined in the advertisement, while links between normal nodes can be
+created in Flack only if interfaces for them are defined in the advertisement
+RSPEC.
+On the Shadownet CM side, we modified the Shadownet CM to report back
+to the Slice Authority with the manifest after it creates a slice
+successfully so that Flack is able to determine when the logical router setup
+is complete.  Because logical router setup and initialization can take a long
+time, we had to adjust the apache server's timeout to prevent XLMRPC calls
+from failing, but we are exploring other ways to both speed up the creation
+process and to more accurately report the creation status to Flack.
+
+The major focus of our work this past period has been to integrate the
+Shadownet CM with the INSTOOLS measurement and instrumentation tools.
+We found that this was a non-trivial task for a number of reasons.
+
+First, Juniper logical routers differ in many ways from the "PC router"
+resources available at the UK and Utah aggregates.  For example, INSTOOLS
+relies on each (PC) router running an SNMP daemon to collect network
+statistics.  However, Juniper logical routers do not have their own SNMP
+servers.
+Instead, there is one SNMP server for the entire physical Juniper router.
+The physical router's SNMP server records all the information from all the
+logical routers.  In order to obtain the information for a specific logical
+router, the SNMP request to the physical SNMP server must be formulated in a
+particular way.  Moreover, all information from all logical routers is
+available to any client that has access to the physical SNMP server.
+To ensure that a slice's MC only accesses the data for the
+logical routers that are part of the slice, we implemented a proxy server
+that checks permissions and then translates the request for logical router
+information into the appropriate calls to the physcial SNMP daemon.
+To ensure MC's cannot access the physical SNMP server, we configure the
+physical SNMP server so that it can only be accessed by our known set of
+proxies. When
+the MC of a slice wants to collect data from logical routers, it sends the
+requests to the proxy.  Whenever the proxy gets an SNMP request, it checks a
+local database to determine whether the logical router is ready and whether
+the IP address for the MC node is permitted to access the routers. Only those
+authorized MCs will be able to get the SNMP request through the proxy to
+collect measurement information about the routers.
+
+A second challenge was the fact that the Shadownet aggregate manager only has
+logical router resources to offer and no general purpose PCs.  Consequently,
+there are no general purpose nodes in the Shadownet aggregate that can serve
+as MCs.  The INSTOOLS software is designed around the concept of a local MC;
+a general purpose node allocated from the local aggregate.  Each aggregate
+in the slice is given its own MC to localize data collection.  Because
+Shadownet has no nodes to offer as MCs, we modified the
+INSTOOLS Service API calls at the Shadownet CM to
+"borrow" nodes from another aggregate (the ukgeni.cm) to use for its MCs.
+Although the MC is running in a different aggregate, it believes it is part
+of the Shadownet aggregate and is configured to monitor the logical routers
+in the Shadownet aggregate.  Because the Shadownet CM looks and feels like
+other CMs (i.e., it pretends it has MC nodes to allocate just like other
+CMs), the normal ProtoGENI API calls can be used by INSTOOLS to create and
+initialize the MC.
+
+In order to accommodate logical routers in ProtoGENI/INSTOOLS, we created a new
+attribute value to define a Juniper logical juniper in RSPECs and Manifests:
+(virtualization_type="juniper-lrouter").
+Both the MC's drupal content management system and the portal in INSTOOLS have
+been modified to recognize these special node types as logical
+routers. Correspondingly, the login method to the node will use the logical
+router name instead of the user name. In addition, some normal PC
+measurements (such as CPU load, memory usage) are disabled because they are not
+well supported on the logical routers.
+
+We demonstrated the new Shadownet CM at the GEC 12 conference.  The demo
+showed how logical routers in the backbone of Internet 2 can be allocated and
+linked together using VLANs and then showed how the INSTOOLs software can
+provide live views of the traffic for each logical router.
+
+During this last reporting period, we continued to support and operate
+the Juniper routers in the Internet 2 backbone, and have been using the
+routers in our testing and evaluation of the new Shadownet CM.
+
+Finally, we have continued to have discussions with the PerfSONAR team to
+better understand how we can more tightly integrate the data collection from
+our logical routers with PerfSONAR.
+
+=== B. Project participants ===
+
+The following individuals are involved with the project in one way or another:
+ * Jim Griffioen - Project PI (Kentucky)
+ * Zongming Fei - Project Co-PI (Kentucky)
+ * Kobus van der Merwe - Project Co-PI (AT\&T)
+ * Eric Boyd - Subcontract Lead (Internet2)
+ * Brian Cashman - Network Planning Manager (Internet2)
+ * Lowell Pike - Network administrator (Kentucky)
+ * Hussamuddin Nasir - Technician/Programmer (Kentucky)
+ * Charles Carpenter - Researcher/Programmer (Kentucky)
+ * Emmanouil Mavrogiorgis - Research Staff (AT\&T)
+
+=== C. Publications (individual and organizational) ===
+
+=== D. Outreach activities ===
+
+We gave a Shadownet Demo at GEC 12 showing how logical routers can be
+allocated and initialized via the ProtoGENI Flack GUI.
+
+=== E. Collaborations ===
+
+Most of our collaborations have been with the Shadownet team.  In particular,
+it involves participants from Kentucky,
+AT\&T, and Internet2, but we have also had several conversations
+with our ProtoGENI and perfSONAR colleagues.
+
+=== F. Other Contributions ===
+