T4) GENI I&M Testing Environment

Platform Targets
Experiment and Measurement Orchestration Tools
Interfaces/Protocols Between Tools and Services

1) Goals

Provide a concise definition of the GENI environment for acceptance testing and operating GENI I&M tools.
Define a step-by-step process for the GEMINI and GIMI tools sets over Spirals 4, 5 and 6 such that:

They deliver useful tools in each spiral
They eventually provide tools that can be included in slices built with any type of GENI hosts
Included GENI hosts should be those provided by ExoGENI and InstaGENI racks, as soon as they are available

2) Tasks

Define the environment where the GENI I&M tools must operate and will be tested, including:

User tools and services for experiment and measurement, setup and orchestration
Interfaces/protocols/APIs between user tools and GENI aggregates
Included GENI aggregtaes, with enumerated topologies

3) Team

LEAD Harry Mussman (GPO)
Martin Swany (IU)
Jim Griffioen (U Kentucky)
Ilia Baldine, for ExoGENI racks (RENCI)
?, for InstaGENI racks (?)
David Irwin (UMass Amherst)
Mike Zink (UMass Amherst)
Christoph Dwertmann (NICTA)
Tom Mitchell (GPO)
Aaron Helsinger (GPO)
Vic Thomas (GPO)

4) Meetings

(organized calls or meetings before GEC13?)

Review conclusions in pre-meeting at GEC13

Review with working team at GEC13

5) Open Issues

6) Definition of GENI I&M Testing Environment

6.1) Spiral-by-Spiral Goals

Define a step-by-step process for the GEMINI and GIMI tools sets over Spirals 4, 5 and 6 such that:

They deliver useful tools in each spiral
They eventually provide tools that can be included in slices built with any type of GENI hosts
Included GENI hosts should be those provided by ExoGENI and InstaGENI racks, as soon as they are available

Start where they are now:

GEMINI is based on LAMP and INSTOOLS, that have been demonstrated/used on Emulab/protoGENI resources
GIMI is based on OML, that has operated in the OMF environment. However, a GIMI co-PI (Ilia Baldine) is also PI of the ORCA and ExoGENI projects

Goals for Spiral 4:

By the end of Spiral 4, GEMINI tools will test/operate in slices built in Emulab/protoGENI/InstaGENI environment
By the end of Spiral 4, GIMI tools will test/operate in slices built in ORCA/ExoGENI environment
As much as possible, the two groups will use common user tools and common interfaces/protocols/APIs
As much as possible, the two groups will avoid arrangements customized to one environment
Both groups will be ready to provide tutorials on their capabilities starting at GEC-14.
Not yet defined: support for PlanetLab/InstaGENI environment

Goals for Spiral 5:

By the end of Spiral 5, GEMINI tools will test/operate in slices built in both the Emulab/protoGENI/InstaGENI and ORCA/ExoGENI environments (but not in one slice)
By the end of Spiral 5, GIMI tools will test/operate in slices built in both the Emulab/protoGENI/InstaGENI and ORCA/ExoGENI environments (but not in one slice)

Goals for later:

Both GEMINI and GIMI tools will test/operate in slices built with slices that combine both Emulab/protoGENI/InstaGENI and ORCA/ExoGENI environments. However, this requires stitching functions that are not yet available.

6.2) Spiral 4 I&M Testing Environment

Interfaces/protocols/APIs between user tools and GENI aggregates

Must use GENI AM API v2 and GENI rspec v3, as the common way to acquire resources, load images/software and setup connectivity

User tools and services for experiment and measurement, setup and orchestration

Must use Gush with OMNI as the common tools to acquire resources, load images/software, setup connectivity and manage processes
May use OMF Experiment Controller (EC) to orchestrate experiment/application
These are to be used because they can be driven by scripts, which are essential to repeatable regression testing of I&M tools

Included GENI aggregates, with enumerated topologies

Emulab/protoGENI/InstaGENI, for servers or VMs, with VLANs between hosts, and IP to backbone/Internet
ORCA/ExoGENI, for servers or VMs, with VLANs between hosts, and IP to backbone/Internet
PlanetLab/InstaGENI, for VMs, with IP to backbone/Internet

6.4) Interfaces/protocols/APIs between User Tools and GENI Aggregates

Spiral 4 interfaces/protocols/APIs between user tools and GENI aggregates

Must use GENI AM API v2 and GENI rspec v3, as the common way to acquire resources, load images/software and setup connectivity

Spiral 5 interfaces/protocols/APIs between user tools and GENI aggregates

When will GENI AM API v3 be defined? what can be done with it? UpdateSliver?
When will GENI rspec v4 be defined? what can be done with it?
When will external stitching be possible?

6.5) User Tools and Services for Experiment and Measurement, Setup and Orchestration

Essential user tools and services for experiment and measurement, setup and orchestration

Must use Gush with OMNI as the common tools to acquire resources, load images/software, setup connectivity and manage processes
May use OMF Experiment Controller (EC) to orchestrate experiment/application
These are to be used because they can be driven by scripts, which are essential to repeatable regression testing of I&M tools

Have defined a Linux "User Environment" to hold various user tools; see topic T8)

User tools may have GUI and/or API interfaces
Environment provides a place to run scripts, e.g., to run regression tests of I&M tools
Environment provides a place to store keys/certificates/credentials, so that scripts can run without manual intervention

"User Environment" also holds other useful tools, that can also be tested in this environment

Portal to GUIs (GEMINI/INSTOOLS)
Directory Archive service (CNRI)
MDOD Creator/Editor service (CNRI and GPO)
MC/MAP services (GIMI)

6.6) Emulab/protoGENI/InstaGENI Aggregates

Emulab/protoGENI/InstaGENI, for servers or VMs, with VLANs between hosts, and IP to backbone/Internet

Utah Emulab/protoGENI cluster
I2 backbone protoGENI cluster (p/o Utah cluster)
Kentucky Emulab/protoGENI cluster
BBN Emulab/protoGENI cluster?
BBN InstaGENI rack, when available
Northwestern InstaGENI rack, when available

Node options supported?

Connectivity options supported?

6.7) Environment for GEMINI GEC-13 Demo

Tentative plan:

6.8) ORCA/ExoGENI Aggregates

ORCA/ExoGENI, for servers or VMs, with VLANs between hosts, and IP to backbone/Internet

RENCI ORCA cluster
Duke ORCA cluster
UNC ORCA cluster
Texas ORCA cluster
RENCI ExoGENI rack, when available
BBN ExoGENI rack, when available

Node options supported:

Servers, at some sites
VMs at all sites, using OpenStack, with NEuca
Many images, from "ImageProxy" web server

Connectivity options supported:

VLANs with aggregate (inta-site)
VLANs accross backbone (inter-site)
See "regression test suite for ORCA network code"

6.9) Environment for GIMI GEC-13 Demo

Tentative plan: