Changes between Version 18 and Version 19 of TIEDClearinghouse


Ignore:
Timestamp:
07/02/09 13:07:06 (15 years ago)
Author:
faber@isi.edu
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • TIEDClearinghouse

    v18 v19  
    11= The TIED Clearinghouse =
    22
    3 The TIED clearinghouse presents two primary services to users:
     3The TIED clearinghouse presents four primary services to experimenters:
    44
    55 * Experimenter Registration
    … …  
    88 * Experimental tools
    99
    10 This page outlines those services and how an experimenter can make use of them.  Most of these services are provided to one extent or another by DETER's [http://fedd.isi.deterlab.net fedd] software.  We [wiki:TIEDFeddMapping describe] the mapping to fedd as well.
     10This page outlines those services and how an experimenter can make use of them.  Most of these services are provided to one extent or another by DETER's [http://fedd.isi.deterlab.net fedd] software and the DETER testbed.  We describe fedd's implementation more technically [wiki:TIEDFeddMapping elsewhere].
     11
     12We summarize these services here before describing in detail how to access them.
    1113
    1214== Experimenter Registration ==
    … …  
    1416TIED currently makes access control decisions based on both the identity of a user and on the attributes that TIED knows about the user.  The user's identity is encoded in a [http://fedd.isi.deterlab.net/trac/wiki/FeddAbout#GlobalIdentifiers:Fedids fedid], the TIED version of a GENI ID.  A [http://fedd.isi.deterlab.net/trac/wiki/FeddConfig#MakingaFedidCertificate fedid can be created by any user using standard SSL commands], but the advantage of using the TIED experimenter registry is that it binds additional information to that identity.  Specifically, this creates a [http://fedd.isi.deterlab.net/trac/wiki/FeddAbout#GlobalIdentifiers:Three-levelNames three-level name] for the experimenter.  This allows the TIED federation daemon to negotiate different levels of access with the aggregates with which the TIED clearinghouse federates.
    1517
     18A registered TIED user has access to the various aggregates TIED works with and can create slices across them.
     19
    1620== Slice Management ==
    1721
    18 For TIED, slice management is the creation and manipulation of experiments that span multiple facilities.  One can think of the facilities as aggregates, each of whaich controls access to a set of resources.  Currently TIED supports creation of slices across testbeds with an Emulab interface and that run instances of [http://fedd.isi.deterlab.net fedd] that act as aggregate managers.  In addition ti the experimenter interface to slices, TIED provides interfaces that allow experimental tools to be used in the slice.
     22For TIED, slice management is the creation and manipulation of experiments that span multiple facilities.  The facilities are aggregates, each of whaich controls access to a set of resources.  Currently TIED supports creation of slices across testbeds with an Emulab interface and that run instances of [http://fedd.isi.deterlab.net fedd] that act as aggregate managers.  In addition to the experimenter/slice interface, TIED provides interfaces that allow experiment management tools to be used in the slice.
     23
     24The TIED slice manager creates slices from an abstract [http://fedd.isi.deterlab.net/trac/wiki/FeddAbout#ExperimentDescriptions "nodes and networks" description] using multiple facilities, guided by the user.
    1925
    2026== Component/Aggregate Management ==
    … …  
    2228The TIED clearinghouse manages the access control of multiple sites as aggregates.  It provides a unified access control model across aggregates and provides a unified interface for alloacting resources from them.  Currently Emulab-based testbeds are used as aggregates, but development is underway to access other systems through the same interfaces.
    2329
     30Most experimenters will not need to access this facility directly, as the high level requests for slice creation manage all this behind the scenes, but it is worth noting that it is happening.
     31
    2432== Experiment Tools ==
    2533
    26 Most tools that use the Emulab testbed interfaces can be seamlessly used on TIED slices.  In particular the DETER [http://seer.isi.deterlab.net SEER] tool can be used to transparently manipulate experiments inside TIED slices.
     34Most tools that use the Emulab testbed interfaces can be seamlessly used on TIED slices.  In particular the DETER [http://seer.isi.deterlab.net SEER] tool can be used to transparently manipulate experiments inside TIED slices.  SEER is an extensible, agent-based experiment management system that enables node configuration, traffic generation, and data gathering.  There are many generic services available to users and a documented extension model.
    2735
    2836= Using the TIED Clearinghouse =
    2937
    30 This brief tutorial will show how to register with TIED, create a slice, connect to it using SEER, and finally to terminate the slice.
     38This brief tutorial will show how to
     39
     40 * Register with TIED
     41 * Create, manipulate and terminate a slice using both a web interface and a command-line interface
     42 * Connect to the slice and manipulate it using SEER
     43
     44Throughout the tutorial are links to more detailed documentation.  Questions or comments are [mailto:faber@isi.edu welcome].
    3145
    3246== Registering with TIED ==
    … …  
    3650Joining the TIED project on DETER is just like joining a project on any Emulab-based testbed.  If you are a DETER user, just log in and choose "Join an Existing Project" from the Experimentation menu.  If not the detailed instructions follow.
    3751
    38 Connect to http://www.isi.deterlab.net and click on the "Request Account" button and from the next screen the "Join an Existing Project" link.  Fill out the form as instructed and be sure to enter "TIED" in the "Project Name" field.  Shortly the TIED staff will approve you as a TIED member and also add appropriate configuration to the slice management portion of fedd to allow you to create slices.  You will receive two e-mail messages, one stating you've been approved in the TIED group and one indicating that you've been granted the authority to create slices.  Once you have received both messages, you can create slices.
     52Connect to http://www.isi.deterlab.net and click on the "Request Account" button.  From the next screen the "Join an Existing Project" link.  Fill out the form as instructed and be sure to enter "TIED" in the "Project Name" field.  Shortly the TIED staff will approve you as a TIED member and also add appropriate configuration to the slice management portion of fedd to allow you to create slices.  You will receive two e-mail messages, one stating you've been approved in the TIED group and one indicating that you've been granted the authority to create slices.  Once you have received both messages, you are ready to create slices.
    3953
    4054[[Image(screen1.png)]]  [[Image(screen2.png)]]
    … …  
    4862=== Using the Web Interface ===
    4963
    50 TIED keeps a record of slices and allows users to recreate them or terminate running ones through a web interface.  The current list is available from https://www.isi.deterlab.net/fedd_list.php3 (you must be logged in using your TIED account to view this page).  An example is below.
     64TIED keeps a record of slices created through the web interface and allows users to recreate them or terminate running ones through a web interface.  The current list is available from https://www.isi.deterlab.net/fedd_list.php3 (you must be logged in using your DETER/TIED account to view this page).  An example is below.
    5165
    5266[[Image(screen3.png)]]
    5367
    54 The image is of the logical slice topology, it does not indicate which aggregates donated which nodes.  The master testbed column indicates which testbed is exporting its user environment into the experiment.  For TIED slices, this will usually be the DETER testbed.  The project column will be TIED for TIED experimenters; it indicates that the DETER TIED project is being exported into the slice.  The name of the slice is a mnemonic name attached to the slice.  It also has a [http://fedd.isi.deterlab.net/trac/wiki/FeddAbout#GlobalIdentifiers:Fedids fedid] this is not visible from this interface.  Finally there are buttons to create or terminate this slice, active based on the current status of the slice.
     68The image shows the logical topology of the slice. It does not include extra nodes in the slice used to induce network delay or loss, or extra nodes used to connect nodes between aggregates.  The master testbed column indicates which testbed is exporting its user environment into the experiment; for TIED slices, this will usually be the DETER testbed.  The project column will be TIED for TIED experimenters; it indicates that the DETER TIED project is being exported into the slice.  The name of the slice is a mnemonic name attached to the slice; the slice also has a [http://fedd.isi.deterlab.net/trac/wiki/FeddAbout#GlobalIdentifiers:Fedids fedid] that is not visible from this interface.  There is also a button to retrieve the slice description used to create this slice, either to inspect it or to copy it for extension.  Finally there is an indication of the slice's status ('''active''', '''inactive''', '''swapping''', or '''unknown''').  Based on the current status there are buttons to create or terminate it.  Termination removes the nodes from active use,but the description remains in this database, so a user may recreate it at the touch of a button.
    5569
    5670==== Creation ====
    … …  
    6478Fedd's documentation on [http://fedd.isi.deterlab.net/trac/wiki/FeddAbout#Experiments experiment creation] provides detail on how experiments are created from the description.  In reading that document from a GENI perspective, it helps to think of experiments as slices and testbeds as aggregates (the aggregate interface is provided by fedd).
    6579
     80The current web interface does not provide a progress report on the creation process, which is a known shortcoming.  TO monitor the creation, use the [https://www.isi.deterlab.net/fedd_list.php3 the slice listing page].  The status will go from '''swapping''' to '''active''' as the resources are allocated and configured from aggregates.  Error messages are also sparse on  the web interface, but the command line interface provides more information in the event of failure.
     81
     82Creation is a matter of allocating and configuring resources from many aggregates.  If the aggregates are loaded, this can be a lengthy process, on the order of minutes.  This is not so much a TIED limitation - TIED parallellizes the process aggressively - but simply a side effect of making potentially complex allocations from various testbeds.  Allow minutes of time at least.
     83
     84Persistent creation problems should be brought to the attention of [mailto:faber@isi.edu the fedd maintainer].
     85
    6686==== Termination ====
    6787
    6888The simplest way to return a slice's resources is to use the terminate button from [https://www.isi.deterlab.net/fedd_list.php3 the slice listing page], which is active when a slice is active.  Only the experimenter who created the slice can delete it from the web interface, though the command line interface is more versatile.  Alternatively, one can terminate a slice from the [https://www.isi.deterlab.net/fedd_terminate.php3 termination page], where one must type the slice name by hand.
     89
     90Termination releases the resources from each aggregate back to that aggregate to be reassigned.  The configuration and name are retained in the web interface so that the slice can be easily recreated.
    6991
    7092==== Slice Status ====
    … …  
    7698Creating, terminating and accessing information about the slice can be done through the [http://fedd.isi.deterlab.net/trac/wiki/FeddDevelop published fedd interfaces] accessible through the command line using the [http://fedd.isi.deterlab.net/trac/wiki/FeddCommands#Fedd_client.py fedd_client.py program].
    7799
    78 Replicating the detailed information at [http://fedd.isi.deterlab.net/trac/wiki/FeddCommands#Fedd_client.py the link above] unnecessary, but it is worth noting that one creates a slice using '''create''', terminates a slice using '''terminate''' and that there is a range of information available about a slice using the other informational requests.  Of particular interest are the '''image''' and '''ns_image''' commands that produce graphical representations of the topologies of either an existing slice or a slice description file.
     100Replicating the detailed documentation of fedd_client.py at [http://fedd.isi.deterlab.net/trac/wiki/FeddCommands#Fedd_client.py the link above] unnecessary, but it is worth noting that one creates a slice using '''create''', terminates a slice using '''terminate''' and that there is a range of information available about a slice using the other informational requests.  Of particular interest are the '''image''' and '''ns_image''' commands that produce graphical representations of the topologies of either an existing slice or a slice description file, and the '''status''' command that provides a brief summary of the slice's status.
    79101
    80 All of these commands can be run from `users.isi.deterlab.net` and will use the experimenter's default GID/fedid by default.  Using the command line interface allows more fine-grained control of slice management functions.  Specifically, by capturing an experiment capability using the '''--experiment_cert''' parameter to '''create''' and the '''--experiment_file''' parameters to information and '''terminate''' commands, the creator of the slice can allow others to manipulate the slice.
     102All of these commands can be run from the shell on `users.isi.deterlab.net` and will use the experimenter's default GID/fedid by default.  By registering as a TIED user on DETER, experimenters have ssh access to that machine.  Using the command line interface allows more fine-grained control of slice management functions.  Specifically, by capturing an experiment capability using the '''--experiment_cert''' parameter to '''create''' and the '''--experiment_file''' parameters to information and '''terminate''' commands, the creator of the slice can allow others to manipulate the slice.
    81103
    82104When using the command line, be sure to use the '''--url='''''!https://users.isi.deterlab.net:23235'' parameter.
    … …  
    93115The [http://seer.isi.deterlab.net SEER experiment control tool] is a useful way to manipulate a TIED experiment to both carry out the experiment and to examine real time data from the experiment.  Other Emulab-style measurement gathering and event systems function as well, but SEER is a convenient, extensible system.
    94116
    95 We focus on SEER, DETER's agent-based experiment management system.  Without modification SEER presents a diverse set of agents that can configure nodes, send traffic (both benign and malevolent), and collect data.  In addition, real-time data can be collected graphically.  The following short tour gives an idea how to use SEER and shows some of its potential.  More complete [http://seer.isi.deterlab.net documentation of SEER] (including video demonstrations) is available.
     117We focus on SEER, DETER's agent-based experiment management system.  Without modification SEER presents a diverse set of agents that can configure nodes, send traffic (both benign and malevolent), and collect data.  In addition, real-time data can be collected graphically.  The following short tour gives an idea how to use SEER and shows some of its potential.  More complete [http://seer.isi.deterlab.net documentation of SEER] (including video demonstrations) is available.  There are a lot of screenshots here to make things easy, but SEER is fairly intuitive to use.
    96118
    97119Once an slice is created that includes the SEER code - the [http://seer.isi.deterlab.net/trac/wiki/HOWTO/Setup SEER HOWTO] explains the commands one must include in the experiment configuration language one must make to include the SEER base system - the SEER front end can be used.  One can start the front end using the [http://seer.isi.deterlab.net/gui/SEER-GUI-1.5b3.jnlp webstart link] on the SEER page.  Alternatively one can download the source from there and compile and run SEER locally, though we don't recommend it.
    … …  
    99121As a quick tour of SEER's capabilities, we show a 3-node slice, split across two aggregates, controlled by SEER.  This is the TIED/one experiment listed on [https://www.isi.deterlab.net/fedd_list.php3 the slice listing page].  The control node is a node dedicated to letting SEER see the whole slice, even though parts of it are remotely allocated.  In TIED, control nodes are best allocated inside the DETER testbed.
    100122
    101 After activating the webstart link, and replying to any of your browser's questions about starting the interface, the seer window and a dialog requesting the kind of interface to use will appear.  Choose "Federated Emulab" from the list.
     123After activating the webstart link, and replying to any of your browser's questions about starting the interface, the SEER window and a dialog requesting the kind of interface to use will appear.  Choose "Federated Emulab" from the list.
    102124
    103125[[Image(seer1.png)]]
    … …  
    115137[[Image(seer6.png)]]
    116138
    117 One can get a real time summary of traffic between nodes by selecting the node on the topology, right clicking to access the node menu, and requesting a graph.  The graph appears under the "graphs" tab.  In this case the graph shows infrequent routing traffic (which is largely extraneous for such a simple slice).  To see more, we initiate a ping below.
     139One can get a real time summary of traffic between nodes by placing the mouse pointer on the node on the topology, right clicking to access the node menu, and requesting a graph.  The graph appears under the "graphs" tab.  In this case the graph shows infrequent routing traffic (which is largely extraneous for such a simple slice).  To see more, we initiate a ping below.
    118140
    119141[[Image(seer11.png)]] [[Image(seer12.png)]]