wiki:ConnectivityGuidelines

Version 24 (modified by jwilliams@bbn.com, 14 years ago) (diff)

--

NOTE: This information is subject to change, but is actively maintained. Please email us (see footer) with updates and corrections.

Summary

This wiki outlines the steps required to aquire various types of connections between two campuses. Before following these guidelines it's recommended to review ConnectivityOverview for a high-level summary of connectivity approaches. See ConnectivityOptions for a matrix of potential options that various campuses support.

Overview

Every campus's situation is unique. This page is a general common-case guideline. Your campus's particular paths and goals may deviate from this outline.

In order for two sites to establish a connection they need some help from every service provider along the network path between the sites. The service providers involved may include: campus IT, regional providers, national research backbones.

First, you must decide which method(s) your campus can use to connect with other campuses. See ConnectivityOptions for a matrix of which options are supported by which campuses. While establishing connectivity you should also update your info on your campus's site page. This will allow for other campuses to find the necessary information to connect with you in the future.

Note that it's assumed that all of this VLAN provisioning is discussing 801.q tagged (aka trunked) VLANs.

Layer 2 Connections

Backbone Options

This is a high-level summary page, see the corresponding Backbone's GENI wiki page for more information including Links to webpages, instructions for creating accounts and provisioning VLANs.

This section lays out the general procedure for establishing an end-to-end connection over a backbone network.

Once Per Backbone

  1. Join the backbone service (contracts, negotiations, bartering, etc).

If this is your campus's first time connecting to a backbone you'll need to establish a relationship with your backbone and establish a backbone endpoint. There may already be arrangements for your Regional Provider to share/provide access to a backbone's endpoint.

  1. Obtain an account for provisioning connections in the backbone network. Strictly speaking this isn't always necessary if your partner campus has access - they can provision the VLAN.
  1. Determine the name of your backbone endpoint.

Once Per Campus

You may be required to grant permissions to your partner campus to connect to your backbone endpoint This is the case For NLR

Once per Connection

These section outlines the steps necessary for your campus to get connectivity to your backbone endpoint. Your partner campus will also need to do these steps, though they may allocated extra paths at a previous point. Note that you'll need a unique VLAN per unique connection that you want to establish. If you wish to connect to multiple campuses, you'll need to provision multiple VLANs. You may also wish to discuss IP address assignments for testing at this time.

Regional Provisioning

Your campus will need to request your regional provider to provision the VLAN from your campus's site edge to your backbone endpoint. If you plan on connecting to multiple campuses, or have multiple unique connections to a campus, you may want to request multiple VLANs at this time.

Campus Provisioning

Now that you know which VLAN(s) are available in the regional network to reach your campus's backbone endpoint, You'll need to provision the same VLAN IDs from your campus's site edge to the particular network gear that you wish to share.

Your partner campus will need to provision VLAN(s) to their campus's edge as well.

Backbone Provisioning

Now that both your campus and your partner campus have VLANs provisioned to your corresponding endpoints, you can, finally, provision the VLAN in the backbone to connect the endpoints into one network. You, or your partner campus, can perform this action via the backbone's web provisioning service.

Notes and Gotchas

  • NLR's web provisioning service (Sherpa), by default, does not provide VLAN translation; your campus, your partner campus, and all regional networks will need to provision the same VLAN ID to establish a connection.

Two campuses - One Regional

Sometimes campuses share the same regional provider. It's possible that the regional can provide a direct layer 2 connection between your campus's regional endpoint and your partner campus's regional endpoint. Your campus, your partner campus, and your shared regional provider can then provision a VLAN as outlined above. You'll need to discuss with your regional whether your campus and your partner campus need to negotiate the same VLAN ID or if the regional offers any VLANIDConflicts VLAN services.

Testing

Typically both campuses will assign IP addresses to various hosts to allow common IP-based programs to quickly verify Layer-2 connectivity. You'll talk with your partner campus to decide on a IP subnet that can be used on both campuses. You'll then want to share a list of IP addresses, as well as MAC addresses, that each campus will use.

TIP If you're the first person to start IP negotiations specify what IP address you want to use. Such as "I plan on using 10.37.45.12 - what do you plan on using". This may help prevent the case where both campuses use the same IP address.

  • ping your partner site's IP address.
  • traceroute your partner site's IP address.
  • arp your partner site's IP Address. this will display your partner's MAC address.

When Things Go Wrong

Talk to your partner campus. Each Site should:

  • Check to see if your edge switch see's your MAC address in the appropriate VLAN.
  • Check to see if your edge switch see's your partner sites MAC address in the appropriate VLAN.

If you used a backbone network you can use a service to inspect whether various back bone switches are seeing the MAC address in question.

NLR
httpe://routerproxy.grnoc.iu.edu/nlr/
  1. Select FrameNet Core Switch that you want to inspect (start with your endpoint).
  2. select the show mac-address-table and specify the vlan you wish to inspect (e.g. type "vlan 3705" in the text box).
  3. run the command, and look for your mac address.
Internet2
http://routerproxy.grnoc.iu.edu/internet2/ Note that currently this service doesn't offer Layer 2 debugging facilities.
  1. Select the switch that you want to inspect.
  2. run the command.

If your MAC is shown in your endpoint switch, but not on your partner's endpoint switch, then the issue may be within the backbone.

I the MAC is not making it to the campus endpoint, it may be a regional network issue. sometimes regionals will attach a node with an IP address for you to ping to confirm that you can reach a particular point in their Network.

Note, sometimes there are spanning tree issues.

VLAN ID conflicts

Given the limited number of VLAN IDs, it's conceivable to run into a conflict when provisioning a common VLAN between two end points. Here are a few common options to resolve the conflict.

Renegotiate

Quite often there's another range of VLANs that are available which may work within your network. VLAN negotiations are typically an iterative process.

QinQ

QinQ is a tunneling option which "wraps" your frames (marked with your VLAN ID) within another VLAN ID that is available within the regional network. For example, you provisioned VLAN 1234 to NLR from your network, but your regional is already using this VLAN ID. Your regional can assign you with another VLAN ID, say 2345, for this connection and tunnel VLAN ID 1234 through this connection so that you can reach NLR with your VLAN ID intact. Depending on the situation, your regional may do this transparently or your campus may need to be involved.

Your campus and your partner campus would likely be using the same VLAN ID in this scenario.

VLAN Translation

VLAN translation translates one VLAN ID to another allowing two separate VLAN ID's to coexist as if it was one VLAN topology. For example, ION? provides VLAN translation. This allows your campus to provision a VLAN ID through your campus and regional network to your ION endpoint without concern for your partner campus's VLAN provisioning. Your partner campus does the same. Once both campuses can reach their ION endpoints, the ION connection can be provisioned to translate your campus's and your partner campus's VLAN ID.

Your Campus and your partner campus would likely be using different VLAN IDs in this scenario.

VLAN Translation Options in Backbone Networks

Layer 3 Connections

If your campus doesn't have layer 2 connectivity (no backbones, must go through a router, etc) then layer 3 connectivity may be an option. For GENI experiments this requires tunneling.

Connections

Backbone
Your campus may have access to a backbone Layer 3 connection such as NLR's !PackeNet.
Commodity Internet
You use the normal "commodity" internet path's through your ISP(s).

Tunneling Options

Generic Route Encapsulation (GRE)
http://en.wikipedia.org/wiki/Generic_Routing_Encapsulation
Software Encapsulation
There are several software solutions available. OpenFlow switches can use the Encapsulator package available at http://www.openflowswitch.org/wk/index.php/Capsulator
Mutli-Path Label Switching (MPLS)
http://en.wikipedia.org/wiki/MPLS

Email us with questions and feedback on this page!