wiki:GENIRacksAdministration/InstaGENIFAQ

InstaGENI Administrative Frequently Asked Questions

This page shows frequently asked question from administrators that have deployed InstaGENI racks and is part of the GENI Rack Administration pages.

User Administration

Q. I am the administrator. What can I access in my rack?

There are three potential access scenarios that may be in place

  • Local Administrator: Each InstaGENI rack is initially configured to allow ssh access for one administrator account to all hosts (physical and virtual). If you are this person, than you have access to all hosts (boss, ops, control, flowvisor ) in the rack. Do note that the 'flowvisor' host is now a light-weight VM used for the purposes of forwarding OF traffic from the data plane switch to experimenter controllers and that the actual FOAM/FlowVisor OpenFlow packages are no longer supported. Click here for more information.
  • Additional administrative accounts : User has submitted a request for an administrative account by joining the emulab-ops group and uploading his/her SSH keys via the Emulab web interface. Once the request is approved by an existing administrator, the new administrator can access web administrative functions, and some hosts (boss and ops) via SSH.
  • Access to additional nodes: To get a new administrator SSH access to the control, and FlowVisor nodes , an existing administrator must install the new administrator's SSH public key from the control host by running the makeadmin.pl script as instructed here to each of the hosts control, and flowvisor. Once the SSH keys have been installed, the newly added administrator will have access to all hosts (boss, ops, control, flowvisor).

Q. How do I request an administrative account?

Use your browser to go to the Emulab web interface for your rack, and select Request Account. On the next page select Join Existing Project. Fill out the form, taking care that you join the project named emulab-ops . Once approved, you have administrative privileges for the web interface. If you also want SSH administrative access on the hosts in the rack, send your SSH public key to your administrator and request that they create an account as described here.

Q. I tried logging into the boss node with my administrative ID and got a "permission denied" error. What is wrong?

Access to the hosts in the InstaGENI rack requires SSH public keys. You must upload your SSH public key into the Emulab web interface, which will result in access to the boss and ops node. If you want access to additional nodes (control, flowvisor) then you must ask the system administrator to install your keys on the control host as specified here.

Q. I have an administrative account and am logged into the web interface, but I cannot access administrative functions. What should I do?

To enable administrative features in the Emulab web interfaces, simply click on the green dot at the top of the page. The dot will become red and the administrator functions are enabled. To disable, click on red dot.

Q. What are the roles and responsibilities for a site administrator for the addition of new VLANs? Will GPO or Utah InstaGENI team handle this remotely?

The team responsible will depend on the stage of the rack deployment. During the initial setup process, both Utah and GPO teams set up various site VLANs required for rack operation in the GENI environment. If your site needs to set up additional VLAN, both the Utah and GPO teams are available to provide support.

Q. What are the roles and responsibilities for a site administrator for site configuration changes? Will GPO/Utah InstaGENI team handle this remotely?

It depends on what the change is, but for the foreseeable future the Utah team will probably make site changes. If you are thinking of making some changes yourself to the rack configuration, please check with the Utah InstaGENI team. If you want to see the details of a proposed change, contact the InstaGENI team for more information about their repositories and releases.

Q. What are the roles and responsibilities for a site administrator for a physical change at a site?

If your site is getting new hardware or if your site needs to be power cycled, you may be contacted by InstaGENI or GPO engineers. Both are unlikely scenarios. If your site needs to schedule an outage, move, or other maintenance that will affect the rack, contact the GMOC to open a ticket with the time and reason for the maintenance. If necessary, InstaGENI or GPO engineers will contact you to coordinate.

Q. Assuming my site needs to be powered-off, who will be contacting the our site?

In the unlikely event that your site needs to be powered-off or shut down, any one of the GMOC, GPO, or Utah may be contacting you. If you decide that your site is to be shut down for any reason, you must notify the GMOC. If you have any questions about what to do technically to properly shut down your InstaGENI rack, contact the InstaGENI Utah team.

Q. Who has the authority to mandate/delegate changes? And when these changes are approved, how are they audited and tracked?

Once your rack is officially opened to experimenters, the InstaGENI Utah team will periodically update the software in the racks. The GMOC might occasionally ask site administrators to investigate some activity (like a runaway experiment in your rack). If you want to make a change to your rack, for example by connecting a new local LAN to the OpenFlow switch, contact the GMOC to coordinate the change. Please provide as much lead time as possible, because your change might affect running experiments, and they need time to adjust their plans. The GPO may also ask for minor changes, but this is uncommon.

Q. What is the InstaGENI hierarchy? Will Utah/GPO teams provide first line support while local site administrators are second in line?

The hierarchy is somewhat in flux and depends on the situation. But site administrators shouldn't expect too many requests, and should only be asked for quick turnaround time for critical scenarios such as emergency stop scenarios or for a Legal, Law enforcement and Regulatory agency (LLR) requests.

Rack Resources

Q. What types of servers are in the racks ?

Earlier racks have HP ProLiant DL380 G7 server while most racks have HP ProLiant DL380p Gen8 servers.

Q. How do I tell if my rack has G7 or G8 (Gen8) servers?

There are two way to tell which type of servers are in your rack.

  • Via your rack's web interface you can select a host and then its "Node type" and on the resulting page you can see the processor field, or you can simply use https://boss.instageni.gpolab.bbn.com/shownodetype.php3?node_type=dl360 (but for your own rack name, of course).
  • Via the omni command "listresources" for your rack. You will find results include <hardware_type name="dl360-G7"> for G7 Servers and <hardware_type name="dl360"> for G8.

Q. What are the CPU, RAM and storage details for rack servers

The G7 nodes have 12 cores, but with hyper threading it is 24 CPUs. The G8s have 16 cores and 32 CPUs. The CPU freq on the G7s is 2666 while on the G8s it is 2094. All servers have 48GB Ram. The G7 servers have a 500GB disk and the G8 servers have a 1TB disk.

Q. What type of notifications will I get from the InstaGENI rack resources?

When experimenters request resources, the InstaGENI processes generate email notifications to the testbed-ops@your.domain mail list. There are several types of notifications that you may receive with these subject lines:

  • "YOURINSTAGENI: protogeni-wrapper.pl" - These notifications include details for successful and unsuccessful experimenter operations. These emails do not require any administrative action, but may be monitored to determine the activity on your rack.
  • "YOURINSTAGENI: Membership $operation emulab-ops/emulab-ops" - Where operation may include "request", "approval", etc. for user account requests on your rack to access the rack's web interface. These emails require administrative action.
  • "YOURINSTAGENI: 4 PCs idle 10 hours: emulab-ops/xen-guests" - Notification that idle resources are being released from experimenter's sliver. No administrator action required.
  • "YOURINSTAGENI: 1 nodes failed to boot" - Notification that experimenter resource failed to boot, may require action by InstaGENI team to reset the node, and site administrator may need to take action, if the team asks for assistance.

Q. What are the email lists associated with my rack?

There are several maillist used for each InstaGENI Rack, see the Rack Mail Lists page for the list of email lists and how to get access and support for messages.

Network Requirements

Q. Which ports will need to be open for my InstaGENI rack?

The list of required ports for the InstaGENI rack can be found in the InstaGENI Site Checklist. If your rack is inside a campus firewall then the following ports must be allowed through your campus network firewall to the entire rack subnet:

  • ssh (port 22)
  • smtp (port 25 - outbound connections only, from boss/ops)
  • http (port 80 - must also allow outbound connections from boss/ops)
  • https (port 443 - must also allow outbound connections from boss/ops)
  • Flash Policy Server (port 843)
  • InstaGENI XMLRPC (port 12369)
  • iLo remote console (port 17990)
  • iLo remote media (port 17988)
  • Ports used by software (ports 25000+)
  • Allow GRE IP protocol, required for constructing tunnels between racks.

Q. Which ports will GENI experimenters need on my InstaGENI rack?

Ports used by experimenters depend on the type of experiments run. For a list of current experimenter ports see the Known GENI Ports page.

Monitoring

Q. Will GMOC monitor my rack? Do I have visibility into GENI monitoring?

Monitoring is currently set up for each InstaGENI rack as part of the deployment process. Each rack reports data to the GMOC, and the GMOC runs a web service for viewing the data collected, at the location https://gmoc-db.grnoc.iu.edu/protected. To access your site data, you can use OpenID, InCommon, or a GlobalNOC account. As a site administrator you will also have the ability to download data from the GMOC and plug that into any existing monitoring infrastructure that you may have (e.g. Nagios).

Last modified 3 years ago Last modified on 04/17/17 14:16:22