[[PageOutline(1-2)]]

= OPS-006-A GENI WiMAX Base Station Debugging Procedure =


This procedure describes the steps taken to identify and remedy misconfigured WiMAX base stations at GENI wireless sites. WiMAX issues will most likely be reported by Sprint Corporation, who will detect a change in frequency of operation of GENI WiMAX base station as their interference software detects an unauthorized frequency as noise. GENI experimenters or Site Contacts may not be aware of the problem, however the misconfiguration can be detected by the [http://genimon.uky.edu/ GENI Monitoring System].  Regardless of the source for the reported event, a ticket must be written to handle the investigation and resolution of the problem. Ticket must copy the issue reporter, the WiMAX development team wimax-developer@winlab.rutgers.edu and the GENI Experimenters at geni-users@googlegroups.com. 

= 1. Issue Reported =

GMOC gathers technical details for failures including:
 - Requester Organization 
 - Requester Name 
 - Requester email
 - Requester GENI site-name 
 - Problem symptoms 

== 1.1 GENI Event Type Prioritization ==

GMOC should classifies a WiMAX Base Station failure as `High` priority. This type of issue may be deemed `Critical` if the person reporting the issue identifies it as `Critical`. For example if the issue impacts a demo, a training session, or a conference. 

== 1.2 Create Ticket ==

The GMOC ticketing system is used to capture information above. GMOC may follow up to request additional information as problem is investigates. This operation results in the problem reporter getting a ticket email for the issue reported.

= 2. Investigate and Identify Response =

== 2.1 Investigate the Problem ==

Login int the [http://genimon.uky.edu/ GENI Monitoring System] dashboard and select `Nodes`. 

In the `Search` box in the upper right hand of the `Nodes` panel, search for "44-51". This is the GENI WiMAX prefix that narrows the list to all GENI WiMAX base stations reporting to the monitoring service.

Look at the `Aggregate` field in the list of monitored `Nodes` and select the WiMAX aggregate reported as experiencing the problem the GMOC ticket. The resulting page provides details for the WiMAX aggregate along with its `Interfaces`.  Select each of the interfaces and look for the the `tx_frequency` variable. If the `tx_frequency` variable in the node description identified above has been modified there is an issue.  This variable should __NEVER__ be modified.  

This misconfiguration does not affect a GENI site or experimenter. However, Sprint Corporation will detect a change in frequency of operation of GENI WiMAX base station as their interference software will detect an unauthorized frequency as noise.

== 2.2 Identify the Response == 

Identify the site Administrator for the site. The master list of GENI WiMAX site admins is posted at [http://wimax.orbit-lab.org/wiki/aaOperations#SitesandOperationsContacts Site Contacts].   Also identify the site Principal Investigator(PI) in the Site Contacts list. 

Contact the site administrator and request that the the frequency be reverted to the assigned site frequency.

Escalate to the site Principal Investigator(PI) also in [http://wimax.orbit-lab.org/wiki/aaOperations#SitesandOperationsContacts Site Contacts] list, if no response is received within 8 hours.

= 3. GMOC Response =

The GMOC implements the actions outlined to identify the source of the issue and updates the ticket to capture actions taken.

== 3.1 Implement Response ==

The GMOC executes the steps outlined. If GMOC finds procedure to be lacking, then steps should be taken to get the procedures updated.

== 3.2 Procedure Updates ==

When instructions in a procedure are found to miss symptoms, required actions, or potential impact, then action must be taken by the GMOC to provide feedback to enhance the procedure for future use.

= 4. Resolution =

GMOC verifies the the problem is no longer happening by checking the [http://genimon.uky.edu/ GENI Monitoring System] information for the base stations as described in section 2.1.  GMOC can also verify that the problem is resolved by coordinating with the original problem reporter.

== 4.1 Document Resolution and Close Ticket ==

GMOC captures how the problem is resolved in the ticket and closes the ticket. This should result in notification back to the problem reporter.