[[PageOutline]] = Detailed test plan for IG-ADM-2: Rack Administrator Access Test = ''This page is GPO's working page for performing IG-ADM-2. It is public for informational purposes, but it is not an official status report. See [wiki:GENIRacksHome/InstageniRacks/AcceptanceTestStatus] for the current status of InstaGENI acceptance tests.'' ''Last substantive edit of this page: 2012-05-12'' == Page format == * The status chart summarizes the state of this test * The high-level description from test plan contains text copied exactly from the public test plan and acceptance criteria pages. * The steps contain things i will actually do/verify: * Steps may be composed of related substeps where i find this useful for clarity * Each step is identified as either "(prep)" or "(verify)": * Prep steps are just things we have to do. They're not tests of the rack, but are prerequisites for subsequent verification steps * Verify steps are steps in which we will actually look at rack output and make sure it is as expected. They contain a '''Using:''' block, which lists the steps to run the verification, and an '''Expect:''' block which lists what outcome is expected for the test to pass. == Status of test == Meaning of states: * [[Color(green,Pass)]]: Step is completed and passed (for a verification step), or is completed (for a prep step) * [[Color(red,Fail)]]: Step is completed and failed, and is not being revisited * in progress: We are currently testing or iterating on this step * [[Color(orange,Blocked)]]: Step is blocked by some other step or activity || '''Step''' || '''State''' || '''Date completed''' || '''Tickets''' || '''Comments''' || || 1A || || || || ready to test || || 1B || || || || ready to test || || 1C || [[Color(orange,Blocked)]] || || || blocked on sudo access to boss || || 2A || || || || ready to test || || 2B || || || || ready to test || || 2C || [[Color(orange,Blocked)]] || || || blocked on sudo access to ops || || 3A || [[Color(orange,Blocked)]] || || || blocked on access to FOAM VM || || 3B || [[Color(orange,Blocked)]] || || || blocked on 3A || || 3C || [[Color(orange,Blocked)]] || || || blocked on 3A || || 4A || [[Color(orange,Blocked)]] || || || blocked on access to FlowVisor VM || || 4B || [[Color(orange,Blocked)]] || || || blocked on 4A || || 4C || [[Color(orange,Blocked)]] || || || blocked on 4A || || 5A || [[Color(orange,Blocked)]] || || || blocked on access to infrastructure VM server || || 5B || [[Color(orange,Blocked)]] || || || blocked on 5A || || 5C || [[Color(orange,Blocked)]] || || || blocked on 5A || || 6A || [[Color(orange,Blocked)]] || || || blocked on sudo access to boss; allocation of OpenVZ node || || 6B || [[Color(orange,Blocked)]] || || || blocked on 6A || || 6C || [[Color(orange,Blocked)]] || || || blocked on 6A || || 7A || [[Color(orange,Blocked)]] || || || blocked on access to switches || || 7B || [[Color(orange,Blocked)]] || || || blocked on 7A || || 7C || [[Color(orange,Blocked)]] || || || blocked on 7A || || 7D || [[Color(orange,Blocked)]] || || || blocked on serial access to switches || || 8A || [[Color(orange,Blocked)]] || || || blocked on access to switches || || 8B || [[Color(orange,Blocked)]] || || || blocked on 8A || || 8C || [[Color(orange,Blocked)]] || || || blocked on 8A || || 8D || [[Color(orange,Blocked)]] || || || blocked on serial access to switches || || 9 || [[Color(orange,Blocked)]] || || || blocked on access to rack iLO || == High-level description from test plan == This test verifies local and remote administrative access to rack devices. ==== Procedure ==== 1. For each type of rack infrastructure node, including the server host itself, the boss VM, the ops VM, the FOAM VM and an experimental host configured for OpenVZ, use a site administrator account to test: * Login to the node using public-key SSH. * Verify that you cannot login to the node using password-based SSH, nor via any unencrypted login protocol. * When logged in, run a command via sudo to verify root privileges. 2. For each rack infrastructure device (switches, remote PDUs if any), use a site administrator account to test: * Login via SSH. * Login via a serial console (if the device has one). * Verify that you cannot login to the device via an unencrypted login protocol. * Use the "enable" command or equivalent to verify privileged access. 3. Test that iLO (the InstaGENI remote console solution for rack hosts) can be used to access the consoles of the server host and an experimental host: * Login via SSH or other encrypted protocol. * Verify that you cannot login via an unencrypted login protocol. === Criteria to verify as part of this test === * V.01. For all rack infrastructure Unix hosts, including rack servers (both physical and VM) and experimental VM servers, site administrators should be able to login at a console (physical or virtual). (C.3.a) * V.02. Site administrators can login to all rack infrastructure Unix hosts using public-key SSH. (C.3.a, C.3.b) * V.03. Site administrators cannot login to any rack infrastructure Unix hosts using password-based SSH, nor via any unencrypted login protocol. (C.3.a) * V.04. Site administrators can run any command with root privileges on all rack infrastructure Unix hosts. (C.3.a) * V.05. Site administrators can login to all network-accessible rack infrastructure devices (network devices, remote KVMs, remote PDUs, etc) via serial console and via SSH. (C.3.a, C.3.b) * V.06. Site administrators cannot login to any network-accessible rack device via an unencrypted login protocol. (C.3.a) == Step 1: verify access to rack boss node == === Step 1A: verify that SSH to boss succeeds and allows public keys only === '''Using:''' * SSH to `chaos@boss.instageni.gpolab.bbn.com` from outside of the rack using public-key SSH * SSH to `chaos@boss.instageni.gpolab.bbn.com` from outside of the rack using password-based SSH '''Verify:''' * Public-key SSH succeeds * Password-based SSH does not succeed === Step 1B: verify the absence of common unencrypted login protocols === '''Using:''' * Use netstat to enumerate the network-listening processes running on boss * Identify each process and determine whether it is a common unencrypted login protocol * For any unencrypted login protocols found to be listening, try to access the relevant port remotely and determine whether login is possible '''Verify:''' * No unencrypted login protocols are listening on accessible networks * Login does not succeed via any unencrypted login protocol === Step 1C: verify sudo and sudo logging === '''Using:''' * On boss, run: `sudo whoami` * Look for a syslog file containing a record of the sudo command which was run '''Verify:''' * The sudo command should succeed * The command which was run should be recorded in a log == Step 2: verify access to rack ops node == === Step 2A: verify that SSH to ops succeeds and allows public keys only === '''Using:''' * SSH to `chaos@ops.instageni.gpolab.bbn.com` from outside of the rack using public-key SSH * SSH to `chaos@ops.instageni.gpolab.bbn.com` from outside of the rack using password-based SSH '''Verify:''' * Public-key SSH succeeds * Password-based SSH does not succeed === Step 2B: verify the absence of common unencrypted login protocols === '''Using:''' * Use netstat to enumerate the network-listening processes running on ops * Identify each process and determine whether it is a common unencrypted login protocol * For any unencrypted login protocols found to be listening, try to access the relevant port remotely and determine whether login is possible '''Verify:''' * No unencrypted login protocols are listening on accessible networks * Login does not succeed via any unencrypted login protocol === Step 2C: verify sudo and sudo logging === '''Using:''' * On ops, run: `sudo whoami` * Look for a syslog file containing a record of the sudo command which was run '''Verify:''' * The sudo command should succeed * The command which was run should be recorded in a log == Step 3: verify access to rack foam node == === Step 3A: verify that SSH to foam succeeds and allows public keys only === '''Using:''' * SSH to `chaos@foam.instageni.gpolab.bbn.com` from outside of the rack using public-key SSH * SSH to `chaos@foam.instageni.gpolab.bbn.com` from outside of the rack using password-based SSH '''Verify:''' * Public-key SSH succeeds * Password-based SSH does not succeed === Step 3B: verify the absence of common unencrypted login protocols === '''Using:''' * Use netstat to enumerate the network-listening processes running on foam * Identify each process and determine whether it is a common unencrypted login protocol * For any unencrypted login protocols found to be listening, try to access the relevant port remotely and determine whether login is possible '''Verify:''' * No unencrypted login protocols are listening on accessible networks * Login does not succeed via any unencrypted login protocol === Step 3C: verify sudo and sudo logging === '''Using:''' * On foam, run: `sudo whoami` * Look for a syslog file containing a record of the sudo command which was run '''Verify:''' * The sudo command should succeed * The command which was run should be recorded in a log == Step 4: verify access to rack FlowVisor node == === Step 4A: verify that SSH to FlowVisor succeeds and allows public keys only === '''Using:''' * SSH to `chaos@flowvisor.instageni.gpolab.bbn.com` from outside of the rack using public-key SSH * SSH to `chaos@flowvisor.instageni.gpolab.bbn.com` from outside of the rack using password-based SSH '''Verify:''' * Public-key SSH succeeds * Password-based SSH does not succeed === Step 3B: verify the absence of common unencrypted login protocols === '''Using:''' * Use netstat to enumerate the network-listening processes running on foam * Identify each process and determine whether it is a common unencrypted login protocol * For any unencrypted login protocols found to be listening, try to access the relevant port remotely and determine whether login is possible '''Verify:''' * No unencrypted login protocols are listening on accessible networks * Login does not succeed via any unencrypted login protocol === Step 4C: verify sudo and sudo logging === '''Using:''' * On flowvisor, run: `sudo whoami` * Look for a syslog file containing a record of the sudo command which was run '''Verify:''' * The sudo command should succeed * The command which was run should be recorded in a log == Step 5: verify access to rack infrastructure VM server host == === Step 5A: verify that SSH to foam succeeds and allows public keys only === '''Using:''' * SSH to `chaos` account on rack infrastructure server from outside of the rack using public-key SSH * SSH to `chaos` account on rack infrastructure server from outside of the rack using password-based SSH '''Verify:''' * Public-key SSH succeeds * Password-based SSH does not succeed === Step 5B: verify the absence of common unencrypted login protocols === '''Using:''' * Use netstat to enumerate the network-listening processes running on the infrastructure VM server * Identify each process and determine whether it is a common unencrypted login protocol * For any unencrypted login protocols found to be listening, try to access the relevant port remotely and determine whether login is possible '''Verify:''' * No unencrypted login protocols are listening on accessible networks * Login does not succeed via any unencrypted login protocol === Step 5C: verify sudo and sudo logging === '''Using:''' * On the infrastructure VM server, run: `sudo whoami` * Look for a syslog file containing a record of the sudo command which was run '''Verify:''' * The sudo command should succeed * The command which was run should be recorded in a log == Step 6: verify access to experimental OpenVZ node == === Step 6A: verify that SSH to experimental OpenVZ node succeeds and allows public keys only on public IPs === '''Using:''' * SSH to `root` on the experimental node from boss using public-key SSH * View `sshd_config` file and determine whether password-based login is allowed '''Verify:''' * Public-key SSH succeeds from boss * Password-based SSH does not succeed from outside of the rack === Step 6B: verify the absence of common unencrypted login protocols === '''Using:''' * Use netstat to enumerate the network-listening processes running on the OpenVZ node * Identify each process and determine whether it is a common unencrypted login protocol * For any unencrypted login protocols found to be listening, try to access the relevant port remotely and determine whether login is possible '''Verify:''' * No unencrypted login protocols are listening on accessible networks * Login does not succeed via any unencrypted login protocol === Step 6C: verify sudo and sudo logging === '''Using:''' * On the OpenVZ node, run: `sudo whoami` * Look for a syslog file containing a record of the sudo command which was run '''Verify:''' * The sudo command should succeed * The command which was run should be recorded in a log == Step 7: verify access to control network switch == === Step 7A: verify SSH access === '''Using:''' * Login to the control network switch via SSH '''Verify:''' * SSH login succeeds === Step 7B: verify privileged access to the control network switch === '''Using:''' * On the control network switch, run the enable command to enter privileged mode * On the control network switch, view the running configuration * On the control network switch, view the MAC address table '''Verify:''' * Entering privileged mode should succeed * Viewing the running configuration should succeed * Viewing the MAC address table should succeed === Step 7C: verify absence of unencrypted login access === '''Using:''' * From boss, attempt to connect via telnet (port 23) to the control network switch * From boss, attempt to connect via http (port 80) to the control network switch * If a port 80 connection is successful, determine whether login is allowed via that interface * Look at the control switch running configuration, and identify any configuration lines which might represent login services * Determine whether any such services are encrypted or disabled '''Verify:''' * The device does not allow telnet access on the standard port * The device does not allow unencrypted http authentication * No other services appear to allow remote unencrypted authentication === Step 7D: verify serial console access to the device === '''Using:''' * Access the control network switch via the serial console * Authenticate to the control network switch if necessary * Enter enable mode if necessary * View the running configuration of the control network switch '''Verify:''' * The control network switch should be accessible via serial console * It should be possible to perform privileged operations via the serial console * It should be possible to view the running configuration via the serial console == Step 8: verify access to dataplane switch == === Step 8A: verify SSH access === '''Using:''' * Login to the dataplane switch via SSH '''Verify:''' * SSH login succeeds === Step 8B: verify privileged access to the dataplane switch === '''Using:''' * On the dataplane switch, run the enable command to enter privileged mode * On the dataplane switch, view the running configuration * On the dataplane switch, view the MAC address table '''Verify:''' * Entering privileged mode should succeed * Viewing the running configuration should succeed * Viewing the MAC address table should succeed === Step 8C: verify absence of unencrypted login access === '''Using:''' * From boss, attempt to connect via telnet (port 23) to the dataplane switch * From boss, attempt to connect via http (port 80) to the dataplane switch * If a port 80 connection is successful, determine whether login is allowed via that interface * Look at the dataplane switch running configuration, and identify any configuration lines which might represent login services * Determine whether any such services are encrypted or disabled '''Verify:''' * The device does not allow telnet access on the standard port * The device does not allow unencrypted http authentication * No other services appear to allow remote unencrypted authentication === Step 8D: verify serial console access to the device === '''Using:''' * Access the dataplane switch via the serial console * Authenticate to the dataplane switch if necessary * Enter enable mode if necessary * View the running configuration of the dataplane switch '''Verify:''' * The dataplane switch should be accessible via serial console * It should be possible to perform privileged operations via the serial console * It should be possible to view the running configuration via the serial console == Step 9: verify that iLO is not accessible via unencrypted protocols == '''Using:''' * From boss, attempt to connect via telnet (port 23) to the pc1 iLO IP * From boss, attempt to connect via http (port 80) to the the pc1 iLO IP * If a port 80 connection is successful, determine whether login is allowed via that interface * If any prospective unencrypted protocols were identified in the iLO configurations during IG-ADM-1 step 3F, attempt to connect to those ports from boss '''Verify:''' * The pc1 iLO does not allow telnet access on the standard port * The device does not allow unencrypted http authentication * No other services appear to allow remote unencrypted authentication