[[PageOutline]] = Configuring dataplane interfaces on MyPLC PlanetLab nodes = This page discusses how to configure secondary interfaces on MyPLC-controlled PlanetLab nodes. == Interface numbering gotchas == PlanetLab nodes boot using a two-stage process. When multiple interface cards are present, the first-stage boot and second-stage boot may not number the interfaces consistently. This typically causes the node to be unusable. We don't know how to control which interface the first-stage boot will use as eth0. However, it is easy to use the MyPLC interface to control which MAC address should be assigned as each interface in the second-stage boot. This section describes how we force MyPLC to use the same interface allocation in second-stage boot as it did in first-stage, allowing the node to be used even if these settings do not natively match. === Step 1: Boot into first-stage with the control interface connected === Before booting the node, physically connect the control interface cable to the interface which first-stage boot will choose as `eth0`. On our lab nodes (1U Dell R300s), first-stage boot always chooses the leftmost interface on the leftmost expansion NIC to be `eth0` (or the leftmost interface on the motherboard if there are no expansion cards). In your environment, you may need to find this interface by guesswork: if the node can successfully enter first-stage boot and seems to reach a server, you guessed right. === Step 2: Use the first-stage boot logs to identify the MAC address of eth0 === The MyPLC gathers logs from each node's first-stage boot. These logs will contain the MAC address of first-stage `eth0` on your node. To find this, login to MyPLC, and do: {{{ sudo find /var/log/bm/raw | grep | sort | tail -1 | xargs sudo grep macs }}} That command should report output similar to this: {{{ $ sudo find /var/log/bm/raw | grep | sort | tail -1 | xargs sudo grep macs 19:48:22(UTC) net:InitInterfaces macs = {'00:00:00:00:00:00': 'lo', 'aa:bb:cc:dd:ee:ff': 'eth0'} }}} Use the reported MAC address: * ``: `aa:bb:cc:dd:ee:ff` === Step 3: Configure MyPLC to use the correct MAC in stage 2 === Use the PlanetLab UI to force the second-stage boot to use the leftmost interface on the leftmost expansion card: * Browse to the MyPLC web interface and login as an administrator * Click "My Site Nodes" in the left menubar * Click "``" in the table of nodes * Expand the "One interface" menu * Click the IP of the primary interface * Set MAC to `` * Click "Update" * Click "Back to Node" * Click "Update Node" * Reboot the node === Step 4: Modify udev to fix the live interface configuration === Sometimes, Fedora gets confused by eth0 switching identities, as a result of which eth1 or eth2 (if you have that many interfaces) does not appear, and is replaced by a weird `devNNNNN` interface. This appears to be fixable by modifying udev's configuration: * Login to the node as root * Edit `/etc/udev/rules.d/70-persistent-net.rules` * Find a configuration section containing the comment `(custom name provided by external tool)`, e.g.: {{{ # PCI device 0x8086:0x10c9 (igb) (custom name provided by external tool) SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:1b:21:4b:3f:ac", ATTR{type}=="1", NAME="eth0" }}} Make sure that there is only one such section, and that its MAC address matches `` * Delete all SUBSYSTEM comment-and-line blocks except this one, and save the file * Reboot the node