Opened 12 years ago
Closed 12 years ago
#20 closed (fixed)
Capture documentation and the process required to make a custom images available in InstaGENI
Reported by: | lnevers@bbn.com | Owned by: | somebody |
---|---|---|---|
Priority: | minor | Milestone: | IG-EXP-2 |
Component: | Experiment | Version: | SPIRAL4 |
Keywords: | Cc: | ||
Dependencies: |
Description
This ticket track the activities and documentation required to upload a custom image to an InstaGENI rack. The process start with an email request to geni-dev-utah@flux.utah.edu. Following is the initial email request:
From: Luisa Nevers <lnevers@bbn.com> Date: 5/14/12 4:40 PM To: geni-dev-utah@flux.utah.edu, instageni-design@geni.net Hello, I am executing the InstaGENI Acceptance tests and need to upload two Ubuntu images to be used in experiments on the InstaGENI rack at Utah. Please let me know if there are any special requirements that I should consider in creating the Ubuntu images. I plan to run one Ubuntu image with "emulab-openvz" and one Ubuntu with "raw-pc" type. Also, if any documentation already exists that I should reference, please let me know. Luisa
Attachments (7)
Change History (12)
comment:1 follow-up: 2 Changed 12 years ago by
Changed 12 years ago by
Attachment: | install-ubuntu1204-log.txt added |
---|
comment:2 Changed 12 years ago by
Capturing Update for the Custom Image creation:
On 5/16/12 10:39 AM, Leigh Stoller wrote:
The approach for creating images to use on the racks, is to use the Emulab interface on the rack. See this tutorial section on how to create an image.
https://users.emulab.net/trac/emulab/wiki/Tutorial#CustomOSImages
In short, you would log into your Emulab account, allocate a node, load your image, test your image, take a snapshot, etc. Others at BBN are familiar with this process so they can help you out.
I followed the instruction given in the tutorial and created a one node experiment which was defined with "tb-set-node-os $node0 UBUNTU11-64-STD" as a starting OS. Experiment was started and I was able to login to the node without problem.
My goal custom image is Ubuntu 12.04, so I upgraded the Ubuntu 11.04 node to Ubuntu 12.04, but upon reboot the node did not come up. Logs for the assigned node (pc250.emulab.net) show the following:
PXE version 2.1, real mode entry point @9c3e:0106 (!PXE) BIOS 640kB/2095872kB available memory FreeBSD/i386 BTX-based Emulab PXE boot loader, Revision 1.17 (root@node.freebsd73-node.testbed.emulab.net, Fri Jun 10 16:38:21 MDT 2011) pxe_open: using PXE DHCP info pxe_open: ipaddr: 155.98.39.50/255.255.252.0 pxe_open: server addr: 155.98.38.163 pxe_open: server path: /tftpboot pxe_open: gateway ip: 155.98.36.1 DHCP Info: IP: 155.98.39.50, Server: 155.98.38.163, Gateway: 155.98.36.1 Type a key for interactive mode (quick, quick!) |Loading disk 0x80 MBR and booting system partition 2 /-\
The upgrade is captured in the attached file named install-ubuntu1204-log.txt.
Determine what to do next....
comment:3 Changed 12 years ago by
InstaGENI Custom Linux Image
The following procedure was followed to customize and Ubuntu image for use in the InstaGENI Utah rack.
Initial email requests was sent to the geni-dev-utah@flux.utah.edu list. I was pointed to the Tutorial page in the Linux Custom OS Image section. Also found the Windows Custom Images page, which will be captured in separate procedure.
Here are the high level steps for creating a custom Ubuntu image for an InstaGENI rack:
- Create a NS experiment with one of the existing image.
- Once the experiment is running login to the node and customize.
- Create a new Image Descriptor and snapshot
- Wait for the email saying the image creation is done.
- Verify new customized image.
- Make image available
This scenario generated a custom Ubuntu 12.04 image based on an available Ubuntu 11 image.
Step 1. Create an Experiment with an existing image
The first step in the Custom Images section requires defining a Network Simulator (NS) file with a Java GUI found at https://boss.utah.geniracks.net/clientui.php3. Used this client to define a one nodes NS files as suggested, or one can just simply use the following file:
set ns [new Simulator] source tb_compat.tcl set node0 [$ns node] tb-set-node-os $node0 UBUNTU11-64-STD $ns rtpproto Static $ns run
Referred to the current list of available OS Image Descriptors to choose starting OS tb-set-node-os UBUNTU11-64-STD.
Once the ns file is defined, you can create an experiment. Login to https://boss.utah.geniracks.net/ and choose "Experimentation"-> "Begin an Experiment". On the "Begin a Testbed Experiment" page name your experiment, and load the NS file shown above, make sure to run the "Syntax Check" and modify "Linktest Option" to skip the Link test.
(see attached capture of the form named BeginNSExperiment.jpg)
Once the form is filled, you can now press "Submit" which brings up a new pages showing how the experiment is being set up and which node is used.
(see attached capture of the page named nodesetup.jpg)
Step 2. Login to node and customize
Once the experiment is set up, you may login and customize your node.
lnevers@sendaria:~/gcf-1.6.1$ ssh pc250.emulab.net Warning: Permanently added 'pc250.emulab.net,155.98.39.50' (RSA) to the list of known hosts. Welcome to Ubuntu 11.04 (GNU/Linux 2.6.38.7-1.0emulab x86_64) * Documentation: https://help.ubuntu.com/ /users/lnevers% cat /etc/issue Ubuntu 11.04 \n \l /users/lnevers%
The customization in this procedure executed an upgrade from Ubuntu 11.04 to Ubuntu 12.04.
Step 3. Create a new Image Descriptor and snapshot
Once the customization in step 2 is completed, use the Create new Image Descriptor page to create the Image Descriptor which will be associated with your new custom image.
(see attached capture of the page named customUbuntu1204.jpg)
Note: "*Which DOS Partition[1]: (DOS partitions are numbered 1-4)" value used was 2 for the Ubuntu custom image.
When the request to create a new image descriptor is submitted, user told that email notification will be sent upon completion.
(see attached capture of the page named customUbuntu1204CreateDescriptor.jpg)
Step 4. Wait for the email saying the image creation is done
The following email notification was generated from the successful image creation:
Subject: UTAHGENIRACK: Image Creation on pc2 Completed: gpo/LNUBUNTU1204 From: Luisa Nevers <lnevers@bbn.com> Date: 5/21/12 6:05 PM To: Luisa Nevers <lnevers@bbn.com> Image creation on pc2 has completed. As you requested, the image has been written to /proj/gpo/images/LNUBUNTU1204.ndz. You may now os_load this image on other nodes in your experiment. --------- /tmp/logfile.Itj0ZZ -------- reboot (pc2): Attempting to reboot ... reboot (pc2): Successful! reboot: Done. There were 0 failures. pc2 Waiting for nodes to come up. All nodes are up. pc2: started image capture, waiting up to 72 minutes pc2: still waiting ... it has been 2 minutes. Current image size: 504365056 bytes. pc2: still waiting ... it has been 4 minutes. Current image size: 1008730112 bytes. pc2: image capture has completed: status='0' All nodes have completed their command. reboot (pc2): Attempting to reboot ... Connection to pc2.utah.geniracks.net closed by remote host. reboot (pc2): Successful! reboot: Done. There were 0 failures. pc2 /proj/gpo/images/LNUBUNTU1204.ndz: 1194 chunks, 2082 regions, 41970 hashregions, 2678062080 data bytes 2678062080 bytes: inflate cycles: 50950875517 Swapout signature file created Image creation succeeded. Image written to /proj/gpo/images/LNUBUNTU1204.ndz. ------------------ Prepare Output ---------------- Legacy library ctime.pl will be removed from the Perl core distribution in the next major release. Please install the separate libperl4-corelibs-perl package. It is being used at /usr/local/etc/emulab/prepare, line 9. Cleaning node; removing configuration files Running /usr/local/etc/emulab/rc/rc.config to clean up ... Checking manifest... Running 'diff -q -u /etc/emulab/gshadow /etc/emulab/gshadow.new' Files /etc/emulab/gshadow and /etc/emulab/gshadow.new differ Files /etc/emulab/gshadow.new and /etc/emulab/gshadow differ, but we can't show the changes, and I was told not to update the master files! Running 'diff -q -u /etc/emulab/shadow /etc/emulab/shadow.new' Files /etc/emulab/shadow and /etc/emulab/shadow.new differ Files /etc/emulab/shadow.new and /etc/emulab/shadow differ, but we can't show the changes, and I was told not to update the master files! Running 'diff -u /etc/emulab/group /etc/emulab/group.new' --- /etc/emulab/group 2011-08-25 11:59:34.000000000 -0600 +++ /etc/emulab/group.new 2012-05-21 15:54:31.000000000 -0600 @@ -51,3 +51,6 @@ ssl-cert:x:111: postfix:x:112: postdrop:x:113: +messagebus:x:114: +utempter:x:116: +_cvsadmin:x:115: Files /etc/emulab/group.new and /etc/emulab/group differ, but I was told not to update the master files!. Running 'diff -u /etc/emulab/passwd /etc/emulab/passwd.new' --- /etc/emulab/passwd 2011-08-25 11:59:00.000000000 -0600 +++ /etc/emulab/passwd.new 2012-05-21 15:54:31.000000000 -0600 @@ -23,3 +23,4 @@ statd:x:103:65534::/var/lib/nfs:/bin/false ntp:x:104:110::/home/ntp:/bin/false postfix:x:105:112::/var/spool/postfix:/bin/false +messagebus:x:106:114::/var/run/dbus:/bin/false Files /etc/emulab/passwd.new and /etc/emulab/passwd differ, but I was told not to update the master files!. Resetting passwd and group files /bin/cp: cannot stat `/etc/emulab/hosts': No such file or directory *** /usr/local/etc/emulab/rc/rc.hostnames: Could not copy default /etc/hosts into place! Failed running rc.hostnames (512)! at /usr/local/etc/emulab/libsetup.pm line 1085. Removing old DB files ... Removing old /etc/dumpdates file ... Creating stub /etc/dumpdates file ... Cleaning logfiles ... Removing accounting files ... Removing root's history ... Removing root's mailfile ... Resetting NTP drift ... Clearing out /var/run ... Clearing out /tmp ... Cleaning out /local/logs ... Removing SFS files ... Clearing out directories in /var ... Cleaning up /var/tmp .... Clearing out directories in /var/emulab ... Clearing out old Emulab scripts and binaries in /usr/local/etc/emulab ... Removing backup files in /etc Removing assorted unix-domain socket files prepare ran successfully!
Step 5. Verify new customized image
Once the image is available, verify that it can be used by creating a new one node experiment that uses the image. For this scenario the following ns file was generated to test the new custom image:
set ns [new Simulator] source tb_compat.tcl set node0 [$ns node] tb-set-node-os $node0 LNUBUNTU1204 $ns rtproto Static $ns run
Begin a new experiment by choosing "Experimentation"-> "Begin an Experiment". On the "Begin a Testbed Experiment" page name your experiment, and load the NS file shown above, run the "Syntax Check" and modify "Linktest Option" to skip the Link test.
(see attached capture of form named testCustomImage.jpg)
Once you click "Submit", the experiment will be started.
(see attached capture of the image named runCustomImage.jpg)
Step 6. Make image available
To make the image available to other experimenters via the AM API send a request to the site administrator, who will forward the request to instageni-ops@flux.utah.edu.
Changed 12 years ago by
Attachment: | BeginNSExperiment.jpg added |
---|
Changed 12 years ago by
Attachment: | customUbuntu1204.jpg added |
---|
Changed 12 years ago by
Attachment: | customUbuntu1204CreateDescriptor.jpg added |
---|
Changed 12 years ago by
Attachment: | nodesetup.jpg added |
---|
Changed 12 years ago by
Attachment: | runCustomImage.jpg added |
---|
Changed 12 years ago by
Attachment: | testCustomImage.jpg added |
---|
comment:4 Changed 12 years ago by
The custom Ubuntu image has been used in several experiments.
Only documentation item remains for defining site admin responsibilities, as stated in email:
On 5/23/12 11:08 AM, Leigh Stoller wrote:
- The tutorial custom image instructions did not give any indication that
the image could be shared with others. How will the experimenter know? Should this be mentioned in the tutorial?
- How will the site admins know that they need to forward the request to
instageni-ops?
I've added a todo item to document this ...
Lbs
comment:5 Changed 12 years ago by
Resolution: | → fixed |
---|---|
Status: | new → closed |
We implemented an image-creation call as an extension to the GENI AM API. Leigh's mail about this was:
I added a new method to the CM (and AM) API, which allows a geni user to create a sliver, customize the node, and then take a snapshot (possibly creating a new image descriptor) without having to use the Emulab web interface. The API looks like: string CreateImage(slice_urn, imagename, sliver_urn, credentials[]); Imagename is just an identifier like "myfedora" or whatever. The slice must be unlocked and the sliver in the ready state. A new image descriptor will be created if it does not exist, and then the node will be rebooted and a snapshot taken. When complete, the node is rebooted again. The return value is the urn of the image. For example: urn:publicid:IDN+emulab.net+image+pgeni-gpolab-bbn-com:myfedora which is how you request the new image in your rspec. Once the operation starts, the slice is locked until the backend finishes. This is something that I might revisit later, but this was the easiest approach that ensures consistency. It means you cannot ask for sliverstatus either (another issue to address someday) for a while. The image is written to a file on the NFS server that is accessible from the node. Look in /proj/$pid/images. Note that these images are not going to show up in discover since they are private. You can still request them using the URN above, but unless the image is "exported" by an admin, it is hidden and only available to slices created belonging to the same SA.
Capturing responses by Leigh
On 5/16/12 10:28 AM, Leigh Stoller wrote:
On 5/16/12 10:39 AM, Leigh Stoller wrote:
Thank you, now reading https://users.emulab.net/trac/emulab/wiki/Tutorial#CustomOSImages