wiki:Dingbot

Version 1 (modified by Josh Smift, 6 years ago) (diff)

--

Dingbot

Dingbot is a post-boot script for bridging the gaps between different AMs and/or different node images. We expect those gaps to narrow over time, but meanwhile, it's an example of how to get a more uniform experience out of some not-entirely-uniform resources. At GEC 17, it was mentioned in the Uniform Experimenter Experience session, and described in more detail at the Experimenter Roundtable session; see those pages for slides.

It's named after Agatha Heterodyne's clank minion Dingbot Prime. (That page contains spoilers for Girl Genius, if you care about that sort of thing.)

Background

Dingbot's purpose is to run on a newly-provisioned node, and do any one-time setup tasks that aren't otherwise done for you automatically. To use it, you'd put something like

<services>
  <install url="CONFIG-URL" install_path="/tmp" />
  <install url="DINGBOT-URL" install_path="/tmp" />
  <execute shell="/bin/bash" command="sudo /tmp/dingbot/dingbot CONFIG-FILE HOSTNAME"/>
</services>

into the <node> section(s) of your rspec, where:

  • DINGBOT-URL is the URL of the a tarball containing Dingbot
  • CONFIG-URL is the URL of a tarball containing a config file to download and use
  • CONFIG-FILE is the path to the config file (so this needs to match the filename in the tarball, and the location where you're unpacking it)
  • HOSTNAME is the hostname that you want this system to use

It'd be better if Dingbot could deduce the hostname from something that's already in the rspec, like the client_id attribute, but right now the node doesn't have reliable access to that at boot time. (You can use $self.Name() on ExoGENI, via their Velocity template engine, to avoid having to hardcode in a hostname in your rspec.)

Dingbot needs to run as root; you can (and may have to) leave out the sudo on AMs where that's already the case (e.g. ExoGENI).

Operation

The script starts by checking for the presence of a /.dingbot-done file (and just prints a message and exits, if one exists), and finishes by creating that file, so it should only ever actually do anything once, even though it runs every time the node boots. If it does do anything, it logs to dingbot-{output,errors}-$TIMESTAMP.log files in /var/log.

The simplest way to see what it does is to look at the code, which is commented in astonishingly verbose detail.

...but which isn't yet available here; we plan to publish it soon, stay tuned.

Configuration

Dingbot uses a JSON configuration file to specify things that different experimenters might want to do differently, such as set up certain user accounts, or install certain software packages. You can also use it to do different things on different nodes in your experiment: If you always want to do the same things on all your nodes, you can use the same config file in all of your rspecs; or if you want different things on different nodes, you can have as many different config files as you want, just put the appropriate URL into the appropriate <node> section in each rspec.

Because the <install url=...> element expects a .tar.gz file, the config file also needs to be in a tarball.

A sample config file:

{
 "packages": [ "fping", "iperf", "rsync", "sudo" ],
 "users":
 {
  "jbs":
  {
   "username": "jbs",
   "fullname": "Josh Smift",
   "shell": "/bin/bash",
   "authorized_keys": "ssh-rsa [* Josh's key]"
  },
  "nriga":
  {
   "username": "nriga",
   "fullname": "Niky Riga",
   "shell": "/bin/bash",
   "authorized_keys": "ssh-rsa [* Niky's key]"
  },
  "mbrinn":
  {
   "username": "mbrinn",
   "fullname": "Marshall Brinn",
   "shell": "/bin/bash",
   "authorized_keys": "ssh-rsa [* Marshall's key]"
  }
 }
}

The names of the packages in the list are handed to YUM/APT/etc for installation. It doesn't gracefully handle packages with different names on different OSes/distributions, yet.

Dingbot creates accounts for the listed users if they don't exist already. The only change it makes to existing accounts is to set the user's shell as specified in the config file; it doesn't change anything else about existing accounts, even if the Dingbot config differs from the current state on the system.

Future work

Dingbot makes various simplifying assumptions about its environment; see the code for more details. (The word "FIXME" is a good indicator.)

One significant limitation of Dingbot is that it only runs once, so if it encounters any transient failures (e.g. it can't install software packages because the node temporarily can't reach the package repo, for whatever reason), it doesn't automatically retry or anything. Using a real system configuration management system (like Cfengine, Puppet, Chef, Salt, etc) could make this more robust.

Other more comprehensive experimenter tools include post-boot scripts as well, and do everything Dingbot does and a whole lot more. It was originally written to make life easier for one casual amateur experimenter, and isn't really intended to be a comprehensive solution for a wide range of people. But if it helps you use GENI, or serves as a starting point for your own ideas, or just amuses and entertains you, all the better.