gmoc.py download should allow a threshold to only download recently-reported data

[chaos@bbn.com]

At the moment, when data is downloaded using gmoc.py, the API searches for all matching data regardless of last_update time. This isn't sustainable in the long-term because of data size, and, in the short term, may get us into trouble with badly-formed data from the past.

Modify gmoc.py to allow an optional threshold argument to GMOCClient.load(), and only load values whose last_update time is more recent than that threshold.

[chaos@bbn.com]

Unfortunately, thresholds can't be universally used, because some data (like Organization objects) is almost never updated. Experimental data should be updated every 5 minutes or so, but we can't count on metadata to be treated that way. Similarly, we probably never want lookup of a mandatory attribute of an object to fail if that attribute is stale and the invoking object is non-stale.

My suggestion for these problems was:

• Whenever you're looking up parameters of an object using a threshold:
  • For any parameter for which the object has exactly one value (e.g. Aggregate.pop), ignore the threshold when looking up that object (but pass it along in case that object itself has other items to lookup)
  • For any parameter for which the object has 0-N values (a list, e.g. Aggregate.slivers), use the same threshold when selecting objects from the list, as was initially used in selecting the calling object.

[chaos@bbn.com]

This was implemented.