Changes between Initial Version and Version 1 of OperationalMonitoring/DataSchema20


Ignore:
Timestamp:
08/26/14 22:20:49 (7 years ago)
Author:
dwiggins@bbn.com
Comment:

update for release 2.0

Legend:

Unmodified
Added
Removed
Modified
  • OperationalMonitoring/DataSchema20

    v1 v1  
     1[[PageOutline]]
     2
     3== REST Data Schema ==
     4All of these inherit from the [http://unis.incntre.iu.edu/schema/20120709/networkresource network resource schema] with the exception of the opsconfig schema (which inherits from [http://unis.incntre.iu.edu/schema/20120709/domain domain schema]).
     5
     6 * [http://www.gpolab.bbn.com/monitoring/schema/20140501/opsconfig]: operational configuration metadata, including locations of and information about other datastores
     7 * [http://www.gpolab.bbn.com/monitoring/schema/20140501/aggregate]: metadata about an aggregate (including lists of resources and slivers at that aggregate, and where to look for measurements about the aggregate)
     8 * [http://www.gpolab.bbn.com/monitoring/schema/20140501/node]: metadata about a resources (including resource properties termed "config" above, and a list of relevant ports (or of all ports, whichever is easier) on that resource)
     9 * [http://www.gpolab.bbn.com/monitoring/schema/20140501/port]: metadata about a network interface (including resource properties termed "config" above)
     10 * [http://www.gpolab.bbn.com/monitoring/schema/20140501/port-vlan]: metadata about a tagged VLAN subinterface of a network interface
     11 * [http://www.gpolab.bbn.com/monitoring/schema/20140501/sliver]: metadata about a sliver at an aggregate, including resources to which that sliver is mapped
     12 * [http://www.gpolab.bbn.com/monitoring/schema/20140501/authority]: metadata about a GENI authority (including lists of GENI users and slivers at that authority)
     13 * [http://www.gpolab.bbn.com/monitoring/schema/20140501/slice]: metadata about a slice at an authority, including GENI users with roles on that slice
     14 * [http://www.gpolab.bbn.com/monitoring/schema/20140501/user]: metadata about a GENI user at an authority, including contact information
     15 * [http://www.gpolab.bbn.com/monitoring/schema/20140501/experiment]: metadata about an operational monitoring experiment for monitoring the dataplane.
     16 * [http://www.gpolab.bbn.com/monitoring/schema/20140501/data]: Schema based on a combination of [http://unis.incntre.iu.edu/schema/20120709/metadata] and [http://unis.incntre.iu.edu/schema/20120709/tsdatum], contains both metadata and data about measurements.
     17 * For now, we would use the `ops_monitoring` namespace for operations monitoring, meaning:
     18   * When we add monitoring-relevant optional properties to objects, we'll put them in an `ops_monitoring` dictionary
     19   * When we setup operations monitoring measurements, we'll give them the eventType `ops_monitoring:<something>`
     20
     21=== Data schema usage example ===
     22
     23Some examples usages of the above schemas to encode metadata and data needed for use cases 3 and 6 follow.
     24
     25These examples assume the following (fictitious) local datastore URLs.  These are arbitrary, and any place they appear, they can be replaced by whatever name the deployers prefer.  For simplicitly, i've shown one local datastore per aggregate here, but the architecture does ''not'' require that --- it would be perfectly fine to have one datastore for relational metadata and one for the data itself, or one for certain types of relational metadata and one for others.
     26 * `https://datastore.geni.net/`: datastore containing configuration data for operational monitoring
     27 * `https://datastore.instageni.gpolab.bbn.com/`: datastore for gpo-ig aggregate
     28 * `https://datastore.ch.geni.net/`: datastore for ch.geni.net authority
     29 * `https://datastore.externalchecks.geni.net/`: datastore for external checks in the monitoring framework (e.g. inter-aggregate pings, AM external availability)
     30
     31==== Data about operational monitoring configuration ====
     32
     33Operational monitoring configuration data tells collectors where to find local datastores, and includes relevant metadata (like URNs and metadata about datastore or aggregate types) that collectors can use to decide which datastores to query.  It is described using [http://www.gpolab.bbn.com/monitoring/schema/20140501/opsconfig# the opsconfig schema].  Examples:
     34 * geni-prod: a hypothetical config datastore listing production aggregates and authorities:
     35{{{
     36{
     37  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/opsconfig#",
     38  "id": "geni-prod",
     39  "selfRef": "https://datastore.geni.net/opsconfigs/geni-prod",
     40  "ts": 1391192685740849,
     41  "aggregates": [
     42    {
     43      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm",
     44      "amtype": "protogeni",
     45      "href": "https://datastore.instageni.gpolab.bbn.com/aggregates/gpo-ig"
     46    }
     47  ],
     48  "authorities": [
     49    {
     50      "urn": "urn:publicid:IDN+ch.geni.net+authority+ch",
     51      "href": "https://datastore.ch.geni.net/authorities/ch.geni.net"
     52    }
     53  ]
     54}
     55}}}
     56
     57==== Data about an aggregate ====
     58
     59Aggregates are indexed by GENI-agreed short name and described using [http://www.gpolab.bbn.com/monitoring/schema/20140501/aggregate# the aggregate schema].  Examples:
     60
     61Hypothetical REST call for GPO-IG:
     62{{{
     63https://datastore.instageni.gpolab.bbn.com/info/aggregate/gpo-ig
     64}}}
     65 Response:
     66{{{
     67{
     68  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/aggregate#",
     69  "id": "gpo-ig",
     70  "selfRef": "https://datastore.instageni.gpolab.bbn.com/info/aggregate/gpo-ig",
     71  "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm",
     72  "ts": 1391192685740849,
     73  "measRef": "https://datastore.instageni.gpolab.bbn.com/data",
     74  "resources": [
     75    {
     76      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1",
     77      "href": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1"
     78    },
     79    {
     80      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2",
     81      "href": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc2"
     82    },
     83    {
     84      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+link_001",
     85      "href": "https://datastore.instageni.gpolab.bbn.com/info/link/instageni.gpolab.bbn.com_link_001"
     86    }
     87  ],
     88  "slivers": [
     89    {
     90      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+26947",
     91      "href": "https://datastore.instageni.gpolab.bbn.com/info/sliver/instageni.gpolab.bbn.com_sliver_26947"
     92    }
     93  ]
     94}
     95}}}
     96
     97==== Data about a node ====
     98
     99Nodes have an ID which is a URL-sanitized version of their URN and are described using [http://unis.incntre.iu.edu/schema/20140501/node# the node schema].  Examples:
     100 Hypothetical REST call for GPO-IG PC1:
     101{{{
     102https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1
     103}}}
     104 Response:
     105{{{
     106{
     107  "$schema": "http://unis.incntre.iu.edu/schema/20140501/node#",
     108  "id": "instageni.gpolab.bbn.com_node_pc1",
     109  "ts": 1391192705275101,
     110  "selfRef": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1",
     111  "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1",
     112  "ops_monitoring:mem_total_kb": 50331648,
     113  "ops_monitoring:vm_server_type": "xen",
     114  "ports": [
     115    {
     116      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth0",
     117      "href": "https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1%3Aeth0"
     118    },
     119    {
     120      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth1",
     121      "href": "https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1%3Aeth1"
     122    },
     123    {
     124      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth2",
     125      "href": "https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1%3Aeth2"
     126    },
     127    {
     128      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth3",
     129      "href": "https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1%3Aeth3"
     130    }
     131  ]
     132}
     133}}}
     134
     135==== Data about a link ====
     136
     137Links have an ID which is a URL-sanitized version of their URN and are described using [http://www.gpolab.bbn.com/monitoring/schema/20140501/link# the link schema].  Examples:
     138 Hypothetical REST call for GRNOC link salt_port_1750:
     139{{{
     140https://datastore.grnoc.iu.edu/info/link/salt_port_1750
     141}}}
     142 Response:
     143{{{
     144{
     145  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/link#",
     146  "id": "datastore.grnoc.iu.edu_link_001",
     147  "ts": 1391192705275101,
     148  "selfRef": "https://datastore.grnoc.iu.edu/info/link/salt_port_1750",
     149  "urn": "urn:publicid:IDN+datastore.grnoc.iu.edu+salt_port_1750",
     150  },
     151  "endpoints": [
     152    {
     153      "urn": "urn:publicid:IDN+al2s.net.internet2.edu+interface+sdn-sw.salt.net.internet2.edu_100GigabitEthernet3_2_1750",
     154      "href": "https://datastore.grnoc.iu.edu/info/interfacevlan/sdn-sw.salt.net.internet2.edu_100GigabitEthernet3_2_1750"
     155    },
     156    {
     157      "urn": "urn:publicid:IDN+al2s.net.internet2.edu+interface+sdn-sw.port.net.internet2.edu_100GigabitEthernet3_2_1750",
     158      "href": "https://datastore.grnoc.iu.edu/info/interfacevlan/sdn-sw.port.net.internet2.edu_100GigabitEthernet3_2_1750"   
     159    }
     160  ],
     161  "l2path": [ 
     162    {
     163      "urn": "urn:publicid:IDN+al2s.net.internet2.edu+interface+sdn-sw.salt.net.internet2.edu_100GigabitEthernet3_2_1750",
     164      "href": "https://datastore.grnoc.iu.edu/info/interfacevlan/sdn-sw.salt.net.internet2.edu_100GigabitEthernet3_2_1750"
     165    },
     166    {
     167      "urn": "urn:publicid:IDN+al2s.net.internet2.edu+interface+sdn-sw.salt.net.internet2.edu_100GigabitEthernet3_1_47",
     168      "href": "https://datastore.grnoc.iu.edu/info/interfacevlan/sdn-sw.salt.net.internet2.edu_100GigabitEthernet3_1_47"
     169    },
     170    {
     171      "urn": "urn:publicid:IDN+al2s.net.internet2.edu+interface+sdn-sw.seat.net.internet2.edu_100GigabitEthernet3_2_47",
     172      "href": "https://datastore.grnoc.iu.edu/info/interfacevlan/sdn-sw.seat.net.internet2.edu_100GigabitEthernet3_2_47"   
     173    },
     174    {
     175      "urn": "urn:publicid:IDN+al2s.net.internet2.edu+interface+sdn-sw.seat.net.internet2.edu_100GigabitEthernet3_1_38",
     176      "href": "https://datastore.grnoc.iu.edu/info/interfacevlan/sdn-sw.seat.net.internet2.edu_100GigabitEthernet3_1_38"   
     177    },
     178    {
     179      "urn": "urn:publicid:IDN+al2s.net.internet2.edu+interface+sdn-sw.port.net.internet2.edu_100GigabitEthernet3_1_38",
     180      "href": "https://datastore.grnoc.iu.edu/info/interfacevlan/sdn-sw.port.net.internet2.edu_100GigabitEthernet3_1_38"   
     181    },
     182    {
     183      "urn": "urn:publicid:IDN+al2s.net.internet2.edu+interface+sdn-sw.port.net.internet2.edu_100GigabitEthernet3_2_1750",
     184      "href": "https://datastore.grnoc.iu.edu/info/interfacevlan/sdn-sw.port.net.internet2.edu_100GigabitEthernet3_2_1750"   
     185    }
     186  ]
     187}
     188}}}
     189
     190
     191==== Data about an interface ====
     192
     193Interfaces have an ID which is a URL-sanitized version of their URN and are described using [http://unis.incntre.iu.edu/schema/20120709/port# the port schema].  Notes:
     194 * I adopted the control/experimental terminology for interface roles from ProtoGENI listresources output.  We could also use control/data; at any rate, we should be consistent among all monitoring uses.
     195 * All bandwidths are fiction.
     196
     197Examples:
     198
     199 Hypothetical REST call for pc1 eth0 (control) at gpo-ig:
     200{{{
     201https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1:eth0
     202}}}
     203 Response:
     204{{{
     205{
     206  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/port#",
     207  "selfRef": "https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1:eth0",
     208  "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth0",
     209  "ts": 1391194147100678,
     210  "id": "instageni.gpolab.bbn.com_interface_pc1:eth0",
     211  "address": {
     212    "type": "ipv4",
     213    "address": "192.1.242.140"
     214  },
     215  "ops_monitoring:role": "control",
     216  "ops_monitoring:max_bps": 10000000,
     217  "ops_monitoring:max_pps": 1000000
     218}
     219}}}
     220
     221 Hypothetical REST call for pc1 eth1 (dataplane) interface at gpo-ig:
     222
     223{{{
     224https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1:eth1
     225}}}
     226 Response:
     227{{{
     228{
     229  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/port#",
     230  "selfRef": "https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1:eth1",
     231  "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth1",
     232  "ts": 1391194147100678,
     233  "id": "instageni.gpolab.bbn.com_interface_pc1:eth1",
     234  "address": {
     235    "type": "mac",
     236    "address": "aa:aa:aa:aa:aa:ab"
     237  },
     238  "ops_monitoring:role": "experimental",
     239  "ops_monitoring:max_bps": 10000000,
     240  "ops_monitoring:max_pps": 1000000
     241}
     242}}}
     243
     244==== Data about a slice authority ====
     245
     246GENI slice authorities are indexed by domain name and described using [http://www.gpolab.bbn.com/monitoring/schema/20140501/authority# the authority schema].  Examples:
     247
     248
     249 Hypothetical REST call for authority of ch.geni.net:
     250{{{
     251https://datastore.ch.geni.net/info/authority/ch.geni.net
     252}}}
     253 Response:
     254{{{
     255{
     256  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/authority#",
     257  "id": "ch.geni.net",
     258  "selfRef": "https://datastore.ch.geni.net/info/authority/ch.geni.net",
     259  "urn": "urn:publicid:IDN+ch.geni.net+authority+ch",
     260  "ts": 1391192685740849,
     261  "users": [
     262    {
     263      "urn": "urn:publicid:IDN+ch.geni.net+user+tupty",
     264      "href": "https://datastore.ch.geni.net/info/user/tupty"
     265    }
     266  ],
     267  "slices": [
     268    {
     269      "urn": "urn:publicid:IDN+ch.geni.net:gpo-infra+slice+tuptyexclusive",
     270      "href": "https://datastore.ch.geni.net/info/slice/ch.geni.net_gpo-infra_slice_tuptyexclusive"
     271    }
     272  ]
     273}
     274}}}
     275
     276
     277==== Data about a VLAN ====
     278
     279Examples:
     280
     281 Hypothetical REST call for VLAN 101 from nox to core controlled by ION (probably an incorrect example):
     282{{{
     283https://datastore.instageni.gpolab.bbn.com/info/interfacevlan/instageni.gpolab.bbn.com_interface_pc1:eth1:1750
     284}}}
     285 Response:
     286{{{
     287{
     288  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/port-vlan#",
     289  "selfRef": "https://datastore.instageni.gpolab.bbn.com/interfacevlan/instageni.gpolab.bbn.com_interface_pc1:eth1:1750",
     290  "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth1:1750",
     291  "ts": 1391194147100678,
     292  "id": "instageni.gpolab.bbn.com_interface_pc1:eth1:1750",
     293  "tag": 1750,
     294  "port": {
     295    "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth1",
     296    "href": "https://datastore.instageni.gpolab.bbn.com/ports/instageni.gpolab.bbn.com_interface_pc1:eth1"
     297  }
     298}
     299}}}
     300
     301
     302==== Data about a GENI user ====
     303
     304GENI users have an ID based on the username and are described using [http://www.gpolab.bbn.com/monitoring/schema/20140501/user# the GENI user schema].  Examples:
     305
     306 Hypothetical REST call for user tupty at ch.geni.net:
     307{{{
     308 https://datastore.ch.geni.net/info/user/tupty
     309}}}
     310 Response:
     311{{{
     312{
     313  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/user#",
     314  "id": "tupty",
     315  "selfRef": "https://datastore.ch.geni.net/info/user/tupty",
     316  "urn": "https://datastore.ch.geni.net/info/user/tupty",
     317  "ts": 1391192685740849,
     318  "authority": {
     319    "urn": "urn:publicid:IDN+ch.geni.net+authority+ch",
     320    "href": "https://datastore.ch.geni.net/info/authority/ch.geni.net"
     321  },
     322  "fullname": "Tim Exampleuser",
     323  "email": "tim@example.com"
     324}
     325}}}
     326
     327==== Data about a GENI slice ====
     328
     329GENI slices have an ID based on the URN and are described using [http://www.gpolab.bbn.com/monitoring/schema/20140501/slice# the GENI slice schema].  Examples:
     330Hypothetical REST call for tuptyexclusive slice:
     331{{{
     332 https://datastore.ch.geni.net/info/slice/ch.geni.net_gpo-infra_slice_tuptyexclusive
     333}}}
     334
     335{{{
     336{
     337  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/slice#",
     338  "id": "ch.geni.net_gpo-infra_slice_tuptyexclusive",
     339  "selfRef": "https://datastore.ch.geni.net/info/slice/ch.geni.net_gpo-infra_slice_tuptyexclusive",
     340  "urn": "urn:publicid:IDN+ch.geni.net:gpo-infra+slice+tuptyexclusive",
     341  "uuid": "8c6b97fa-493b-400f-95ee-19accfaf4ae8",
     342  "ts": 1391192685740849,
     343  "authority": {
     344    "urn": "urn:publicid:IDN+ch.geni.net+authority+ch",
     345    "href": "https://datastore.ch.geni.net/info/authority/ch.geni.net"
     346  },
     347  "created": 1391626683000000,
     348  "expires": 1391708989000000,
     349  "members": [
     350    {
     351      "urn": "urn:publicid:IDN+ch.geni.net+user+tupty",
     352      "href": "https://datastore.ch.geni.net/info/users/tupty",
     353      "role": "lead"
     354    }
     355  ]
     356}
     357}}}
     358
     359==== Data about a GENI sliver ====
     360
     361GENI sliver have an ID based on the URN and are described using [http://www.gpolab.bbn.com/monitoring/schema/20140501/sliver# the GENI sliver schema].  Examples:
     362 Hypothetical REST call for tuptyexclusive instageni sliver:
     363{{{
     364https://datastore.instageni.gpolab.bbn.com/info/sliver/instageni.gpolab.bbn.com_sliver_26947
     365}}}
     366 Response:
     367{{{
     368{
     369  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/sliver#",
     370  "id": "instageni.gpolab.bbn.com_sliver_26947",
     371  "selfRef": "https://datastore.instageni.gpolab.bbn.com/info/sliver/instageni.gpolab.bbn.com_sliver_26947",
     372  "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+26947",
     373  "uuid": "30752b06-8ea8-11e3-8d30-000000000000",
     374  "ts": 1391192685740849,
     375  "aggregate": {
     376    "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm",
     377    "href": "https://datastore.instageni.gpolab.bbn.com/info/aggregates/gpo-ig"
     378  },
     379  "slice_urn": "urn:publicid:IDN+ch.geni.net:gpo-infra+slice+tuptyexclusive",
     380  "slice_uuid": "8c6b97fa-493b-400f-95ee-19accfaf4ae8",
     381  "creator": "urn:publicid:IDN+ch.geni.net+user+tupty",
     382  "created": 1391626683000000,
     383  "expires": 1391708989000000,
     384  "resources": [
     385    {
     386      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1",
     387      "href": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1",
     388    },
     389    {
     390      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2",
     391      "href": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc2",
     392    }
     393  ]
     394}
     395}}}
     396
     397==== Data about an external check datastore ====
     398
     399{{{
     400https://externalcheckstore.gpolab.bbn.com/info/externalcheck/gpo
     401}}}
     402
     403{{{
     404{
     405   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/externalcheck#",
     406   "id": "gpo",
     407   "selfRef": "https://externalcheckstore.gpolab.bbn.com/info/externalcheck/gpo",
     408   "ts": 1397762559284773,
     409   "measRef: "https://externalcheckstore.gpolab.bbn.com/data/",
     410   "experiments": [
     411      {
     412         "href": "https://externalcheckstore.gpolab.bbn.com/info/experiment/missouri_ig_to_gpo_ig"
     413      }
     414   ],
     415   "monitored_aggregates": [
     416      {
     417         "id": "gpo-ig",
     418         "href": "https://datastore.instageni.gpolab.bbn.com/info/aggregate/gpo-ig"
     419      },
     420      {
     421         "id": "rci-eg",
     422         "href": "https://rci-hn.exogeni.net/ops-monitoring/info/aggregate/rci-eg"
     423      }
     424   ]
     425}
     426}}}
     427==== Data about a monitoring experiment ====
     428
     429Operational monitoring measures the dataplane with a set of simple tests between aggregates described using [http://www.gpolab.bbn.com/monitoring/schema/20140501/experiment# the operational monitoring experiment schema].  Examples:
     430 Hypothetical REST call for missouri-ig to gpo-ig experiment:
     431{{{
     432https://datastore.externalchecks.geni.net/info/experiment/missouri-ig_to_gpo-ig
     433}}}
     434 Response:
     435{{{
     436{
     437  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/experiment#",
     438  "id": "missouri-ig_to_gpo-ig",
     439  "selfRef": "https://datastore.externalchecks.geni.net/info/experiment/missouri-ig_to_gpo-ig",
     440  "slice_urn": "urn:publicid:IDN+ch.geni.net:gpo-infra+slice+tuptyexclusive",
     441  "slice_uuid": "8c6b97fa-493b-400f-95ee-19accfaf4ae8",
     442  "ts": 1391192685740849,
     443  "source_aggregate": {
     444    "urn": "urn:publicid:IDN+instageni.missouri.edu+authority+cm",
     445    "href": "https://datastore.instageni.missouri.edu/info/aggregates/missouri-ig"
     446  },
     447  "destination_aggregate": {
     448    "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm",
     449    "href": "https://datastore.instageni.gpolab.bbn.com/info/aggregates/gpo-ig"
     450  }
     451}
     452}}}
     453
     454
     455==== Measurements used for the use cases ====
     456
     457Measurements have an opaque ID which is generated by the local datastore which serves them, and must be persistent, so that the caller has the option of asking for the measurement by ID.  They are described using the data schema outlined above.  Examples:
     458Hypothetical REST call for CPU utilization metric on pc1:
     459{{{
     460https://datastore.instageni.gpolab.bbn.com/data/?q={"filters":{"eventType": ["ops_monitoring:cpu_util"],"ts":{"gte":0, "lte": 1391199016651285},"obj":{"type":"node","id":["instageni.gpolab.bbn.com_node_pc1"]}}}
     461}}}
     462Response
     463{{{
     464{
     465  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/data#",
     466  "id": "cpu_util:instageni.gpolab.bbn.com_node_pc1",
     467  "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1",
     468  "eventType": "ops_monitoring:cpu_util",
     469  "description": "CPU utilization percentage",
     470  "units": "percent",
     471  "tsdata": [
     472    { "ts": 1391198716651283, "v": 45 },
     473    { "ts": 1391198776651284, "v": 44 },
     474    { "ts": 1391198836651284, "v": 44 },
     475    { "ts": 1391198896651284, "v": 47 },
     476    { "ts": 1391198956651284, "v": 46 },
     477    { "ts": 1391199016651285, "v": 47 }
     478  ]
     479}
     480}}}
     481Hypothetical REST call for percentage of swap available on pc1:
     482{{{
     483https://datastore.instageni.gpolab.bbn.com/data/?q={"filters":{"eventType": ["ops_monitoring:swap_free"],"ts":{"gt":0,"lt":1391199016651285},"obj":{"type":"node","id":["instageni.gpolab.bbn.com_node_pc1"]}}}
     484
     485}}}
     486Response:
     487{{{
     488{
     489  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/data#",
     490  "id": "swap_free:instageni.gpolab.bbn.com_node_pc1",
     491  "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1",
     492  "eventType": "ops_monitoring:swap_free",
     493  "description": "Percentage of swap available",
     494  "units": "percent",
     495  "tsdata": [
     496    { "ts": 1391198716651283, "v": 95 },
     497    { "ts": 1391198776651284, "v": 95 },
     498    { "ts": 1391198836651284, "v": 95 },
     499    { "ts": 1391198896651284, "v": 95 },
     500    { "ts": 1391198956651284, "v": 95 },
     501    { "ts": 1391199016651285, "v": 95 }
     502  ]
     503}
     504}}}
     505Hypothetical REST call for memory in active use on pc1:
     506{{{
     507https://datastore.instageni.gpolab.bbn.com/data/?q={"filters":{"eventType": ["ops_monitoring:mem_active_kb"],"ts":{"gte":0, "lt": 1391299016651285},"obj":{"type":"node","id":["instageni.gpolab.bbn.com_node_pc1"]}}}
     508}}}
     509Response:
     510{{{
     511{
     512  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/data#",
     513  "id": "mem_active_kb:instageni.gpolab.bbn.com_node_pc1",
     514  "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1",
     515  "eventType": "ops_monitoring:mem_active_kb",
     516  "description": "Memory in active use",
     517  "units": "integer",
     518  "tsdata": [
     519    { "ts": 1391198716651283, "v": 20030048 },
     520    { "ts": 1391198776651284, "v": 20031148 },
     521    { "ts": 1391198836651284, "v": 20031148 },
     522    { "ts": 1391198896651284, "v": 22222222 },
     523    { "ts": 1391198956651284, "v": 22222222 },
     524    { "ts": 1391199016651285, "v": 22222222 }
     525  ]
     526}
     527}}}
     528 Hypothetical call for bytes per second received by `pc1:eth0`:
     529{{{
     530https://datastore.instageni.gpolab.bbn.com/data/?q={"filters":{"eventType": ["ops_monitoring:rx_bytes"],"ts":{"gte":0, "lte":1391299016651285},"obj":{"type":"interface","id":["instageni.gpolab.bbn.com_node_pc1:eth0"]}}}
     531}}}
     532Response:
     533{{{
     534{
     535  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/data#",
     536  "id": "rx_bps:instageni.gpolab.bbn.com_interface_pc1:eth0",
     537  "subject": "https://datastore.instageni.gpolab.bbn.com/info/ports/instageni.gpolab.bbn.com_interface_pc1:eth0",
     538  "eventType": "ops_monitoring:rx_bps",
     539  "description": "bytes per second received on this interface",
     540  "units": "float",
     541  "tsdata": [
     542    { "ts": 1391198716651283, "v": 2453.64 },
     543    { "ts": 1391198776651284, "v": 800.2 },
     544    { "ts": 1391198836651284, "v": 2400.3 },
     545    { "ts": 1391198896651284, "v": 1984.3 },
     546    { "ts": 1391198956651284, "v": 0 },
     547    { "ts": 1391199016651285, "v": 0 }
     548  ]
     549}
     550}}}
     551Hypothetical REST call for boolean metric indicating whether pc1 is available for use according to the aggregate responsible for it:
     552{{{
     553https://datastore.instageni.gpolab.bbn.com/data/?q={"filters":{"eventType": ["ops_monitoring:is_available"],"ts":{"gte":0, "lte":1391299016651285},"obj":{"type":"node","id":["instageni.gpolab.bbn.com_node_pc1"]}}}
     554}}}
     555Response:
     556{{{
     557{
     558  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/data#",
     559  "id": "is_available:instageni.gpolab.bbn.com_node_pc1",
     560  "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1",
     561  "eventType": "ops_monitoring:is_available",
     562  "description": "is the subject node available, according to the aggregate",
     563  "units": "boolean",
     564  "tsdata": [
     565    { "ts": 1391198716651283, "v": 1 },
     566    { "ts": 1391198776651284, "v": 1 },
     567    { "ts": 1391198836651284, "v": 1 },
     568    { "ts": 1391198896651284, "v": 1 },
     569    { "ts": 1391198956651284, "v": 0 },
     570    { "ts": 1391199016651285, "v": 0 }
     571  ]
     572}
     573}}}
     574
     575==== Bulk Data Queries ====
     576Queries on multiple eventTypes and object ID's will be presented in a list format.  Here is an example query for event types cpu utilization and active memory utilization for nodes pc1 and pc2 of instageni-bbn.
     577Also it is advised to provide timestamp filters with data queries.  Here is the format for [wiki:OperationalMonitoring/DatastorePolling#DataQueries timestamp filters] .
     578
     579Example call:
     580{{{
     581https://datastore.instageni.gpolab.bbn.com/data/?q={"filters":{"eventType": ["ops_monitoring:mem_used","ops_monitoring:cpu_util"],"ts":{"gte":1391192225475202,"lt":1391192225480000},"obj":{"type":"node","id":["instageni.gpolab.bbn.com_node_pc1","instageni.gpolab.bbn.com_node_pc2"]}}}}
     582}}}
     583
     584{{{
     585[
     586 {
     587   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/data#",
     588    "id": "cpu_util:instageni.gpolab.bbn.com_node_pc1",
     589    "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1",
     590    "eventType": "ops_monitoring:cpu_util",
     591    "description": "CPU utilization percentage",
     592    "units": "percent",
     593    "tsdata": [
     594      { "ts": 1391198716651283, "v": 45 },
     595      { "ts": 1391198776651284, "v": 44 },
     596      { "ts": 1391198836651284, "v": 44 },
     597      { "ts": 1391198896651284, "v": 47 },
     598      { "ts": 1391198956651284, "v": 46 },
     599      { "ts": 1391199016651285, "v": 47 }
     600    ]
     601 }
     602,
     603 {
     604   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/data#",
     605   "id": "cpu_util:instageni.gpolab.bbn.com_node_pc2",
     606   "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc2",
     607   "eventType": "ops_monitoring:cpu_util",
     608   "description": "CPU utilization percentage",
     609   "units": "percent",
     610   "tsdata": [
     611     { "ts": 1391198716651283, "v": 45 },
     612     { "ts": 1391198776651284, "v": 48 },
     613     { "ts": 1391198836651284, "v": 45 },
     614     { "ts": 1391198896651284, "v": 49 },
     615     { "ts": 1391198956651284, "v": 50 },
     616     { "ts": 1391199016651285, "v": 51 }
     617    ]
     618 }
     619,
     620 {
     621  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/data#",
     622  "id": "mem_active_kb:instageni.gpolab.bbn.com_node_pc1",
     623  "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1",
     624  "eventType": "ops_monitoring:mem_active_kb",
     625  "description": "Memory in active use",
     626  "units": "integer",
     627  "tsdata": [
     628    { "ts": 1391198716651283, "v": 30030048 },
     629    { "ts": 1391198776651284, "v": 30031148 },
     630    { "ts": 1391198836651284, "v": 30031148 },
     631    { "ts": 1391198896651284, "v": 32222222 },
     632    { "ts": 1391198956651284, "v": 32222222 },
     633    { "ts": 1391199016651285, "v": 32222222 }
     634  ]
     635 }
     636,
     637 {
     638  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140501/data#",
     639  "id": "mem_active_kb:instageni.gpolab.bbn.com_node_pc2",
     640  "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc2",
     641  "eventType": "ops_monitoring:mem_active_kb",
     642  "description": "Memory in active use",
     643  "units": "integer",
     644  "tsdata": [
     645    { "ts": 1391198716651283, "v": 20030048 },
     646    { "ts": 1391198776651284, "v": 20031148 },
     647    { "ts": 1391198836651284, "v": 20031148 },
     648    { "ts": 1391198896651284, "v": 22222222 },
     649    { "ts": 1391198956651284, "v": 22222222 },
     650    { "ts": 1391199016651285, "v": 22222222 }
     651  ]
     652 }
     653]
     654}}}