Changes between Version 14 and Version 15 of OperationalMonitoring/DataSchema20


Ignore:
Timestamp:
08/29/14 10:48:38 (10 years ago)
Author:
lnevers@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • OperationalMonitoring/DataSchema20

    v14 v15  
    1 [[PageOutline]]
    21
    3 == REST calls and JSON Replies for Monitoring Data ==
    4 
    5 Monitoring data collectors make REST calls to monitoring datastores.  Datastores reply with JSON data that conforms to a JSON schema that is specific to the type of REST call being responded to.
    6 All of these schemas inherit from the [http://unis.incntre.iu.edu/schema/20120709/networkresource UNIS network resource schema] with the exception of the opsconfig schema.
    7 This page gives examples of these REST calls and responses.
    8 These examples assume the following (fictitious) local datastore URLs.
    9 For simplicitly, we show one local datastore per aggregate here, but the architecture does ''not'' require that.  It is fine to have the data for an aggregate be split among multiple datastores.
    10  * `https://datastore.geni.net/`: datastore containing configuration data for operational monitoring
    11  * `https://datastore.instageni.gpolab.bbn.com/`: datastore for gpo-ig aggregate
    12  * `https://datastore.ch.geni.net/`: datastore for ch.geni.net authority
    13  * `https://datastore.externalchecks.geni.net/`: datastore for external checks in the monitoring framework (e.g. inter-aggregate pings, AM external availability)
    14 
    15 ==== Aggregate call and response ====
    16 
    17 Retrieves information about an aggregate, including lists of resources and slivers at that aggregate, version information, and where to look for measurements about the aggregate.
    18 
    19 General form of the REST call:
    20 {{{
    21 https://<datastore-host>/info/aggregate/<aggregate-id>
    22 }}}
    23 
    24 <aggregate-id> is usually the aggregate's GENI-assigned nickname (common name).
    25 
    26 Example REST call for aggregate gpo-ig:
    27 {{{
    28 https://datastore.instageni.gpolab.bbn.com/info/aggregate/gpo-ig
    29 }}}
    30 
    31 The response conforms to the [http://www.gpolab.bbn.com/monitoring/schema/20140828/aggregate# aggregate schema], [http://www.gpolab.bbn.com/monitoring/schema/20140828/aggregate.schema.jwc commented version of the aggregate schema].
    32 {{{
    33 {
    34   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/aggregate#",
    35   "id": "gpo-ig",
    36   "selfRef": "https://datastore.instageni.gpolab.bbn.com/info/aggregate/gpo-ig",
    37   "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm",
    38   "ts": 1391192685740849,
    39   "measRef": "https://datastore.instageni.gpolab.bbn.com/data",
    40   "monitoring_version": "v2.0",
    41   "operational_status": "production",
    42   "resources": [
    43     {
    44       "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1",
    45       "resource_type": "node",
    46       "href": "https://datastore.instageni.gpolab.bbn.com/nodes/instageni.gpolab.bbn.com_node_pc1"
    47     },
    48     {
    49       "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2",
    50       "resource_type": "node",
    51       "href": "https://datastore.instageni.gpolab.bbn.com/nodes/instageni.gpolab.bbn.com_node_pc2"
    52     }
    53   ],
    54   "slivers": [
    55     {
    56       "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+26947",
    57       "href": "https://datastore.instageni.gpolab.bbn.com/slivers/instageni.gpolab.bbn.com_sliver_36947"
    58     }
    59   ]
    60 }
    61 }}}
    62 
    63 ==== Node call and response ====
    64 
    65 Retrieves information about a node.
    66 
    67 General form of the REST call:
    68 {{{
    69 https://<datastore-host>/info/node/<node object id>
    70 }}}
    71 
    72 Example REST call for gpo-ig node pc1:
    73 {{{
    74 https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1
    75 }}}
    76 
    77 The response conforms to the [http://www.gpolab.bbn.com/monitoring/schema/20140828/node node schema], [http://www.gpolab.bbn.com/monitoring/schema/20140828/node.schema.jwc commented version of the node schema].
    78 {{{
    79 {
    80   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/node#",
    81   "id": "instageni.gpolab.bbn.com_node_pc1",
    82   "ts": 1391192705275101,
    83   "selfRef": "https://datastore.instageni.gpolab.bbn.com/nodes/instageni.gpolab.bbn.com_node_pc1",
    84   "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1",
    85   "ops_monitoring:mem_total_kb": 50331648,
    86   "node_type": "server",
    87   "virtualization_type": "xen",
    88   "interfaces": [
    89     {
    90       "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth0",
    91       "href": "https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1%3Aeth0"
    92     },
    93     {
    94       "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth1",
    95       "href": "https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1%3Aeth1"
    96     },
    97     {
    98       "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth2",
    99       "href": "https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1%3Aeth2"
    100     },
    101     {
    102       "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth3",
    103       "href": "https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1%3Aeth3"
    104     }
    105   ]
    106 }
    107 }}}
    108 
    109 ==== Link call and response ====
    110 
    111 Retrieves information about a link.
    112 
    113 General form of the REST call:
    114 {{{
    115 https://<datastore-host>/info/link/<link object id>
    116 }}}
    117 
    118 Example REST call for arbitrary_id_001:
    119 {{{
    120 https://datastore.grnoc.iu.edu/info/link/arbitrary_id_001
    121 }}}
    122 
    123 The response conforms to the [http://www.gpolab.bbn.com/monitoring/schema/20140828/link# link schema], [http://www.gpolab.bbn.com/monitoring/schema/20140828/link.schema.jwc commented version of the link schema].
    124 {{{
    125 {
    126   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/link#",
    127   "selfRef": "https://datastore.instageni.gpolab.bbn.com/info/link/arbitrary_id_001",
    128   "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+link+arbitrary_id_001",
    129   "ts": 1391194147100678,
    130   "id": "arbitrary_id_001",
    131   "endpoints": [
    132     {
    133       "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc2:eth1:1750",
    134       "href": "https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc2:eth1:1750"
    135     },
    136     {
    137       "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth1:1750",
    138       "href": "https://datastore.instageni.gpolab.bbn.com/info/interfacevlan/instageni.gpolab.bbn.com_interface_pc1:eth1:1750"
    139     }
    140   ]
    141 }
    142 }}}
    143 
    144 ==== Interface call and response ====
    145 
    146 Retrieves information about a network interface, a.k.a. port.
    147 
    148 General form of the REST call:
    149 {{{
    150 https://<datastore-host>/info/interface/<interface object id>
    151 }}}
    152 
    153 Example REST call for pc1 eth0 at gpo-ig:
    154 {{{
    155 https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1:eth0
    156 }}}
    157 
    158 The response conforms to the [http://www.gpolab.bbn.com/monitoring/schema/20140828/interface interface schema], [http://www.gpolab.bbn.com/monitoring/schema/20140828/interface.schema.jwc commented version of the interface schema].
    159 {{{
    160 {
    161   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/interface#",
    162   "selfRef": "https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1:eth0",
    163   "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth0",
    164   "ts": 1391194147100678,
    165   "id": "instageni.gpolab.bbn.com_interface_pc1:eth0",
    166   "addresses": [
    167     {
    168       "addrtype": "IPv4",
    169       "address": "192.1.242.140",
    170       "scope": "private"
    171     },
    172     {
    173       "addrtype": "IPv6",
    174       "address": "2001:cdba::3257:9652"
    175     }
    176   ],
    177   "ops_monitoring:role": "control",
    178   "ops_monitoring:max_bps": 10000000,
    179   "ops_monitoring:max_pps": 1000000
    180 }
    181 }}}
    182 
    183 ==== Interface-VLAN call and response ====
    184 
    185 Retrieves information about a tagged VLAN subinterface of a network interface.
    186 
    187 General form of the REST call:
    188 {{{
    189 https://<datastore-host>/info/interfacevlan/<interfacevlan object id>
    190 }}}
    191 
    192 Example REST call for VLAN 1750 on pc1 eth1:
    193 {{{
    194 https://datastore.instageni.gpolab.bbn.com/info/interfacevlan/instageni.gpolab.bbn.com_interface_pc1:eth1:1750
    195 }}}
    196 
    197 The response conforms to the  [http://www.gpolab.bbn.com/monitoring/schema/20140828/interfacevlan interfacevlan schema], [http://www.gpolab.bbn.com/monitoring/schema/20140828/interfacevlan.schema.jwc commented version of the interfacevlan schema].
    198 {{{
    199 {
    200   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/interfacevlan#",
    201   "selfRef": "https://datastore.instageni.gpolab.bbn.com/info/interfacevlan/instageni.gpolab.bbn.com_interface_pc1:eth1:1750",
    202   "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth1:1750",
    203   "ts": 1391194147100678,
    204   "id": "instageni.gpolab.bbn.com_interface_pc1:eth1:1750",
    205   "tag": 1750,
    206   "interface": {
    207     "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth1",
    208     "href": "https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1:eth1"
    209   }
    210 }
    211 }}}
    212 
    213 ==== Slice authority call and response ====
    214 
    215 Retrieves information about a slice authority.
    216 
    217 General form of the REST call:
    218 {{{
    219 https://<datastore-host>/info/slice/<slice object id>
    220 }}}
    221 
    222 Example REST call for authority of ch.geni.net:
    223 {{{
    224 https://datastore.ch.geni.net/info/authority/ch.geni.net
    225 }}}
    226 
    227 The response conforms to the [http://www.gpolab.bbn.com/monitoring/schema/20140828/authority authority schema], [http://www.gpolab.bbn.com/monitoring/schema/20140828/authority.schema.jwc commented version of the authority schema].
    228 {{{
    229 {
    230   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/authority#",
    231   "id": "ch.geni.net",
    232   "selfRef": "https://datastore.ch.geni.net/info/authority/ch.geni.net",
    233   "urn": "urn:publicid:IDN+ch.geni.net+authority+ch",
    234   "ts": 1391192685740849,
    235   "users": [
    236     {
    237       "urn": "urn:publicid:IDN+ch.geni.net+user+tupty",
    238       "href": "https://datastore.ch.geni.net/info/user/tupty"
    239     }
    240   ],
    241   "slices": [
    242     {
    243       "urn": "urn:publicid:IDN+ch.geni.net:gpo-infra+slice+tuptyexclusive",
    244       "href": "https://datastore.ch.geni.net/info/slice/ch.geni.net_gpo-infra_slice_tuptyexclusive"
    245     }
    246   ]
    247 }
    248 }}}
    249 
    250 
    251 ==== User call and response ====
    252 
    253 Retrieves information about a GENI user.
    254 
    255 General form of the REST call:
    256 {{{
    257 https://<datastore-host>/info/user/<user object id>
    258 }}}
    259 
    260 Example REST call for user tupty at ch.geni.net:
    261 {{{
    262 https://datastore.ch.geni.net/info/user/tupty
    263 }}}
    264 
    265 The response conforms to the [http://www.gpolab.bbn.com/monitoring/schema/20140828/user user schema], [http://www.gpolab.bbn.com/monitoring/schema/20140828/user.schema.jwc commented version of the user schema].
    266 {{{
    267 {
    268   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/user#",
    269   "id": "tupty",
    270   "selfRef": "https://datastore.ch.geni.net/info/user/tupty",
    271   "urn": "https://datastore.ch.geni.net/info/user/tupty",
    272   "ts": 1391192685740849,
    273   "authority": {
    274     "urn": "urn:publicid:IDN+ch.geni.net+authority+ch",
    275     "href": "https://datastore.ch.geni.net/info/authority/ch.geni.net"
    276   },
    277   "fullname": "Tim Exampleuser",
    278   "email": "tupty@example.com"
    279 }
    280 }}}
    281 
    282 ==== Slice call and response ====
    283 
    284 Retrieves information about  a slice at an authority.
    285 
    286 General form of the REST call:
    287 {{{
    288 https://<datastore-host>/info/slice/<slice object id>
    289 }}}
    290 
    291 Example REST call for tuptyexclusive slice:
    292 {{{
    293 https://datastore.ch.geni.net/info/slice/ch.geni.net_gpo-infra_slice_tuptyexclusive
    294 }}}
    295 
    296 The response conforms to the [http://www.gpolab.bbn.com/monitoring/schema/20140828/slice slice schema], [http://www.gpolab.bbn.com/monitoring/schema/20140828/slice.schema.jwc commented version of the slice schema].
    297 {{{
    298 {
    299   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/slice#",
    300   "id": "ch.geni.net_gpo-infra_slice_tuptyexclusive",
    301   "selfRef": "https://datastore.ch.geni.net/info/slice/ch.geni.net_gpo-infra_slice_tuptyexclusive",
    302   "urn": "urn:publicid:IDN+ch.geni.net:gpo-infra+slice+tuptyexclusive",
    303   "uuid": "8c6b97fa-493b-400f-95ee-19accfaf4ae8",
    304   "ts": 1391192685740849,
    305   "authority": {
    306     "urn": "urn:publicid:IDN+ch.geni.net+authority+ch",
    307     "href": "https://datastore.ch.geni.net/info/authority/ch.geni.net"
    308   },
    309   "created": 1391626683000000,
    310   "expires": 1391708989000000,
    311   "members": [
    312     {
    313       "urn": "urn:publicid:IDN+ch.geni.net+user+tupty",
    314       "href": "https://datastore.ch.geni.net/info/users/tupty",
    315       "role": "lead"
    316     }
    317   ]
    318 }
    319 }}}
    320 
    321 ==== Sliver call and response ====
    322 
    323 Retrieves information about a sliver.
    324 
    325 General form of the REST call:
    326 {{{
    327 https://<datastore-host>/info/sliver/<sliver object id>
    328 }}}
    329 
    330 Example REST call for tuptyexclusive instageni sliver:
    331 {{{
    332 https://datastore.instageni.gpolab.bbn.com/info/sliver/instageni.gpolab.bbn.com_sliver_26947
    333 }}}
    334 
    335 The response conforms to the [http://www.gpolab.bbn.com/monitoring/schema/20140828/sliver sliver schema], [http://www.gpolab.bbn.com/monitoring/schema/20140828/sliver.schema.jwc commented version of the sliver schema].
    336 {{{
    337 {
    338   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/sliver#",
    339   "id": "instageni.gpolab.bbn.com_sliver_26947",
    340   "selfRef": "https://datastore.instageni.gpolab.bbn.com/info/sliver/instageni.gpolab.bbn.com_sliver_26947",
    341   "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+26947",
    342   "uuid": "30752b06-8ea8-11e3-8d30-000000000000",
    343   "ts": 1391192685740849,
    344   "aggregate": {
    345     "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm",
    346     "href": "https://datastore.instageni.gpolab.bbn.com/info/aggregate/gpo-ig"
    347   },
    348   "slice_urn": "urn:publicid:IDN+ch.geni.net:gpo-infra+slice+tuptyexclusive",
    349   "slice_uuid": "8c6b97fa-493b-400f-95ee-19accfaf4ae8",
    350   "creator": "urn:publicid:IDN+ch.geni.net+user+tupty",
    351   "created": 1391626683000000,
    352   "expires": 1391708989000000,
    353   "resource": {
    354     "resource_type": "node",
    355     "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1",
    356     "href": "urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1"
    357   }
    358 }
    359 }}}
    360 
    361 ==== External check datastore call and response ====
    362 
    363 Retrieves information about a sliver.
    364 
    365 General form of the REST call:
    366 {{{
    367 https://<datastore-host>/info/externalcheck/<extchk-id>
    368 }}}
    369 
    370 Example REST call for GPO externalcheck store:
    371 {{{
    372 https://externalcheckstore.gpolab.bbn.com/info/externalcheck/gpo
    373 }}}
    374 
    375 The response conforms to the [http://www.gpolab.bbn.com/monitoring/schema/20140828/externalcheck external check schema], [http://www.gpolab.bbn.com/monitoring/schema/20140828/externalcheck.schema.jwc commented version of the external check schema].
    376 {{{
    377 {
    378    "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/externalcheck#",
    379    "id": "gpo",
    380    "selfRef": "https://externalcheckstore.gpolab.bbn.com/info/externalcheck/gpo",
    381    "ts": 1391192685740849,
    382    "measRef": "https://externalcheckstore.gpolab.bbn.com/data",
    383    "experiments": [
    384       {
    385          "href": "https://externalcheckstore.gpolab.bbn.com/info/experiment/missouri_ig_to_gpo_ig"
    386       }
    387    ],
    388    "monitored_aggregates": [
    389       {
    390          "id": "gpo-ig",
    391          "href": "https://datastore.instageni.gpolab.bbn.com/info/aggregate/gpo-ig"
    392       },
    393       {
    394          "id": "rci-eg",
    395          "href": "https://rci-hn.exogeni.net/ops-monitoring/info/aggregate/rci-eg"
    396       }
    397    ]
    398 }
    399 }}}
    400 
    401 ==== Experiment call and response ====
    402 
    403 Retrieves information data about an operational monitoring experiment for monitoring the dataplane between aggregates.
    404 
    405 General form of the REST call:
    406 {{{
    407 https://<datastore-host>/info/experiment/<experiment id>
    408 }}}
    409 
    410 Example REST call for missouri-ig to gpo-ig experiment:
    411 {{{
    412 https://datastore.externalchecks.geni.net/info/experiment/missouri-ig_to_gpo-ig_rtt
    413 }}}
    414 
    415 The response conforms to the [http://www.gpolab.bbn.com/monitoring/schema/20140828/experiment experiment schema], [http://www.gpolab.bbn.com/monitoring/schema/20140828/experiment.schema.jwc commented version of the experiment schema].
    416 {{{
    417 {
    418   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/experiment#",
    419   "id": "missouri-ig_to_gpo-ig_rtt",
    420   "selfRef": "https://datastore.externalchecks.geni.net/info/experiment/missouri-ig_to_gpo-ig_rtt",
    421   "slice_urn": "urn:publicid:IDN+ch.geni.net:gpo-infra+slice+tuptyexclusive",
    422   "slice_uuid": "8c6b97fa-493b-400f-95ee-19accfaf4ae8",
    423   "ts": 1391192685740849,
    424   "source_aggregate": {
    425     "urn": "urn:publicid:IDN+instageni.missouri.edu+authority+cm",
    426     "href": "https://datastore.instageni.missouri.edu/info/aggregates/missouri-ig"
    427   },
    428   "destination_aggregate": {
    429     "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm",
    430     "href": "https://datastore.instageni.gpolab.bbn.com/info/aggregates/gpo-ig"
    431   }
    432 }
    433 }}}
    434 
    435 
    436 ==== Time-series measurements call and response ====
    437 
    438 Retrieve time series measurements, a.k.a. simply "data" or "events."
    439 
    440 General form of the REST call:
    441 {{{
    442 https://<datastore-host>/data/?q=
    443         {"filters":
    444             {"eventType":[<list of measurementNames>],
    445              "ts":{"gte":<lower bound timestamp>, "lte":<upper bound timestamp},
    446              "obj":{"type":"<object type name>", "id":[<list of object ids>]}}}
    447 }}}
    448  (The above is split into multiple lines for readability; actual calls are all on one line.)
    449 If <list of object ids> is "*", it means "all objects known to the datastore of type <object type name>."
    450 
    451 The response is a list "[...]" in which each element conforms to the  [http://www.gpolab.bbn.com/monitoring/schema/20140828/data data schema], [http://www.gpolab.bbn.com/monitoring/schema/20140828/data.schema.jwc commented version of the data schema].
    452 Each element of the list applies to one (eventType, object id) pair.
    453 
    454 Example REST call for CPU utilization metric on pc1:
    455 {{{
    456 https://datastore.instageni.gpolab.bbn.com/data/?q={"filters":{"eventType": ["ops_monitoring:cpu_util"],"ts":{"gte":0, "lte": 1391199016651285},"obj":{"type":"node","id":["instageni.gpolab.bbn.com_node_pc1"]}}}
    457 }}}
    458 
    459 Response:
    460 {{{
    461 [
    462   {
    463     "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/data#",
    464     "id": "cpu_util:instageni.gpolab.bbn.com_node_pc1",
    465     "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1",
    466     "eventType": "ops_monitoring:cpu_util",
    467     "description": "CPU utilization percentage",
    468     "units": "percent",
    469     "tsdata": [
    470       { "ts": 1391198716651283, "v": 45 },
    471       { "ts": 1391198776651284, "v": 44 },
    472       { "ts": 1391198836651284, "v": 44 },
    473       { "ts": 1391198896651284, "v": 47 },
    474       { "ts": 1391198956651284, "v": 46 },
    475       { "ts": 1391199016651285, "v": 47 }
    476     ]
    477   }
    478 ]
    479 }}}
    480 
    481 Example REST call for percentage of swap available on pc1:
    482 {{{
    483 https://datastore.instageni.gpolab.bbn.com/data/?q={"filters":{"eventType": ["ops_monitoring:swap_free"],"ts":{"gt":0,"lt":1391199016651285},"obj":{"type":"node","id":["instageni.gpolab.bbn.com_node_pc1"]}}}
    484 
    485 }}}
    486 
    487 Response:
    488 {{{
    489 [
    490   {
    491     "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/data#",
    492     "id": "swap_free:instageni.gpolab.bbn.com_node_pc1",
    493     "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1",
    494     "eventType": "ops_monitoring:swap_free",
    495     "description": "Percentage of swap available",
    496     "units": "percent",
    497     "tsdata": [
    498       { "ts": 1391198716651283, "v": 95 },
    499       { "ts": 1391198776651284, "v": 95 },
    500       { "ts": 1391198836651284, "v": 95 },
    501       { "ts": 1391198896651284, "v": 95 },
    502       { "ts": 1391198956651284, "v": 95 },
    503       { "ts": 1391199016651285, "v": 95 }
    504     ]
    505   }
    506 ]
    507 }}}
    508 
    509 Example REST call for memory in active use on pc1:
    510 {{{
    511 https://datastore.instageni.gpolab.bbn.com/data/?q={"filters":{"eventType": ["ops_monitoring:mem_active_kb"],"ts":{"gte":0, "lt": 1391299016651285},"obj":{"type":"node","id":["instageni.gpolab.bbn.com_node_pc1"]}}}
    512 }}}
    513 
    514 Response:
    515 {{{
    516 [
    517   {
    518     "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/data#",
    519     "id": "mem_active_kb:instageni.gpolab.bbn.com_node_pc1",
    520     "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1",
    521     "eventType": "ops_monitoring:mem_active_kb",
    522     "description": "Memory in active use",
    523     "units": "integer",
    524     "tsdata": [
    525       { "ts": 1391198716651283, "v": 20030048 },
    526       { "ts": 1391198776651284, "v": 20031148 },
    527       { "ts": 1391198836651284, "v": 20031148 },
    528       { "ts": 1391198896651284, "v": 22222222 },
    529       { "ts": 1391198956651284, "v": 22222222 },
    530       { "ts": 1391199016651285, "v": 22222222 }
    531     ]
    532   }
    533 ]
    534 }}}
    535 
    536 Example REST call for bytes per second received by `pc1:eth0`:
    537 {{{
    538 https://datastore.instageni.gpolab.bbn.com/data/?q={"filters":{"eventType": ["ops_monitoring:rx_bytes"],"ts":{"gte":0, "lte":1391299016651285},"obj":{"type":"interface","id":["instageni.gpolab.bbn.com_node_pc1:eth0"]}}}
    539 }}}
    540 
    541 Response:
    542 {{{
    543 [
    544   {
    545     "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/data#",
    546     "id": "rx_bps:instageni.gpolab.bbn.com_interface_pc1:eth0",
    547     "subject": "https://datastore.instageni.gpolab.bbn.com/info/interface/instageni.gpolab.bbn.com_interface_pc1:eth0",
    548     "eventType": "ops_monitoring:rx_bps",
    549     "description": "bytes per second received on this interface",
    550     "units": "float",
    551     "tsdata": [
    552       { "ts": 1391198716651283, "v": 2453.64 },
    553       { "ts": 1391198776651284, "v": 800.2 },
    554       { "ts": 1391198836651284, "v": 2400.3 },
    555       { "ts": 1391198896651284, "v": 1984.3 },
    556       { "ts": 1391198956651284, "v": 0 },
    557       { "ts": 1391199016651285, "v": 0 }
    558     ]
    559   }
    560 ]
    561 }}}
    562 
    563 Example REST call for boolean metric indicating whether pc1 is available for use according to the aggregate responsible for it:
    564 {{{
    565 https://datastore.instageni.gpolab.bbn.com/data/?q={"filters":{"eventType": ["ops_monitoring:is_available"],"ts":{"gte":0, "lte":1391299016651285},"obj":{"type":"node","id":["instageni.gpolab.bbn.com_node_pc1"]}}}
    566 }}}
    567 
    568 Response:
    569 {{{
    570 [
    571   {
    572     "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/data#",
    573     "id": "is_available:instageni.gpolab.bbn.com_node_pc1",
    574     "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1",
    575     "eventType": "ops_monitoring:is_available",
    576     "description": "is the subject node available, according to the aggregate",
    577     "units": "boolean",
    578     "tsdata": [
    579       { "ts": 1391198716651283, "v": 1 },
    580       { "ts": 1391198776651284, "v": 1 },
    581       { "ts": 1391198836651284, "v": 1 },
    582       { "ts": 1391198896651284, "v": 1 },
    583       { "ts": 1391198956651284, "v": 0 },
    584       { "ts": 1391199016651285, "v": 0 }
    585     ]
    586   }
    587 ]
    588 }}}
    589 
    590 Example REST call requesting multiple eventTypes and object ids:
    591 {{{
    592 https://datastore.instageni.gpolab.bbn.com/data/?q={"filters":{"eventType": ["ops_monitoring:mem_used","ops_monitoring:cpu_util"],"ts":{"gte":1391192225475202,"lt":1391192225480000},"obj":{"type":"node","id":["instageni.gpolab.bbn.com_node_pc1","instageni.gpolab.bbn.com_node_pc2"]}}}}
    593 }}}
    594 
    595 Response:
    596 {{{
    597 [
    598  {
    599    "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/data#",
    600     "id": "cpu_util:instageni.gpolab.bbn.com_node_pc1",
    601     "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1",
    602     "eventType": "ops_monitoring:cpu_util",
    603     "description": "CPU utilization percentage",
    604     "units": "percent",
    605     "tsdata": [
    606       { "ts": 1391198716651283, "v": 45 },
    607       { "ts": 1391198776651284, "v": 44 },
    608       { "ts": 1391198836651284, "v": 44 },
    609       { "ts": 1391198896651284, "v": 47 },
    610       { "ts": 1391198956651284, "v": 46 },
    611       { "ts": 1391199016651285, "v": 47 }
    612     ]
    613  }
    614 ,
    615  {
    616    "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/data#",
    617    "id": "cpu_util:instageni.gpolab.bbn.com_node_pc2",
    618    "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc2",
    619    "eventType": "ops_monitoring:cpu_util",
    620    "description": "CPU utilization percentage",
    621    "units": "percent",
    622    "tsdata": [
    623      { "ts": 1391198716651283, "v": 45 },
    624      { "ts": 1391198776651284, "v": 48 },
    625      { "ts": 1391198836651284, "v": 45 },
    626      { "ts": 1391198896651284, "v": 49 },
    627      { "ts": 1391198956651284, "v": 50 },
    628      { "ts": 1391199016651285, "v": 51 }
    629     ]
    630  }
    631 ,
    632  {
    633   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/data#",
    634   "id": "mem_active_kb:instageni.gpolab.bbn.com_node_pc1",
    635   "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc1",
    636   "eventType": "ops_monitoring:mem_active_kb",
    637   "description": "Memory in active use",
    638   "units": "integer",
    639   "tsdata": [
    640     { "ts": 1391198716651283, "v": 30030048 },
    641     { "ts": 1391198776651284, "v": 30031148 },
    642     { "ts": 1391198836651284, "v": 30031148 },
    643     { "ts": 1391198896651284, "v": 32222222 },
    644     { "ts": 1391198956651284, "v": 32222222 },
    645     { "ts": 1391199016651285, "v": 32222222 }
    646   ]
    647  }
    648 ,
    649  {
    650   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/data#",
    651   "id": "mem_active_kb:instageni.gpolab.bbn.com_node_pc2",
    652   "subject": "https://datastore.instageni.gpolab.bbn.com/info/node/instageni.gpolab.bbn.com_node_pc2",
    653   "eventType": "ops_monitoring:mem_active_kb",
    654   "description": "Memory in active use",
    655   "units": "integer",
    656   "tsdata": [
    657     { "ts": 1391198716651283, "v": 20030048 },
    658     { "ts": 1391198776651284, "v": 20031148 },
    659     { "ts": 1391198836651284, "v": 20031148 },
    660     { "ts": 1391198896651284, "v": 22222222 },
    661     { "ts": 1391198956651284, "v": 22222222 },
    662     { "ts": 1391199016651285, "v": 22222222 }
    663   ]
    664  }
    665 ]
    666 }}}
    667 
    668 ==== Operational monitoring configuration (opsconfig) call and response ====
    669 
    670 Retrieves a variety of configuration information, including:
    671  * locations (URLs) of and information about other datastores
    672  * GENI authorities
    673  * Database schemas used to store monitoring information
    674  * A list of possible event types for time-series measurements
    675 
    676 Example REST call for geni-prod,  a hypothetical config datastore listing production aggregates and authorities:
    677 {{{
    678 https://datastore.geni.net/info/opsconfig/geni-prod
    679 }}}
    680 
    681 The response conforms to the [http://www.gpolab.bbn.com/monitoring/schema/20140828/opsconfig opsconfig schema], [http://www.gpolab.bbn.com/monitoring/schema/20140828/opsconfig.schema.jwc commented version of the opsconfig schema].
    682 {{{
    683 {
    684   "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140828/opsconfig#",
    685   "id": "geni-prod",
    686   "selfRef": "https://datastore.geni.net/opsconfigs/geni-prod",
    687   "ts": 1391192685740849,
    688   "aggregatestores": [
    689     {
    690       "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm",
    691       "amtype": "protogeni",
    692       "href": "https://datastore.instageni.gpolab.bbn.com/info/aggregate/gpo-ig"
    693     }
    694   ],
    695   "externalcheckstores": [
    696     {
    697       "href": "https://externalcheckstore.gpolab.bbn.com/info/externalcheck/gpo"
    698     }
    699   ],
    700   "authorities": [
    701     {
    702       "urn": "urn:publicid:IDN+ch.geni.net+authority+ch",
    703       "href": "https://datastore.ch.geni.net/authorities/ch.geni.net"
    704     }
    705   ],
    706   "info": [
    707     {  "name": "aggregate",
    708        "db_schema": [
    709                  ["$schema", "varchar"],
    710                  ["id", "varchar"],
    711                  ["selfRef","varchar"],
    712                  ["urn","varchar"],
    713                  ["ts","int8"],
    714                  ["measRef","varchar"]
    715           ]
    716     },
    717     {   "name":"node",
    718         "db_schema": [
    719                  ["$schema", "varchar"],
    720                  ["id", "varchar"],
    721                  ["selfRef","varchar"],
    722                  ["urn","varchar"],
    723                  ["ts","int8"],
    724                  ["properties$mem_total_kb","int8"]
    725           ]
    726     },
    727     {  "name": "link",
    728        "db_schema": [
    729                  ["$schema", "varchar"],
    730                  ["id", "varchar"],
    731                  ["selfRef","varchar"],
    732                  ["urn","varchar"],
    733                  ["ts","int8"]
    734           ]
    735     },
    736     {  "name": "sliver",
    737        "db_schema": [
    738                  ["$schema", "varchar"],
    739                  ["id", "varchar"],
    740                  ["selfRef","varchar"],
    741                  ["urn","varchar"],
    742                  ["uuid","varchar"],
    743                  ["ts","int8"],
    744                  ["aggregate_urn","varchar"],
    745                  ["aggregate_href","varchar"],
    746                  ["slice_urn","varchar"],
    747                  ["slice_uuid","varchar"],
    748                  ["creator","varchar"],
    749                  ["created","int8"],
    750                  ["expires","int8"]
    751           ]
    752     },
    753     {  "name":"interface",
    754        "db_schema": [
    755                  ["$schema", "varchar"],
    756                  ["id", "varchar"],
    757                  ["selfRef","varchar"],
    758                  ["urn","varchar"],
    759                  ["ts","int8"],
    760                  ["address_type","varchar"],
    761                  ["address_address","varchar"],
    762                  ["properties$role","varchar"],
    763                  ["properties$max_bps","int8"],
    764                  ["properties$max_pps","int8"]
    765           ]
    766     },
    767     {   "name": "interfacevlan",
    768         "db_schema": [
    769                  ["$schema", "varchar"],
    770                  ["id", "varchar"],
    771                  ["selfRef","varchar"],
    772                  ["urn","varchar"],
    773                  ["ts","int8"],
    774                  ["tag","int8"],
    775                  ["interface_urn","varchar"],
    776                  ["interface_href","varchar"]
    777           ]
    778    },
    779    {   "name": "slice",
    780        "db_schema": [
    781                  ["$schema", "varchar"],
    782                  ["id", "varchar"],
    783                  ["selfRef","varchar"],
    784                  ["urn","varchar"],
    785                  ["uuid","varchar"],
    786                  ["ts","int8"],
    787                  ["authority_urn","varchar"],
    788                  ["authority_href","varchar"],
    789                  ["created","int8"],
    790                  ["expires","int8"]
    791           ]
    792     },
    793     {   "name":"user",
    794         "db_schema": [
    795                  ["$schema", "varchar"],
    796                  ["id", "varchar"],
    797                  ["selfRef","varchar"],
    798                  ["urn","varchar"],
    799                  ["ts","int8"],
    800                  ["authority_urn","varchar"],
    801                  ["authority_href","varchar"],
    802                  ["fullname","varchar"],
    803                  ["email","varchar"]
    804           ]
    805     },
    806     {   "name":"authority",
    807         "db_schema": [
    808                  ["$schema", "varchar"],
    809                  ["id", "varchar"],
    810                  ["selfRef","varchar"],
    811                  ["urn","varchar"],
    812                  ["ts","int8"]
    813           ]
    814     },
    815     {   "name":"externalcheck",
    816         "db_schema": [
    817                  ["$schema", "varchar"],
    818                  ["id", "varchar"],
    819                  ["selfRef","varchar"],
    820                  ["ts","int8"],
    821                  ["measRef","varchar"]
    822           ]
    823     },
    824     {   "name":"experiment",
    825         "db_schema": [
    826                  ["$schema", "varchar"],
    827                  ["id", "varchar"],
    828                  ["selfRef","varchar"],
    829                  ["ts","int8"],         
    830                  ["slice_urn","varchar"],
    831                  ["slice_uuid","varchar"],
    832                  ["source_aggregate_urn","varchar"],
    833                  ["source_aggregate_href","varchar"],
    834                  ["destination_aggregate_urn","varchar"],
    835                  ["destination_aggregate_href","varchar"]
    836           ]
    837     },
    838     {   "name":"opsconfig",
    839         "db_schema": [
    840                  ["$schema", "varchar"],
    841                  ["id","varchar"],
    842                  ["selfRef","varchar"],
    843                  ["ts","int8"]
    844         ]
    845     },
    846     {   "name":"aggregate_resource",
    847         "db_schema" : [
    848                  ["id","varchar"],
    849                  ["aggregate_id","varchar"],
    850                  ["urn","varchar"],
    851                  ["selfRef","varchar"]
    852           ]
    853     },
    854     {   "name":"aggregate_sliver",
    855         "db_schema" : [
    856                  ["id","varchar"],
    857                  ["aggregate_id","varchar"],
    858                  ["urn","varchar"],
    859                  ["selfRef","varchar"]
    860          ]
    861     },
    862     {   "name":"link_interfacevlan",
    863         "db_schema" : [
    864                  ["id","varchar"],
    865                  ["link_id","varchar"],
    866                  ["urn","varchar"],
    867                  ["selfRef","varchar"]
    868           ]
    869     },
    870     {   "name":"sliver_resource",
    871         "db_schema" : [
    872                  ["id","varchar"],
    873                  ["sliver_id","varchar"],
    874                  ["urn","varchar"],
    875                  ["selfRef","varchar"]
    876          ]
    877     },
    878     {   "name":"node_interface",
    879         "db_schema": [
    880                  ["id","varchar"],
    881                  ["node_id","varchar"],
    882                  ["urn","varchar"],
    883                  ["selfRef","varchar"]
    884           ]
    885     },
    886     {   "name":"slice_user",
    887         "db_schema": [
    888                  ["id","varchar"],
    889                  ["slice_id","varchar"],
    890                  ["urn","varchar"],
    891                  ["role","varchar"],
    892                  ["selfRef","varchar"]
    893         ]
    894     },
    895     {   "name":"authority_user",
    896         "db_schema": [
    897                  ["id","varchar"],
    898                  ["authority_id","varchar"],
    899                  ["urn","varchar"],
    900                  ["selfRef","varchar"]
    901         ]
    902     },
    903     {   "name":"authority_slice",
    904         "db_schema": [
    905                  ["id","varchar"],
    906                  ["authority_id","varchar"],
    907                  ["urn","varchar"],
    908                  ["selfRef","varchar"]
    909         ]
    910     },
    911     {   "name":"opsconfig_aggregate",
    912         "db_schema": [
    913                  ["id","varchar"],
    914                  ["opsconfig_id","varchar"],
    915                  ["amtype","varchar"],
    916                  ["urn","varchar"],
    917                  ["selfRef","varchar"]
    918         ]
    919     },
    920     {   "name":"opsconfig_authority",
    921         "db_schema": [
    922                  ["id","varchar"],
    923                  ["opsconfig_id","varchar"],
    924                  ["urn","varchar"],
    925                  ["selfRef","varchar"]
    926         ]
    927     },
    928     {   "name":"opsconfig_event",
    929         "db_schema": [
    930                  ["object_type","varchar"],
    931                  ["name","varchar"],
    932                  ["id","varchar"],
    933                  ["ts","varchar"],
    934                  ["v","varchar"],
    935                  ["units","varchar"]
    936         ]
    937     },
    938     {   "name":"externalcheck_experiment",
    939         "db_schema": [
    940                  ["id","varchar"],
    941                  ["externalcheck_id","varchar"],
    942                  ["selfRef","varchar"]
    943         ]
    944     }
    945   ],
    946   "events":
    947     {
    948       "node": [
    949           {    "name": "cpu_util",
    950                "id": "varchar",
    951                "ts": "int8",
    952                "v": "float4",
    953                "units": "percent"
    954           },
    955           {    "name": "mem_used_kb",
    956                "id": "varchar",
    957                "ts": "int8",
    958                "v": "int8",
    959                "units": "kilobytes"
    960           },   
    961           {    "name": "swap_free",
    962                "id": "varchar",
    963                "ts": "int8",
    964                "v": "float4",
    965                "units": "percent"
    966           },
    967           {    "name": "is_available",
    968                "id": "varchar",
    969                "ts": "int8",
    970                "v": "int2",
    971                "units": "boolean"
    972           },
    973           {    "name": "disk_part_max_used",
    974                "id": "varchar",
    975                "ts": "int8",
    976                "v": "float4",
    977                "units":"percent"
    978           }
    979        ],
    980        "aggregate": [
    981           {    "name": "num_vms_allocated",
    982                "id": "varchar",
    983                "ts": "int8",
    984                "v": "int4",
    985                "units": "count"
    986           },
    987           {    "name": "is_available",
    988                "id": "varchar",
    989                "ts": "int8",
    990                "v": "int2",
    991                "units": "boolean"
    992           }
    993        ],
    994        "interface": [
    995           {    "name": "rx_bps",
    996                "id": "varchar",
    997                "ts": "int8",
    998                "v":"float4",
    999                "units":"bps"
    1000           },
    1001           {    "name": "tx_bps",
    1002                "id": "varchar",
    1003                "ts": "int8",
    1004                "v":"float4",
    1005                "units":"bps"
    1006           },
    1007           {    "name":"rx_pps",
    1008                "id": "varchar",
    1009                "ts": "int8",
    1010                "v":"float4",
    1011                "units":"pps"
    1012           },
    1013           {    "name":"tx_pps", 
    1014                "id": "varchar",
    1015                "ts": "int8",
    1016                "v":"float4",
    1017                "units":"pps"
    1018           },
    1019           {    "name":"rx_eps", 
    1020                "id": "varchar",
    1021                "ts": "int8",
    1022                "v":"float4",
    1023                "units":"pps"
    1024           },
    1025           {    "name":"tx_eps",
    1026                "id": "varchar",
    1027                "ts": "int8",
    1028                "v":"float4",
    1029                "units":"pps"
    1030           },
    1031           {    "name":"rx_dps",
    1032                "id": "varchar",
    1033                "ts": "int8",
    1034                "v":"float4",
    1035                "units":"pps"
    1036           },
    1037           {    "name":"tx_dps",
    1038                "id": "varchar",
    1039                "ts": "int8",
    1040                "v":"float4",
    1041                "units":"pps"
    1042           }
    1043        ],
    1044        "interfacevlan": [
    1045           {    "name": "rx_bps",
    1046                "id": "varchar",
    1047                "ts": "int8",
    1048                "v":"float4",
    1049                "units":"bps"
    1050           },
    1051           {    "name": "tx_bps",
    1052                "id": "varchar",
    1053                "ts": "int8",
    1054                "v":"float4",
    1055                "units":"bps"
    1056           },
    1057           {    "name":"rx_pps",
    1058                "id": "varchar",
    1059                "ts": "int8",
    1060                "v":"float4",
    1061                "units":"pps"
    1062           },
    1063           {    "name":"tx_pps", 
    1064                "id": "varchar",
    1065                "ts": "int8",
    1066                "v":"float4",
    1067                "units":"pps"
    1068           },
    1069           {    "name":"rx_eps", 
    1070                "id": "varchar",
    1071                "ts": "int8",
    1072                "v":"float4",
    1073                "units":"pps"
    1074           },
    1075           {    "name":"tx_eps",
    1076                "id": "varchar",
    1077                "ts": "int8",
    1078                "v":"float4",
    1079                "units":"pps"
    1080           },
    1081           {    "name":"rx_dps",
    1082                "id": "varchar",
    1083                "ts": "int8",
    1084                "v":"float4",
    1085                "units":"pps"
    1086           },
    1087           {    "name":"tx_dps",
    1088                "id": "varchar",
    1089                "ts": "int8",
    1090                "v":"float4",
    1091                "units":"pps"
    1092           }
    1093        ],
    1094        "experiment": [
    1095           {    "name": "ping_rtt_ms",
    1096                "id": "varchar",
    1097                "ts": "int8",
    1098                "v":"float4",
    1099                "units":"milliseconds"
    1100           }
    1101        ]
    1102     }
    1103 }
    1104 }}}
     2Moved to http://groups.geni.net/geni/wiki/OperationalMonitoring/DataSchema