Changes between Version 14 and Version 15 of OperationalMonitoring/DataSchema


Ignore:
Timestamp:
02/10/14 11:03:45 (10 years ago)
Author:
chaos@bbn.com
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • OperationalMonitoring/DataSchema

    v14 v15  
    9696
    9797I propose a schema based on, and partially compatible with, [http://unis.incntre.iu.edu/schema/20120709/], for measurement data and metadata.  In particular, i suggest using:
    98  * [http://unis.incntre.iu.edu/schema/20120709/domain]: metadata about an aggregate (including a list of resources at that aggregate)
     98 * [http://www.gpolab.bbn.com/monitoring/schema/20140131/aggregate]: (inherits from [http://unis.incntre.iu.edu/schema/20120709/domain]): metadata about an aggregate (including lists of resources and slivers at that aggregate, and where to look for measurements about the aggregate)
    9999 * [http://unis.incntre.iu.edu/schema/20120709/node]: metadata about a resources (including resource properties termed "config" above, and a list of relevant ports (or of all ports, whichever is easier) on that resource)
    100100 * [http://unis.incntre.iu.edu/schema/20120709/port]: metadata about a network interface (including resource properties termed "config" above)
    101  * http://www.gpolab.bbn.com/monitoring/schema/20140131/data (not yet posted): novel schema based on a combination of [http://unis.incntre.iu.edu/schema/20120709/metadata] and [http://unis.incntre.iu.edu/schema/20120709/tsdatum]:
    102 {{{
    103 {
    104   "$schema": "http://json-schema.org/draft-03/hyper-schema#",
    105   "id": "http://www.gpolab.bbn.com/monitoring/schema/20140131/data#",
    106   "description": "Operational monitoring data",
    107   "name": "Data",
    108   "type": "object",
    109   "additionalProperties": false,
    110   "extends": {
    111     "$ref": "http://unis.incntre.iu.edu/schema/20120709/metadata#"
    112   },
    113   "properties": {
    114     "units": {
    115       "units": "Valid units for the values in this metric",
    116       "type": "string",
    117       "required": true
    118     },
    119     "description": {
    120       "description": "Description of this metric",
    121       "type": "string",
    122       "required": false
    123     },
    124     "tsdata": {
    125       "description": "Time-series data",
    126       "name": "tsdata",
    127       "type": "array",
    128       "required": false
    129     }
    130   }
    131 }
    132 }}}
     101 * [http://www.gpolab.bbn.com/monitoring/schema/20140131/data]: novel schema based on a combination of [http://unis.incntre.iu.edu/schema/20120709/metadata] and [http://unis.incntre.iu.edu/schema/20120709/tsdatum], contains both metadata and data about measurements.
     102 * [http://www.gpolab.bbn.com/monitoring/schema/20140131/sliver]: (inherits from [http://unis.incntre.iu.edu/schema/20120709/networkresource]): metadata about a sliver at an aggregate, including resources to which that sliver is mapped
    133103 * For now, we would use the `ops_monitoring` namespace for operations monitoring, meaning:
    134104   * When we add monitoring-relevant optional properties to objects, we'll put them in an `ops_monitoring` dictionary
     
    144114==== Data about an aggregate ====
    145115
    146 Aggregates are indexed by GENI-agreed short name and described using [http://unis.incntre.iu.edu/schema/20120709/domain# the domain schema].  Examples:
     116Aggregates are indexed by GENI-agreed short name and described using [http://www.gpolab.bbn.com/monitoring/schema/20140131/data # the aggregate schema].  Examples:
    147117 * gpo-ig:
    148118{{{
    149119{
    150   'id': 'gpo-ig',
    151   'selfRef': 'https://datastore.instageni.gpolab.bbn.com/domains/gpo-ig',
    152   'urn': 'urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm',
    153   'ts': 1391192685740849,
    154   'nodes': [
    155     {
    156       'href': 'https://datastore.instageni.gpolab.bbn.com/nodes/instageni.gpolab.bbn.com_node_pc1',
    157     },
    158     {
    159       'href': 'https://datastore.instageni.gpolab.bbn.com/nodes/instageni.gpolab.bbn.com_node_pc2',
    160     },
     120  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140131/aggregate#",
     121  "id": "gpo-ig",
     122  "selfRef": "https://datastore.instageni.gpolab.bbn.com/domains/gpo-ig",
     123  "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+authority+cm",
     124  "ts": 1391192685740849,
     125  "measRef": "https://datastore.instageni.gpolab.bbn.com/data",
     126  "resources": [
     127    {
     128      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1",
     129      "href": "https://datastore.instageni.gpolab.bbn.com/nodes/instageni.gpolab.bbn.com_node_pc1"
     130    },
     131    {
     132      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc2",
     133      "href": "https://datastore.instageni.gpolab.bbn.com/nodes/instageni.gpolab.bbn.com_node_pc2"
     134    }
    161135  ],
     136  "slivers": [
     137    {
     138      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+sliver+26947",
     139      "href": "https://datastore.instageni.gpolab.bbn.com/slivers/instageni.gpolab.bbn.com_sliver_36947"
     140    }
     141  ]
    162142}
    163143}}}
     
    169149{{{
    170150{
    171   'id': 'instageni.gpolab.bbn.com_node_pc1',
    172   'ts': 1391192705275101,
    173   'selfRef': 'https://datastore.instageni.gpolab.bbn.com/nodes/instageni.gpolab.bbn.com_node_pc1',
    174   'urn': 'urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1',
    175   'properties': {
    176     'ops_monitoring': {
    177       'mem_total_kb': 50331648,
    178     },
     151  "$schema": "http://unis.incntre.iu.edu/schema/20120709/node#",
     152  "id": "instageni.gpolab.bbn.com_node_pc1",
     153  "ts": 1391192705275101,
     154  "selfRef": "https://datastore.instageni.gpolab.bbn.com/nodes/instageni.gpolab.bbn.com_node_pc1",
     155  "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+node+pc1",
     156  "properties": {
     157    "ops_monitoring": {
     158      "mem_total_kb": 50331648
     159    }
    179160  },
    180   'ports': [
    181     {
    182       'href': 'https://datastore.instageni.gpolab.bbn.com/ports/instageni.gpolab.bbn.com_interface_pc1%3Aeth0',
    183     }
    184     {
    185       'href': 'https://datastore.instageni.gpolab.bbn.com/ports/instageni.gpolab.bbn.com_interface_pc1%3Aeth1',
    186     }
    187     {
    188       'href': 'https://datastore.instageni.gpolab.bbn.com/ports/instageni.gpolab.bbn.com_interface_pc1%3Aeth2',
    189     }
    190     {
    191       'href': 'https://datastore.instageni.gpolab.bbn.com/ports/instageni.gpolab.bbn.com_interface_pc1%3Aeth3',
    192     }
    193   ],
     161  "ports": [
     162    {
     163      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth0",
     164      "href": "https://datastore.instageni.gpolab.bbn.com/ports/instageni.gpolab.bbn.com_interface_pc1%3Aeth0"
     165    },
     166    {
     167      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth1",
     168      "href": "https://datastore.instageni.gpolab.bbn.com/ports/instageni.gpolab.bbn.com_interface_pc1%3Aeth1"
     169    },
     170    {
     171      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth2",
     172      "href": "https://datastore.instageni.gpolab.bbn.com/ports/instageni.gpolab.bbn.com_interface_pc1%3Aeth2"
     173    },
     174    {
     175      "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth3",
     176      "href": "https://datastore.instageni.gpolab.bbn.com/ports/instageni.gpolab.bbn.com_interface_pc1%3Aeth3"
     177    }
     178  ]
    194179}
    195180}}}
     
    205190{{{
    206191{
    207   'selfRef': 'https://datastore.instageni.gpolab.bbn.com/ports/instageni.gpolab.bbn.com_interface_pc1:eth0',
    208   'urn': 'urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth0',
    209   'ts': 1391194147100678,
    210   'id': 'instageni.gpolab.bbn.com_interface_pc1:eth0',
    211   'properties': {
    212     'ops_monitoring': {
    213       'role': 'control',
    214       'max_bps': 10000000,
    215       'max_pps': 1000000,
    216     },
     192  "$schema": "http://unis.incntre.iu.edu/schema/20120709/port#",
     193  "selfRef": "https://datastore.instageni.gpolab.bbn.com/ports/instageni.gpolab.bbn.com_interface_pc1:eth0",
     194  "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth0",
     195  "ts": 1391194147100678,
     196  "id": "instageni.gpolab.bbn.com_interface_pc1:eth0",
     197  "address": {
     198    "type": "ipv4",
     199    "address": "192.1.242.140"
    217200  },
     201  "properties": {
     202    "ops_monitoring": {
     203      "role": "control",
     204      "max_bps": 10000000,
     205      "max_pps": 1000000
     206    }
     207  }
    218208}
    219209}}}
     
    221211{{{
    222212{
    223   'selfRef': 'https://datastore.instageni.gpolab.bbn.com/ports/instageni.gpolab.bbn.com_interface_pc1:eth1',
    224   'urn': 'urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth1',
    225   'ts': 1391194147100678,
    226   'id': 'instageni.gpolab.bbn.com_interface_pc1:eth1',
    227   'properties': {
    228     'ops_monitoring': {
    229       'role': 'experimental',
    230       'max_bps': 10000000,
    231       'max_pps': 1000000,
    232     },
     213  "$schema": "http://unis.incntre.iu.edu/schema/20120709/port#",
     214  "selfRef": "https://datastore.instageni.gpolab.bbn.com/ports/instageni.gpolab.bbn.com_interface_pc1:eth1",
     215  "urn": "urn:publicid:IDN+instageni.gpolab.bbn.com+interface+pc1:eth1",
     216  "ts": 1391194147100678,
     217  "id": "instageni.gpolab.bbn.com_interface_pc1:eth1",
     218  "address": {
     219    "type": "mac",
     220    "address": "aa:aa:aa:aa:aa:ab"
    233221  },
     222  "properties": {
     223    "ops_monitoring": {
     224      "role": "experimental",
     225      "max_bps": 10000000,
     226      "max_pps": 1000000
     227    }
     228  }
    234229}
    235230}}}
     
    241236{{{
    242237{
    243   'id': '1',
    244   'subject': 'https://datastore.instageni.gpolab.bbn.com/nodes/instageni.gpolab.bbn.com_node_pc1',
    245   'eventType': 'ops_monitoring:cpu_utilization',
    246   'description': 'CPU utilization percentage',
    247   'units': 'percent',
    248   'tsdata': {
    249     { 'ts': 1391198716651283, 'val': 0.45, },
    250     { 'ts': 1391198776651284, 'val': 0.44, },
    251     { 'ts': 1391198836651284, 'val': 0.44, },
    252     { 'ts': 1391198896651284, 'val': 0.47, },
    253     { 'ts': 1391198956651284, 'val': 0.46, },
    254     { 'ts': 1391199016651285, 'val': 0.47, },
    255   },
     238  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140131/data#",
     239  "id": "1",
     240  "subject": "https://datastore.instageni.gpolab.bbn.com/nodes/instageni.gpolab.bbn.com_node_pc1",
     241  "eventType": "ops_monitoring:cpu_utilization",
     242  "description": "CPU utilization percentage",
     243  "units": "percent",
     244  "tsdata": [
     245    { "ts": 1391198716651283, "val": 0.45 },
     246    { "ts": 1391198776651284, "val": 0.44 },
     247    { "ts": 1391198836651284, "val": 0.44 },
     248    { "ts": 1391198896651284, "val": 0.47 },
     249    { "ts": 1391198956651284, "val": 0.46 },
     250    { "ts": 1391199016651285, "val": 0.47 }
     251  ]
    256252}
    257253}}}
    258254 * Percentage of swap available on pc1:
    259255{{{
    260 # Metadata and data for the swap_free metric on pc1 at gpo-ig.
    261 {
    262   'id': '2',
    263   'subject': 'https://datastore.instageni.gpolab.bbn.com/nodes/instageni.gpolab.bbn.com_node_pc1',
    264   'eventType': 'ops_monitoring:swap_free',
    265   'description': 'Percentage of swap available',
    266   'units': 'percent',
    267   'tsdata': {
    268     { 'ts': 1391198716651283, 'val': 0.95, },
    269     { 'ts': 1391198776651284, 'val': 0.95, },
    270     { 'ts': 1391198836651284, 'val': 0.95, },
    271     { 'ts': 1391198896651284, 'val': 0.95, },
    272     { 'ts': 1391198956651284, 'val': 0.95, },
    273     { 'ts': 1391199016651285, 'val': 0.95, },
    274   },
     256{
     257  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140131/data#",
     258  "id": "2",
     259  "subject": "https://datastore.instageni.gpolab.bbn.com/nodes/instageni.gpolab.bbn.com_node_pc1",
     260  "eventType": "ops_monitoring:swap_free",
     261  "description": "Percentage of swap available",
     262  "units": "percent",
     263  "tsdata": [
     264    { "ts": 1391198716651283, "val": 0.95 },
     265    { "ts": 1391198776651284, "val": 0.95 },
     266    { "ts": 1391198836651284, "val": 0.95 },
     267    { "ts": 1391198896651284, "val": 0.95 },
     268    { "ts": 1391198956651284, "val": 0.95 },
     269    { "ts": 1391199016651285, "val": 0.95 }
     270  ]
    275271}
    276272}}}
     
    278274{{{
    279275{
    280   'id': '3',
    281   'subject': 'https://datastore.instageni.gpolab.bbn.com/nodes/instageni.gpolab.bbn.com_node_pc1',
    282   'eventType': 'ops_monitoring:mem_active_kb',
    283   'description': 'Memory in active use',
    284   'units': 'kilobytes',
    285   'tsdata': {
    286     { 'ts': 1391198716651283, 'val': 20030048, },
    287     { 'ts': 1391198776651284, 'val': 20031148, },
    288     { 'ts': 1391198836651284, 'val': 20031148, },
    289     { 'ts': 1391198896651284, 'val': 22222222, },
    290     { 'ts': 1391198956651284, 'val': 22222222, },
    291     { 'ts': 1391199016651285, 'val': 22222222, },
    292   },
    293 }
    294 }}}
     276  "$schema": "http://www.gpolab.bbn.com/monitoring/schema/20140131/data#",
     277  "id": "3",
     278  "subject": "https://datastore.instageni.gpolab.bbn.com/nodes/instageni.gpolab.bbn.com_node_pc1",
     279  "eventType": "ops_monitoring:mem_active_kb",
     280  "description": "Memory in active use",
     281  "units": "kilobytes",
     282  "tsdata": [
     283    { "ts": 1391198716651283, "val": 20030048 },
     284    { "ts": 1391198776651284, "val": 20031148 },
     285    { "ts": 1391198836651284, "val": 20031148 },
     286    { "ts": 1391198896651284, "val": 22222222 },
     287    { "ts": 1391198956651284, "val": 22222222 },
     288    { "ts": 1391199016651285, "val": 22222222 }
     289  ]
     290}
     291}}}