{\rtf1\ansi\ansicpg1252\cocoartf1138
{\fonttbl\f0\froman\fcharset0 TimesNewRomanPSMT;\f1\fswiss\fcharset0 Helvetica;}
{\colortbl;\red255\green255\blue255;\red23\green54\blue93;\red54\green95\blue145;\red79\green129\blue189;
\red0\green0\blue255;\red0\green0\blue153;}
{\*\listtable{\list\listtemplateid1\listhybrid{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid1\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li720\lin720 }{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid2\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li1440\lin1440 }{\listname ;}\listid1}
{\list\listtemplateid2\listhybrid{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid101\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li720\lin720 }{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid102\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li1440\lin1440 }{\listname ;}\listid2}
{\list\listtemplateid3\listhybrid{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid201\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li720\lin720 }{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid202\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li1440\lin1440 }{\listname ;}\listid3}
{\list\listtemplateid4\listhybrid{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid301\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li720\lin720 }{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid302\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li1440\lin1440 }{\listname ;}\listid4}
{\list\listtemplateid5\listhybrid{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid401\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li720\lin720 }{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid402\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li1440\lin1440 }{\listname ;}\listid5}
{\list\listtemplateid6\listhybrid{\listlevel\levelnfc0\levelnfcn0\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{decimal\}.}{\leveltext\leveltemplateid501\'02\'00.;}{\levelnumbers\'01;}\fi-360\li720\lin720 }{\listname ;}\listid6}
{\list\listtemplateid7\listhybrid{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid601\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li720\lin720 }{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid602\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li1440\lin1440 }{\listname ;}\listid7}
{\list\listtemplateid8\listhybrid{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid701\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li720\lin720 }{\listname ;}\listid8}
{\list\listtemplateid9\listhybrid{\listlevel\levelnfc23\levelnfcn23\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{disc\}}{\leveltext\leveltemplateid801\'01\uc0\u8226 ;}{\levelnumbers;}\fi-360\li720\lin720 }{\listname ;}\listid9}
{\list\listtemplateid10\listhybrid{\listlevel\levelnfc0\levelnfcn0\leveljc0\leveljcn0\levelfollow0\levelstartat1\levelspace360\levelindent0{\*\levelmarker \{decimal\}.}{\leveltext\leveltemplateid901\'02\'00.;}{\levelnumbers\'01;}\fi-360\li720\lin720 }{\listname ;}\listid10}}
{\*\listoverridetable{\listoverride\listid1\listoverridecount0\ls1}{\listoverride\listid2\listoverridecount0\ls2}{\listoverride\listid3\listoverridecount0\ls3}{\listoverride\listid4\listoverridecount0\ls4}{\listoverride\listid5\listoverridecount0\ls5}{\listoverride\listid6\listoverridecount0\ls6}{\listoverride\listid7\listoverridecount0\ls7}{\listoverride\listid8\listoverridecount0\ls8}{\listoverride\listid9\listoverridecount0\ls9}{\listoverride\listid10\listoverridecount0\ls10}}
\vieww17220\viewh13560\viewkind1
\deftab720
\pard\pardeftab720\ri0\sa300

\f0\fs52 \cf2 GIMS Passive Measurement System
\fs22 \cf0 \

\b\fs28 \cf3 Relationship with GENI I&M Architecture\
\pard\pardeftab720\ri0\sl276\slmult1

\b0\fs24 \cf0 \
In this section, we describe how the GIMS Passive Measurement System (GPMS) fits with the GENI I&M Architecture.  The job of the GPMS in relation to the GENI I&M Architecture is to gather, transform, and store packets from taps on links in the GENI infrastructure.  Before we go into detail on how the interface between GPMS and the control framework integrates GPMS into the GENI I&M Architecture we will briefly discuss some of the functions of GPMS. The main role of GPMS is to capture packets for a given experiment as well as transform and store the collected data.  The system allows for a single packet capturing device to receive packets for several experiments at a time and to perform transformation and storage independently for each experiment.  Thus, the system design easily enables the GPMS system to expose a slice-type abstraction.\
\
The basic components of the GPMS are (1) a control framework integration component based on the reference component manager that interacts with a (2) GIMS \'93backend\'94 component.  This component has its own database that stores information regarding various sensor devices that may be deployed in the GENI infrastructure.  Furthermore, it translates the control framework actions into XML/RPC calls to control the sensor devices.  (3) The third component is the sensor device, which contains three subcomponents: (a) a software component that communicates with the GIMS backend, (b) the software component that actually performs the packet capture, transformation, metadata creation, and local staging of collected data, and (c) a storage manager which handles transfer of collected data to specified storage locations (e.g., an sftp server or Amazon S3).\
\
Below, we describe how the GPMS components map to the architectural components of the overall GENI I&M architecture:\
\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0\sl276\slmult1
\ls1\ilvl0\cf0 \uc0\u9632 	Measurement orchestration: how is workflow set up in our system?   \
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0\sl276\slmult1
\ls1\ilvl1\cf0 \uc0\u9675 	The standard functions defined by the component manager (create slice, delete slice) are available to instantiate a new packet capture process.  An experimenter may also interact with a separate GUI environment to modify the behavior of individual measurement points, and to set up storage transfer.  Furthermore, an experimenter may interact with the GUI to collect status information about an ongoing packet capture session.\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0\sl276\slmult1
\ls2\ilvl0\cf0 \uc0\u9632 	Measurement point\
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0\sl276\slmult1
\ls2\ilvl1\cf0 \uc0\u9675 	The functions, and types of services and data available from the measurement point are capture of raw packets, optional sampling, optional aggregation into simple counts (byte/packet counts) or IPFIX flow records, optional anonymization of IP addresses, and storage of the collected data.  Metadata are added to the collected data, as described below.\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0\sl276\slmult1
\ls3\ilvl0\cf0 \uc0\u9632 	Measurement information\
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0\sl276\slmult1
\ls3\ilvl1\cf0 \uc0\u9675 	Transformation and annotation of measurement data occurs within the capture system.  This is not a separate component of the system, but rather co-resident with the measurement point functionality.\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0\sl276\slmult1
\ls4\ilvl0\cf0 \uc0\u9632 	 Measurement collection\
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0\sl276\slmult1
\ls4\ilvl1\cf0 \uc0\u9675 	Collected data and metadata can be transferred to an experimenter-specified location.  At present, this may be an sftp server or Amazon S3.\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0\sl276\slmult1
\ls4\ilvl0\cf0 \uc0\u9632 	Measurement analysis and presentation\
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0\sl276\slmult1
\ls4\ilvl1\cf0 \uc0\u9675 	These functions are outside the scope of the GPMS.  The system is used for configuration and collection of network packet traces.\
\uc0\u9675 	There are many (external) basic tools available for analysis of traces.  Any pcap-based tool can be used to analyze a pcap trace collected by the system.  The open source yafscii tool, which is part of the YAF and libfixbuf software, may be used to convert a yaf-collected IPFIX trace to text, which then may be further analyzed.\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0\sl276\slmult1
\ls4\ilvl0\cf0 \uc0\u9632 	Measurement data archive\
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0\sl276\slmult1
\ls4\ilvl1\cf0 \uc0\u9675 	This function is essentially outside scope of our system.  The GPMS facilitates transfer of data and metadata to configured data storage locations, but these locations are not under the control or management of the GPMS, per se.\
\pard\pardeftab720\ri0\sl276\slmult1
\ls4\ilvl0\cf0 \
Below, we describe how the GPMS fits into the GENI I&M functions, following the discussion in the current GENI I&M architectural document.\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0\sl276\slmult1
\ls5\ilvl0\cf0 \uc0\u9632 	Discover Resources and Assign Slivers\
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0\sl276\slmult1
\ls5\ilvl1\cf0 \uc0\u9675 	Resources must be pre-configured with a control framework to be used by an experimenter.\
\uc0\u9675 	An experimenter may use standard control framework interfaces to create a slice that then may be used for passive packet collection.  Our system implements standard calls via the reference component manager, and has currently been tested and integrated with protogeni.\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0\sl276\slmult1
\ls5\ilvl0\cf0 \uc0\u9632 	Configure and Program Slivers\
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0\sl276\slmult1
\ls5\ilvl1\cf0 \uc0\u9675 	An experimenter may use the GIMS GUI to configure specific capabilities of individual capture devices, such as sampling, transformation, and storage.\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0\sl276\slmult1
\ls5\ilvl0\cf0 \uc0\u9632 	Manage Services\
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0\sl276\slmult1
\ls5\ilvl1\cf0 \uc0\u9675 	An experimenter may use the GIMS GUI to check status of capture devices (slivers), to stop, start, and/or pause capture.  There are no asynchronous notifications at present of events on capture devices.\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0\sl276\slmult1
\ls5\ilvl0\cf0 \uc0\u9632 	Measurement Data Flows/Transfer\
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0\sl276\slmult1
\ls5\ilvl1\cf0 \uc0\u9675 	Data and metadata are periodically transferred to the configured repository.  The interval at which data/metadata are transferred may be configured through the GIMS GUI.  The protocols/interfaces used for data transfer depend on the configured storage system, which may be an sftp server (ssh protocol) or Amazon S3 (REST-like https service).\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0\sl276\slmult1
\ls5\ilvl0\cf0 \uc0\u9632 	Register Availability of Measurement Data from I&M Service\
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0\sl276\slmult1
\ls5\ilvl1\cf0 \uc0\u9675 	The GIMS system currently does not include capabilities for advertising collected measurements; such capabilities are outside the scope of the system.  It is the responsibility of an individual experimenter to make collected data/metadata available to a wider group.\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0\sl276\slmult1
\ls5\ilvl0\cf0 \uc0\u9632 	Discover Availability of Measurement Data from I&M Service and Start MD Flow/Transfer\
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0\sl276\slmult1
\ls5\ilvl1\cf0 \uc0\u9675 	Again, this capability is outside the scope of GIMS.\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0\sl276\slmult1
\ls5\ilvl0\cf0 \uc0\u9632 	Observe I&M Service Status and Examine Measurement Data\
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0\sl276\slmult1
\ls5\ilvl1\cf0 \uc0\u9675 	Service status may be observed through the GIMS GUI.  Measurement data may be examined at the configured repository; how this is accomplished depends on the storage and transformation parameters configured by an experimenter.\
\pard\pardeftab720\ri0\sl276\slmult1
\ls5\ilvl0\cf0 \
\pard\pardeftab720\ri0\sb480
\ls5\ilvl0
\b\fs28 \cf3 Data and Metadata Formats and Relationships to Emerging GENI I&M Standards\
\pard\pardeftab720\ri0
\ls5\ilvl0
\b0\fs24 \cf4 \
\pard\pardeftab720\ri0\sl276\slmult1
\ls5\ilvl0\cf0 In this section we discuss the current data/meta-data formats supported by the GIMS passive measurement system, and how those formats related to discussion and emerging standards within the GENI I&M working group. \
\pard\pardeftab720\ri0
\ls5\ilvl0
\i \cf4 \
Interfaces between measurement services:\
\pard\pardeftab720\ri0
\ls5\ilvl0
\i0 \cf0 The measurement services provided by GIMS are (1) passive packet capture,(2)  transformation of measurements, (3) storage and (4) testing. All four of these services have interfaces linking them and facilitating communication between the control framework and the four different services offered. Each of the services uses distinct technologies and works with each other in order to compose GIMS.\
\
(1) Passive packet capture.  The passive packet capturing capability is provided by the capture daemon. This component provides the basic functionality in GIMS, described in section 3.2.1 from 
\i \'93Requirements and Specifications for the Instrumentation and Measurement Systems for GENI.\'94
\i0  Packet capturing is managed by a capture proxy.\
\
(2) Measurement transformation.  The next capability, transformation of measurements, includes, anonymization, sampling and flow aggregation. All of these features are also provided in the capture daemon using the 
\i YAF/FIXBUF
\i0  libraries. \
\
(3) Storage.  After capturing packets and generating more data pertaining a particular experiment, the storage component then takes care of transporting data across the network to user-specific locations. The storing of data takes advantage of the multiple resources offered by companies such as Amazon and others, but still offers a wide range of high performance storing options. The three different ways that data storage is supported is through Amazon\'92s S3 service, SFTP  in an SSH server and local storage. The data being stored consists of the measurement data and the metadata as well.\
\
(4) Testing.  The last main service offered by GIMS is testing. Testing is achieved through the capture client, which tests different orderings of the 
\i XML/RPC
\i0  method calls as well as the managing interface for the capture daemon. Another way testing is performed during an experiment is to check if the storage service specified in the configuration parameters is working and if not it raises an error.  For example, an experimenter may errantly enter bad authentication information for a storage service; the testing interfaces enable him or her to check whether data can be uploaded to the configured storage service.\
\ls5\ilvl0
\i \
\pard\pardeftab720\ri0
\ls5\ilvl0\cf4 Protocols for MD flows:\
\pard\pardeftab720\ri0
\ls5\ilvl0
\i0 \cf0 There are two protocols used for the measurement data flow aggregation they are the SNMP-like aggregation and the standard IPFIX flow aggregation. The SNMP-like aggregation consists of adding the packet and byte counts, and periodically exporting these in text format. The standard IPFIX record export [RFC 3917] uses YAF, libfixbuf, and related libraries from CERT ({\field{\*\fldinst{HYPERLINK "http://tools.netsa.cert.org/yaf/"}}{\fldrslt \cf5 \ul \ulc5 http://tools.netsa.cert.org/yaf/}}).  The tool used to dump records to text for conversion is yafscii, which allows to print flow data files in an ASCII format ({\field{\*\fldinst{HYPERLINK "http://tools.netsa.cert.org/yaf/yafscii.html"}}{\fldrslt 
\fs22 \cf5 \ul \ulc5 http://tools.netsa.cert.org/yaf/yafscii.html}}
\fs22 ).
\fs24 \
\pard\pardeftab720\ri0
\ls5\ilvl0
\fs22 \cf0 \
\pard\pardeftab720\ri0
\ls5\ilvl0
\fs24 \cf0 The other types of protocols are those use by the storage agent to transfer data about an experiment. First, data are collected using the libpcap library by capturing packets.  Second, as the measurement data are captured, they may be transformed.  Furthermore, some metadata are created during live capture.  Next, the data and metadata are locally staged and readied for transfer to a configured storage location.  There are two methods used in GIMS to aggregate data. One method is through SNMP-like aggregation in which the packets and bytes are simply counted and embedded in measurement metadata. The second method of aggregation is Flow aggregation in which the raw libpcap trace is transformed using tools such as YAF, libfixbuf and other related libraries from CERT \cf6 \ul \ulc6 (http://tools.netsa.cert.org/yaf/)\cf0 \ulnone  and formatted to fit the IPFIX [RFC 3917] format. After the aggregation/transformation of the measurement data there are two types of files generated, pcap-format traces and IPFIX records.  Then, the measurement data and metadata are transferred to the user specified storage location. The storage location of the measurement data depends on the protocol the user defined in the configuration. The three possible storage options are (1) Amazon\'92s S3, (2) SFTP and in a (3) local directory.\
\
The protocol followed in (1) Amazon\'92s S3 storage system consists of using the secret and access keys to enter the user-specified account, also the user must specify the bucket name, which is a logical space used to organize data.  Access to Amazon S3 is based on RESTful-style HTTP interface, making each server transaction independent of each other\'92s state.  We use the boto python module for interacting with S3.  (2) SFTP stands for SSH File Transfer Protocol and requires four main parameters: the host-name, user-name, a private key and the file list to be uploaded. All four of these parameters are used to establish a connection with the server and then upload  the list of  that was specified. We use the paramiko python module for interacting with ssh/sftp-based storage locations.  Another way to store files from an experiment is in a (3) local directory, in which the user specifies the base directory to store the files.  This option assumes that users have direct access to a measurement end system, which in general is likely not to be the case.\
 \
\pard\pardeftab720\ri0
\ls5\ilvl0
\i \cf4 Schema for metadata:\
\pard\pardeftab720\ri0
\ls5\ilvl0
\i0 \cf0 The current schema for the metadata created by the GIMS passive measurement system is based on the well-known CRAWDAD and DatCat formats, as well as other systems. The following schema was used after taking into account many elements such as the object oriented structure seen in DatCat, the division of metadata into data, tools and authors seen in CRAWDAD, the XML interface between the manager and capturing device in PerfSonar,  GMOC and the use of IPFIX formatted flow records.  Thus, the current structure of the GIMS metadata (and data formats) is consistent with existing I&M protocols and formats.  The current structure of the GIMS metadata can be classified in three main fields Data, Tools and Author/Creators.  (See below for a more detailed description of each field.)\
\pard\pardeftab720\ri0\sb480
\ls5\ilvl0
\b\fs28 \cf3 Identifiers, Annotation, Provenance, Privacy, Description and Processing \
\pard\tx0\tx720\pardeftab720\li720\fi-360\ri0
\ls6\ilvl0
\b0\fs20 \cf0 1.	
\i\fs24 \cf4 Identifiers:
\i0\fs22 \cf0  Simply refer to the different elements being stored for Data, Tools and Authors. Such as trace name, format, start time, author etc.
\i\fs24 \cf4 \
\pard\pardeftab720\ri0
\ls6\ilvl0
\i0\fs22 \cf0 The three main fields in the metadata of each experiment are Data, Tools and Authors. Each of this fields is composed by various attributes that provide more detailed information about the GENI experiment.\
\
The identifiers associated with each field in the GIMS metadata schema are the following:\
\pard\pardeftab720\ri0
\ls6\ilvl0
\i\b \cf4 \
Data:\
\pard\pardeftab720\ri0
\ls6\ilvl0
\f1\i0\b0 \cf0 At capturing time the elements that are stored as Metadata are:\
\pard\pardeftab720\ri0
\ls6\ilvl0
\f0 \cf0 \
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0
\ls7\ilvl0\cf0 \uc0\u9679 	
\b Trace name: 
\b0 Refers to the name of the experiment trace.
\b \
\ls7\ilvl0
\b0 \uc0\u9679 	
\b Byte Count Format:  
\b0 This refers to the format in which bytes of data should be represented. It is set to represent a floating point number in kilobytes.
\b \
\pard\pardeftab720\ri0
\ls7\ilvl0
\b0 \cf0 \
\pard\pardeftab720\ri0\qc
\ls7\ilvl0
\i\b \cf0 <byte_count format=\\"kilobytes\\">112</byte_count>\
\pard\pardeftab720\ri0
\ls7\ilvl0
\i0\fs24 \cf0 \
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0
\ls7\ilvl0
\b0\fs22 \cf0 \uc0\u9679 	
\b\fs24 Start time/End time: 
\b0 Flow start or end time in ISO 8601 format, with milliseconds (
\i YYYY
\i0 -
\i MM
\i0 -
\i DD
\i0  
\i hh
\i0 :
\i mm
\i0 :
\i ss
\i0 .
\i ssss
\i0 ). Start time is printed with a date; end time is not. End time is only present if the flow has a non-zero duration.
\b \
\pard\pardeftab720\ri0\qc
\ls7\ilvl0
\i\b0 \cf0 \
\pard\pardeftab720\ri0\qc
\ls7\ilvl0
\b \cf0 <start_time>2011-06-20 13:44:12</start_time>\
<end_time>13:44:43</end_time>\
\pard\pardeftab720\ri0\qc
\ls7\ilvl0
\b0 \cf0 		\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0
\ls7\ilvl0
\i0\fs22 \cf0 \uc0\u9679 	
\b\fs24 Time zone: 
\b0 Time Zone that will be used to normalize the start/end time and other timestamps.
\b \
\ls7\ilvl0
\b0\fs22 \uc0\u9679 	
\b\fs24 Geographical location: 
\b0 Three char code used to identify the geographical location of the current GIMS node. Originally set to \'91UNK\'92.
\b \
\pard\pardeftab720\ri0
\ls7\ilvl0
\b0 \cf0 \
\pard\pardeftab720\ri0\qc
\ls7\ilvl0
\i\b \cf0 		<gims_location>UNK</gims_location>\
\pard\pardeftab720\ri0\qc
\ls7\ilvl0
\i0 \cf0 \
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0
\ls7\ilvl0
\b0\fs22 \cf0 \uc0\u9679 	
\b\fs24 Capture Configuration: 
\b0 This attribute refers to the experiment configuration and it includes aggregation, sample type, sample rate, experiment name, device, pcap filter, file rollover and anonymization.
\b \
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0
\ls7\ilvl1
\b0\fs22 \cf0 \uc0\u9675 	
\b\fs24 Agreggation
\b0 : The method of aggregation used. This could be one out of three values: byte packet count, IPFIX or none. 
\b \
\pard\pardeftab720\ri0\qc
\ls7\ilvl0
\i \cf0 \
<aggregation>ipfix</aggregation>\
\pard\pardeftab720\ri0
\ls7\ilvl0
\i0\b0 \cf0 \
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0
\ls7\ilvl1
\fs22 \cf0 \uc0\u9675 	
\b\fs24 Sample Type: 
\b0 The method used to capture packets by the device. It can be set to capture all packets, capture packets at a particular time interval or capture a packet based on probability. 
\b \
\pard\pardeftab720\ri0\qc
\ls7\ilvl0
\i \cf0 <sample_type>all</sample_type>\
\pard\pardeftab720\ri0
\ls7\ilvl0
\i0\b0 \cf0 \
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0
\ls7\ilvl1
\fs22 \cf0 \uc0\u9675 	
\b\fs24 Sample Rate:
\b0  The rate at which samples are taken from the packet flow coming through the device.
\b \
\pard\pardeftab720\ri0
\ls7\ilvl0
\i \cf0 		\
\pard\pardeftab720\ri0\qc
\ls7\ilvl0\cf0 <sample_rate>0.000000</sample_rate>\
\pard\pardeftab720\ri0
\ls7\ilvl0
\i0\b0 \cf0 \
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0
\ls7\ilvl1
\fs22 \cf0 \uc0\u9675 	
\b\fs24 Experiment Name
\b0 : The name used to identify the experiment.
\b \
\pard\pardeftab720\ri0
\ls7\ilvl0
\i \cf0 		\
\pard\pardeftab720\ri0\qc
\ls7\ilvl0\cf0 <exp_name>test</exp_name>\
\pard\pardeftab720\ri0
\ls7\ilvl0
\i0\b0 \cf0 \
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0
\ls7\ilvl1
\fs22 \cf0 \uc0\u9675 	
\b\fs24 Device
\b0 : The device used to capture packets.
\b \
\pard\pardeftab720\ri0
\ls7\ilvl0
\i \cf0 		\
\pard\pardeftab720\ri0\qc
\ls7\ilvl0\cf0 <device>eth0</device>\
\pard\pardeftab720\ri0
\ls7\ilvl0
\i0\b0 \cf0 \
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0
\ls7\ilvl1
\fs22 \cf0 \uc0\u9675 	
\b\fs24 pcap filter
\b0 : Filter to be used to limit what packets will be processed.
\b \
\pard\pardeftab720\ri0
\ls7\ilvl0
\i \cf0 		\
\pard\pardeftab720\ri0\qc
\ls7\ilvl0\cf0 <pcap_filter> ip</pcap_filter>\
\pard\pardeftab720\ri0
\ls7\ilvl0
\i0\b0 \cf0 \
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0
\ls7\ilvl1
\fs22 \cf0 \uc0\u9675 	
\b\fs24 File Rollover Time: 
\b0 Time before it starts overwriting from the begining of the file.
\b \
\pard\pardeftab720\ri0
\ls7\ilvl0
\b0 \cf0 \
\pard\pardeftab720\ri0\qc
\ls7\ilvl0
\i\b \cf0 		<file_rollover_time>30</file_rollover_time>\
\pard\pardeftab720\ri0
\ls7\ilvl0
\i0 \cf0 \
\pard\pardeftab720\ri0
\ls7\ilvl0
\b0 \cf0 \
\pard\tx1080\tx1440\pardeftab720\li1440\fi-360\ri0
\ls7\ilvl1
\fs22 \cf0 \uc0\u9675 	
\b\fs24 Anonymization Key:
\b0  Indicates whether a particular set of data should be anonymized to maintain privacy of experimenters.
\b \
\pard\pardeftab720\ri0
\ls7\ilvl0
\b0 \cf0 \
\pard\pardeftab720\ri0\qc
\ls7\ilvl0
\i\b \cf0 		<anonymization>abk4da23</anonymization>\
\pard\pardeftab720\ri0
\ls7\ilvl0
\i0\b0 \cf0 \
 Sample of Capture Configuration metadata:\
\
\pard\pardeftab720\ri0
\ls7\ilvl0
\i\b \cf0 <capture_config>\
		<aggregation>none</aggregation>\
		<pcap_version> 2.4 </pcap_version>\
		<sample_type>all</sample_type>\
		<sample_rate>0.000000</sample_rate>\
		<exp_name>test</exp_name>\
		<exp_name>test</exp_name>\
		<device>eth0</device>\
		<pcap_filter> ip</pcap_filter>\
		<file_rollover_time>30</file_rollover_time>\
		<anonymization>none</anonymization>\
	</capture_config>\
\pard\pardeftab720\ri0
\ls7\ilvl0
\i0\b0 \cf0 \
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0
\ls7\ilvl0
\fs22 \cf0 \uc0\u9679 	
\b\fs24 Storage type: 
\b0 This defines the type of storage to be used in to store data in the experiment. Three possible options are Amazon\'92s S3 storage service (\'91s3\'92), SFTP (\'91ssh\'92) and in a local directory (\'91local\'92).
\b \
\ls7\ilvl0
\b0\fs22 \uc0\u9679 	
\b\fs24 Packets dropped: 
\b0 The amount of packets dropped per time interval from the raw stream.
\b \
\pard\pardeftab720\ri0
\ls7\ilvl0
\b0 \cf0 \
\pard\pardeftab720\ri0\qc
\ls7\ilvl0
\i\b \cf0       <device_packets_dropped>number</device_packets_dropped>\
\pard\pardeftab720\ri0
\ls7\ilvl0
\i0\b0\fs22 \cf0 \
\
\pard\pardeftab720\ri0
\ls7\ilvl0
\i\b \cf4 Tools:\
\pard\pardeftab720\ri0
\ls7\ilvl0
\i0\b0\fs24 \cf0 Versioning details for ancillary libraries (e.g., YAF,\
libpcap) such as:\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0
\ls8\ilvl0\cf0 \uc0\u9679 	
\b GIMS version:
\b0  This refers to the version of GIMS used.
\b \
\pard\pardeftab720\ri0
\ls8\ilvl0
\b0 \cf0 		\
\pard\pardeftab720\ri0\qc
\ls8\ilvl0
\i\b \cf0 <gims_version>0.1</gims_version>\
\pard\pardeftab720\ri0
\ls8\ilvl0
\i0\b0 \cf0 \
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0
\ls8\ilvl0\cf0 \uc0\u9679 	
\b IPFIX library
\b0 : The IPFIX library installed in the node used to perform the aggregation and generate the record.
\b \
\pard\pardeftab720\ri0
\ls8\ilvl0
\b0 \cf0  \
\pard\pardeftab720\ri0\qc
\ls8\ilvl0
\i\b \cf0   <ipfix_library>" YAF_URL "</ipfix_library>\
\pard\pardeftab720\ri0
\ls8\ilvl0
\i0\b0 \cf0 \
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0
\ls8\ilvl0\cf0 \uc0\u9679 	
\b YAF version:
\b0  Refers to the version of YAF installed. 
\b \
\pard\pardeftab720\ri0
\ls8\ilvl0
\b0 \cf0    \
\pard\pardeftab720\ri0\qc
\ls8\ilvl0
\i\b \cf0 <yaf_version>" YAF_VERSION "</yaf_version>\
\pard\pardeftab720\ri0
\ls8\ilvl0
\i0\b0 \cf0 \
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0
\ls8\ilvl0\cf0 \uc0\u9679 	
\b Pcap version:
\b0  Refers to the version of libpcap installed.
\b \
\pard\pardeftab720\ri0
\ls8\ilvl0
\b0 \cf0 		\
\pard\pardeftab720\ri0\qc
\ls8\ilvl0
\i\b \cf0 <pcap_version> 2.4 </pcap_version>\
\pard\pardeftab720\ri0
\ls8\ilvl0
\i0\b0 \cf0 \
\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0
\ls8\ilvl0\cf0 \uc0\u9679 	
\b FIXBUF version:
\b0  Refers to the version of FIXBUF installed
\b \
\pard\pardeftab720\ri0
\ls8\ilvl0
\b0 \cf0   \
\pard\pardeftab720\ri0\qc
\ls8\ilvl0
\i\b \cf0  <fixbuf_version>" FIXBUF_VERSION "</fixbuf_version>\
\pard\pardeftab720\ri0
\ls8\ilvl0
\i0\b0 \cf0 \
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0
\ls8\ilvl0\cf0 \uc0\u9679 	
\b Platform
\b0 : Operating system used to run experiment.
\b \
\pard\pardeftab720\ri0
\ls8\ilvl0
\b0 \cf0 		\
\pard\pardeftab720\ri0\qc
\ls8\ilvl0
\i\b \cf0 <platform>ubuntu</platform>\
\pard\pardeftab720\ri0
\ls8\ilvl0
\i0\b0 \cf0 \
\
\pard\pardeftab720\ri0
\ls8\ilvl0
\i\b\fs22 \cf4 Creators:\
\pard\pardeftab720\ri0
\ls8\ilvl0
\i0\b0\fs24 \cf0 The user-specified metadata such as:\
\pard\tx360\tx720\pardeftab720\li720\fi-360\ri0
\ls9\ilvl0\cf0 \uc0\u9679 	
\b Creator
\b0 : Consists of a name and email pair to identify the creator(s) of an experiment. Notice that there can be multiple creators per experiment.
\b \
\ls9\ilvl0
\b0 \uc0\u9679 	
\b Primary contact
\b0 : The name and email of the main contact or manager responsible for an experiment.
\b \
\pard\pardeftab720\ri0
\ls9\ilvl0
\b0 \cf0 \
\pard\pardeftab720\ri0
\ls9\ilvl0
\b \cf0    
\i <creators>
\i0 \
\ls9\ilvl0
\i                 <creator>\
                        <name>John Smith</name>\
                        <email>jsmith@fixme.edu</email>\
                </creator>\
	   <creator>\
                        <name>Bob Smith</name>\
                        <email>bsmith@fixme.edu</email>\
                </creator>\
                <primary_contact>\
                        <name>John Smith</name>\
                        <email>jsmith@fixme.edu</email>\
                </primary_contact>\
        </creators>\
\pard\pardeftab720\ri0
\ls9\ilvl0
\i0\b0 \cf0 \
\pard\tx0\tx720\pardeftab720\li720\fi-360\ri0
\ls10\ilvl0
\fs20 \cf0 2.	
\i\fs24 \cf4 Annotation:
\i0\fs22 \cf0  Refers to notes related to a particular metadata object. The notes may contain any sort of information for which there is no specified field defined in the current metadata structure.
\i\fs24 \cf4 \
\ls10\ilvl0
\i0\fs20 \cf0 3.	
\i\fs24 \cf4 Provenance:
\i0\fs22 \cf0  Where does the data come from, such as data collection location and any information about the network infrastructure around the collection point.  In the Data field, attributes such as start time, end time, time zone, geographical location, collection system and anonymization key. These attributes compose what is known as the provenance of a particular experiment at a given collection point.  
\i\fs24 \cf4 \
\ls10\ilvl0
\i0\fs20 \cf0 4.	
\i\fs24 \cf4 Privacy:
\i0\fs22 \cf0  How does data anonymization work and what part of the metadata object is being anonymized.  In the Data field, the anonymization key attribute tells the program if the packets being aggregated should be anonymized at run time. This is done by hiding the source/destination IP addresses in a prefix-preserving fashion, which ensures the users privacy while preserving the research value of data. 
\i\fs24 \cf4 \
\ls10\ilvl0
\i0\fs20 \cf0 5.	
\i\fs24 \cf4 Description:
\i0\fs22 \cf0  Simply describe the attributes in each of the three fields of the current metadata schema we are using, data, tools and author/creators.
\i\fs24 \cf4 \
\ls10\ilvl0
\i0\fs20 \cf0 6.	
\i\fs24 \cf4 Processing:
\i0\fs22 \cf0  What data are obtained after processing such as hash of trace, trace size and other synchronization information.  Processing is done by a device, either a dedicated hardware component of a software component depending on the node. The data obtained after processing a packet is mainly the timestamp that the packet receives when being processed by the device along with the MD5 hash of trace and trace size. Also filtering, aggregation and sampling are also done by the processing device.
\i\fs24 \cf4 \
\pard\pardeftab720\ri0
\ls10\ilvl0
\i0\fs22 \cf0 \
\
Sample Metadata from an experiment called \'93
\i test01
\i0 \'94 using local storage, without aggregation or anonymization:\
\
\pard\pardeftab720\ri0
\ls10\ilvl0
\i\b \cf0 <?xml version="1.0"?>\
<gims_metadata>\
	<data_file>test_SYR_20110720134035.pcap</data_file>\
	<data_type>pcap</data_type>\
	<start_time>2011-06-20 13:40:35</start_time>\
	<creators>\
		<creator>\
			<name>John Smith</name>\
			<email>jsmith@fixme.edu</email>\
		</creator>\
		<primary_contact>\
			<name>John Smith</name>\
			<email>jsmith@fixme.edu</email>\
		</primary_contact>\
	</creators>\
	<setting>\
		<gims_version>0.1</gims_version>\
		<platform>ubuntuhost4</platform>\
		<gims_location>SYR</gims_location>\
	</setting>\
	<capture_config>\
		<aggregation>none</aggregation>\
		<pcap_version> 2.4 </pcap_version>\
		<sample_type>all</sample_type>\
		<sample_rate>0.000000</sample_rate>\
		<exp_name>test01</exp_name>\
		<exp_name>test01</exp_name>\
		<device>eth0</device>\
		<pcap_filter> ip</pcap_filter>\
		<file_rollover_time>30</file_rollover_time>\
		<anonymization>none</anonymization>\
	</capture_config>\
	<end_summary>\
		<end_time>13:41:06</end_time>\
		<packets_observed>277</packets_observed>\
		<packets_saved>276</packets_saved>\
		<bytes_observed>32600</bytes_observed>\
	</end_summary>\
</gims_metadata>}