| 20 | |
| 21 | |
| 22 | |
| 23 | [[PageOutline]] |
| 24 | |
| 25 | = Instrumentation and Measurement (I&M): GIMI and GEMINI Design Topics at GEC15 = |
| 26 | |
| 27 | Tuesday, October 23, 8:30am - 10am [[BR]] |
| 28 | Room: [[BR]] |
| 29 | Session leader: Harry Mussman (GENI Project Office, Raytheon BBN Technologies) [[BR]] |
| 30 | |
| 31 | == Description == |
| 32 | |
| 33 | This I&M session will focus on the key issues of interest to both the GIMI and GEMINI projects, as they continue to work towards a comprehensive set of I&M tools for GENI experimenters and for infrastructure measurement. In particular, we will consider the services and functions that can be used by both the GIMI and GEMINI projects.[[BR]] |
| 34 | |
| 35 | We will form teams that can resolve these issues. [[BR]] |
| 36 | |
| 37 | Finally, we will discuss the I&M goals for the next 4 months, particularly how to best support experimenters and those doing infrastructure measurement. [[BR]] |
| 38 | |
| 39 | == Agenda == |
| 40 | This is a tentative agenda, which may change. [[BR]] |
| 41 | |
| 42 | == 1) Introductions == |
| 43 | 8:30am [[BR]] |
| 44 | |
| 45 | == 2) Topics for this meeting == |
| 46 | |
| 47 | === T9) GENI Storage and Archive Service (based on iRODS) === |
| 48 | 8:35am [[BR]] |
| 49 | |
| 50 | Shu Huang (RENCI) [[BR]] |
| 51 | |
| 52 | Tasks |
| 53 | * Design and deploy GENI Storage and Archive Service for use by all GENI services. |
| 54 | * Provide functions similar to the Measurement Data Archive (MDA) service prototype, designed by CNRI. |
| 55 | * Use to store all types of experiment artifacts (not just measurement data), and identify objects to be moved to a long-term archive. |
| 56 | * Implement using the iRODS data grid service, by defining rules (interfaces) to provide the necessary GENI functions. [[BR]] |
| 57 | |
| 58 | Design goals and approach |
| 59 | * [wiki:InstMeasTopic_4.9ArchiveService status at GEC13] [[BR]] |
| 60 | * [http://groups.geni.net/geni/attachment/wiki/GEC13Agenda/InstrumentationAndMeasurement/T9b%29%20%20gec13_irods_im_anirban.pptx slides at GEC13] [[BR]] |
| 61 | * iRODS service has been established at RENCI [[BR]] |
| 62 | * iRODS service has been established at Univ of Kentucky [[BR]] |
| 63 | * [http://groups.geni.net/geni/attachment/wiki/GEC14Agenda/IMDesignTopics/irods-gec14-1.pptx iRODS configuration at GEC14] [[BR]] |
| 64 | * Proposed configuration? [[BR]] |
| 65 | * Operations plan? [[BR]] |
| 66 | |
| 67 | Current implementation and next steps |
| 68 | |
| 69 | * [http://groups.geni.net/geni/attachment/wiki/GEC15Agenda/IMDesignTopics/iRODS%20GENI%20plan.docx iRODS GENI Plan] [[BR]] |
| 70 | |
| 71 | Open issues |
| 72 | * Who is lead? [[BR]] |
| 73 | * Will all iRODS services be federated, for redundancy? |
| 74 | * How to move data to service: move file; move SQL DB; or? [[BR]] |
| 75 | * How is persistent object identifier (i.e., handle) generated? also registered? and later resolved? Include necessary Object Identifier (OI) service.[[BR]] |
| 76 | * How are partitions for different users managed? [[BR]] |
| 77 | * How is authorization for a user done? [[BR]] |
| 78 | * How is data in MDOD used to set archive sharing policy? what is required in MDOD? [[BR]] |
| 79 | * How will the GEMINI project use the iRODS service? [[BR]] |
| 80 | * Will other parts of GENI want to use the iRODS service? [[BR]] |
| 81 | * Do we need a breakout session for further discussion? when? [[BR]] |
| 82 | |
| 83 | Team |
| 84 | * Shu Huang (RENCI), GIMI project, Lead |
| 85 | * ? , GEMINI project |
| 86 | * Giridhar Manepalli (CNRI) |
| 87 | * Harry Mussman and Jeanne Ohren (GPO) |
| 88 | |
| 89 | |
| 90 | === T5) Digital Objects, Identifiers and Descriptors === |
| 91 | 9:05am [[BR]] |
| 92 | |
| 93 | Giridhar Manepalli (CNRI) [[BR]] |
| 94 | |
| 95 | Tasks |
| 96 | * Define a comprehensive system for identifying and describing a wide range of digital objects, including all types of experiment artifacts (e.g., rspecs, scripts, measurement data). |
| 97 | * Show how to group objects into one combined object (e.g., all artifacts associated with one experiment), and then identify, describe adn archive this combined object |
| 98 | |
| 99 | Design goals |
| 100 | |
| 101 | Approach and next steps |
| 102 | |
| 103 | * [http://groups.geni.net/geni/attachment/wiki/GEC15Agenda/IMDesignTopics/MD_Workspace_And_Archive_GEC15.pptx Archive, DOI (handle), Descriptors] [[BR]] |
| 104 | |
| 105 | Open issues |
| 106 | |
| 107 | |
| 108 | Team |
| 109 | * Giridhar Manepalli and Larry Lannom (CNRI) |
| 110 | * Harry Mussman and Jeanne Ohren (GPO) |
| 111 | * ?, GIMI project |
| 112 | * ?, GEMINI project |
| 113 | |
| 114 | |
| 115 | === T10) I&M Service-to-Service Authentication and Authorization === |
| 116 | 9:25am [[BR]] |
| 117 | |
| 118 | Ezra Kissel (IU) [[BR]] |
| 119 | |
| 120 | |
| 121 | Tasks |
| 122 | * Establish a approach for providing authentication and authorization between GENI I&M service when they are required. |
| 123 | * Note: One approach included in LAMP tools from IU |
| 124 | * Note: Most authentication and authorization work in GENI has been focused on resource assignment, between an experimenter's tolls and an aggregate manager |
| 125 | |
| 126 | Design goals and approach |
| 127 | |
| 128 | Current implementation and next steps |
| 129 | |
| 130 | [http://groups.geni.net/geni/attachment/wiki/GEMINI/AA-workflow.png GEMINI AA Workflow Configuration] [[BR]] |
| 131 | [http://groups.geni.net/geni/attachment/wiki/GEMINI/101112%20%20gemini_aa.txt GEMINI AA Workflow Process] [[BR]] |
| 132 | [http://groups.geni.net/geni/attachment/wiki/GEMINI/092812b_gemini_aa_continued.txt Status] [[BR]] |
| 133 | [http://groups.geni.net/geni/attachment/wiki/GEMINI/Visio-083112_AuthenticationAuthorization_Figures.pdf AA Reference Configurations] [[BR]] |
| 134 | [http://groups.geni.net/geni/attachment/wiki/GEC15Agenda/IMDesignTopics/IM_AA.pptx I&M A&A Considerations (slides)] [[BR]] |
| 135 | |
| 136 | Open issues |
| 137 | |
| 138 | |
| 139 | Team |
| 140 | * Ezra Kissel (IU), GEMINI project, Lead |
| 141 | * ?, GIMI project |
| 142 | * Harry Mussman and Jeanne Ohren (GPO) |
| 143 | |
| 144 | |
| 145 | == 3) Goals for the next 4 months == |
| 146 | 9:45am [[BR]] |
| 147 | |
| 148 | Harry Mussman (GPO) [[BR]] |
| 149 | |
| 150 | Goals |
| 151 | * Both GIMI and GEMINI tools should be available to Users during this period [[BR]] |
| 152 | * Acceptance testing continued by the GPO (Jeanne Ohren) [[BR]] |
| 153 | * Bug tracking system, plus periodic bug fixes, continued (who are leads?) [[BR]] |
| 154 | * How can we facilitate feedback from Users, after EVERY use? [[BR]] |
| 155 | |
| 156 | * Both GIMI and GEMINI must provide a high level of support to Users [[BR]] |
| 157 | * Tutorials must be kept up-to-date and operational, so that Users can teach themselves. (who are leads?) [[BR]] |
| 158 | * Instructions must be continuously refined, to make the tools easier to use. (who are leads?) [[BR]] |
| 159 | * Requested tool extensions must be gathered from users, based upon their experience. (who are leads?) [[BR]] |
| 160 | * Users with advanced requirements must be found, better identify gaps in the tools. How? (who are leads?) [[BR]] |
| 161 | * Tool extensions must be proposed by the teams, and reviewed with the users. How? (who are leads?) [[BR]] |
| 162 | * A list of tool extensions needs to be developed for inclusion into v1.1, due at GEC15. (who are leads?) [[BR]] |
| 163 | |
| 164 | * Persistent multi-user services must be reliably available to all Users [[BR]] |
| 165 | * User Worksapce (Jeanne Ohren) [[BR]] |
| 166 | * XML messaging service (who is lead?) [[BR]] |
| 167 | * iRODS archive service (who is lead?) [[BR]] |
| 168 | * IREEL analysis and presentation service (who is lead?) [[BR]] |
| 169 | * INSTOOLS portal-to-GUIs service (who is lead?) [[BR]] |
| 170 | * Unified Network Information Service (UNIS) (who is lead?) [[BR]] |
| 171 | |
| 172 | |
| 173 | Discussion |
| 174 | * Do we have leads identified for all items? [[BR]] |
| 175 | |
| 176 | |
| 177 | == 4) Adjourn == |
| 178 | 10:00am [[BR]] |
| 179 | |
| 180 | |
| 181 | = I&M References = |
| 182 | |
| 183 | |
| 184 | |
| 185 | == GENI I&M Architecture and Tools: Goals, Overview and Status == |
| 186 | |
| 187 | * GIMI I&M tool set design [[BR]] |
| 188 | * Based on OML, provides basic host and active network measurements [[BR]] |
| 189 | * Ready for use on ExoGENI racks; successful tutorial [[BR]] |
| 190 | * Introduces XML messaging service, iRODS archive service, IREEL(lab wiki) portal service; use by others? iRODS by GEMINI [[BR]] |
| 191 | * Spiral 5 goals: also work with InstaGENI racks, WiMAX sites [[BR]] |
| 192 | |
| 193 | * GEMINI I&M tool set design [[BR]] |
| 194 | * Based on INSTOOLS and perfSONAR/LAMP, provides basic host and active network measurements [[BR]] |
| 195 | * Ready for use on InstaGENI/protoGENI racks ; successful tutorial [[BR]] |
| 196 | * Introduces portal to GUIs service, UNIS unified ntwk info service; use by others? [[BR]] |
| 197 | * Spiral 5 goals: also work with ExoGENI racks [[BR]] |
| 198 | |
| 199 | * Need simplified MDOD measurement data object descriptor schema [[BR]] |
| 200 | * Also service for creation and editing of MDOD [[BR]] |
| 201 | * Possible extensions to other GENI objects, events [[BR]] |
| 202 | |
| 203 | [http://groups.geni.net/geni/attachment/wiki/GEC14Agenda/IMDesignTopics/071112%20I%26M%20GEC14%20Wrap-Up%20Agenda.ppt slides] [[BR]] |
| 204 | |
| 205 | |
| 206 | |
| 207 | === GIMI I&M Tools: Goals, Overview and Status === |
| 208 | |
| 209 | '''PI''' Michael Zink, University of Massachusetts at Amherst (mailto:zink@ecs.umass.edu) [[BR]] |
| 210 | '''Co-PI''' Max Ott, NICTA (mailto:max.ott@nicta.com.au) [[BR]] |
| 211 | '''Key personnel''' Ilia Baldine, RENCI (mailto:ibaldin@renci.org) [[BR]] |
| 212 | |
| 213 | [wiki:GIMI GIMI wiki] [[BR]] |
| 214 | [http://gimi.ecs.umass.edu/ GIMI website at UMass Amherst] [[BR]] |
| 215 | |
| 216 | Goals [[BR]] |
| 217 | * Starts with OML, and provides easy-to-use tools for GENI environment [[BR]] |
| 218 | |
| 219 | * First focus: I&M use cases for experiments, with default OML Client in nodes that gathers passive measurements like INSTOOLS, and provides basic node-to-node connectivity tests using ping and iperf [[BR]] |
| 220 | * Second focus: I&M use cases for infrastructure measurement [[BR]] |
| 221 | * Later focus: add sensor measurement data, carried in data network [[BR]] |
| 222 | |
| 223 | * Spiral 4 supported aggregates: ORCA servers/VMs and ExoGENI (RENCI) racks [[BR]] |
| 224 | * Later, support: WiMAX sites [[BR]] |
| 225 | * Spiral 5 supported aggregates: protoGENI/Emulab servers/VMs and InstaGENI (HP) racks [[BR]] |
| 226 | |
| 227 | * Introduces XML messaging service, with pub/sub, in public IP space [[BR]] |
| 228 | * Introduces iRODS service, as a measurement data archive. [[BR]] |
| 229 | * Introduces IREEL portal service, for measurement data analysys and presentation. [[BR]] |
| 230 | |
| 231 | |
| 232 | Summary: [[BR]] |
| 233 | |
| 234 | * [http://groups.geni.net/geni/attachment/wiki/GEC14Agenda/IMDesignTopics/GIMI%20GEC%2014%20I%26M%20Session.pptx GIMI architecture, design and status at GEC14; plan for GEC-14 tutorial; next steps during Spiral 5] [[BR]] |
| 235 | * For more information, attend or audit [http://groups.geni.net/geni/wiki/GIMIv1.0Tutorial GIMI v1.0 tutorial] [[BR]] |
| 236 | |
| 237 | |
| 238 | |
| 239 | === GEMINI I&M Tools: Goals, Overview and Status === |
| 240 | |
| 241 | '''PI''' Martin Swany, Indiana University (mailto:swany@iu.edu) [[BR]] |
| 242 | '''Co-PI''' Chris Small, Indiana University (mailto:chsmall@indiana.edu) [[BR]] |
| 243 | '''Co-PI''' James Griffioen, University of Kentucky (mailto:griff@netlab.uky.edu) [[BR]] |
| 244 | '''Co-PI''' Zongming Fei, University of Kentucky (mailto:fei@netlab.uky.edu) [[BR]] |
| 245 | |
| 246 | [wiki:GEMINI GEMINI wiki] [[BR]] |
| 247 | |
| 248 | Goals [[BR]] |
| 249 | |
| 250 | * Includes LAMP code and features in GEMINI tools, to provides flexible tools based on perfSONAR for active infrastructure measurements [[BR]] |
| 251 | * Includes INSTOOLS code and features in GEMINI tools, to gather basic host measurements [[BR]] |
| 252 | * Later, integrates LAMP and INSTOOLS code and features into new code base for GEMINI tools. [[BR]] |
| 253 | |
| 254 | * First focus: I&M use cases for infrastructure measurement [[BR]] |
| 255 | * Include sharing of data with other users (e.g., operators), at an interface with authorization using a GENI credential [[BR]] |
| 256 | * Show how to monitor clusters/racks [[BR]] |
| 257 | * Show how to measure Layer 2 and OpenFlow paths (with input from OnTimeMeasure project) [[BR]] |
| 258 | |
| 259 | * Second focus: I&M use cases for experiments [[BR]] |
| 260 | * Gather basic host measurements (as was done by INSTOOLS using SNMP) [[BR]] |
| 261 | * Show how to gather customized data from applications (as provided by OML Client) [[BR]] |
| 262 | |
| 263 | * Spiral 4 supported aggregates: protoGENI/Emulab servers/VMs and InstaGENI (HP) racks [[BR]] |
| 264 | * Spiral 5 supported aggregates: ORCA servers/VMs and ExoGENI (RENCI) racks [[BR]] |
| 265 | |
| 266 | * Continues INSTOOLS portal service, to find GUIs [[BR]] |
| 267 | * Continues LAMP local and global Unified Network Information Service (UNIS), to register available measurement data, and network topology [[BR]] |
| 268 | |
| 269 | Summary [[BR]] |
| 270 | |
| 271 | * [http://groups.geni.net/geni/attachment/wiki/GEC14Agenda/IMDesignTopics/GEMINI%20Overview%20and%20Direction-GEC14.pptx GEMINI architecture, design and status at GEC14; plan for GEC-14 tutorial; next steps during Spiral 5] [[BR]] |
| 272 | * For more information, attend or audit [http://groups.geni.net/geni/wiki/GEMINITutorial GEMINI v1.0 tutorial] [[BR]] |
| 273 | |
| 274 | |
| 275 | |
| 276 | |
| 277 | |
| 278 | == Persistent Multi-user I&M Services == |
| 279 | |
| 280 | |
| 281 | Goals [[BR]] |
| 282 | * Use by both GIMI and GEMINI tools sets [[BR]] |
| 283 | * Use by other parts of GENI [[BR]] |
| 284 | * Identify need for further reviews and discussions [[BR]] |
| 285 | |
| 286 | |
| 287 | === S1) GENI (XML) Messaging Service === |
| 288 | |
| 289 | === S2) GENI Storage and Archive Service (based on iRODS) === |
| 290 | 9:15am [[BR]] |
| 291 | |
| 292 | Shu Huang (RENCI) [[BR]] |
| 293 | |
| 294 | Tasks |
| 295 | * Establish iRODS service(s), and define rules (interfaces) to provide GENI measurement data archive functions. [[BR]] |
| 296 | |
| 297 | Summary |
| 298 | * [wiki:InstMeasTopic_4.9ArchiveService status at GEC13] [[BR]] |
| 299 | * [http://groups.geni.net/geni/attachment/wiki/GEC13Agenda/InstrumentationAndMeasurement/T9b%29%20%20gec13_irods_im_anirban.pptx slides at GEC13] [[BR]] |
| 300 | * iRODS service has been established at RENCI [[BR]] |
| 301 | * iRODS service has been established at Univ of Kentucky [[BR]] |
| 302 | * [http://groups.geni.net/geni/attachment/wiki/GEC14Agenda/IMDesignTopics/irods-gec14-1.pptx iRODS configuration at GEC14] [[BR]] |
| 303 | * Proposed configuration? [[BR]] |
| 304 | * Operations plan? [[BR]] |
| 305 | |
| 306 | Discussion |
| 307 | * Who is lead? [[BR]] |
| 308 | * Will all iRODS services be federated, for redundancy? |
| 309 | * How to move data to service: move file; move SQL DB; or? [[BR]] |
| 310 | * How is persistent object identifier (i.e., handle) generated? also registered? and later resolved? Include necessary Object Identifier (OI) service.[[BR]] |
| 311 | * How are partitions for different users managed? [[BR]] |
| 312 | * How is authorization for a user done? [[BR]] |
| 313 | * How is data in MDOD used to set archive sharing policy? what is required in MDOD? [[BR]] |
| 314 | * How will the GEMINI project use the iRODS service? [[BR]] |
| 315 | * Will other parts of GENI want to use the iRODS service? [[BR]] |
| 316 | * Do we need a breakout session for further discussion? when? [[BR]] |
| 317 | |
| 318 | === S3) GIMI Portal Service === |
| 319 | 9:30am [[BR]] |
| 320 | |
| 321 | Max Ott and Christoph Dwertmann (NICTA); Mike Zink and Cong Wang (UMass Amherst) [[BR]] |
| 322 | |
| 323 | Tasks |
| 324 | * Establish IREEL service, to provide measurement analysis and presentation functions, starting with GIMI project [[BR]] |
| 325 | * Extend/change to become Lab wiki service [[BR]] |
| 326 | |
| 327 | Summary |
| 328 | * IREEL service has been established at NICTA [[BR]] |
| 329 | * IREEL service has been established at UMassAmherst [[BR]] |
| 330 | * Current configuration [ slides] [[BR]] |
| 331 | * Proposed configuration [ slides] [[BR]] |
| 332 | * Operations plan [ slides] [[BR]] |
| 333 | |
| 334 | Discussion |
| 335 | * Who is lead? [[BR]] |
| 336 | * How are partitions for different users managed? [[BR]] |
| 337 | * How is authorization for a user done? [[BR]] |
| 338 | * Can IREEL be used to provide other functions, e.g., experiment meangement? if so, how? [[BR]] |
| 339 | * Will the GEMINI project want to use the IREEL service? [[BR]] |
| 340 | * Will other parts of GENI want to use the IREEL service? [[BR]] |
| 341 | * Do we need a breakout session for further discussion? when? [[BR]] |
| 342 | |
| 343 | === S4) GEMINI Portal Service === |
| 344 | 9:40am [[BR]] |
| 345 | |
| 346 | Charles Carpenter (U Kentucky) [[BR]] |
| 347 | |
| 348 | Tasks: |
| 349 | * Continue INSTOOLS portal-to-GUIs service, to allow user to locate GUIs within their slice, for use with INSTOOLS and GEMINI tools [[BR]] |
| 350 | |
| 351 | Summary |
| 352 | * [http://groups.geni.net/geni/attachment/wiki/GEC14Agenda/IMDesignTopics/GEMINI%20Portal.ppt current GEMINI portal service configuration] [[BR]] |
| 353 | |
| 354 | Discussion |
| 355 | * Who is lead? [[BR]] |
| 356 | * How are partitions for different users managed? [[BR]] |
| 357 | * How is authorization for a user done? [[BR]] |
| 358 | * What additional features are under consideration? [[BR]] |
| 359 | * Will the GIMI project want to use the INSTOOLS portal-to-GUIs service? [[BR]] |
| 360 | * Will other parts of GENI want to use the INSTOOLS portal-to-GUIs service? [[BR]] |
| 361 | * Do we need a breakout session for further discussion? when? [[BR]] |
| 362 | |
| 363 | === S5) Unified Network Information Service (UNIS) === |
| 364 | 9:45am [[BR]] |
| 365 | |
| 366 | Ahmed El-Hassany (IU) [[BR]] |
| 367 | |
| 368 | Tasks [[BR]] |
| 369 | * Continue Unified Network Information Service (UNIS), to provide combined Lookup and Topology Services, initially for use with GEMINI tool set. |
| 370 | |
| 371 | Summary |
| 372 | * [wiki:InstMeasTopic_4.7LookupService status at GEC13] [[BR]] |
| 373 | * [http://groups.geni.net/geni/attachment/wiki/GEC14Agenda/IMDesignTopics/UNIS_GEC14.pdf UNIS configuration at GEC14; proposed configuration] [[BR]] |
| 374 | |
| 375 | Discussion |
| 376 | * Who is lead? [[BR]] |
| 377 | * Will you refactor UNIS (combined Lookup and Topology Services) to support hierarchical operation with local and global instances? [[BR]] |
| 378 | * What additional features are under consideration? [[BR]] |
| 379 | * What new interfaces are under consideration? [[BR]] |
| 380 | * Will the GIMI project want to use the UNIS service? [[BR]] |
| 381 | * Will other parts of GENI want to use the UNIS service, e.g., stitching? [[BR]] |
| 382 | * Do we need a breakout session for further discussion? when? [[BR]] |