| 494 | == Change Set T: Long Lived Slices == |
| 495 | This topic was raised at the [wiki:GEC18Agenda/CodingSprintTutoring#SessionSummary GEC18 Coding Sprint]. |
| 496 | |
| 497 | This proposal provides a way to support long running experiments that require slices and resource reservations with long lives. This proposal requires changes to Slice Authorities and aggregates. |
| 498 | |
| 499 | '''Proposal''': |
| 500 | |
| 501 | Augment existing slice credentials to add a new possible privilege value `longlived` that means that the issuing Slice Authority (SA) believes that the given slice should be allowed to have an extra long reservation. The proposal does not specify how the SA decides to include this privilege. Requesting a credential for such a slice results in a slice credential that adds this privilege. |
| 502 | |
| 503 | Aggregates must accept slice credentials that include the new privilege and understand that a privilege value of `*` does not include this privilege. When an aggregate receives a slice credential that includes this privilege, the aggregate should allow resources in this slice to be renewed beyond the usual maximum resource reservation lifetime at that aggregate. However, the maximum expiration time for the resources remains subject to local aggregate policy. Some aggregate may allow arbitrary expirations for some resources when given a long lived slice credential. Some aggregates may double the maximum duration of the reservation. And other aggregates for scarce resources may only allow a slight increase over the usual duration. |
| 504 | |
| 505 | '''Question''': Does the privilege say anything about how long the slice' reservations should be good for? I think not. |
| 506 | |
| 507 | '''Alternative''': Add a new credential, in addition to the existing slice credential, which looks just like a normal slice credential, but adds this additional privilege. This requires tools to retrieve and pass around multiple credentials. |
| 508 | |
| 509 | This change requires aggregates to accept and handle these new credentials. |
| 510 | |
| 511 | == Change Set U: Scheduling == |
| 512 | The ability to schedule resource reservations in the future has been discussed multiple times as desirable. This proposal would add support in the AM API to allow aggregates to provide this ability. |
| 513 | |
| 514 | This proposal is incomplete and requires further discussion. |
| 515 | |
| 516 | === An Analogy === |
| 517 | First, an analogy to reserving a table at a restaurant: |
| 518 | |
| 519 | The current AM API does not support reservations - we only support walk-ins. |
| 520 | In AM API v2, we support "Can I have a table?" (`Createsliver`), with the possible answers "No" or "I'll bring you your menus." |
| 521 | |
| 522 | In AM API v3, we support "Can I have a table?" (`Allocate`) with the answers "No" or "I have a table free, would you like it?". Then the experimenter can say "No" (`Delete`) or "Yes, I'll take it" (`Provision`), or "Can you give me a minute to consult with my friends?" (Renew) |
| 523 | |
| 524 | With this proposal, we want to support reservations: |
| 525 | Experimenter: Can I have a table on Tuesday? (`Allocate`) |
| 526 | |
| 527 | Aggregate: I have a table by the door, would you like it? |
| 528 | |
| 529 | - And the restaurant will not give away that table until the experimenter answers, but only waiting a short time. |
| 530 | |
| 531 | E: Give me a minute to consult? (`Renew`) |
| 532 | |
| 533 | or |
| 534 | |
| 535 | E: No thanks (`Delete`) |
| 536 | |
| 537 | or |
| 538 | |
| 539 | E: Yes, please reserve that table for me on Tuesday. (`Provision`?) |
| 540 | |
| 541 | Then when Tuesday rolls around, the restaurant will only hold the table for a limited time. |
| 542 | |
| 543 | E: I'm here, ready to sit down. (`Provision`? `PerformOperationalAction` of `geni_start`?) |
| 544 | |
| 545 | or |
| 546 | |
| 547 | E: Never mind, I don't want the table. (`Delete`) |
| 548 | |
| 549 | Or the experimenter doesn't show up, and the reservation "expires" and the restaurant is free to give the table away. |
| 550 | |
| 551 | === Goals === |
| 552 | - Experimenters can specify both start and end time for when they want resources |
| 553 | - Experimenters must commit to using the resources, so that aggregates are not holding resources that the experimenter will never use |
| 554 | - Experimenters know when they have the resources |
| 555 | - Experimenters know how long they may be able to hold the resources, and when the resources are likely taken by someone else |
| 556 | - Support for scheduling is optional |
| 557 | |
| 558 | === Proposal === |
| 559 | |
| 560 | - `Allocate` takes both a `geni_start_time` (new) and `geni_end_time` (already allowed). |
| 561 | - The `geni_start_time` option to `Allocate` indicates when the experimenter wants the reservation to start. The semantics are such that the aggregate may give the client the resources early, or up to 10 minutes later than requested, but no more. If the aggregate cannot reserve the requested resources for the client within that time window, then the request must fail with the error code indicating the resources are UNAVAILABLE. Aggregates may include in the error some message indicating if a slightly different start time would succeed, if known. |
| 562 | - Aggregates that do not support scheduled reservations will ignore this option, always treat the request as a request for resources now. Clients may examine the return from `Allocate` to and see that `geni_start_time` and `geni_end_time` is not specified in the return, indicating that this is not a scheduled reservation, but an immediate reservation. |
| 563 | - Note that `geni_start_time` is not an option to `Provision` |
| 564 | |
| 565 | - Add a `geni_max_expiration` field wherever we currently return a sliver expiration. This is advisory, indicating the latest datetime that you should expect to be able to renew your reservation until. This may be based on local aggregate policy only, or based on known other scheduled future reservations for the same resource(s). Note though that this is not a guarantee: a request to extend a reservation up until this time may still be denied (other resource reservations that come in in the interim for example), and it is even possible that a request to renew past this time might succeed, though clients should not expect this. |
| 566 | |
| 567 | - Add a `geni_start_time` field to the return wherever we currently return a sliver expiration. This field is required only for reservations that begin in the future. |
| 568 | |
| 569 | - `geni_expires` is slightly redefined. This now indicates the end of the time window in which you must act on the sliver(s) / reservation, or else the slivers will expire and be given to others. This is consistent with how the field is used by aggregates that do not support scheduling today. Aggregates that do support scheduling however will use this and a new option. |
| 570 | |
| 571 | - Add a `geni_end_time` return element wherever `geni_expires` is returned, to be required only for aggregates that support scheduling, and only when both a `geni_start_time` and `geni_end_time` was supplied in the original reservation request. This element specifies the time at which the reservation has been requested to end. For example, `geni_start_time` might be tomorrow at 1pm, `geni_end_time` might be tomorrow at 3pm, and `geni_expires might be 10 minutes from now. This indicates that the client has 10 minutes in which to confirm the reservation for resources tomorrow from 1 to 3pm. |
| 572 | |
| 573 | - After `Allocate`, scheduled requests are `geni_allocated` as before. As before, such slivers have a short expiration time (given in `geni_expires`), during which time `Delete` and `Renew` are both valid. |
| 574 | - `Renew` does not change the requested start and end times of the reservation, it only extends the time window in which the client must confirm the reservation. To change the requested start or end times of an allocated reservation, use `Update`, or call `Delete` and then `Allocate`. |
| 575 | - Experimenters must call `Provision` as before (arguments are unchanged) to confirm the reservation during this time window (before `geni_expires`). |
| 576 | - If the client does not call `Provision` during that time window, the slivers expire and the reservation is cancelled. |
| 577 | - If the client ''does'' call `Provision` for a reservation in the future (has a `geni_start_time`), then the sliver(s) transition to the new `geni_scheduled` state. The aggregate does ''not'' begin instantiating the resources at this point. The aggregate only changes the state of the slivers and the `geni_expires` time at which the slivers will expire (see below). |
| 578 | |
| 579 | - Add a new `geni_scheduled` allocation state after `geni_allocated` and before `geni_provisioned`. Slivers in this state are being held for the client for some future time. The client has already confirmed that they want the resources as promised, but the requested start time has not arrived or the experimenter has not confirmed that they are ready to start using their reservation. This state is only used by aggregates that support scheduling and only for requests for resources in the future. |
| 580 | - While in this state, the client may call `Describe`, `Status`, `Delete`, or `Update`. `Renew` is not legal on slivers in the `geni_scheduled` state: aggregates may raise an error, or simply return the existing expiration times and states. |
| 581 | - `Update` moves the slivers to the `geni_updating` state; clients must confirm the new request with a new call to `Provision` before the time in `geni_expires`, or else the update request expires, the slivers return to the `geni_scheduled` state, and the reservation remains unchanged. Regardless of the specified `geni_start_time`, resources will not be instantiated until the slivers move out of the `geni_updating` state. `Update` ''may'' be used to request a change to both what resources are reserved, and the requested `geni_start_time` and `geni_end_time`. |
| 582 | - When `Provision` is used on `geni_allocated` slivers with a `geni_start_time` in the future, it moves the slivers to the `geni_scheduled` state. The return must include `geni_start_time`, `geni_end_time`, and `geni_expires. `geni_end_time` is the end of the requested reservation window. `geni_expires is a time some small delta after the `geni_start_time`. This indicates the time at which the slivers will expire, if the client has not claimed the slivers with a second call to `Provision`. |
| 583 | |
| 584 | - When the `geni_start_time` arrives, the client must call `Provision` (arguments are unchanged), indicating the client is ready to use the resources. Only at this point does the aggregate begin to instantiate the resources. The slivers become `geni_provisioned` as before. |
| 585 | - If the client does not call `Provision` before the sliver expiration time given in `geni_expires`, then the slivers expire and become `geni_unallocated` (as before). |
| 586 | - `Renew` has no effect on slivers that are `geni_scheduled`. |
| 587 | |
| 588 | - `Update` on slivers that are already `geni_provisioned` will not change the `geni_start_time`. It may request a change to the `geni_end_time`, along with changes to what resources are reserved. |
| 589 | |
| 590 | === Questions === |
| 591 | - Is there any way to query the reservation schedule of a particular resource? |
| 592 | |
| 593 | == Change Set V: Create and Manage Disk Images == |
| 594 | This proposal would allow creating, listing and deleting disk images based on the state of existing disks in your slice at an aggregate. |
| 595 | |
| 596 | This proposal is based on the functionality exported by ProtoGENI based aggregates. '''FIXME''': This proposal must be made more generic. |
| 597 | |
| 598 | '''FIXME''' |
| 599 | - Do images _have_ to be kept local to where they started? Or is it possible you can do list/delete at other aggregates? Do images get copied such that now you can list/delete them elsewhere? |
| 600 | - Must images that you snapshot be available at other aggregates, or is that separable / optional? |
| 601 | |
| 602 | These images are created by taking a "snapshot" of the disk of an existing compute resource. Images "belong to" the aggregate at which the original compute resource existed, and will only be listed or deletable there. |
| 603 | |
| 604 | === `CreateImage` === |
| 605 | Create a disk snapshot or image based on the state of the disk for an existing sliver. |
| 606 | |
| 607 | Arguments: |
| 608 | - slice URN |
| 609 | - name for the image: An alphanumeric non empty string |
| 610 | - sliver URN of the machine whose disk should be snapshotted |
| 611 | - credentials giving the caller rights on this slice |
| 612 | - options |
| 613 | - The option `global` defines whether this image will be publicly visible or not. The value is an XML-RPC boolean indicating whether the image should be public, defaulting to False meaning private. |
| 614 | |
| 615 | Returns: |
| 616 | - URN naming the new image. The image URN looks like `urn:publicid:IDN+<aggregate authority>+image+<aggregate authority with punctuation replace by hyphens>:<image name supplied in the argument>`, for example `urn:publicid:IDN+foo.net+image+foo-net:booboo`. |
| 617 | - URL for referencing the image, for use at other aggregates, for example: `https://www.foo.net/image_metadata.php?uuid=c0b47d37-f6cd-11e1-9f72-001143e453fe` |
| 618 | |
| 619 | If an image of the same name already exists, then this image replaces the previous image of the same name. '''FIXME''' This should only be true for private images - public images should not be replaceable and you should get an error. |
| 620 | |
| 621 | This function should return quickly, asynchronously snapshotting the disk. While the disk is being snapshotted, do not modify the state of your VM - results will be unpredictable. |
| 622 | |
| 623 | Only slivers in the `geni_ready` state may be imaged. While the image is being created, the sliver will be in a new operational state, `geni_imaging`. While the sliver is in this state, no operational actions are permitted. When imaging is complete, the operational state transitions back to `geni_ready`. |
| 624 | |
| 625 | The aggregate will email the user who created the snapshot at the email address found in the user's certificate, providing status and results. |
| 626 | |
| 627 | === `DeleteImage` === |
| 628 | Delete the named disk image owned by / created by the named user. |
| 629 | |
| 630 | Arguments: |
| 631 | - image URN |
| 632 | - credentials |
| 633 | - options |
| 634 | - `creator_urn`: image creator URN (a valid "user" URN). If not supplied, the current user is assumed |
| 635 | |
| 636 | Return: XML-RPC boolean for success. On error, return codes include `SEARCHFAILED`. |
| 637 | |
| 638 | '''FIXME: Can you really delete another user's image? Public and Private?''' |
| 639 | |
| 640 | === `ListImages` === |
| 641 | List the disk images created by the given user at the aggregate you are calling. |
| 642 | Arguments: |
| 643 | - creator URN |
| 644 | - credentials |
| 645 | - options |
| 646 | |
| 647 | The creator URN must be from the same authority as the caller (i.e. the portion of the user URN before `user` must be identical). |
| 648 | |
| 649 | Return: a list of structs, each containing the url and urn of the images at this aggregate. |
| 650 | ''' FIXME: Details''' |
| 651 | |
| 652 | === Image URN type === |
| 653 | This proposal introduces a new "type" of URN, `image`, to be used to name disk images. |
| 654 | |
| 655 | ''FIXME: Restrictions on the characters legal in the name of an image or image name length?''' |
| 656 | |
| 657 | === Import Disk Images === |
| 658 | Should there be a common API for asking an aggregate to import a disk image from some other location? |
| 659 | |
| 660 | '''Questions''': |
| 661 | - What about exporting a disk image? |
| 662 | - Are there any more fine grained privacy controls beyond public and private? For example, shared within the current slice? |
| 663 | - Do disk images expire? Is there any quota on the disk space used or # of images a slice or experimenter can have? |
| 664 | - How do aggregates reclaim disk space from disk images? |
| 665 | |
| 666 | == Declined: No Reservation Allocate == |
| 667 | Tools should be able to do complex topology embedding to figure out where a particular request can be instantiated. Aggregates know best what they can do. This proposal provides a cheap way to ask "could you satisfy this request, and if so what would I get?" |
| 668 | |
| 669 | This proposal is ''declined'': `Allocate` itself is pretty cheap. |
| 670 | |
| 671 | - Add a new option to `Allocate` named `geni_no_reservation`. If supplied with a non-empty value (not null or an empty string), the request is not asking for the resources to actually be reserved. Instead, the aggregate should calculate what it ''would'' reserve, and return the usual `allocate` return structure, but without actually making the reservation. As such, `geni_expires` should be set to the current time, and the status should remain `geni_unallocated`, and the `geni_sliver_urn` element should be empty (no sliver is created or assigned). Since there is no sliver URN, no calls to other methods should return any information on this sliver. |
| 672 | |
| 673 | == Change Set W: Rename fields to drop `geni` prefix == |
| 674 | In the interest of making this API better apply to international federations, change all options and return elements that are currently named `geni_foo` to just plain `foo`. Non standard options and return elements should use a local aggregate type specific prefix. |
| 675 | |
| 676 | That is, where the spec currently says: |
| 677 | {{{ |
| 678 | The prefix geni_ is reserved for members that are part of this API specification. Implementations should choose an appropriate prefix to avoid conflicts. |
| 679 | }}} |
| 680 | Instead, it should say: |
| 681 | {{{ |
| 682 | Implementations may add additional options or return elements that are not standard to this API. |
| 683 | To avoid conflicts with options in this specification or other implementations, implementations |
| 684 | should name these non-standard members with their aggregate type specific prefix. |
| 685 | }}} |
| 686 | |
| 687 | See [wiki:GAPI_AM_API_DRAFT#ChangeSetN3:Useconsistenttypesandprefixes Change Set N3]. |
| 688 | |
| 689 | == Change Set X: Split API into services advertised in `GetVersion` == |
| 690 | |
| 691 | The Uniform Federation API defines multiple "services". Each service is a coherent piece of functionality that a given server / authority may implement. For example, some slice authorities support projects and some do not. This proposal would split up the AM API into services, and add a `SERVICES` element in the return from `GetVersion` that is a list of the string names of services supported by this aggregate. |
| 692 | |
| 693 | '''FIXME''' |
| 694 | |
| 695 | '''Proposed services''': |
| 696 | - `SCHEDULING`: Indicates that the aggregate supports scheduled reservations per the previous proposal. Aggregates that support this will accept a `geni_start_time` option to `Allocate`, others will ignore such an option. |
| 697 | - `IMAGES`: This aggregate allows creating and deleting and listing disk images, per the previous proposal. |
| 698 | - `COMPUTE`: This aggregate supports reserving nodes and will list nodes in its Ad RSpec. |
| 699 | - `OPENFLOW`: This aggregate supports reserving or configuration !OpenFlow links |
| 700 | - `STITCHING`: This aggregate supports the stitching extension and creating stitched links between aggregates. |
| 701 | - `STORAGE`: This aggregate supports reserving storage resources. |
| 702 | - `UPDATE`: This aggregate supports the `Update` method |
| 703 | |
| 704 | == Change Set Y: Support UTF8 == |
| 705 | To more fully support international users and sites, support UTF8 characters in usernames, slice names, sliver names, authorities, tool names, and RSpecs. |
| 706 | |
| 707 | Regular expressions defining those fields must change appropriately. Slice authorities, tools and aggregates must all change. |
| 708 | |
| 709 | See [wiki:GAPI_AM_API_DRAFT#ChangeSetO3:Allowunicodevalues Change Set O3]. |
| 710 | |
| 711 | '''FIXME''': Is that proposal sufficient as is? |
| 712 | |
| 713 | == Change Set Z: Support `Shutdown` on a single sliver == |
| 714 | This proposal would allow calling `Shutdown` with a specific sliver or slivers specified, and would only shut down the named slivers, not the entire slice at this aggregate. |
| 715 | |
| 716 | '''Questions''': |
| 717 | - What if there are interdependencies among slivers such that other slivers must also be shut down? |
| 718 | - Can an aggregate specify that only the whole slice may be shut down? |
| 719 | - Is this like the options for how `Allocate` works? |
| 720 | |
| 721 | '''FIXME''' |
| 722 | |