Subject: [alto] Updated ALTO Interoperability test cases (version 0.5)

From: "Vijay K. Gurbani" <[email-masked]>
Date: Tue, 28 Jun 2011 14:38:35 -0500

Folks: I have updated the ALTO test case document based on
the feedback I have received on the list so far.

More specifically:

- Added an additional CIDR block to mypid1.

- Added an IPv6 address block to peeringpid2, and defaultpid

- Fleshed out Test Case 5 (retrieving a network map) with HTTP
   request and response payload.

   I generated these by hand, so someone may want to make sure that
   the syntax is correct using a JSON parser.

- Fleshed out Test Case 6 (retrieving a cost map) with HTTP request
   and response payload.

   I generated these by hand, so someone may want to make sure that
   the syntax is correct using a JSON parser.

- Fleshed out Test Case 14 (send a request with a bad URI)

- Fleshed out Test Case 15 (change in a network map)

- Fleshed out Test Case 16 (filtered network map)

- Fleshed out Test Case 17 (filtered cost map)

Please take a look at the above changes and make sure they are good.
The latest version of the test document is included below.

In addition, I am looking for some victims who will help me flesh out
the details for the following remaining 10 test cases:

   Test case 1, 7-13, 18, 19

If 4-5 people get in touch with me, I can assign 1 or two test cases
to each.  Please let me know if you can volunteer.  That will be very
helpful.

Thanks so much.

- vijay

---- BEGIN ALTO INTEROPERABILITY DOCUMENT (Version 0.5)

For the ALTO interoperability, we will consider the following topology
from the view point of a service provider:

PID            IP Address Block
-------------------------------
mypid1         10.0.0.0/8, 15.0.0.0/8
mypid2         192.168.0.0/16
mypid3         192.168.10.0/24
peeringpid1    128.0.0.0/16
peeringpid2    130.0.0.0/16, 2001:DB8::/32
transitpid1    132.0.0.0/16
transitpid2    135.0.0.0/16
defaultpid     0.0.0.0/0, ::/0

And here is the cost map for the above topology:

Source PID         Destination PID            Cost (mode: numerical)
--------------------------------------------------------------------
mypid1             mypid1                               0
   "                mypid2                               0
   "                mypid3                               0
   "                peeringpid1                          0
   "                peeringpid2                          0
   "                transitpid1                          5
   "                transitpid2                         10
   "                defaultpid                           4
mypid2             mypid1                               0
   "                mypid2                               0
   "                mypid3                               0
   "                peeringpid1                          0
   "                peeringpid2                          0
   "                transitpid1                          7
   "                transitpid2                          8
   "                defaultpid                           4
mypid3             mypid1                               0
   "                mypid2                               0
   "                mypid3                               0
   "                peeringpid1                          0
   "                peeringpid2                          0
   "                transitpid1                          8
   "                transitpid2                          8
   "                defaultpid                           5.1

Test Cases

1) Retrieving the Information Resource Directory (IRD, Section 7.6)

    - Client retrieve IRD through a well known URI:
         http://alto.ietf.org/directory
    - Server returns the following directory:
      (TBD)

2) Longest prefix match (c.f. Section 4.2.1)

    - Client retrieves a PID for IP address 192.168.1.23
      Use the following request:

      POST /endpointprop/lookup HTTP/1.1
      Host: alto.ietf.org
      Content-Length: ...
      Accept: application/alto-endpointprop+json

      {
         "properties" : [ "pid" ],
         "endpoints" : [ "ipv4:192.168.1.23" ]
       }

    - Server returns mypid2 using the following response:

    HTTP/1.1 200 OK
    Content-Length: ...
    Content-Type: application/alto-endpointprop+json

    {
      "meta" : {},
      "data": {
        "ipv4:192.168.1.23"    : { "pid": "mypid2" }
      }
    }

3) Longest prefix match (c.f. Section 4.2.1)

    - Client retrieves a PID for IP address 192.168.10.23
      Use the following request:

      POST /endpointprop/lookup HTTP/1.1
      Host: alto.ietf.org
      Content-Length: ...
      Accept: application/alto-endpointprop+json

      {
         "properties" : [ "pid" ],
         "endpoints" : [ "ipv4:192.168.10.23" ]
       }

    - Server returns mypid3 using the following response:

    HTTP/1.1 200 OK
    Content-Length: ...
    Content-Type: application/alto-endpointprop+json

    {
      "meta" : {},
      "data": {
        "ipv4:192.168.10.23"    : { "pid": "mypid3" }
      }
    }

4) Returning default PID:

    - Client retrieves a PID for IP address 201.1.13.12
      Use the following request:

      POST /endpointprop/lookup HTTP/1.1
      Host: alto.ietf.org
      Content-Length: ...
      Accept: application/alto-endpointprop+json

      {
         "properties" : [ "pid" ],
         "endpoints" : [ "ipv4:201.1.13.12" ]
       }

    - Server returns defaultpid using the following response:

    HTTP/1.1 200 OK
    Content-Length: ...
    Content-Type: application/alto-endpointprop+json

    {
      "meta" : {},
      "data": {
        "ipv4:201.1.13.12"    : { "pid": "defaultpid" }
      }
    }

5) Retrieving a network map (Section 4, Section 7.7.2)

    - Client uses information from IRD to retrieve a network map.
      Use the following request:

      GET /networkmap HTTP/1.1
      Host: alto.ietf.org
      Accept: application/alto-networkmap+json,
              application/alto-error+json

    - Server returns the following:

    HTTP/1.1 200 OK
    Content-Length: 799
    Content-Type: application/alto-networkmap+json

    {
       "meta" : {},
       "data" : {
          "map-vtag" : "1266506139",
          "map" : {
             "mypid1" : {
                "ipv4" : [ "10.0.0.0/8", "15.0.0.0/8" ]
             },
             "mypid2" : {
                "ipv4" : [ "192.168.0.0/16" ]
             },
             "mypid3" : {
                "ipv4" : [ "192.168.10.0/24" ]
             },
             "peeringpid1" : {
                "ipv4" : [ "128.0.0.0/16" ]
             },
             "peeringpid2" : {
                "ipv4" : [ "130.0.0.0/16" ],
                "ipv6" : [ "2001:DB8::/32"]
             },
             "transitpid1" : {
                "ipv4" : [ "132.0.0.0/16" ]
             },
             "transitpid2" : {
                "ipv4" : [ "135.0.0.0/16" ]
             },
             "defaultpid" : {
                "ipv4" : [ "0.0.0.0/0" ],
                "ipv6" : [ "::/0" ]
             }
          }
       }
    }

6) Retrieving a cost map (Section 5)

    - Client uses information from IRD to retrieve a cost map.
      Use the following request:

    GET /costmap/num/routingcost HTTP/1.1
    Host: alto.ietf.org
    Accept: application/alto-costmap+json,application/alto-error+json

    - Server returns the following:

    HTTP/1.1 200 OK
    Content-Length: 787
    Content-Type: application/alto-costmap+json

    {
      "meta" : {},
      "data" : {
        "cost-mode" : "numerical",
        "cost-type" : "routingcost",
        "map-vtag"  : "1266506139",
        "map" : {
          "mypid1": { "mypid1" : 0,  "mypid2" : 0,  "mypid3" : 0,
                      "peeringpid1" : 0, "peerinpid2" :  0,
                      "transitpid1" : 5, "transitpid2" : 10,
                      "defaultpid" : 4},
          "mypid2": { "mypid1" : 0,  "mypid2" : 0,  "mypid3" : 0,
                      "peeringpid1" : 0, "peerinpid2" :  0,
                      "transitpid1" : 7, "transitpid2" : 8,
                      "defaultpid" : 4},
          "mypid3": { "mypid1" : 0,  "mypid2" : 0,  "mypid3" : 0,
                      "peeringpid1" : 0, "peerinpid2" :  0,
                      "transitpid1" : 8, "transitpid2" : 8,
                      "defaultpid" : 5.1}
        }
      }
    }

    NOTE: Make sure that the vtag in the response for the cost
    map is the same as the vtag in the response for Test case 5.

7) Error: JSON parsing error (Section 7)

    - Client sends a malformed JSON body in the request (details TBD)
    - Server returns HTTP response code of 400 with ALTO error code
      of E_JSON_SYNTAX (c.f., Table 1).

8) Error: Required field missing (Section 7)

    - Client sends a request with a missing field (details TBD)
    - Server returns HTTP response code of 400 with ALTO error code
      of E_JSON_FIELD_MISSING (c.f., Table 1).

9) Error: Unexpected JSON value (Section 7)

    - Client sends a request with a JSON value of unexpected type
     (details TBD)
    - Server returns HTTP response code of 400 with ALTO error code
      of E_JSON_VALUE_TYPE (c.f., Table 1).

10) Error: Invalid JSON cost mode (Section 7)

    - Client sends a request with a cost mode of 'foo'
    - Server returns HTTP response code of 400 with ALTO error code
      of E_JSON_COST_MODE (c.f., Table 1).

11) Error: Invalid JSON cost type (Section 7)

    - Client sends a request with a cost type of 'foo'
    - Server returns HTTP response code of 400 with ALTO error code
      of E_JSON_COST_TYPE (c.f., Table 1).

12) Error: Invalid JSON property type (Section 7)

    - Client sends a request with an invalid property type (details TBD)
    - Server returns HTTP response code of 400 with ALTO error code
      of E_JSON_PROPERTY_TYPE (c.f., Table 1).

13) Error: multiple errors in request (Section 7)

     - Client sends a request with a missing required field and an
       invalid cost mode ('foo')
     - Server should detect at least one error and return that in a
       response.  The server chooses which error to report.

      Question: If the server is capable of detecting and reporting
      multiple errors, then what?  How does it let the client know?

14) Error: send a request with a bad URI

     - Client sends the following request:

       GET /foo/bar HTTP/1.1
       Host: alto.ietf.org
       Accept: application/alto-costmap+json,application/alto-error+json

     - Server responds with:

       HTTP/1.1 404 Not Found
       Content-Length: 0

15) Change in network map (Section 5.3)

     - Add a new network, changing the map.  For example, add the
       following subnet 201.1.2.0/24 to peeringpid2 giving the new
       network map entry for peeringpid2 as follows:

       peeringpid2    130.0.0.0/16, 2001:DB8::/32, 201.1.2.0/24

     - Have a client retrieve the new network map.

     - The Version tag on the network map should be different than the
       one retrieved in Test Case 5.

     - The server will respond with the following:

    HTTP/1.1 200 OK
    Content-Length: 827
    Content-Type: application/alto-networkmap+json

    {
       "meta" : {},
       "data" : {
          "map-vtag" : "1266506155",
          "map" : {
             "mypid1" : {
                "ipv4" : [ "10.0.0.0/8", "15.0.0.0/8" ]
             },
             "mypid2" : {
                "ipv4" : [ "192.168.0.0/16" ]
             },
             "mypid3" : {
                "ipv4" : [ "192.168.10.0/24" ]
             },
             "peeringpid1" : {
                "ipv4" : [ "128.0.0.0/16" ]
             },
             "peeringpid2" : {
                "ipv4" : [ "130.0.0.0/16" ],
                "ipv4" : [ "201.1.2.0/24" ],
                "ipv6" : [ "2001:DB8::/32"]
             },
             "transitpid1" : {
                "ipv4" : [ "132.0.0.0/16" ]
             },
             "transitpid2" : {
                "ipv4" : [ "135.0.0.0/16" ]
             },
             "defaultpid" : {
                "ipv4" : [ "0.0.0.0/0" ],
                "ipv6" : [ "::/0" ]
             }
          }
       }
    }

16) Filtered network map (Section 7.7.3)

    - Client sends a request to get a filtered map of mypid2:
    Use the following request:

    POST /networkmap/filtered HTTP/1.1
    Host: alto.ietf.org
    Content-Length: 26
    Content-Type: application/alto-networkmapfilter+json
    Accept: application/alto-networkmap+json,application/alto-error+json

    {
      "pids": [ "mypid2" ]
    }

   - Server returns the following response:

    HTTP/1.1 200 OK
    Content-Length: 172
    Content-Type: application/alto-networkmap+json

    {
       "meta" : {},
       "data" : {
          "map-vtag" : "1266506155",
          "map" : {
             "mypid2" : {
                "ipv4" : [ "192.168.0.0/16" ]
             },
          }
       }
    }

17) Filtered cost map (Section 7.7.3.2, Section 7.7.2.2)

     - Client sends a request to get a filtered cost map from a source
       PID to a set of destination PIDs.
       Use the following request:

    POST /costmap/filtered HTTP/1.1
    Host: alto.ietf.org
    Content-Type: application/alto-costmapfilter+json
    Content-Length: 174
    Accept: application/alto-costmap+json,application/alto-error+json

    {
      "cost-mode" : "numerical",
      "cost-type" : "routingcost",
      "pids" : {
        "srcs" : [ "mypid1", "mypid3" ]
        "dsts" : [ "mypid2", "peeringpid1", "transitpid2" ]
      }
    }

   - Server responds with the following:

    HTTP/1.1 200 OK
    Content-Length: 294
    Content-Type: application/alto-costmap+json

    {
      "meta" : {},
      "data" : {
        "cost-mode" : "numerical",
        "cost-type" : "routingcost",
        "map-vtag" : "1266506155",
        "map" : {
          "mypid1": { "mypid2": 0,  "peeringpid1": 0,  "transitpid2": 10 },
          "mypid3": { "mypid2": 0,  "peeringpid1": 0,  "transitpid2": 8 }
        }
      }
    }

18) Endpoint cost service (Section 7.7.5)
     TBD

19)  IPv6 test cases
      (TBD)

Miscellaneous notes

1) Section 4.2.1 says that "Each endpoint MUST map into exactly one
    PID."

    What happens if a client finds out that the above is not true?

    Update: This is being debated on the mailing list right now.

---- END ALTO INTEROPERABILITY DOCUMENT (Version 0.5)