Auto Program OpenAPI (Swagger) Spec on Enroute Universal API Gateway

Simplifying operations at DevOps speed

As organizations are moving towards a DevOps driven application delivery, the need for infrastructure to keep up at this pace of change is vital. enroutectl provides that flexibility and automation to quickly program Enroute Universal API Gateway for most common use-cases.

In this article, we describe how enroutectl radically simplifies programming the openapi spec on Enroute Universal Gateway

enroutectl is packaged in the enroute-gw docker image under the path /bin/enroutectl

Up and running with OpenAPI spec in 30 seconds

Here we demonstrate how easy it is to get up and running. This example demonstrates running the standalone gateway. Note that similar process can also be followed for kubernetes ingress gateway.

Let us start by running the standalone gateway in host mode. This makes the host network stack accessible to the running container. This does not change anything from functionality perspective. The standalone API gateway container can also be run in non-host mode with similar configuration.

sudo docker run --net=host saarasio/enroute-gw:v0.4.0

The above command starts the API Gateway. When run in host mode, the accessible ports are -

Interesting Ports

Envoy Ports
8002 - Envoy stats on 8002/stats
9001 - Envoy admin interface on 9001/admin
Enroute Interface Ports
1323 - REST interface to interact with standalone API gateway
8001 - xDS server listen port for Envoy
Enroute Process Ports
 
6060 - Debug info for enroute pprof tool
8000 - prometheus metrics for enroute
8888 - GraphQL server used by control plane to configure Enroute
5432 - postgres server when running in stateful mode
Ports used by Rate-Limit service
8003 - RateLimit server listen port 
6379 - Runs redis for rate-limit and other Enroute state. 
 
Enroute ports for traffic visibility
 
9090 - prometheus server
3000 - port to access grafana-server
Enroute ports for traffic
 
8080 - http traffic
8443 - https traffic

Note that when running in non-host mode, port redirection can be achieved as needed while exposing selected ports. For instance, when running https traffic, we can expose the following ports for operation -

 
sudo docker run -p 443:8443 -p 3000:3000 -p 80:8080 saarasio/enroute-gw:v0.4.0

The above command will run the gateway that will forward traffic ports and expose grafana port to look at the metrics. The rest of the ports are isolated in the container network stack. If direct access to Envoy admin interface is desired, the following command can be used -

 
sudo docker run -p 443:8443 -p 3000:3000 -p 80:8080 -p 9001:9001 saarasio/enroute-gw:v0.4.0

For the current example, we’ll run with –net=host mode and make the host network stack accessible to the API gateway container.

Accessing enroutectl

enroutectl is packaged along with docker container of standalone gateway and can be accessed by opening an interactive shell on the container. Here are the steps to do it.

 $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b8072f7b4b8c saarasio/enroute-gw:v0.4.0 "/usr/bin/supervisor…" 24 minutes ago Up 24 minutes silly_bohr
 
$ docker exec -it silly_bohr /bin/bash

 
silly_bohr:/bin#
# ls -l enroutectl
-rwxrwxr-x 1 root root 51513918 Jun 29 18:16 enroutectl

silly_bohr:/bin# ./enroutectl

enroutectl can be used to generate declarative configuration for

- Enroute Standalone Gateway
- Kubernetes Ingress Gateway

enroutectl can also read openapi spec to generate config for gateway

Usage:
enroutectl [flags]
enroutectl [command]

Available Commands:
help Help about any command
init Initialize enroutectl
openapi Automatically program Enroute Universal API Gateway using OpenAPI Spec

Flags:
--enroutectl-config string config for enroutectl if no enroutectl.yaml (or enroutectl.json) found in current OR home directory
-h, --help help for enroutectl
--verbose be verbose when running

Use "enroutectl [command] --help" for more information about a command.
The petstore example OpenAPI (swagger) spec is under the examples directory in this container -

silly_bohr:/examples# ls -l /examples
total 16
-rw-rw-r-- 1 postgres postgres 13843 Jun 29 18:16 petstore.json

To program the API gateway, simply run enroutectl with petstore.json as an argument

silly_bohr:/bin#./enroutectl openapi --openapi-spec /examples/petstore.json --to-standalone-url http://localhost:1323/

Note that enroutectl uses the REST interface on port 1323 to program the OpenAPI spec on the gateway. We can query the gateway state using dump verb. Note that we use the gateway name gw to dump state -

$curl -s localhost:1323/proxy/dump/gw | jq
{
    "data": {
        "saaras_db_proxy": [
            {
                "proxy_globalconfigs": [],
                "proxy_name": "gw",
                "proxy_services": [
                    {
                        "service": {
                            "fqdn": "*",
                            "routes": [
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "POST"
                                            }
                                        ],
                                        "prefix": "/v2/user/createWithArray$"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-createUsersWithArrayInput-3ae533b2b5-d0fee9541d",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "POST"
                                            }
                                        ],
                                        "prefix": "/v2/user/createWithList$"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-createUsersWithListInput-7011820c21-69bee49818",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "DELETE"
                                            }
                                        ],
                                        "prefix": "/v2/store/order/(?P<orderId>.*)$"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-deleteOrder-a0e2a93d66-6e736e9382",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "DELETE"
                                            }
                                        ],
                                        "prefix": "/v2/pet/(?P<petId>.*)$"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-deletePet-35222ce8e8-9fe883ae9e",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "DELETE"
                                            }
                                        ],
                                        "prefix": "/v2/user/(?P<username>.*)$"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-deleteUser-ea777d883f-d17f08eca4",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "GET"
                                            }
                                        ],
                                        "prefix": "/v2/pet/findByStatus"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-findPetsByStatus-090b3721b1-d1a5b91889",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "GET"
                                            }
                                        ],
                                        "prefix": "/v2/pet/findByTags"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-findPetsByTags-224f0e4d2d-b45f3e6a76",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "GET"
                                            }
                                        ],
                                        "prefix": "/v2/store/inventory"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-getInventory-85650e00a2-b0f5441374",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "GET"
                                            }
                                        ],
                                        "prefix": "/v2/store/order/(?P<orderId>.*)$"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-getOrderById-a0e2a93d66-3afe640f9d",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "GET"
                                            }
                                        ],
                                        "prefix": "/v2/pet/(?P<petId>.*)$"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-getPetById-35222ce8e8-2e85250528",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "GET"
                                            }
                                        ],
                                        "prefix": "/v2/user/(?P<username>.*)$"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-getUserByName-ea777d883f-1963d3289a",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "GET"
                                            }
                                        ],
                                        "prefix": "/v2/user/login"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-loginUser-c69fdf5bba-4aeb13de96",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "GET"
                                            }
                                        ],
                                        "prefix": "/v2/user/logout"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-logoutUser-6f8b3a245c-3713915009",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "POST"
                                            }
                                        ],
                                        "prefix": "/v2/store/order$"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-placeOrder-f930c1447d-13d63d6189",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "PUT"
                                            }
                                        ],
                                        "prefix": "/v2/pet$"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-updatePet-v2pet$-147d5289ee",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "POST"
                                            }
                                        ],
                                        "prefix": "/v2/pet/(?P<petId>.*)$"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-updatePetWithForm-35222ce8e8-26bb2fc896",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "PUT"
                                            }
                                        ],
                                        "prefix": "/v2/user/(?P<username>.*)$"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-updateUser-ea777d883f-97c79328fd",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "POST"
                                            }
                                        ],
                                        "prefix": "/v2/pet/(?P<petId>.*)/uploadImage$"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-uploadFile-67b0e00cbb-0b5b39c320",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "POST"
                                            }
                                        ],
                                        "prefix": "/v2/pet$"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-addPet-v2pet$-b021280030",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                },
                                {
                                    "config_json": {
                                        "header": [
                                            {
                                                "header_name": ":method",
                                                "header_value": "POST"
                                            }
                                        ],
                                        "prefix": "/v2/user$"
                                    },
                                    "route_filters": [],
                                    "route_name": "openapi-createUser-v2user$-8185d9fbf4",
                                    "route_prefix": "",
                                    "route_upstreams": [
                                        {
                                            "upstream": {
                                                "upstream_ip": "petstore.swagger.io",
                                                "upstream_name": "openapi-upstream-petstore.swagger.io",
                                                "upstream_port": 80
                                            }
                                        }
                                    ]
                                }
                            ],
                            "service_name": "openapi-petstore.swagger.io",
                            "service_secrets": [],
                            "service_filters": []
                        }
                    }
                ]
            }
        ]
    }
}

Attaching advanced rate-limiting to securing APIs.

Enroute uses a filter based architecture to attach additional functionality at the global level and route level. Note the route_filters: [] is an empty list above. Filters can be attached here. Eg: attaching a rate-limit filter here will apply rate-limits to this route. Similarly the service_filters:[] is an empty list above. Filters can be attached here for service level traffic. An example of such filter is the lua filter.

For a more detailed example on rate-limit, refer to the article on - getting started with advanced rate-limiting.

enroutectl also provides the ability to follow a similar workflow to program OpenAPI spec on kubernetes ingress gateway which also follows a similar filter architecture for configuration. Refer to the article on getting started with kubernetes ingress gateway for more details

Using Prometheus and Grafana to monitor Gateway state

Enroute Standalone comes packaged with Prometheus and Grafana to monitor the gateway. It provides information with request-level stats, latency and overall health of the system. The dashboard can also be tweaked as per the requirement of the user and notifications can be programmed to be notified on different conditions. As mentioned above, the dashboard can be accessed on port 3000 with default login password admin/admin

Grafana with swagger

Conclusion


enroutectl``` simplifies operations and helps automate configuring Enroute Universal Gateway. Both the kubernetes ingress gateway and standalone gateway can be configured.