EnRoute Kubernetes Ingress Gateway

Published May 11, 2021
Last Modified October 19, 2021

EnRoute Universal Gateway

EnRoute Universal Gateway is a an API gateway built to support traditional and cloud-native use cases. It is designed to run either as a Kubernetes Ingress Gateway, Standalone Gateway, Horizontally scaling L7 API gateway or a Mesh of Gateways. Depending on the need of the user, the environment, the application, either one or many of these solutions can be deployed.

EnRoute also supports plugins/filters to extend functionality and enforce policies. The features page lists the available plugins for the Gateway. More details about each of the plugins can also be found on plugin pages.

A consistent policy framework across all these network components makes the EnRoute Universal Gateway a versatile and powerful solution.

What this article covers

This article covers how to get started with the EnRoute Kubernetes Ingress Gateway. The minimum requirement is a working Kubernetes cluster.

To get a more detailed understanding of EnRoute Universal Gateway and its architecture, refer to the article here To run EnRoute outside of kubernetes as a standalone gateway, refer to the article on standalone gateway

EnRoute also supports several other topologies including a Standalone Gateway. Only the Kubernetes Ingress topology is covered in this article.

Configure EnRoute using helm

A simple non-SSL example of EnRoute gateway with Lua and Rate-Limit filters can be programmed using the EnRoute helm chart.

Add the helm chart -

helm repo add saaras https://getenroute.io

Check repositories -

helm search repo

NAME                            CHART VERSION   APP VERSION     DESCRIPTION
jetstack/cert-manager           v1.5.3          v1.5.3          A Helm chart for cert-manager
jetstack/cert-manager-istio-csr v0.2.3          v0.2.1          A Helm chart for istio-csr
saaras/enroute                  0.4.1           0.8.0           EnRoute API Gateway
saaras/service-policy           0.3.1           0.8.0           Service L7 Policy using EnRoute API Gateway

Install the helm chart -

helm install enroute-demo saaras/enroute   \
    --set enrouteService.rbac.create=true

This installs the EnRoute Ingress API Gateway.

kubectl get svc -n enroutedemo

NAME      TYPE           CLUSTER-IP       EXTERNAL-IP                                                               PORT(S)                      AGE
enroute   LoadBalancer   10.100.227.212   a3523e69d10ea4da19d50af16f2dc272-1512456672.us-west-1.elb.amazonaws.com   80:31207/TCP,443:30109/TCP   18m

Questions about the helm chart? Slack Send Us a Note

However we still need to program the gateway to expose a service. Let us start by creating a demo namespace and a demo service.

kubectl create namespace demo-service

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: httpbin
  namespace: demo-service
  labels:
    app: httpbin
spec:
  containers:
  - name: httpbin
    image: kennethreitz/httpbin
    ports:
      - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: httpbin
  namespace: demo-service
spec:
  selector:
    app: httpbin
  ports:
  - port: 80
EOF

This creates a demo service in demo-service namepspace.

Now lets expose this service using a simple helm command

helm install httpbin-service-policy saaras/service-policy    \
     --set service.name=httpbin                              \
     --set service.namespace=demo-service                    \
     --set service.prefix=/get                               \
     --set service.port=80

The above command makes the service httpbin running on port 80 externally visible on path /get

Let’s check all the helm charts installed

helm ls --all-namespaces

NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
enroute-demo            default         1               2021-09-03 01:02:53.76747176 +0000 UTC  deployed        enroute-0.4.1           0.8.0
httpbin-service-policy  default         1               2021-09-03 01:03:50.33880541 +0000 UTC  deployed        service-policy-0.3.1    0.8.0

Use the External IP to send traffic

The EnRoute service can be reached using the External-IP and a request on path /get sends it to the httpbin service

curl -vvv a3523e69d10ea4da19d50af16f2dc272-1512456672.us-west-1.elb.amazonaws.com/get

*   Trying 13.56.87.172:80...
* TCP_NODELAY set
* Connected to a3523e69d10ea4da19d50af16f2dc272-1512456672.us-west-1.elb.amazonaws.com (13.56.87.172) port 80 (#0)
> GET /get HTTP/1.1
> Host: a3523e69d10ea4da19d50af16f2dc272-1512456672.us-west-1.elb.amazonaws.com
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< server: envoy
< date: Fri, 03 Sep 2021 01:22:20 GMT
< content-type: application/json
< content-length: 388
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 3
<
{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Host": "a3523e69d10ea4da19d50af16f2dc272-1512456672.us-west-1.elb.amazonaws.com",
    "User-Agent": "curl/7.68.0",
    "X-Envoy-Expected-Rq-Timeout-Ms": "15000",
    "X-Envoy-Internal": "true"
  },
  "origin": "192.168.63.101",
  "url": "http://a3523e69d10ea4da19d50af16f2dc272-1512456672.us-west-1.elb.amazonaws.com/get"
}

* Connection #0 to host a3523e69d10ea4da19d50af16f2dc272-1512456672.us-west-1.elb.amazonaws.com left intact

Trigger rate-limits by sending a burst of traffic

while true; do curl -I a3523e69d10ea4da19d50af16f2dc272-1512456672.us-west-1.elb.amazonaws.com/get; done;

HTTP/1.1 200 OK
server: envoy
date: Tue, 11 May 2021 16:55:31 GMT
content-type: application/json
content-length: 300
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 2
vary: Accept-Encoding

HTTP/1.1 200 OK
server: envoy
date: Tue, 11 May 2021 16:55:31 GMT
content-type: application/json
content-length: 300
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 2
vary: Accept-Encoding

HTTP/1.1 200 OK
server: envoy
date: Tue, 11 May 2021 16:55:31 GMT
content-type: application/json
content-length: 300
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 23
vary: Accept-Encoding

HTTP/1.1 200 OK
server: envoy
date: Tue, 11 May 2021 16:55:31 GMT
content-type: application/json
content-length: 300
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 2
vary: Accept-Encoding

HTTP/1.1 200 OK
server: envoy
date: Tue, 11 May 2021 16:55:31 GMT
content-type: application/json
content-length: 300
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 2
vary: Accept-Encoding

HTTP/1.1 429 Too Many Requests
x-envoy-ratelimited: true
date: Tue, 11 May 2021 16:55:30 GMT
server: envoy
transfer-encoding: chunked

HTTP/1.1 429 Too Many Requests
x-envoy-ratelimited: true
date: Tue, 11 May 2021 16:55:30 GMT
server: envoy
transfer-encoding: chunked

HTTP/1.1 429 Too Many Requests
x-envoy-ratelimited: true
date: Tue, 11 May 2021 16:55:30 GMT
server: envoy
transfer-encoding: chunked

HTTP/1.1 429 Too Many Requests
x-envoy-ratelimited: true
date: Tue, 11 May 2021 16:55:31 GMT
server: envoy
transfer-encoding: chunked

HTTP/1.1 429 Too Many Requests
x-envoy-ratelimited: true
date: Tue, 11 May 2021 16:55:31 GMT
server: envoy
transfer-encoding: chunked

HTTP/1.1 429 Too Many Requests
x-envoy-ratelimited: true
date: Tue, 11 May 2021 16:55:31 GMT
server: envoy
transfer-encoding: chunked

Questions about rate-limit? Slack Send Us a Note

Understanding EnRoute configuration setup in previous step

`GatewayHost`

GatewayHost is a way to expose your services running inside Kubernetes.

kubectl describe  gatewayhosts.enroute.saaras.io  --all-namespaces

Name:         httpbin-80-gatewayhost
Namespace:    demo-service
Labels:       app=httpbin
              app.kubernetes.io/managed-by=Helm
Annotations:  meta.helm.sh/release-name: httpbin-service-policy
              meta.helm.sh/release-namespace: default
API Version:  enroute.saaras.io/v1
Kind:         GatewayHost
Metadata:
...
Spec:
  Routes:
    Conditions:
      Prefix:  /get
    Filters:
      Name:  httpbin-80-rl2
      Type:  route_filter_ratelimit
    Services:
      Health Check:
        Healthy Threshold Count:    3
        Host:                       hc
        Interval Seconds:           5
        Path:                       /
        Timeout Seconds:            3
        Unhealthy Threshold Count:  3
      Name:                         httpbin
      Port:                         80
  Virtualhost:
    Filters:
      Name:  httpbin-80-luatestfilter
      Type:  http_filter_lua
    Fqdn:    *
Events:      <none>

Filters attached to GatewayHost can be used to control traffic flowing to the exposed service

Questions about GatewayHost? Slack Send Us a Note

`Filter`

Filters provide fine-grained control to control access to service. There are several features supported by EnRoute.

Lua Filter

This config invokes a function for an incoming request on request and response path

kubectl describe -n demo-service httpfilters.enroute.saaras.io httpbin-80-luatestfilter

Name:         httpbin-80-luatestfilter
Namespace:    demo-service
Labels:       app=httpbin-app
              app.kubernetes.io/managed-by=Helm
Annotations:  meta.helm.sh/release-name: httpbin-service-policy
              meta.helm.sh/release-namespace: default
API Version:  enroute.saaras.io/v1
Kind:         HttpFilter
Metadata:
...
Spec:
  Http Filter Config:
    Config:  function envoy_on_request(request_handle)
   request_handle:logInfo("Hello World request");
end

function envoy_on_response(response_handle)
   response_handle:logInfo("Hello World response");
end

  Name:  httpbin-80-luatestfilter
  Type:  http_filter_lua
Events:  <none>

Questions about lua filter? Slack Send Us a Note

Rate Limit config for Filter

The following config limits every http request from a unique IP. The x-forwarded-for provides the remote-ip and x-forwarded-proto is set to http for requests. A combination of IP address (from x-forwarded-for) and protocol (from x-forwarded-proto) is rate-limited to 5 requests per second.

kubectl describe -n demo-service globalconfigs.enroute.saaras.io httpbin-rl-global-config

Name:         httpbin-rl-global-config
Namespace:    demo-service
Labels:       app=httpbin-app
              app.kubernetes.io/managed-by=Helm
Annotations:  meta.helm.sh/release-name: httpbin-service-policy
              meta.helm.sh/release-namespace: default
API Version:  enroute.saaras.io/v1beta1
Kind:         GlobalConfig
Metadata:
...
Spec:
  Config:  {
  "domain": "enroute",
  "descriptors" :
  [
    {
      "key": "x-forwarded-for",
      "descriptors" :
      [
        {
          "key" : "x-forwarded-proto",
          "value" : "http",
          "descriptors" : [
           {
             "key" : "generic_key",
             "value" : "default_route",
             "rate_limit" : { "unit" : "second", "requests_per_unit" : 5 }
            }
          ]
        },
        {
          "key" : "x-forwarded-proto",
          "value" : "https",
          "descriptors" : [
           {
             "key" : "generic_key",
             "value" : "default_route",
             "rate_limit" : { "unit" : "second", "requests_per_unit" : 2 }
           }
          ]
        }
      ]
    }
  ]
}

  Name:  httpbin-rl-global-config
  Type:  globalconfig_ratelimit
Events:  <none>

Questions about filters? Slack Send Us a Note

Adding SSL

The above config does not add SSL config. For SSL to work, we need a certificate installed with the appropriate domain name.

Let’s install a SSL certificate for both the httpbin and echo service. We will first install Jetstack cert manager. Then invoke EnRoute helm chart to issue and install certificates.

Install Jetstack Cert Manager

helm repo add jetstack https://charts.jetstack.io
helm repo update
helm search repo

We can now see the helm repo for Jetstack cert manager

NAME                            CHART VERSION   APP VERSION     DESCRIPTION
jetstack/cert-manager           v1.5.3          v1.5.3          A Helm chart for cert-manager
jetstack/cert-manager-istio-csr v0.2.3          v0.2.1          A Helm chart for istio-csr
saaras/enroute                  0.4.1           0.8.0           EnRoute API Gateway
saaras/service-policy           0.3.1           0.8.0           Service L7 Policy using EnRoute API Gateway

Let’s go ahead an install the cert manager

helm install cert-manager jetstack/cert-manager \
    --namespace cert-manager                    \
    --version v1.2.0                            \
    --create-namespace                          \
    --set installCRDs=true

NAME: cert-manager
LAST DEPLOYED: Wed May 12 18:13:26 2021
NAMESPACE: cert-manager
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
cert-manager has been deployed successfully!

In order to begin issuing certificates, you will need to set up a ClusterIssuer
or Issuer resource (for example, by creating a 'letsencrypt-staging' issuer).

More information on the different types of issuers and how to configure them
can be found in our documentation:

https://cert-manager.io/docs/configuration/

For information on how to configure cert-manager to automatically provision
Certificates for Ingress resources, take a look at the `ingress-shim`
documentation:

https://cert-manager.io/docs/usage/ingress/

The cert manager chart got installed, lets list all the charts we have

helm ls --all-namespaces

Let’s verify the cert manager chart installation

NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
cert-manager            cert-manager    1               2021-09-03 01:18:07.524580777 +0000 UTC deployed        cert-manager-v1.2.0     v1.2.0
enroute-demo            default         1               2021-09-03 01:02:53.76747176 +0000 UTC  deployed        enroute-0.4.1           0.8.0
httpbin-service-policy  default         2               2021-09-03 01:16:25.003355577 +0000 UTC deployed        service-policy-0.3.1    0.8.0

We can also check the artifacts installed for with the cert manager installation

kubectl get all -n cert-manager

NAME                                           READY   STATUS    RESTARTS   AGE
pod/cert-manager-85f9bbcd97-6z2sl              1/1     Running   0          29s
pod/cert-manager-cainjector-74459fcc56-lp9kl   1/1     Running   0          29s
pod/cert-manager-webhook-57d97ccc67-pjksz      1/1     Running   0          29s

NAME                           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/cert-manager           ClusterIP   10.245.101.63   <none>        9402/TCP   29s
service/cert-manager-webhook   ClusterIP   10.245.10.42    <none>        443/TCP    29s

NAME                                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/cert-manager              1/1     1            1           29s
deployment.apps/cert-manager-cainjector   1/1     1            1           29s
deployment.apps/cert-manager-webhook      1/1     1            1           29s

NAME                                                 DESIRED   CURRENT   READY   AGE
replicaset.apps/cert-manager-85f9bbcd97              1         1         1       29s
replicaset.apps/cert-manager-cainjector-74459fcc56   1         1         1       29s
replicaset.apps/cert-manager-webhook-57d97ccc67      1         1         1       29s

We will use the HTTP01 challenge to issue the cert. For that, we will first need to create a DNS record to map httpbin21.enroutedemo.com to the external public domain name a3523e69d10ea4da19d50af16f2dc272-1512456672.us-west-1.elb.amazonaws.com

Once you create the DNS record, verify that the DNS name maps to the correct IP

ping httpbin21.enroutedemo.com

ping httpbin21.enroutedemo.com
PING a3523e69d10ea4da19d50af16f2dc272-1512456672.us-west-1.elb.amazonaws.com (13.56.87.172) 56(84) bytes of data.
^C
--- a3523e69d10ea4da19d50af16f2dc272-1512456672.us-west-1.elb.amazonaws.com ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

Now we are ready to install a SSL certificate for this service

Adding SSL to `httpbin` service

helm upgrade httpbin-service-policy saaras/service-policy   \
      --set service.namespace=demo-service                  \
      --set service.name=httpbin                            \
      --set service.prefix=/get                             \
      --set service.port=80                                 \
      --set service.enableTLS=true                          \
      --set service.createGlobalConfig=false                \
      --set autoTLS.certificateCN=httpbin21.enroutedemo.com \
      --set autoTLS.createIssuers=true                      \
      --set autoTLS.email=myemail@mydomain.com              \
      --set autoTLS.enableProd=true

Note: In the step above, the actual email has been redacted.

Release "httpbin-service-policy" has been upgraded. Happy Helming!
NAME: httpbin-service-policy
LAST DEPLOYED: Fri Sep  3 01:33:33 2021
NAMESPACE: default
STATUS: deployed
REVISION: 3

The above command enables TLS for the service by using the fqdn of httpbin21.enroutedemo.com

It also issues the certificate using the CN httpbin21.enroutedemo.com

The flag autoTLS.createIssuers=true creates Issuers, which are needed to work with Jetstack Cert manager. Note that this is only required the very first time. This flag is not required when issuing certificates again in the same cluster.

The autoTLS.email=myemail@mydomain.com needs to be set to a valid email address that can be used by the Cert manager to request a certificate. Note that the current value of myemail@mydomain.com won’t work

Let’s check the status of the certificate.

kubectl get certificates.cert-manager.io --all-namespaces

NAMESPACE      NAME                        READY   SECRET                      AGE
demo-service   httpbin21.enroutedemo.com   True    httpbin21.enroutedemo.com   117s

The True status above tells us that certificate issue was successful. More details are also captured in troubleshooting and more information about EnRoute with Jetstack Cert manager

Let’s send some traffic to verify the certificate got installed

curl -vvv https://httpbin21.enroutedemo.com/get

*   Trying 13.56.87.172:443...
* TCP_NODELAY set
* Connected to httpbin21.enroutedemo.com (13.56.87.172) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=httpbin21.enroutedemo.com
*  start date: Sep  3 00:33:58 2021 GMT
*  expire date: Dec  2 00:33:57 2021 GMT
*  subjectAltName: host "httpbin21.enroutedemo.com" matched cert's "httpbin21.enroutedemo.com"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x562b4fd7c550)
> GET /get HTTP/2
> Host: httpbin21.enroutedemo.com
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 2147483647)!
< HTTP/2 200
< server: envoy
< date: Fri, 03 Sep 2021 01:35:59 GMT
< content-type: application/json
< content-length: 297
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 3
<
{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Host": "httpbin21.enroutedemo.com",
    "User-Agent": "curl/7.68.0",
    "X-Envoy-Expected-Rq-Timeout-Ms": "15000",
    "X-Envoy-Internal": "true"
  },
  "origin": "192.168.63.101",
  "url": "https://httpbin21.enroutedemo.com/get"
}
* Connection #0 to host httpbin21.enroutedemo.com left intact

Let’s check the helm charts installed.

helm ls

NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
enroute-demo            default         1               2021-09-03 01:02:53.76747176 +0000 UTC  deployed        enroute-0.4.1           0.8.0
httpbin-service-policy  default         3               2021-09-03 01:33:33.070360281 +0000 UTC deployed        service-policy-0.3.0    0.8.0

Summary of artifacts created

We graphically show the summary of steps, the helm commands and the corresponding artifacts created

A helm chart instance per-service

EnRoute helm chart helps manage policy per-service by installing a version of the chart for each service.

The previous example showed how a simple service can receive external traffic. Note that the previous service GatewayHost installs a fqdn of * which matches traffic destined to all hosts. However, real-world deployments often route on the basis of fqdn first and then the route.

In this example, we demonstrate how traffic destined to a specific service is first routed on the basis of fqdn and then the route for the service.

We start by installing the another example workload

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: echo-server
  template:
    metadata:
      labels:
        app: echo-server
    spec:
      containers:
        - name: echo-server
          image: jmalloc/echo-server
          ports:
            - name: http-port
              containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: echo-service
spec:
  ports:
    - name: http-port
      port: 80
      targetPort: http-port
      protocol: TCP
  selector:
    app: echo-server
EOF

This creates an echo service in default namespace.

Now lets expose this service using a simple helm command

helm install echo-service-policy saaras/service-policy \
    --set service.name=echo-service                    \
    --set service.namespace=default                    \
    --set service.prefix=/get                          \
    --set service.enableTLS=false                      \
    --set service.port=80                              \
    --set service.fqdn=echo3.enroutedemo.com

Note that this time, we use a couple of other flags. service.enableTLS=false disables https on this service. The service.fqdn=echo3.enroutedemo.com flag sets the fqdn. Only traffic destined to echo3.enroutedemo.com will be directed to this service

Let’s send some traffic to test this (Note that we did create a DNS CNAME record for echo3.enroutedemo.com to a3523e69d10ea4da19d50af16f2dc272-1512456672.us-west-1.elb.amazonaws.com before sending traffic).

curl echo3.enroutedemo.com/get

Request served by echo-deployment-869b7bf9c7-bbx6h

HTTP/1.1 GET /get

Host: echo3.enroutedemo.com
Accept: */*
X-Request-Id: 2e2fb8f2-b9b0-467c-a37a-3917791b6ee8
X-Request-Start: t=1630640611.106
X-Envoy-Internal: true
X-Envoy-Expected-Rq-Timeout-Ms: 15000
User-Agent: curl/7.68.0
X-Forwarded-For: 192.168.63.101
X-Forwarded-Proto: http

Adding SSL to `echo` service

Installing a SSL certificate for the echo service is similar to the previous step, except that this time we don’t need to create issuers. We leave out the autoTLS.createIssuers flag this time. The steps below demonstrate that

First create a DNS CNAME record from echo3.enroutedemo.com to a3523e69d10ea4da19d50af16f2dc272-1512456672.us-west-1.elb.amazonaws.com. Then use the helm command below to issue a certificate and install it for the service

helm upgrade echo-service-policy saaras/service-policy \
    --set service.namespace=default                    \
    --set service.name=echo-service                    \
    --set service.prefix=/get                          \
    --set service.port=80                              \
    --set service.enableTLS=true                       \
    --set autoTLS.certificateCN=echo3.enroutedemo.com  \
    --set autoTLS.email=myemail@mydomain.com           \
    --set autoTLS.enableProd=true

Note that we don’t use the createIssuers=true flag above since it is only needed the very first time.

Release "echo-service-policy" has been upgraded. Happy Helming!
NAME: echo-service-policy
LAST DEPLOYED: Fri Sep  3 03:49:13 2021
NAMESPACE: default
STATUS: deployed
REVISION: 2

Let’s check the status of the new certificate we requested

kubectl get certificates.cert-manager.io --all-namespaces

NAMESPACE      NAME                        READY   SECRET                      AGE
default        echo3.enroutedemo.com       True    echo3.enroutedemo.com       59s
demo-service   httpbin21.enroutedemo.com   True    httpbin21.enroutedemo.com   136m

Now that the certificate is ready, let’s send some traffic to test it

curl -vvv https://echo3.enroutedemo.com/get

*   Trying 13.56.87.172:443...
* TCP_NODELAY set
* Connected to echo3.enroutedemo.com (13.56.87.172) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=echo3.enroutedemo.com
*  start date: Sep  3 02:49:39 2021 GMT
*  expire date: Dec  2 02:49:38 2021 GMT
*  subjectAltName: host "echo3.enroutedemo.com" matched cert's "echo3.enroutedemo.com"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55e029392550)
> GET /get HTTP/2
> Host: echo3.enroutedemo.com
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 2147483647)!
< HTTP/2 200
< content-type: text/plain
< date: Fri, 03 Sep 2021 03:50:38 GMT
< content-length: 338
< x-envoy-upstream-service-time: 2
< server: envoy
<
Request served by echo-deployment-869b7bf9c7-bbx6h

HTTP/1.1 GET /get

Host: echo3.enroutedemo.com
X-Forwarded-For: 192.168.63.101
X-Forwarded-Proto: https
X-Request-Id: 9eaa44d1-ceca-4ee4-9660-9a2d2be7069e
X-Envoy-Expected-Rq-Timeout-Ms: 15000
User-Agent: curl/7.68.0
Accept: */*
X-Envoy-Internal: true
X-Request-Start: t=1630641038.275
* Connection #0 to host echo3.enroutedemo.com left intact

Creation of SSL certificates is automated using ACME protocol and a Certificate Authority like Let’s Encrypt. EnRoute integrates with Jacks Cert Manager to achieve this. The EnRoute helm chart also provides switches to achieve this.

Questions about SSL/TLS? Slack Send Us a Note

Helm switches to enable/disable features

EnRoute helm chart provides configuration options for multiple clouds and switches to enable/disable plugins or filters. These switches provide a fine-grained control over L7 policy.

The EnRoute Helm chart supports the following filter switches -

Switch	Valid Values	Description
envoySettings.logLevel	One of - ["trace", "debug", "info", "error"]	Log level for underlying Envoy
EnRoute Ingress Controller Service Settings
enrouteService.enable	One of - ["true", "false"]	Disabled by default. Enable when installing EnRoute the first time. Enables creation of EnRoute deployment and service
enrouteService.replicaCount	Number of replicas eg: 3	By default, one instance of EnRoute pod is created Change this value to increase/decrease count of replicas
Cloud Settings
awsSettings.enable	One of - ["true", "false"]	Disabled by default. Enable when running on AWS. Enables propagation of client-IP to backend service
ociSettings.enable	One of - ["true", "false"]	Disabled by default. Enable when running on OCI. Enables propagation of client-IP to backend service
digitaloceanSettings.enable	One of - ["true", "false"]	Disabled by default. Enable when running on DigitalOcean. Enables propagation of client-IP to backend service
Filter Settings
filters.lua.enable	One of - ["true", "false"]	Enabled by default. Enable when you want to install a Lua filter for service.
filters.ratelimit.enable	One of - ["true", "false"]	Enabled by default. Enable when you want to install a RateLimit filter for service.
filters.cors.enable	One of - ["true", "false"]	Disabled by default. Enable when you want to install a CORS filter for service.
filters.jwt.enable	One of - ["true", "false"]	Disabled by default. Enable when you want to install a JWT filter for service.
Service Settings
service.name	A string representing service name in Kubernetes	Default is "hello-enroute" service. Set it to your own service name
service.port	Port on which to access this service	Default is 9090, the port for "hello-enroute" service. Set it to your own service port
service.enableTLS	When securing the service using TLS, use this flag after cert-manager has been setup.	Default is false. Enable this flag to automatically get a certificate for a service from Let's encrypt
service.namespace	The namespace in which the service is running	This setting is used to determine the namespace in which to create EnRoute Filters and ```GatewayHost```
service.prefix	The prefix on which the service is reachable	This installs the prefix route for this service
Auto-TLS Settings
autoTLS.issueCert	One of - ["true", "false"]	Enable this to issue a certificate signed using the ACME protocol (from Let's Encrypt)
autoTLS.certificateCN	Common Name for Certificate	Common Name to use for Certificate. This also gets installed as ```fqdn``` for the service
autoTLS.enableProd	One of - ["true", "false"]	Default is false. When set to false, it uses the ACME staging server `https://acme-staging-v02.api.letsencrypt.org/directory` . When set to true, it uses the ACME production server `https://acme-v02.api.letsencrypt.org/directory`

As filters/plugins are added to EnRoute, simple switches will allow enabling/disabling these features.

Questions about the helm chart? Slack Send Us a Note

Enforcing L7 Policies

EnRoute uses filter/plugin to add functionality to Ingress for a service. These Filters/Plugins loosely map to Envoy filters/plugins and can be enabled/disabled using the helm chart. The complete list of plugins supported in the helm chart and examples are covered in the Enforce L7 Policy blog

EnRoute Config Model across Kubernetes Ingress Gateway and Standalone Gateway

EnRoute can be used to protect services outside kubernetes using the standalone gateway or services running inside kubernetes.

EnRoute follows a configuration model similar to Envoy and is extensible using Filters. It uses filters to extend functionality at the global Service level and per-route level. The config objects used for this are - GatewayHost, Route, HttpFilter, RouteFilter and Service as shown here for Kubernetes gateway.

Regardless of where the workload runs, a consistent service policy can be defined once and applied to secure any service running inside or without Kubernetes using Envoy.

EnRoute provides key functionality using modular filters which make it easy to secure any service.

Need Questions Answered? Slack Send Us a Note

Next Steps

Working with EnRoute Ingress API Gateway is organized in following steps

Step-1 Getting Started : Enforcing L7 Policy for a micro-service API using helm chart
Step-2 Getting Started : Understanding Artificats of EnRoute Ingress Controller
Step-3 Understanding the How and Why of Kubernetes Ingress and Networking

kubernetes-ingress

EnRoute Kubernetes Ingress Gateway