OpenTracing on 3scale

Introduction

This document will walk you through how to setup Jaeger, enable OpenTracing on APIcast and Echo-api, then confirm the integration on OpenShift 4.x.

Prerequisites

Software requirements

• Red Hat OpenShift Container Platform 4.6
• Red Hat 3scale API Management 2.9

My tested environment

• Fedora 32
• Red Hat CodeReady Containers(CRC) 1.24.0
• OpenShift 4.7.0
• 3scale 2.9

Setup procedure

Install Red Hat OpenShift Jaeger operator

1. Open OperatorHub.

   

2. Search jaeger, then click Red Hat OpenShift Jaeger.

   

3. Click Install

   

4. Click Install again.

   

5. Wait until the installation complete.

   

Create Jaeger

1. Open Installed Operators page, make sure the project is your 3scale project. Then click Jaeger link.

   

2. Click Create Jaeger.

   

3. Click Create.

   

4. jaeger-all-in-one-in-memory will be created.

   

5. Search jaeger-all-in-one-in-memory route, then open the location.

   

6. Click Login with OpenShift.

   

7. Login to OpenShift as a user.

   

8. Click Allow selected permissions.

   

9. Now you will see Jaeger UI.

   

10. You can see 4 Jaeger services in the project.

    $ oc get svc|grep '\(jaeger\|NAME\)'
    NAME                                            TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                                  AGE
    jaeger-all-in-one-inmemory-agent                ClusterIP   None           <none>        5775/UDP,5778/TCP,6831/UDP,6832/UDP      5d
    jaeger-all-in-one-inmemory-collector            ClusterIP   10.217.5.218   <none>        9411/TCP,14250/TCP,14267/TCP,14268/TCP   5d
    jaeger-all-in-one-inmemory-collector-headless   ClusterIP   None           <none>        9411/TCP,14250/TCP,14267/TCP,14268/TCP   5d
    jaeger-all-in-one-inmemory-query                ClusterIP   10.217.4.66    <none>        443/TCP                                  5d

Configure APIcast

Reference: Configuring APIcast with OpenTracing

1. Configure jaeger_config.json locally.

   {
       "service_name": "apicast",
       "disabled": false,
       "sampler": {
         "type": "const",
         "param": 1
       },
       "reporter": {
         "queueSize": 100,
         "bufferFlushInterval": 10,
         "logSpans": false,
         "localAgentHostPort": "jaeger-all-in-one-inmemory-agent:6831"
       },
       "headers": {
         "jaegerDebugHeader": "debug-id",
         "jaegerBaggageHeader": "baggage",
         "TraceContextHeaderName": "uber-trace-id",
         "traceBaggageHeaderPrefix": "testctx-"
       },
       "baggage_restrictions": {
           "denyBaggageOnInitializationFailure": false,
           "hostPort": "127.0.0.1:5778",
           "refreshInterval": 60
       }
    }

2. Register jaeger_config.json as a ConfigMap

   $ oc create configmap jaeger-config --from-file=jaeger_config.json

3. Mount the file on apicast-production

   $ oc set volume dc/apicast-production --add --overwrite --name=jaeger-config --mount-path=/opt/app-root/src/conf.d/opentracing/jaeger_config.json --sub-path=jaeger_config.json --source='{"configMap":{"name":"jaeger-config","items":[{"key":"jaeger_config.json","path":"jaeger_config.json"}]}}'

4. Add OPENTRACING_TRACER and OPENTRACING_CONFIG environment variables in apicast-production deploymentconfig.

   $ oc set env dc/apicast OPENTRACING_TRACER=jaeger OPENTRACING_CONFIG=/opt/app-root/src/conf.d/opentracing/jaeger_config.json

5. After deployed apicast-production, call a production base url of a product

6. Open Jaeger UI, select apicast service then click Find Traces.

   

7. Traces are listed.

   

Setup echo-api

1. Install echo-api

   $ oc new-app -f https://raw.githubusercontent.com/3scale/echo-api/master/contrib/openshift/echo-api-template.yml --param ECHOAPI_HOST=echo-api.apps-crc.testing

2. Configure echo-api

   $ oc set env dc/echo-api OPENTRACING_TRACER=jaeger JAEGER_AGENT_HOST=jaeger-all-in-one-inmemory-agent JAEGER_AGENT_PORT="6831"

3. Open Jaeger UI, select echo-api service then click Find Traces. Traces are listed.

   

Configure a product

1. Configure the echo-api route as a backend

   

2. Configure the the backend to a product

   

3. Promote the configuration to the staging and production environment

Test the integration

1. Send a request to the production public base url

2. Confirm whether the response returned from the echo-api

   example:

   $ curl "https://api-3scale-apicast-production.apps-crc.testing:443/?user_key=b382b63a376dc1d41de338509066e2d3" -k

   {
     "method": "GET",
     "path": "/",
     "args": "user_key=b382b63a376dc1d41de338509066e2d3",
     "body": "",
     "headers": {
       "HTTP_VERSION": "HTTP/1.1",
       "HTTP_X_REAL_IP": "10.217.0.1",
       "HTTP_X_3SCALE_PROXY_SECRET_TOKEN": "Shared_secret_sent_from_proxy_to_API_backend_5817cd301754f6b2",
       "HTTP_USER_AGENT": "curl/7.69.1",
       "HTTP_ACCEPT": "*/*",
       "HTTP_X_FORWARDED_HOST": "api-3scale-apicast-production.apps-crc.testing, echo-api.apps-crc.testing",
       "HTTP_X_FORWARDED_PORT": "443, 80",
       "HTTP_X_FORWARDED_PROTO": "https, http",
       "HTTP_FORWARDED": "for=192.168.130.1;host=api-3scale-apicast-production.apps-crc.testing;proto=https, for=10.217.0.117;host=echo-api.apps-crc.testing;proto=http",
       "HTTP_X_FORWARDED_FOR": "192.168.130.1, 10.217.0.117",
       "HTTP_UBER_TRACE_ID": "ebd6331bd080b480:b7cca0910ca55d3d:ebd6331bd080b480:1",
       "HTTP_HOST": "echo-api.apps-crc.testing"
     },
     "uuid": "4029588a-77de-44b8-a80a-388e5506aa50"
   }

3. Open Jaeger UI, select apicast service then click Find Traces. Traces are listed.

   

4. Click a trace that contais apicast and echo-api. APIcast and Echo-api is included in one trace.