You can debug problems, such as performance issues or site failures, in your ABL HTTP client program's communication with a web server, by implementing HTTP tracing and analyzing the resultant tracing data.

ABL HTTP client tracing enables you to log key aspects of an HTTP session, such as the URLs invoked, request and response data, request and response times, and so on, in a configurable way.

ABL HTTP tracing provides a better mechanism to debug HTTP problems than using the LOG-MANAGER or setting SESSION:DEBUG-ALERT to true, because it produces a single output for the entire request-response exchange and the output data is not overwritten on redirects or on subsequent requests. Further, an OpenEdge administrator can enable or disable tracing for certain requests.

To set up HTTP tracing in OpenEdge, an OpenEdge ABL developer must first allow tracing in the ABL HTTP client. An OpenEdge administrator can then use a JSON configuration file in a development, test, or production environment to define rules that determine which request-response exchanges are traced.

Note that tracing must be allowed or enabled in the HTTP client as well as in the JSON configuration for OpenEdge to collect tracing information.

How tracing works

  1. When a request is sent, an AllowTracing() method in the ABL HTTP client instance determines whether tracing is allowed on that client. If tracing is not allowed, no further action is taken.
  2. If tracing is allowed in the ABL HTTP client, a tracing configuration is loaded. If tracing is enabled in the tracing configuration, a series of rules are applied to the data.
  3. Filtering rules determine whether tracing data must be collected for the request-response exchange. For example, a filtering rule may be based on whether the request URL matches a defined string pattern.
  4. If filtering rules allow tracing data to be collected, request data rules are applied to the request message. Request data rules determine which parts of the message, such as body, headers, and cookies, are included in the tracing data.
  5. Similarly, when a response is received, response data rules are applied on the response message.
  6. The collected tracing data is formatted by a formatter class specified in the tracing configuration.
  7. Finally, the tracing data is sent to an output based on a writer class in the tracing configuration.

Tracing data in HAR format

The OpenEdge.Net library has built-in support for logging the HTTP tracing information in an HTTP Archive (HAR) format. The HAR format is a standard format for HTTP tracing data. Many browsers, including Chrome and Firefox, enable you to download tracing data in a HAR file.

A HAR file is a JSON-formatted file. It provides you a complete view of a request-response exchange between your client and a web server. Moreover, you can take advantage of a number of third-party tools that provide visualization based on the HAR format.

If generating a HAR file does not meet your requirements, you can customize the tracing behavior by developing your own output formatter. See Customize ABL HTTP tracing to learn more.

Overview of steps to implement HTTP tracing

Perform the following steps to implement HTTP tracing:

  1. Allow tracing in the ABL HTTP client instance.
  2. Create a tracing configuration file or object.
  3. Use the tracing configuration as follows:
    • Enable tracing.
    • Specify an output formatter and writer.
    • Define the following rules:
      • Filter rules
      • Request and response data rules

Allow tracing in an ABL HTTP client

To allow tracing in an ABL HTTP client, set the AllowTracing(TRUE) method in the HTTP client object as shown in this example.

var IHttpClient oClient.
...
oClient = ClientBuilder:Build():AllowTracing(TRUE):Client.

How to set up a tracing configuration

If tracing is allowed in the ABL HTTP client, the OpenEdge.Net library uses a tracing configuration to determine which requests to trace. You can set up the tracing configuration in one of the following ways:

  • Create a JSON tracing configuration file. This approach allows OpenEdge administrators to modify the file in a test or production environment when they need to debug ABL HTTP-related problems. See Create a tracing configuration file to learn more.
  • Specify a tracing configuration in the ABL program when building the ABL HTTP client instance. This approach enables you to specify separate tracing configurations for HTTP client instances, use conditions to load configurations from database tables, and so on. See Specify a tracing configuration in the ABL HTTP client to learn more.

Create a tracing configuration file

Create a JSON file named hctracing.config in a directory that is included in the PROPATH.

Note:
  • Only the first file with the name hctracing.config in the PROPATH is used.
  • While the format of the tracing configuration file is JSON, it should use the file extension .config, not .json.

The tracing configuration file is checked for modifications each time an ABL HTTP client's Execute() method is called. If the configuration has changed since the last check, the updated configuration is used.

See JSON schema for tracing configuration to read about the tracing configuration schema.

See Sample tracing configuration for a complete example of a tracing configuration.

Enable tracing in a tracing configuration

Define a JSON field named enabled in the tracing configuration and set it to true.

Note: Setting enabled to false disables tracing, even when tracing is allowed in the ABL HTTP client program.
{
    "enabled": true
}

Define an output formatter and writer in the tracing configuration

Define an output object containing the following fields:

  • type: Specify one of the following values:
    • local: Formats and writes the tracing information locally. If you choose local you should specify a formatter and a writer.
    • proxy: Delegates tracing to a proxy server. See Delegate to a proxy server for more information.
    • none: Collects the tracing information in memory but does not take any further action; may be used for testing or other advanced behavior.
  • formatter: Specify a formatter if you set the type to local. To define a formatter, create a formatter object. Inside the formatter object, create an implementation object and specify the fully qualified name of an ABL class that formats the tracing information. To format the tracing information in HAR format, specify OpenEdge.Net.HTTP.Trace.Formatter.HarFormatter.
    Note: If type is set to local and no formatter is specified, a default JSON formatter will be used to format the tracing data.
  • writer: Specify a writer if you set the type to local. To define a writer, create a writer object. Inside the writer object, create an implementation object and specify the fully qualified name of an ABL class that processes the formatted tracing information in some way, such as writing it to a file. Specify OpenEdge.Net.HTTP.Trace.Writer.FileWriter to write the tracing information to a file.
    Note: If type is set to local and no writer is specified, a default file writer will be used to write the tracing data to Session-Temp-Directory/TraceOutput.
You can pass parameters to formatter or writer classes using an options object. To specify a filename for a file writer, include a fileName property in the file writer's options object.
{
    "enabled": true,
    "output": {
        "type": "local",
        "formatter": {
            "implementation": "OpenEdge.Net.HTTP.Trace.Formatter.HarFormatter"
        },
        "writer": {
            "implementation": "OpenEdge.Net.HTTP.Trace.Writer.FileWriter",
            "options": {
                "fileName": "HttpClient.har"
            }
        }
    }
}
Note: You can include tokens in the filename, as shown in this example. 
"options": {
    "fileName": "${session.temp-dir}/TraceOutput/HttpClient_${t.now}.har"
}

For a list of allowed tokens, see Logging tokens.

Define tracing rules

ABL HTTP tracing gives you a lot of flexibility in choosing which request-response exchanges to trace and what data to collect. You do this by defining rules in the tracing configuration.

Defining rules is the recommended approach to tracing. If no rules are defined, all tracing data is logged for all HTTP requests (provided tracing is enabled in the HTTP client and the tracing configuration), which may result in confidential data also getting logged.

To define rules, create a rules object. Inside the rules object, you can:
  • Define filters that determine which request-response exchanges get traced
  • Define rules for collecting request data
  • Define rules for collecting response data
{
    "enabled": true,
    "output": {
        ...
    },
    "rules": {
        "filter": {
            ...
        },
        "request": {
            ...
        },
        "response": {
            ...
        }
    }
}

For each rule, you must set a policy—allow-all or deny-all—to define whether to allow or deny tracing.

For filter rules, this means allowing or denying tracing on a message transaction as a whole. For example, if you set deny-all in a filter rule, request and response data rules are not evaluated and no tracing data is collected for any part of the request-response exchange.

For request and response data rules, the policy dictates whether to collect tracing data from the part of the request or response message that the rule applies to. For example, you can set allow-all in a HeaderRule in the request object to add request headers to the tracing data.

For each rule, you may optionally define an array of exceptions.

Take the following snippet as an example. The UrlRule filter has a deny-all policy but an exception is set for requests to https://example.com. This means that tracing data will be collected only on requests to http://example.com.

"filter": {
            "OpenEdge.Net.HTTP.Trace.Rule.UrlRule": {
                "policy": "deny-all",
                "exceptions": [
                    "http://example.com"
                ]
            }
        }

Filter rules

Use filter rules to specify which message transactions get traced.

To specify filter rules, create a filter object. Inside the filter object, create an object for each filter rule, named after the ABL class that contains the rule logic.


    "rules": {
        "filter": {
            "<Filter.Rule.Class.Name>": {
                "policy": "<allow-all | deny-all>",
                "exceptions": [
                    ...
                ]
            }
        }
    }

The OpenEdge.Net library provides the following built-in classes for filtering:

  • OpenEdge.Net.HTTP.Trace.Rule.UrlRule
  • OpenEdge.Net.HTTP.Trace.Rule.QueryStringFilterRule
  • OpenEdge.Net.HTTP.Trace.Rule.BodyFilterRule
  • OpenEdge.Net.HTTP.Trace.Rule.StatusCodeRule

OpenEdge.Net.HTTP.Trace.Rule.UrlRule

Use this class name to define a filter based on the URL pattern.

In the exceptions array, specify absolute URLs such as http://www.example.com or URL patterns that include asterisks or periods as wildcards, for example, http://*.example.com. To escape wildcards, use the tilde (~) symbol.
Note: When HTTP requests are executed, the request URLs are compared with the specified URL pattern using the ABL MATCHES operator. If the MATCHES operation returns true, the exception is applied.

For each URL or URL pattern in the exceptions array, you can also define an HTTP method (GET, PUT, POST, DELETE, and so on) that the exception applies to.

In the following example, tracing is denied on all message transactions except requests to https://example.com and POST requests to http://httpbin.org/post:


    "rules": {
        "filter": {
            "OpenEdge.Net.HTTP.Trace.Rule.UrlRule": {
                "policy": "deny-all",
                "exceptions": [
                    {
                        "url": "http://httpbin.org/post",
                        "method": "POST"
                    },
                    "http://example.com"
                ]
            }
        }
    }

OpenEdge.Net.HTTP.Trace.Rule.QueryStringFilterRule

Use this class name to define a filter based on query strings in the URL.

In the exceptions array, specify query parameter names such as filter, CustNum, or apiKey.

In the following example, tracing is allowed on all message transactions except for those originating with a request URL containing the parameter names apiKey or CustNum. 


    "rules": {
        "filter": {
            "OpenEdge.Net.HTTP.Trace.Rule.QueryStringFilterRule": {
                "policy": "allow-all",
                "exceptions": [
                    "apiKey", "CustNum"
                ]
            }
        }
    }

When this configuration is loaded, tracing will be allowed on message transactions that use this URL— http://example.com?CustName=Lift%20Tours—but denied for message transactions that use this URL—http://example.com?CustNum=1.

OpenEdge.Net.HTTP.Trace.Rule.BodyFilterRule

Use this class name to define a filter based on the content type of the request or response body.

In the exceptions array, specify content types that you want to exclude from the policy, such as application/json or text/plain.

In the following example, tracing is allowed on all message transactions except those in which the Content-Type of the request OR response body is text/plain.


    "rules": {
        "filter": {
            "OpenEdge.Net.HTTP.Trace.Rule.BodyFilterRule": {
                "policy": "allow-all",
                "exceptions": [
                    "text/plain"
                ]
            }
        }
    }

OpenEdge.Net.HTTP.Trace.Rule.StatusCodeRule

Use this class name to define a filter based on the status code in the response message. For example, assuming you want to collect tracing data only when the response status code is 403 or 404, you would use this rule as shown in the following example:


    "rules": {
        "filter": {
            "OpenEdge.Net.HTTP.Trace.Rule.StatusCodeRule": {
                "policy": "deny-all",
                "exceptions": [
                    403, 404
                ]
            }
        }
    }

Request and response data rules

Request and response rules determine which parts of the request or response are included in the trace. You can configure separate rules to gather tracing data from the message body, message headers, cookies, and query strings.

Create a request object and a response object at the same hierarchical level as the filter object. Define rules in the request and response objects as follows:

  • Message body: Inside the request or response object, specify the fully qualified name of the ABL class that processes rules for the message body. The built-in ABL class for this is OpenEdge.Net.HTTP.Trace.Rule.BodyDataRule. Within this object, set a policy field to allow (allow-all) or deny (deny-all) tracing. You can also optionally define an exceptions array and specify content types that you want to exclude from the policy, as shown in the code example below.
  • Cookies: Inside the request or response object, specify the fully qualified name of the ABL class that processes rules for cookies. The built-in ABL class for this is OpenEdge.Net.HTTP.Trace.Rule.CookieRule. Within this object, set a policy field to allow (allow-all) or deny (deny-all) tracing. You can also optionally define an exceptions array and specify names of cookies that you want to exclude from the policy, as shown in the code example below.
  • Message headers: Inside the request or response object, specify the fully qualified name of the ABL class that processes rules for headers. The built-in ABL class for this is OpenEdge.Net.HTTP.Trace.Rule.HeaderRule. Within this object, set a policy to allow (allow-all) or deny (deny-all) tracing. You can also optionally define an exceptions array and specify the names of headers that you want to exclude from the policy, as shown in the code example below.
  • Query parameter data (requests only): Inside the request object, specify the fully qualified name of the ABL class that processes rules for query parameter data. The built-in ABL class for this is OpenEdge.Net.HTTP.Trace.Rule.QueryStringDataRule. Within this object, set a policy to allow (allow-all) or deny (deny-all) tracing. You can also optionally define an exceptions array and specify the names of query parameters that you want to exclude from the policy, as shown in the code example below.
{
    "enabled": true,
    "output": {
        ...
    },
    "rules": {
        "filter": {
            ...
        },
        "request": {
            "OpenEdge.Net.HTTP.Trace.Rule.BodyDataRule": {
                "policy": "allow-all"
            },
            "OpenEdge.Net.HTTP.Trace.Rule.CookieRule": {
                "policy": "deny-all"
            },
            "OpenEdge.Net.HTTP.Trace.Rule.HeaderRule": {
                "policy": "allow-all",
                "exceptions": [
                    "X-API-Key", "User-Agent", "Accept"
                ]
            },
            "OpenEdge.Net.HTTP.Trace.Rule.QueryStringDataRule": {
                "policy": "deny-all",
                "exceptions": [
                    "CustNum", "CustName"
                ]
            }
        },
        "response":{
            "OpenEdge.Net.HTTP.Trace.Rule.BodyDataRule": {
                "policy": "allow-all",
                "exceptions": [
                    "text/plain"
                ]
            },
            "OpenEdge.Net.HTTP.Trace.Rule.CookieRule": {
                "policy": "allow-all",
                "exceptions": [
                    "JSESSIONID"
                ]
            }
        }
    }
}
CAUTION: Request and response messages may have confidential data, so you must take care to comply with your organization's security policies when configuring tracing on request or response data. You may also consider developing your own custom formatter to encrypt confidential request or response data and using this formatter instead of the built-in HAR formatter in the tracing configuration.
Note that values for the following security headers are obfuscated in logs even when tracing is allowed on them, to prevent sensitive data such as passwords or security tokens from being logged:
  • Authorization
  • Proxy-Authorization
Note: If you unintentionally specify a request or response data rule in the filter object or a filter rule in the request or response objects, the rule is ignored.

Specify a tracing configuration in the ABL HTTP client

By default, the OpenEdge.Net library looks for a file named hctracing.config to load the tracing configuration. Alternatively, you can pass the tracing configuration to the ClientBuilder:TracingConfig() method, when building the ABL HTTP client instance.

The TracingConfig() method takes one of the following parameters:

  • A path to a tracing configuration file.
    var IHttpClient httpClient.

httpClient = ClientBuilder:Build()
             :AllowTracing(TRUE)
             :TracingConfig('HttpClientTracing/hctracing.json')
             :Client.
  • A JSON object containing the tracing configuration.
    var JsonObject config.
var JsonObject child.
var JsonObject grandchild.
var JsonObject detail.
var IHttpClient httpClient.


// Create Json
config = new JsonObject().
config:Add('enabled', true).
child = new JsonObject().
config:Add('output', child).
child:Add('type', 'local').
grandchild = new JsonObject().
child:Add('writer', grandchild).
detail = new JsonObject().
grandchild:Add('options', detail).
detail:Add('fileName', session:temp-dir + 'HttpClient.trace').


// Build client
httpClient = ClientBuilder:Build()
             :AllowTracing(TRUE)
             :TracingConfig(config)
             :Client.

Delegate to a proxy server

If you want to send tracing data to a proxy server, specify the proxyUri in the tracing configuration as shown here:

{
    "enabled": true,
    "output": {
        "type": "proxy",
        "proxyUri": "http://localhost:8888"        
    }
}
If you are using a secured proxy, you must implement the required authentication or authorization in the ABL HTTP client.
Note: If a proxy is defined in the ABL HTTP client or request through the ClientBuilder:ViaProxy() or RequestBuilder:ViaProxy() methods, tracing data is sent to that proxy and the proxyUri property is ignored.

Customize ABL HTTP tracing

If the default formatter, writer, or tracing rules do not meet your requirements, you can customize these components as follows:

Custom formatter
A formatter is an ABL class that receives raw tracing data and formats it in a desired way. To develop a custom formatter, create an ABL class that implements the OpenEdge.Net.HTTP.Trace.Formatter.ITraceFormatter interface. Add a default public constructor with no parameters and implement the Format(input pTraceData as ExchangeData extent) method.
The ExchangeData class is a built-in ABL class that represents the raw tracing data. It has public properties such as StartedAt, RequestMethod, and ServerAddress that you can use in your custom formatter class. To learn more about these properties and methods, see the API documentation.
To use the custom formatter in the tracing configuration, specify its fully qualified class name, just like when using the default HAR formatter.
"formatter": {
            "implementation": "MyABLPackage.MyCustomFormatterClass"
        }
Note that your custom class properties get mapped to an options field in the tracing configuration. This allows additional values to be passed from the tracing configuration to your custom class. See Options for formatters and writers to learn more.
Custom writer
A writer is an ABL class that receives tracing data after it has been formatted by either a built-in or a custom formatter. To develop a custom writer, create an ABL class that implements the OpenEdge.Net.HTTP.Trace.Formatter.ITraceWriter interface. Add a default public constructor with no parameters and implement the Write (input pData as IMemptrHolder) method.
To use the custom writer in the tracing configuration, specify its fully qualified class name, just like when using the default file writer.
"writer": {
            "implementation": "MyABLPackage.MyCustomWriterClass"
        }
Note that your custom class properties get mapped to an options field in the tracing configuration. This allows additional values to be passed from the tracing configuration to your custom class. See Options for formatters and writers to learn more.
Custom tracing rules
You can create two types of tracing rules: filter rules and data rules.
To create a custom filter rule, create an ABL class that implements the OpenEdge.Net.HTTP.Trace.Rule.ITraceFilterRule interface and inherits from the OpenEdge.Net.HTTP.Trace.Rule.TraceRule class.
To create a custom data rule, create an ABL class that implements the OpenEdge.Net.HTTP.Trace.Rule.ITraceDataRule interface and inherits from the OpenEdge.Net.HTTP.Trace.Rule.BodyRule class.
See the OpenEdge.NET API documentation for details on how to implement these custom classes.
To use custom rules in the tracing configuration, specify the fully qualified class name in the filter, request, or response properties. 
"filter": {
            "MyABLPackage.MyCustomFilteringClass": {
                "policy": "...",
                "exceptions": [
                    ...
                 ]
            }
},
"request": {
            "MyABLPackage.MyCustomBodyRuleClass": {
                "policy": "...",
                "exceptions": [
                    ...
                 ]
            }
}

Options for formatters and writers

If you need to pass additional values from the tracing configuration to your custom formatter or writer classes, create public properties in the custom classes with setters. You can then specify values for the properties using options in the tracing configuration.

For example, assuming you want to write trace data to a database, you could implement this custom behavior by specifying the database connection details in the tracing configuration, thus passing those details to your custom writer class. To set this up, you would add public properties in your custom ABL writer class, as shown in this example:

class MyDatabaseWriter implements ITraceWriter:

    define public property DatabaseUrl as character no-undo
        get():
            // implementation of get
        end get.
        set.

    define public property DatabaseUsername as character no-undo
        get():
            // implementation of get
        end get.
        set.

	define public property DatabasePassword as character no-undo
        get():
            // implementation of get
        end get.
        set.
    
...

end class.

Then, in the tracing configuration, you could specify the property values using the options field:

"writer": {
            "implementation": "MyABLPackage.MyDatabaseWriter",
            "options": {
                 "databaseUrl": "jdbc:dbserver://localhost:2345;databaseName=Test",
                 "databaseUsername": "user",
                 "databasePassword": "password"
            }
}

JSON schema for tracing configuration

The JSON schema for the tracing configuration is available in the OpenEdge.Net procedure library. Perform the following steps to obtain the schema:

  1. Launch Proenv.
  2. Change the directory to $DLC/src/netlib. (The netlib directory contains the OpenEdge.Net procedure library.)
    cd C:\Progress\OpenEdge\src\netlib
  3. Extract the schema file, which is named hctracing.config.schema, using prolib as shown in this example:
    prolib OpenEdge.Net.pl -extract hctracing.config.schema
    This extracts the schema file to the netlib directory.

Changes in the ABL Socket library

The OpenEdge.Net.HTTP.Lib.ABLSockets.ABLSocketLibrary class converts ABL HTTP request objects into the bytes required by the HTTP specification.

Prior to the HTTP tracing feature, the ABLSocketLibrary sent a single set of bytes containing all of the HTTP message elements, including the request line, headers, message body, and so on.

Because the HTTP tracing feature enables you to choose what parts of the message to log, it became necessary to split the message into distinct sets of bytes; one set each for the request line, headers, and message body. Each of these elements is now sent as separate messages to the server. If you use a custom implementation of the OpenEdge.Net.ServerConnection.ClientSocket class, which is used by the ABLSocketLibrary, you may need to update the implementation to reflect this change.

Sample tracing configuration

Here is a complete tracing configuration example that you can copy and modify to your requirements. This tracing configuration collects tracing data on all HTTP message transactions. It includes all parts of the request and response message in the tracing data and formats the data in HAR format. Note that the configuration does not include a writer, so if the configuration is used as given, a default writer will output the tracing data to session-temp-directory/TraceOutput.

{
   "enabled":true,
   "output":{
      "type":"local",
      "formatter":{
         "implementation":"OpenEdge.Net.HTTP.Trace.Formatter.HarFormatter"
      }
   },
   "rules":{
      "filter":{
         
      },
      "request":{
         "OpenEdge.Net.HTTP.Trace.Rule.BodyDataRule":{
            "policy":"allow-all"
         },
         "OpenEdge.Net.HTTP.Trace.Rule.CookieRule":{
            "policy":"allow-all"
         },
         "OpenEdge.Net.HTTP.Trace.Rule.HeaderRule":{
            "policy":"allow-all"
         },
         "OpenEdge.Net.HTTP.Trace.Rule.QueryStringDataRule":{
            "policy":"allow-all"
         }
      },
      "response":{
         "OpenEdge.Net.HTTP.Trace.Rule.BodyDataRule":{
            "policy":"allow-all"
         },
         "OpenEdge.Net.HTTP.Trace.Rule.CookieRule":{
            "policy":"allow-all"
         },
         "OpenEdge.Net.HTTP.Trace.Rule.HeaderRule":{
            "policy":"allow-all"
         }
      }
   }
}

Related information

OpenEdge.Net.HTTP.Trace API documentation

HAR Specification

ABL MATCHES operator