HTTP GET Requests within Kepware: Purpose and Use

Hypertext Transfer Protocol (HTTP) is a well-defined communications protocol that outline several request methods and services to enable requests to, or take action on a target server. HTTP is the standard protocol for a wide range of data transfer throughout our daily lives – whether we know it or not. For example, when you browse social media on your phone, your device is issuing HTTP requests to retrieve data from the social media server. When you use online banking, again, the underlining communications protocol being used is often HTTP. HTTP is everywhere today, even in the industrial plant.

There are a number of examples of how HTTP is being used in the industrial space. Whether it be a new smart sensor, or more generically a simple software-based API. HTTP is more than a protocol that’s used strictly on the public internet. It can be confined within the walls of the factory as well.

HTTP defines several request methods to be used to take action on a target server. The defined HTTP request methods are GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS and PATCH. Each request method differs in its desired action. In addition to the basic CRUD activities (Create, Read, Update and Delete), there are also more specialized methods to take more specific action on the server.

This blog post and the associated demonstrated example will explore the GET request method in more detail.

Consider our everyday lives, where we use an internet browser to access data from a web site. This website contains all sorts of data to be displayed on the web page within our browser. The internet browser often issues HTTP GET requests to retrieve some sort of data set to be displayed to you, the user. The response to these GET requests contain the specific data requested. As you click around into different areas of the site, several GET requests are being sent to retrieve new content. Let’s take a closer look at how this all works.

The GET request that is issued includes a target HTTP endpoint. After all, the HTTP client, which in our example is our internet browser, needs to know where to issue the request for data. These endpoints include the protocol to be used (HTTP) as well as the target location of the server. The endpoint may also include key pieces of information that tells the target server what data is being requested.

Continuing with our example, if you were to use your internet browser to browse search results for the keyword “Kepware”, the user would populate the browser with this string: http://google.com/search?q=kepware

Notice the protocol definition (HTTP), the target server (Google), as well as the data that’s being requested – in this case, a search for the keyword “Kepware”. The HTTP server (a search engine), then responds accordingly with all its lookup results for Kepware.

Once these results are returned, and you click on one of the returned links, another GET request is issued to retrieve a new set of data from a different HTTP endpoint, say http://velotic.com/products/kepware. And so on.

For the sake of the upcoming demonstration, we will be working with the Open Weather Map API located at api.openweathermap.org. This open API provides the ability to issue HTTP GET requests to read weather data for a given area. Here is an example of the HTTP endpoint being accessed by a web browser and the associated payload within the response.

http://api.openweathermap.org/data/2.5/onecall?lat=43.65&lon=70.25&exclude=hourly,daily,minutely&appid={Your app key}

Response:

{
"lat": 43.65,
"lon": -70.25,
"timezone": "America/New_York",
"timezone_offset": -14400,
"current": {
"dt": 1687889314,
"sunrise": 1687856474,
"sunset": 1687911995,
"temp": 294.15,
"feels_like": 294.66,
"pressure": 1009,
"humidity": 90,
"dew_point": 292.45,
"uvi": 1.35,
"clouds": 100,
"visibility": 10000,
"wind_speed": 5.14,
"wind_deg": 160,
"weather": [
{
"id": 804,
"main": "Clouds",
"description": "overcast clouds",
"icon": "04d"
}
]
}
}

Behind the scenes, like in the other examples, an HTTP GET request is sent by the browser to the API. You can find the structure of the GET request using network monitoring tools, like Wireshark. Within Wireshark, there is a feature that enables you to follow a specific TCP stream. Or, in other words, you can view all messages being sent within that TCP socket. When following the TCP stream of the connection, the GET request is outlined:

GET /data/2.5/onecall?lat=43.65&lon=-70.25&exclude=hourly,daily,minutely&appid={my personalized app key} HTTP/1.1

Host: api.openweathermap.org

The HTTP response is as follows:

HTTP/1.1 200 OK
Server: openresty
Date: Tue, 27 Jun 2023 18:21:00 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 393
Connection: keep-alive
X-Cache-Key: /data/2.5/onecall?exclude=hourly,daily,minutely&lat=43.65&lon=-70.25
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET, POST

{"lat":43.65,"lon":-70.25,"timezone":"America/New_York","timezone_offset":-14400,"current":{"dt":1687890040,"sunrise":1687856474,"sunset":1687911995,"temp":294.21,"feels_like":294.72,"pressure":1009,"humidity":90,"dew_point":292.51,"uvi":1.35,"clouds":100,"visibility":10000,"wind_speed":5.14,"wind_deg":160,"weather":[{"id":804,"main":"Clouds","description":"overcast clouds","icon":"04d"}]}}

The goal within the Kepware custom driver is to rebuild this exact based HTTP request and to parse the associated response.

The HTTP payload is constructed of a series of ASCII strings separated by a few spaces, carriage returns, and line feeds. The U-CON driver is a great tool to recreate this string of ASCII characters. The Transaction Editor (located in the Device Properties of the U-CON driver configuration) simply builds the ASCII message to be sent by this custom-built driver to the target endpoint.

Within the Transaction Editor, after you create a “tag block”, and the associated tags you’d like to update within that block, you can then populate the “read transaction” with several commands. This command set will be run through when a read request is sent from an incoming client to one of the tags within the block. Here this read transaction configured to issue the HTTP GET request.

The response contains the data sent by the REST server to the U-CON driver in response to the GET request—in this case, as a JSON object. The driver then interprets this response to ultimately update tag values. Adding "Read Response", “Seek String” and "Update Tag" commands within the Transaction Editor allows the driver to find the keys of the JSON structure and update the appropriate tags.

The sequence of the commands within the Transaction Editor depends on the format of the response message. A description of all the available commands can be found within the driver manual.

The OPC Quick Client provides a clear review of the returned tag value. An internal OPC DA tool, it quickly and easily displays the name, value, quality, and timestamp of any available tags from the server. Simply browse to the newly created tag to review the results. Seeing numeric values and good quality data is a sign of success!

As the need to integrate even more disparate data sets into your industrial data collection infrastructure increases, you need software tools that enable you to be flexible, and ultimately successful. The Kepware User Configurable Driver is still a great tool to graphically build custom driver behavior when there isn’t an off-the-shelf option.

With that said, Kepware is now equipped with a new custom driver named the Universal Device Driver. This new custom driver enables users to build custom driver behavior using predefined JavaScript functions that are then executed by the Kepware communications platform. Please contact us today for more information.