Adding new Bridges

Integration Service allows the creation of custom readers and writers for any protocol. This feature is the most powerful of IS, because it allows communicating any DDS application with any other protocol, for example, NGSIv2 from FIWARE Orion ContextBroker, which is a protocol based on WebServices.

_images/Bidirectional_connector.png

To allow new protocols, IS provides an intuitive interface in its classes ISWriter and ISReader that must be inherited by the new implementations, and a Bridge Library that will allow to load the new components into IS through a Connector.

Bridge usage and configuration

An XML Configuration file configures Integration Service. In this case, is needed a Bridge Library to provide functions that will allow IS to create new instances of ISBridge, ISWriter, and ISReader. All these components must be declared (and optionally configured) in the Bridge configuration.

The library of the example will be named libprotocol.so, so it’s necessary to include its location path into the <bridge> section:

<bridge name="protocol">
    <library>libprotocol.so</library>
    <properties>
        <property>
            <name>property1</name>
            <value>value1</value>
        </property>
    </properties>
    <!-- [...] -->
</bridge>

The library can accept configuration parameters through the <properties> tag. In the next section, readers and writers must be declared to make them available for the connectors:

<bridge name="protocol">
    <!-- [...] -->
    <reader name="protocol_subscriber">
        <property>
            <name>property1</name>
            <value>value1</value>
        </property>
        <property>
            <name>property2</name>
            <value>value2</value>
        </property>
    </reader>
</bridge>
<bridge name="protocol">
    <!-- [...] -->
    <writer name="protocol_publisher">
        <property>
            <name>property1</name>
            <value>value1</value>
        </property>
        <property>
            <name>property2</name>
            <value>value2</value>
        </property>
    </writer>
</bridge>

Both, again, can be configured using the <properties> tag. And the Connectors are declared below:

<connector name="shapes_protocol">
    <reader participant_profile="2Dshapes" subscriber_profile="2d_subscriber"/>
    <writer bridge_name="protocol" writer_name="protocol_publisher"/>
    <transformation file="libprotocoltransf.so" function="transformFrom2D"/>
</connector>

<connector name="protocol_shapes">
    <reader bridge_name="protocol" reader_name="protocol_subscriber"/>
    <writer participant_profile="2Dshapes" publisher_profile="2d_publisher"/>
    <transformation file="libprotocoltransf.so" function="transformTo2D"/>
</connector>

In the previous example, there are defined a DDS participant 2Dshapes with a publisher 2d_publisher and a subscriber 2d_subscriber. In the connectors, we bound 2d_subscriber with protocol_publisher and protocol_subscriber with 2d_publisher.

_images/NEW_BRIDGE.png

Typically a transformation function will be useful to split responsibilities and allow readers and writers to only worry about protocol details without data transformations. The Data transformation use case focuses on the usage of Transformation Libraries.

Integration Service’s Bridges

To allow IS instantiate custom ISWriter and ISReader, it’s necessary to add a custom Bridge Library. In this case, and typically, there is no need to override the default ISBridge behavior (and normally isn’t recommended).

As previously explained, the name of the library will be libprotocol.so. The next step in the creation process is to create a new source file named protocol.cpp that must implement the functions create_bridge, create_writer, and create_reader.

Starting the implementation from scratch, it needs to include the required libraries, in this case, the header file that gives access to the implementation of the ISWriter and the ISReader, ISBridgeProtocol.h.

#include "ISBridgeProtocol.h"

The next part is optional, but it helps to make the library portable between different operating systems and keeps the source code clear to read.

#if defined(_WIN32) && defined (BUILD_SHARED_LIBS)
	#if defined (_MSC_VER)
		#pragma warning(disable: 4251)
	#endif
  #if defined(integration_services_EXPORTS)
  	#define  USER_LIB_EXPORT __declspec(dllexport)
  #else
    #define  USER_LIB_EXPORT __declspec(dllimport)
  #endif
#else
  #define USER_LIB_EXPORT
#endif

This optional section should be taken in mind during the creation of the CMakeLists.txt file to configure the project.

Now, the library is in condition to start with the implementation of each function.

As established before the library doesn’t need to override the default ISBridge behavior, therefore, the create_bridge function will return nullptr.

extern "C" USER_LIB_EXPORT ISBridge* create_bridge(const char* name,
    const std::vector<std::pair<std::string, std::string>> *config)
{
    return nullptr;
}

The function create_reader will call directly to the constructor that will return the configured instance of the ISReader.

extern "C" USER_LIB_EXPORT ISReader* create_reader(ISBridge *bridge, const char* name,
    const std::vector<std::pair<std::string, std::string>> *config)
{
    return new ProtocolReader(name, config);
}

In the same way, the function create_writer will call directly to the constructor that will return the configured instance of the ISWriter.

extern "C" USER_LIB_EXPORT ISWriter* create_writer(ISBridge *bridge, const char* name,
    const std::vector<std::pair<std::string, std::string>> *config)
{
    return new ProtocolWriter(name, config);
}

The Bridge API section will detail these functions, along with the implementation of ISReader and ISWriter.

Bridge API

The same source code file can be used to implement the new ISReader and ISWriter, but it is preferable to split the code into several files, and these implementations are very likely to be written in a separated file. In the example, both implementations are written inside two header files ProtocolWriter.h and ProtocolReader.h, both included in ISBridgeProtocol.h, but normally it will be split into source and header files instead only use header files.

ProtocolWriter.h will declare the class ProtocolWriter following the instructions of ISWriter section.

First, the typical definitions must be used to avoid including a header file multiple times:

#ifndef _PROTOCOL_WRITER_H_
#define _PROTOCOL_WRITER_H_

And, in the last line of the file

#endif // _PROTOCOL_WRITER_H_

Then, the file must include the required headers. The example will make use of the cURLpp library.

#include <curlpp>
#include "ISBridge.h"

The next step is to declare the class ProtocolWriter that must inherit from ISWriter.

class ProtocolWriter : public ISWriter
{
private:
    std::string config1;
    std::string config2;
    bool config3;
public:
    ProtocolWriter(const std::string &name, const std::vector<std::pair<std::string, std::string>> *config);
    ~ProtocolWriter() override;

    bool write(SerializedPayload_t*) override;
    bool write(eprosima::fastrtps::types::DynamicData*) override { return false; }; // We don't use it
};

Then the method’s implementation must be defined. The constructor will receive the configuration parameters and will parse them in the example.

ProtocolWriter::ProtocolWriter(const std::string &name, const std::vector<std::pair<std::string, std::string>> *config)
{
    // Configure ProtocolWriter instance with the given config
    // For example:
    for (auto pair : *config)
    {
        try
        {
            if (pair.first.compare("CONFIG1") == 0)
            {
                config1 = pair.second;
            }
            else if (pair.first.compare("CONFIG2") == 0)
            {
                config2 = pair.second;
            }
            else if (pair.first.compare("CONFIG3") == 0)
            {
                config3 = pair.second.compare("TRUE") == 0;
            }
        }
        catch (...)
        {
            return;
        }
    }
}

The destructor will free any taken resource and memory allocation.

ProtocolWriter::~ProtocolWriter()
{
    // Free any taken resources, memory, etc.
}

Finally, the example’s write method implementation will update an entity using a WebService.

bool ProtocolWriter::write(SerializedPayload_t* payload)
{
    // Manage the payload to write into destination protocol
    // For example: A POST request to a WebService
    long code = 600;
    try
    {
        curlpp::Cleanup cleaner;
        curlpp::Easy request;

        // Custom TopicDataType that encapsulates a JSON as:
        // struct Json
        // {
        //     string entityId; // To which entity apply the data (goes into the URL)
        //     string data; // JSON data
        // };
        JsonPubSubType json_pst;
        Json json;
        json_pst.deserialize(payload, &json); // Deserialize the payload into the Json structure

        // Retrieve the data
        std::string entityId = json.entityId();
        std::string payload = json.data();
        // Create a Curl request
        request.setOpt(new curlpp::options::Url(url + "/entity/" + entityId + "/update"));
        std::list<std::string> header;
        header.push_back("Content-Type: application/json");
        request.setOpt(new curlpp::options::HttpHeader(header));
        request.setOpt(new curlpp::options::PostFields(payload));
        request.setOpt(new curlpp::options::PostFieldSize((long)payload.length()));

        // Perform the request
        request.perform();
        code = curlpp::infos::ResponseCode::get(request);
    }
    catch (curlpp::LogicError & e)
    {
        LOG_ERROR(e.what()); // A LOG System. You should have access to the Fast-RTPS's Logger too.
        code = 601;
    }
    catch (curlpp::RuntimeError & e)
    {
        LOG_ERROR(e.what());
        code = 602;
    }

    return (code / 100) == 2; // Return true if success (code family 200)
}

Now, ProtocolReader.h will declare the class ProtocolReader following the instructions of the ISWriter section.

Again, the typical definitions must be used to avoid including a header file multiple times:

#ifndef _PROTOCOL_READER_H_
#define _PROTOCOL_READER_H_

And, in the last line of the file

#endif // _PROTOCOL_READER_H_

Then, the file must include the required headers. The example will make use of the cURLpp library.

#include "ISBridge.h"

The next step is to declare the class ProtocolReader that must inherit from ISReader.

class ProtocolReader : public ISReader
{
private:
    std::string config1;
    std::string config2;
    bool config3;
public:
    ProtocolReader(const std::string &name, const std::vector<std::pair<std::string, std::string>> *config);
    ~ProtocolReader() override;

    void checkUpdates(); // Our custom reading method
};

Then the method’s implementation must be defined. The constructor will receive the configuration parameters and will parse them in the example.

ProtocolReader::ProtocolReader(const std::string &name, const std::vector<std::pair<std::string, std::string>> *config)
{
    // Configure ProtocolReader instance with the given config
    // For example:
    for (auto pair : *config)
    {
        try
        {
            if (pair.first.compare("CONFIG1") == 0)
            {
                config1 = pair.second;
            }
            else if (pair.first.compare("CONFIG2") == 0)
            {
                config2 = pair.second;
            }
            else if (pair.first.compare("CONFIG3") == 0)
            {
                config3 = pair.second.compare("TRUE") == 0;
            }
        }
        catch (...)
        {
            return;
        }
    }
}

The destructor will free any taken resource and memory allocation.

ProtocolReader::~ProtocolReader()
{
    // Free any taken resources, memory, etc.
}

Finally, the example’s checkUpdates method implements a query to a WebService.

void ProtocolReader::checkUpdates()
{
    // Ask the source protocol for updates. Maybe called by a timer, or by event...
    // For example: A Get request to a WebService
    try
    {
        curlpp::Cleanup myCleanup;
        curlpp::Easy myRequest;
        std::ostringstream response;

        // Set the URL.
        // entity/{entityId}/attrs/{attrName}
        myRequest.setOpt<Url>(url + "/entity/" + config1 + "/attrs/" + config2);
        // Store the result in response
        myRequest.setOpt(new curlpp::options::WriteStream(&response));

        // Send request and get a result.
        myRequest.perform();

        // Custom TopicDataType that encapsulates the response:
        // struct Response
        // {
        //     string entityId; // To which entity apply the data (goes into the URL)
        //     string data; // response
        // };
        ResponsePubSubType response_pst;
        Response responseData;
        // Fill the data
        responseData.entityId = config1;
        responseData.data = response.str();
        SerializedPayload_t payload;
        response_pst.serialize(&payload, &responseData); // Serialize the Response into the payload

        // Call on_received_data method
        on_received_data(&payload);
    }
    catch (curlpp::LogicError & e)
    {
        LOG_ERROR(e.what()); // A LOG System. You should have access to the Fast-RTPS's Logger too.
    }
    catch (curlpp::RuntimeError & e)
    {
        LOG_ERROR(e.what());
    }
}

Putting all together

After that, the bridge library is implemented, but it still need to be built. Any build system can be used for this task, but IS provides a CMakeLists.txt template that will be used as starting point of an example.

First, the cmake project will be named to protocol.

project(protocol)

It’s recommendable to keep all C++11 and CMake version as it is but to create the CMakeLists.txt from scratch, it’s important to keep in mind that FastRTPSGen generates files that depend on Fast CDR and Fast RTPS, so they must be included as dependencies to the CMakeLists.txt.

In this case, the example will use cURLpp also.

find_package(fastcdr)
find_package(fastrtps)
find_package(curlpp) # We use curlpp

To make the library more portable the cmake file needs to add the preprocessor definitions to build the library exporting symbols.

add_definitions(-DEPROSIMA_USER_DLL_EXPORT -DBUILD_SHARED_LIBS)

set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(BUILD_SHARED_LIBS TRUE)

Finally, CMake needs the source code of the library to build, along with its dependencies.

file(GLOB USER_LIB_SOURCES_CPP "protocol.cpp") # It includes the rest of files
add_library(protocol SHARED ${USER_LIB_SOURCES_CPP})
target_link_libraries(protocol fastrtps fastcdr curlpp ${CMAKE_DL_LIBS}) # We use curlpp

After that, the library will be generated using CMake.

$ cmake .
$ make

It should generate the libprotocol.so file in the current directory that is the library that IS expects when loads the config.xml file.

At this point, the configuration file config.xml is created and the bridge library libprotocol.so built. IS can be launch with the config.xml and will allow receiving and sending data to the new protocol thanks to the bridge library.

$ integration_service config.xml

Creating new Bridges

The steps needed to define bridge libraries to receive and send data to new protocols are:

  • Create and configure the needed Bridge configuration in the XML configuration file.
  • Create the needed Connectors in the XML configuration file.
  • Implement the custom Bridge Library.
  • Implement the custom ISReader, ISWriter, and, optionally but not recommended, ISBridge.
  • Generate the library binary.
  • Executing IS with the XML configuration file.

Bridge examples

FIROS2

FIROS2 is the implementation of a Bridge Library that allows the intercommunication between ROS2 and Fiware Orion ContextBroker.

HelloWorld To File

To illustrate this use case, there is an example named Helloworld_to_file. This example creates a new bridge to save all received data from the Fast-RTPS HelloWorldExample into a file. IS must be already installed to execute the example.

To achieve that target, it creates a bridge library named isfile. The library only instantiates FileWriter that implements the logic to save the data to a file.

The file config.xml of the example configures IS with the bridge library in a connector that receives data from HelloWorldExample.

Preparation

The HelloWorldExample from Fast-RTPS already must be compiled.

The helloworld_to_file example must be compiled too.

Linux:

$ mkdir build
$ cd build
$ cmake ..
$ make

Windows:

$ mkdir build
$ cd build
$ cmake -G "Visual Studio 14 2015 Win64" ..
$ cmake --build .

The build process will generate the binary of the bridge library.

Execution

In one terminal, launch HelloWorldExample as a publisher:

$ HelloWorldExample publisher

Launch IS in another terminal with the config.xml (config_win.xml if you are on Windows) file from the example folder:

Linux:

$ cd <path_to_is_source>/examples/helloworld_to_file
$ integration_service config.xml

Windows:

$ cd <path_to_is_source>/examples/helloworld_to_file
$ integration_service config_win.xml

Once IS is running, HelloWorldExample will match and a file named output.txt will be created with the received data from HelloWorldExample.

_images/HW_TO_FILE.png