Data transformation

One of the most common issues during the integration of new systems is making them compatible with the ones already implemented in the current environment. Even using RTPS systems there could be situations where two different data types cause direct communication impossible. In this scene is where Integration Service, and its ability to transform data, is useful to adapt different data types to be compatible.

The following image shows the data flow throw transformation function.

_images/TRANF_CASE.png

Transformation usage and configuration

Integration Service must be configured through its XML Configuration file. In this case, a Transformation Library is needed to provide transformation functions that will transform the data from DDS World A to DDS World B and vice versa.

This example supposes that all the configuration is stored in a file named config.xml. The endpoints must be configured in the Fast-RTPS profiles section.

<profiles>
    <participant profile_name="DDS World A">
        <!-- DDS World A participant attributes -->
    </participant>

    <participant profile_name="DDS World B">
        <!-- DDS World B participant attributes -->
    </participant>

    <subscriber profile_name="subscriber A">
        <!-- DDS World A subscriber attributes -->
    </subscriber>

    <subscriber profile_name="subscriber B">
        <!-- DDS World B subscriber attributes -->
    </subscriber>

    <publisher profile_name="publisher A">
        <!-- DDS World A publisher attributes -->
    </publisher>

    <publisher profile_name="publisher B">
        <!-- DDS World B publisher attributes -->
    </publisher>
</profiles>

And the needed Connectors are declared below:

<connector name="A_to_B">
    <reader participant_profile="DDS World A" subscriber_profile="subscriber A"/>
    <writer participant_profile="DDS World B" publisher_profile="publisher B"/>
    <transformation file="libtransformationData.so" function="transformA_to_B"/>
</connector>

<connector name="B_to_A">
    <reader participant_profile="DDS World B" subscriber_profile="subscriber B"/>
    <writer participant_profile="DDS World A" publisher_profile="publisher A"/>
    <transformation file="libtransformationData.so" function="transformB_to_A"/>
</connector>

It’s important to associate the correct participant with the correct publisher or subscriber, in this case: publisher A and subscriber A endpoints belong to DDS World A participant, and publisher B and subscriber B endpoints belong to DDS World B participant.

To transform the data between the DDS World A data and the DDS World B data, Integration Service will use a Transformation Library named libtransformationData.so that have transformation functions implemented: transformA_to_B, to perform the transformation in one way, and transformB_to_A to perform the other way.

Data transformation with Integration Service

This example needs a custom Transformation Library implementing two functions: transformA_to_B to transform the data from DDS World A to DDS World B, and transformB_to_A to transform the data from DDS World B to DDS World A. The name of this library will be libtransformationData.so.

The implementation of both functions will be stored in a source file named transformationDDS.cpp.

The first step of the implementation is including the TopicDataTypes involved. In the example, each DDS World data type is related to an IDL file.

For DDS World A data type:

struct DDSTypeA
{
    long index;
    string category;
    string info;
};

After generating the C++ code of this IDL file, FastRTPSGen will generate a file named DDSTypeAPubSubTypes.h. If you want to know more about FastRTPSGen, please go to FastRTPSGen Documentation.

In the same way for DDS World B data type:

struct DDSTypeB
{
    octet sequenceNumber;
    string message;
};

After generating the C++ code of this IDL file, FastRTPSGen will generate a file named DDSTypeBPubSubTypes.h. So with these types in mind, it’s mandatory to include both PubSubTypes headers in the transformation library.

#include "DDSTypeAPubSubTypes.h"
#include "DDSTypeBPubSubTypes.h"

The next part is optional, but it helps to make the library portable between different operating systems and keeps the source code clear to read.

#if defined(_WIN32) && defined (BUILD_SHARED_LIBS)
    #if defined (_MSC_VER)
        #pragma warning(disable: 4251)
    #endif
  #if defined(integration_services_EXPORTS)
      #define  USER_LIB_EXPORT __declspec(dllexport)
  #else
    #define  USER_LIB_EXPORT __declspec(dllimport)
  #endif
#else
  #define USER_LIB_EXPORT
#endif

This optional section should be taken in mind during the creation of the CMakeLists.txt file to configure the project. After these steps to set the workspace, this will be the code of transformA_to_B:

extern "C" void USER_LIB_EXPORT transformA_to_B(
        SerializedPayload_t *serialized_input,
        SerializedPayload_t *serialized_output)
{
    // Types definition
    DDSTypeA input_data;
    DDSTypeAPubSubType input_pst;
    DDSTypeB output_data;
    DDSTypeBPubSubType output_pst;

    // Deserialization
    input_pst.deserialize(serialized_input, &input_data);

    // Data transformation from A to B
    output_data.sequenceNumber = input_data.index % 256;
    output_data.message = input_data.category + ":" + input_data.info;

    // Serialization
    serialized_output->reserve(output_pst.m_typeSize);
    output_pst.serialize(&output_data, serialized_output);
}

After writing this function and transformB_to_A with the opposite conversion, the transformation library has been implemented, but it needs to be built. Integration Service provides a CMakeLists.txt template that can be used as in this example.

The first step is renaming the cmake project to transformationData.

project(transformationData)

It’s recommendable to keep all C++11 and CMake version as it is, but to create the CMakeLists.txt from scratch it’s important to keep in mind that FastRTPSGen generates files that depend on Fast CDR and Fast RTPS, so both libraries must be included as dependencies to the CMakeLists.txt.

find_package(fastcdr)
find_package(fastrtps)

To make the library more portable the cmake file needs to add the preprocessor definitions to build the library exporting symbols.

add_definitions(-DEPROSIMA_USER_DLL_EXPORT -DBUILD_SHARED_LIBS)

set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(BUILD_SHARED_LIBS TRUE)

The final step is indicating to CMake the source code files and the library to build, along with its dependencies.

file(GLOB USER_LIB_SOURCES_CPP "transformation_data.cpp")
add_library(transformationData SHARED ${USER_LIB_SOURCES_CPP})
target_link_libraries(transformationData fastrtps fastcdr ${CMAKE_DL_LIBS})

After that, CMake will generate the library running these commands:

$ cmake .
$ make

It should generate the file libtransformationData.so in the current directory, and that’s the library that IS expects when loads the config.xml file.

At this point, there is a configuration file config.xml created, and a transformation library libtransformationData.so built. Integration Service now allows connecting both DDS Worlds adapting their data types to be compatible.

$ integration_service config.xml

Creating new transformations

The steps needed to define transformation functions adapting data types between two different domains are:

  • Create and configure the needed Fast-RTPS profiles in an XML configuration file.
  • Create the needed Connectors in the XML configuration file.
  • Implementing custom transformation functions.
  • Generating the binary of the library.
  • Executing Integration Service with the XML configuration file.

Transformation Data example

There is an example implemented in dynamic_types example that shows the use of a transformation function. IS must be already installed to execute the example.

$ mkdir build
$ cd build
$ cmake ..
$ make

Windows:

$ mkdir build
$ cd build
$ cmake -G "Visual Studio 14 2015 Win64" ..
$ cmake --build .

This example creates the communication bridge between the HelloWorld and the Keys examples from FastRTPS. HelloWorldExample must be started as a publisher and the Keys example as a subscriber.

$ ./HelloWorldExample publisher

And in another terminal:

$ ./keys subscriber

In this step, the applications don’t communicate between them, but after starting the Integration Service with the provided configuration file, the communication starts.

$ ./integration_service static_static_config_win.xml