Dynamic Data Integration

Fast-RTPS 1.7.0 includes Dynamic Data Types and XML Types as new features. Integration Service benefits from these features allowing to integrate Dynamic Data Types for the transformation of data types.

This use case illustrates a scenario using both types of Dynamic Data declarations (Dynamic Data Types and XML Types) and a transformation function. For a full use case explaining transformation functions, you can see Data transformation section.

The following image shows the data flow through a transformation function.

_images/DYNAMIC_CASE.png

Dynamic Data usage and configuration

There are two ways to declare Dynamic Data Types in Integration Service. They could be created using IS Types Libraries, or they can be declared directly as Fast-RTPS XML Types in the XML configuration file.

<is_types>
    <!-- Fast-RTPS XML Types -->
    <types>
        <type>
            <struct name="HelloWorld">
                <unsignedlong name="index"/>
                <string name="message"/>
            </struct>
        </type>
    </types>

    <!-- IS Types Libraries -->
    <type name="ShapeType">
        <library>libshape.so</library>
    </type>
    <types_library>libdefault.so</types_library>
</is_types>

This example needs to define two Dynamic Data Types: TypeA and TypeB, that will be used to communicate the DDS World A with DDS World B, applying a transformation function, that will be the typical scenario.

The first step is creating the XML configuration file to indicate which library implements each data type. In this case, both types TypeA and TypeB will be implemented in the same library.

<is_types>
    <type name="TypeA">
        <library>libtypeatypeb.so</library>
    </type>
    <type name="TypeB">
        <library>libtypeatypeb.so</library>
    </type>
</is_types>

Then, in the Fast-RTPS profiles section, the publisher and subscriber can refer the data types by name.

<profiles>
    <participant profile_name="DDSWorldA">
        <rtps>
            <builtin>
                <domainId>0</domainId>
            </builtin>
        </rtps>
    </participant>

    <participant profile_name="DDSWorldB">
        <rtps>
            <builtin>
                <domainId>5</domainId>
            </builtin>
        </rtps>
    </participant>

    <subscriber profile_name="is_subscriber">
        <topic>
            <name>DDSTopicA</name>
            <dataType>TypeA</dataType>
        </topic>
    </subscriber>

    <publisher profile_name="is_publisher">
        <topic>
            <name>DDSTopicA</name>
            <dataType>TypeB</dataType>
        </topic>
    </publisher>
</profiles>

And finally it needs a Connector to relate all the components:

<connector name="DDS_A_to_B">
    <reader participant_profile="DDSWorldA" subscriber_profile="is_subscriber"/>
    <writer participant_profile="DDSWorldB" publisher_profile="is_publisher"/>
    <transformation file="libtransform.so" function="transformAtoB"/>
</connector>

As this scenario focus on Dynamic Topic Data, it isn’t going to explain how to create the Transformation Library, but there is a complete use case about it in Data transformation section.

Dynamic Data with Integration Service

As explained in the previous section, there are two ways to declare Dynamic Data Types in IS. Using Fast-RTPS XML Types, there is no need to perform any additional step, and the Data Types will be available to use them directly. And using a Types Library, it’s mandatory to implement the library and build it.

In this example, the name of the library is going to be libtypeatypeb.so, and at least it must implement the function GetTopicType. There is a template file typeslib.cpp in resources folder in the IS repository.

This part of the example shows the full process to create a Types Library. The source file is going to be named typeatypeb.cpp.

Firstly it needs to include all required headers to use Fast-RTPS Dynamic Types.

#include <iostream>
#include <fastrtps/TopicDataType.h>
#include <fastrtps/types/DynamicTypeBuilderFactory.h>
#include <fastrtps/types/DynamicDataFactory.h>
#include <fastrtps/types/DynamicTypeBuilder.h>
#include <fastrtps/types/DynamicTypeBuilderPtr.h>
#include <fastrtps/types/DynamicPubSubType.h>
#include <fastrtps/types/DynamicType.h>
#include <fastrtps/types/DynamicData.h>
#include <fastrtps/types/DynamicDataPtr.h>

The next part is optional, but it helps to make the library portable between different operating systems and keeps the source code clear to read. Additionally, in this case, may be useful to add some using namespaces.

#if defined(_WIN32) && defined (BUILD_SHARED_LIBS)
#if defined (_MSC_VER)
#pragma warning(disable: 4251)
#endif
#if defined(EPROSIMA_SHAPES_DLL_EXPORT)
#define  USER_LIB_EXPORT  __declspec(dllexport)
#else
#define  USER_LIB_EXPORT  __declspec(dllimport)
#endif
#else
#define USER_LIB_EXPORT
#endif

using eprosima::fastrtps::TopicDataType;
using namespace eprosima::fastrtps;
using namespace eprosima::fastrtps::types;

This optional section should be taken in mind during the creation of the CMakeLists.txt file to configure the project.

Usually, it’s useful to create auxiliary functions to generate each type of data.

To create DataTypeA:

static DynamicPubSubType* GetDataTypeA()
{
    // Create basic types
    DynamicTypeBuilder_ptr created_type_ulong = DynamicTypeBuilderFactory::GetInstance()->CreateUint32Builder();
    DynamicTypeBuilder_ptr created_type_string = DynamicTypeBuilderFactory::GetInstance()->CreateStringBuilder();
    DynamicTypeBuilder_ptr struct_type_builder = DynamicTypeBuilderFactory::GetInstance()->CreateStructBuilder();

    // Add members to the struct.
    struct_type_builder->AddMember(0, "index", created_type_ulong.get());
    struct_type_builder->AddMember(1, "message", created_type_string.get());
    struct_type_builder->SetName("DataTypeA");

    DynamicType_ptr dynType = struct_type_builder->Build();
    DynamicPubSubType *psType = new DynamicPubSubType(dynType);
    return psType;
}

To create DataTypeB:

static DynamicPubSubType* GetDataTypeB()
{
    // Create basic types
    DynamicTypeBuilder_ptr created_type_octet = DynamicTypeBuilderFactory::GetInstance()->CreateByteBuilder();
    DynamicTypeBuilder_ptr struct_type_builder = DynamicTypeBuilderFactory::GetInstance()->CreateStructBuilder();

    // Add members to the struct.
    struct_type_builder->AddMember(0, "index", created_type_octet.get());
    created_type_octet->ApplyAnnotation("@Key", "true");
    struct_type_builder->AddMember(1, "key_value", created_type_octet.get());
    struct_type_builder->SetName("DataTypeB");
    struct_type_builder->ApplyAnnotation("@Key", "true");

    DynamicType_ptr dynType = struct_type_builder->Build();
    DynamicPubSubType *psType = new DynamicPubSubType(dynType);
    return psType;
}

And after the creation of these functions, the next step is filling the GetTopicType function to return an instance of the declared types:

extern "C" USER_LIB_EXPORT TopicDataType* GetTopicType(const char *name)
{
    if (strncmp(name, "DataTypeA", 10) == 0)
    {
        return GetDataTypeA();
    }
    else if (strncmp(name, "DataTypeB", 10) == 0)
    {
        return GetDataTypeB();
    }
    return nullptr;
}

After that, the types library is implemented, but it needs to be built. Integration Service provides a CMakeLists.txt template that can be used as in this example.

The first step is renaming the cmake project to typeatypeb.

project(typeatypeb)

It’s recommendable to keep all C++11 and CMake version as it is but to create the CMakeLists.txt from scratch, it’s important to keep in mind that FastRTPSGen generates files that depend on Fast CDR and Fast RTPS, so they must be included as dependencies to the CMakeLists.txt.

find_package(fastcdr)
find_package(fastrtps)

To make the library more portable the cmake file needs to add the preprocessor definitions to build the library exporting symbols.

add_definitions(-DEPROSIMA_USER_DLL_EXPORT -DBUILD_SHARED_LIBS)

set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(BUILD_SHARED_LIBS TRUE)

The final step is indicating to CMake the source code files and the library to build, along with its dependencies.

file(GLOB USER_LIB_SOURCES_CPP "typeatypeb.cpp")
add_library(typeatypeb SHARED ${USER_LIB_SOURCES_CPP})
target_link_libraries(typeatypeb fastrtps fastcdr ${CMAKE_DL_LIBS})

After that, CMake will generate the library running these commands:

$ cmake .
$ make

It should generate the file libtypeatypeb.so in the current directory, and that’s the library that IS expects when loads the config.xml file.

This example needs a Transformation Library too. There is a complete use case about transformation libraries in Data transformation section.

At this point, there is a configuration file config.xml created, and a types library libtypeatypeb.so built. Integration Service now allows generating both dynamic types and connect two DDS Worlds adapting their data types to be compatible.

$ integration_service config.xml

Creating new Dynamic Data

The steps needed to define types libraries to create Dynamic Data Types are:

  • Create and configure the needed IS Types configuration in an XML configuration file.
  • Create and configure the needed Fast-RTPS profiles in the XML configuration file.
  • Create the needed Connectors in the configuration file.
  • Implementing a custom Types Library, if needed.
  • Generating a types library binary, if needed.
  • Executing IS with the XML configuration file.

Dynamic Types example

IS must be already installed to execute the example. There is an example implemented in the dynamic_types example where you can see the use of dynamic types.

$ mkdir build
$ cd build
$ cmake ..
$ make

Windows:

$ mkdir build
$ cd build
$ cmake -G "Visual Studio 14 2015 Win64" ..
$ cmake --build .

This example allows the communication between the HelloWorldExample and the Keys examples from FastRTPS. HelloWorldExample must be started as a publisher and the Keys example as a subscriber.

$ ./HelloWorldExample publisher

And in another terminal:

$ ./keys subscriber

In this step, both applications don’t have any communication between them, and after starting the Integration Service with the given configuration file, the communication will be started. dyn_dyn_config[_win].xml uses Dynamic Types in both flavors.

$ ./integration_service dyn_dyn_config_win.xml