failed to create data sample from cdr buffer

11 posts / 0 new
Last post
Offline
Last seen: 9 months 1 day ago
Joined: 04/09/2019
Posts: 5
failed to create data sample from cdr buffer

Hi, I have this error  "Failed to create Foo from cdr buffer" when I implemented the code below. The struct Foo is written inside an IDL file. This only happens if there's sequence<> in the IDL file. Is this a known issue in RTI Connext DDS 5.3.1?

struct Foo{
    sequence<double, 1000> x;
};

int main(int argc, char* argv[]) {

 Foo foo_original;
 foo_original.x().resize(1);
 foo_original.x()[0] = 0.112;

 std::vector<char> buffer;
 dds::topic::topic_type_support<Foo>::to_cdr_buffer(buffer, foo_original);

 Foo foo_reconstructed;
 dds::topic::topic_type_support<Foo>::from_cdr_buffer(foo_reconstructed, buffer);

 return 0;

}

Offline
Last seen: 9 months 1 day ago
Joined: 04/09/2019
Posts: 5

Referring to my earlier post, no issues if a fixed array is used instead.

struct Foo{
    double x[1000];
};

Disadvantages of using fixed array[] compared to sequence<> :

1. wastage of memory because arrays are fixed in size

2. unneccessary transfer of unused data 

Offline
Last seen: 1 month 3 days ago
Joined: 10/22/2018
Posts: 59

Hi ychua,

I have run a test using 5.3.1 and reproduced your issue.

I ran the same reprocuer using our latest release, 6.0.0, and it worked fine (i.e., no exceptions were thrown) - please let me know if it's an option for you to move to 6.0.0.

Either way, I agree that this behaviour is a little strange, I will investigate further when I get the chance and update you with any findings.

Thanks,
Sam

Offline
Last seen: 1 month 3 days ago
Joined: 10/22/2018
Posts: 59

Hi ychua,

After some internal discussions I now have a good understanding of what is going on.

Short answer:

The  topic_type_support::to_cdr_buffer and  topic_type_support::from_cdr_buffer APIs shouldn’t be used directly. Instead, you should use  rti::topic::to_cdr_buffer and rti::topic::from_cdr_buffer:

Foo foo_original;
foo_original.x().resize(1);
foo_original.x()[0] = 0.112;

std::vector<char> buffer;
rti::topic::to_cdr_buffer(buffer, foo_original);

Foo foo_reconstructed = rti::topic::from_cdr_buffer<Foo>(buffer);

Long answer:

The default C++11 plugin in Connext DDS 5.3.1 requires a call to  rti::topic::to_cdr_buffer first. This call pre-allocates the sample’s sequences, strings, etc., meaning that calls to  rti::topic::to_cdr_buffer_no_alloc are then allowed.

The topic_type_support functions which you are currently using shouldn’t be used directly, they behave in a similar way to the to_cdr_buffer_no_alloc function.

This plugin has been deprecated in Connext DDS 6.0.0 (-legacyPlugin option to rtiddsgen). With the newer plugin (the default in Connext DDS 6.0.0, -stl option to rtiddsgen in 5.3.1), the first call to pre-allocate the fields is not required.

Let me know if this this clears things up,

Sam

Offline
Last seen: 9 months 1 day ago
Joined: 04/09/2019
Posts: 5

Thank you Samr for your thorough investigation and detailed explanation. Your solution worked.

Offline
Last seen: 1 month 3 weeks ago
Joined: 06/05/2020
Posts: 3

Hi Sam,

We have the same issue here. we do something like 

dds::topic::topic_type_support<Foo> topic_type_support;
Foo foo_msg;
topic_type_support.from_cdr_buffer(foo_msg, buffer);

And we try 

Foo foo_msg = rti::topic::from_cdr_buffer<Foo>(buffer)

but we still have this issue:

Failed to create Foo from cdr buffer

We use 6.0.0. and we found that rti::topic::from_cdr_buffer is super slower than topic_type_support.from_cdr_buffer.

Chao Wang

Offline
Last seen: 1 month 3 days ago
Joined: 10/22/2018
Posts: 59

Hey Chao Wang,

The topic_type_support.from_cdr_buffer is for internal use only. The rti::topic::from_cdr_buffer API should not be slower.
Could you expand on what you are doing to measure speed? And how the results.

 

Sam

Offline
Last seen: 1 month 3 weeks ago
Joined: 06/05/2020
Posts: 3

Hi Sam,

I have a application which parse the DDS store and do some calculation.

I try both  topic_type_support.from_cdr_buffer and rti::topic::from_cdr_buffer with the same  DDS store.

topic_type_support.from_cdr_buffer cost about 10 second,

and  rti::topic::from_cdr_buffer cost about 48 second.

I restart my laptop before every run.

I believe the 38 second diff totaly come from the diff between topic_type_support.from_cdr_buffer and rti::topic::from_cdr_buffer.

Chao

Offline
Last seen: 1 month 3 days ago
Joined: 10/22/2018
Posts: 59

Wow that's a big difference.
Whilst I look into it more this side, could you use a profiler to see where the additional 38 seconds are going? You may need to use the debug libraries which were shipped alongside your Connext DDS Pro installation. They will be in $NDDSHOME/lib/<arch>. The debug libraries are suffixed with a 'd'.

Offline
Last seen: 1 week 4 days ago
Joined: 04/02/2013
Posts: 131

Hi Chao,

When performance is a concern,  rti::topic::from_cdr_buffer should be used the first time only, then you can use  rti::topic::from_cdr_buffer_no_alloc.

For example:

auto sample = rti::topic::from_cdr_buffer<Foo>(buffer);

for (...) {

    update(buffer);

    rti::topic::from_cdr_buffer_no_alloc(sample, buffer);

}

The call to from_cdr_buffer ensures that the returned sample preallocates all its sequences, strings, etc. 

Calling topic_type_support directly is not safe.

Offline
Last seen: 1 month 3 weeks ago
Joined: 06/05/2020
Posts: 3

Hi Alexc

This way's performance is similar to topic_type_support.from_cdr_buffer. I will this function to replace topic_type_support.from_cdr_buffer .

Thanks

Chao Wang