Conversion of an UUID fails

[ptrdtznr]

We are facing an error of the conversion of UUIDs... this relates to the FiwareObjectConverter from my point of view but needs to be verified and fixed. If you need the messages, let me know.

  File "/usr/lib/python2.7/json/encoder.py", line 209, in encode
    chunks = list(chunks)
  File "/usr/lib/python2.7/json/encoder.py", line 434, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/usr/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib/python2.7/json/encoder.py", line 443, in _iterencode
    for chunk in _iterencode(o, _current_indent_level):
  File "/usr/lib/python2.7/json/encoder.py", line 434, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/usr/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib/python2.7/json/encoder.py", line 443, in _iterencode
    for chunk in _iterencode(o, _current_indent_level):
  File "/usr/lib/python2.7/json/encoder.py", line 434, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/usr/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib/python2.7/json/encoder.py", line 443, in _iterencode
    for chunk in _iterencode(o, _current_indent_level):
  File "/usr/lib/python2.7/json/encoder.py", line 434, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/usr/lib/python2.7/json/encoder.py", line 390, in _iterencode_dict
    yield _encoder(value)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xef in position 0: invalid continuation byte

here is the ROS message from the rostopic echo

MoveBaseSimpleState: 0
---
header: 
  seq: 58
  stamp: 
    secs: 61
    nsecs: 200000000
  frame_id: "map"
robot_id: 
  uuid: [239, 199, 236, 49, 108, 114, 67, 11, 143, 62, 64, 182, 39, 17, 109, 130]
  description: ''
MoveBaseSimpleState: 0

[Luxxii]

This Error might also be a Bug/Error/intended Behaviour caused by ROS. On this site ROS (or more specific 'rospy') describes how it converts specific datatypes into Python-Data-Types. If you take a look at the Array-Handling, uint8[]-Arrays are converted into str (???) for Python2 and into byte in Python3.

Surprisingly int8[]-Arrays are always converted into tuples or lists for Python2 or 3 which makes it very easy to convert.

So a quick fix would be to avoid uint8[]-Arrays, cause they behave (in my opinion) very strangely.

[Luxxii]

The new ObjectConverter now handles such cases. Keep in mind that this is just a construct on top which handles uint8[]-Arrays differently.

This can easily break, if ROS decides to convert it differently into e.g. Int's (or any other construct you can imagine).

[dondiegodelabega]

Pulled from the development branch the new version. It works and updates to OCB the uuids as promised (thank you for the quick bugfix), but it still prints the ROS_ERROR.
I attach a bag subset with a topic (pls remove the .zip extension).
subset.bag.zip
If needed, I can provide more feedbacks.
Thank you <3

[Luxxii]

The submodule wasn't added properly in the last commit. With your bag i am able to send the Message to the Contextbroker.

I could not find any specific ROS_ERROR. Can you give me some more details?

[marija185]

Hello Luxxi!
I am experiencing the same issue with uint8[16] for uuid (about 'utf8' codec like here in the first message). I tried with int8[16] and it works but I get a message too large (payload size: 1312003, max size supported: 1048576). The reason why I changed the message to uint8[16] is to reduce the size of the message, but it is interesting that now the message is bigger than when I used string uuids. With string uuids for the same example, it works. Do you know what can be a problem? Is it possible that:

int8[16] uuid: [-35, 2, -125, -117, 117, -120, 80, -13, -114, 12, -109, -72, 98, -43, -70, -63]

is larger than

string 63dc1b40-2634-5a5c-9434-b2efffb77a8a

when converted to OCB?

Many thanks,
Marija

[Luxxii]

Hi @marija185,

Yes, this can happen. In your case: The uuid-Array is (correctly) larger than the single string of the uuid.

Here an example of the array, where each value is wrapped:

...
"type":"YOUR UUID TYPE",
"value":{
    "uuid":{
        "type":"array",
        "value":[
        {
            "type":"number",
            "value":-35
        },
        {
            "type":"number",
            "value":2
        },
        ...
        ]
...

And here the String version:

...
"type":"YOUR UUID TYPE",
"value":{
    "uuid":{
        "type":"string",
        "value":"63dc1b40-2634-5a5c-9434-b2efffb77a8a"
...

The payload size of the OCB should now be customizable. If you are experiencing that your messages are slightly to large, you could increase the limit.

[marija185]

Hello @Luxxii !
Thanks for your reply! I will stick to the string representation of uuid. My message has a lot of uuids (array of custom messages that contain uuid) and when using strings I don't have a message about payload exceeded. The idea was to make the message smaller, not bigger, which sounded like a good idea since int8[16] is smaller than 36 Bytes of string representing uuid. But the representation of messages in firos makes int8[16] bigger than one string of 36 Bytes.
Thanks for clarifying this!
Best regards
Marija

[Luxxii]

Hello again @marija185,

There is a slightly better representation of arrays in FIWARE-Entities, which i found: #1 (comment)

But i cannot guarantee, that we implement this without more documentation about this..

Best regards,
Luxxii

[marija185]

Oho! Thanks! That means int8[16] could be compressed too in the future.
best regards
Marija