Unexpected Serial Response

[tom-gangemi]

I'm trying to use PCIProtocol to write my own python cbus-web interface but am having an issue where writes to the serial interface are getting an unexpected reply.

PC Interface: 5500PC
Firmware Version: 4.2.00

For example, when I create a new PCIProtocol, I get the following output:

send: '~~~'
send: 'A3210038h'
send: 'A3420002i'
send: 'A3300059j'

recv: '~~~'
recv: caught exception. line state = '~~~'
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cbus-0.1-py2.7.egg/cbus/protocol/pciprotocol.py", line 76, in lineReceived
    self.decode_cbus_event(line)
  File "/usr/lib/python2.7/site-packages/cbus-0.1-py2.7.egg/cbus/protocol/pciprotocol.py", line 98, in decode_cbus_event
    p, line = decode_packet(line, checksum=True, server_packet=True)
  File "/usr/lib/python2.7/site-packages/cbus-0.1-py2.7.egg/cbus/protocol/packet.py", line 104, in decode_packet
    raise ValueError, "Non-base16 input: %r in %r" % (c, data)
ValueError: Non-base16 input: '~' in '~~~'

recv: 'A3210038hh.322100AD'
recv: caught exception. line state = 'A3210038hh.322100AD'
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cbus-0.1-py2.7.egg/cbus/protocol/pciprotocol.py", line 76, in lineReceived
    self.decode_cbus_event(line)
  File "/usr/lib/python2.7/site-packages/cbus-0.1-py2.7.egg/cbus/protocol/pciprotocol.py", line 98, in decode_cbus_event
    p, line = decode_packet(line, checksum=True, server_packet=True)
  File "/usr/lib/python2.7/site-packages/cbus-0.1-py2.7.egg/cbus/protocol/packet.py", line 104, in decode_packet
    raise ValueError, "Non-base16 input: %r in %r" % (c, data)
ValueError: Non-base16 input: 'h' in 'A3210038hh.322100AD'

recv: 'i.3242008C'
dce: packet: <cbus.protocol.confirm_packet.ConfirmationPacket object at 0x75efafd0>
recv: confirmation: code = 'i', success = True
recv: caught exception. line state = 'i.3242008C'
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cbus-0.1-py2.7.egg/cbus/protocol/pciprotocol.py", line 76, in lineReceived
    self.decode_cbus_event(line)
  File "/usr/lib/python2.7/site-packages/cbus-0.1-py2.7.egg/cbus/protocol/pciprotocol.py", line 98, in decode_cbus_event
    p, line = decode_packet(line, checksum=True, server_packet=True)
  File "/usr/lib/python2.7/site-packages/cbus-0.1-py2.7.egg/cbus/protocol/packet.py", line 145, in decode_packet
    p = DeviceManagementPacket.decode_packet(data, checksum, flags, destination_address_type, rc, dp, priority_class)
  File "/usr/lib/python2.7/site-packages/cbus-0.1-py2.7.egg/cbus/protocol/dm_packet.py", line 50, in decode_packet
    assert ord(data[1]) == 0, 'second byte of DeviceManagementPacket must be 0'
IndexError: string index out of range

recv: 'j.8601010032300016'
dce: packet: <cbus.protocol.confirm_packet.ConfirmationPacket object at 0x75edbe70>
recv: confirmation: code = 'j', success = True
recv: caught exception. line state = 'j.8601010032300016'
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cbus-0.1-py2.7.egg/cbus/protocol/pciprotocol.py", line 76, in lineReceived
    self.decode_cbus_event(line)
  File "/usr/lib/python2.7/site-packages/cbus-0.1-py2.7.egg/cbus/protocol/pciprotocol.py", line 98, in decode_cbus_event
    p, line = decode_packet(line, checksum=True, server_packet=True)
  File "/usr/lib/python2.7/site-packages/cbus-0.1-py2.7.egg/cbus/protocol/packet.py", line 150, in decode_packet
    p = PointToPointPacket.decode_packet(data, checksum, flags, destination_address_type, rc, dp, priority_class)
  File "/usr/lib/python2.7/site-packages/cbus-0.1-py2.7.egg/cbus/protocol/pp_packet.py", line 103, in decode_packet
    handler = REQUESTS[cmd]
KeyError: 50

Thanks!

[micolous]

It looks like your PCI is doing a local echo. It's been a while since I looked at this, but I think this is intentional, at least partially. In your traceback, after A3210038h you don't get local echo anymore.

But after that, I'm not sure what's going on. I tried to consult the documentation to make more sense of this, but Clipsal has taken that down. I also no longer have easy access to an installed CBus system to confirm that this is 100% intentional behaviour.

From memory, during that reset/initialisation process, the PCI sends a bunch of replies, but the rest of the code never actually uses them, and we assume that everything else is configured correctly after that point. Some of the messages are in a different protocol mode that this software doesn't use, and they weren't fully documented.

I didn't care about them too much with cdbusd, because I'd never see the errors at startup time. Then whatever other application I wanted to use would connect to that, and the PCI would be in the correct state.

Did pciprotocol.py work after this point? ie: if you activate some light switch, are you getting messages on the bus, and can you send lighting control events to the network?

Also, if you're building your own application on the pciprotocol layer, bear in mind that you'll need to keep a single persistent connection open to the PCI. If you have modules that require a time source, you'll also need to provide that. cdbusd does that for you, and that is what I would recommend as a base for writing some webserver control without dbus.

[tom-gangemi]

Thanks for the reply.

The main issue appears to be a result of the library working via lineReceived but confirmation replies not being "\r\n" terminated. Subsequently, the confirmation character is being read as the start of the following reply.

Besides the error in parsing these initial replies, the library does work.

I've managed to find a copy of the serial dev guide for firmware version 4. So as a learning exercise, I've started my own C-Bus PCI interface library using Python 3 and asyncio in which I've used this library as a bit of a guide.

[micolous]

A few months ago I found the docs again (fetch_protocol_docs.py should download them all) -- however these guides only mention up to Firmware 3.16 (doc version 1.17, published 2008-12-09).

I've since moved all of this project to Python 3, and ended up moving it to the asyncio library. That's now merged as master as of today.

When I did this migration, I've also done lots of refactoring and replaced Twisted with asyncio, and built a protocol layer that handles some of C-Bus' protocol inconsistencies better. I'm pretty sure that the issue with confirmations not showing up until after a \r\n is fixed now.

There are still some edge cases where this library is simulating a PCI where it doesn't give you events even when it could probably decode them (eg: using ? to cancel a command, and probably everything to do with "BASIC mode"), but they generally only impact interactive use where a human is typing out C-Bus serial commands into a terminal.

Unfortunately I haven't been able to fix local echo properly (it's complicated), and you will need to restart the program if the PCI loses power (but most likely if your PCI loses power your computer will too...).

[lowgoz]

came across this when i googled my own weird output - similar to the above but seems slightly different??

2020-03-05 00:57:18+0000 [CBusHandler,client] recv: b'~x~\x7fx~\x7f0E0207E40305030D010A382A0080' 2020-03-05 00:57:18+0000 [CBusHandler,client] dce: packet: InvalidPacket(payload=b'~x~\x7fx~\x7f0E0207E40305030D010A382A0080', exception=ValueError("Non-base16 input: 7e in b'~x~\\x7fx~\\x7f0E0207E40305030D010A382A0080'")) 2020-03-05 00:57:18+0000 [CBusHandler,client] dce: unhandled other packet InvalidPacket(payload=b'~x~\x7fx~\x7f0E0207E40305030D010A382A0080', exception=ValueError("Non-base16 input: 7e in b'~x~\\x7fx~\\x7f0E0207E40305030D010A382A0080'"))