Greg Young - MQTT text string?I have a device...
MQTT text string?
I have a device (ESP32) that is sending data to signalk MQTT broker,
for values (eg /vessels/self/electrical/generator/Oil/pressure ) which works fine.
however Im also trying to send a text string which is the generator status (eg “Running”)
when using MQTT tool (or similar) to look at messages on MQTT server, … i see two entries , the second of which is reported as NaN
(& in signalk data browser it reports as null)
It looks like MQTT is ok with me sending simple values, but it doesnt like a string?
Am i correct that i need to do some specific formatting in the MQTT.publish( ) message?
27 Replies
I wonder if it’s expecting json
im guessing yes …
and for the simple case wheree it just has a numeric value … it tolerates no formatting
… but im far from competent in json … im good and modify/cutnpaste and following examples ..
I think it may be your mqtt tool that is expecting a string that can be parsed as number and the result is NotANumber
Mqtt as protocol does not have a type for the payload
ok,
but the value field in sk data browser is showing null,
is that because i havent set units? or meta? for that path
.. although when using same tool to look at other MQTT messages (eg coming from venus plugin) … i can see just text/string..
… so i assumed it was because of my formatting of the mqtt publish message
using same tool to look at different MQTT msgs on broker,
it is actually seeing the (correct) string (eg status1) coming in, but also its getting NaN at same time (immediatly after the status string) …. and NaN is the most recent recorded by broker
I am confused about your setup - you talk about sending from esp to mqtt but then about databrowser
Since it is your code can’t you just log exactly what you are sending?
Databrowser does not refesh realtime, so if a sensor always sends two values in succession like A and B, in databr you will practically always see just B
sorry for the confusion
when im looking at MQTT (broker) messages (eg using MQTT Tool) - i see two entries each time a MQTT msg is published by client
the first one is correct (eg a text string) followed immediately (at same time stamp) by NaN..
and hence NaN is effectively always the latest message in MQTT broker.
When i look at the path using data browser (that is being created by MQTT plugin) , it has correct path , but the value is shown as null.
Im assuming thats because of the NaN issue that is being created in the broker.
Im using python on an ESP32, with a simple client to publish the MQTT messages.
Its working fine when i publish numeric values (string) …
but for reason unknown to me, when i publish a text string, im getting this double entry issue in MQTT broker.
Further testing
Using a “standard” Mosquitto broker (2.0.11 ). installed onto RPi ,
my same ESP MQTT publish code, works fine … aka no NaN recorded.
So at present, im leaning towards there being some? difference in the way the signalk MQTT broker is treating “non numeric” characters in a string.
for example.
I can publish payloads 1, “1” and they are treated correctly & appear in sk path as a 1.
But as soon as I try to include non numeric characters in the string/ payload ,
the broker records the text characters (correctly), but also recorrds NaN (as a second entry) immediately after the correct text is recorded (per above snapshots)
So my thinking is that for some reason the sk MQTT broker is “noting” that the payload includes “not a number” (aka NaN)
I do this for a number of esp32 devices. A local Mosquitto broker receives all the data from the devices and then I have a separate SignalK plugin that ingests the various topics into SignalK.
I made it work by creating fully formed SignalK deltas that become the MQTT payload. The MQTT topic is just the SignalK path with '/' instead of '.'.
All my wee devices uses the same system for formatting and sending.
Curious why you chose the mqtt pattern over direct websocket connection. Are there other subscribers to the same message queue?
Two reasons. The devices fall back to a remote broker if the local broker/signalk server is down or off (assuming there is connectivity.) And I found a local broker bridged to a remote broker to be a simple way to mirror the local(boat) signal k data without having to manage a proxy.
So you have some remote subscriber for the messages for remote monitoring via some other consumer? Seems like a good application for streaming
What I have is a neurosis. I want to know what's going on my boat all the time. I have a virtual boat signalk system in my living room that subscribes to the remote broker(AWS) and mirrors whatever data is coming from the boat. Similarly, I have signalk running on the AWS instance. It also gives me some ability to turn things on and off remotely.
I suspect there are other more graceful ways of doing this but MQTT/Mosquitto is so reliable and simple I have never felt the need to look at other methods.
I could imagine it being a decent solution if I were managing/watching a small fleet of boats.
Have you outputted the payload using a print statement and then put it into the data fiddler?
not sure i understand what your suggesting?
in my esp32 i issue a print statement (locally) to check/confirm data is being sent (for local debugging)
.. the problem im getting is that the MQTT broker (from sk MQTT plugin) is generating a NaN entry (which prevents it from then updating the text string into sk path)
you should be able to output the payload that you are sending to the broker to the terminal. Take that string and see what is looks like.
@Tony
your mention of web socket got me thinking .. and checking the signalk API’s.
web socket method seems overly difficult for what im trying to achieve …
I have an ESP32 device that needs to publsh a few deltas , at around 5 sec update intervals ..
… I saw the mention of “Put requests” … i dont have any experience with this method, and not sure if its an efficient way to
“stream” deltas into signalk? (a few deltas every 5 seconds)
any thoughts or direction woulkd be appreciated
Put requests bypass the SensESP library. There are repeat classes that facilitate the time interval for sending deltas via websocket
Im not using sensESP environment on my ESP device.
my ESP module is running some modbus code to read a Cummins/Onan generator modbus registers, ststus et al
it displays the generator operating parameters on a local (inbuilt) display …
and as an additional reqt, Im now trying to send a few of the parameters (eg coolant temp) to signalk (so i can generate notifications/alarms etc)
… so my programming environment is micropython (im not a very experienced coder!)
.. and trying to find the “easiest” way to get a few of the parameters updating signalk paths
Is the esp already connected to the network? You can just send the delta over http via PUT /signalk/v1/api/vessels/self/{path}
yes connected to same network,
& yes I have generated a token.
however i keep getting a 405 error when i try the PUT.
and a lot of reading suggested that maybe this was not the preffered or correct method to update paths in server
No, you can not just send deltas via PUT without a PUT handler, but u can send deltas over ws
thanks, and yes, thats where i most recently “landed”.
… its a learning experience for me …
im now at stage of getting my head around ws authentication ..
How difficult would it be to port your sensor code to sensesp cpp from python? I think this is where you are having to do a lot more setup with your device regarding networking, sk auth, etc. this is all handled via sensesp. Is it because you don’t have the same libraries for your mod bus translation?
My device is a ESP32 C6 with integrated 1.4” display, i have very limited space to mount it on the existing panel beside generator controls… hence the very small footprint device i selected.
Primary purpose of device is to read the generator (cummins onan) RS485 Modbus messages and display “locally” oil pressure and coolant temp, as well as status & error messages.
In addition - i wanted to send the “data” into Sk, as i can then generate notifications, alarms etc .. as well as view the data on my various displays around the boat.
I have a bit of upython experience, also sensESP, however primary purpose was to read local modbus and display on ESP32 touchscreen ..
my experience with sensESP has been checkered… I have a number of V1 & V2 sensESP devices working ok on boat,
but an attempt to update to V3 , met with frustration and no success.
I wasnt confident in trying to “merge” my touchscreen operation, and modbus etc into the sensESP framework.
Also upython is fast & easy to develop and debug the displays and touch UI… vs a compiled environment/framework.
First attempt : MQTT - i have a lot of familiarity with MQTT, and got the device working and sening numeric data to the inbuilt MQTT broker/plugin , with it populating SK paths just fine, but as soon as i tried to send text strings, i get the failure described above in original post… the inbuilt MQTT broker (“aedes”?) doesnt appear to like non numeric data, and generates a NaN message, which prevents the plugin from populating the SK paths.
Ive tested with a generic mosquitto MQTT broker - and it works just fine .. aka no NaN message handler issue.
so unclear why, but the inbuilt broker is behaving “differently” and preventing non numeric data from flowing thru to sk path.
Second Attempt - tried using PUT, got all the coding done, with token, and was succesfully sending PUT data, but ran into the “no PUT handler” issue.
Third attempt: started down the websocket path …. but having a bunch of challenges with appropriate libraries etc
to get upython running with SSL/TLS etc websocket authentication
Have done massive googling, chatGPT et al … but examples are few and far between…
(obviously the compiled eg arduino IDE framework is more wiedley supported)
.. so thats what im trying to achieve, and my journey thus far!
One trick that would make this easy is to push numbers since you know that works, then use node red to convert the value to text. It’s a hack but it will work
Yes, already thought of that … but the problem is that aside from some obvious “status” messages that I can simulate on the generator … eg stopped, running etc>
it generates faults codes and various other modbus messages - NONE of which are published..
indeed i cant find any useful referernces to their implmentation of modbus … so its been a “suck and see”
to read modbus registers and figure out the meaning of each …
but without being able to simulate “all” the possible generator fault conditions … I will never know what the actual text messages are …
… and Im not so keen to “force” fault conditions..
eg a low oil pressure situation!
just to see the relevant fault code!
Iknow there are likely more than 20++ fault code text string generated … but dont know the “exact” format of each
Update:
given further investigation, I can now see that signalk-mqtt-gw has limited configurability… ..whcih has also previously created additional overhead for me (with selected mqtt devices,) to use node-red flows to parse mqtt msgs into signalk …
so im going to move to a generic moscquitto broker and one of the signalk MQTT client plugins.