night199uk - @Scott Bender hey, I remember a wh...
@Scott Bender hey, I remember a while ago working on the string support for raymarine in canboat/canboatjs. been doing far too much boatwork and finally got back to futzing with signalk, now with a new and fully upgraded axiom 2 pro.
I notice that raymarine can't display the SignalK/canboat strings properly still. didn't dig into it yet but though I'd highlight it before I started digging in.
57 Replies
looks like a variation of the null termination-vs-fixed-length-strings stuff.
this is using signalk-aisstream to generate AIS data into my raymarine via signalk.
first check that the data is good that signalk-aisstream produces? as in the vessel names have proper length? @Karl-Erik Gustafsson
@night199uk Can you try if this is removing extra characters?
npm install git+https://github.com/KEGustafsson/signalk-aisstream.git#nameSure, let me test today.
I need to just dump the actual can traffic as well, just didn’t get time yesterday.
okay, this is not to do with aisstream
i see this on any AIS data originated from SignalK via the SignalK to NMEA2000 plugin on my Axiom 2 Pro.
weird. dumping the data from my raymarine AIS-700 i see that the raymarines leave junk at the end of the STRING_FIXED fields. what's weird is they don't seem to be null terminating now (like they were on the older generation e126 i replaced).
here's two dumped AIS frames from the AIS-700, one has the STRING_FIXED fields padded with 0x20 (space), one has it padded with 0x40. weird. i don't see a null terminator in here, so I wonder how the Axiom is discerning the end-of-string and removing the padding?
oh, i guess in the SPIRIT OF AMERICA case they filled up the text box on their side with spaces when configuring their AIS. the 0x40 padding seems to be the common case.
comparing this to what signalk/canboat sends: i see canboat/signalk consistently pads unused string_fix fields with 0xff.
as a test, i wonder if i can build a custom canboat to pad with 0x40 and see if that makes raymarine happy. that would be a yucky fix though.
ahhh. I remember. from canboat/print.c:
so it seems like lighthouse 4 on the raymarine axiom 2 pro expects/strips 0x40 ('@') instead of 0xff in AIS strings and doesn't consider 0xff to be a proper terminator?
same issue:
https://forum.openmarine.net/showthread.php?tid=4681
@Scott Bender fyi:
https://github.com/canboat/canboatjs/pull/373
This fixes the issues with Raymarine for me with a surgical fix to change the fill character from 0xff to 0x40 only for the NMEA AIS message types.
I have no idea if this is the "right" fix; I guess without access to the spec no-one really knows.
But this does work for my Raymarine Axiom's and gets rid of all the junk.
I don't have any access to B&G to test. but the implication of:
https://forum.openmarine.net/showthread.php?tid=4681
Is that SIMRAD AIS units also use 0x40 as the pad character.
since a ton of AIS units like Raymarine are all based on SRT technologies chipsets & firmware, I'd guess if the Raymarine AIS-700 does it a large number of the AIS units do it that way...
SRT and non-SRT clones.
https://continuouswave.com/whaler/reference/AISB.html
Would be interesting to dump a Vesper and see what it uses to pad the AIS strings.
I have em-trak B951 and haven't seen any name padding issues with SK or commercial chart plotters...
Do you have any candumps of AIS messages from the em-trak? That’s also an SRT clone I believe. I would imagine they are padding with 0x40; assuming the similar hardware all has some similar “base” firmware.
I have nmea0183 interface in use. Also I don't have n2k receiver to collect the other output, at least now.
Interesting. Here are two more datapoints:
- PredictWind RDS Datahub: no terminator, padded with 0x40.
- Garmin CORTEX: 0x00 terminator, padded with 0xff.
I haven't really thought of what to send to NMEA 2000 but my feeling is that what CORTEX does is probably the best solution. Certainly if that fixes the Raymarine issue. I don't like making a special case for AIS messages which happen to originate from 6 bit code, so 0x00 in 6 bit maps to 0x40 in 8 bit which is '@' which is the AIS way to encode end of string.
I think it should be illegal to put @ in AIS vessel names.
@Kees Verruijt interesting! i'm not sure that the Garmin Cortex is the right way to do it though?
the raymarine Axiom's only accept 0x40 as the end of string/pad character for AIS messages. 0x00 and 0xff show up on the screen as junk characters so AIS messages padded with those look horrible.
your findings imply that if you had a garmin cortex AIS with a raymarine Axiom MFD, you'd see similar junk on the raymarine axiom to what we see today.
i didn't understand your point about 6-bit 0x00 and 0x40? but i'm not familiar with nmea-2000 history, i guess this implies there was some older 6-bit stuff somewhere?
though i have a lot of respect for the garmin (nee: Vesper) stuff. it is high quality stuff, but i don't know about their software.
let me run some tests soon with 0x00 terminators, 0xff, and 0x40 characters.
I believe my em-track does no terminator, padded with 0x40
its also possible the axiom's have some special case code that detects garmin AIS units and displays it correctly, i guess. thats the kind of thing a prop vendor can do.
so turns out em-track: no terminator, sometimes pads with 0x20 , sometimes 0x20 . No obious pattern there. Works fine with Rayamarine Axiom
sometimes 0x20 sometimes 0x40?
i'm guessing the 0x20 is if the owner has padded the text box on their sending side with spaces, right? that's my guess at least. i think they're often configured by non-technical folks who probably just fill up the box with space.
Could be
the one i need to test on axiom is 0x00 followed by 0xff. if that displays fine that would fix the cortex case.
currently just 0xff without the 0x00 null terminator causes issues at least on my axiom.
@Scott Bender i know you have an axiom display as well, are you not seeing the same issue with SignalK sourced AIS objects?
or are you not doing signalk sourced AIS yet
I don’t have Raymarine anymore
ahhh. good choice 🙂
Put all B&G/Simrad on the new boat
wish i had done the same. this raymarine generation is junk.
or that the sending device pads it, no matter how short a name u enter
I just tested my B&G Zeus SR and it doesn't care how we terminate: 0x00 x n, 0x40 x n, 0x00 followed by 0xff x n - 1, 0x00 followed by 0x40 x n - 1.
Unfortunately, I have to agree with you. I have been testing my radar packaga (mayara) on a friends new Axiom + Quantum and other than basic chart usage I am constantly thinking what a piece of junk. Lighthouse 4 may have been okay in the beginning but it is already showing heavy signs of neglect and "we dont care" attitude, with messy setup menus. Also radar is way below HALO.
Let me elaborate on why only AIS uses 0x40, I think I may have skipped a few steps making it unclear.
AIS is sent over the air, in a binary protocol. The easiest way to see this is the NMEA 0183 representation, which is basically "here are the bits straight from the radio" (with some breaking up to make the messages fit 0183 structure). As you can imagine, bits are scarce at 38400 baud. So they use only 6 bits for text, using a mapping from ASCII to the 6 bits:
32 -> 63 => 32 -> 63, 64 -> 95 => 0 -> 31. Now in the AIS standard (which can be obtained) it clearly states that strings are padded with all-zero-bits. When recoding all zero in AIS back to ASCII, that maps back to 64 == '@'.
Now of course, in "normal" NMEA2000 there is never any 6 bit involved. I just checked and PGN 126996 (Product Information) on my devices show the following:
- padded with 0x20 x n (spaces)
- padded with 0xff x n (unused n2k value)
- padded with 0x00 x n (C end of string)
- padded with 0x00 + 0xff x n - 1
with some devices even using two different paddings in the same sentence!!!!!
so i guess the question is which of these formats is most widely supported / correct?
currently it seems that raymarine doesn't support what we use now, which is 0xff x n
so it would seem like the better options would be:
- padded with 0x00 x n
- padded with 0x00 + 0xff x n -1 (what we see from Vesper/Garmin Cortex AIS)
- padded with 0x40 (what we see from SRT based AIS'
- padded with 0x20 (feels weird?)
i can update this PR to whatever I guess
https://github.com/canboat/canboatjs/pull/373
Seems like we should just make it an option
i'm inclined to think we don't need an option - maybe?
i think if i look at the venn diagram of what we've seen, either 0x40 or 0x00 + 0xff seems to be supported everywhere? i need to test the latter one though.
From my point of view, we should keep it the way it is since no one else is complaining.
i mean, there is at least one other forum report that i mentioned in the pr
But with Raymarine, right?
sure, but it would suck to just abandon raymarine, right?
even if it is shit 🙂
let me try and update this to use 0x00 + 0xff and see if this solves it?
https://github.com/canboat/canboatjs/pull/373
Agreed. I just don’t know if we change it, will it break for many other devices?
I don’t like the PR because it is hardcoded for specific pgns.
well, if i update that PR it seems like we have a mix of folks with B&G and Garmin? Seems like supporting B&G, Garmin, and Raymarine should get you the best coverage. If Lowrance/whoever else do something odd I can't believe they wouldn't fall into line with the "big 3"
yes, that was kind of hacky. i'd like to remove the pgn special casing. if we found the "correct" terminator i'd think we want to update all PGNs?
i just did the pgn limit for just the ais stuff for testing. i was trying to be clear in the PR it was a WIP / proposal 🙂
let me try tomorrow. i've been tied up trying to build some halmet's for engine monitoring, so haven't go back to the JS side of things for a while.
My preference would be to create a new string type used for AIS pgns. And then an option to specify the encoding that should be used.
I think this was @Kees Verruijt issue also. But canboat does not do encoding, so it doesn’t really affect you that much?
@Kees Verruijt question for you: on the PredictWind RDS DataHub, do they only pad with 0x40 for AIS messages? what do you see in 126996 for padding on the PredictWind?
PredictWind gets its data over the 'nets. that means its data is not AIS sourced: so the 0x40 padding choice must have been a purposeful encoding choice on their part; and not a hangover from 6-bit conversion, like the AIS units.
I don’t know about PredictWind, but MarrineTraffic and AisHub get their data from the net also, but does get sent AIS encoded.
to be clear, i'm supportive of having a configuration choice and adding more string_fixed; but it just feels like there might be a way to avoid adding yet more config options if we can find something universally supported.
so i'm just trying to advocate for taking the harder road to find a cleaner solution without just bumping it back to the user.
I totally agree. I would much rather find a universal way
let me do some testing tomorrow.
But there many, many products and vendors out there.
i feel like using 0x00 + 0xff across the board seems to be the sweet spot.
All I really have to go off of is bug reports.
100%, but i feel like if we find something that works for the "big 3" then the other vendors are naturally going to fall into line anyway. like, if em-trak AIS' didn't work with B&G, Raymarine and Garmin, em-trak would be forced to do work to fix it as "most" of their customers would want it. so i feel like most of the market probably does/should fall into line with what the big 3 do.
could be wrong with that though 🙂
They would all have the same issue we have. Making changes risks breaking things for lots of people.
And I am thinking not just about the big 3. But all the homegrown stuff out there, all the n2k to 0183 devices out there from many vendors. (YD, Digital Yacht, Shipmodule, etc.)
Because I have no data on usage. I have to assume that changing the encoding could break a ton of stuff.
sure. but what i meant was that if we are doing the same as garmin, raymarine... vendors like YD, Digital Yacht, etc should have support.
e.g. 0x00 + 0xff is what a Vesper Cortex sends. if digital yacht didn't work with 0x00 + 0xff, they'd have a bunch of vesper cortex users yelling at them. so long as we stay to one of the "known" mechanisms we see being done by SRT (raymarine, em-trak, b&g, etc) or Garmin/Vesper then you can make some perhaps reasonable or unreasonable assumptions that YD/DY/Shipmodule etc already support those.
I totally get it. And totally logical!
there are really three things:
1) 0xff - this is what we do today
2) 0x40 - this is what we see from all SRT firmware based devices we have samples of
3) 0x00 + 0xff - this is what we see from Vesper Cortex devices
so assuming we're working within these 3 known samples, i believe the change risk should be minimal
Agreed. And I like choosing #2.
but the question is then - just for AIS PGNs, or also for all other string_fixed fields? that was hence my question about how PredictWind DataHub encodes NON-AIS PGNs
if PredictWind datahub encodes 126996 (Product Information) with 0x40, then I think #2 is a slam-dunk.
I say only AIS The risk is way to high otherwise.
but if PredictWind datahub encodes 126996 with 0x00+0xff, and AIS PGNs with 0x40, then I might advocate for something different
I am also happy to do the work to make it an option.
I am definitely not changing the default for other pgns.
i mean, i'll never turn down getting the fix for free, saves me updating the PR 🤣 but i'm still an advocate of finding a way without adding more knobs for users that htey won't understand
Agreed. And the option will not presented to users unless there is a great need. But at least then I can give people a quick fix.
Let’s wait for @Kees Verruijt input on creating a new string type?
sounds good
btw, apologies for drawing this out, i'm new here so trying to build consensus and land useful PRs + understand the mindset
No worries!
This is part of why I love doing this.
Can’t do this right without people like you pushing.
I appreciate it
Morning guys, although I was originally against making a different string type I think I can live with a Canboat STRING_AIS that documents that end padding with 0x40 is to be expected.
I will add a sample concentrated file with all strings that I can get my hands on.