Crowdsourcing depth for openseamap
41 Replies
Oh wow, I had no idea that project exists. My main motivation for getting into SignalK a couple months ago was so I could start tracking my own depths and maybe do something like this eventually.
Looks like at least one other person has tried to make a SignalK plugin https://github.com/OpenSeaMap/depth_webfrontend/issues/230
Probably not a good sign that this is the second result when I search "openseamap depth" https://www.reddit.com/r/openstreetmap/comments/116k1ue/openseamap_depth_data_dead_project/
It"s dead because it's not share enough. If we can have a simple plugin / opt-in in signalk, all people that are using signalk will share datas automatically.
GarminQuickDraw vs OpenSeaMap Depth
This has already been discussed. It is not useful to collect depth data from people, as there is no way to determine the reference point from which the depth was measured.
That's not actually true. Navionics does this with their sonar charts and it's really good. Given shared reference points, you can determine the offset of an individual sensor. You just need enough data
@edouard_k are you thinking about building a plugin?
You just need enough dataHow much?
It depends, but the more data you have, the more accurate you can get. The perfect case would be you have a reading from two sensors from the exact same position at the exact same time. Obviously this is not possible. But given enough readings from enough reference points, you can calculate offsets of a sensor. Tides also make this interesting. It's definitely a hard problem, but it's not impossible.
Don't know how to do it, but i can try
Navionics Sonar vs GarminQuickDraw
I'm afraid you don't fully understand the problem.
I did a little more research on this.
The openseamaps project seems to be dead. There has been little to no activity in mailing lists or code repositories for the last several years.
But the good news is, there is some really interesting progress in this space happening elsewhere.
The most promising is from the International Hydrographic Organization (IHO), which has a crowdsourced bathymetry project. From https://iho.int/en/crowdsourced-bathymetry:
Crowdsourced bathymetry (CSB) is the collection and sharing of depth measurements from vessels, using standard navigation instruments, while engaged in routine maritime operations.These slides from a Workshop on crowd-sourced bathymetry are very interesting and worth a skim. https://iho.int/uploads/user/Inter-Regional%20Coordination/CSBWG/CSBWG_IRCC_CSB_Workshop/IRCC_CSB_workshop_April24_Master.pdf I have reached out a good friend who is a professional surveyor to get his thoughts on all this. At a minium, I think it would be great to have a Signal K plugin that contributes to one of these efforts. IHO has a process for becoming a "trusted node" that contributes data, so i'll inquire about what all that involves. It would also be nice to then have some tooling for using and consuming this data in freeboard/opencpn/etc.
I would at least like to be able to overly my own data like I can in navionics. The yellow is my own depth soundings, showing where I ran agound a few days ago.
Here is IHO's guidance on collecting crowd-source bathymetry https://iho.int/uploads/user/pubs/bathy/B_12_CSB-Guidance_Document-Edition_3.0.0_Final.pdf
That doc is good. I love this:
"Data Quality" relates to the extent to which a data set is fit for its intended purpose. Good data quality does not necessarily mean that the quality of the data has to be good rather it means that the end user is well informed regarding how good the quality of the data is. The best available data for an area may not be the best possible quality data according to standards, but as long as the quality of this data is quantified or qualified, it offers the best available data if nothing else exist.
Could you not just save the data to influxdb & send that with lat/long & anything else on interest? I end up with masses of data after an ocean passage, uploading it for an Ai to look at can be interesting. Would be nice to give it to someone academic if it was any use to them. Ocean temperatures might be of use. 🤔
I reached out to IHO to inquire about becoming a trusted publisher. 🤞
In the mean time, time to start coding 😆 https://github.com/bkeepers/signalk-bathymetry
Yep, here's a sample of the data they want: https://github.com/bkeepers/signalk-bathymetry/blob/efe9cca3debf20850ef7f69554bbee56398e1687/docs/geo-sample-suggested.json?short_path=5873305
Not too bad then, something an Ai could write python to do from signalk & influxdb in no time. more here . https://iho.int/uploads/user/pubs/bathy/B_12_CSB-Guidance_Document-Edition_3.0.0_Final.pdf
To submit Crowdsourced Bathymetry (CSB) data to the International Hydrographic Organization (IHO), the expected format and process are as follows:
Data Format
The IHO Data Centre for Digital Bathymetry (DCDB) primarily accepts data in GeoJSON and/or CSV formats for bathymetric soundings.
Data should be accompanied by mandatory metadata, such as vessel identifier, sensor configuration, and data license (which must be Creative Commons Zero, CC0 1.0).
All measurements (e.g., depth, offsets) should be in SI units (meters), with speed in knots allowed.
Tidal corrections should not be applied to the soundings before submission.
Submission Process
Data is typically submitted via a Trusted Node (an organization or individual acting as a liaison), but individuals can also contact the DCDB directly to arrange submission.
Submissions use standard network protocols such as FTP or HTTP POST, authenticated with a unique token provided by the DCDB.
Example HTTP POST submission (with GeoJSON file):
bash
curl 'https://www.ngdc.noaa.gov/ingest-external/upload/csb/test/geojson' -i -X POST \
-H 'x-auth-token: <valid token>' \
-H 'Content-Type: multipart/form-data' \
-F 'file=@<geoJson.json>;type=multipart/form-data' \
-F 'metadataInput={
"uniqueID": "<uniqueID>"
}'
Next Steps
Review the IHO B-12 Guidance on Crowdsourced Bathymetry for detailed metadata requirements and sample data formats.
Contact the DCDB at bathydata@iho.int to express your interest and receive guidance tailored to your setup.
Looks interest, would be nice to do something useful out there -😎
I would say the OpenSeaMap Depth project is somewhat "dormant". It could probably be reactivated with
- volunteers
- promotion
- software to collect data and upload (-> SK plugin).
If anyone is interested, I can put you in contact with the people there.
@Copprhead yes please!
You can use my TrackViewer plugin to export your past tracks (InfluxDB required) with GPS coords and depth info and upload/send to OpenSeaMap.
I'm working on a plugin to send data to the IHO, it would not be hard to add support for multiple destinations
@Copprhead Does your TrackViewer plugin normalize gps antenna position vs sounder position?
Just read the webpage and it looks like GPX is not a supported format... 😦 Somehow I thought it was. So forget that... (And no, TrackViewer doesn't normalize, it takes the data as it is saved in InfluxDB)
Ok, no worries.
CSB look like just a money grab. "Попил" in Russian, sorry for my English.
You are quite the ray of sunshine over there. First you say that I don't understand the problem when I say that crowdsourcing depth data is technically feasible, and now legit internationally-coordinated efforts to do it are money grabs? Anything constructive you would like to add to the conversation?
hmm genuinely curious which part is a money grab. I read through the thing and here's what stood out:
- Data can be accessed through the Data Centre for Digital Bathymetry (DCDB) Map Viewer
- Data is contributed under a Creative Commons Zero (CC0 1.0)
- Even the S3 bucket is courtesy of NOAA, so they're not even asking money for that...
Anyone else getting a dead link?
https://github.com/bkeepers/signalk-bathymetry/blob/main/docs/geo-sample-suggested.json?short_path=5873305
Yeah, sorry, I removed it because I'm actually going to use their more concise format, but here's the original…
https://github.com/bkeepers/signalk-bathymetry/blob/efe9cca3debf20850ef7f69554bbee56398e1687/docs/geo-sample-suggested.json?short_path=5873305
There are no real specifications:
to data
to verification procedure.
Criteria for selecting trusted data sources are not specified too.
Nothing specific at all, just general words.
I gave two maps of the same area kind of based on "crowdsourced" data. Do you understand why there is nothing in common between them?
Yes, I understand the one spot you picked does not have good data on Navionics. I don't know where that is, so i can't speak to how much data is available in that area.
I've been cruising full time on the US east coast for 6 years and use sonar charts every day. Where I sail there are a lot of Navionics users and the data is surprisingly good most of the time.
I am excited about the potential for open crowd-sourced data and am putting in effort to build a signalk plugin to contribute to that data. If you have something constructive to add to that effort, I welcome your insight.
https://iho.int/uploads/user/pubs/bathy/B_12_CSB-Guidance_Document-Edition_3.0.0_Final.pdf is quite big doc to wade through, you could try https://notebooklm.google.com/ and paste the doc address as a web link then get it to make a podcast out of the document, 2 people talking about it. Takes a few minutes, Star Trek is here 😎😂
Well, compare the NavionicsSonar and GarminQuickDraw data for a place you know. Try to understand why there is nothing in common between them.
I don't know where that isThere's a lighthouse in the upper left corner.
😆 I'd like to hear that
That doc is long, but it's good! I'm impressed at how thorough they're being. They have definitely put a lot of work into it.
Status update:
The plugin works and is currently capturing position/depth/time in a CSV (the xyz format that can be reported to IHO).
And I have the config done for all of the metadata that they want reported.
I contacted IHO to figure out the process for becoming a trusted published, but haven't heard back yet.
It sounds like part of the application process is to send them sample data and they run tests against it. So I'm debating about building a small aggregation service that this plugin can report to in the mean time so I can get people using it and have some data to test against.
Depth data from my sail today converted to mbtiles overlay. Hoping to automate the ability to view my own past depth data on my charts similar to how Navionics live sonar charts currently work 🤞
Own depth data with SignalK:
1) Collect the data using naiveGPXlogger
Very cool. I haven't used GaladrielMap, but will check it out.
If anyone is interested in trying it out or helping, here's where I'm working from: https://github.com/bkeepers/signalk-bathymetry/pull/1
cool! please consider how this could work in a way so that each plugin is not using just its own data
for example I have multiple years' data including track & depth in influxdb. if these plugins would work like TrackViewer by @Copprhead, using the provisional History API, the plugins would be immediately usable for all users of the InfluxDb2 plugin
GitHub
GitHub - tkurki/signalk-to-influxdb2
Contribute to tkurki/signalk-to-influxdb2 development by creating an account on GitHub.
@Teppo Kurki yes, will do! I've been using the influxdb plugin since I started using Signal K in January, so I'm looking forward to being able to extract all of that data.
Now if only I hade my previous 6 years of data that I gave to Garmin/Navionics 😭
I would love an intro to anyone at OpenSeaMaps if you have a contact.
It's not pretty yet, but I now have a depth chart for my entire history of using Signal K, with the data from the influxdb plugin, and the charts are available via the charts provider plugin. Here it is in Freeboard. Now I need to learn how to style mbtiles so it's not just dark polygons.
