Sometimes KlipperScreen / Mainsail wont start, SSH is possible

So every so often, I turn on the power of the printer (and pi, which is powered with a dedicated 5V 3A meanwell PSU going to the USB-c port) My klipperscreen shows the usual pi bootup console, then it turns black and stays like that. Usually KlipperScreen comes online after a couple seconds and the Mainsail is available. However sometimes the screen stays black with the console cursor in the corner, and no connection to mainsail ist possible. A hard reset fixes is sometimes, sometimes I have to cut power several times until it boots successfully. I did notice tho that I can ssh into the pi, to the problem is likely klipper related. Havent found anything in the logs tho (mainly cause I dont know what to look for). On time I issued a reboot command via ssh, which didn't help and the pi wasnt reachable at all afterwards? Any suggestion on what to do for troubleshooting (via SSH) once this happens again?
37 Replies
blacksmithforlife
blacksmithforlife•6mo ago
have you gone through https://klipperscreen.readthedocs.io/en/latest/Troubleshooting/ ?
I did notice tho that I can ssh into the pi, to the problem is likely klipper related
Not klipper, klipperscreen
On time I issued a reboot command via ssh, which didn't help and the pi wasnt reachable at all afterwards?
this is completely unrelated. You probably didn't send the correct command and it just shutdown instead of restarting
Oskait
Oskait•6mo ago
Weird thing is every time this happens, I also can't access the webgui. only ssh into the pi works. Thats why I think it is Klipper / mainsail related and not just klipperscreen. But I'll have a look at the troubleshooting guide, thanks. I used sudo reboot now which should be the same as sudo shutdown -r now if I'm not mistaken.
miklschmidt
miklschmidt•6mo ago
I'd try a clean install on a new sd card
Oskait
Oskait•6mo ago
Was hoping I could avoid that.... I do have sonar installed because I sometimes had disconnection issues, maybe that is causing the startup problems. It hasn't happened in the last few days, so maybe it got fixed by some update. But yea If the issue persists I'll try to uninstall sonar or even a fresh install
Oskait
Oskait•6mo ago
Well speaking of the devil, it just happened again. After not finding anything obvious in the logs I issued sudo shutdown -r now, ssh closed but the screen didn't change and nothing happened for 10 minutes. So likely the pi is bonked. Rebooting it via power cycling but still cant access anything other than the pi via ssh. Checking the running services confirmed that RatOS is indeed not running for some reason (as indicated by the [ - ]) I'm backing up everything now and will try a fresh install on a new sd card.
No description
Oskait
Oskait•6mo ago
Well did a fresh install on a new SD card, had everything set up and working fine. Didn't install sonar again. This morning, same issue again. The weird thing is that my controller fans do stop spinning after the while, just like during a normal boot. When i disconnect the octopus board they don't stop, so some part of ratos/klipper does work I guess
miklschmidt
miklschmidt•6mo ago
The RatOS service is not a persistent process, it's just a script. (it's supposed to be stopped) Sonar is by choice not included in RatOS as it conflicts with the hotspot So, just to clear it up. A vanilla image works? What modifications are you doing besides Sonar? Could you post your moonraker.log ? maybe check syslog
Oskait
Oskait•6mo ago
Yea I figured that out as the service status was the same when everything is working. Nope same issue on a fresh install on a different (new) sd card. So not sonar related. No other modifications despite the printer.cfg
miklschmidt
miklschmidt•6mo ago
That's indeed very strange
Oskait
Oskait•6mo ago
Heres the moonraker log from yesterday. I think the first entry might be the "failed" boot. I couldn't find any issues despite no connections being logged by moonraker. The second entry is probably after the power cycle (unsafe shutdown incresed from 2 to 3) where it worked and it did log connections
Oskait
Oskait•6mo ago
yep. I might try switching to a normal usb-c power supply. maybe its somehow power related but its super weird that the controller fans stop, ssh is working, but mainsail and klipper screen do not at all
miklschmidt
miklschmidt•6mo ago
2023-12-26 09:59:02,444 [proc_stats.py:run()] - VCIO vcgencmd failed
This is odd
Oskait
Oskait•6mo ago
oh good find. Any idea what it means?
miklschmidt
miklschmidt•6mo ago
Try:
vcgencmd measure_volts
vcgencmd get_throttled
vcgencmd measure_volts
vcgencmd get_throttled
Oskait
Oskait•6mo ago
No description
miklschmidt
miklschmidt•6mo ago
That's a relatively low core voltage And... there's no ram voltage reported 🤔 Oh never mind, cpu scales from 0.8 to 1.4 dynamically with load. so 0.936 is to be expected nothing out of the ordinary there.. vcgencmd seems to work fine maybe it doesn't when it boots in borked mode Yeah it seems to work on the next moonraker start:
2023-12-26 02 [proc_stats.py:init()] - Detected 'vcgencmd', throttle checking enabled
Oskait
Oskait•6mo ago
possible. Will definitely try this when its borked up again. Yep the VCIO vcgencmd failed only showed up once (where it failed)
miklschmidt
miklschmidt•6mo ago
that does indicate there's some kind of hardware wonkyness with the pi.
Oskait
Oskait•6mo ago
2023-12-26 09:59:00,252 [proc_stats.py:init()] - Detected 'vcgencmd', throttle checking enabled was also in the first borked boot. it failed later 2023-12-26 09:59:02,444 [proc_stats.py:run()] - VCIO vcgencmd failed Didnt fail during the other starts tho well thats not what I wanted to hear. Bought this pi a month ago for the printer...
miklschmidt
miklschmidt•6mo ago
You may want to try this: https://github.com/alwye/get_throttled/blob/master/get_throttled.sh The output of get_throttled is just a hex string where different values mean different things you'd have to look up. (this script fixes that)
GitHub
get_throttled/get_throttled.sh at master · alwye/get_throttled
Human-readable vcgencmd get_throttled. Contribute to alwye/get_throttled development by creating an account on GitHub.
miklschmidt
miklschmidt•6mo ago
Hmm.. Well that doesn't make much sense. Have you tried disconnecting the screen for a while? What's the output of
grep -a -e "hwmon" /var/log/syslog
grep -a -e "hwmon" /var/log/syslog
?
Oskait
Oskait•6mo ago
Not yet. Kinda hard to test as it happens randomly. But I'll try that for a couple of days and see if it fixes
Oskait
Oskait•6mo ago
nothing...
No description
miklschmidt
miklschmidt•6mo ago
Huh... So doesn't seem to be a power issue at least
Oskait
Oskait•6mo ago
looks good as well
No description
miklschmidt
miklschmidt•6mo ago
run that script next time it borks Maybe it'll tell us nothing, but i'm hoping there's something 😂
Oskait
Oskait•6mo ago
yea would be nice to know what causes the issue
Oskait
Oskait•6mo ago
not the only weird issue im fighting atm. My bed thermistor is wonky as well (in orange) But I'll open a fix-my-printer about this tmrw once i double check with a second thermistor
No description
Oskait
Oskait•6mo ago
Thanks a lot for your help, now I at least have something to try
miklschmidt
miklschmidt•6mo ago
Sure thing!
Oskait
Oskait•6mo ago
well... it hasn't happened again. During troubleshooting of the thermistor I noticed that I my power plug was plugged in reversed, so AC Neutral / Live were switched. After reversing it every boot has been successful for now (and the bed thermistor is happy again). I thought the 5V meanwell PSU wouldn't mind but apparently that fixed it
miklschmidt
miklschmidt•6mo ago
Oh dang i completely forget this is a thing as we use non-reversible earthed plugs here.
miklschmidt
miklschmidt•6mo ago
Look how happy they are
No description
Oskait
Oskait•6mo ago
no problem. I would have expected that it does not work at all with N/L switched or work every time, since its AC and reversing polarity often doesnt matter. Especially with a DC powersupply in the middle. Or at least the boot to fail every time and not randomly. Anyways, its fixed 🙂
miklschmidt
miklschmidt•6mo ago
AC is confusing, L/N matters, polarity is switched on L. 😄
Oskait
Oskait•6mo ago
Ah yes thats what I meant. Still never payed attention which way a plug goes in, and never had any similar issues with my PCs, Pis, Arduinos or ESPs
miklschmidt
miklschmidt•6mo ago
yeah most consumer electronics don't care