HThanks but I cannot take the credit. Credit should go to you, Propeler3D, and Washout! I just hacked my way through borrowing everyone's wisdom along...
H
H@walmis Hi, I'm still trying to debug the DCS freeze issue even though that I have improved all of the ground and power supply connections. The USB controller is now directly connected to a 24V lab power supply and the freeze issue still persists. I use a program called Wireshark to capture the USB traffic in order to understand the underline activities that cause DCS to freeze. I notice on the captured data that at some point the URB interrupt from the host (i.e. DCS) is not acknowledged by the USB controller. Maybe either something times out or prolonged waiting on the ACK causes DCS to hang. Funny enough, unplugging the USB cable to the USB controller after freezing forces the corresponding ACK's to be finally sent so these ACK's have been in the buffer all along and just didn't get sent timely. Have you seen anything like this before? Please refer to the screenshots and captured data for detail. The relevant data packets are:
USB Host to Stick URB_INTERRUPT
64992 46.373221 IRP ID:0xffffd782f98bc010
66728 47.427333 IRP ID:0xffffd782fe1196e0
USB Stick to Host URB_INTERRUPT
78621 61.816345 IRP ID:0xffffd782f98bc010
78622 61.816350 IRP ID:0xffffd782fe1196e0
Any suggestion is welcome!
USB Host to Stick URB_INTERRUPT
64992 46.373221 IRP ID:0xffffd782f98bc010
66728 47.427333 IRP ID:0xffffd782fe1196e0
USB Stick to Host URB_INTERRUPT
78621 61.816345 IRP ID:0xffffd782f98bc010
78622 61.816350 IRP ID:0xffffd782fe1196e0
Any suggestion is welcome!
WAh Sh!t these USB issues are hard to debug. And yours is the first such case to my knowledge. Did you try running it through a usb hub? Also could you check the FFB board revision number for me?
HI'm sorry to ruin your day. I'm like a magnet that seems to attract outliers. Anyway, I did try using a USB hub earlier as you suggested and observed pretty much the same freeze symptom. The revision of my Rhino FFB board is Rev. 1.2, dated 2022.06.02.
WThanks, I'll do some stress testing by flooding that endpoint with data.
WHopefully will be able to reproduce it
H
H@walmis Thank you!
WPerhaps your PC mainboard has some USB ports on a different controller ? Like separate USB 3.0 and USB 3.1
WThis one is head scratcher
W@kevjuan
W@kevjuan Here's a Python script to stress test that HID endpoint, uploads data at max rate. Let me know if you can reproduce the hangup.
hid_out_stress.zip76.81KB
WOn my windows machine after 26M transfers still chugging along nicely
HMy motherboard provides USB 2.0, USB 3.2 Gen 1, and USB 3.2 Gen 2. So far, I only tried on multiple USB 3.2 Gen 1 ports and saw the issue. For some reason, I couldn't get your stress test program started. I kept getting this write error.
H
H@walmis When I enter 'dev.get_error()', I get 'WriteFile: (0x00000057) The parameter is incorrect.'
H? The emergency button must be : 3 Pin 1no 1nc OR 6 Pin 2no 2nc ??? Thanks for your support 
WThe first one, it should just break the positive power supply line when activated
Wgotcha, change the report_id=0x55 to report_id=106
should work
should work
HYep, report_id=106 does work. I'll let you know later if something happens. Thanks!
HYou mean it's better to order the second one ?
W3-pin
HH@walmis I got to play with your USB stress test program a bit more. I found out it could go for a long long time when the stick (i.e. motors) was not under any load. As soon as I moved the stick around, the USB error started to occur. Interestingly enough, the Wireshark capture shows the ACK from the endpoint was eventually sent but after 1 second instead of the typical 1 ms. The error check shows 'hid_write/WaitForSingleObject: (0x000003E5) Overlapped I/O operation is in progress.'. Apparently, time-out happens after the long wait, hence, the error. What do you think may cause this kind of delay in ACK from the endpoint?
H
WOne quick thing you could try is to isolate the PSU ground from the mains earth connection, other than that I'm baffled. Looked the code, USB driver is pretty simple, maybe errors occur on cable and are not recoverable. I could send you another board with a different MCU to test.
W
HIs the power supply (+19 to 24V) ground tied to USB ground through a 0ohm resistor somewhere on the PCB that I can remove? If not, how do I separate USB ground from power supply ground? Thanks!
HThank you for your offer! Let me play with the current board some more. I actually enjoy learning stuff about the USB interface that I knew nothing about before. If, say, I still struggle in a few days, I'll take up on your offer. I'm willing to pay for the shipping to offset some of your cost.
WYou can completely disconnect the 20V (both positive and negative) from the USB board (the fan will not work then though) - that will break the GND loop.
Or 2. remove the GND finger on the mains plug/or the earth wire from the mains lead that connects to the power supply input/or use a mains cable without grounding.
Or 2. remove the GND finger on the mains plug/or the earth wire from the mains lead that connects to the power supply input/or use a mains cable without grounding.
HMy +19 to 24V supply is a lab power supply, which has red(+), black(-), and green(chassis ground) terminals. My understanding is, since I'm connecting through red and black, I don't think the earth ground is part of the ground connection. Anyway, I did try tying +19 to 24V directly to the motors bypassing the FFB USB control board as you suggested and I still got the same USB issue. Looking closely at my build, I did notice the USB panel-mount connector that I got through Aliexpress does not have the shield tied to ground. I don't think it would matter but I would try to connect it to ground just to be sure.
WAlright, it's not GND loop then, perhaps the USB wires go parallel and close with the power wires?
You could try to add a ferrite to the USB wires. Or maybe try another power source? Man, you have a strange case there
You could try to add a ferrite to the USB wires. Or maybe try another power source? Man, you have a strange case there
HMan, I agree with you that this is a real doozy. I tried AC outlets from different part of my house and from different circuit breakers. I also found a USB cable that had a built-in ferrite core and tried. All of the tried cases so far reproduce the same USB issue (occasional 1 sec ACK from endpoint). I haven't tried tying the shield of the panel-mount USB connector to ground (a somewhat difficult mod) but I'm skeptical that anything will be different. I'm close to the end of my wit unless you can come up with something else for me to try. I was in a process of building a home-made reflow oven but got distracted and didn't finish. I'm seriously considering completing this reflow oven project so that I can put the FFB control board in to reflow for a while just to see if my luck changes... Sigh...
SMaybe the issue is the lab power supply. I know Chinese rebranded ones are very noisy. Maybe you can try a different one like a laptop power brick or one that uses a transformer. Worth a try?
HI actually started out with a laptop power brick and noticed the issue. I thought it would be better by switching to a lab power supply but the problem persisted regardless of form of supply.
SHave you tried a different computer?
Stry it on a laptop if you have one, run the same tests. It could be the usb controller if your on amd motherboard.
Syou can also try turning off all settings related to usb power savings/ standby/ sleep in windows and in bios.
S
HThank you for your response! My motherboard is AMD based Asus Tuf Gaming X570 Pro WiFi. I finally had a chance to do the same test on my Intel based laptop and could not reproduce the issue. I updated BIOS and AMD's chipset driver for Windows 10 to the latest and turned off all USB power/standby/sleep/selective suspend settings as you suggested. I still see the same issue. Unfortunately, my laptop is not powerful enough for gaming so I'm pretty much stuck. Where did you hear about this USB thing with AMD motherboards? Is it something that people complained a lot in the past?
Sits a problem with other devices too
Shp had to send me a special cable for my hp reverb g2 so it doesn't crash all the time specifically for amd motherboard
Sreddit
402 votes and 680 comments so far on Reddit
Sthis is what they did to it so it can work with my motherboard
SI have the asus strix x570i
SSo you can try a usb hub or pci usb hub, apparently that fixes the issue if you find a good one
Sapparently amd tried to fix it through bios updates multiple times and failed
S
Syou can read more about it here