Raspberry Pi 4B has issue with MT7612U USB dongles in OpenWrt AP mode?

[fakemanhk]

I have 2 RPi 4B, also at least 2 x MT7612U, in fact not only me, but most people trying with OpenWrt with RPi 4B + almost any MT7612U will end up having dongle driver crash (will recover by itself after a few mins) after putting it into AP mode and running with some traffic.

Something silimar to kern.err kernel: [ 304.423350] mt76x2u 2-1:1.0: error: mt76x02u_mcu_wait_resp failed with -110 showing up in dmesg log, following with certain lines of kernel dump.

To prove that's not OpenWrt driver issue, I used NanoPi R4S, or older Raspberry Pi 3B with same dongle (both are AARCH64), both having no issue (and speed was fast), just tested with laptop to download ~6GB file from Microsoft showing that a continous 3mins download stream wasn't causing any crash at all. Also powered USB hub was used to make sure there was no power issue with dongle, but this doesn't help.

[morrownr]

Hi @fakemanhk

Been there, heard this story many times before. It also happens with adapters based on the rtl8812bu (also AC1200). There is no fix for the rtl8812bu. The only work around is to run the rtl8812bu adapters in usb2 mode. For the mt7612u, there is a parameter to turn Scatter-Gather off:

https://github.com/morrownr/7612u

Look for known issue 2. You can ssh into OpenWRT and add the parameter.

The Pi4 has a problematic usb3 controller... maybe the hardware or maybe the software. The RasPi folks have not done a good job picking hardware for the Pi's.

Let me know how it goes.

[fakemanhk]

Hi @fakemanhk

Been there, heard this story many times before. It also happens with adapters based on the rtl8812bu (also AC1200). There is no fix for the rtl8812bu. The only work around is to run the rtl8812bu adapters in usb2 mode. For the mt7612u, there is a parameter to turn Scatter-Gather off:

https://github.com/morrownr/7612u

Look for known issue 2. You can ssh into OpenWRT and add the parameter.

The Pi4 has a problematic usb3 controller... maybe the hardware or maybe the software. The RasPi folks have not done a good job picking hardware for the Pi's.

Let me know how it goes.

Thanks, I will try it later with my Pi4B again, however I noticed that a month ago when I tried RPi 4B with OpenWrt snapshot (kernel 6.1.x) there was a moment that the crash didn't happen. In Raspberry Pi forum I observed that many people with RPi OS + kernel 5.15 LTS (which is same as the one using on OpenWrt 23.05 stable) having similar problem when using various USB devices (like USB-SATA bridge), and improvements observed after using kernel 6.1 LTS.

[fakemanhk]

Just did a test with echo 1 > /sys/module/mt76_usb/parameters/disable_usb_sg

On USB3 port it's never working, same issue, on USB2 now it's working but the speed is of course terrible, download never > 100Mbps while upload I barely getting 200Mbps.

And with USB2, now after reboot it won't detect my WiFi anymore, I need to unplug -> plug to same port (or another USB2 port) to trigger a plugin event to use it.

[morrownr]

I just reread your original post. I use openwrt myself but not on RasPi's. On RasPi's I use RasPiOS and my AP Mode guide that is on the Main Menu. I do have some rules that I follow when using RasPi4B's and some rules that are good when using usb wifi adapters:

• Never use a powered hub with a RasPi4B. There are some powered hubs that can work with RasPi's without problem but they are rare. There is a long history of powered hub problems on RasPi's so the info is there if you want to search for it.
• Never use more than one usb wifi adapter with the same chipset in the same computer. You can use multiple usb adapters with a computer but make sure the adapters use different chipsets.
• Keeping the number of devices plugged into usb low is a good idea.
• The Mediatek based usb wifi adapters generally use less power than the equivalent Realtek based adapters so that helps.
• Keep the overall power load on the usb subsystem in the Pi4B to < 1200 mA.

Following these rules, I can't say that I have had a problem with a mt7612u based adapter in AP mode for a long long time.

[morrownr]

Oh, and make sure you are using a very good power supply and cable. The error you are seeing can be caused by power problems.

[fakemanhk]

I have double checked that my power: No throttling during my tests

I disabled onboard wireless, only using 1 testing WiFi each time (and it's the only USB device on Pi4B to avoid USB bus traffic congestion/interference)

Even with powered USB hub, I use those without "back powering" one to ensure nothing can go wrong with it, but of course after I found out that the USB port was enough to bring up the WiFi I only direct connect them.

I don't know if there is anything to do with mult-state adaptor, tonight I will test more with my PIX-LINK, if that works then it could be a problem with multi-state WiFi.

[fakemanhk]

Tried to grab more log, the failure looks like to be USB port related, I am already using official RPi power supply so I don't think I have power issue. Once this happens no matter how I plug/unplug the whole USB bus seems dead.

Wed Apr  3 00:00:03 2024 kern.warn kernel: [  274.275890] xhci_hcd 0000:01:00.0: WARN: TRB error for slot 2 ep 7 on endpoint
Wed Apr  3 00:00:03 2024 kern.warn kernel: [  274.283267] xhci_hcd 0000:01:00.0: WARN waiting for error on ep to be cleared
Wed Apr  3 00:00:03 2024 kern.err kernel: [  274.283421] mt76x2u 2-2:1.0: tx urb failed: -84
Wed Apr  3 00:00:03 2024 kern.err kernel: [  274.290412] mt76x2u 2-2:1.0: tx urb submit failed:-22
Wed Apr  3 00:00:03 2024 kern.warn kernel: [  274.300093] xhci_hcd 0000:01:00.0: WARN waiting for error on ep to be cleared
Wed Apr  3 00:00:03 2024 kern.err kernel: [  274.307238] mt76x2u 2-2:1.0: tx urb submit failed:-22
Wed Apr  3 00:00:05 2024 kern.err kernel: [  276.666422] mt76x2u 2-2:1.0: error: mt76x02u_mcu_wait_resp failed with -110

[fakemanhk]

OK, after digging the RPi forum for more, I finally came across a "workaround" and managed to make it working in AP mode without crashing the USB bus: Enabling Packet Steeing & CPU affinity by myself.

Former one is easy, just go to Network > Interface > Global Option to check the Enable Packet Steeting option, after that the USB crashing issue is already resolved, however performance was extremly terrible (speed fluctuates and difficult to get past 100Mbps throughput).

To solve this issue, I perform CPU affinity manual setup. By default the XHCI USB is using the same core 0 for interrupts, then I found out which interrupts of onboard NIC by grep eth0 /proc/interrupts, in my case IRQ 35 & 36 are responsible for eth0 interrupts, then:

 echo 2 > /proc/irq/35/smp_affinity
 echo 2 > /proc/irq/36/smp_affinity

Above example was assigning both interrupts to core 1, more configuration can refer to this OpenWrt guide.

After that I continued to test, performance increased drastically, basically similar to what I observed when using it on NanoPi R4S, and I tried to download a Windows ISO + Linux ISO (total size roughly 8GB) with download speed not less than 300Mbps from closest mirror. Not sure if you want to put this information into your guide to remind RPi 4 users.

[morrownr]

This is interesting. Thanks for writing this up. I will try to put a short blurb in my guide about it. It does seem to just apply to OpenWRT and its specific tuning as I have never seen this problem with my guide on a Pi4B.

Did you try irqbalance?

[fakemanhk]

This is interesting. Thanks for writing this up. I will try to put a short blurb in my guide about it. It does seem to just apply to OpenWRT and its specific tuning as I have never seen this problem with my guide on a Pi4B.

Did you try irqbalance?

irqbalance doesn't help here.
I believe OpenWrt doesn't expect people using USB WiFi (or basically any USB NIC) so it's never optimized like those full feature distro (RPi OS for example), for other SBC like NanoPi R4S which I own, I can see that by default OpenWrt had preconfigured certain CPU affinity assignments for different NICs onboard to allow maximum performance.