Airoha en7523 Ethernet #9
Open
Sirherobrine23
wants to merge 15 commits from
airoha_en7523_eth into airoha_en7523
pull from: airoha_en7523_eth
merge into: :airoha_en7523
:airoha_en7523
:airoha_en7523_en8801s
:airoha_en7523_eth
:airoha_en7523_trng
:airoha_en7523_thermal
:airoha_en7523_snor
:airoha_en7523_pon
:airoha_en7523_pinctrl
:airoha_en7523_pciephy
:airoha_en7523_new_clk
:airoha_en7523_i2c
:airoha_en7523_dram-controller
:airoha_en7523_cpufreq
:pse-pd-gpio
Dismiss Review
Are you sure you want to dismiss this review?
Labels
Clear labels
Compat/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Breaking change that won't be backward compatible
Something is not working
Documentation changes
Improve existing functionality
New functionality
This is security issue
Issue or pull request related to testing
Priority
Critical
1
The priority is critical
Priority
High
2
The priority is high
Priority
Low
4
The priority is low
Priority
Medium
3
The priority is medium
Reviewed
Confirmed
1
Issue has been confirmed
Reviewed
Duplicate
2
This issue or pull request already exists
Reviewed
Invalid
3
Invalid issue
Reviewed
Won't Fix
3
This issue won't be fixed
Status
Abandoned
3
Somebody has started to work on this but abandoned work
Status
Blocked
1
Something is blocking this issue or pull request
Status
Need More Info
2
Feedback is required to reproduce issue or to continue work
No labels
Milestone
No items
No Milestone
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: airoha_en7523/kernel#9
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
ToDo
@merbanan I had to revert airoha_eth, and when I re-enabled
NETIF_F_IP_CSUMandNETIF_F_TSOI had no TCP network traffic; several packets were being sent out of sequence or out of order, but ICMP was working normally.Something else I'm seeing is IPv6, it works occasionally, but it must be a problem on my end. Another thing I'm noticing in Wireshark is multiple retransmissions, ipv6 icmp working normally
@@ -1152,5 +1152,5 @@* the MT7988 SoC. Trapped frames will be forwarded to the CPU port that* is affine to the inbound user port.*/if (priv->id == ID_MT7531 || priv->id == ID_MT7988 ||priv->id == ID_EN7581 || priv->id == ID_AN7583)same here
@@ -2588,5 +2588,5 @@/* Allow mirroring frames received on the local port (monitor port). */mt7530_set(priv, MT753X_AGC, LOCAL_EN);/* Enable Special Tag for rx frames */if (priv->id == ID_EN7581 || priv->id == ID_AN7583)@merbanan i'm surprised that my switch is working without adding
|| priv->id == ID_EN7523Well I am using the EN7581 settings so I think you can get away with alot. It just wont work optimally.
@@ -491,0 +518,4 @@if (airoha_is(eth, en7523)) {/* map GDMP sram to fe */airoha_wr(eth->gdmp_regs, 0x74, 3);@merbanan this is fist build with
gdmpenabled in my code and workingDo you get ~250mbit tx now ?
Openwrt -> Computer: 620~812Mbps
Computer -> Openwrt: 1.05~2.10Mbps
router
computer
It seems the PPE will be tweaked later; some things appear to be working, but there's a lot of packet loss with PPE/NPU enabled
I am getting over 200mbit from the SoC tx path. There must be something missing in the code.
@@ -606,3 +666,3 @@break;case 0x10 ... 0x14:case 0x8 ... 0x14:port = 0; /* GDM1 */I think this hunk is bogus. 0x10-0x14 should be correc.t
@@ -1957,2 +2066,2 @@qid = skb_get_queue_mapping(skb) % ARRAY_SIZE(qdma->q_tx);tag = airoha_get_dsa_tag(skb, dev);if (skb->len > 2048)goto error;Get rid of this. It was just debug code.
@@ -1979,2 +2097,3 @@tcp_hdr(skb)->check = (__force __sum16)csum;msg0 |= FIELD_PREP(QDMA_ETH_TXMSG_TSO_MASK, 1);if (!airoha_is(qdma->eth, en7523))msg0 |= FIELD_PREP(QDMA_ETH_TXMSG_TSO_MASK, 1);This is unneeded.
@@ -627,1 +655,4 @@static const struct airoha_npu_soc_data an7583_npu_soc_data = {.use_memremap = false,.extra_cores = true,.num_cores = 8,This can be 2-6 depending on the model of the SoC. A dts property would be better in this case.
If the cores actually are fused off is unclear but thermals must be respected.
but from
computer->routeris slowbut download is okay
and with NPU enable is okay
Hmm I was able to get 200mbit. Are you still having issues with tx?
I was away for the weekend, I got back on Monday and didn't even have time to mess with the kernel. I haven't had much time to look at the pinctrl driver yet, I'll try it today, And I'll also try to take a look at the xPON phy code today
I'll test it with gdmp disabled to see if there's any change, but I don't think there will be much of a difference in speed
Out of the box from current tree I get ~250 mbit.
That's much better, but why is the upload so slow, while the download seems to reach almost GbE speeds?
I'm away from home and I don't know when I'll be back. I have the router with me, but I don't know when I'll be able to test it. Could you send me your settings so I can test them when possible?
There are no settings. I just start iperf3 -s on the router and do the test. One thing that might cause the issue is that I think TSO is broken. My Realtek network card seems to ignore that the MTU is huge so it just works anyway. Maybe that is why you have issues.
Regarding low tx performance it might be this dsa tag mem move that is redundant.
Anyway the ref software gets like 350mbits. So it is possible to get more but the target is to get hardware acceleration going and then it doesnt matter anyway. 250mbit is enough to handle the control traffic.
So, from what I understand, the upload speed won't exceed 350Mbps without hardware acceleration on the switch for wireless and on the switch for CPU?
I'd like to be able to test iperf3 at stock speeds to see what speed it's currently running at.
Well I would expect that more engineering would be able to beat the ref driver. Top gives 70% idle.
For example I think that the we also only use one core. At least we only trigger interrupts on one core.
So IMO there is more performance to be unlocked.
Ok, I found one thing now. But it should be possible to get more performance. This only fixes TCP, udp seems to be capped at 84.5Mbits/s.
Just add back the missing TSO flag.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/net/ethernet/airoha/airoha_eth.c?h=v6.19-rc1#n1939
That one was missing from the main code. Now I get this:
This is good, it makes it more acceptable, but in terms of internet access, when I tested with https://speed.cloudflare.com without TSO msg, I got around 200-320Mbps download and 30-60Mbps upload.
I have a 600Mbps download and 300Mbps upload connection, I don't have another Ethernet device to test the switch locally 🙃🙃
Yeah, there is something else also. If I disabled TSO in hwcaps I only get like 100mbits/s and the cpu is far from running at 100%. TSO is just hardware acceleration kicking in.
@@ -605,3 +672,3 @@port = 2; /* GDM3 */break;case 0x10 ... 0x14:case 0x8 ... 0x14:This should be reverted.
@@ -1660,0 +1753,4 @@airoha_fe_clear(qdma->eth, REG_GDM_INGRESS_CFG(port->id),GDM_STAG_EN_MASK);}The following 2 lines should be removed.
from this
to this?
Those lines should be removed.
@@ -1015,3 +1083,3 @@airoha_qdma_rmw(qdma, REG_TX_IRQ_CFG(id), TX_IRQ_THR_MASK,FIELD_PREP(TX_IRQ_THR_MASK, 1));FIELD_PREP(TX_IRQ_THR_MASK, 0x20));Revert this also.
@@ -37,6 +37,8 @@#define TX_DSCP_NUM 1024Wouldn't it be good to add this here?
0a55a19aa2/xx230v/src_output/opt/bba/bba_3_0_platform/sdk/en7529/modules/private/qdma/EN7516/qdma_bmgr.c (L11-L20)I guess, I am not convinced all this is good (ie the raf code ring buffer sizes). I think all this queue stuff is over engineered from the hardware side. Some of the queues need to be large but the others will only allocate unnecessary memory.
If we ever get to adding GPON support then yes we maybe need to differentiate the sizes of the descriptor (dma) rings. If we add a printout there:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/net/ethernet/airoha/airoha_eth.c?h=v6.19-rc1#n1956
We will notice if we ever run out of descriptors.
Anyway, the networking part is now better then what the reference software can reach but the question is what happens with the wifi performance. We might need to tune some parameters when wifi gets in the mix.
We need to get the PPE working properly, and get the NPU working with the MT76 as well. Currently, the Wi-Fi is working at the best upload and download speeds possible on the LAN, and communication between devices is normal, as I can get 1.2Gbps~880Mbps on the 5GHz wireless band
with Flow offloading enabled
CPU utilization
stock router is strange compared with now
(this is in stock firmware)
(this in stock -> openwrt)
I guess the cpu performance is ok for now. Will you update the openwrt PR?
Yes, I will, but I have to resolve a problem with pinctrl, the device is crashing and restarting after 45Secs
Here you should split out the mt7530 parts and send those upstream. They are ok, it is all the other stuff that will be difficult. But splitting out things into smaller parts is always the best way forward.
@@ -1654,3 +1745,3 @@if (netdev_uses_dsa(dev) || port->id > 2)airoha_fe_set(qdma->eth, REG_GDM_INGRESS_CFG(port->id),GDM_STAG_EN_MASK);GDM_STAG_EN_MASK);Redundant hunk
Anyway my main point is that it will take alot of time to get everything upstream and we need to start that process. Divide and conquer, take small and easy parts and then go from there. I am not sure how we should handle the difference in position of the dma descriptor members. This is something I think we till get an answer from the kernel devs. But to get there we need patch set that is easy to review. So split out the mt7530 parts and the npu stuff. The phy change should be its own patch also. From there we start chipping off airoha_eth.c into a patch set with like 90% easy stuff and 10% of the tricky parts.
I suggest you start with sending the phy path upstream. It is tiny and easy to review.
I will have to make a patch for mt76 to activate wed on mt7915
Sirherobrine23 referenced this pull request2026-01-24 11:08:02 +00:00
Hi @merbanan, @lilmayofuksu tested the ethernet on the EN7562, here's the iperf test:
@merbanan needs this large qdma size?
ecb65624e5/target/linux/airoha/dts/en7523.dtsi (L34-L42)I'm having trouble loading initramfs due non free memory
I am quite sure that we can reduce the buffer allocation quite a lot. Maybe you can look at the memory allocation after boot up. The NPU memory reservation can temporary be removed also.
I've noticed that if the traffic goes through the cpu (wan port), I can only get around 500mbps
I ran the speed test on OpenWrt to check the network speed (speedtestcpp and speedtest-go installed). Between the interfaces (WAN -> LAN), at least 60 to 80% of the traffic between them is expected. Wireless traffic should be about 70% of what arrives on the WAN.
@merbanan the npu boot sequence is quite different from v2, v2 has more definitions
522a0ca806/include/global_inc/modules/npu/wifi_mail_type.h (L57-L85)while v1 has
4d76b59039/sdk/en7529/release_bsp/UNION_EN7523_GLIBC_7915_7915_ActiveEthWan_demo/global_inc/modules/npu/wifi_mail_type.h (L61-L78),Currently, we call
airoha_npu_wlan_init_memorywhen the mt76 module is loadedThe log we should have:
By configuring it to be set even before the module loads, we have this:
4d76b59039/sdk/en7529/linux-4.4.115/include/ecnt_hook/ecnt_hook_fe.h (L1784-L1793)Look at the end of this file and you'll see that it's also freezing after the switch interfaces come up
I have some NPU-related information. Apparently the NPUs access the console port uart directly. Lines with [CX] (X=0-3) are emitted directly from the NPU cores running the loaded code.
If the following is run:
devmem 0x1ec0c194 32 0x00000001 (any non zero value).
The NPU log output to the console stops.
The NPUs seems to be RiscV 32 based. Ghidra can partially disassemble the firmware.
I still don't know what's happening, but when airoha_eth loads, it freezes the entire system trying to access something on the NPU. I'll take a closer look at what's going on
0 => Enable NPU UART Log
1 => Disable NPU UART log
522a0ca806/econet_public/arch/arm/mach-econet/ecnt_npu.c (L704-L708)crashing after this
@merbanan I tested it and made several changes to the NPU code. The entire system crashes when the switch is enabled, but when I disable the switch, it doesn't crash. and it always crashes at
[ 17.287891] mt7530-mmio 1fb58000.switch wan: configuring for phy/internal link mode. Regarding the NPU on the mt76 for themt7915e, I will still have to change a few things:I ran tests today, and I'm still having problems with the switch. If I boot with only one cable connected to LAN3, I get traffic, but if I use more than one cable, it doesn't detect the connection on any of the switch ports
But the LEDs light up when I connect the cables and blink initially when connected
The analog parts are always tricky. What is the result if you only calibrate rext?
I had a problem with my script to applying patches, but I just uploaded commit i'm using
I'm seeing mt7530 drivers in the mt7621 and they use these interrupts:
interruptsshould not start with0I looked at the dtsi and interrupt-parent is missing.
GIC_INT <-> SPI_37 <-> MT7530_INT <-> 1,2,3,4
This is how the interrupt routing needs to be.
What do you mean? I don't understand. The interrupt is on the eth:
2b693a6bf8/target/linux/airoha/dts/en7523.dtsi (L639-L640)Will I need to add it to the switch?
The MT7530 switch is connected to the GIC interrupt controller pin 31 (not 37).
The 1G switch port phys are connected to the MT7530 switch interrupt controller.
I am unsure if this node will resolve to the correct interrupt controller (MT7530). If cat /proc/interrupts show that the mt7530 interrupts are allocated to the phy ports then everything is fine. If not then the interrupt parent is needed.
When the port phy emits an interrupt is travels to the the GIC via pin 31, then the mt7530 interrupt handler is run to handle the event.
this is interrupts in xx230v:
LGTM. But the dtsi should disable the phys and the dts should enable the used ports.
b58177150ato92609bdc3b92609bdc3btodaee516910daee516910to3069f60542503d0ff11btocd8c2512abcd8c2512abto3069f60542the Ethernet driver seems to be having some kind of problem, but I haven't yet identified where
and in the npu same:
3069f60542tobb30e68e37bb30e68e37todb790e36ec@merbanan i had to do a rollback to commit
470ae207b6...c3f3209d67of the ethernet driver to boot normaly but ethernet driver no working in en7529 SoC's, on en7562 working but hw offload stop working with TCP packet with hw offload without rollback commitEN7529CT Bootlog with rollback commit
10c24861d1toc0bea29f43c0bea29f43to5a2e2608195a2e260819toa766e9204ea766e9204etoabac5a43b1abac5a43b1to2fff451466728b2776d4to23acbe6fa2@@ -553,12 +614,13 @@ static int airoha_qdma_fill_rx_queue(struct airoha_queue *q)q->queued++;nframes++;offset += AIROHA_RX_HEADROOM;@merbanan
I had to remove that part because the driver not having network traffic. I didn't have to make many changes with the commit airoha_en7523/openwrt@dda777dd44
23acbe6fa2to9ec90a52399ec90a5239to58b92e371558b92e3715to16bf7a886516bf7a8865tob61f1f3a2ab61f1f3a2ato52092d6ac7@merbanan
The en7523 PPE supports both 64-byte and 80-byte files, so why don't we use the 80-byte file since it's supported by the en7523?
and from what i've seen, the an7581 and an7583 are already using 80 bytes by default
I wonder if the airoha_ppe and friends support 64byte at all. Anyway we probably should use that same as an7581 for now at least.
from what I've seen in the code, only the en7523 and the mips SoCs support 64 bytes, the an7581, an7583, and an7552 force 80 bytes in the PPE, and the EN7523 has both.
They support it as much as a providing a bit to toggle it.
I will be adding the values for 7523_80Bytes, then I will also take a look at hw_offload through the NPU
Should I continue with 80 or 64?
80, we can always change back if we need the ram for some reason.
okay
4dba462531to39d9b6841f39d9b6841ftof7f93b63a3f7f93b63a3toac0acfff1fcomputer:
openwrt:
Can you try adding the LRO patches posted by lorenzo?
As far as I know, they are already included here, and I have already added the en7523
i added a print dump to airoha_ppe_foe_get_entry_locked, and it's returning a completely empty table:
ac0acfff1ftod6f46892f0This is for the AN7581 but might work on EN7523 also.
it's returning the same values in raw_hwe that i added in ppe_debugfs, it's all coming back as zero.
d6f46892f0toca70a218e0i added some changes, now it seems the entries are being added to the PPE, but I don't recognize some addresses, so I can't consider it working
i say this because my IPv6 is disabled on the local network
furthermore, I achieved stability in the NPU initialization by changing MEMREMAP_WB to MEMREMAP_WC in devm_memremap
Entries
i just added dev_info to the push of entries in the PPE and we are having the data inserted incorrectly in the PPE
ppe
airoha_ppe_foe_flow_commit_entry/sys/kernel/debug/ppe/entries
ca70a218e0to33a58913c333a58913c3to9941ce9a549941ce9a54to8ec31f73538ec31f7353to25fa0a541a@merbanan I added your mt76 patcher for the NPU's RX, and that's it for now. I'll test it with the NPU firmware from the xx230v, because the Heingard one doesn't work with my xx230v's wireless chip
my laptop:
25fa0a541atofc3c0a8425fc3c0a8425to33da58c29d5719b0a676to33da58c29dWhen the PPE rules are working properly we can start looking into the wifi acceleration via WED and NPU. Alot of things needs to be wired in properly but in theory the same logic that is used in AN7581 for wifi offload can be used.
The
ppe/entriesis being populated, but the offload is still going to the CPU, so I'm being misled here 😩We need proper tools to look at the rules. And of course the PPE rules have another bit packing on EN7523...
Lots of PPE config registers seem to differ abit also.
Have you added WED support also? Or are those patches not yet merged?
Your patches caused a kernel panic with WED, but the NPU wouldn't finish configuring the ring, so I removed the patchers
Based on what I have from
hw_nat/, I realized this, I added some, but it seems that some files are missing from the TP-Link GPLs that contain the codes@merbanan i got to that part, besides having a speed 100mbps higher than what I'm paying for, the CPU usage by qdma is quite variable, but probably because of the logs in uart.
new tests without uart logs
local tests (Mikrotik mt7621 -> xx230v -> laptop)
e466df2fd8to9bf6f49282@merbanan now sfp working in hex s with changes i added
Bootlog
\o/
@merbanan confirm if working with u devices
Just merge the change as a new commit. If all is good later we can squash the commits. As long as it is a nice small commit it is easy to bisect. I'm going for the PPE so I'm not going to test any new code right now.
Okay, I'm noticing that network traffic isn't working very well from gdm3 to gdm1 (switch), but from gdm3 to gdm2 it is working, but without the PPE
The is something to look at after the PPE is working.
b3f7d0a972tobfe037d726@merbanan I think this test in mkt hexS
It seems that gdm2 and gdm3 aren't communicating very well with gdm1
Is device to gdm2/gdm3 pings working well ?
We need to add the xsi mac resets also. Right now the code toggles the reset regs. I'm not sure the reset driver has the support.
214b35bd92toac69130e0aOn HexS I get decent speed via the WAN directly on openwrt, but if I use lan2 the traffic doesn't go to TCP/UDP, it reaches openwrt, but doesn't go to the wan port on gdm3
Is it working well from the device?
The (en8801sn) phy -> gdm3 pcie1 -> wan sounds like broken phy setup. Try the new kernel driver for it.
gdm1 -> wan should work fine but 1000Base-X might need additional code for the driver. Have you requested the Hex GPL packge? It should be in there.
In my tests before the new changes, with only the patches I sent earlier, it was like this:
iperf -c 192.168.1.10 -R)Ok, my device have the usb-serdes connected to gdm3 so I should hit this issue also. I'll get there eventually. There might still be an issue with the serdes code.
GDM1 (LAN2) -> GDM3 PCIE1:
GDM1: LAN2 -> LAN5
GDM3 -> GDM2
GDM{1,2,3} -> Openwrt host
GDM1:
GDM2 -> Openwrt Host
GDM3 PCIE1:
This is a broken generated rule. The rule written sends the packets somewhere where they are dropped. Eventually it times out then some traffic gets through when it it learning the rule again. And then the broken state repeats. Might be a bad nbq id.
Can you run linux shell on the vendor software ?
No, not on RouterOS, it sends me to its limited shell
I made the request, but they sent the 2022 RouterOS GPL, without any content about Econet/Airoha in that GPL, so I don't have much information about it
77127f3d5etocf59175bfdcf59175bfdto8af48e1ab18af48e1ab1to1cf05aeaacThe EN7523 PON PCS has phya=INVALID, so mainline never initialises the analog serdes PMA. The attached RTL8221B drives an SGMII (1.25G) serdes for a 1G copper link and a 2500Base-X (3.125G) serdes at 2.5G; without the matching analog PMA bring-up the serdes link never establishes and the XSI-AE MAC stays in local-fault with no traffic. Port the vendor sgmii_api_pon0_force_{sgmii,hsgmii}() PMA bring-up: pulse the serdes + XSI-MAC clock-domain reset, then program SSR3/WAN_CONF and the rate-specific PMA registers (the reset clears SSR3/WAN_CONF, so they must be written after it), saving/restoring the MAC global config across the reset. Dispatch the SGMII or 2500Base-X init from pcs_config() by interface for the PON port, so the link comes up at either copper speed. Signed-off-by: Benjamin Larsson <benjamin.larsson@genexis.eu> Signed-off-by: Matheus Sampaio Queiroga <srherobrine20@gmail.com>The USB and PCIe serdes (GDM3 external ports) have a real phya so their analog is handled by set_phya_mode(), but mainline never programs the rate-adapt/AN/PCS-control values nor pulses their clock-domain reset, so the serdes link to the SoC never establishes. Port the vendor sgmii_api_{usb0,pcie0,pcie1}_force_hsgmii() bring-up: a shared helper writes RATE_ADAPT_CTRL_0/AN0/CTROL_6 and pulses RST_CFG with the per-port reset bits (PCIe0 0x12000, PCIe1 0x24000, USB 0x48000), saving/restoring the MAC global config across the reset. Dispatch it by PCS type in the 2500Base-X path. Signed-off-by: Benjamin Larsson <benjamin.larsson@genexis.eu> Signed-off-by: Matheus Sampaio Queiroga <srherobrine20@gmail.com>GDM3 (the EN7523 e650 "lan4" port) rides the USB (SSUSB) serdes via the usb_pcs PCS (compatible "airoha,en7523-pcs-usb", driver type EN7523_PCS_USB), not the PON PCS. 941-01 only brings up the PON serdes (its analog PMA init is gated on res->setup_pon and the PON PMA mapping on data->type == EN7523_PCS_PON), so the USB serdes never gets its XSI-MAC/serdes clock-domain reset. The result: the USB PCS digital layer is configured and link/AN train ("link up" at the copper speed), but the MAC/serdes RX datapath is never reset into a working state, so the frame engine receives zero frames (GDM3 FE RX counter stays 0; lan4 fails at every copper speed). The vendor sgmii_api_usb0_force_{sgmii,hsgmii}() pulse SCU_RST bits 15+18 around a save/restore of the PCS MAC global config to bring the RX datapath up. Port just that reset for the USB PCS (the digital PHYA / rate-adapt / AN / PCS control writes are already done by the common pcs_config path); run it after the PCS digital config, matching the vendor order. Signed-off-by: Benjamin Larsson <banan@ludd.ltu.se> Signed-off-by: Matheus Sampaio Queiroga <srherobrine20@gmail.com>1cf05aeaacto8180e76ac9View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.