EDIT: The setup described below worked fine for me for six months, with only the annoyances described below (mostly not being able to traverse DHCP requests across the bridge.) However, I recently upgraded to OpenWRT Backfire and took the opportunity to replace my proprietary ADSL router with a Fonera. This allowed me to switch over to WDS as described in this post. So this setup is no longer in use at my house.

Renting sucks when you’re a nerd. When you move, you have to re-network all your bits and pieces, without the benefits of laying cable. Wireless is great, but not every network device has it. I have a VoIP phone and an XBMC XBox, each without wireless and each in a separate room.

I need a way to connect the phone and the Xbox to my existing wireless access point.

Here’s a rough diagram of my network:

Wired & Unwired Diagram

Rough diagram of my network

Options

I want a flat network, single broadcast domain, for all of the (wired and unwired) devices in the diagram above.

I already have two Linux-compatible wireless routers, an Asus WL-500G and a Fonera. Both running OpenWRT.

If my routers had Broadcom wireless chipsets then this task would be easy. Thanks to driver support, OpenWRT can automatically bridge network segments through a wireless client (at least on 2.4 kernels, not sure about 2.6.) Neither of my routers have Broadcom. The Asus used to have it, but I swapped in an Atheros card. Oops.

WDS is a possibility, using each router as a repeater station. However, each repeater costs half the available bandwidth. I also can’t get WDS+WPA to work with my access point. YMMV.

You can set up separate networks, one for each segment, and the routers can route(!) between them using iptables. I’ve tried this and I don’t like it. It’s much easier when you have one single network with a single broadcast domain: simple DHCP, simple NetBIOS lookups, no routing tables to maintain, etc.

The final option, and the one I settled on, is using ARP-NAT.

What is ARPNAT?

ARPNAT is a clever hack. It’s the same idea as NAT for IP networks, except it works one layer deeper. Instead of translating IP network addresses, the router translates between the MAC hardware addresses on each side of it.

If something on the wired side of the router makes an ARP request for the MAC address of an IP on the wireless side, then the router forwards the request as if it came from the router. When the response comes back, it mangles that too. Instead of passing back the real MAC (which lives on the wireless network), the router gives its own wired MAC address. Then, when it receives frames for IP addresses on the wireless network, it forwards them through. It does this to both sides of the bridge.

Here’s an example, taken from my network:

  1. XBMC Xbox asks “What is the MAC address of 10.0.0.1? Tell me: 10.0.0.5.” This is a broadcast ARP request over the wired ethernet.
  2. Asus router sees the ARP request, and forwards the broadcast onto the wireless network, as “What is the MAC address of 10.0.0.1? Tell me: 10.0.0.4.”
  3. Access point (at 10.0.0.1) gets the ARP request from the wireless, and replies with its MAC address: AA:AA:AA:AA:AA:AA.
  4. Asus router gets the ARP response, sent to 10.0.0.4. It mangles it and forwards it back to the Xbox as “10.0.0.1 is at BB:BB:BB:BB:BB:BB”, where BB:BB:BB:BB:BB:BB is the wired address of the Asus itself.
  5. Now, any time the Xbox wants to talk to 10.0.0.1, it will talk to BB:BB:BB:BB:BB:BB, and the Asus will forward that frame onto the wireless network, to the real destination AA:AA:AA:AA:AA:AA.

On Linux, this is done using a patched version of the ethernet filtering program ebtables.

How to do it?

Out of the box, OpenWRT does not support either ebtables or ARPNAT. The developers have expressed opinions that ARPNAT is a hack, and ebtables is too slow (see here & here.) We’ll get to that issue, later.

However, there are some forum posts that show how to do it. In this post, a developer of the OpenWRT fork Gargoyle links to their patch for ARPNAT. So, one option is to just use Gargoyle. Gargoyle apparently has a very nice web interface, and is good for intermediate users. I prefer flat config files, so I’m going to stick with OpenWRT.

Building OpenWRT with ARPNAT Support

If you’ve never compiled OpenWRT before, start here. Alternatively, I have some prebuilt 8.09.2 images for Atheros devices (like the Fonera), here.

To build, you’ll need two patches. One is from Gargoyle. The other is a small bugfix to allow a DHCP client to run on the router. The patches can be found here. These patches are against the 8.09 branch of OpenWRT SVN, I don’t think either will apply to the trunk.

EDIT: I’ve just posted a slightly different patch, 04-arpnat-brcm47xx.patch, for brcm47xxx devices like the Asus WL-500g. It’s different because these devices have an older kernel version in 8.09, and also because the arpnat kernel module doesn’t seem to load the first time around. Also, on this architecture DHCP responses don’t seem to pass back correctly to devices on the other side of the bridge (while on my Atheros-based Fonera they are passed back.) I’m just using a static IP on those devices, for now. Things will also work fine if the DHCP server is on the bridge router, itself.

You’ll need to enable ‘ebtables’ in your configuration (ie when you make menuconfig.) You’ll probably want wpa-supplicant as well. The .config that I used for the Fonera can be found at the link above, as well.

Once up and running, /etc/config/wireless only needs one tweak. Here’s mine:

config wifi-device  wifi0
	option type     atheros
	option channel  auto

config wifi-iface
	option device	wifi0
	option network	lan
	option mode	sta
	option client_bridge 1
	option ssid	<your ssid here>
	option encryption psk
	option key <your key here>

The only difference is “option client_bridge 1“, which indicates that you want ARPNAT enabled.

/etc/config/network is simple, too. Here’s the bridge section of mine:

config interface lan
	option ifname	eth0
	option type 	bridge
	option macaddr  CC:CC:CC:CC:CC:CC
	option proto	dhcp

The only difference here is option macaddr. You will need this only if you want to assign the router’s IP address dynamically via DHCP. If you do this, “option macaddr” must be set to the MAC address of the interface where the DHCP server lives (ie wired side or wireless side.) The DHCP request will only go out on a single interface, so you need to make sure it is the correct one.

If you don’t need DHCP then you don’t need to manually assign any MAC address. “option macaddr” can be totally removed.

That’s it! When the router boots up, it will connect to the wireless network, and any devices on the wired side will automatically be bridged onto the wireless (and vice-versa.) To them, it just looks like one big seamless network.

Bear in mind that not all routers support failsafe mode (Fonera for one), so if these values are wrong then you may end up locked out of your router, especially if you run the router with a DHCP-assigned address. If you’re not certain, I recommend running with a static IP first, so you can always plug in a network cable and still connect to the router.

Thanks again to Eric Bishop and the Gargoyle project, because it’s really his patch (building on other people’s work for ARPNAT) that makes all this possible.

Problems with ARPNAT

The OpenWRT developers will not consider ARPNAT, or even include ebtables, in the default package set of OpenWRT. They say this is because of performance impact. Every ethernet frame that hits the router needs to be inspected in software, so it’s understandable that there will be a cost.

Some cost is OK by me, in exchange for convenience. So I did some tests:

The test methodology was to ping flood the Fonera from a PC:

Performance of ebtables & ARPNAT (longer is better)

  • “Ethernet, no ebtables” is a vanilla Openwrt 8.09.2 install
  • “Ethernet, ebtables installed” is with patches and ebtables installed, but no ARPNAT enabled.
  • “Ethernet, ARPNAT enabled” is with ARPNAT turned on, but no traversal (the pings still go just to the router.)
  • “Direct WiFi” is pinging the router directly over the wireless network. It’s slower, but only because WiFi is slower than ethernet.
  • “WiFi via ARPNAT” is pinging a client on the wireless side, from a client on the wired side. So the ping is going through ARPNAT translation on the router.

Under all of these tests the CPU on the router was pegged at >95%.

The results seem to show that ebtables & ARPNAT do not drastically affect the performance of the Fonera. Turning on ARPNAT causes direct traffic to the router to be about 17% more costly. Actually bridging packets through ARPNET on the router is only 32% more costly than directly accessing the router itself.

I’m surprised these figures aren’t worse, given the doom & gloom spouted by the OpenWRT devs. It’s possible there’s another circumstance under which ARPNAT falls over, or possibly it performs worse on certain architectures. I’ll post back if I learn anything interesting.

EDIT: One more discovery, obvious in retrospect, is that IPv6 won’t get ARPNATted by this config. If you have any devices that use IPv6 (like iTunes talking to an Apple Airport Express), you’ll either need to disable IPv6 or tweak the ebtables scripts somehow to translate IPv6 as well. I opted for disabling IPv6.

21 thoughts on “Wireless client bridging with OpenWRT

  1. Great work !
    I applied the arpnat patch in openwrt version 8.09.2 (svn) but this not work for me. The patch apply with no errors and compilation report ok. I upgrade fonera routers com my vesion and modules ebtables arpnat are up but no arpnat reply. Dou you make compilatiom over openwrt version 8.09.2 ? Your images are ok e arpnat work perfectely. Please, post instructions for compilation withyour version. Thank you e great job !

  2. Hi James,

    Nothing springs to mind, sorry. I can only think you’re missing something from the build config, or something from one of /etc/config/network or /etc/config/wireless .

    If you run ‘ebtables -t nat -L’ do see any entries for arpnat?

    If you diff the config-arpnat-fonera (under the patch link above) against the .config that you’re building with, do see any noteworthy differences? That’s the config that I built the images with, so they should give identical results.

    Finally, did you ‘make clean’ after you applied the patch? I’ve found that sometimes OpenWRT’s build system doesn’t rebuild changed files like you’d expect.

    Sorry for the delay in getting back to you, btw.

    Hth. – Angus

  3. Hi Angus,

    Thank you for your reply!

    Well, I go to show the steps that I followed:

    Openwrt compilation:
    mesh@LPtelecom:~$ svn co svn://svn.openwrt.org/openwrt/tags/8.09.2 arpnat-test

    mesh@LPtelecom:~$ cd arpnat-test

    mesh@LPtelecom:~/arpnat-test$ wget http://projectgus.com/files/openwrt/arpnat/patches/04-arpnat.patch

    mesh@LPtelecom:~/arpnat-test$ patch -p1 < 04-arpnat.patch
    patching file package/madwifi/files/etc/hotplug.d/net/20-client_bridge
    patching file package/ebtables/patches/100-arpnat.patch
    patching file target/linux/generic-2.6/patches-2.6.26/701-arpnat.patch
    patching file target/linux/generic-2.6/config-2.6.26

    mesh@LPtelecom:~/arpnat-test$ wget http://projectgus.com/files/openwrt/arpnat/patches/08-config-hwaddr.patch

    mesh@LPtelecom:~/arpnat-test$ patch -p1 192.168.251.100 (openwrt fonera) reply “Destination Port Unreachable”

    Please, Angus, help me.
    Thank you again.

  4. James,

    I can’t see any problems with your setup as shown.

    What’s the output of

    arp -n 192.168.251.100
    arp -n 192.168.251.251

    …when run from your laptop?

    Also, what does the routing table on each machine look like?

    This same setup works fine when you use the image that I built, correct?

  5. Angus,

    Yes, your image work fine (same setup). My arp table report correct MAC:
    192.168.251.100 ether 00:12:cf:81:7e:34 C eth0
    192.168.251.251 ether 00:15:6d:aa:86:25 C eth0

    My laptop no have default gateway. Please, how do you do building your images ?

    Thanks

  6. James,

    If that’s the arp table on your laptop, then it’s wrong. The MAC for 192.168.251.251 (the router) should be spoofed by the Fonera to be the same as the Fonera’s MAC. Otherwise, ARP NATting can’t work.

    The Fonera’s ARP table should be the only ARP table with correct MACs for everything, because it “knows” both sides of the ARPNAT bridge.

    You could try ‘tcpdump -i br-lan -v arp’ on the Fonera, and the equivalent on your Laptop, and watch where the ARP requests are going from/to.

    The images I supplied were built using the .config file that’s in the patches directory as config-arpnat-fonera. Try downloading that file, ‘make clean’, copy it as .config in the root OpenWRT build directory, then do a ‘make’ and see if the resulting image turns out. If it does work, then after thart you can try ‘make menuconfig’ and add whatever options you need on top of that.

    - Angus

  7. Hi Angus

    That’s a great work.

    I have a question about it. Is there any patch to work with ath5k driver instead of madwifi?

    Thanks in advance

    oMatrix

  8. Hi oMatrix,

    Good question. You can probably adapt the patch which is there. The only madwifi-specific part is the client-bridge hotplug script, which picks up “client-bridge 1″ and then run ebtables with the correct argument for the specific interface.

    You could either adapt that script for ath5k, or just put hard-coded ‘ebtables’ calls into your rc.local or a similar startup file. I think this section is all you really need:

    ebtables -t nat -F PREROUTING
    ebtables -t nat -F POSTROUTING
    ebtables -t broute -F BROUTING
    ebtables -t nat -A PREROUTING –in-interface $INTERFACE -j arpnat –arpnat-target ACCEPT
    ebtables -t nat -A POSTROUTING –out-interface $INTERFACE -j arpnat –arpnat-target ACCEPT
    ebtables -t broute -A BROUTING –in-interface $INTERFACE –protocol 0x888e -j DROP

    Where you replace $INTERFACE with your wireless interface.

    Of course, you still need to apply the rest of the patch – you need the part which adds arpnat support to ebtables.

    Hth.

    - Angus

  9. it works well on our test but few glitches are seen.
    1) when the link is flapping. Its on the margin of loosing we saw mac address of the client changing in desktop (arp -a ) sometimes of ath0 and sometimes of eth0 which make the station unreachable.
    2) broadcast arp from wireless interface from inside the station. Like pinging anything on the wireless side crashes the station and it reboots.

  10. Normunds, thanks for taking the time to report back.

    I’ve been running with this setup at home since March now, and like you it’s generally been good. If by “link is flapping” you mean the wireless then I’ve seen similar symptoms – sometimes it seems to take a while for the ARPed clients to recover from a wireless network restart. The situation you describe makes perfect sense, if that’s the case.

    I haven’t seen the broadcast ARP crash situation you’re talking about, though. Can you describe it in detail for me?

    Cheers,

    - Angus

  11. Hi angus

    I saw your comment on openwrt website (https://forum.openwrt.org/viewtopic.php?pid=116595).

    I have a problem with ath5k and wds mode. My setup is something like yours, just my AP and client systems are like each other. Now when I use wds option without encryption or encryption without wds everything work fine. but when I use wds and encryption together, my link is connected but I can’t send or receive any thing. even I can’t ping AP from client side console or vice versa.

    I thought may be you know anything about this problem.

    Thanks in advance

    neo

  12. Hey angus,
    great work … i want to mention that you must be careful with the kernel modules (kmod) form the default repository … you must compile all kmods with the above kernel config. with the default kmods the userland applications will segfault …

  13. better late than never. I lost this site. ;)
    sta…….. ap

    pinging from station to ap reboots the station running arpnat.
    Had eric developed fresh patch for openwrt with newer kernels 2.6.32 onwards ? If yes then plz provide me the link.
    One instance in installed scdp cisco discovery protocol on br-lan and the station started rebooting in loops as soon as scdp sends broadcast to network.

  14. normunds: sorry, since your previous comment I’d actually given up on arpnat and gone back to WDS, which I’m quite happy with now.

    I think broadcasts are never handled particularly well by the arpnat hacks, that’s one of the reasons why I moved back (DHCP broadcast/response was not quite working correctly.)

  15. Orginaly ARPNAT was created by Kestutis Barkauskas in November 2005.
    Rewritten by the same guy and Kestutis Kupciunas in November 2006 from ubnt.com. There were about 7 major fixes/changes since that.
    Last modifications on original sources were done by Darius Ivanauskas in May 2011.
    Since first public release there are a lot of different clones of original code.

    ARPNAT becomes kind of obsolete feature. Almost all APs support WDS now.

  16. Hello,

    I’m using the latest Gargoyle build 1.5.10 and I want to run your MAC address DHCP patch on my wr741nd. Is this possible? Can you help me on how to do it?

    Thanks

  17. Hi there, Your work about arpnat is great even now in 2014 it can be very helpful on some old devices like WA5210G
    I successfully added arpnat rebuild it from gargoyle distribution of kamikadze openwrt
    All works like charm.
    But.
    I have one question if there any possibility to NAT PPPOE request with arp nat. Because my ISP is only with PPPOE server and the request do not nat with arpnat
    Is there some solution (i know for rp-pppoe-relay but it is slow)

    • Hi Jordan,

      I don’t know of any solution. You can’t ARP-NAT PPPoE specifically because it doesn’t use ARP (no IP addresses.) You could probably do something similar with ebtables where you forward the PPPoE discovery frames and response and then forward the PPPoE frames, but I don’t know how.

      If you can use it I really recommend WDS if possible, much less hacky.

      - Angus

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>