802.11b runs at 11Mbps. 802.11g was the new standard, fully backwards compatible with 802.11b, and runs at 54Mbps.
As for switching ethernet cables, no, that defeats the purpose of having a router. A router splits your internet connection between multiple devices. It will have 4 ports on the back, an uplink port, and a WAN port (though, the new Linksys 802.11g one I bought didn't have an uplink port anymore).
Uplink is used when you want no networking done at all. So if you connected it to your modem, your modem would read that you have say 3 computers hooked up to it. It's good when you're connecting to another device, if you have multiple routers and hubs. It also disables port 1.
WAN is what you're looking for. Connect the WAN port to your modem. Connect all your devices to 1-4, as well as setup the wireless network, and you'll be set. Everything will be on a LAN, and everything will have access to the internet at the same time.
My recommendation? Get the 802.11g since it is faster. Will you need 54Mbps online though? Of course not. But I have no experience with D-Link, so personally, I'd buy the Wireless B. Another option, get the 802.11b for right now, $5 is change, and wait for the prices to drop and then buy one that support 802.11a/b/g. 802.11a is still 54Mbps, but operates at 5.0Ghz instead of 2.4Ghz so the signal is a little better. That way you have better support for future devices, such as the wireless adapter for Xbox 360 which supports a/b/g, any laptops you buy (Think Different), and who knows what Nintendo and Sony will use.
P.S. - If you get the LinkSys, don't install any of their software, despite what they may want to convince you, it's not necessary (they don't even make it for Mac). Open your web browser, go to
http://192.168.1.1 user: *blank* pass: admin and go from there.