No idea about bicycles but local price for a motorbike is $5 a day. Don't bother with your bike if you get this in time. LyLy guesthouse is run by a German guy, Paul who rents out rooms as cheap as you can get. Air con rooms are usually full but Dec/Jan are cool enough to stay in a fan room for cheap. There is a kitchen that you can use and beer fridge that you have access to where you keep track of your own bill.
Buses to Mui Ne from Saigon are easy enough to find. If you go to the backpacker area, Pham Nhu Lao street has tons of travel companies. All charge the same price.
As for your visa issue... even though you are visa exempt for two weeks... getting an extension will cost you more (specially being in Mui Ne) than getting a one month visa on arrival. Getting a new visa from a 2 week exemption will cost $60 for a one month in Saigon. You're looking at more money outside of Saigon. Check out www.vietnamvisacenter.org
for a visa on arrival, they're kite friendly.