/usr/share/doc/rssh/examples/mkchroot.sh /var/chroot

It created most of the files, but didn’t include the right libraries. Here’s the minimum file listing I’ve been able to use:

libnsl was the tricky one that I found by trial and error. I installed bash inside the chroot and tested running each binary. Everything runs without libnsl, but its required if you want it to work with rssh.

$HTTP["remoteip"] != "192.168.1.0/24" {
  $HTTP["url"] =~ "^/~" {
    url.access-deny = ( "" )
    dir-listing.activate = "disable"
  }
}

This allows any host with an IP in the 192.168.1.1-254 range view any URL that begins with /~username. It denies everyone else with a 403 – Forbidden message. Note that for some reason, listing the conditions in the opposite order (url first, remoteip second) did not produce the correct results.

Finally, make sure that you have both the appropriate modules enabled:

server.modules += ( "mod_access", "mod_userdir" )

More interestingly, I’ve actually found that they are pretty Linux-friendly. Actually, Linux-friendly might be too strong – lets just say that they have not locked me into using Windows for their service. I did have to use activatemyfios.verizon.net, which has a Windows/OS X only Firefox extension. But other than that, I haven’t felt the vendor lock-in blues.
Here are some good points:

• As far as I can tell, Verizon only blocks port 80. I run both SSH and HTTPS from my home box.
• Once I spoofed the correct MAC address, I was able to use my own non-Verizon router
• There is no PPPoE authentication or the like
• My IP address has not changed since I started the service
• Latency is very low (about 12ms to Google)
• Service has not gone down

I don’t use their phone or TV services, so I can’t comment on those. My guess would be that services like Vonage or Skype would work pretty well on my FIOS connection.

Bad points:

• The backup battery is supposedly pretty weak. It only provides voice service during a power outage, not Internet. For that, you’d have to use a UPS. Fortunately, I haven’t had the power go out.
• It took forever to get the service installed. Even though the previous house owner already had FIOS installed, they couldn’t come out for 2.5 weeks.

Verizon does pretty well here. So if you’re thinking about switching – I would recommend it.

Interestingly enough, the rest of the search results are the traffic statistics pages of various Wordpress blogs around the Internet. There doesn’t seem to be a clear answer for what “radianrss” is, or why 142.166.3.122-123 is constantly indexing all of my blog posts.

[evian]$ grep "142.166.3.123" access.log
142.166.3.123 - - [15/Oct/2007:01:02:23 -0700] "GET /2007/10/14/if-youre-not-already-start-using-dd-wrt/ HTTP/1.1" 200 14022 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:01:52:14 -0700] "GET /feed/atom/ HTTP/1.1" 200 36826 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:03:36:42 -0700] "GET /feed/atom/ HTTP/1.1" 200 36826 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:04:02:18 -0700] "GET /2007/10/14/if-youre-not-already-start-using-dd-wrt/ HTTP/1.1" 200 14023 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:06:23:53 -0700] "GET /2007/10/06/the-new-ubuntu-is-coming-already/ HTTP/1.1" 200 10915 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:07:00:07 -0700] "GET /2007/10/07/apt-get-does-have-an-option-for-automatic-security-updates/ HTTP/1.1" 200 12223 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:07:01:49 -0700] "GET /2007/10/14/if-youre-not-already-start-using-dd-wrt/ HTTP/1.1" 200 14023 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:07:57:22 -0700] "GET /2007/10/08/use-mozilla-firefox-under-wine-to-reach-those-windows-only-sites/ HTTP/1.1" 200 13046 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:09:06:34 -0700] "GET /2007/10/12/rails-ruby-scriptconsole-has-tab-completion/ HTTP/1.1" 200 16010 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:09:17:26 -0700] "GET /feed/atom/ HTTP/1.1" 200 13032 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:09:27:21 -0700] "GET /feed/atom/ HTTP/1.1" 200 36826 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:10:02:07 -0700] "GET /2007/10/14/if-youre-not-already-start-using-dd-wrt/ HTTP/1.1" 200 14022 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:10:17:18 -0700] "GET /2007/10/14/slow-ssh-logins-in-ubuntu-feisty-704/ HTTP/1.1" 200 12895 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:12:46:43 -0700] "GET /2007/10/13/use-ps2pdf-to-create-pdfs-from-any-linux-application/ HTTP/1.1" 200 11739 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:13:01:23 -0700] "GET /2007/10/14/if-youre-not-already-start-using-dd-wrt/ HTTP/1.1" 200 14022 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:13:03:56 -0700] "GET /2007/10/14/slow-ssh-logins-in-ubuntu-feisty-704/ HTTP/1.1" 200 12882 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:14:32:48 -0700] "GET /feed/atom/ HTTP/1.1" 200 36826 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:15:08:14 -0700] "GET /feed/atom/ HTTP/1.1" 200 36825 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:15:08:21 -0700] "GET /2007/10/14/slow-ssh-logins-in-ubuntu-feisty-704/ HTTP/1.1" 200 12894 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:15:08:26 -0700] "GET /2007/10/14/if-youre-not-already-start-using-dd-wrt/ HTTP/1.1" 200 14014 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:16:01:29 -0700] "GET /2007/10/14/if-youre-not-already-start-using-dd-wrt/ HTTP/1.1" 200 14023 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:16:03:03 -0700] "GET /2007/10/14/slow-ssh-logins-in-ubuntu-feisty-704/ HTTP/1.1" 200 12882 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:18:29:20 -0700] "GET /2007/10/14/slow-ssh-logins-in-ubuntu-feisty-704/ HTTP/1.1" 200 12901 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:18:29:21 -0700] "GET /2007/10/14/if-youre-not-already-start-using-dd-wrt/ HTTP/1.1" 200 14008 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:18:47:46 -0700] "GET /feed/atom/ HTTP/1.1" 200 36826 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:19:01:27 -0700] "GET /2007/10/14/if-youre-not-already-start-using-dd-wrt/ HTTP/1.1" 200 14028 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:19:02:41 -0700] "GET /2007/10/14/slow-ssh-logins-in-ubuntu-feisty-704/ HTTP/1.1" 200 12896 "-" "radianrss-1.0"
142.166.3.123 - - [15/Oct/2007:19:49:08 -0700] "GET /feed/atom/ HTTP/1.1" 200 36818 "-" "radianrss-1.0"

This is traffic just from today – less than 24 hours! Looking back a little further I also found

142.166.3.123 - - [14/Oct/2007:17:35:13 -0700] "GET /2007/10/14/slow-ssh-logins-in-ubuntu-feisty-704 HTTP/1.1" 200 450 "-" "Java/1.5.0_11"

which has decided to use Java 1.5 as its user agent string.

I’m curious to know why this IP address is retrieving all of my (and others’) Wordpress blog(s), and why so frequently. Its not like the page has changed between each retrieval. Have you found this IP address in your logs, the “radianrss-1.0″ user agent string, or anything else of interest?

Say no to GSSAPIAuthentication

For some reason, GSSAPIAuthentication is turned on by default in Ubuntu Feisty (7.04). If you ssh -v somewhere.com, you’ll see this in the output:

...
debug1: Local version string SSH-2.0-OpenSSH_4.3p2 Debian-8ubuntu1
debug1: An invalid name was supplied
Cannot determine realm for numeric host address

debug1: An invalid name was supplied
A parameter was malformed
Validation error

debug1: An invalid name was supplied
Cannot determine realm for numeric host address

debug1: An invalid name was supplied
A parameter was malformed
Validation error

debug1: SSH2_MSG_KEXINIT sent
...

There are two ways to fix it. You can edit either /etc/ssh/ssh_config to include

GSSAPIAuthentication no

Or, you can edit ~/.ssh/config and add the same line. The only difference is that /etc/ssh/ssh_config will affect all systems users – probably a good thing.

Slow/broken DNS lookups

Each time you log into an OpenSSH server, it will typically perform a reverse-lookup on your IP address and store it in wtmp (try running the command “last”). If that DNS lookup times out, you’ll be in for a big delay. Whats worse, OpenSSH has a habit of performing the lookup 3 or 4 times, further extending the delay to 30 seconds or more.

Its generally best to fix the source of the problem by fixing whatever problem exists in your DNS infrastructure. This could be as simple as pointing /etc/resolv.conf to the correct nameserver.

If you’re really in a bind, you can pass the -u0 flag to sshd. Unfortunately, the sshd man page does not reveal any config file options to accomplish the same thing. But, the -u0 flag will stop sshd from performing any lookups that are not absolutely necessary as part of the authentication mechanism.

The nice part about the service is the latency. I have a 12ms ping to Google. I doubt most people get that at home on their cable or DSL lines. The best I’ve ever had at home was 40ms. At any rate, the low latency helps web pages load that extra bit faster (and it is noticeable).

The one thing about their service is their router. Apparently, I’m not the first one that thinks it is a little creepy. Verizon has customized it to provide their IPTV (cable) service as well as perform the normal router functions. I decided to use my Linksys WRT54G instead.

If you want to use their IPTV services, I’m willing to bet that you can chain their router behind yours. Just make sure to either change your router’s DHCP subnet to something other than 192.168.1.0/24, or change the Verizon router to use a different subnet (ie. 192.168.2.0/24). Most consumer routers will puke if you try to give them a non-routable IP address on their WAN interface in the same subnet as the one they are supposed to give to their own clients. Rightly so, I guess. Please comment about your experience with chaining the Verizon router below your own.

While setting up my own router, I noticed a few things:

1. Verizon sets up your router with a random ESSID (network name) and WEP encryption. My tech wrote down the WEP key when she installed the service.
2. Even if you do stick with the Verizon router, please switch to WPA. WEP is easy enough to crack and won’t keep your data secure from a moderately skilled attacker.
3. The default login credentials for the router are admin/password. I’ve also heard some people having success with using “password1″ as the password. Please change this the first time you log in.
4. Verizon uses MAC filtering, so you must spoof/clone the MAC on your router to match the Verizon router.
5. There is a MAC address printed on the outside of the router. Unfortunately, this is NOT the MAC address used by the WAN interface on the router. Instead, you must log into the router and choose the “System Monitoring” section (upper-right corner). This will give you what they call a “Broadband MAC Address.”

Once I used the correct MAC address, I was able to get onto the FIOS network using my own router with no problems. They say your own router may not perform as well as theirs, but I didn’t notice any difference.

One last thing: You might notice that each time you type a character in the password field, several characters will appear. A brief look at the code suggests that they are hashing the data as you type. While it is incredibly annoying to have those characters randomly appear, at least they are using BSD-licensed code from a reputable source. Thank you Paul Johnston!

There was a definite speed boost! I have Verizon FIOS and the OpenDNS servers are more responsive than Verizon’s DNS servers. I’ve noticed that most pages now load in about 3/4 of a second (time to look up the DNS name, fetch the page, and render it). I use fasterfox to time it.

OpenDNS also provides some interesting reports. You can turn off the DNS logs if you like (privacy and all), but I found them useful. I have a cron job that runs every 5 minutes and fetches my mail via POP and mails it to another account. The cron job had done over 24,000 DNS lookups of the same IP address in the last 8 days. My router is supposed to be caching those requests (isn’t that why you run local DNS?), but its easy to see that caching is not happening.

Also, OpenDNS gives you the ability to block domains, and automatically will block phishing and pornographic sites if you want. This can be pretty useful, for kid-safe and normal-user-safe browsing.

I’ve been pretty happy with using the service so far – especially since it is free. The only thing I’ve noticed is that I haven’t seen any *nix update clients. Anyone found one that they like?

Update: here are some screenshots from their interface

Recent Activity

Unique Domains

Top Domains

Sample blocked domain

The ISP was First Step Internet. When I was in college, I used to live in this apartment complex that bundled Internet with their lease agreement. It was really a good idea, because it puts your Internet bill into your rent, so you have more predictable monthly expenses. It also forced your roommates to help pay for Internet. For some reason I always had roommates that didn’t want to pay because they didn’t need the Internet. And at $15 a month, you really couldn’t beat it since most other offerings were at least $25 per month. When you’re in college, $10 per month is a lot of money.

But the service from First Step Internet was terrible.

In the beginning, I was actually pretty excited. They came and wired all the rooms in the apartment complex with CAT5 cables, and we were eventually given the OK to plug in and use the Internet. There was no proprietary setup with a lot of ISPs like Verizon – just get an IP address from the DHCP server and you were connected.

It was initially opened up a week or two before the school year started. When I was first testing the line, I was happy with it since I was getting about 3 megs down and 1 meg up. While not the best, I definitely couldn’t complain for $15/3 roommates = $5 per month.

But as soon as everyone came back for the school year and plugged into their computers, the service came to a screeching halt. Here is one of the results I got from their speed test results:

Can you imagine only getting 85kbps to your ISP, and trying to share that among 3 people? Just think about the speed you would get to basically any site on the Internet. Now magnify that by about 10 times and you will get an idea for the pain that we felt while using their service. This screenshot was taken about 1am, so its not like their lines were busy. This was a good speed test. I’ve seen them 35kbps and below on their broadband lines.

There were many problems with their service, and I am going to list some of those that I experienced.

• Very slow link speeds (as seen above)
• Very high latency (150ms+ ping to Google.com was common)
• We were not given a public IP address. Instead I had a non-routable IP in the 192.168.x.y range. I talked with their technical support and they absolutely would not give me a public IP address or even forward a single port to my machine for SSH.
• They used default/weak passwords or vulnerable switches/routers. Someone from our apartment complex was obviously as frustrated as I was and hacked their equipment. Unfortunately they weren’t too smart and when First Step tracked them down they had their network privileges revoked.
• They subjectively blacklisted certain apartments for using too much bandwidth. When I sent them the screenshot above the tech confirmed my apartment was blacklisted, and that is why I was getting slow speeds. He then later told me that when they put in the bandwidth rules, they put them in backwards so that I had 85kbps down and something like 300kbps up. Not only did they severely limit the speed, they put the rules in backwards to make the service even more unbearable? What kind of ISP is this?
• Their switches limited bandwidth by switch port and NOT by throttling the connection to the remote site you were connecting to. So, if I wanted to share a file with my roommate or play a LAN game, I was still limited by the 85/300kbps rule.
• Even though First Step has a dedicated wireless link to my university, I still could not get under 200ms pings to the CS department Solaris servers. WTF?
• First Step was at their worst when I contacted them about a problem I knew at least a dozen people in the apartment complex were having. For some reason they couldn’t access Hotmail through Internet Explorer. It worked fine with Firefox – but Internet Explorer would simply stop loading the page after the initial log in. Since I run Linux I never experienced the problem myself, but I tried for weeks to get them to understand that many people that I knew were having this issue. I even considered starting a petition, having people sign it and sending it to them. Finally I sent them a packet dump detailing all of the problem frame numbers and showing the difference in behavior between IE and Firefox. They came back a few days later saying that their MikroTik router had a bug in its firmware.
• My entire apartment complex (200+ people) was served Internet by a single DSL line. This was a direct quote from one of their techs after a long conversation about why my service was so slow.

At the end of the day the worst part of everything was that their service was consistently bad. Their techs were about the skill level I would expect – able to help most home users with general Internet problems. But sometimes I did feel like they were trying to fool me by telling me that the Internet was fine and 200ms pings were normal.

Man, I’m glad I don’t even live within range of First Step Internet anymore. They are hazardous to your health.

So many times I’ve search the ‘net for a simple network uptime script. I always wanted to know the exact time the network went down (and came back up) so that I could track down the cause of the problem. Here is a dead-simple script that will do just that:

#!/bin/bash
## connect_test.sh ##
## This script continually pings an IP address / hostname and reports *only*
## when it is unable to reach the destination.
##
## Example: ./connect_test.sh www.google.com
 
if [[ $1 == "" ]];
then
        echo "Please provide an IP/host to ping"
        exit
fi
 
while [ true ]; do
        p=`ping -c 4 $1 > /dev/null 2>&1`
        if [[ $? != 0 ]];
        then
                d=`date`
                echo "Unable to reach $1 at $d"
                sleep 1
        fi
done

It works best if you open up multiple terminal windows and try to connect to different portions of your network simultaneously. For example:

./connect_test.sh [my neighboring workstation]
./connect_test.sh [my gateway]
./connect_test.sh [my ISPs DNS]
./connect_test.sh [random domain - ie. google.com or yahoo.com]

This way, you can determine exactly where the failure is AND you have a record of the time it occurred. Hope that helps!