Wednesday, October 24, 2007

Google IMAP will be a Yahoo mail killer!

If you haven't heard already, Google with be and is offering IMAP for their Gmail service! You might not have it available on your account immediately, but it should be within a few days or a week.

I'm one of the lucky ones that already has IMAP set up. I tried it out in Thunderbird and the setup was quite smooth. The only problem I ran into is that some of my old chats came up with a timestamp of 2:33am this morning. So, they were out of order and weird, but it wasn't that big of an issue.

The real point I wanted to make in this post is that this will be a killer (imo) for Yahoo mail. Back in the day when I signed up for Yahoo mail, they were on the leading edge for features and interface. I remember when they moved to the interface they have which they now call their 'classic' interface, and I have saved HTML pages that show what their REAL classic interface looks like.

But until now I have not had a reason to move to Gmail. While I do have a Gmail account, I don't use it as my primary account because there was no real advantage to switch to it. Some of the features might be nicer and sexier, but webmail is still webmail. Now that Gmail offers IMAP along with a host of other features that set it apart from other webmail clients, there is a huge reason to move to Gmail. For this reason, Yahoo will lose many 'techie' customers it hasn't already lost unless it offers the same feature.

In case you're curious, here are the settings:

Incoming: imap.gmail.com
Outgoing: smtp.gmail.com
Username: username@gmail.com
SSL: yes

Have fun!

Saturday, October 20, 2007

LDS General Conference Podcast RSS feed

LDS General Conference audio and video is available via podcast!

After watching conference the LDS General Conference at the beginning of October, I really wanted to download all the talks via an RSS feed. I didn't see one available right away, so I considered making my own. Fortunately, I found that an official podcast does exist. Its just not publicized well by the Church.

MP3 audio: http://feeds.lds.org/ldsgccomplete_eng

Video: http://feeds.lds.org/ldsgccomplete_eng_mp4

This should get you the most recent General Conference. If you haven't used RSS before, you just need to give either (or both) of the above feed URLs to a podcast downloader (such as iTunes). The downloader will download all the current talks, and theoretically, will download the new talks next conference.

As an unrelated side note, feeds.lds.org is actually hosted by feedburner. Thanks to ldswebguy for originally posting these podcast feeds.

Friday, October 19, 2007

Verizon FIOS and running a Linux operating system

Despite the well known fact that Verizon Can't Do Math, I have to say that I am surprised to find that their FIOS service is pretty good. I've had it for over a month, and haven't noticed it go down or have any speed problems.

More interestingly, I've actually found that they are pretty Linux-friendly. Actually, Linux-friendly might be too strong - lets just say that they have not locked me into using Windows for their service. I did have to use activatemyfios.verizon.net, which has a Windows/OS X only Firefox extension. But other than that, I haven't felt the vendor lock-in blues.
Here are some good points:

  • As far as I can tell, Verizon only blocks port 80. I run both SSH and HTTPS from my home box.

  • Once I spoofed the correct MAC address, I was able to use my own non-Verizon router

  • There is no PPPoE authentication or the like

  • My IP address has not changed since I started the service

  • Latency is very low (about 12ms to Google)

  • Service has not gone down


I don't use their phone or TV services, so I can't comment on those. My guess would be that services like Vonage or Skype would work pretty well on my FIOS connection.

Bad points:

  • The backup battery is supposedly pretty weak. It only provides voice service during a power outage, not Internet. For that, you'd have to use a UPS. Fortunately, I haven't had the power go out.

  • It took forever to get the service installed. Even though the previous house owner already had FIOS installed, they couldn't come out for 2.5 weeks.


Verizon does pretty well here. So if you're thinking about switching - I would recommend it.

Thursday, October 18, 2007

How to create a lookup table in Ruby on Rails

I'm a big fan of the Rails way, but sometimes the simple things get you. I like to set up "lookup tables" in Rails, ie. tables in the database that hold commonly used, (often) fixed values. For example, if I have a Car model, I might want to define a Car type model. This would have two purposes:

  1. Give me the ability to associate a type with the Car model (ie. Car.type => 'sport')

  2. Pre-load common values into the database (ie. sport, coupe, sedan, etc)


New version of Rails available

If you didn't know already, there is a new version of Rails out there.

And I'm not referring to the Rails 2.0 preview. This is Rails 1.2.5, which contains fixes for a JSON XSS (cross-site scripting) vulnerability. I'm not horribly familiar with the details, but the site does say that you don't have to worry about it if you're not using JSON. Probably a good idea to upgrade anyway.

Monday, October 15, 2007

Who the heck is 142.166.3.122 and 142.166.3.123 (radianrss-1.0)?

I've been perusing through my logs lately and found the user agent "radianrss-1.0" numerous times. I'd never heard of this program (maybe an RSS reader?), so I did a google search. The only commentary I found was this post speculating that 142.166.3.123 was possibly involved in the compromise of katester.net.

Interestingly enough, the rest of the search results are the traffic statistics pages of various Wordpress blogs around the Internet. There doesn't seem to be a clear answer for what "radianrss" is, or why 142.166.3.122-123 is constantly indexing all of my blog posts.

Sunday, October 14, 2007

If you're not already, START USING DD-WRT!


==========================================================

____ ___ __ ______ _____ ____ _____
| _ \| _ \ \ \ / / _ \_ _| __ _|___ \|___ /
|| | || ||____\ \ /\ / /| |_) || | \ \ / / __) | |_ \
||_| ||_||_____\ V V / | _ < | | \ V / / __/ ___) |
|___/|___/ \_/\_/ |_| \_\|_| \_/ |_____|____/

DD-WRT v23 SP2
http://www.dd-wrt.com

==========================================================

I installed dd-wrt on my Linksys WRT54g because I was having problems with the Sveasoft Alchemy firmware. The wireless was dropping constantly, which can be real annoying when you're trying to do a video call on Skype. My suspicion was that the firmware image was too large and too many services were trying to be run at once, and the unit was running out of memory.

Once long ago I had problems with the Alchemy firmware getting an IP address from my DSL connection. I managed to get syslog working (shouldn't have been hard, but it didn't work well with Alchemy) and the errors seemed to point to the fact that the unit was running out of memory. It would eventually work, it just took 20 minutes to get an IP address after it booted. Then it would stay up for months.


dd-wrt status page


I guess I've just been out of the loop with regard to the newer firmwares. dd-wrt has much of the same basic functionality in their 'mini' version as the Sveasoft Alchemy fully featured version did. And to think that at point I paid for Alchemy because I wanted a more featureful, stable firmware :|

The wireless issues have gone away, and the web interface has a much more polished feel to it. I like logging into the router just to look at the interface and see the random stats. It also gives you status on the signal strength of the wireless clients, which can be very useful!

The dd-wrt firmware is famous for being able to do client mode bridging, i.e. act as a transparent wired-to-wireless bridge. I'm looking forward to trying that out some day.

By the way, although my unit will broadcast at a power of 251mW, I read on the dd-wrt wiki that 84mW is ideal for most hardware in terms of getting the best signal-to-noise ratio. At 84mW, I get a SNR of somewhere between 30-40, so I feel like that is probably pretty accurate information, considering I probably only need a SNR of 10-15 to get a very good wireless signal.

Try out dd-wrt today!

Slow SSH logins in Ubuntu Feisty (7.04)

There are two major reasons (I know of) that can cause extremely slow SSH logins - anywhere in the range of 15-30 seconds. It doesn't seem like a long time, but it takes forever to wait that out sitting at the terminal.

Say no to GSSAPIAuthentication


For some reason, GSSAPIAuthentication is turned on by default in Ubuntu Feisty (7.04). If you ssh -v somewhere.com, you'll see this in the output:
...
debug1: Local version string SSH-2.0-OpenSSH_4.3p2 Debian-8ubuntu1
debug1: An invalid name was supplied
Cannot determine realm for numeric host address

debug1: An invalid name was supplied
A parameter was malformed
Validation error

debug1: An invalid name was supplied
Cannot determine realm for numeric host address

debug1: An invalid name was supplied
A parameter was malformed
Validation error

debug1: SSH2_MSG_KEXINIT sent
...

There are two ways to fix it. You can edit either /etc/ssh/ssh_config to include
GSSAPIAuthentication no

Or, you can edit ~/.ssh/config and add the same line. The only difference is that /etc/ssh/ssh_config will affect all systems users - probably a good thing.

Slow/broken DNS lookups


Each time you log into an OpenSSH server, it will typically perform a reverse-lookup on your IP address and store it in wtmp (try running the command "last"). If that DNS lookup times out, you'll be in for a big delay. Whats worse, OpenSSH has a habit of performing the lookup 3 or 4 times, further extending the delay to 30 seconds or more.

Its generally best to fix the source of the problem by fixing whatever problem exists in your DNS infrastructure. This could be as simple as pointing /etc/resolv.conf to the correct nameserver.

If you're really in a bind, you can pass the -u0 flag to sshd. Unfortunately, the sshd man page does not reveal any config file options to accomplish the same thing. But, the -u0 flag will stop sshd from performing any lookups that are not absolutely necessary as part of the authentication mechanism.

Saturday, October 13, 2007

Use ps2pdf to create PDFs from any Linux application

If you want to share a PDF of a web page or any other document with another person, PDF is usually the way to go. There are free PDF printers out there for Windows, but fortunately the necessary software is probably already installed on your Linux distribution. Just use ps2pdf.

First, print from your application (ie. Firefox) and make sure to check "Print to file":

Print to file

Press Print, save the file as filename.ps , and then run the following command:

$ ps2pdf filename.ps

The resulting file will be filename.pdf.

This is also a good way to convert old PS files (which can get pretty large) to the smaller PDF format. And, if for some reason you want to go the other direction, you can always use the pdf2ps utility.

Friday, October 12, 2007

Rails' ruby script/console has tab completion!

While I recently found out about Ruby's 'methods' method, I also found that Rails' "ruby script/console" will do tab completion:

>> a = Array.new
=> []
>> a.
Display all 157 possibilities? (y or n)
a.fetch a.method a.slice
a.__id__ a.fill a.methods a.slice!
a.__send__ a.find a.min a.sort
a.all? a.find_all a.nil? a.sort!
a.any? a.first a.nitems a.sort_by
a.assoc a.flatten a.object_id a.split
a.at a.flatten! a.pack a.subclasses_of
a.b64encode a.freeze a.partition a.sum
a.blank? a.frozen? a.pop a.suppress
a.class a.gem a.pretty_inspect a.taguri
a.clear a.grep a.pretty_print a.taguri=
a.clone a.group_by a.pretty_print_cycle a.taint
a.collect a.hash a.pretty_print_inspect a.tainted?
a.collect! a.id a.pretty_print_instance_variables a.to_a
a.compact a.in_groups_of a.private_methods a.to_ary
a.compact! a.include? a.protected_methods a.to_default_s
a.concat a.index a.public_methods a.to_formatted_s
a.copy_instance_variables_from a.index_by a.push a.to_json
a.daemonize a.indexes a.rassoc a.to_param
a.dclone a.indices a.reject a.to_s
a.decode64 a.inject a.reject! a.to_sentence
a.decode_b a.insert a.remove_subclasses_of a.to_set
a.delete a.inspect a.replace a.to_xml
a.delete_at a.instance_eval a.require a.to_yaml
a.delete_if a.instance_exec a.require_gem a.to_yaml_properties
a.detect a.instance_of? a.require_library_or_gem a.to_yaml_style
a.display a.instance_values a.respond_to? a.transpose
a.dup a.instance_variable_get a.returning a.type
a.each a.instance_variable_set a.reverse a.uniq
a.each_index a.instance_variables a.reverse! a.uniq!
a.each_with_index a.is_a? a.reverse_each a.unloadable
a.empty? a.join a.rindex a.unshift
a.enable_warnings a.kind_of? a.select a.untaint
a.encode64 a.last a.send a.values_at
a.entries a.length a.shift a.with_options
a.eql? a.load a.silence_stderr a.yaml_initialize
a.equal? a.map a.silence_stream a.zip
a.extend a.map! a.silence_warnings
a.extend_with_included_modules_from a.max a.singleton_methods
a.extended_by a.member? a.size

Unfortunately the same thing doesn't work for the irb. Of course you can always

a.methods.sort

to see the same information.

Wednesday, October 10, 2007

Convert has_and_belongs_to_many to a has_many :through association

So there are plenty of resources out there to learn how to use has_many :through associations.

I followed them over and over again but couldn't get my code to work. I knew I had the basic structure setup correctly, since the examples are pretty straightforward, and the concept is not difficult. My has_and_belongs_to_many code originally looked like this:

class Soda < ActiveRecord::Base
has_and_belongs_to_many :distributors
end

class Distributor < ActiveRecord::Base
has_and_belongs_to_many :sodas
end

Of course there was also a many-to-many join table migration:

class DistributorsSodasJoinTable < ActiveRecord::Migration
def self.up
create_table :distributors_sodas, :id => false do |t|
t.column :soda_id, :int
t.column :distributor_id, :int
end
end

def self.down
drop_table :distributors_sodas
end
end

This works quite nicely:

>> Soda.find(1).distributors
=> []

Later found that I needed to add attributes in the join table to associate extra fields on the Distributors <-> Sodas relationship. has_and_belongs_to_many does not have a Rails way to access those extra fields in the join table. I've successfully done it through SQL, but much guilt and remorse lead me to finally learn has_many :through.

This was my best initial attempt:

class Soda < ActiveRecord::Base
has_many :distributors_sodas
has_many :distributors, :through =>; :distributors_sodas
end

class Distributor < ActiveRecord::Base
has_many :distributors_sodas
has_many :distributors, :through => :distributors_sodas
end

class DistributorsSodas < ActiveRecord::Base
belongs_to :soda
belongs_to :distributor
end

All goes well until I try to do a quick test:

>> Soda.find(1).distributors
NameError: uninitialized constant Soda::DistributorsSoda
...
from (irb):4

Umm... what? I never tried to instantiate an object of the type Soda::DistributorsSoda. Instead, I was simply trying to use the DistributorsSodas ActiveRecord object, right?

It turns out that has_many :through (apparently) can't handle using the join tables created by has_and_belongs_to_many. Its just a naming issue - has_many :through will work fine using a one-to-many join table like distributor_sodas (note the missing 's' on distributor). If you need a many-to-many join, you have to rename the table to fix the (pluralization?) problem. I deleted the DistributorsSodas model and created the Store model.

class Soda < ActiveRecord::Base
has_many :stores
has_many :distributors, :through => :stores
end

class Distributor < ActiveRecord::Base
has_many :stores
has_many :sodas, :through => :stores
end

class Store < ActiveRecord::Base
belongs_to :soda
belongs_to :distributor
end

This proved a much better result:

>> Soda.find(1).distributors
=> []

In the end, the association naming convention actually make more sense. I was bummed to have to change the table/model names though.

Please comment if you know how to create the association without changing the model name.

Monday, October 8, 2007

Use Mozilla Firefox under WINE to reach those Windows-only sites

Being denied based on operating system

Its long been known that you can use something like the User Agent Switcher to make it appear like you're using a different operating system than you really are. This can be useful on some websites that insist you need to run Windows - when in fact Linux or practically any other operating system will work fine.

But for those sites that are actually telling the truth, you can often run Firefox under WINE with very good results. Just head over to www.getfirefox.com and download the Windows binary. The site actually does its own OS detection, so you'll probably want to choose "Other Systems and Languages." Once you have it, make sure you have WINE installed. I also installed the MS truetype fonts because it looks terrible otherwise.

# apt-get install wine msttcorefonts

And then start the installer

$ wine Firefox\ Setup\ 2.0.0.7.exe

You'll go through the normal install process, and when you're finished you'll see a new Firefox icon on your Linux desktop:

Windows Firefox on my Linux desktop

Double-click it and you should be good to go! If you want to visit a site with audio, you'll want to run winecfg, visit the Audio tab, choose your settings (I just left the defaults) and click Apply. Otherwise you might run into some issues with Firefox crashing. If its any consolation, Firefox will crash on certain sites powered by Move Networks in Windows, too, if there is no audio driver installed. This is probably related to the Firefox extension you must install to view the site.

Sunday, October 7, 2007

Apt-get DOES have an option for automatic security updates

I recently wrote about the cron job that I run to keep my Debian and Debian-like servers up to date automatically.

It turns out that apt-get (or libapt, more specifically) has a mechanism to automatically download and install security updates. This will work across package management applications (apt-get, synaptic, aptitude, etc) and is very simple to setup. Edit /etc/apt/apt.conf.d/10periodic to include the following:

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::AutocleanInterval "0";
APT::Periodic::Unattended-Upgrade "1";


Your distribution should already have a cron job (usually in /etc/cron.daily/apt) that runs every day and updates as per these configuration options. To see the entire config, run

$ apt-config dump

For those interested, this configuration setting is what Synaptic sets when using its "Install security updates without confirmation" option.

Thanks to g for pointing this out.

Saturday, October 6, 2007

The new Ubuntu is coming already

If you didn't already know, Ubuntu 7.10 Gutsy Gibbon will be coming soon!



Maybe I'll finally get suspend support for my dual core Dell XPS desktop?

Any guesses if the Ruby and Rails packaging will be any better?

Thursday, October 4, 2007

Using syntax enable and other vim commands in Ubuntu and Debian

So I ran into the strangest thing today. Vim complains because I tried to use 'syntax enable' in my .vimrc.

$ vim myfile.rb
Error detected while processing /home/solipsistic/.vimrc:
line 1:
E319: Sorry, the command is not available in this version: syntax enable
Press ENTER or type command to continue

Apparently Debian-like distributions don't package the full version of vim by default. They pack a 'lite' version.
# apt-get install vim

And you'll be rolling again.

Tuesday, October 2, 2007

Installing Mongrel on Ubuntu Feisty with rubygems

When I recently re-installed Ubuntu 7.04 (Feisty), I found that I couldn't build the native fastthread gem (a required dependency). If you get this error:

Building native extensions. This could take a while...
extconf.rb:1:in `require': no such file to load -- mkmf (LoadError)
from extconf.rb:1


ERROR: While executing gem ... (RuntimeError)
ERROR: Failed to build gem native extension.
Gem files will remain installed in /var/lib/gems/1.8/gems/fastthread-1.0 for inspection.

then you need the ruby development libraries and the build-essential package. I've found that these commands will make sure mongrel (and rails) is installed correctly:

$ sudo apt-get install ruby1.8-dev rubygems build-essential rails
$ sudo gem install mongrel --include-dependencies

Monday, October 1, 2007

Using the Ubuntu Feisty Live CD as your primary OS

I had one of the most dreaded computer woes happen last week - my hard drive crashed.

Well, it didn't crash completely. I have about a 70% boot rate. It makes a load noise like the bearings aren't well lubricated when the drive won't boot. I figured it was better (and less frustrating) to stop using the drive and just wait until the new hard drive arrived.

Luckily, I use unison over SSH to backup my files, so everything was already mirrored on my Debian file server at home, even though I was across the country when the failure occured. I've reminded myself that I needed to have my data backed up if something catastrophic ever happened to my laptop. In my mind the scenario was that my laptop was going to be run over by a truck (gotta think big), but I knew the hard drive crash was probably inevitable. Unison is a good tool to sync two file systems, however it does work best if you manually sync. If you want something to automatically back up files, set up rsync as a cron job on your system.

Coping with the problem


At first, I thought I should just go without my laptop for a few days (oh the horror!). Newegg ships fast, right? I should have known myself better than that. I immediately started thinking of how to run my system with a broken hard drive. Naturally, the idea of using a Linux Live CD came to mind.

To preserve the old hard drive, I physically removed it from the laptop. After that, I remembered I had ordered some Ubuntu 7.04 pressed CDs a while back so I popped one in.

To my surprise, Ubuntu 7.04 (Feisty) as a Live CD rivals my normal hard drive install in terms of performance! The bootup time is quite slow, but once it has started, it runs nicely. Since I spend a lot of time using Firefox, I don't experience the spin-up-spin-down as often as I remember in previous Live CD usage. I think Feisty must also do a good job of caching programs in RAM when you first access them because I hardly ever hear the CD drive spin up unless I'm starting a new program.

The other nice thing is that the laptop fan almost never turns on. The old hard drive ran somewhat noisy and warm, causing the fan system to kick on about 75% of the time. I've really enjoyed the cool, quiet performance of running on a Live CD. The laptop is absolutely silent.

The only real obstacle I've faced is that I can't shutdown my laptop unless I want to spend time re-setting up my Thunderbird profile and re-installing Firefox extensions. I guess you can't have everything.

Why not switch completely?


If this setup works so well, I thought to myself, why not just switch to always using a Live CD and save $60 on a new hard drive? After thinking about it, I've decided that I would be willing to switch if a few conditions were met:

  1. A decent amount of persistent storage were possible

  2. I could install updates and new software without losing them each time I reboot

  3. Most or all of the OS could be loaded in RAM at boot


I could probably accomplish #1 with a USB drive and a persistent home directory. I think #2 would be harder with the USB drive - like what happens when you move between systems with the same flash drive - but still possible. In the end, I think I would be happiest by buying a Compact Flash to IDE adapter and a large (16GB or more) Compact Flash card. These are nice because you get the cool, quiet operation and persistent operation. I considered doing this instead of getting a new hard drive, but in the end I figured I would get much more storage but the price (80GB @ $60 or 16GB @ $150 or more). In the future I think many laptop hard drives will be completely flash based memory.

As for #3, I wish that Ubuntu Feisty had the ability to use the "toram" kernel boot option and load the OS into RAM during boot. I've done this with Knoppix CDs in the past. The speed increase is amazing. Granted, less RAM is available for program execution, but RAM is cheap enough these days that the productivity gain is worth the extra couple of dollars.

All in all, I give the Ubuntu Feisty Linux Live CD a surprisingly high thumbs up for everyday use.

Silly Verizon and their creepy new router

I recently signed up for the Verizon FIOS service. Its nice, but probably not in the way you'd think. I only have 5 mbps down / 2 mbps up, which is the basic plan, and in my opinion, the only plan that makes financial sense. I just can't see myself paying $180 per month for home internet service.

The nice part about the service is the latency. I have a 12ms ping to Google. I doubt most people get that at home on their cable or DSL lines. The best I've ever had at home was 40ms. At any rate, the low latency helps web pages load that extra bit faster (and it is noticeable).

The one thing about their service is their router. Apparently, I'm not the first one that thinks it is a little creepy. Verizon has customized it to provide their IPTV (cable) service as well as perform the normal router functions. I decided to use my Linksys WRT54G instead.

If you want to use their IPTV services, I'm willing to bet that you can chain their router behind yours. Just make sure to either change your router's DHCP subnet to something other than 192.168.1.0/24, or change the Verizon router to use a different subnet (ie. 192.168.2.0/24). Most consumer routers will puke if you try to give them a non-routable IP address on their WAN interface in the same subnet as the one they are supposed to give to their own clients. Rightly so, I guess. Please comment about your experience with chaining the Verizon router below your own.

While setting up my own router, I noticed a few things:

  1. Verizon sets up your router with a random ESSID (network name) and WEP encryption. My tech wrote down the WEP key when she installed the service.

  2. Even if you do stick with the Verizon router, please switch to WPA. WEP is easy enough to crack and won't keep your data secure from a moderately skilled attacker.

  3. The default login credentials for the router are admin/password. I've also heard some people having success with using "password1" as the password. Please change this the first time you log in.

  4. Verizon uses MAC filtering, so you must spoof/clone the MAC on your router to match the Verizon router.

  5. There is a MAC address printed on the outside of the router. Unfortunately, this is NOT the MAC address used by the WAN interface on the router. Instead, you must log into the router and choose the "System Monitoring" section (upper-right corner). This will give you what they call a "Broadband MAC Address."


Verizon Router Broadband MAC

Once I used the correct MAC address, I was able to get onto the FIOS network using my own router with no problems. They say your own router may not perform as well as theirs, but I didn't notice any difference.

One last thing: You might notice that each time you type a character in the password field, several characters will appear. A brief look at the code suggests that they are hashing the data as you type. While it is incredibly annoying to have those characters randomly appear, at least they are using BSD-licensed code from a reputable source. Thank you Paul Johnston!