Assuming you're dealing with the US, addresses are fairly public. The only real privacy concern is matching addresses to individuals -- and I don't think that an email address alone (at least, from what you've shared with us of your schema) is enough to present that risk. I work for SmartyStreets where we handle a lot of address data (standardizing, verifying, etc).
Consequently, we maintain a database that stores millions of official USPS-supplied address records. We take privacy seriously. I don't think, though, that there's a need in your case to actually encrypt all of those fields. As Jason said, if you later wanted to query that data for a search, performance will probably nosedive unless it's done right.
By the way, if you're collecting addresses (especially for a customer database), you will want to verify that they are correct; I suggest a service like LiveAddress. There's also services to process existing databases for you.