Tip: Click here to scan for System Errors and Optimize PC performance

[ATNO/TW]

October 29th, 2008, 11:07 am

Anyone with some expertise in MS Exchange 2003 (server is 2003 SBS) - I could use some help.

I have two employees that work remotely. For over four years I've had them set up with mail forwarders to pass mail from their exchange accounts to their Roadrunner emails. All has worked fine with zero issues until Monday. Since then emails sent from within the company domain are forwarded to them just fine, but emails received from external sources appear to reach their accounts at our exchange server (I've verified by opening their accounts here that emails are being received at their domain accounts), however, instead of being forwarded to their Roadrunner accounts an NDR is being created and returned to the sender.

Here is the contents of the NDR:

Quote:

Reporting-MTA: dns;emailservername.alaron-nuclear.com
Received-From-MTA: dns;web63406.mail.re1.yahoo.com
Arrival-Date: Wed, 29 Oct 2008 13:09:43 -0400

Final-Recipient: rfc822;myuser@cinci.rr.com
Action: failed
Status: 5.6.0
Diagnostic-Code: smtp;550 5.6.0 Lone CR or LF in headers (see RFC2822 section 2.2)
X-Display-Name: My User

I have no clue why a problem is being indicated with the "Lone carriage return or line feed in headers", nor do I know what started it. I looked up RFC2822 section 2.2 and I'm a bit clueless what it's talking about:

Quote:

2.2. Header Fields

Header fields are lines composed of a field name, followed by a colon
(":"), followed by a field body, and terminated by CRLF. A field
name MUST be composed of printable US-ASCII characters (i.e.,
characters that have values between 33 and 126, inclusive), except
colon. A field body may be composed of any US-ASCII characters,
except for CR and LF. However, a field body may contain CRLF when
used in header "folding" and "unfolding" as described in section
2.2.3. All field bodies MUST conform to the syntax described in
sections 3 and 4 of this standard.

http://www.rfc-archive.org/getrfc.php?rfc=2822

Events that occurred immediately prior to this problem:

-Servers were shut down normally and safely Friday due to a planned power outage.
-Servers were rebooted normally Saturday morning. All appeared to be normal
-Sunday evening I noticed email was not working and the company website, located on the same server as Exchange, were not working. Remote access to the server was unavailable, and could not be pinged from other servers or workstations internally. I couldn't physically get to them until Monday morning.
-Monday checked the servers event logs.
-Application log displayed multiple MSExchangeAL Address List Synchronization errors Event ID 8331
-Server could could not be pinged from external computers nor did outward pings work.
-System log showed an IPSec error at the time the server was rebooted on Saturday Event ID 4292 with the following error message: "The IPSec driver has entered Block mode. IPSec will discard all inbound and outbound TCP/IP network traffic that is not permitted by boot-time IPSec Policy exemptions"
-The recommended action was disable the IPSec service and reboot the server, which I did.
-After reboot everything seemed restored to normal working order and no more errors were displayed in the logs.

But apparently this is when the forwarding issue with these two user accounts began. I'm not sure how it's related, and less sure how to fix it.

As always any help and advice is appreciated.

October 29th, 2008, 12:54 pm

http://www.tspec.net/support-outlookcrvulnerability.asp

[ATNO/TW]

October 29th, 2008, 1:34 pm

Thanks Don. That helps me understand the RFC better but doesn't really help me know what to do about it yet.

I do have one update, though. I thought to look at recent security updates, and there was one update that was done during server shutdown. KB958644 Security Bulletin MS08-067

Hoping that was the cause I uninstalled it. Rebooted and on a whim set IPSec back to Automatic and rebooted again. Removing that security update fixed the IPSec problem mentioned above. However it doesn't appear to have been the cause of the NDRs. I sent test emails from a yahoo account and I can already see that an NDR was generated for those.

However, I did install several updates on last week tuesday the 21st. I'm not certain the problem traces as far back as that, but it may, so will research those next and uninstall if necessary.

October 29th, 2008, 5:12 pm

There isn't much on it but I think you're on the right track looking at security updates as the possible cause.

Another post said showed the same error but a little different.

Diagnostic-Code: X-Postfix; host
xxxx.xxx.net[140.211.166.39] said: 550 Lone CR or LF in headers (see RFC2822 section 2.2) (in
reply to end of DATA
command)

See where it says "end of DATA command"?

Let's say I wanted to send an anonymous email from an opened relay.
telnet openedrelay.com 25
mail from:don2007@fakespam.net
rcpt to:ATNO@ozzu.com
data
blah blah blah
.
^D

See the . on the line by itself? That ends the text input in the body of the email that was started by the data command. Then the ^D ends the session. CR and LF are used throughout a sendmail program although the user has no control of it other than the body.

That was an example of one anonymous email done by hand but there are bots to mass produce emails on vulnerable servers. Apparently someone or something has decided that certain formation of headers denotes the use an opened relay, spam bot or an email carrying a virus. That's why I think some update is blocking the headers based on that.

I hope all that made sense.

[ATNO/TW]

October 30th, 2008, 5:48 am

Yeah, I understand the issue now. Just clueless what caused it. I changed nothing so has to be an update. Uninstalling all the ones I did last week one at a time and testing.

October 30th, 2008, 6:23 am

I think that's all you can do.

[ATNO/TW]

October 30th, 2008, 8:01 am

Update:

Besides the update I listed above as being uninstalled, I uninstalled the following updates that were installed Tuesday Oct 21, 1 at a time and tested with no resolution to the problem:

KB957257, KB954211, KB954211, KB959391, KB956390, KB957095, KB956841, KB956803, KB956390, KB957095

I've confirmed that emails are being received at the recipient's Exchange accounts on our server. The NDR is being generated when the mail is being forwarded to their Roadrunner accounts. I've come to the conclusion that the problem is at Roadrunner.

Next step is have them set up gmail accounts, and reroute the forwarding to those. Its a work-around, but I'll update how it goes.

October 30th, 2008, 5:52 pm

What about contacting Road Runner?
703-345-3416

[ATNO/TW]

October 31st, 2008, 1:53 am

We did. Response was they rebooted their server and for me to reboot mine. Thankfully it was one of the affected people that called and not me. I'd have done just a little more than laugh at them.

October 31st, 2008, 5:53 am

Unbelievable.

[ATNO/TW]

November 11th, 2008, 8:21 am

Just a quick update to this, it still hasn't been resolved, but I did verify that this is 100% a problem on Roadrunner's end. I have another user that is set up exactly the same way except on AOL and he is having no issues receiving forwarded mail.

As a work around the alternative solutions I've suggested are to use OWA, VPN to work and access mail over the VPN pipe, or set up GMAIL accounts and set the forwarder to those and set their Outlook to retrieve their GMAIL.

I've concluded struggling with Time Warner customer service is useless. They are outsourced and not only can you barely understand them, they completely had no comprehension of what I was trying to explain. I couldn't even get to a Level 2 tech.

November 11th, 2008, 5:26 pm

It annoys me when people who are supposed to be in tech support or customer service, are that incompetent. I'm crazy enough to find out who is in charge and call him at his house, if I can find the number.

[ATNO/TW]

April 14th, 2009, 10:43 am

Well, I finally figured out what the problem was and resolved this.

The problem apparently originated from the Spam Filter I was using which was ORFilter. Apparently when patches came out that addressed this vulnerability apparently they caught this "bug" in the program on roadrunner's end. Unfortunately the program is no longer updated or really supported and there was no way to alter it. Bottom line - find a different spam filter.

Ended up discovering that Exchange Server 2003 SP2 includes Intelligent Message Filter V2 which I had never heard of. I never had a pressing need to install the service pack and had obviously missed that. So I updated Exchange with SP2 and configured IMF V2 and it solved the forwarding issue and appears to be handling spam as efficiently if not better than ever.