Hijack my DNS and I’ll be annoyed. Blame me for it and I’ll go ballistic.

Things got off to a good start with Time Warner Cable’s Road Runner service. I switched over recently when it became clear that Fairpoint cannot or will not maintain its infrastructure. The Time Warner kit showed up, I plugged everything in, my new digital phone and Internet services worked right out of the box. Nice!

There was just one annoying glitch. My searches kept getting redirected to dnssearch.rr.com. So for example, if the search term was “Jon Udell”, I’d land here. The landing page poses the question “Why am I here?” and answers thusly:

You entered an unknown web address that was used to present site suggestions that you may find useful. Clicking any of these suggestions provides you with search results, which may include relevant sponsored links.

If this service is not right for you, please visit your Preferences page to opt out. At any point in time, you can opt back in to the service by visiting your Preferences page.

You might wonder why search would trigger this hijacking. I looked into it and found that my DoubleSearch search provider, which queries Google and Bing side-by-side, reveals an odd Road Runner quirk. When I use it on a Road Runner connection, the Google search works normally but the Bing search gets hijacked. This wouldn’t happen normally, but it turns out that I never updated the DoubleSearch provider when search.live.com was redirected to search.bing.com. So when the provider invokes this URL:

http://search.live.com/results.aspx?q=”Jon Udell”

I should be redirected to:

http://search.bing.com/results.aspx?q=”Jon Udell”

But instead, Road Runner sends me to:

http://dnssearch.rr.com/?q=”Jon Udell”

Evidently you don’t need to fail a DNS lookup outright to trigger the hijacking. It even happens when your first destination redirects you to a second.

When I went to the Preferences page to end this interference I found not one but three “services”:

  1. Web Address Error Redirect Service
  2. Typo Correction Service
  3. Safe Search Filter

As others before me have discovered, the first of these — the “non-existing domain landing service,” aka DNS hijacking — is enabled by default. That rubs me the wrong way. I don’t want Time Warner Cable hijacking DNS lookups at all. Doing it in a way that involves “relevant sponsored links” is even worse. And triggering on a redirect instead of an outright failed lookup is just plain weird. But OK, it’s a setting, I can disable it once, and then forget about it, right?

Wrong. It turns out that to “disable” the “service” doesn’t mean ending the hijacking for my local network. Instead it means dropping a cookie into whichever browser I happen to be using at the time. This fails to address the various problems detailed on Wikipedia’s DNS Hijacking page.

So I called Time Warner to ask them if they will implement the setting correctly. Unlikely, but it never hurts to ask. Things got off to a really bad start with the first support agent, Kerwin, though.

Me: Your Web Address Error Redirect Service is creating a problem and I’d like to see if we can resolve it.

Kerwin: Where are you being redirected to? It sounds like your computer is infected with a virus, so…

Me: Hold it right there, pal. Let me speak with your supervisor.

After some backpedaling, during which I learned that Kerwin didn’t even know what DNS hijacking is, never mind that Road Runner does it, I connected with Bill at level two support. I told Bill to take Kerwin out to the woodshed for a spanking, and explained the situation again. Bill, who says he’s worked at Time Warner for 8 years, also claims not to know that this “service” exists on his company’s network.

I am waiting (but not really expecting) to hear back from somebody at level three. Meanwhile I just had to get this rant off my chest. If you hijack my network pipe, I’ll be annoyed. If you make it hard for me to stop you from doing that, I’ll be angry. But if you blame me for creating a problem you claim not to know about or understand, I’ll go ballistic.

32 Comments

  1. I’ve had some big disappointments with TWC as well. For instance, rather than contact the account holder (who they send bills to) to let them know their computer may be infected with a virus and they need to remediate the problem or suffer having their network access blocked, they simply took us off the network one day. I came home to no network, spent many minutes powering down and powering up the Cable Modem, the router/access point, etc. All to no avail finally ending in me calling. And then I had the same kind of run around where nobody knew why we didn’t have access to the network. The first time I called, I got dropped because I was calling on our hardline through TWC and they had to reset the modem remotely. Guess what genius, that dropped the call too. So I had to call back got another guy who couldn’t tell me why the network was not working, got put on hold, transferred, and got a voice mail message! I called back again, this time they successfully got me to a level 2 support guy (but I was on hold for 15 minutes). The level 2 guy finally figures out what’s going on and explains one of our computers is suspected of being infected and our network access was blocked. And to add insult to the injury, he made some kind of comment that he could temporarily turn it back on if I promised not to let that computer back on the network while I worked on it. And so I promised and he said, usually they require all TWC customers who have been blocked to provide a copy of a receipt from a computer maintenance outfit showing proof that the computer was ‘brought in’ and ‘cleaned’. Some how you must fax this to them (because it certainly will be impossible to do through SMTP and the TWC provided network which is being blocked). Which leads me to believe I could forge a receipt and easily get back on the network. TWC is the amateur hour writ large.

  2. Jon,

    do you perhaps have enough configuration control over the Roadrunner gateway/router box to change the DNS servers used? If you switch to Google’s public servers (8.8.8.8 and 8.8.4.4, IIRC), I believe they do _not_ hijack.

  3. This is Sandvine’s “technology.”

    It’s more than just DNS hijacking, they are proxying HTTP requests too it looks like, though I’m not 100% sure. I think that’s how they inspect the URL to see if they should redirect it or not.

  4. Paradigm: I agree it does — or did — look that way.

    Ken: Yes, the modem is theirs but the router is mine. While trying your suggestion I noticed that I cannot now reproduce the original behavior. It’s almost as if somebody read this and responded silently, though somehow I really doubt that.

  5. I’ll emphatically second Ken’s suggestion to use the Google Public DNS servers. I was originally skeptical about giving Google access to a new source of information about the websites I’m visiting but it seems now that ISPs are doing much more sketchy things than just recording my DNS lookups. And at least Google is subject to enough public scrutiny that if they start hijacking or messing with the results, it’ll draw a big public outcry.

    On the other hand, if they were in fact mucking around with the HTTP stream, changing your DNS provider wouldn’t help. That’s an awfully big ‘if’, though — since presumably you hadn’t configured your browser to use a TWC proxy, so they’d have to be intercepting the HTTP packets, reassembling the response, processing it, modifying it, and re-sending the modified response, all at wire speed.

    You didn’t run any kind of disc they sent you with the self-install kit, did you?

  6. If anything is sorely needed with Net Neutrality it is precisely this – meddling with the customer HTTP request and its standard operation (like the valid redirect) must be, IMHO, an OPT IN thing – TWC must have to actually sell you on this and give you something in return (and also let you revert at any point if you so wish).

    Jon is not an average person – he knows a thing or two about how the network works. 99% of the others will not get the service they choose and will not even know about it.

    @Jon – you do have a much wider audience than most of us and you obviously do care about this issue. Let your voice be heard!

    Dror

  7. Well at this point somebody else might need to document the behavior I was seeing, because I’m not seeing it now. If I disable or enable the service in one browser, the cookie I was seeing before isn’t created or deleted. Meanwhile a different browser sees the change.

    Although nobody from Time-Warner has responded to me, the system really is acting as if they heard and acquiesced to my request.

    It would interesting to replay all this on a new TWC installation.

  8. Went through similar rigamarole today with Bright House after getting opted back into the program after opting out when it was first implemented in 2006 (?).

    They ended up blaming me and my OS X machines for their DNS hijacking and recommended I call Apple. After I reiterated that it was their hijacking at fault, the support agent stranded me on hold for 40 minutes after which point I gave up.

    Received a bunch of email spam from them this evening with links to “useful” articles with wonderful titles like “CA Internet Security Suite – CA Anti-Virus”, “Removing ActiveX Controls and Java Applets”, and “Abuse Reporting: Child Porn Complaints”.

    Not a happy customer. Considering calling back later this week and escalating higher (which the real threat of changing ISPs).

  9. Tonight I heard back from a (presumably level 3) TWC person.

    Q: Is opt-in the default?

    A: Yes.

    Q: Why?

    A: Don’t know.

    Q: Why was opt-out only per-browser rather than local-network-wide?

    A: I think it is local-network-wide.

    Q: If so, why drop a cookie into the browser that requests the opt-out?

    A: Don’t know.

    Q: And why didn’t the opt-outs stick until I called to complain?

    A: Don’t know.

    Q: How and why is TWC redirecting when there is no failure of DNS lookup, but only an HTTP-level redirect (e.g. search.live.com -> bing.com via HTTP 301)?

    A: Don’t know.

  10. I suppose I should be glad that I am not alone. I spent 2 hours on the phone with support, switched to 6 different people (and having to give them my account info each time)

    If you do get this resolved, please get us a contact, reference number or even just the correct words to say so we can get it fixed too.

  11. And BTW I don’t believe the fix-it link does anything with cookies (despite what RoadRunnaround support thinks) I get the same redirect if I run NSLOOKUP directly against their DNS servers. And the options page modifies that behavior as well…for a few hours anyways

  12. I’m a non-expert internet user who’s had his address bar searches hijacked by Road Runner off and on for over a year. This time I discovered that the “Safe Search Filter” cannot be disabled. I was wondering if maybe that’s their trick: could it be that if the safe search filter remains enabled, the web address redirection also is, automatically?

    Anyway, on previous occassions and again on this one, it turned out that if I go to their opt in/opt out page and disable all that stuff, and go back and do it again, and then again: several times–I didn’t count–it works. For some months, anyway. I’ve just disabled it that way, so I wish me luck.

  13. @Nick, There definitely seemed to be a correlation between the setting and a cookie, at least initially. I observed it on multiple browsers on multiple machines. Since I called to complain, though, there isn’t.

    Meanwhile the “Don’t hijack” setting refuses to stick. It behaves as though opt-out means “Stop doing this for a while” instead of “Stop doing this period.”

  14. I thought about what you said about cookies. This morning I made sure the redirect was happening. I checked by browsing to “http://somewhere.local” on my ipad and my work laptop. In both I got the redirect.

    Next, I used fiddler (a web traffic analyzer) to analyze the responses as I clicked trough to the preferences page and disabled redirect. There are no cookies received in response to saving settings.

    With settings saved, I tried my test URL again. Both laptop and the iPad are affected by the change.

    Incidentally, while testing I recorded that “Save Setting” does a simple GET. Meaning if you bookmark “http://dnssearch.rr.com/?cat=pref&con=dns&optout=yes”, you can visit the bookmark instead of navigating through “why am i here”.

    After another 3 hour call with no resolution, I might make that my home page until this is resolved.

  15. The comment system rejected my longer reply (twice). I went ahead and analyzed the web traffic when saving settings. There are no cookies in the exchange.

  16. Thank you, Jon, for fixing my mess there with the multiple corrective posts.

    Just thought that having commented here I owed you guys an update:

    Unfortunately, Road Runner’s back. I guess this time they mean business.

  17. The great mystery is _why_ do they do this ?

    After dealing with similar problems with Verizon DSL (and their port 25 blocking – different rant), I just gave up, and installed dnsmasq as my local DNS proxy, pointing to real DNS servers. It does exactly what I need, and runs without a glitch.

    I run it on a local GNU/Linux server, but it’s also available with the dd-wrt router firmware.

  18. Tonight Time Warner Cable DNS hijacked my Facebook.

    It’s one thing when a lookup failure takes you to some dumb search page, it’s another thing entirely when it blocks off actual existing sites! Thanks for this post, for a minute I thought I was in Iran.

    Of course, which is eviller – the ISP that blocks me from getting to Facebook, or Facebook … ? :D

  19. Update:

    One of my coworkers showed the hijacking behavior to a tech during a service call. Tech supposedly called somebody to look into it.

    For myself, I setup a scheduled task to ‘wget “http://dnssearch.rr.com/?cat=pref&con=dns&optout=yes”‘ so I’m now opting out every 5 minutes. Doesn’t cure the problem but sure cures the symptoms.

  20. @Nick, I had another conversation with TWC yesterday. The guy was very helpful: read this blog entry, started a trouble ticket, included a reference to this entry, really wanted to help.

    BTW, he too was unaware of the existence of dnssearch.rr.com, as have been all the front-line people I’ve spoken with.

    I got a call back a few hours later, from somebody who made a change on their end, and so far so good. This may indicate a bug in their opt-out mechanism.

    (Never ascribe to malice what is adequately explained by incompetence.)

    If this works, it’ll prove that at a minimum others can get the fix applied manually, though of course that shouldn’t be necessary if it is just a bug and if the bug gets fixed.

    Still unexplained: the HTTP-level redirection. I asked them to follow up on that.

  21. Thank you! Those idiots at TWC had me thinking I was insane. As soon as another ISP comes to town I’m switching (I hope you’re listening you jackasses :). I’ve had the same conversation over and over again with 10 different “level 3” techs and none of them believe me.

    I’ve had it happen to every website imaginable – wiki, facebook, youtube, and most annoyingly, google. I’ve tried to clear my cache/cookies/etc. but the only thing that works (for google anyway) is this: type in images.google.com, type in my search, click search, and after all the results load, click the “everything” button.

    I work in the adult industry and couldn’t help but be paranoid and think they might have “flagged” me….I don’t know, maybe there’s some bullshit in their TOS about programming for porn sites… :/

  22. Just wondering if any of you had any hardware changes before the opt in problem reoccurred? Did you have to replace a modem? Did you have any scripts run which may have wiped out deeply saved settings on the computer?
    I had two things happen at the same time, a new modem from timewarner, and then i noticed that a lot of my customized ‘shortcut’ items in my Mac finder ( os x 10.4.11, i know, i am ancient ), disappeared. It has been a long time since i had the issue in the finder, and i don’t recall what i may have thought caused that in the past, but i am wondering if there may have been some corruption after a cron job was run, … i am pretty much guessing here.
    But, then i found that dnsredirect / opt in happened to me, and i am trying to figure out which may be the cause. I am wondering if the setting may have been saved on the modem itself? Now with a new modem, the setting needs to be reset? Or, of course, I have the very unusual problem which did not coincide with the new modem, the Mac settings lost, but the coincidence in timing is there, and therefore the troubleshooting of the cause becomes a bit more difficult. ah well, them computers…. if we didn’t love to hate them, then we would hate the wrong things! cheers, wnyer

  23. I finally reached the end of my rope with TW’s excuses, the endless reverts to the redirects. I set my home router to dish our The Evil Google DNSs: 8.8.8.8 and 8.8.4.4.
    Steve

  24. It happened to me too. After no help from Microsoft or Time Warner I found a fix to get Bing back. This fix does not disable TWC’s redirect when you type int an incorrect URL (404 hijacking). If you log in to the back end of your router and turn off “Port Scan Detection” it should fix it.

    To do this go through you browser. In my case I would…
    1. Type 192.168.0.1 into a browser’s URL area (if that doesn’t work, look at your router for where to go)
    2. Enter the user name and password (here is a site that will help if you don’t know http://www.routerpasswords.com/)
    3. Disable “Port Scan Detection”

    That is all.

  25. I know this is an old thread, but another option (instead of using Google’s DNS) is to set your routers DNS servers to dyn.com (or a similar service). This way you can create a free account and setup your own filter, to filter out any trash you don’t want/need in front of your family (dyn.com calls it the Internet Guide). Here are there DNS servers:
    216.146.35.35
    216.146.36.36

    Enjoy!

  26. I realize this was originally posted in 2010 the problem continues on. TWC is continually redirecting 404 errors to http://www.dnsrsearch.com . I haven’t had any legitimate sites redirect but even the 404 redirect bothers me. Is there any way to stop this behaviour?

    I’m not an expert but I’ve tried a few things to no avail: Adblock, editting the HOSTS file, and changing to Google’s public DNS but all to no avail. I would really rather see nothing.. a blank page.. rather than dnsrsearch

  27. Thanks a lot. Knowing that there’s no disable switch, and I’m forced to use google’s DNS servers, saved me considerable time.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s