DNS lookups are a little complicated. Depending on how your environment has been configured, your DNS lookup requests could be going first to a DNS "server" process in any of ...
- your local router for it to try and respond
- an AT&T machine for it to try and respond
- a public DNS server for it to try and respond
And by "try and respond" I mean that the DNS server process will reply if it knows the answer, or it will ask another DNS server for the answer and forward the response to your computer when the answer comes back.
So for DNS to work, lots of other things need to work too. In particular:
- Your computer needs to be able to exchange requests/answers with which ever DNS server it's using
- That DNS server needs to be able to exchange requests/answers with the rest of the Internet
So my point is that if your DSL connection is bad, the errors you see might be full of "DNS not responding" ... even though a more appropriate error message would be that the DSL connection has failed. Your windows machine just knows its DNS lookups didn't work, it doesn't know why.
Note that power cycling your router doesn't just reset the software & hardware of the router. You're also hanging up and re-establishing the DSL connection between the router and AT&T.
Bottom line is that this might just be an issue with the quality of the DSL circuit with AT&T and not a fault of your router.
But ... yes, nine years is pretty old for a DSL modem. Even if there is a poor DSL circuit, a newer modem/router might help keep the circuit connected. And (depending on other factors) your local WiFi might be faster too.
AT&T "should" be able to tell if the circuit is bad or if you really need a new modem now.