Hmm, that's very interesting... I am honestly not too sure. As you said, when the adapter is enabled, it's likely some sort of latency or interference. I can imagine the only workaround is a different adapter.
Forgive my ignorance....Is it possible that I just download a different driver or should I change the hardware related to it ? And being a laptop, is it worth all the effort ?
It's likely a hardware specific issue causing the interference, and likely with the adapter itself. If you've reinstalled the latest version of your Atheros wireless adapter, then you've done all you can.