Yes, I know RDP means Remote Desktop Protocol, you're technically using Microsoft Terminal Services Client (mstsc.exe) a.k.a. the Remote Desktop Client (RDC) to connect, but it seems like folks throw around the acronym RDP as in "I'm going to RDP into that box", so frequently that I'm sticking with it for the moment.
You are trying to connect to a desktop or server via a Windows XP desktop that you are already connected to via another Remote Desktop Session. The computer you are physically using is computer "A", running any operating system. Computer "B", the one in the middle, is a Windows XP SP2 or SP3 workstation running version 6.x or 7 of the Microsoft Terminal Services Client, which you are then using to connect to computer "C" via the Microsoft Terminal Services Client, which is any desktop or server operating system. Upon attempting to make this connection, you might see either a gray screen or a light blue screen that gets stuck somewhere in the process of trying to display the login prompt. You cannot close the Terminal Services client on computer "B", it is hung. You can kill it via Task Manager or Taskkill by stopping the mstsc.exe process.
Surprisingly simple and nonsensical. Click Start, choose Run on computer "B" (the "middle" computer) and type "mstsc", hit the down arrow next to "Options", choose "Local Resources", under the section "Local devices and resources", click the "More" button. Uncheck the "Smart cards" box. Using that same window, try to connect again and it should work.
Oddly with this one, on the computers we could get this to repeat, neither computer "A", "B" or "C" had a smart card reader or hardware on it. I'll post an update if I ever figure out what the real root cause was, but that was fascinating.
Also, in my attempts to find a solution to this problem, I found that Microsoft released Version 7 of the RDC for Windows XP and Vista (scroll to the bottom of the KB article for links to the download).