I have a similar problem, but I don't think the workaround from archer works for me. When I try to close the window, I get the "Test Window can't be closed..." box. Clicking Terminate give me the error:
Error during debug step:
ORA-03114: not connected to ORACLE
Debug session terminated.

but the debug session remains; the green arrow and the red lightning bolt are both active, and clicking them gives the same error.

Can PL/SQL Developer be modified to actually terminate the debug session when it says "Debug session terminated"? (I'm using Version 9.0.6.1665)

One other note: Right-clicking on the window in the Window List shows the "Close Window" disabled, but the "Execute" is enabled. (Choosing Execute just locks up PSD as "Initializing...")