The Oracle Debug API (sys.dbms_debug) internally uses dbms_pipe messages to communicate between the controlling session and the debug session, which apparently can fail on RAC systems. The only workaround is to modify the tnsnames.ora file to connect to only 1 node of the cluster without load balancing and failover.