The problem with that is that the cloak disruptor is still just a reaction piece. While a cloak or a JD gives you opportunities to act, the cloak disruptor is only useful in reaction to something someone else does. Inversely, that means that it is useless unless you are facing a situation in which you can expect these things to be used against you, and for such cases, you would really rather have a cloak yourself or a CM.
Spazzy once made a remark about making cloak disruptors a piece of equipment like the JD that is mounted on an equipment slot, or just fixed to the ship like an armor upgrade so it doesn't take away the CM slot. That could solve this issue, but until that is feasible, I remain against the idea that CDs shouldn't be able to disrupt the charge time.