I said it in some thread on here before but the funny thing is that this argument isn't even true.
X11 always had a security extension and can differentiate between trusted and untrusted programs. As you might guess, untrusted programs are not allowed to see the screen contents of trusted programs or the clipboard. It's kinda poorly documented but I had that set up for a while, security wise it works best if you also can sandbox the untrusted applications in other ways so they have no chance to access the magic cookie.
The big problem I had with it was that programs just automagically assumed they're trusted (firefox was a big offender) and then would end up crashing if they couldn't access some things. There might've been more problems, it's been a while.