Cover image banner for fixing X11 connection rejected because of wrong authentication OpenSSH error on Linux

X11 forwarding is a feature of the SSH (Secure Shell) protocol that enables graphical applications found on a remote computer to be accessed over an SSH connection.

The GUI application is executed on the remote server, but the application’s window is displayed on the local computer where the SSH session was started. This allows us to use the GUI application as if it was installed on the local computer itself.

Backstory

I’ve used X11 forwarding several times in the past, but a few weeks ago, I was unable to launch GUI apps from a freshly provisioned AWS Lightsail instance. Instead of seeing the application’s window, I got this error message:

X11 connection rejected because of wrong authentication.

Below are the steps I took to fix this error.

1. Remote Server Configuration

First, connect to the remote sever via SSH normally (without enabling X11 forwarding by omitting the -X option):

ssh user@xxx.xxx.xxx.xxx
Bash

Step 1: Create a Group for X11 Forwarding

Let’s create a group called x11users on the remote server. In Step 2, we’ll configure SSH to allow X11 forwarding for all users who are in this group.

export GROUP_X11="x11users"
sudo groupadd $GROUP_X11
Bash

Next, we add our user to the newly created group.

sudo usermod -a -G $GROUP_X11 my_user
Bash

Step 2: Enable X11 Forwarding (Server-side)

We now need to configure SSH to enable X11 forwarding for the x11users group. Add the following block at the end of the /etc/ssh/sshd_config file on the remote server:

## Uncomment the line below only if you have disabled IPv6
#AddressFamily inet

Match Group x11users
	X11Forwarding yes
	X11UseLocalhost no
	X11DisplayOffset 10
	AllowTcpForwarding yes
vim

Note

If you have disabled IPv6 on the remote server, you should also set the value of the AddressFamily directive to inet by uncommenting the second line above.

Then, reload the SSH service to pick up the new configuration, and close the SSH session:

sudo sshd -t
sudo systemctl reload ssh
exit
Bash

2. Client (Local Computer) Configuration

Step 1: Enable X11 Forwarding (Client-side)

The following directives should be present in the /etc/ssh/ssh_config file on your client (local computer):

Host *
	ForwardAgent yes
	ForwardX11 yes
	ForwardX11Trusted yes
vim

Just like before, the SSH service (this time locally) is reloaded with the updated configuration:

sudo systemctl reload ssh
Bash

Step 2: Connect to the SSH Server

Let’s now connect to the remote server with X11 forwarding enabled:

ssh -X user@xxx.xxx.xxx.xxx
Bash

Launching any GUI app from the SSH session should display that app’s user interface on your local computer. If you’re still experiencing X11 authentication errors, the steps below explain how to troubleshoot and resolve the error.

Troubleshooting: Check $DISPLAY variable on the SSH Server

If X11 forwarding is working properly, the $DISPLAY variable will be set to some value by default.

echo $DISPLAY
Bash

Note

The $DISPLAY environment variable should be checked inside the SSH session (when connected to the remote server).

On the other hand, if the value of $DISPLAY is (null) or empty, it means the X11 proxy was not properly started. This could be because xauth is missing on the remote computer and/or the client computer. You should install xauth on both the client and server as follows:

  1. On Manjaro and Arch Linux derivatives:
sudo pacman -S --needed xorg-xauth
Bash
  1. On Ubuntu and Debian derivatives:
sudo apt install xauth
Bash
  1. On RedHat, RHEL, and Fedora derivarives:
sudo dnf xorg-x11-xauth
Bash

Note

Ensure that xauth is installed on both the remote server and on your local computer!

Fix X11 Wrong Authentication Errors

The commands in this section are intended for the remote (SSH) server, not the local (client) computer!

Solution 1: Set $XAUTHORITY on the SSH Server

echo $XAUTHORITY
Bash

If you get an empty output, the $XAUTHORITY environment variable needs to be set on the SSH server before launching the GUI application:

export XAUTHORITY=$HOME/.Xauthority
Bash

Solution 2: Grant Permission to the New User

If you switched users during the SSH session (for example using sudo or su), then that new user (e.g user2) differs from the original user (e.g user1) that started the SSH session. Consequently, the new user (user2) does not have permission to access your X server, thereby causing the X11 forwarding to fail.

This new user (user2) can be granted permission to your X server using xhost. But first you need to switch to the original user that started the SSH session on the server.

su user1
Bash

Note

You should replace user1 in the above command with the proper username!

Then, running this command on the SSH server will grant the new user (e.g user2) access to your X server:

xhost +SI:localuser:user2
Bash

You should now be able to launch GUI apps over SSH even if you switch to the new user user2.

Note

If you'd like to reset the permissions before closing the SSH session, this can be achived as follows:

xhost -
Bash

Solution 3: Use the Original User’s Authorization File

As a last resort, you can add the authorization entry of the original (non-root) user that started the SSH session to the current user’s /.Xauthority file on the server:

export USER_ORG="user_who_started_ssh_session"
xauth add $(xauth -f ~${USER_ORG}/.Xauthority list | tail -1)
Bash

Note

You should replace user_who_started_ssh_session in the above command with the proper username!

# Footnotes

https://www.ibm.com/support/pages/x11-forwarding-ssh-connection-rejected-because-wrong-authentication

https://wiki.archlinux.org/title/Xhost

https://linux.die.net/man/1/xauth