first, for a functional xorg setup you need at least a video driver and an input driver:
local/xf86-input-libinput 1.4.0-1 (xorg-drivers)
Generic input driver for the X.Org server based on libinput
local/xf86-video-amdgpu 23.0.0-1 (xorg-drivers)
X.org amdgpu video driver
If I recall correctly I had to remove xf86-input-evdev and install xf86-input-libinput for my rootless xorg to work. Didn't check why evdev doesn't work with rootless.
Now if you have consolekit or seatd you should be set. But if like me you don't, you must also make sure that your user is in the video
and input
groups:
$ groups
lp video audio scanner input
Without the video group, xorg won't even start, and without the input group, your keyboard and mouse won't work (which might be what you're experiencing).