Arch Linux User Repository

Thanks to both of, you, @jfk and @ravenblackdusk.

@hexd, I've changed the test locale to LC_ALL=C.UTF-8 in python2-pyopenssl's check().

According to @jfk's comment, it resolved the UnicodeEncodeError there.

Based on input from both @jfk and @ravenblackdusk, as well as my own testing, it seems that if we set any <lang>.UTF-8 locale that is not available on a particular system (neither in glibc nor via locale-gen), python2 will fall back to the ancient ANSI_X3.4-1968 encoding for its filesystem access. (The same as if we were forcing LC_ALL=C).

Only locales POSIX, C and C.UTF-8 are universally present on up-to-date Arch-based systems. The latter has just been added recently to glibc, on 6 June 2022, with package version 2.35-6 (systemd needs that as fallback now, and "forced" the Arch devs to include it, according to them).

It is possible some users still don't have it; they'll have to update to at least glibc 2.35-6.

Fortunately even Manjaro's roughly one-month-delayed stable-branch repos received this glibc version 2 days after Arch, as a fast-tracked package upgrade.

So @hexd, please change your testing locale to C.UTF-8 and cross your fingers! :D

I'm using manjaro I don't use testing repositories AFAIK.

git clone https://aur.archlinux.org/python2-scandir.git
cd python2-scandir
makepkg --syncdeps --rmdeps --install

seemingly same error: UnicodeEncodeError: 'ascii' codec can't encode character u'\u018f' in position 72: ordinal not in range(128)

$ env | grep -E '^(LANG|LC_)' | sort
LANG=en_GB.UTF-8
LC_CTYPE=en_GB.utf8

$ python2 -c 'import sys; print("default: " + sys.getdefaultencoding()); print("filesystem: " + sys.getfilesystemencoding())'
default: ascii
filesystem: UTF-8

$ LC_ALL=en_US.UTF-8 python2 -c 'import sys; print("default: " + sys.getdefaultencoding()); print("filesystem: " + sys.getfilesystemencoding())'
default: ascii
filesystem: ANSI_X3.4-1968

$ locale --all-locales
C
C.UTF-8
en_GB.utf8
POSIX

$ LC_ALL=en_US.UTF-8 python2 -c 'import sys; print("default: " + sys.getdefaultencoding()); print("filesystem: " + sys.getfilesystemencoding())'
default: ascii
filesystem: ANSI_X3.4-1968

$ python2 -c 'import sys; print("default: " + sys.getdefaultencoding()); print("filesystem: " + sys.getfilesystemencoding())'
default: ascii
filesystem: UTF-8

$ locale --all-locales
C
C.UTF-8
fr_CH.utf8
POSIX

EDIT: build eventually went through after installing python2-pyopenssl (see MarsSeed's thread for details). I cannot pinpoint a reason though.

Just wanted to say that in my case (vanilla Arch and locale set to catalan) I had to comment the line containing "LC_ALL" for it to work.

+1: I'm also interested in the output of this command:

locale --all-locales

Thanks for both of you for your replies. On the surface, everything looks fine with those LANG settings.

But could you please also share the output of the following statement? (I have a hunch and I've already found some potential discrepancy; will discuss later).

python2 -c 'import sys; print("default: " + sys.getdefaultencoding()); print("filesystem: " + sys.getfilesystemencoding())'

EDIT: And also please report the results of the following variant:

LC_ALL=en_US.UTF-8 python2 -c 'import sys; print("default: " + sys.getdefaultencoding()); print("filesystem: " + sys.getfilesystemencoding())'

I'm running Gnome on Arch with default repos (meaning I haven't set anything in particular).

$ env | grep -E '^(LANG|LC_)' | sort
LANG=fr_CH.UTF-8

$ findmnt -T ~/.cache/yay/python2-scandir 
TARGET SOURCE                  FSTYPE OPTIONS
/home  /dev/mapper/volume-home ext4   rw,relatime

I am getting the same errors both with Pamac and Yay, I am using EndeavourOS.

$ env | grep -E '^(LANG|LC_)' | sort
LANG=en_GB.UTF-8
LC_COLLATE=en_GB.UTF-8

@ravenblackdusk, are you using Manjaro? In that case, which branch?

@jfk, same question to you - which distro are you using? And within that, do you use the default 'branch' / default repos, or maybe 'testing' / 'unstable' etc.?

Also to both of you: what is the output of the following command on your system(s)?

env | grep -E '^(LANG|LC_)' | sort

Thank you for your responses in advance.

Still getting the same errors on my system as well.