It seems that Qt honors neither ~/.fonts.conf, nor what's in ~/.fonts.conf.d, nor what's in /etc/fonts/conf.d. This is a problem if you want to run light-and-lean and not use the GNOME or Xfce daemons in Openbox, Fluxbox, and likely many other window managers.
After much hair pulling, I finally managed to work around the problem by using Xresources. Adding the code below to either ~/.Xresources (for user-only changes) or to a new file /etc/X11/Xresources/fonts-default (for system-wide changes) fixed the issue for me. It also fixes problems with ugly font rendering in OpenOffice.org and LibreOffice.
What I added is:
Xft.dpi: 96
Xft.antialias: true
Xft.hinting: true
Xft.rgba: rgb
Xft.autohint: false
Xft.hintstyle: hintslight
Xft.lcdfilter: lcddefault
The dpi entry is there just for good measure. Change settings as needed according to taste.