I have tried so many things. The light block (item 514) will not work at all with RTX on. I have tried adding a light map, adding a texture, adding everything, but they will not work in the slightest. I have been searching around but apparently no one uses light blocks so I feel out of luck.
Works just fine if I turn off rtx.

I've added an example below, same situation, both with a light block placed at my feet. In RTX, it's totally dark.
As far as I can tell, the method nvidia gives doesn't seem to work.