I was reading this book recently where it has short journal or diary entries from Himalayan expeditions from various different noted people. It mentioned the exact same thing from Jahangir. He was basically saying the same thing - that the workers who picked up saffron were getting intoxicated themselves.