Human desires are endless. When hungry, a person's only need is to eat enough, but once satiated, they will think about how to eat well, live well, and dress well.
The Radiance followers who fled to Harbor City quickly discovered that although, thanks to the Lord of Radiance's grace, they had more pancakes than they could eat every day, there was no longer any worry about food or hunger.
But what could be offered to them was only those firm pancakes. They were filling, but truly not delicious.
Initially, having only pancakes was bearable, after all, they were refugees. It's not like they could be choosy, being able to be full was already good enough. But when given the opportunity to choose, spiritual willpower struggles to withstand bodily cravings.