"To die for"

Somebody earlier remarked that so-and-so something was "to die for". Although I know that it's simply an exaggeration, it struck me how odd that expression was. I mean I would certainly die for somebody or for a cause. But for some object? I wouldn't want to die for it otherwise I wouldn't be around to enjoy it.