of course you are right there – but, on the other hand, there is no such thing as “absolute” morality. Morality for a tiger is different than morality for a human, so we are always thrown back to the “conditio humana” – and the Golden Rule/Monolaw gives us a good heuristic, a starting point, for fleshing out more detailed rules. Also, it is important that when we speak of the monolaw we assume a normal/average human being, ie not someone who does not feel pain etc.
In conditions of severe stress, where survival is at stake, we can also draw on the monolaw: when somebody is starving and he steals an apple from your orchard, you would probably consent (this is where the “standard” human comes in again).
Dietary restriction – in case of vegetarianism – is a perfect case of the Monolaw – you don’t want to be eaten – and an animal neither (Disclaimer: I do eat meat from time to time).
And things like polygamy are, in my opinion, not a case for morality, but rather local customs.