Whether someone is a musician or not doesn't matter. It's like saying that only a baker can judge the taste of bread.
By the way, if we take your metaphor – even if you're not a baker, you won't be able to tell by taste what kind of flour was used, whether it's yeast bread or not, etc. But simply saying "it's delicious" – that's fine.
But, as they say, there's no accounting for taste. This applies to music as well.:smile01:
