The customs of Israel are so important, that it is safe to say that they give more spirituality to a person than the Mitzvot themselves. This is so although breaking a custom does not yield punishment, and breaking a judgment does yield punishment. Still, concerning the benefit, meaning producing fear of heaven, the customs yield more spirituality, since the great ones who established the customs arranged it so that spirituality would shine through them.
This is why he said that he who avoids the custom of eating meat and fish on Shabbat denies spirituality of himself. However, this concerns a person who has not achieved perfection, meaning seeing what he does. This means that he has still not been rewarded with the flavors of the Mitzvot, so he needs to observe the customs.
It is like an apple that is spoiled before it rots, but when it is spoiled, rotting is certain. Similarly, when a person becomes free, he rejects the customs, and following the rejection either he becomes free or his sons become free.