To address your question:
Saying "no" and walking off is not wrong provided you are an O-1 or above and provided the chief was not being disrespectful in tone, words, or approach (insofar as we're addressing good order and discipline). If the chief was in anyway disrespectful, then pausing for correction is in order provided the circumstances allows for this to be accomplished constructively - if not then report the chief's conduct to your commander.
(If your commander does not seem to care, that's a shame, but for your part you can recall this event when you're the commander and your folks come to you with similar experiences)
Assuming the chief was being respectful, but possibly mistaken (or perhaps you are mistaken), don't you think it preferable to take charge of the situation and discuss a little more fully? - e.g. introduce yourself, get his/her name unit and contact information, let the chief know that you're not aware of a published rule that clearly prohibits this small utility light, and since you are not subject to his/her direct authority you will continue to wear the light until presented with proof of this rule. Keep your words and tone good natured and professional. State your position clearly and move along - you are under no obligation to engage in debates with the enlisted force.*
(*I am not suggesting that being routinely dismissive of the opinions and ideas of our enlisted members is an acceptable SOP, but the scenario you present would be resolved by reference to published directives - no need for a debate)
I might anticipate replies that my comment demonstrates ignorance of how things actually are with regard to officer/chief conduct and authority. I know full well how it is, but the chiefs are not going to fix this - it's up to ALL the officers.
All-in-all, even if you were mistaken on the rule on the wearing of the light, what you did was better than removing the light and conveying that the chief has authority to direct your actions. But, if you knew you were in error, then you should not have been wearing the %$&# light on your uniform out where this chief was going to see you (sorry guys).