I wonder whether your band leader sees the potential in you playing the instrument you already are and was frustrated when you said you wanted to change.
Once my star runner decided he wanted to quit the Athletics team. I was disappointed not for myself but for him! His potential was exciting, and if he applied himself (which he eventually did) I knew he could do really well ( which he did)
The reason he wanted to quit was because he was anxious and scared of losing so he decided not to compete at all.
I don’t know if that may help you because you mentioned that you get really anxious playing your instrument. Is that the real reason that you want to switch instruments?If it is then your band leader will probably be aware of this as he will know you well and probably over reacted because of his disappointment.
You need to decide what you really want. If the desire to play a new instrument is to try and escape the anxiety and stay in a comfortable environment, then it may be worth exploring how to manage your anxiety better as that may be the real root of the problem. It is possible to do that. I used to have major anxiety over performances in public even though I was very good and did not sing for about 5 years. I regret not dealing with my anxiety sooner as it freed me to do what I wanted.
Buy this shirt: https://nemoshirt.com/product/black-girl-stay-woke-shirt/