It is most unfortunate and extremely inappropriate that your coworker stairs and makes comments to you when you wear clothing that you would like to wear to work. I think that you look very elegant and relaxed in your photo, certainly no reason why you should not be able to wear clothing like that to work whenever you wanted.

I do wish that we could trade the weather as well. It is not cold where I am (other parts of Canada are minus 20 degrees C or more) but it is very rainy and windy here. We had a little bit of snow fall exactly on Christmas night but that was about it.