I cannot change it anymore. When I click on "remove avatar" I get a '?' avatar but whenever I try to upload a new one this avatar appears again instead of the uploaded avatar...

No, your avatar is changed. Sometimes it doesn't show your new avatar on the uploading page though.

I encountered the same problem.

Clear your cache and refresh.

Interesting... I thought that kind of things didn't happen with firefox, but yeah it's better caching that having to reload all the avatars when loading the page.
Sorry for the topic then. It should probably be closed now.