When adding "server-icon.png" to your server directory you can have a logo for your server now, and if you update that file a restart is needed to update the clients cache.

However, if you delete "server-icon.png" from your servers directory and reboot, the clients cache will not update, leaving the logo there. So, if you wanted to remove your logo you can't, unless you just upload a transparent image.