The message "There are objects that cannot be removed" indicates that some objects have been damaged by attacks.
If you examine objects emitting smoke, as shown in the image below, the quest "Post-Raid Cleanup" will be triggered.
After completing this quest and repairing the damaged objects, you will be able to remove them.