I would move them as well. Those ants are little demons. They will end up causing issues sooner or later.