Ensure you have "Enable EFI" checked. High Sierra will not boot via Legacy BIOS in VirtualBox. Also, in Storage settings, set the ISO to "Optical Drive" not "Floppy".