I have had great success with installing ZeroShell in ESXi4 by installing it to an IDE drive instead of Sata in VMWare Machine Setup.
First make 2GB IDE disk in the VM setup & 512MB Ram
I then boot the VM Guest off Ubuntu-Desktop-32.iso (Cause I already have it in my datacenters ISO folder, I bet you can use another distro)
Download the ZeroShell IDE Sata USB img.gz
Open a shell – Apps / Acc / Terminal
gunzip -c /home/ubuntu/Downloads/Zero*.img.gz>/dev/sda
(NOTE: sda not sda1)
shutdown, reduce RAM & remove the iso cd-rom from VM.