I have ESXi 5.1 running on two Mac Mini's. The recommended way I found was to install VMware Fusion on the Mac Mini and then install ESXi as a VM under VMware Fusion. I have had to edit the config file for the ESXi VM and then I can assign 14 Megs of RAM to ESXi. I have good luck with this configuration. I have vCenter running on one ESXi and several VM's running on the other Mac Mini. Both of them are talking to an iSCSI system being served up from a 3rd ESXi server running on Dell T300 server.
Ron