The scenario: ESX 4.1 Cluster and a Solaris 10u8 64 bit VM running on RDM Luns. Clone the LUNs at the storage level (a crash consistent cloning) and map them to a new VM configured in the same exact way, same virtual hardware version, same SCSI Controller, same Hard Disk SCSI ID.
The problem: the cloned Solaris guest will fail to boot, since for a still unexplained reason the PCI device mapping of the controller has changed and the kernel while booting will be unable to find the disk(s). This is really strange since VMware ESX should provide the same exact hardware to the guest, like in Linux OS guests.
Booting in failsafe mode and mounting the root file system on /a should show something like this:

The /devices/pci@0,0/pci15ad,1976/sd@0,0:a is the new device, while on /a/dev/dsk/c1t0d0s0 you can see the broken symlink to the wrong PCI path.
The solution: you have to reconfigure the devices inside the cloned guest; unfortunately for Solaris is far more complicated than in Linux OS.
1. Start the VM and at the grub prompt, enter fail-safe mode and allow the root file system to be mounted as read/write under /a.
2. Identify the new device name using the format command. In this case it is /pci@0,0/pci15ad,1976@10/sd@0,0 so replace where necessary from now on. vi may not display correctly so fix the terminal type with the following command:
# TERM=sun-color; export TERM
3. Now edit /a/boot/solaris/bootenv.rc and update the line that starts with setprob boothpath so that it reads:
setprop bootpath ‘/pci@0,0/pci15ad,1976@10/sd@0,0:a’
4. Update the boot archive:
# bootadm update-archive -R /a
5. Then edit /a/etc/vfstab (making a backup copy first) and modify the line mounting the root file system so instead of /dev/dsk/c1t0d0s0 and /dev/rdsk/c1t0d0s0 it reads the following absolute paths. Do not forget to suffix a: and ,raw respectively.
/devices/pci@0,0/pci15ad,1976@10/sd@0,0:a /devices/pci@0,0/pci15ad,1976@10/sd@0,0:a,raw / ufs 1 no -
6. Now we need to force a reconfiguration boot so that the system recreates the /etc/path_to_inst file that contains physical device to logical instance mappings. You can do this in 2 different ways: “touch /a/reconfigure” before reboot or at the grub prompt, I prefer to use grub.
Anyway it is time to restart:
# reboot
7. At the grub prompt, edit the grub entry thath sould automatically boot with ‘e’ and edit the kernel command line adding -r -s at the end of the line, then press ‘b’ to boot that entry. -r is to force reconfiguration, -s to start in single-user mode
8. Login and edit /etc/vfstab again so that you can replace those absolute paths for the / file system. If you have /var on a separate file system mount it manually to make vi happy. If you made a backup of this file before the last update, it would be easier to start with this. Before you do this use format to determine the new disks and update this file as appropriate. An example with the correct disk paths may be:
/dev/dsk/c3t0d0s1 – - swap – no -
/dev/dsk/c3t0d0s0 /dev/rdsk/c3t0d0s0 / ufs 1 no -
/dev/dsk/c3t0d0s7 /dev/rdsk/c3t0d0s7 /export/home ufs 2 yes -
9. Reboot again in multi-user mode and you’re done! System is now ready for use.
It seems that ZFS volumes will not be affected by the device reconfiguration, they should come up automatically.
YMMV.
Useful links:
- http://communities.vmware.com/message/907527
- http://rdiggle.blogspot.com/2010/06/using-solaris-1009-virtual-appliance.html
((enjoy))