Rebel IT Mumblings of a crazy programmer…

14Sep/1146

ESXi 5.0 Dell PERC 6i Health Status Monitoring

After upgrading my ESXi whitebox server using the official ESXi 5.0 install DVD I noticed that the health status monitoring for my PERC 6i RAID card was not showing up anymore. Everything else went smoothly during the upgrade and the test VMs all powered on from the datastore on the PERC 6i without issues. When checking health status only the processors and software components were listed. As it turns out VMware has removed the vendor specific VIBs for health monitoring in ESXi 5.0.

In order to restore health monitoring for the PERC 6i to the health status screen you will need the latest LSI offline bundle VIB for ESXi 5.0. I tried using the Dell OpenManage offline bundle but it stopped displaying all monitoring after the reboot and the system would not reset the sensors. After removing the installed OpenManage VIB and after a few hours of scouring the internet I managed to find the solution. The Dell PERC 6i cards use the LSI MegaRAID chipset for their controller.

LSI’s latest offline bundle package supports a variety of cards. After finding the proper version (500.04.V0.24) I was able to locate the download on one of the other controller card pages. Doing a search for "LSI 500.04.V0.24 site:lsi.com" on Google brought up several results. I selected the first result for the MegaRAID SAS 9260CV-4i - LSI and scrolled down to the Management Tools section. Here you will find VIB downloads for 4.x and 5.x. Download the file for ESXi 5.x from any of the listed card pages. You will need to extract the offline bundle from the archive otherwise it will not install and you will get errors about being unable to load the index.xml file.

You will need VMware vSphere CLI installed on a machine. The update requires maintenance mode and a host reboot so if you are using a vMA make sure it's on another physical host. Using CLI on my Windows desktop machine I first copied the extracted offline bundle zip to the root of the ESXi host datastore via the vSphere Client. Then on the machine with CLI installed I opened command prompt and browsed to the folder C:\Program Files (x86)\VMware\VMware vSphere CLI\bin.

I put the ESXi host in maintenance mode using the following command,

vicfg-hostops.pl -server x.x.x.x -operation enter

Note: Several times CLI returned connection errors or said that operation is a mandatory argument. I found that pasting the command was the culprit and manually typing in each command solved the issue. Also note that the VMs must be powered off to enter maintenance mode.

Update: If you have the latest patches installed on your ESXi 5.0 host you may get a cryptic error when trying to put the host in Maintenance Mode from CLI,

Fault string: fault.RestrictedVersion.summary
Fault detail: RestrictedVersionFault

This appears to be a bug fix done to fully disable CLI on free hosts.  Use the vCenter client to put the host in Maintenance Mode.

After the server was in maintenance mode I verified the status by running the following command,

vicfg-hostops.pl -server x.x.x.x -operation info

Once the host was in maintenance mode I ran the following command to install the vib offline bundle,

esxcli.exe -s x.x.x.x software vib install -d [datastore]VMW-ESX-5.0.0-LSIProvider-500.04.V0.24-261033-offline_bundle-456178.zip

When running the command and supplying credentials CLI sat at a flashing cursor for a few minutes. If it's going to throw an error it will do it right away, otherwise it's installing and you should leave it alone. There are no status updates until the install has completed.

Once the install was complete the following was returned,

Now you need to restart the ESXi host in order for the changes to work. You can also do this with CLI running the following command,

vicfg-hostops.pl -server x.x.x.x -operation reboot

After the host was done rebooting I logged in with the vSphere Client and checked the Health Status. It now shows the Storage category and displays all of the information related to my Dell PERC 6i including battery status.

I removed the host from maintenance mode and powered all of the test VMs on without any issues. I hope this helps any users out there upgrading with a PERC 6i RAID controller that want to retain the ability to monitor their storage array.

Filed under: VMware Leave a comment
Comments (46) Trackbacks (3)
  1. Worked perfectly. Thank you for this COMPLETE step by step guide. Only problem I had was had a few errors when I was trying to run the *.pl script to do maintenance mode and etc, I just used the vsphere client to enter maintenance mode (and reboot). I’m guessing my CLI install got goofed or something silly. Regardless, this guide worked perfectly, thanks a bunch!

  2. were do I get the Vsphere CLI version? I can’t find it on Vmware’s website. :(

    Thanks

  3. Thank you! Did the trick for me as well.

    It’s actually slightly easier to complete if you skip the vCLI and just log in using using the “Remote Access” SSH capability straight into the ESXi console. You can then use the same “esxcli” commands (minus the “.exe” suffix) to do the same work.

    Avoids the need for the vCLI install.

    Regardless – thank you very much. Kind of annoying that VMware dropped this.

  4. I found that by using the new ESXi 5.0 install ISO:
    VMWARE-ESXi-5.0.0-Rollup-20110921.iso
    you don’t need to do this, all the VIB bundles are already there.

    • ESXi Free License users do not have access to download the driver rollup 1 ISO. There is an available update for the MegaRAID bootbank which can be downloaded separately from here however ESXi still wont show Health Status on the PERC 6i without the LSI VIB using this method. I also tested the driver rollup ISO on several whitebox machines running PERC 6i’s and none of them report Health Status for the RAID and are still missing the required VIB bundles.

  5. I get the following error when trying to do this:
    [MetadataDownloadError]
    Could not download from depot at zip:/vmfs/volumes/datastore/VMW-ESX-5.0.0-LSIP
    rovider-500.04.V0.24-261033-offline_bundle-456178.zip?index.xml, skipping ((‘zip
    :/vmfs/volumes/datastore/VMW-ESX-5.0.0-LSIProvider-500.04.V0.24-261033-offline_b
    undle-456178.zip?index.xml’, ”, “Error extracting index.xml from /vmfs/volumes/
    datastore/VMW-ESX-5.0.0-LSIProvider-500.04.V0.24-261033-offline_bundle-456178.zi
    p: [Errno 2] No such file or directory: ‘/vmfs/volumes/datastore/VMW-ESX-5.0.0-L
    SIProvider-500.04.V0.24-261033-offline_bundle-456178.zip'”))
    url = zip:/vmfs/volumes/datastore/VMW-ESX-5.0.0-LSIProvider-500.04.V0.24
    -261033-offline_bundle-456178.zip?index.xml
    Please refer to the log file for more details.

    I’m using ESXi 5.0 on a poweredge 2050 with a perc 6i. Any thoughts?

    • Looks like you extracted the offline bundle zip ok just make sure you leave it zipped. Also open the zip and make sure there is an index.xml file in there. If there is another zip extract that and upload it to the datastore. I had issues with copy and paste and had to type the commands manually to get them working properly.

    • Check that your datastore is named datastore and not, like me datastore1 or anything else.
      In my case i had to put [datastore1] in the command line.

    • Just extract and upload the internal zip file from the original compressed file. Will work properly.

    • I had the same issue. I had renamed the datastore from ‘datastore’ to ‘Datastore_1′ – it seems like it is case-sensitive although that doesn’t make a lot of sense to me. After I made that change it went through fine.

      Also, I find that when copying/pasting commands doesn’t work – instead of typing all the commands in manually, as long as you edit the command it should be fine. ie. after you copy/paste, removed the ‘p’ at the end of ‘.zip’ then retype it.

  6. Hi, thanks for this detailed write-up on this.

    In case it’s of interest, I have a health check script with email alerting for ESXi on my blog at http://blog.peacon.co.uk/hardware-health-alerting-with-esxi

  7. Has anyone gotten this to work with a Perc 6/E (External)? Any links or help is much appreciated!

    • From my understanding the Perc 6/E uses the same LSI 1078 chipset and should work using these same directions. I don’t however have any to test as my lab currently only runs 6/i and H200 RAID cards.

      • Right now I am using 1U Dell SC1435’s that have been upgraded on memory and CPU coupled with a Dell MD1000 shelf. Not having that information in the Health Status is horrible.

        I’ll give it a shot and let you know in the next couple days. Thanks!

        • I agree with you there. I have even heard that health status is missing from the dell provided ESXi installer but I can’t confirm that. Since your using a Dell server I would suggest contacting dell to see if they can provide a vib bundle for health status since VMWare removed all of them. If you can see the datastore under storage on the attached MD1000 the vib should fix the health monitoring. If you can’t see the storage you have larger problems than just health status. Also I am pretty sure for monitoring other built in Dell hardware you need the Dell provided ESXi installer. If this bundle doesn’t work you can always remove and reboot just be aware the first boot can take a lot longer after installing.

          • Yeah on the SC1435 I don’t think vSphere or the PERC 6/e is supported. It works great with upgraded quad core processors and a pile of memory so I’m not complaining other than not having Health Status.

            Everything works great on my 2950’s in my vSphere cluster with regards to the health status w/ Perc 5/i’s. As a side question, do you know of any tie-in’s for vSphere/vCenter for the Dell MD3000i that work well to administer the array?

            I’ll give the process above a shot and see if that works, otherwise I think I’m just SOL and will need to just go into the basement periodically to see if I have any dead drives. :)

            What’s the process to delete the package if it doesn’t work – or would I really even need to? If it doesn’t work, it just doesn’t work right – It doesn’t affect any of the other functionality correct?

            My datastores show up fine from the MD1000 via the Perc 6/E currently and ESXi is installed on a 4GB USB stick.

            One last question for you – Do you know if it’s possible with ESXi to maintain the array as well? Say I throw in a few more drives, to build a Raid 1 from those new drives. You can do this in Linux and other OS’s I believe but not sure if anyone got anything working with ESXi’s limited ability.

          • If the storage array shows up just fine the drivers are working properly. The 5i/6i/6e etc. are all supported by ESXi 5.0 just the vendor related vib bundles for monitoring were removed. The vib bundle only enables health monitoring for the RAID card and drives in the array and does not mess with the underlying driver functions. To update those you need the rollup update ISO or the seperate driver from here using the LSI MegaRAID SAS driver but if you can see the datastore I wouldn’t mess with the drivers too much.

            To remove a bundle you would first need to place the host in Maintenance Mode,
            vicfg-hostops –server=server_name –operation enter
            Then list out the currently installed vib bundles,
            esxcli –server=server_name software vib list
            Then remove the bundle by name,
            esxcli –server=server_name software vib remove –vibname=name
            Reboot after the removal is successful.

            Also note that it can take over 5 minutes for the health status to show up after a reboot so if the [+] is missing in vCenter give it some time. If after that you still cannot expand the server health status you may need to close and reconnect vCenter to the host again. If it expands and the array is still missing after a reboot the vib bundle will not work for you and you may need to look into contacting Dell for their offline bundles.

            The only way that I am aware of managing the array with ESXi booted is to have a paid license and use OpenManage from Dell which also takes care of the offline bundles and installs the correct vibs for health monitoring.

  8. FYI to all, these instructions work on the Perc 6/E as well.. Thanks again for all the help!

  9. Hi, i got myself a Perc 5/i and have been trying to get the Health Monitoring up and running. I am not sure if the card is broken tho, cause i install the bundle as you write but when i reboot there is no Storage-part in Health Status. It even sometimes drops the Software Components sometimes.

    I had firmware 5 something on the card, and i reflashed it with 7.0.1-0083_SAS_FW_Image_APP-1.12.330-1300 in hope that it would solve my issues.

    Except the health thing my storage pool is acting up, when i try to write data sometimes to the pool it kinda.. drops the whole storage, so i have to remount it. After i flashed the new firmware that stopped, but when i do my converter thing i get write errors. This is pretty much why i wanted to got some kind of health status out from the card.

    Any idea about this or do i need to find a new card or something? Thanks for the guide btw.

    I have tried ESXi 5 and ESXi 5 Rollup 1-2 to get this working

    • From my understanding the PERC 5i uses a different chipset. You would need to get the VIB bundle for that chipset and follow the same process. When buying my cards I opted for the PERC 6i card as it runs a lot better than an old PERC 5i card I used to have in a windows box. If you get a new card I suggest getting a PERC 6i.

  10. Hello,
    Is there an equivalent VIB for the Dell Perc H200 RAID card?
    I tried with the one linked in this article but it doesn’t work (actually installing it make things worse, as the health status window only contains 3 of the 10 usual entries once it’s installed)
    Thanks

    • I’ve heard mixed results from this method on the H200 cards. There are several different VIB bundles on the LSI website you can try. I would suggest finding what chipset it uses and match that. You can also try the Dell OMSA VIB bundle and see if that helps any. You can find it on the Dell website along with the Dell version of ESXi 5.

      • I already tried Dell OMSA VIB 6.5A02 and 7.0 with no changes.
        H200 uses LSISAS2008, like the 9240.
        The only other VIBs I can find on LSI site are drivers, the management tool I can only find once.

        • I suggest you contact Dell for the solution. The VIB bundle should work without issue after a host reboot. The SAS 9240-4i and 8i both use the same VIB bundle I used on the PERC 6i. Since I don’t have any of those controllers I don’t have a way to test but it should be supported. We use the PERC 6i in our machine for the write back caching and BBU features.

  11. This was hard, but I couldn’t have done this without these instructions as a basis. I have a new Fujitsu Server running ESXi 5 on a USB drive. The Fujitsu actually has built in RAID – but is too expensive to use because the BBU is over $200. So I bought a used Perc 5/i for RAID and wanted to see the disks in the health section of the vSphere client.

    I found the exicli command simply would not work against my esxi 5 host. I kept on getting the error “esxcli can only be used with version 4.0 or newer servers” which I couldn’t figure out since my assumption that esxi 5 was correctly matched to the CLI package I had installed on my Win7 box.

    So, in the end, I started playing with the console shell and then with putty. So I renamed the offline zip file to something short and used the datastore browser to copy it to the root of the datastore. Then, in putty, I ran the command:

    ~ # esxcli software vib install -d [LocalDataStore]offlinebundle.zip
    Installation Result
    Message: The update completed successfully, but the system needs to be rebooted for the changes to be

    effective.
    Reboot Required: true
    VIBs Installed: LSI_bootbank_LSIProvider_500.04.V0.24-261033
    VIBs Removed:
    VIBs Skipped:
    ~ #

    Now I see more information in the health status area. Unfortunately, the battery info is listed, but with a question mark and “unknown” and charging rather than the desired green mark. But I’m happy to have gone this far.

    Suggestions?

    • The BBU on my PERC 6i shows unknown/charging for around 6-8ish hours when I replace it unless i reset the sensors. Hopefully that is the only issue! I agree with you though, having a health status was extremely important for me as I don’t have any other method of monitoring my drives when they fail.

      • Battery status just showed up. Yes, health status is important. My expensive servers at work have nice red lights and built in monitoring systems. DIY takes a little bit more.

        Between the Perc Card in the server, the extra cables and adapters, I probably spent $100. As I said before, the Fujitsu had built in Raid, but to get RAID 5 you needed a Fujitsu BBU. It was an obscure part number and they all seemed to run well over $200. But, it sure would have been cleaner inside without all this extra “stuff.” Sigh. Did I really need to save that $100? Oh well. Probably learned more in the process.

        • I would say it was worth it the next time you have to replace the BBU and it only costs a 1/4 of the Fujitsu!

          • I’ll be running a WHS 2011 box and a Win7 box for recording TV shows. Right now, I have an HP Media Smart running WHS 1. The Fujitsu will ultimately replace the HP.

            WHS can provide info on the health of the machine, but I doubt that it can if it’s a VM. Are there other tools that can provide things like temperature in an ESXi environment?

          • Sounds about what I use mine for! I wasn’t able to get temp readings on my whitebox so I use an Arduino to run a fan controller, take temps, and show the HDD status using some LEDs and a 20×4 LCD display.

  12. Hello,
    Has someone achieved to make the LSI provider work with the Free version of ESXi 5 or does it requires another license from VMware to work?
    Thanks

  13. I had trouble finding the right file to download. This link takes you straight to the file you need: http://www.lsi.com/downloads/Public/MegaRAID%20Common%20Files/VMW-ESX-5.0.0-LSIProvider-500.04.V0.24-261033-456178.zip

  14. Thanks for sharing..

    Regards
    Raamesh Keerthi N.J

  15. I have installed the bundle and now have the PERC 6/i health monitoring enabled, but my battery status is always listed as “Partially Charged”, just like in the screenshot above. I bought a new battery and installed it and that didn’t help.

    Did I get a bad replacement or is it the software?

    • I did literally the exact same thing thinking it was the BBU that came with several of the cards I got. It should change to Fully Charged after a few minutes but will only change in vCenter if you refresh or change pages after the charge has topped off. Rebooting will also put it back in Partially Charged status for a few minutes.

      • I’ve refreshed, updated, and reset and it still stays at “Partially Charged”, so if it’s working for you then the new battery probably is bad.

        I don’t think the Dell tools run on non-Dell servers otherwise I’d throw another drive in and install Windows or Linux to use their tools to check.

        • Yea that’s a weird one. Mine definitely changes to Fully Charged after it’s done charging. Are you using the latest VIB from the LSI site? I had issues with it. Try the one from the ESXi 5.1 post and see if that helps any maybe? I also get the Partially Charged status when the BBU is going through a cycle charge.

          • It turns out it was the cable.

            I used a spare drive to do a Windows 7 install and I loaded the LSI MegaRaid monitor. It showed the battery had a low charge and wasn’t charging (both the old and new battery).

            Someone suggested that I check the cable, so I did. I didn’t see anything, but I used a small screwdriver to push on the wires on the connector anyway. Once I powered everything back up I could see the battery was charging, along with an estimate to fully charged. I then check the old battery and it worked, too. I left it alone for about an hour and came back and the battery had built up about 80% of the charge.

            I removed the temporary drive and booted into ESXi. I can now see a status of “Battery Charging”.

            I’m going to get a replacement for the cable.

          • Nice! Glad it worked out before you bought more batteries! I’ll definitely keep it in mind if mine ever starts freaking out like that.

  16. This is really awesome and worked very well.

    With that said, have you had any experience expanding an existing array using a whitebox esxi solution with a perc6i card?

    • I have not since 4.1 but you would need to either install OMSA and Open Manage Essentials or use the card BIOS to get it done and then return to this method for monitoring. As always have a backup of all your data when reshaping your raid array :)

  17. Here’s how I got Health Monitoring for the LSI SAS 3041E-R to work.
    1) There is no VMWare Provider available on the download page for the 3041E, but the one from newer controllers works fine – even though the card is not even listed in the readme of those SMSI Downloads as compatible.
    2) I went to the download page for the LSI 9260-8i and downloaded the newest VMWARE SMIS Provider which currently is “00.39.V0.02 _VMWare_Provider.zip” (for ESXi v5)
    3) Switch ESXi into Maintenance mode and upload the VIB file that is contained in the ZIP to /tmp/ using SCP.
    4) login using SSH and run: esxcli software vib install -v /tmp/vmware-esx-provider-lsiprovider.vib
    5) it should now say: “update completed successfully, but the system needs to be rebooted” and “VIBs Installed: LSI_bootbank_lsiprovider_500.04.V0.39-0006″
    6) exit Maintenance Mode and reboot ESXi
    7) In vSphere Client => Configuration => Health Status you should now see “Storage” entries for the SAS1064E – which is the chipset the 3041E-R card is based on.

  18. Just wanted to add my thanks. I had similar problems to Adam in that I couldn’t get it to work with vCLI 4.1 onto my ESXi 5.1 host so used putty to connect directly to it.

    I also couldn’t get it to work using VMW-ESX-5.0.0-LSIProvider-500.04.V0.24-261033-offline_bundle-456178.zip. I tried VMW-ESX-5.0.0-LSIProvider-500.04.V0.24-261033-456178.zip but no dice.

    Specifying vmware-esx-provider-LSIProvider.vib however did the trick !

    (might have been fat-finger syndrome but I don’t think so)

    Thank you


Leave a comment