Invalid Virtual Machine Configuration

When a Snapshot of a VM is created and one of the disks is removed prior to removal of the Snapshot, the error “Invalid Virtual Machine Configuration” will appear when attempting to delete that snapshot. This will also prevent any additional snapshots from being created.

In our situation, a snapshot was taken using NetApp Virtual Storage Console plugin during a schedule backup job. At the time of snapshot removal an Oracle load test was being performed on the same storage system. This caused excessive latency and prevented the snapshot from being removed. Follow the steps below to fix this issue.

1. Locate the Virtual machine in vCenter that is throwing this error and select it
2. Click on the “Summary” tab for the VM
invalidVM121214-step2
3. Under “Storage, Right-click on the OS data drive and click “Browse datastore”
invalidVM121214-step3
4. Locate the Folder for this Virtual Machine and open it
5. Locate the file <vmname>.vmsd
invalidVM121214-step5
6. Right click on the .VMSD file and choose “rename”
invalidVM121214-step6
7. Change the name to <vmname>.vmsd.old
invalidVM121214-step7
8. Right click on the Virtual Machine, hover over “Snapshot” then choose “Take Snapshot”
invalidVM121214-step8
9. Enter a name and ensure both boxes are unchecked and click “OK”
invalidVM121214-step9
10. The VM snapshot might fail with a message “A general system error occurred”, this is normal.
11. Right click on the Virtual Machine, hover over “Snapshot”, then choose “Snapshot Manager”
invalidVM121214-step11
12. The previous Snapshot that was there will be gone, but the recent snapshot will remain (this is normal). Click the Snapshot and click “Delete All” and “Yes” to confirm delete
invalidVM121214-step12
13. Try taking a new snapshot and ensure it works
14. As a matter of clean, ensure that you delete the <vmname>.vmsd.old file once you’re finished. No need to leave stale files laying around

Invalid Virtual Machine Configuration

Why VMUG?

As I think back on my career and where I began I’m reminded of the people in my professional life that have helped me along the way. My first job was at a nationwide company primarily doing desktop support for the local office as well as some of the remote locations. My boss, Jason, was someone I related to and we had a great working relationship. I could ask for help on anything and he would stop what he was doing and make time to show me the right way to do it; not just do it for me. While I reported to him, he treated me as an equal.

My next job was for a startup and I was the lone IT guy. No matter the task, whether IT or not, I did it. It was a job where I had ownership over everything. The attitude that the success of our company rested on my shoulders was something that drove me to work harder each day. My boss, Ron, was a great person that I could freely talk to about everything. We brainstormed together how this business would run, where it would go, and how we’d get there. There was a mutual respect.

At a later job I met one of my favorite boss’s and one of my better friends. While we both struggled with the company itself,¬†we made the best of our situation and we worked hard for each other. He trusted me and believed in me. He saw a lot in me and throughout the years has pushed me to do more and to be better than what I was. Not just in my career, but in my life as well.

My favorite jobs all had one thing in common: 1 person that made me better, that kept me learning, and that pushed and encouraged me. The person I bounced all of my (sometimes ridiculous) ideas off of and they listen to all of them. When you’re building your career, that is what you need. You need the help of people around you to build you up.

The point of this post isn’t to talk about how important your boss or good co-workers are. Sometimes you don’t have the benefit of having a boss or good co-workers who CAN or WILL teach you. Sometimes you’re a 1 man IT shop and all you have is yourself and Google. When you don’t have that support at your job that’s when you need a good community.

Why is VMUG and this community so important? . When I started working with VMware I learned how to edit some Virtual Machine settings and then made a ton of assumptions on how things worked because I never had the time to learn and didn’t always have the benefit of someone to ask. Each and every day I’m just trying to keep my head above water, but I’m not growing in my career. Those were the days I wished I had a community I could go to and ask all the questions I had.

With so many VMware products and so many configuration options, every member adds value to VMUG. Being active in in VMUG isn’t just for “experts”. Sharing the knowledge you’ve gained in YOUR environment can help someone in theirs. When you share your struggles the community is there for you. When someone else shares their struggles, you can be there for them.

I have recently joined as a leader for VMUG in Portland, Oregon. The vision I have for us is a community that is actively working to help each other succeed. We can ask questions, share ideas, or just talk over beers during a happy hour. I didn’t want to be a VMUG leader because I’m an expert, far from it. I wanted to be a leader because I want to see us succeed. I want each and every VMUG member to know they have a place to turn whenever they need help. The only way we can be successful is if our members are active and talk to each other.

The more events you come to, the more connections you’ll make, the larger your community will grow and the better this VMUG will be. Everyone has value regardless of how big or how small the environment they support. No matter the skill level, years of experience, certifications or any other factor. The VMware User Group is nothing without its users.

We are all in this together and your VMUG community is on your side.

Why VMUG?

Restore Files & AD Objects From NetApp & Veeam v8

With the release of Veeam Backup & Replication v8 we can restore directly from NetApp Snapshots. Whether it’s an entire VM, individual files, or just some objects in Active Directory, you can do it all from the Veeam console. For a guide on installing and configuring Veeam v8 with NetApp storage, click here

We’ll be testing the restore of individual files and some Active Directory objects for this blog post. In this scenario we have a couple Domain Controllers (2008 R2) and a couple of member servers with some files that we’ll delete. We also have an OU with a couple users, a member server, and a group.

Each of these VMs sit on either of these two volumes, Win_2008 and Win_2012. If you click on “Storage Infrastructure” in the Veeam Backup and Replication console, then expand your NetApp storage you’ll see a list of all the volumes available and their snapshots.
veeamrest120114-part1

1. I’ve taken a snapshot in NetApp System Manager of these volumes. To list these snaps, refresh the volume by right-clicking on the volume and choosing “Rescan volume” or right click on the storage array and choose “Rescan Storage” (Since we have 2 volumes to refresh, we’ll rescan storage.
veeamrest120114-step1
2. A new window will popup showing the progress
veeamrest120114-step2
3. Once completed, we now see the new snapshot I created called “Pre-delete”
veeamrest120114-step3
4.I’m going to delete a file from the server “Lab2008” (on the Win_2008 datastore) and “Lab2012” (on the Win_2012 datastore) that are sitting on my desktop.
veeamrest120114-step4a
veeamrest120114-step4b
5. And let’s also delete the OU “Delete Test” which contains a couple test users, a group they are apart of and the VM “Lab2008”
veeamrest120114-step5
6. Now that those files and OU\objects have been delete, let’s go back to the Veeam console and see what we can recover. We’ll start with the files for the “Lab2012” VM.
7. Expanding “Win_2012” datastore in “Storage Infrastructure” view, click on the name of the snapshot I created earlier and we see the “Lab2012” VM
veeamrest120114-step7
8. We right-click on “Lab2012”, hover over “Restore guest files” and then choose “Microsoft Windows”
veeamrest120114-step8
9. Under the “File Level Restore” screen, click “Customize” in the bottom right corner
veeamrest120114-step9
10. As long as you’re restoring to a vCenter/Host that’s already been added to Veeam, choose the host, resource pool (if any) and folder. Click “OK” then click “Next”
veeamrest120114-step10
11. Enter a reason for the restore and click “Next”
veeamrest120114-step11
12. Click “Finish”
veeamrest120114-step12
13. The restore session will open and mount the snapshot/VM to the chosen host
veeamrest120114-step13
14. In vCenter, we see these 2 tasks of creating a datastore and registering the virtual machine.
veeamrest120114-step14
15. On the host, we see a new powered off VM with the name of “Lab2012” followed by a GUID.
veeamrest120114-step15
16. Back at the Veeam console, the Backup Browser window appears and we can browse to the location of the deleted file
veeamrest120114-step16
17. From here, we can copy the file to our local machine or restore it directly to the Virtual Machine. Right click on the file and choose “Restore” then “Overwrite”
veeamrest120114-step17
18. We’ll pick “Use the following account” and choose my Lab Domain credentials and click “OK”
veeamrest120114-step18
19. The restore process will start and you’ll see this output if you click “Show Details”
veeamrest120114-step19
20. Logging back in to “Lab2012” we can see the file has been restored
veeamrest120114-step20
21. Close the “Restoring files” window in the Veeam console and the “Backup Browser” window. After they’re closed, the VM will be unregistered on the host and the datastore will be unmounted.
22. I’m doing a restore from “Lab2008” but this time I will just copy the file to my local computer instead of restoring to the guest VM. After browsing the datastore snapshots and choosing “Restore Guest Files”, we’ll browse the directory structure, locate the file, right-click and choose “Copy To”
veeamrest120114-step22
23. A window will pop up to choose the folder location on your machine and whether to preserve permissions and ownership. Then click “OK”
veeamrest120114-step23
24. Now in the root of the C: drive we have the “Lab2008-txt” file
veeamrest120114-step24
25. Let’s look at the “Lab2008” VM now. It was in that OU we deleted and after rebooting it and trying to login we receive the message “The security database on the server does not have a computer account for this workstation trust relationship”. We can fix that.
veeamrest120114-step25
26. Back in the Veeam console and the “Pre-delete” snapshot for the “Win_2008” datastore, we’ll locate the “Lab-DC01” VM. Right click on the VM, hover over “Restore application items” and then click “Microsoft Active Directory objects”
veeamrest120114-step26
27. Our host settings are saved from the last restore we did, so click “Next”
veeamrest120114-step27
28. Enter a restore reason and click “Next”
veeamrest120114-step28
29. Review the summary and click “Finish”
veeamrest120114-step29
30. The Veeam Explorer for Microsoft Active Directory window will appear
veeamrest120114-step30
31. Then the VM will be mounted in vCenter
veeamrest120114-step31
32. Once the Veeam Explorer window for AD opens, you’ll be able to browse your Domain object. We’ll expand the “LabOU” object where we see “Delete Test” with the same 2 test users, “Lab2008” server and the group those users belong to.
veeamrest120114-step32
33. Right click the “Delete Test” OU and choose “Restore container to LabDC.local”
veeamrest120114-step33
34. Enter the credentials for the account with access to add objects to the domain and click “OK”
veeamrest120114-step34
35. You’ll see the progress of the restore and then the summary of how many objects were restored
veeamrest120114-step35

(In order for this to work your Veeam server will need network access to the live domain controller)

36. If we refresh the screen for Active Directory Users and Computers on “Lab-DC01” we’ll see the OU is back with all of it’s objects
veeamrest120114-step36
37. In the properties for the users, we can see that group membership was retained. The group “Email Group” is located in another OU and that membership was restored as well
veeamrest120114-step37
38. And now when we try to login to “Lab2008” with domain credentials it works with no issues.

 

How fast can this restore happen? From the time I opened the Veeam console until the time the OU reported as being restored took 3 minutes and 34 seconds. In an emergency where someone accidentally deletes an entire OU, a user account, a server, or anything else, they can all be restored in under 5 minutes time without the need to reset any passwords and everything will work without anyone ever noticing. Veeam is awesome and just keeps getting better and better.

Restore Files & AD Objects From NetApp & Veeam v8