Thursday, December 31, 2015

vSphere 6 - Installation of vCenter Server 6

As stated in my previous blog post, we will use "vCenter Server" to manage our ESXi hosts unless we have very few of these (perhaps only two or three). Since vCenter Server entails a supplemental (and significant) cost, we have to arbitrate between cost and efficiency. I will assume that we have made the choice of using vCenter Server and present a brief overview of the installation process.

First, we need to download vCenter Server. It is usually part of a larger package that we can download from this site:

Download VMware vSphere

We can select the version of vSphere that we want to download. At the time I composed this blog post, the options were versions 5, 5.1, 5.5 and 6. I will use version 6.

We will have to login (or create a VMware account if we do not already have one) and select the package we want. If we have not purchased a package, we can download an evaluation copy.

The package I selected  is in the form of a rather large .iso file, roughly 2.8 GB in size:



Note: the other file is the ESXi 6 .iso file used in my previous blog post.

We need to extract the vCenter Server install files from this .iso file. I used a tool called "MagicISO" but there are surely other options. The install files are extracted (in my example) to the folder in the screenshot above. Inside this folder, among other subfolders, there is a vCenter-Server subfolder with the following contents:





***


At this point, I want to state that there are numerous prerequisites and configuration options for vCenter Server. I will only present some of those that I believe are most important, referring the reader to the official VMware documentation for complete details:


We can install vCenter Server 6 on a Windows server (Windows 2008 SP2 and above) or as a virtual appliance on an existing ESXi host. In this post, I will use the first option. Even for the smallest implementations (described as "tiny" in the documentation), the minimal hardware requirements are:
  • 2 CPUs
  • 8 GB of RAM
These are "hard" limits. In my test network, the Windows 2008 (SP2) server had 8 GB of RAM but only 1 processor. Therefore, the pre-install check failed with this error message:



Among the many other choices, we can use a "PostgreSQL" database that the vCenter Server installer will install, or designate an external database. I will use the first option. The PostgreSQL database will function with a maximum of 20 ESXi hosts and 200 virtual machines.

Note: for comparison, the limits are 1000 hosts and 10,000 virtual machines with the vCenter Server appliance.

For complete requirements, please consult this document:



***


Now we begin the installation (double click on the executable file shown above). This starts the installation:


Note: click on images to enlarge if necessary.

Accept the license agreement...



At this point, we have to make a choice: will we install vCenter Server and the component known as the (embedded) Platform Services Controller on the same server or on separate servers?



The Platform Services Controller provides shared services (for multiple vCenter Servers) in the following areas:

  • Single Sign-On
  • Licensing
  • Certificates

However, even if we only have a single vCenter Server, we still must install a Platform Services Controller. In that case, the easiest solution is to install "PSC" on the vCenter Server itself.

For further details on the PSC roles listed above, please consult the vCenter Server documentation (although it is possible I may elaborate on this subject myself in a later post).



We then choose a "System Network Name". For best results, we should have DNS configured with both forward and reverse lookup zones.




We can ignore the following warning if we do not intend to implement IPv6 (at least not as the primary IP protocol):



Single Sign-On for vCenter is one of the features of the Platform Services Controller, presented above. We must create a new SSO domain (unless there is already an existing domain). All the information shown in the screenshot below is provided for us, except for the password which we must enter.  For a practice network, we can use the default values.



I will execute vCenter Server using the Local System account (default) which is acceptable if I select the option (next step) to install the embedded "vPostgres" database. If I want to use an external database (on a separate SQL server, for example), I will have to specify an Active Directory domain account.



So here we select the embedded database option:



The next screen lists all the ports used by vCenter. Unless we have special reasons to use different ports, we can use the default ports shown here:



The next screen simply shows where vCenter will be installed. For a test environment, the default values will suffice. In other circumstances, one may consider installing elsewhere (to a separate volume):



Here is the summary of our choices:



We click "Install" and the installation progresses:



If all goes well, we should see this:




***


And this concludes my post about installing vCenter Server 6 (with Platform Services Controller) on a Windows server (2008 SP2 or above). At this point, we could (as illustrated in the final screenshot) open the vSphere Web Client. However, I will save that for a later post - along with other vSphere related subjects.

Thursday, December 17, 2015

vSphere 6 - Introduction - Installation of ESXi 6

For my next series of blog posts, I'm going to work with vSphere 6: installing and configuring ESXi and vCenter Server to begin. The posts will constitute a overview of the process(es), useful to me as reference notes, and perhaps useful to the reader as well. It is not my intention to provide a complete set of instructions covering all possible aspects of the installation and configuration process. It would be both fastidious and repetitive to reproduce, essentially, documentation already made available by VMware:


For example, there are three methods for installing ESXi: 1) manual install, 2) with a script, 3) with auto-deploy. I will perform a simple manual install (see below) which, admittedly, may not be very informative for accomplished VMware experts.

Besides the official VMware documentation mentioned above, I would like to provide some references for those who, for whatever reason, would like to consult other resources that either examine the subjects presented here, but in greater detail, or more advanced subjects that I will not have explored.


Text material

Mastering vSphere 6 by Nick Marshall and Scott Lowe (publisher: Sybex)

You can search for this title at your favorite bookseller (Amazon, for example).

Scott Lowe, alone or with others, has written books on the many of the previous versions of vSphere as well (if you happen to be using one of those earlier versions).


Video training

There are numerous video training series on vSphere, both version 6 and earlier versions, at the following sites (for a fee - various payment options):



In my opinion, both are excellent, especially from the perspective of someone learning vSphere for the first time. David Davis (PluralSight) has been producing training videos on vSphere since (at least) version 4 (back when TrainSignal still existed). His presentations are complete and to the point. I've also watched the videos by Keith Barker (CBTNuggets), perhaps best known for his Cisco material (text and video) and appreciate his effort to create a training environment in VMware workstation. For example, he'll show you how to "hack" vSphere Server Center Appliance (yes, the appliance) so you can run it (and learn it) in VMware workstation.


***


For this first exercise, I'll install ESXi 6 as a virtual host in VMware Workstation (v11). I will not present the preliminary setup in VMware Workstation. First, this would be a very unlikely scenario in practice (ESXi is almost always installed on a physical server) and second, such a presentation would have little value for those using some other virtualization software. So I'll concentrate on what we see (and must do) once the installation process begins.

Some notes on prerequisites all the same...
  • ESXi 6 (Update 1) can be installed on most  (or at least many) "modern" servers (since 2007-2008 for example). I was able to install it on an older IBM x3350 (from around 2009-2010) with no problems at all. However, in a production environment, it is crucial to use hardware on the VMware comptability list to ensure optimal support.
  • Even on supported servers, you may have to adjust BIOS settings to enable virtualization or virtualization of 64 bit operating systems. Consult the VMware documentation for more details.
  • The server must have at least two "CPU cores".
  • Minimum RAM is 4 GB but 8 GB (at least) is recommended for production environments.



We can download ESXi from VMware (after creating an account) as a .iso file. For installation on physical servers, I'll burn the .iso image to a DVD and boot from the DVD. In VMware (and presumably comparable products), we can target the .iso file in the virtual CD/DVD drive and boot from it as if we were booting from media.


We will first see a menu like this. We can press "Enter" or wait for automatic boot:



On the "Welcome" screen, press "Enter" (unless you want to cancel, of course):



Press F11 to accept the EULA:


Tip: inside of VMware workstation, if F11 alone does not work, try Fn + F11.


Select the disk on which you want to install ESXi:



And a keyboard layout:



We need to enter a root password (the default username is "root"):



The disk will be repartitioned so we have one last chance to decide if we really want to do this:



If all goes well, we have at least 60 days to experiment with ESXi in evaluation mode and more if we enter a VMware license code:



After we reboot the server (make sure to remove the installation media), we must login with the password we selected earlier (root is the default username):



How can we manage the ESXi host  (called host because it will "host" virtual guest machines)? As we will see later, we can manage the host directly with the vSphere Client or by adding it to vCenter Server. In either case, we need to configure a static IP address for the host. Press F2 (or Fn+F2) to display the following screen:


Note: besides the first (default) network adapter, we will usually add other adapters so management traffic, replication traffic (if applicable) and other traffic are segmented for greater efficiency.


We select "Network Adpaters":



We enter an IP address, subnet mask and default gateway suitable for our network:



We then restart the management network so the changes take effect:



If you look at the other options (in the Management Network screenshot above), you will see that we can optionally configure IPv6 and DNS also. For now, at least, I will leave those settings at the default values (note that by default an IPv6 link local address is configured automatically).


***

If we had only one ESXi host to manage (or a very small number of hosts), we could, by accessing with a web browser the IP address configured above, download a management tool called vSphere Client (or in more recent versions of ESXi use a web interface for management). This would have the advantage of avoiding the cost of vCenter (licensed separately - with an additional cost - from ESXi). In most networks however, it would not be practical to manage each ESXi host directly with the vSphere Client. Instead, we would install vCenter and add the ESXi hosts to the console for centralized management. That will be the subject of a future blog post.

Monday, November 30, 2015

Exchange 2010 - Restore from a "Recovery Database"

The restoration of a deleted item from a backup may be necessary when all other options have been exhausted. But before resorting to this option, we should ask ourselves:

  1. Can we simply recover the deleted item from the Deleted Items folder (the Outlook "Recycle Bin")?
  2. Can we recover the deleted item from the "Recover Deleted Items" folder (the Outlook "Dumpster")? Remember to highlight the folder from which the item was deleted when using this option.
  3. In the case of a disabled or removed mailbox (there is a slight but important difference), can we locate the mailbox in the "Disconnected Mailboxes" section of the Exchange Mailbox Console (EMC) and reconnect the mailbox to the associated Active Directory user account (or another user account)? Note that if we "removed" the mailbox, we will have deleted the associated Active Directory user account as well and would have to restore this object also.
Note: if the terms or concepts above are unfamiliar, please conduct an online search for clarification.


If we must restore a mailbox (or specific items) from backup, we can use a "Recovery Database". Previously, with Exchange 2007, this was known as a "Recovery Storage Group - RSG)" but since storage groups were eliminated from Exchange 2010 (and more recent versions), the name has changed. In this scenario, I will use Exchange 2010 (SP3 - RU10) but the procedure is very similar for Exchange 2013 (please see references at the end of this article - there is a link to the procedure for Exchange 2013).

The obvious pre-requisite is the availability of a recent backup. I created such a backup with the native Windows 2008 (R2) backup software which is now able to backup Exchange (this was not initially the case when Windows 2008 was released). The reader will have to make adjustments if using a 3rd party backup system such as Commvault or Veeam (etc.).


***

The recovery procedure can be divided into three major steps:
  1. Restore the Exchange database and logfiles to a specific location.
  2. Create (and then mount) the recovery database using these files.
  3. Perform the restore operations.
For step 3, it is essentially a matter of "copying" items from the recovery database (RDB) to the database where these items were originally located with the New-MailboxRestoreRequest cmdlet.



Restore the Exchange database and logfiles

I have created a folder named "A_RestoredFiles" on my C: drive. This is the location to which I will restore the Exchange database and transaction log files before creating (and then mounting) the RDB. Here is the restore procedure...

We open "Windows Server Backup" on the Exchange server. In the Action pane, select "Recover":





In our scenario (please see my previous blog post), we created the backup on a second hard drive. On the "Getting Started" page, we would therefore select "This server (EX13-1)":




We then select the backup we want to restore by date (and time):




Since Exchange is an application, we select the "Applications" option:




The server may have several applications but in our scenario there is only one. So we simply select Exchange:


Note: I opt not to check the roll-forward recovery option. This choice works for me (if you read what follows) but may or may not be suitable in other circumstances.



At this point, we can choose to recover to the original location or to another location. Since I want to create a separate Recovery Database using the restored files, I will select the second option and then designate the location where I want to restore the application:





We can simply click "Recover" on the confirmation screen:




And wait for the restore to complete:




The operation restores the database and log files based on their original location (file path):




Now that we have restored the database and transaction log files, we can use them to create the Recovery Database.




Create (and mount) the Recovery Database (RDB)

 When we restore the Exchange database and transaction log files to a new location, the database is usually in what is called a "Dirty Shutdown" state. We can verify this with the ESEUTIL command line tool (and we can also remedy this situation with the same tool).

First, we go to the location where the .edb file was restored and run the eseutil /mh command as shown here:

[PS] C:\>sl .\A_RestoredFiles
[PS] C:\A_RestoredFiles>sl .\C_
[PS] C:\A_RestoredFiles\C_>sl .\A-DB1
[PS] C:\A_RestoredFiles\C_\A-DB1>eseutil /mh db1.edb

Note: the Exchange database exists in a file with the .edb extension.

Note: "sl" is the abbreviation of the Powershell cmdlet "Set-Location". Similar to the traditional "cd" command, it allows us to move around the file structure at the command line.


This is probably the result that you would obtain in comparable circumstances:

[...]
State: Dirty Shutdown
Log Required: 386-388 (0x182-0x184)
Log Committed: 0-389 (0x0-0x185)
[...]

Note: the output of the eseutil /mh command is quite abundant so I have cut all but the most relevant parts above.

But why is this important?

Because a database will not mount when it is in a "Dirty Shutdown"state.

We can change the shutdown state with the eseutil /r command but eseutil needs to know the location of the transaction log files. We can either designate the location of these files or copy them to the location of the database file (eseutil looks for the log files in this location by default). I will opt for the second option...

Copy...




Paste...




Note the respective path of the database file and the log files.



We now repair the database with the ESEUTIL /R command:

[PS] C:\A_RestoredFiles\C_\A-DB1>eseutil /r E01 /i /d

[...]

Initiating RECOVERY mode...
    Logfile base name: E01
    Log files: <current directory>
    System files: <current directory>
   Database Directory: <current directory>

Performing soft recovery...
                      Restore Status (% complete)

          0    10   20   30   40   50   60   70   80   90  100
          |----|----|----|----|----|----|----|----|----|----|
          ...................................................

Operation completed successfully in 4.212 seconds.



Now we can see that the database is in a clean shutdown state and can be mounted:

[PS] C:\A_RestoredFiles\C_\A-DB1>eseutil /mh db1.edb

[...]

State: Clean Shutdown
Log Required: 0-0 (0x0-0x0)
Log Committed: 0-0 (0x0-0x0)
Log Recovering: 0 (0x0)

[...]


At this point, we need to clarify the status of the DB1.edb file we restored and repaired with ESEUTIL (so it would be in a "Clean Shutdown" state). This file is the restored database but Exchange is not aware of the existence of this file and does not recognize it as a "mailbox database". We can observe this for ourselves with the Get-MailboxDatabase cmdlet:

[PS] C:\>Get-MailboxDatabase

Name                    Server          Recovery        ReplicationType
----                           ------          --------        ---------------
DB1                      EX13-1          False           Remote

Exchange only sees the original copy of the database and not the restored file that we want to use as the Recovery Database.

Therefore, we need to create a new "Mailbox Database" with the restored file and specify that it is a "Recovery" database. We can accomplish that with the following command:

[PS] C:\>New-MailboxDatabase -Recovery -Server EX13-1 -Name RDB1 -EdbFilePath "C:\A_RestoredFiles\C_\A-DB1\DB1.edb" -LogFolderPath "C:\A_RestoredFiles\C_\A-DB1"

Note: the "-Recovery" parameter makes this mailbox database a recovery database (RDB).


We may see the following warning stating that the database must be brought into a clean shutdown state before it can be mounted. In our case, we have already accomplished this task and can disregard the warning:

WARNING: Recovery database 'RDB1' was created using existing file C:\A_RestoredFiles\C_\A-DB1\DB1.edb. The database must be brought into a clean shutdown state before it can be mounted.

Now "RDB1" is recognized as a (recovery) mailbox database.

[PS] C:\>Get-MailboxDatabase

Name                   Server          Recovery    ReplicationType
----                         ------            --------        ---------------
DB1                      EX13-1          False           Remote
RDB1                   EX13-1          True            None


This is what we see in the Exchange Management Console (EMC):



All we have to do next is mount the database:

[PS] C:\>Mount-Database RDB1


If we want to see what mailboxes are in the database, we can use the Get-MailboxStatistics cmdlet. The Get-Mailbox cmdlet does not seem to produce output with a RDB:

[PS] C:\>Get-Mailbox -Database RDB1
[PS] C:\>Get-MailboxStatistics -Database RDB1 | fl DisplayName

DisplayName : Aisha Bhari
DisplayName : Alex Heyne



Restore mailbox items from the RDB to the source database

Since Exchange 2010 SP1, the New-MailboxRestoreRequest cmdlet enables us to restore items from the RDB to the source database . This cmdlet replaces the former Restore-Mailbox cmdlet (now deprecated).

I intend to illustrate some of the possible restore options by presenting the status quo of the user mailbox of Alex Heyne (test user) when the backup mentioned above was taken and just before I deleted the items shown below (not only from the original folder locations but also from the recycle bin and the "dumpster").

Then I will show what his mailbox looks like once the items are restored.

So, when the backup was taken, there were 3 messages in Alex Heyne's Inbox and 3 messages in his Sent Items folder:






Once again, all these messages were deleted after the backup was taken: from the Inbox and Sent Items folder but also the Recycle Bin and "Dumpster".

Of course, it is only later that we realize these messages should have been retained!

How can we retrieve them?

The New-MailboxRestoreRequest with the following parameters restores the messages the their original location:

New-MailboxRestoreRequest -SourceDatabase RDB1 -SourceStoreMailbox "Alex Heyne" -TargetMailbox "Alex Heyne"

I will not post additional screenshots here because you would simply see what you see above: essentially a return to the "status quo ante" or "situation as it was before".

Note: since all the messages were deleted there were no conflicts. But what if the backup included items that were not deleted? What would happen then? Default behavior is to keep the source item or in other words overwrite the item still present in the target database. There are parameters that change this default behavior. Please consult the references at the end of this post and in particular (for this subject) the article by Paul Cunningham.


This combination of parameters restores the entire mailbox to a single folder named "Restored Files" (in my example):

New-MailboxRestoreRequest -SourceDatabase RDB1 -SourceStoreMailbox "Alex Heyne" -TargetMailbox "Alex Heyne" -TargetRootFolder "Restored Items"

And this is what the user would see in Outlook:


The entire folder structure, including the normally hidden "Recoverable Items" folder, is restored to the designated "TargetRootFolder" as well as the content of the folders (Sent Items are shown above).

Note: I have performed the restore operation with Outlook open and in that case, the items usually appear instantly with no user interaction. In the example above, however, it was necessary to close and reopen Outlook. Try this before concluding that the restore operation failed.


This combination of parameters restores only the Inbox to a single folder name "Restored Files":

New-MailboxRestoreRequest -SourceDatabase RDB1 -SourceStoreMailbox "Alex Heyne" -TargetMailbox "Alex Heyne" -TargetRootFolder "Restored Items" -IncludeFolders "#Inbox#"


Note: the "Recoverable Items" folder is still restored. We can avoid this by adding the following parameter:

-ExcludeDumpster


One more example: the following combination allows us to restore Alex Heyne's entire mailbox to the specified folder inside of Alan Reid's mailbox:

New-MailboxRestoreRequest -SourceDatabase RDB1 -SourceStoreMailbox "Alex Heyne" -TargetMailbox "Alan Reid" -TargetRootFolder "Restored Items" -AllowLegacyDNMismatch



This could be useful if the user is no longer with the organization and someone has been designated to recover some important items (pertaining to a contract, for example) from a backup of the former employee's mailbox.

The "AllowLegacyDNMismatch" parameter is mandatory if we want to restore the mailbox into another user's mailbox like in the example above. By default, Exchange will not allow this. We have to specify explicitly that this is our intention. Otherwise, we will see the following error:

Source mailbox's legacyExchangeDN '/O=MYNET/OU=EXCHANGE ADMINISTRATIVE GROUP (FYDIBOHF23SPDLT)/CN=RECIPIENTS/CN=ALEX.HEYNE' doesn't match [...].


The restore request may require some time to finish. We can check the status of the request(s) with this cmdlet:

Get-MailboxRestoreRequest


After the restore is complete, we can remove the restore requests with this combination of cmdlets:

Get-MailboxRestoreRequest | Remove-MailboxRestoreRequest


Furthermore, there is no need to keep the Recovery Database permanently, so we would want to dismount it and then remove it with the following cmdlets:

Dismount-Database RDB1

Remove-MailboxDatabase RDB1

After we execute the cmdlet above, we should see a warning like this: 

WARNING: The specified database has been removed. You must remove the database file located in... C:\RestoredFiles\C_\A-DB1\DB1.edb from your computer manually if it exists. Specified database: RDB1

It is simply a matter of deleting the files in question from either File Explorer or the command line.





References:

Technet reference:



Article by Exchange MVP Mike Pfeiffer:



Article by Exchange MVP Paul Cunningham:


Saturday, November 28, 2015

Exchange 2010 - Native Windows backup of Exchange

This is a short blog post that presents the backup of an Exchange 2010 (SP3 - RU10) server using native Windows backup. It is meant to be informational in itself but also fulfills a preliminary requisite for the following blog post in which I will demonstrate how to restore mailbox items using what is called a "Recovery Database" or "RDB"). Indeed, we cannot perform a restore without a backup to begin with.

***

First, we go to "Administrative Tools" and select "Windows Server Backup". This is a Windows Server  "feature" (as opposed to a "role"). If the "WSB" feature has not been added, an error message will display. We can overcome this obstacle by installing the feature.


For this exercise, we will click on the "Backup Once" option in the Action pane:




Then "Different options":




I will select "Custom" for the backup configuration. This allows me to see exactly what elements I will backup:




On the "Select Items for Backup" page, we have to click on the "Add Items" button:





We will select all the boxes except "Backup (B)". This is an additional hard drive that we will use to store the backup:





We click "OK" and return to the configuration page. We then click on "Advanced Settings" and then the "VSS Settings" tab. I will select the "VSS full Backup" option since I am not using another product (other than Windows native backup) for the backup:




For this exercise, I will use a local drive (a shared folder on the network is the other option):




We then select the specific volume itself:




I confirm the settings selected (and click OK):




I can observe the progress of the backup during the execution of the backup job:




If all goes well, status for all backup operations should read "Completed".




***


In reality, we would probably want to schedule a backup to occur at regular intervals, targeting, perhaps, a remote shared folder elsewhere on the network.

In any case, we now have a "full server backup" which includes, besides the Windows OS, the Exchange system files, databases and log files (some of which have been truncated during the backup operation). We will use this backup in my next blog post to create a "Recovery Database" from which we can (essentially) copy items from the "RDB" to the database where the items existed before they were deleted.