Friday, February 28, 2014

Windows Server 2012 - Active Directory - SYSVOL - migration from FRS to DFS-R

Group Policy settings (in part) and logon scripts are stored in the subfolders of the C:\Windows\SYSVOL folder (unless Windows is installed on another drive of course). It is crucial that these settings and scripts are replicated among domain controllers so the settings are applied and the scripts executed regardless of the domain controller that authenticates the user (or computer).

The traditional mechanism for this replication, for Windows 2000 and 2003, was FRS (File Replication Service). FRS can still be used, even with Windows 2012 R2, but since Windows 2008, there is another possible mechanism: DFS-R. This system is considered more reliable than FRS.

The objective of this blog post is not to present either FRS or DFS-R in detail but to share my experience in migrating from the first to the second.

 There are a number of resources that can be consulted for those interested in such a presentation, for example:


In the lines that follow, I'll presenting my experience with the migration process, using two domain controllers, one running Windows 2008 R2 (SP1) and the other Windows 2012.



Demonstration of the SYSVOL replication mechanism

First, I'll conduct a basic test of the SYSVOL replication mechanism and illustrate how it functions.

Let's create a logon script that should be available on all domain controllers - in our case, both domain controllers.

I'll create it on DC7 and see if it replicates to DC2.

DC2 and DC7 are simply the names of the test machines I had set up for the exercise. No need to worry about what happened to DC1 and DCs 3 to 6.

I'll also use this opportunity to illustrate some file management with Powershell. So everything will happen at the command line: no GUI, no images!

Creation of script on DC2 - and replication to DC7

PS C:\> sl windows
PS C:\windows> sl sysvol
PS C:\windows\sysvol> sl domain
PS C:\windows\sysvol\domain> sl scripts
PS C:\windows\sysvol\domain\scripts> ni logonscript1.txt -type file

Note (Powershell): "ni" is short for "New-Item". There is also a -Name parameter but this is "positional" so if we enter the name directly after the cmdlet, it is not required. There is a -Path parameter as well. If none is indicated, the current location is used.

Note (Powershell): "sl" is short for Set-Location. It is the equivalent of the old "cd" command (which would also function here as well, for that matter).

And yes, I realize a "logon script" with a .txt extension would never be executed. In reality, I would have to change it to .bat or .cmd (and add some content).

I will not press ENTER right away. Let's first look at the corresponding folder on DC7.

We'll initiate a remote session to view this:

PS C:\> New-PSSession dc2.mynet.lan
PS C:\> Enter-PSSession 1

Note: now I'm on DC2, via the remote session. I'll browse to the scripts folder...

[dc2.mynet.lan]: PS C:\Users\admin\Documents> sl \
[dc2.mynet.lan]: PS C:\>
[dc2.mynet.lan]: PS C:\> sl windows
[dc2.mynet.lan]: PS C:\windows> sl sysvol
[dc2.mynet.lan]: PS C:\windows\sysvol> sl domain
[dc2.mynet.lan]: PS C:\windows\sysvol\domain> sl scripts
[dc2.mynet.lan]: PS C:\windows\sysvol\domain\scripts> gci
[dc2.mynet.lan]: PS C:\windows\sysvol\domain\scripts>

Nothing... the Get-ChildItem cmdlet (or the abbreviated version "gci") produces no results.


Now let's press enter on DC7 so the file is created:

PS C:\windows\sysvol\domain\scripts> ni logonscript1.txt -type file

    Directory: C:\windows\sysvol\domain\scripts

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---         2/27/2014   8:41 PM          0 logonscript1.txt


Now... returning to DC2, we execute Get-ChildItem again and almost immediately the file appears:


[dc2.mynet.lan]: PS C:\windows\sysvol\domain\scripts> gci

    Directory: C:\windows\sysvol\domain\scripts

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---         2/27/2014   8:41 PM          0 logonscript1.txt


Preliminary verifications - introduction to the DFSRMIG tool

Before migrating SYSVOL from FRS to DFS-R, we can verify many aspects of Active Directory health and replication health in particular.

We can verify that the SYSVOL folder is shared with the NET SHARE command. Since the migration process changes the shared resource from the SYSVOL folder to a new folder named SYSVOL_DFSR, we can follow the evolution of this process by executing the command after each step. For readbility and "neatness", I may edit (remove) some of the other entries later.

Share name   Resource               Remark
---------------------------------------------------------------
C$           C:\                             Default share
IPC$                                         Remote IPC
ADMIN$  C:\Windows             Remote Admin
NETLOGON C:\Windows\SYSVOL\sysvol\mynet.lan\SCRIPTS [...]
SYSVOL       C:\Windows\SYSVOL\sysvol        Logon server share


We can run DCDIAG and REPADMIN /REPLSUM and REPADMIN /SHOWREPL.

These checks, while useful, are not the subject of this post.

There is, however, one prerequisite that must be met before we can migrate FRS to DFS-R:

The domain functional level (DFL) must be at Windows 2008 (or higher).

Let's verify that... with PowerShell.

PS C:\> Get-ADDomain | fl DomainMode

DomainMode : Windows2008Domain


Perfect.

Now we can begin the migration.

The process is gradual, following 4 "states":

  • 0 - Start = FRS is used for replication.
  • 1 - Prepared = FRS continues to replicate SYSVOL content. In addition, a copy of SYSVOL is created in a folder named SYSVOL-DFSR.
  • 2- Redirected = the SYSVOL share, instead of referring to the folder SYSVOL\sysvol now refers to SYSVOL_DFSR\sysvol. However, the content of both folders continues to replicate.
  • 3 - Eliminated = FRS is not longer used for replication.


DFSRMIG

The tool used is DFSRMIG. It has 3 primary commands:

  • /setglobalstate = this migrates SYSOL replication between the states mentioned above: Start, Prepared, Redirected, Eliminated.
  • /getglobalstate = this show the global or overall status of the migration
  • /getmigrationstatus = this shows the migration status on particular domain controllers.

Migration to the "Prepared" State

We can check migration status with the two "get" commands presented above...

PS C:\> dfsrmig /getglobalstate
DFSR migration has not yet initialized. To start migration please set global state to desired value.

PS C:\> dfsrmig /getmigrationstate
DFSR migration has not yet initialized. To start migration please set global state to desired value.

Note: we can set the global state to 0 first. I proceeded directly to 1.

PS C:\> dfsrmig /setglobalstate 1

Current DFSR global state: 'Start'
New DFSR global state: 'Prepared'

Migration will proceed to 'Prepared' state. DFSR service will
copy the contents of SYSVOL to SYSVOL_DFSR
folder.

If any domain controller is unable to start migration, try manual polling
Or run with option /CreateGlobalObjects.

Migration can start anytime between 15 minutes to 1 hour.
Succeeded.

So where is the new SYSVOL_DFSR folder located? Right next to the SYSVOL folder:

PS C:\windows> gci *sysvol*

    Directory: C:\windows

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
d----         2/25/2014   7:12 PM            SYSVOL
d----         2/27/2014   7:22 PM            SYSVOL_DFSR

Note that it has not yet been created on DC2:

[dc2.mynet.lan]: PS C:\windows> gci *sysvol*

    Directory: C:\windows

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
d----         8/19/2012   9:18 PM            SYSVOL

We have to wait a while:

[dc2.mynet.lan]: PS C:\windows> gci *sysvol*

    Directory: C:\windows

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
d----         8/19/2012   9:18 PM            SYSVOL
d----         2/27/2014  10:25 PM           SYSVOL_DFSR

We can verify that the migration state is now "Prepared" on all domain controllers:

PS C:\windows> dfsrmig /getglobalstate
Current DFSR global state: 'Prepared'
Succeeded.

PS C:\windows> dfsrmig /getmigrationstate
All domain controllers have migrated successfully to the Global state ('Prepared').
Migration has reached a consistent state on all domain controllers.
Succeeded.


At this point, the SYSVOL share is still associated with the SYSVOL folder.

PS C:\windows> net share

Share name   Resource                        Remark
---------------------------------------------------------------
[snip]
NETLOGON C:\Windows\SYSVOL\sysvol\mynet.lan\SCRIPTS [...]
SYSVOL       C:\Windows\SYSVOL\sysvol        Logon server share


Note that although FRS is still used for replication (with the SYSVOL folder still the "target" of the share, the content has been copied - not moved - to the SYSVOL_DFSR folder)...


PS C:\windows\sysvol\domain\scripts> gci

    Directory: C:\windows\sysvol\domain\scripts

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---         2/27/2014   5:41 PM          0 logonscript1.txt

PS C:\windows\sysvol\domain\scripts> sl ..
PS C:\windows\sysvol\domain> sl ..
PS C:\windows\sysvol> sl ..
PS C:\windows> sl sysvol_dfsr
PS C:\windows\sysvol_dfsr> sl domain
PS C:\windows\sysvol_dfsr\domain> sl scripts
PS C:\windows\sysvol_dfsr\domain\scripts> gci

    Directory: C:\windows\sysvol_dfsr\domain\scripts

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---         2/27/2014   5:41 PM          0 logonscript1.txt



Migration to the "Redirected" state

Now let's proceed to state 2: Redirected...

PS C:\windows> dfsrmig /setglobalstate 2

Current DFSR global state: 'Prepared'
New DFSR global state: 'Redirected'

Migration will proceed to 'Redirected' state. The SYSVOL share
will be changed to SYSVOL_DFSR folder,
which is replicated using DFSR.
Succeeded.

Let's verify the status...

PS C:\windows> dfsrmig /getglobalstate
Current DFSR global state: 'Redirected'
Succeeded.

PS C:\windows> dfsrmig /getmigrationstate

The following domain controllers have not reached Global state ('Redirected'):
Domain Controller (Local Migration State) - DC Type
===================================================
DC2 ('Prepared') - Primary DC

Migration has not yet reached a consistent state on all domain controllers.
State information might be stale due to Active Directory Domain Services latency.


So it appears that the change has not yet replicated to DC2. We simply need to wait a moment (several minutes)...

PS C:\windows> dfsrmig /getmigrationstate

All domain controllers have migrated successfully to the Global state ('Redirected').
Migration has reached a consistent state on all domain controllers.
Succeeded.

Note that NET SHARE now shows the SYSVOL_DFSR folder as the resource for the SYSVOL share:

PS C:\> net share

Share name   Resource                        Remark
---------------------------------------------------------------
[snip]
NETLOGON     C:\Windows\SYSVOL_DFSR\sysvol\mynet.lan\SCRIPTS
SYSVOL       C:\Windows\SYSVOL_DFSR\sysvol   Logon server share


Let's see which folder is being used to replicate...

PS C:\windows>
PS C:\windows> sl sysvol
PS C:\windows\sysvol> sl domain
PS C:\windows\sysvol\domain> sl scripts
PS C:\windows\sysvol\domain\scripts> ni logonscript2.txt -type file

    Directory: C:\windows\sysvol\domain\scripts

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---         2/28/2014   4:54 PM          0 logonscript2.txt



So we've created another file on DC7 in the SYSVOL folder (one of its subfolders). Does it replicate to DC2?

[dc2.mynet.lan]: PS C:\windows\sysvol\domain\scripts> gci

    Directory: C:\windows\sysvol\domain\scripts

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---         2/27/2014   8:41 PM          0 logonscript1.txt
-a---         2/28/2014   7:54 PM          0 logonscript2.txt


Observation: the contents of SYSVOL still replicate at this state (even though the share is now associated with SYSVOL_DFSR.

We'll create a file in the SYSVOL_DFSR folder (one of its subfolders) to verify that replication is taking place here as well...

PS C:\windows\sysvol_dfsr\domain\scripts> ni logonscript3.txt -type file

    Directory: C:\windows\sysvol_dfsr\domain\scripts

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---         2/28/2014   4:59 PM          0 logonscript3.txt

Does it replicate to DC2?

[dc2.mynet.lan]: PS C:\windows\sysvol\domain\scripts> sl \
[dc2.mynet.lan]: PS C:\>
[dc2.mynet.lan]: PS C:\>
[dc2.mynet.lan]: PS C:\> sl windows\sysvol_dfsr\domain\scripts
[dc2.mynet.lan]: PS C:\windows\sysvol_dfsr\domain\scripts> gci

    Directory: C:\windows\sysvol_dfsr\domain\scripts

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---         2/27/2014   8:41 PM          0 logonscript1.txt
-a---         2/28/2014   7:59 PM          0 logonscript3.txt

Yes, replication functions in both locations.


Migration back to the "Prepared" state

(strictly optional - to demonstrate ability to revert to a previous state)


At this state, we can still turn back, which I'll demonstrate before irrevocably eliminating FRS from the replication process:

PS C:\> dfsrmig /setglobalstate 1

Current DFSR global state: 'Redirected'
New DFSR global state: 'Prepared'

Migration will proceed to 'Prepared' state. The SYSVOL share
will be changed to SYSVOL folder, which is replicated using FRS.
Succeeded.

However, the change back is not immediate as the following shows:

PS C:\> dfsrmig /getglobalstate

Current DFSR global state: 'Prepared'
Succeeded.

PS C:\> dfsrmig /getmigrationstate

The following domain controllers have not reached Global state ('Prepared'):

Domain Controller (Local Migration State) - DC Type
===================================================
DC2 ('Redirected') - Primary DC
DC7 ('Redirected') - Writable DC

Migration has not yet reached a consistent state on all domain controllers.
State information might be stale due to Active Directory Domain Services latency.

This takes some time (several minutes) and strangely, it is DC7 (the server on which I performed the operation) that reverts back to the "Prepared" state with the longest delay.

PS C:\> dfsrmig /getmigrationstate

The following domain controllers have not reached Global state ('Prepared'):
Domain Controller (Local Migration State) - DC Type
===================================================
DC7 ('Redirected') - Writable DC

Migration has not yet reached a consistent state on all domain controllers.
State information might be stale due to Active Directory Domain Services latency.


After several more minutes, the change is effective:

PS C:\> dfsrmig /getmigrationstate

All domain controllers have migrated successfully to the Global state ('Prepared').
Migration has reached a consistent state on all domain controllers.
Succeeded.



Migration to the "Eliminated" state

Now let's move forward and make DFS-R the one and only replication mechanism (for SYSVOL) on our domain controllers.

This should look familiar...

PS C:\windows> dfsrmig /setglobalstate 2

Current DFSR global state: 'Prepared'
New DFSR global state: 'Redirected'

Migration will proceed to 'Redirected' state. The SYSVOL share
will be changed to SYSVOL_DFSR folder, which is replicated using DFSR.
Succeeded.

Let's verify...

PS C:\> dfsrmig /getglobalstate

Current DFSR global state: 'Redirected'
Succeeded.

PS C:\> dfsrmig /getmigrationstate

All domain controllers have migrated successfully to the Global state ('Redirected').
Migration has reached a consistent state on all domain controllers.
Succeeded.

So we are once again back at "Redirected" state (2).

We are at the point of no return.

We would have to rebuild the domain to return to FRS.

So here we go....

PS C:\> dfsrmig /setglobalstate 3

Current DFSR global state: 'Redirected'
New DFSR global state: 'Eliminated'

Migration will proceed to 'Eliminated' state. It is not possible to revert this step.
If any read-only domain controller is stuck in the 'Eliminating' state for too long
run with option /DeleteRoNtfrsMember.
Succeeded.

It's better to know there is no turning back, because there is no "Are you sure you want to do this?" warning.

After several minutes, the transition is complete:

PS C:\> dfsrmig /getglobalstate

Current DFSR global state: 'Eliminated'
Succeeded.

PS C:\> dfsrmig /getmigrationstate

All domain controllers have migrated successfully to the Global state ('Eliminated').
Migration has reached a consistent state on all domain controllers.
Succeeded.

The SYSVOL folder is eliminated:

PS C:\windows> sl sysvol

sl : Cannot find path 'C:\windows\sysvol' because it does not exist.
[snip]

This completes the migration.


Sunday, February 16, 2014

Windows Server 2012 - Active Directory - Manual Removal of a domain controller, Part 2: metadata cleanup

Now that we have seized the operations masters (FSMO roles) on the second of our two domain controllers (the first being out of commission), we need to remove references to the defunct domain controller so clients are no longer directed to use it (in DNS for example).

With Windows 2012 (and even with Windows 2008) there are a number of methods to achieve this objective.

The "traditional" method uses NTDSUTIL to perform the cleanup. In fact, there are some variations. We can type out the complete commands one after another, we can use abbreviated commands and we can type a single command.

Since Windows 2008, we can also use the GUI for metadata cleanup.

In either case, my experience (and that of some colleagues) has shown that even after performing the cleanup operation in NTDSUTIL (or the GUI equivalent), there is still some follow-up cleanup to perform in DNS.

Note: optionally, we can configure the new PDCe to receive correct time from an outside source as well.




NTDSUTIL - option 1 (step by step, complete commands)

Note: for readibility, I have changed the font of some of the most verbose output so the reader can concentrate on the commands (in bold and underlined), while still having the output for reference.

PS C:\> ntdsutil
C:\Windows\system32\ntdsutil.exe: activate instance ntds
Active instance set to "ntds".
C:\Windows\system32\ntdsutil.exe: metadata cleanup
metadata cleanup: connections
server connections: connect to server DC5
Binding to DC5 ...
Connected to DC5 using credentials of locally logged on user.
server connections: quit
metadata cleanup: select operation target
select operation target: list domains
Found 1 domain(s)
0 - DC=mynet,DC=lan
select operation target: select domain 0
No current site
Domain - DC=mynet,DC=lan
No current server
No current Naming Context
select operation target: list sites
Found 1 site(s)
0 - CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan
select operation target: select site 0
Site - CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan
Domain - DC=mynet,DC=lan
No current server
No current Naming Context
select operation target: list servers in site
Found 2 server(s)
0 - CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan
1 - CN=DC5,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan
select operation target: select server 0
Site - CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan
Domain - DC=mynet,DC=lan
Server - CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan
        DSA object - CN=NTDS Settings,CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan
        DNS host name - DC2.mynet.lan
        Computer object - CN=DC2,OU=Domain Controllers,DC=mynet,DC=lan
No current Naming Context
select operation target: quit
metadata cleanup: remove selected server

Transferring / Seizing FSMO roles off the selected server.
Removing FRS metadata for the selected server.
Searching for FRS members under "CN=DC2,OU=Domain Controllers,DC=mynet,DC=lan".

Removing FRS member "CN=DC2,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=mynet,DC=lan".

Deleting subtree under "CN=DC2,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=mynet,DC=lan".

Deleting subtree under "CN=DC2,OU=Domain Controllers,DC=mynet,DC=lan".
The attempt to remove the FRS settings on CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan failed because "Element not found.";
metadata cleanup is continuing.

"CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan" removed from server "DC5"

metadata cleanup: q
C:\Windows\system32\ntdsutil.exe: q
PS C:\>




NTDSUTIL option 2 - abbreviated commands

Note: the commands entered in the previous section can be more or less abbreviated, as shown below, as long as there is no ambuguity with other ntdsutil commands. Once again, I have made minor edits (font size and spacing) for readbility.

PS C:\Users\ufc> ntdsutil "act ins ntds" "meta clean" conn "co to ser DC5" q "s o t" "l d"

C:\Windows\system32\ntdsutil.exe: act ins ntds
Active instance set to "ntds".
C:\Windows\system32\ntdsutil.exe: meta clean
metadata cleanup: conn
server connections: co to ser DC5
Binding to DC5 ...
Connected to DC5 using credentials of locally logged on user.
server connections: q
metadata cleanup: s o t
select operation target: l d
Found 1 domain(s)

Note: we stopped the command above at "list domains" or "l d" since the choices that follow depend on the number of domains and the names of the sites and servers, which we may not know beforehand. If we do, we can enter all the information on a single line as shown in the next example.

0 - DC=mynet,DC=lan
select operation target: sel dom 0
No current site
Domain - DC=mynet,DC=lan
No current server
No current Naming Context
select operation target: list sites
Found 1 site(s)
0 - CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan
select operation target: sel site 0
Site - CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan
Domain - DC=mynet,DC=lan
No current server
No current Naming Context
select operation target: list serv in site
Found 2 server(s)
0 - CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan
1 - CN=DC5,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan
select operation target: sel ser 0
Site - CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan
Domain - DC=mynet,DC=lan
Server - CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan
        DSA object - CN=NTDS Settings,CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan
        DNS host name - DC2.mynet.lan
        Computer object - CN=DC2,OU=Domain Controllers,DC=mynet,DC=lan
No current Naming Context

select operation target: q
metadata cleanup: rem sel ser

Transferring / Seizing FSMO roles off the selected server.
Removing FRS metadata for the selected server.
Searching for FRS members under "CN=DC2,OU=Domain Controllers,DC=mynet,DC=lan".

Removing FRS member "CN=DC2,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=mynet,DC=lan".

Deleting subtree under "CN=DC2,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=mynet,DC=lan".

Deleting subtree under "CN=DC2,OU=Domain Controllers,DC=mynet,DC=lan".
The attempt to remove the FRS settings on CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan failed because "Element not found.";
metadata cleanup is continuing.

"CN=DC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=mynet,DC=lan" removed from server "DC5"
metadata cleanup:




NTDSUTIL -  option 3 (single command)

Note: in fact, we have to enter three commands before entering the "remove selected server" command with the path to the server to remove.


PS C:\Users\ufc> ntdsutil
C:\Windows\system32\ntdsutil.exe: activate instance ntds
Active instance set to "ntds".
C:\Windows\system32\ntdsutil.exe: metadata cleanup
metadata cleanup: remove selected server cn=DC2,cn=servers,cn=default-first-site-name,cn=sites,cn=configuration,dc=mynet,dc=lan

Binding to localhost ...
Connected to localhost using credentials of locally logged on user.
Transferring / Seizing FSMO roles off the selected server.
Removing FRS metadata for the selected server.

[Same output as in previous examples - please see above]




GUI - delete object in Active Directory Users and Computers


Since Windows Server 2008, we can select the object representing the defunct domain controller in Active Directory Users and Computers. By default, it is located in the Domain Controllers organizational unit:


We right-click on the object, select "Delete" and confirm the operation.


We are prompted once more and must acknowledge that we do indeed want to delete the object:



Just to be certain, we are prompted one last time since this domain controller was also a global catalog:





Metadata Cleanup - final steps

All the methods outlined above remove most references to the defunct domain controller. However, it is recommended to verify the following locations, and in particular DNS:

Active Directory Users and Computers (the domain controllers OU)

I have found that all methods above remove the object (and logically the GUI method where we delete the object directly)


Active Directory Users and Computers (File Replication Service)

Verification in this location requires us to select the "View | Advanced Features" setting.

Once we have done so, we proceed to the following location:

Domain icon (root) | System | File Replication Service | Domain System Volume (SYSVOL share)

Note: I have never found any references to defunct domain controllers in this location.


Active Directory Sites and Services

Although NTDS settings are removed, there still is a reference to the domain controller itself in the following location. It can and should be deleted:




DNS

This is the location where a number of records referring to the failed domain controller remain. The screenshot below is only one example. Each folder should be checked and any references to the domain controller in question be deleted:





References:

Clean Up Server Metadata

A very comprehensive article by a Directory Services MVP




Wednesday, February 12, 2014

Windows Server 2012 - Active Directory - Manual Removal of a domain controller, Part 1: seize FSMO roles

If at all possible, we should aim to remove domain controllers using dcpromo or, most recently (with Windows Server 2012), "Remove Roles and Features".

Sometimes this is not possible: a domain controller suffers catastrophic hardware failure and a so-called "graceful" demotion is not possible.

In such a situation, we have two options. We could attempt to restore the entire domain controller from backup (if we have one... ) or we could simply accept the loss of the machine and create a brand new domain controller on other hardware.

With this second option, we must consider the roles that the defunct domain controller held and how to re-establish them (if and as necessary) on the remaining domain controllers.

Note: I use "roles" in the broadest possible sense - FSMO roles as well as DNS, Global Catalog, etc.. 

In this scenario, I will assume that the failed domain controller held all five FSMO roles. It was also a DNS server and a Global Catalog. 

If we only have two domain controllers, the remaining domain controller must not only seize the FSMO roles but also assume these tasks:

1. DNS server (no Active Directory without DNS)
2. Global Catalog
3. Time source

In "double domain controller" scenario, it is likely that the remaining domain controller is both a DNS server and a Global Catalog. It would have made sense for both domain controllers have these roles for redundancy.

On the other hand, since the remaning domain controller (in our scenario) is not the PDCe, it would not be the authoritative time source.

Note, however, that this would not be critical. As long as the time source is uniform throughout the domain, Active Directory will function, even if the "domain time" is not in sync with the external "atomic time clocks". I will not address that aspect in this blog post.

So even before I seize the FSMO roles, I'm going to double check points 1 and 2.

For DNS, we can see if the domain controller is also a DNS server in Server Manager. By default, DNS is installed with Active Directory Domain Services. Unless someone changed this default, we should see the DNS role present. If we want to verify that DNS is Active Directory Integrated, we can open the DNS console:



As for the Global Catalog, this command will display the servers that hold a copy:

dsquery server -forest -isgc

Otherwise, we can open Active Directory Sites and Services and look at this location:



So we are ready to go. We can adjust time service later. The seizing of some FSMO roles (or operations masters) should be performed rather soon. We can do without the Schema master, the Domain Naming Master and (especially in a single domain) the Infrastructure Master for quite some time. Weeks could go by without any ill effect. But the RID Master (that distributes RIDs necessary for the creation of domain objects) and the PDCe may need to be restored rather rapidly.

Operations master roles (FSMO)


Seizing the FSMO roles (operations masters)

So we first need to seize the FSMO roles. We can do this at the command line with the ntdsutil tool:

(Note: the commands to enter are in bold and underlined - the rest is the resulting output.)

PS C:\> ntdsutil
C:\Windows\system32\ntdsutil.exe: activate instance ntds
Active instance set to "ntds".
C:\Windows\system32\ntdsutil.exe: roles
fsmo maintenance: connections
server connections: connect to server DC5
Binding to DC5 ...
Connected to DC5 using credentials of locally logged on user.
server connections: quit
fsmo maintenance: seize schema master

Attempting safe transfer of schema FSMO before seizure.
ldap_modify_sW error 0x34(52 (Unavailable).
Ldap extended error message is 000020AF: SvcErr: DSID-032103C7, problem 5002 (UNAVAILABLE), data 1722
Win32 error returned is 0x20af (The requested FSMO operation failed. The current FSMO holder could not be contacted.)
Depending on the error code this may indicate a connection,
ldap, or role transfer error.
Transfer of schema FSMO failed, proceeding with seizure ...

Server "DC5" knows about 5 roles
Schema - CN=NTDS Settings,CN=DC5,CN=Servers,CN=Site1,CN=Sites,CN=Configuration,DC=mynet,DC=lan
Naming Master - CN=NTDS Settings,CN=DC2,CN=Servers,CN=Site1,CN=Sites,CN=Configuration,DC=mynet,DC=lan
PDC - CN=NTDS Settings,CN=DC2,CN=Servers,CN=Site1,CN=Sites,CN=Configuration,DC=mynet,DC=lan
RID - CN=NTDS Settings,CN=DC2,CN=Servers,CN=Site1,CN=Sites,CN=Configuration,DC=mynet,DC=lan
Infrastructure - CN=NTDS Settings,CN=DC2,CN=Servers,CN=Site1,CN=Sites,CN=Configuration,DC=mynet,DC=lan


Some remarks...

1. NTDSUTIL asks us to confirm the operation...

2. NTDSUTIL attempts  to transfer the role, which fails.

3. And then displays verbose output in which we can see the status of the FSMO roles.

In this case, we can see that the Schema Master is now DC5 (in bold red) but the other roles are still held by the now defunct DC2.

For concision and readability, I will not post this output for each command:

Note the syntax, incorrect and correct for the domain naming master:

fsmo maintenance: seize domain naming master
Error parsing Input - Invalid Syntax.
fsmo maintenance: seize naming master

We must omit the word "domain". Users familiar with Windows Server 2003 may remember that we had to include the word "domain". Since Windows 2008, his is no longer the case.

These are the commands for the remaning operations masters:

fsmo maintenance: seize rid master
fsmo maintenance: seize infrastructure master
fsmo maintenance: seize pdc

Note: we exit the fsmo maintenance mode like this:

fsmo maintenance: q
C:\Windows\system32\ntdsutil.exe: q

q = quit


***

In the following section of this two-part blog post, we will examine the different methods for "metadata cleanup".

Wednesday, February 5, 2014

Windows Server 2012 - Active Directory - Domain and Forest Functional Levels.

Domain and forest functional levels (abbreviated as DFL and FFL) determine what functionality is available in a domain (or forest) based on the version of domain controllers present.

What do we mean by "functionality"?

For example, the Active Directory Recycle Bin was introduced with Windows 2008 R2 (and later improved in Windows Server 2012). If we want to use the Recycle Bin, our FFL must be - at least - Windows 2008 R2. That means that all domain controllers in the forest must run Windows 2008 R2 (or above). Other features may be available at a lower functional level. If we want to rename the domain, for example, the Windows 2003 FFL would suffice.

We can raise the functional level in at least three places.


ADUC and ADDT

For the domain level, we can open Active Directory Users and Computers (ADUC), right-click on the icon representing the domain, and select "Raise Domain Functional Level"



For the forest, we go to Active Directory Domains and Trusts (ADDT), right-click on the icon and select "Raise Forest Functional Level".

Note: we can also raise the DFL in ADDT as well by right-clicking the domain icon.
 
We can safely click on these options because the level will not be raised at this step but only after we select the new functional level (and click "Raise"). This is what we see in ADDT:
 
 
 
In fact, we can select a number of different levels depending on the domain controllers that are present. Here, we can select up to Windows Server 2012:
 
 
 
 
Note: with Windows Server 2012 R2, there is now yet another possible level.
 
 
ADAC

The Active Directory Administrative Center is the second place we can raise functional levels. Once again, we right-click on the domain icon and select among the options in the resulting drop-down menu:
 





Powershell

The third place we can raise the DFL and FFL (and lower - in some cases) is at the Powershell command line. I'll present the cmdlets in a moment, but first want to take a look at the designations of the various levels. Since we are (in my test network) already at Windows 2003 DFL, we have these options remaining:

- Windows2008Domain - or "3"
- Windows2008R2Domain -  or "4"
- Windows2012Domain - or "5"

And more recently, with the release of Windows 2012 R2:

- Windows2012R2Domain - or "6"

So, if we want to raise the DFL, we can either type "Windows2008Domain" or just the number "3" (without quotes in both cases).



Here is the complete command:

Set-ADDomainMode mynet.lan -DomainMode Windows2008Domain

Reference:

Set-ADDomainMode


There is a similar command for the FFL, for example:

Set-ADForestMode -identity mynet.lan -ForestMode Windows2008Forest

Reference:

Set-ADForestMode

Note: the parameter -identity is positional, which means we can omit it if we prefer.



So that is how we can raise the DFL (or FFL) with Powershell cmdlets - at least in theory.

My first attempts with these cmdlets were not successful. Here are the results:

PS C:\> Get-ADDomain | Set-ADDomainMode -DomainMode Windows2008Domain

[snip - confirmation prompt was here]

Set-ADDomainMode : A referral was returned from the server at line:1 char:16
+ Get-ADDomain | Set-ADDomainMode -DomainMode Windows2008Domain
+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

+ CategoryInfo : ResourceUnavailable: (DC=mynet,DC=lan:ADDomain) [Set-ADDomainMode]

[snip]

However, if I attempt the same operation from the GUI, it is successful.

In fact, I learned that this command apparently requires the designation of the PDC emulator.

Indeed, if I use the syntax below (where DC2 is the PDCe)....

PS C:\> Set-ADDomainMode mynet.lan -DomainMode Windows2008R2Domain -Server DC2

The operation succeeds.

Reference:

Set-ADDomainMode - ResourceUnavailable

And thanks once again to Christopher.

 
 
Reverting to a previous DFL or FFL


With the Set-ADDomainMode and Set-ADForestMode cmdlets, we can also revert the DFL and FFL to a previous level - but only under certain conditions.

First, we can go from Windows 2012 to Windows 2008 R2 or from either of those levels to Windows 2008.

We cannot revert to Windows 2003 (or 2003 R2).

There is one more extremely important condition:

The Recycle Bin must not have been enabled.

If it is, we are "out of luck": it is impossible to revert to a previous level in this case.


In the lines below, I'll revert my DFL from Windows 2008 R2 to Windows 2008.

Note: the operation is only possible at the command line. There (currently) is no equivalent option in the GUI.


We start at the Windows2008R2Domain DFL...

PS C:\> Get-ADDomain | fl name,domainmode

name                       : mynet
DomainMode          : Windows2008R2Domain


We revert to the Windows2008Domain DFL with this cmdlet:

PS C:\> Set-ADDomainMode mynet.lan -DomainMode Windows2008Domain -Server DC2

Confirm
Are you sure you want to perform this action? [snip] y



It may take a moment for the change to take effect. The first time I verified the change, the DFL was still at Windows2008R2:

PS C:\> Get-ADDomain | fl name,domainmode

name                       : mynet
DomainMode         : Windows2008R2Domain


After a moment or two, the change takes effect:

PS C:\> Get-ADDomain | fl name,domainmode

name                       : mynet
DomainMode          : Windows2008Domain


Note: concerning Powershell, we can also display the domain mode (for example) with the following syntax:

PS C:\> (Get-ADDomain).domainmode