Premkumar Yogeswaran's Blog

Active Directory | PowerShell | DNS | DHCP | Exchange Server | VM Ware

Archive for November, 2013

Active Directory – ADUC and LDAP Attributes refrence

Posted by Premkumar Yogeswaran on November 13, 2013


ADUC Field LDAP Attribute Name ADUC Tab
Account expires accountExpires Account
Account is locked out lockoutTime Account
Logon Hours logonHours Account
User logon name (pre-Windows 2000) sAMAccountName Account
Account options userAccountControl Account
User logon name userPrincipalName Account
Log On To userWorkstations Account
Country/region c Address
Country/region co Address
Country/region countryCode Address
City l Address
Zip/Postal Code postalCode Address
P.O. Box postOfficeBox Address
State/province st Address
Street streetAddress Address
Description description General
Display name displayName General
First name givenName General
Initials initials General
E-mail mail General
Phone Number (Others) otherTelephone General
Office physicalDeliveryOfficeName General
Last name sn General
Telephone number telephoneNumber General
Web Page Address (Others) url General
Web page wWWHomePage General
Member of memberOf Member Of
Primary group primaryGroupID Member Of
Canonical name of object canonicalName Object
Object class objectClass Object
Update Sequence Numbes (USNs)/Current uSNChanged Object
Update Sequence Numbes (USNs)/Original USN uSNCreated Object
Modified whenChanged Object
Created whenCreated Object
Company company Organization
Department department Organization
Direct reports directReports Organization
Manager manager Organization
Job Title title Organization
Home folder: Local path homeDirectory Profile
Home folder: Connect homeDrive Profile
User profile:Profile path profilePath Profile
User Profile: Logon script scriptPath Profile

Posted in Active Directory | Leave a Comment »

How to find and remove lingering objects in Active Directory

Posted by Premkumar Yogeswaran on November 12, 2013


Some of the biggest annoyances for any Active Directory administrator are odd little things called lingering objects. These have existed since Windows Server 2000 and will probably never go away completely, although Microsoft has worked to give us some great tools to get rid of them and protect our domain controllers.

While there are already some good articles out there describing lingering objects, I’d like to put my own spin on the issue based on experiences I’ve had with them. I still find many Active Directory admins who either don’t understand what lingering objects are or don’t know what to do about them.

Put simply, a lingering object is any Active Directory object that has been deleted, but gets reanimated when a DC has not replicated the change during the domain’s tombstone lifetime period.

In other words, when an Active Directory object is deleted, it still exists in the AD as a tombstone. This form of the object contains only the mandatory attributes and is moved into the Deleted Objects container. The contents of the Deleted Objects container can be seen using the LDP.exe tool from the Windows Server 2003 Support Tools. Once the object is tombstoned, it will remain in this condition until the tombstone lifetime period expires (which is 60 days by default). At that point, the garbage collection process will purge it from the Active Directory.

Now suppose you have a Global Catalog server in a remote office in Brazil that has not been available on the network for the 60-day tombstone lifetime period. This could be due to maintenance, a network outage, a hardware failures, etc. that prevents the Global Catalog from replicating with the other DCs.

So let’s say you have a multiple domain forest and 100 users were deleted from the United Kingdom domain while the Brazil DC was off the network. Finally, the Brazil Global Catalog comes back online and starts replicating, but since it did not replicate the deletion of those 100 user objects which have now been purged from Active Directory, it thinks that those objects need to be replicated to its partners. So now the partners replicate the objects and those 100 accounts are alive again – sort of. Since the Brazil Global Catalog contains a read-only copy of the United Kingdom domain, it replicates read-only copies of those objects.

In this condition you will see all sorts of anomalies. You may have deleted an account called RBrown several months ago and now another person joins the company with a similar name. You try to create the RBrown account and will get an error saying it already exists. You may also see inconsistencies in the Active Directory such as two copies of an object (a lingering version and a recreated version), or you may see different objects in a user interface depending on which Global Catalog/domain controller you query. You could even get conflicting objects and find that email has failed due to inconsistencies in the Global Address List (GAL).

Preventing lingering objects

Of course, it’s most desirable to prevent lingering objects from being created in the first place. There is a registry key called StrictReplicationConsistency — which we’ll refer to asStrict Mode — that will protect a DC from lingering objects:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\NTDS\Parameters
ValueName = Strict Replication Consistency
Data Type = Reg_DWORD
Value Data = 1 = Strict 0=Loose

If this value is set to 1, it will prevent a partner from replicating lingering objects to the DC it is defined on. Thus, if every domain controller has Strict Mode enabled, they are protected from lingering objects being propagated to them. If the value is set to 0, however, it is said to be in Loose Mode, and will allow the lingering objects to be propagated.

Now in Windows 2000 Server, the default value for StrictReplicationConsistency is loose consistency. This is important to note because if you have a domain that was upgraded to Windows Server 2003 from Windows 2000 and this key remained in the default Loose Mode, the domain will remain in loose mode. On the other hand, if you install a clean Windows Server 2003 domain, without upgrading from Windows 2000 Server, it will be in Strict Mode by default.

I’ve worked with a few organizations that suffered lingering objects because they had not taken the time to check this registry key. Again, you should always define the StrictReplicationConsistency key =1 in normal operations. It should only be set to 0 during removal of lingering objects.

Also, when Strict Mode is enabled on say DC1, and DC2 attempts to replicate an object that has been deleted on DC1, replication will be disabled between DC1 and DC2. Not just replication of the object either — all replication between the two DCs.

Determining the existence of lingering objects in the domain

Various events will either indicate the existence of Active Directory lingering objects, or will warn that they may exist. There are several events that might be logged in the Directory Service event.

Event ID 1864

This event will indicate if there are lingering objects. Note that it contains a count of how many DCs have not replicated in a day, week, month, two months, or the tombstone lifetime. The last entry is important. Unfortunately, the event will not tell us the name of the domain controller that hasn’t replicated in the tombstone lifetime.

Source NTDS Replication
Event ID 1864
User NT AUTHORITY\ANONYMOUS LOGON
This is the replication status for the following directory partition on the local domain controller.
Directory partition:
DC=DomainDnsZones,DC=corp,DC=com

The local domain controller has not recently received replication information from a number of domain controllers. The count of domain controllers is shown, divided into the following intervals.
More than 24 hours:
2
More than a week:
2
More than one month:
1
More than two months:
1
More than a tombstone lifetime:
1
Tombstone lifetime (days):
60

Event 2042 (Error) — Source: NTDS Replication

This identifies that strict replication is enabled, the "source DC" has not replicated in tombstone lifetime days and is attempting to replicate, thus replication has been disabled from the source. The event provides the GUID of the source in the format of the CName (alias) DNS record: 982a942e-40e4-4e3c-8609-bae0cfd2affb._msdcs.corp.net. The friendly name of the domain controller can easily be found by looking at the Alias records in the _msdcs zone in the DNS snap-in.

Event ID 1388 (Error) – Source: NTDS Replication

Description: Another domain controller (DC) has attempted to replicate into this DC an object which is not present in the local Active Directory database. The object may have been deleted and already garbage collected (a tombstone lifetime or more has past since the object was deleted) on this DC.

Event 1988 (Error) — Source: NTDS Replication

Description: Active Directory Replication encountered the existence of objects in the following partition that have been deleted from the local domain controllers (DCs) Active Directory database. This event is being logged because the source DC contains a lingering object which does not exist on the local DCs Active Directory database.

Source DC (Transport-specific network address):
4a8717eb-8e58-456c-995a-c92e4add7e8e._msdcs.Corp.com

Since these are logged individually on each domain controller, you can use a tool likeMicrosoft EventComb, which is part of the Account Lockout tools download. Events 1864 and 1862 indicate the existence of lingering objects.

If you run the command Repadmin /showrepl when replication has been disabled due to strict consistency, you will see verbiage such as:

==== INBOUND NEIGHBORS ======================================

DC=Wtec,DC=adapps,DC=hp,DC=com
Bracknell\GSE-EXCH3 via RPC
DC object GUID: 00b71e7b-46e3-4b2e-8eef-c05f08d2ab82

******* 753 CONSECUTIVE FAILURES since 2008-12-15 11:52:10
Last error: 8614 (0x21a6):
The Active Directory cannot replicate with this server because the time since the last replication with this server has exceeded the tombstone lifetime.

From the error here, it is pretty obvious that the domain controller is being protected from an out-of-date DC. As part of regular Active Directory health maintenance, it’s a good idea to run the following command manually or by script:

Repadmin/replsum /bysrc /bydest /sort:delta

Source DC largest delta fails/total %% error
WTEC-DC2 >60 days 105 /105 100 (1722)The RPC server…
WTEC-DC1 41m:26s 0/20 0
GSE-EXCH3 08m:59s 0/6 0
WTEC-DC6 08m:34s 0/6 0

In this output, we can see how long it has been since each of the other DCs have replicated with the one on which the command has been run. Of course the red flag is WTEC-DC2 that tells us it hasn’t replicated to our target DC for more than 60 days (tombstone lifetime).

The action that should be taken in this instance is to NOT fix it. Luckily, it is not replicating or we would have the danger of lingering objects coming to our source DC if strict consistency is not enabled. The resolution for WTEC-DC1 is to remove it from the network, manually demote it, clean up the server object in Active Directory, wait for replication and re-promote it.

Removing Active Directory lingering objects

If you have found the events and errors noted above, the lingering objects need to be cleaned up. You can refer to the Microsoft articles in the box to the right for more details, but here is the quick version.

The first step is to set the StrictReplicationConsistency registry value =1 (Strict), if it isn’t already set. Be sure to check it – don’t assume you know what it is. Use the Repadmin command to set this value on all DCs:

repadmin /regkey DC_LIST +strict

Simply add the DCs to be affected in the DC_List argument.

In the Windows Server 2003 SP1 (and later) Support Tools, the Repadmin tool has a nice switch called\RemoveLingeringObjects:

/removelingeringobjects [/ADVISORY_MODE]

This is a general lingering object purge. The arguments are:

Dest_DC_List – list of DCs to operate on
Source DC GUID – the DSA GUID of a reliable DC (preferably the PDC)
NC – Naming context of the domain the lingering objects exist in.
/ADVISORY_MODE – identifies what will happen when you execute the command for real.

So a sample command would be:

C:\>Repadmin /removeLingeringObjects wtec-dc1 f5cc63b8-cdc1-4d43-8709-22b0e07b48d1 dc=wtec,dc=adapps,dc=hp,dc=com

RemoveLingeringObjects sucessfull on wtec-dc1.

After running the Repadmin command, check the event log for the events noted previously to ensure the lingering objects are removed.

Obviously for a single domain forest with a few DCs, it will be pretty easy to find the warning signs and run the Repadmin command to remove the lingering objects. In a large forest with multiple domains, however, it isn’t so easy. For instance, the lingering objects may only exist on some subset of DCs in the forest. This command would need to be run for every naming context in the forest.

Posted in Active Directory | 3 Comments »

OpsMgr by Example: The Active Directory 2008 Management Pack

Posted by Premkumar Yogeswaran on November 1, 2013


This blog entry is the next in a series of Operations Manager-related items that review the steps performed to install, configure, and tune management packs in real-world environments:

Installation

1) Download the Active Directory Management Pack (http://www.microsoft.com/downloads/details.aspx?FamilyId=008F58A6-DC67-4E59-95C6-D7C7C34A1447&displaylang=en). The Active Directory Management Pack Guide is included in the download and labeled “OM2007_MP_AD2008.doc.”

2) Read the Management Pack guide – cover to cover. This document spells out in detail some important pieces of information you will need to know.

3) Import the AD Management Pack (using either the Operations console or PowerShell).

4) Deploy the OpsMgr agent to all domain controllers (DCs). The agent must be deployed to all DCs. Agentless configurations will NOT work for the AD Management Pack.

5) Get a list of all domain controllers from the Operations console. In the Authoring space, navigate to Authoring -> Groups -> AD Domain Controller Group (Windows 2008 Server). Right-click on the group(s) and select View Group Members.

6) Enable Agent Proxy configuration on all Domain Controllers identified from the groups. This is in the Administration space, under Administration -> Device Management -> Agent Managed. Right-click each domain controller, select Properties, click the Security tab, and then check the box labeled “Allow this agent to act as a proxy and discover managed objects on other computers.” Perform this action for every domain controller, even if the DC is added after your initial configuration of OpsMgr.

7) Configure the Replication account in the Operations console, under Administration -> Security (full details for this are in the AD MP Guide). Do this for every domain controller, even if a DC is added after your initial OpsMgr configuration.

8) Validate the existence of the “OpsMgrLatencyMonitors” container. Within this container, create sub-folders for each DC, using the name of each domain controller. If the container does not exist, it is often due to insufficient permissions. (See information configuring the Replication account within the AD MP Guide for details.)

9) Open the Operations console. Go to the Monitoring node and navigate to Monitoring -> Microsoft Windows Active Directory -> Topology Views and validate functionality. (You may have to set the scope to the AD Domain Controllers Group to get these views to populate).

10) Check to make sure Active Directory shows up under Monitoring -> Distributed Applications as a distributed application that is in the Healthy, Warning or Critical state. If it is in the “Not Monitored” state, check for domain controllers that are not installed or are in a “gray” state.

11) Create a MicrosoftWindowsActiveDirectory_Overrides management pack to contain any overrides required for the MP (hey, if it’s not created now we’ll never remember to create it and we’ll end up using the default MP and that’s not good – seehttp://cameronfuller.spaces.live.com/blog/cns!A231E4EB0417CB76!1152.entry orSystem Center Configuration Manager 2007 Unleashed for details there).

Deploying the Active Directory 2008 Management Pack was relatively painless. After importing the management pack, there was no significant impact on processors seen on the domain controllers. The Active Directory Topology Root appeared as a distributed application and showed a health state of green. The Active Directory diagram view also worked as expected.

Tuning/Alerts to Look For

We encountered and resolved the following alerts while tuning the Active Directory management pack.

Alert: The AD Last Bind latency is above the configured threshold.

Issue: One domain controller had consistently high AD Last Bind Latency. Logon to the system showed it as extremely unresponsive.

From product knowledge, we used the suggested tasks to validate that the bind was not going slowly and no high CPU processes were identified on the system. The view available in product knowledge pointed to a large spike in the time required for the LDAP query (checking the Active Directory Last Bind counter). The spike occurred while there was a very heavy processor utilization occurring on one of the domain controllers. This monitor checks every 5 minutes. Alert auto-resolved itself after the LDAP query was responding in an acceptable timeframe.
Resolution: Attempts to debug the issue were inconclusive and extremely difficult due to the performance issue with the system. We rebooted the domain controller, it came back online, and the AD Last Bind Latency returned to normal values.

Alert:A problem has been detected with the trust relationship between two domains.

Issue: A server in a location (site 1) lost communication with domain controllers that existed in a second location (site 2). This critical alert did NOT auto-resolve. This was detected by the alert rule “A problem has been detected with the trust relationship between the two domains.” We verified that the Last Modified date occurred during the outage (add this column to the display by personalizing the view on the Active Alerts to include the field) and the Repeat Count was not incrementing.

Resolution: We used the Active Directory Domain Controller Server 2008 Computer Role Task of Enumerate Trusts to validate all trusts were working after site connectivity was re-established. We then logged into the domain controller reporting the error and used the Active Directory Domains and Trusts UI to validate each of the trusts. We closed the alert manually.

Alert:A problem with the inter-domain trusts has been detected.

Issue: A server in a location (site 1) lost communication with domain controllers that existed in a second location (site 2). This critical alert did NOT auto-resolve. This was detected by the AD Trust Monitoring monitor which runs every 5 minutes using the AD Monitor Trusts script. We verified that the Last Modified date occurred during the outage (add this column to the display by personalizing the view on the Active Alerts to include the field) and the Repeat Count was not incrementing.

Resolution: We used the Active Directory Domain Controller Server 2008 Computer Role Task of Enumerate Trusts to validate all trusts were working after site connectivity was re-established. We next logged into the domain controller reporting the error and used the Active Directory Domains and Trusts UI to validate each of the trusts. This alert should auto-resolve when the trust relationships are working, but that functionality does not appear to work. We manually closed the alert.

Alert:AD Op Master is inconsistent.

Issue: We tested using the Alert Monitor “Ad Replication Partner Op Master Consistency,” which runs every minute, to verify the incoming replication partners for the domain controller show the same operations masters. We also used the REPADMIN Replsum task in the Active Directory MP.

Resolution: The REPADMIN Replsum command validated that replication was functioning correctly (we had to override the “Support Tools Install Dir” on Windows 2008 to %windir%\system32 to make the task work correctly). The link between the domain controllers has been running close to fully saturated. The alert auto-resolved once the network utilization slowed down.

Alert:AD Client Side – Script Based Test Failed to Complete.

Issue: This alert is generated by the “AD Replication Partner Op Master Consistency” monitor. The system reporting the error was generating an error of event id 45 in the Operations Manager Log from the source of Health Service Script.

This event is occurring on an hourly basis (12:57, 1:58, and so on):

AD Replication Partner Op Master Consistency : The script ‘AD Replication Partner Op Master Consistency’ failed to execute the following LDAP query: ‘<LDAP://servername.contoso.com/CN=Configuration,DC=CONTOSO,DC=COM>;(&(objectClass=crossRefContainer)(fSMORoleOwner=*));fSMORoleOwner;Subtree’.

The error returned was ‘Table does not exist.’ (0x80040E37)

This alert is linked to “Could not determine the FSMO role holder.” alerts that are occurring.

Resolution: We believe this was related to a misconfiguration of the anti-virus settings on the domain controllers in the environment.

Alert:DC has failed to synchronize its naming context with replication partners.

Issue: A server in a location (site 1) lost communication with domain controllers that existed in a second location (site 2). The rule generating this alert is “DC has failed to synchronize naming context with its replication partner”.

Resolution: The alerts occurred when connectivity was lost between the sites. These alerts had a Repeat Count of 0. We used the REPADMIN Replsum command to validate that replication was functioning correctly (had to override the “Support Tools Install Dir” on Windows 2008 to %windir%\system32 to make the task work correctly). We closed the alerts manually.

Alert:Could not determine the FSMO role holder.

Issue: Each domain controller in the environment reported the error when trying to determine the Schema Op Master on the various domain controllers. The rule generating this was “Could not determine the FSMO role holder”.

Resolution: We used the NETDOM Query FSMO task (changing the Support Tools Install Dir to %windir%\system32) to validate the FSMO role holders on each domain controller.

Alert:DC has failed to synchronize its naming context with replication partners.

Issue: One of the domain controllers in the environment went to a grayed out status.

The server having the issues reported the “DC has failed to synchronize its naming context with replication partners” issue and “A problem has been detected with the trust relationship between two domains” and “AD Replication is occurring slowly” and “Script Based Test Failed to Complete” (for multiple AD related scripts).

Other domain controllers reported “Could not determine the FSMO role holder” and “AD Client Side – Script Based Test Failed to Complete”.

Events also occurred on the client system (21006 OpsMgr Connector, 20057 OpsMgr Connector, 21001 OpsMgr Connector).

Resolution: We installed the Telnet client feature to test connectivity to the management server. Telnet connectivity failed from this system but not from others. We then restarted the OpsMgr Health service but it had no effect on the gray status. After rebooting the system, the status went back to non-gray.

Alert: AD Client Side – Script Based Test Failed to Complete.

Issue: AD Replication Partner Op Master Consistency: The script ‘AD Replication Partner Op Master Consistency’ could not create object ‘McActiveDir.ActiveDirectory’. This is an unexpected error. The error returned was ‘ActiveX component can’t create object’ (0x1AD)

Resolution: In MOM 2005, this was resolved by changing the Action account. In OpsMgr 2007, this alert occurred in a different domain than the one with the OpsMgr RMS server. To resolve this, we created a Run As Account for the domain (DMZ) and assigned the Run As Account to the AD domain controllers in the DMZ domain.

Alert: Script Based Test Failed to Complete.

Issue: AD Lost And Found Object Count: The script ‘AD Lost And Found Object Count’ failed to create object ‘McActiveDir.ActiveDirectory’. This is an unexpected error. The error returned was ‘ActiveX component can’t create object’ (0x1AD)

Resolution: We configured the AD MP Account (Administration / Security / Run As Profiles) for each of the two servers in the domain that were reporting errors.

Alert: Script Based Test Failed to Complete.

Issue: AD Database and Log : The script ‘AD Database and Log’ failed to create object ‘McActiveDir.ActiveDirectory’. The error returned was ‘ActiveX component can’t create object’ (0x1AD).

Resolution: We configured the AD MP Account (Administration -> Security -> Run As Profiles) for each of the two servers in the domain that were reporting errors.

Alert: Performance Module could not find a performance counter.

Issue: In PerfDataSource, could not resolve counter DirectoryServices, KDC AS Requests, Module will be unloaded.

Resolution: We created a Run As Account and configured the AD MP Account (Administration -> Security -> Run As Profiles) for each of the two servers in the domain that were reporting errors.

Alert: Script Based Test Failed to Complete.

Issue: AD Database and Log : The script ‘AD Database and Log’ failed to create object ‘McActiveDir.ActiveDirectory’. The error returned was ‘ActiveX component can’t create object’ (0x1AD)

Resolution: We installed OOMADS from the OpsMgr 2007 SP 1 CD.

Alert: This domain controller has been promoted to PDC.

Issue: No issue, this was an informational message. The message was generated when the PDC emulator role was moved between domain controllers.

Resolution: No actions required, this message is provided for situations where the PDC emulator role was moved unexpectedly.

Alert: The Domain Changes report has data available.

Issue: No issue, this was an informational message. This was generated when the PDC emulator role was moved between domain controllers in the environment.

Resolution: No actions required, this message is provided for situations where the PDC emulator role was moved unexpectedly.

Alert: AD Domain Performance Health Degraded.

Issue: More than 60% of the DCs contained in this AD Domain report a Performance Health problem

Resolution: This alert indicates that there are alerts that are occurring in more than 60% of the domain controllers in a domain. This alert does not require an action for itself but does require analysis to determine what is causing the domain controllers to be in a degraded state.

Alert: AD Site Performance Health Degraded.

Issue: More than 60% of the DCs contained in this AD Site report a Performance Health problem

Resolution: This alert indicates that there are alerts that are occurring in more than 60% of the domain controllers in a site. This alert does not require an action for itself but does require analysis to determine what is causing the domain controllers to be in a degraded state.

Alert:Account Changes Report Available.

Issue: Informational alert, which can be accessed in the AD SAM Account Changes report (available on the right side under Active Directory Domain reports).

Resolution: No resolution required. We checked the AD SAM Account Changes report (available on the right-side under Active Directory Domain reports) to see the changes that were available.

During our testing, we had a period of time when we lost network connectivity to a site that had one of the domain controllers. The result was a flurry of alerts listed below:

Alerts:

Critical Alerts:

§ A problem with the inter-domain trusts has been detected

§ DNS 2008 Server External Addresses Resolution Alert

§ OleDB: Results Error

Warnings:

§ A problem has been detected with the trust relationship between two domains

§ AD Client Side – Script Based Test Failed to Complete (multiple)

§ Could not determine the FSMO role holder. (multiple)

§ DC has failed to synchronize its naming context with replication partners (multiple)

Issue: Loss of network connectivity between one site and another, both of which had domain controllers.

Resolution: Once network connectivity was re-established, we resolved all issues identified above.

UPDATE: 02/25/09

Alert: The Op Master Schema Master Last Bind latency is above the configured threshold.

Issue: A large number of alerts are generated at > 5 seconds for warning and > 15 seconds for error.

Resolution: Per http://technet.microsoft.com/en-us/library/cc749936.aspx the effective thresholds should be changed to warning at > 15 seconds and error at > 30 seconds. Created an override for all types of Active Directory Domain Controller Server 2008 Computer role to change Threshold Error Sec to 30 and Threshold Warning (sec) to 15 and stored it in the ActiveDirectory2008_Overrides management pack.

Alert: The Op Master Domain Naming Master Last Bind latency is above the configured threshold.

Issue: A large number of alerts are generated at > 5 seconds for warning and > 15 seconds for error.

Resolution: Per http://technet.microsoft.com/en-us/library/cc749936.aspx the effective thresholds should be changed to warning at > 15 seconds and error at > 30 seconds. Created an override for all types of Active Directory Domain Controller Server 2008 Computer role to change Threshold Error Sec to 30 and Threshold Warning (sec) to 15 and stored it in the ActiveDirectory2008_Overrides management pack.

Posted in Active Directory | Leave a Comment »