Change in URL.

I had some trouble to change the URL of my blog due to missunderstanding
between my domain hosting provider and their support employees. Finally, it
was done 2 days back and the new url to my blog is working now.
You can access the blog using http://www.ex-geek.com yet the old url will be
still working. If you have any trouble accessing any information please do
feel free to send me an email at milind.naphade@gmail.com

Active Directory from a physicist’s perspective.

By now you understand that Microsoft Exchange 200x family can not live without Active Directory. And that is true because you as a human being also can not stand without your backbone. The backbone should be strong enough so that it can sustain shocks and heavy work done by your body. Unlike backbone in our body Active Directory Services are considered as the considered as the backbone for the network infrastructure.

Am I talking like a doctor? Yes, it’s a physicist’s language. Everybody talks about AD topology, namespace and all other stuff but I have seen very few people giving some credits to active directory database. Well, I am not even interested in this topic but can’t neglect it too. Imagine a situation when your one and only domain controller crashed and will not boot. If your boss is kind and soft hearted he would probably allow you to rebuild everything but mine one would simply kick me off the job. To troubleshoot the DS performance issues and server reboot problems you also need to understand the directory database as well. Let’s have a quick look.

 

If you can recall, AD database file name is ntds.dit and usually put at location WindowsNTDS but it is a good idea to put this file on a separate volume when you are designing a domain controller which will be hell busy once it will be ready to serve requests. This is one of the recommendations by Microsoft to improve the active directory performance.

This database is based on Microsoft Jet Database Technology and uses B-Tree file structure with transactional logging to ensure the recoverability from drive failures. Something like Exchange and SQL. To understand transactional logging you can read at http://technet.microsoft.com/en-us/library/ms345419.aspx

This database is a set of files. In a typical active directory installation if you browse to the location C:WindowsNTDS you will find some files in there. Below is the description of this set of files. Again, these files are your active directory database. So follow me and we will browse through the folder .WindowsNTDS

 

 

  • Ntds.dit. This is the main AD database. NTDS stands for NT Directory Services. The DIT stands for Directory Information Tree. The Ntds.dit file on a particular domain controller contains all naming contexts hosted by that domain controller, including the Configuration and Schema naming contexts. A Global Catalog server stores the partial naming context replicas in the Ntds.dit right along with the full Domain naming context for its domain.
  • Edb.log. This is a transaction log. Any changes made to objects in Active Directory are first saved to a transaction log. During lulls in CPU activity, the database engine commits the transactions into the main Ntds.dit database. This ensures that the database can be recovered in the event of a system crash. Entries that have not been committed to Ntds.dit are kept in memory to improve performance. Transaction log files used by the ESE engine are always 10MB.
  • Edbxxxxx.log. These are auxiliary transaction logs used to store changes if the main Edb. log file gets full before it can be flushed to Ntds.dit. The xxxxx stands for a sequential number in hex. When the Edb. log file fills up, an Edbtemp. log file is opened. The original Edb. log file is renamed to Edb00001. log, and Edbtemp. log is renamed to Edb. log file, and the process starts over again. ESENT uses circular logging. Excess log files are deleted after they have been committed. You may see more than one Edbxxxxx. log file if a busy domain controller has many updates pending.
  • Edb.chk. This is a checkpoint file. It is used by the transaction logging system to mark the point at which updates are transferred from the log files to Ntds.dit. As transactions are committed, the checkpoint moves forward in the Edb. chk file. If the system terminates abnormally, the pointer tells the system how far along a given set of commits had progressed before the termination.
  • Res1.log and Res2.log. These are reserve log files. If the hard drive fills to capacity just as the system is attempting to create an Edbxxxxx. log file, the space reserved by the Res log files is used. The system then puts a dire warning on the screen prompting you to take action to free up disk space quickly before Active Directory gets corrupted. You should never let a volume containing Active Directory files get even close to being full. File fragmentation is a big performance thief, and fragmentation increases exponentially as free space diminishes. Also, you may run into problems as you run out of drive space with online database defragmentation (compaction). This can cause Active Directory to stop working if the indexes cannot be rebuilt.
  • Temp.edb. This is a scratch pad used to store information about in-progress transactions and to hold pages pulled out of Ntds.dit during compaction.
  • Schema.ini. This file is used to initialize the Ntds.dit during the initial promotion of a domain controller. It is not used after that has been accomplished.

Doesn’t that look like a full proof fail safe plan? If anything above goes corrupt and your DC, GC wont boot you can actually play around these files and get them up and running within minutes. And the tools listed below will be your force to fight against the crash and problems with your directory database.

 

Ntdsutil.exe

This utility is unlike a one man army soldier. So, if you identify that the problem is with directory database and nowhere else. What all you need to do is to reboot the box and start it in Directory Services restore mode. Once machine is booted launch this command line utility and you can fix the problems with database files. The table below explains what are the command line parameters and what they are used for.

 

Command

Description

Compact to %s
(where %s identifies an empty target directory)

Invokes Esentutl.exe to compact the existing data file and writes the compacted file to the specified directory. The directory can be remote, that is, mapped by means of the net use command or similar means. After compaction is complete, archive the old data file, and move the newly compacted file back to the original location of the data file.
ESENT supports online compaction, but this compaction only rearranges pages within the data file and does not release space back to the file system. (The directory service invokes online compaction regularly.)

Header

Writes the header of the Ntds.dit data file to the screen. This command can help support personnel analyze database problems.

Info

Analyzes and reports the free space for the disks that are installed in the system, reads the registry, and then reports the sizes of the data and log files. (The directory service maintains the registry, which identifies the location of the data files, log files, and directory service working directory.)

Integrity

Invokes Esentutl.exe to perform an integrity check on the data file, which can detect any kind of low-level database corruption. It reads every byte of your data file; thus it can take a long time to process large databases.
Note that you should always run Recover before performing an integrity check.

Move DB to %s
(where %s identifies a target directory)

Moves the Ntds.dit data file to the new directory specified by % s and updates the registry so that, upon system restart, the directory service uses the new location.

Move logs to %s
(where %s identifies a target directory)

Moves the directory service log files to the new directory specified by % s and updates the registry so that, upon system restart, the directory service uses the new location.

Recover

Invokes Esentutl.exe to perform a soft recovery of the database. Soft recovery scans the log files and ensures all committed transactions therein are also reflected in the data file. The Windows 2000 Backup program truncates the log files appropriately.
Logs are used to ensure committed transactions are not lost if your system fails or if you have unexpected power loss. In essence, transaction data is written first to a log file and then to the data file. When you restart after failure, you can rerun the log to reproduce the transactions that were committed but hadn’t made it to the data file.

Repair

Invokes Esentutl.exe to perform a low-level repair of the data file. Use the repair command only on the advice of qualified service personnel, as it can cause data loss. Furthermore, this can only repair what ESENT knows about. This means that its notion of repair might eliminate some data that is key to the safe operation of the directory service.

Set path backup %s
(where %s identifies a target directory)

Sets the disk-to-disk backup target to the directory specified by % s . The directory service can be configured to perform an online disk-to-disk backup at scheduled intervals.

Set path DB %s
(where %s identifies a target directory)

Updates the part of the registry that identifies the location and file name of the data file. Use this command only to rebuild a domain controller that has lost its data file and that is not being restored by means of normal restoration procedures.

Set path logs %s
(where %s identifies a target directory)

Updates the part of the registry that identifies the location of the log files. Use this command only if you are rebuilding a domain controller that has lost its log files and is not being restored by means of normal restoration procedures.

Set path working dir %s
(where %s identifies a target directory)

Sets the part of the registry that identifies the directory service’s working directory to the directory specified by % s .

 

Overview of Active Directory Schema

Every active directory administrator knows that the active directory schema is a partition which can not be rolled back once changed. We will talk about the active directory schema today as it is really an important and the last topic that every exchange administrator should know. I will cover up the remaining part related to DNS, forests and trust relations in active directory whenever they will be referred. So keep watching the new posts. You will enjoy the upcoming stuff I bet.

Active Directory Schema:

As a quick review, the object-oriented LDAP database that comprises Active Directory is built around a set of object classes and their associated attributes. Individual objects are instances of specific object classes. The schema defines the available object classes, their associated attributes, the data types and permitted ranges for those attributes, and the rules for arranging and managing objects within the Active Directory database.

Below is exactly what is held by Schema Partition:

Object Classes. Define the objects that can appear in Active Directory and their associated attributes.
Class Derivations. Define a method for building new object classes out of existing object classes.
Object Attributes. Define the available attributes. This includes extended attributes that govern actions that can be taken on object classes.
Structure Rules. Determine possible tree arrangements.
Syntax Rules. Determine the type of value an attribute is capable of storing.
Content Rules. Determine the attributes that can be associated with a given class.
Extensible schema. Additions can be made to the list of available classes and attributes.
Dynamic class assignments. Certain classes can be dynamically assigned to a specific object rather than an entire class of objects.
In a day to day administration life an administrator rarely get a chance to play around the schema. To understand the above concepts you can simply logon to http://msdn.microsoft.com/activedirectory . I have not covered up the above things in details to prevent the confusions so that it will make it easy to understand and off course remember too.

Before moving forward, let’s look at how Active Directory uniquely identifies objects. This information is crucial to understanding the more advanced Active Directory tools. Here is a brief attribute listing for a sample User object made using the LDIFDE utility. The unique identifiers are highlighted:

C:>ldifde -d cn=bgates, cn=users, dc=dotnet, dc=com -f con
Connecting to “DC01. Company.com”
Logging in as current user using SSPI
Exporting directory to file con
Searching for entries. . .
Writing out entries. dn: CN=bgates, CN=Users, DC=dotnet, DC=com
changetype: add
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
cn: bgates
distinguishedName: CN=bgates, CN=Users, DC=dotnet, DC=com
instanceType: 4
whenCreated: 20020812134034. 0Z
whenChanged: 20020812134034. 0Z
uSNCreated: 13772
uSNChanged: 13774
name: bgates
objectGUID:: 7swJ8PXwqkWu8N2Qv+jQ+Q==
userAccountControl: 512
badPwdCount: 0
codePage: 0
countryCode: 0
badPasswordTime: 0
lastLogoff: 0
lastLogon: 0
pwdLastSet: 126736332347481024
primaryGroupID: 513
objectSid:: AQUAAAAAAAUVAAAAdbl1VBUlr0cWwOoyVQQAAA==
accountExpires: 0
logonCount: 0
sAMAccountName: bgates
userPrincipalName: bgates@dotnet. com
sAMAccountType: 805306368
objectCategory: CN=Person, CN=Schema, CN=Configuration, DC=dotnet, DC=com

Distinguished Name:
The Distinguished Name (DN) attribute of an object defines the LDAP path all the way to the root of the namespace; therefore, the DN must be unique. If you move an object to a different container in Active Directory, in reality, you are simply changing the DN.

Globally Unique Identifier (GUID):
In Exchange 5.0, Microsoft used the DN as the unique database row identifier for objects in the directory service store. This unfortunate engineering decision created a configuration problem for Exchange. When an object is moved, its DN changes, but a unique row identifier in a database cannot ever change. For this reason, in Exchange 5. 5 and earlier, mailbox recipients cannot be moved but must be freshly created and then linked to a User account in the SAM.

To avoid that problem in Active Directory, Microsoft used a different unique row identifier called the Globally Unique Identifier, or GUID. A GUID is created using an algorithm that virtually guarantees its uniqueness within a system.

Using a GUID permits you to move objects at will between containers in Active Directory without changing the unique row numbers for the objects, thereby maintaining internal referential integrity in the database. Keep this behavior in mind, because you’ll see it at work when we discuss the role of the Infrastructure Master in keeping track of group members from other domains.

Security Identifier (SID):
Three classes of Active Director objects can be placed on the access control lists (ACLs) used to protect security objects. These object classes are User, Computer, and Group. Together, they are termed security principals.
A security principal is assigned a unique number called a Security Identifier, or SID. This is exactly the same SID used by NT to identify users, groups, and computers. A SID for a security principal is made up of the SID of the security principal’s domain and a unique suffix, called a Relative ID, or RID. The series of RIDs for security principals that can be created by an administrator start at decimal 1000. For example, the first User account created following the creation of a domain would be given RID 1000. The next object, call it a group, would be RID 1001, and so forth.
The combination of a domain SID and a RID form a unique number within a domain and within a forest. The pool of RIDs is maintained by a specially designated Windows Server 2003 domain controller called a RID Master.

SAM Account Name
In an NT domain, every object in the SAM must have a unique name. This is true for computers, users, and groups. A unique name guarantees that the object will have a unique NetBIOS presence in the network as well as a one-to-one correspondence between the logon name (in the case of users and computers) and the SID used to control resource access.
The same restriction is left in place in Windows 2000 and Windows Server 2003. Every user, computer, and group in a domain must have a unique name. This attribute is called SAMAccountName, although you might hear it called logon name or flat name. When you create a new security principal, regardless of the container where you place the object, it must have a unique flat name in the domain.

User Principal Name (UPN) and Service Principal Name (SPN)
Just as unique flat names identify security principals in NetBIOS, User Principal Names (UPNs) identify security principals within the hierarchical LDAP namespace in Active Directory. A UPN takes the form User@Company.com. Unique UPNs ensure that users can log on with their UPN rather than the classic domainusername construct. The Global Catalog is used to “crack” the UPN into its constituent parts.

To assure uniqueness, when a security principal is created, the system refers to the Global Catalog to verify that the UPN has not already been used. If a GC server is not available, the system displays an error message prompting the administrator to wait until a GC is available so that uniqueness can be verified.

In a Parent/Child trust configuration, the UPN suffix of the root dom

ain is assigned to every security principal. In a Tree Root trust configuration, you must manually assign a common UPN suffix. This is done using the Properties window of the domain tree in the AD Domains and Trusts console.

Object Identifier (OID) In addition to the attributes that assure uniqueness of a particular object, Active Directory needs a way to assure that objects of the same class all come from the same Schema object. This is done by assigning a unique Object Identifier, or Object Identifier (OID) to each object in the Schema naming context. ISO defines the structure and distribution of OIDs in ISO/IEC 8824:1990, “Information Technology—Open Systems Interconnection—Specification of Abstract Syntax Notation One (ASN. 1).”

ASN.1 provides a mechanism for standards bodies in various countries to enumerate standard data items so that they do not conflict with one other. ASN.1 governs more than just directory services classes and attributes. For example, OIDs are used extensively in SNMP to build hierarchies of Management Information Base (MIB) numbers. They are also assigned to many items associated with the Internet. If you’re interested in the list of organizations that assign OID numbers and their hierarchy, it is available at ftp.isi.edu/in-notes/iana/assignments/enterprise-numbers.
If you ever need to create a new attribute or object class in Active Directory, you must have a unique OID. There are a couple of ways to get one. The first is to apply to ANSI for your own numerical series. This costs a few thousand dollars and takes a while to process. The other is to use the OIDGEN utility from the Resource Kit. This will generate a Class and an Attribute OID out of Microsoft’s address space. The disadvantage to using OIDGEN is that the resultant number is very, very, very long. Here is an example:

C:>oidgen
Attribute Base OID:
1. 2. 840. 113556. 1. 4. 7000. 233. 180672. 443844. 62. 26102. 2020485. 1873967. 207938
Class Base OID:
1. 2. 840. 113556. 1. 5. 7000. 111. 180672. 443844. 62. 199519. 642990. 1996505. 1182366

Active Directory Name Space Structure (Contd..)

I haven’t had a chance to write up anything since I joined into the new project. So without writing anything else lets jump to the topic. This post will cover up all pending concepts of Active Directory.

Someone would certainly think that why would an exchange administrator need to understand active directory first? The answer to this question is very simple. Exchange Server 200x family is tightly integrated with Microsoft Active Directory Services. For 90% of the routine tasks, Exchange Server has to contact Active Directory and retrieve data out of it. A simple example is of an incoming email to some mailbox. When an email comes into your exchange server; it has to query Active Directory to understand where this email needs to be routed or handed over to.

The first one I would like to start with is Domain. A domain can be simply defined as a security boundary, a separate namespace structure. We will not jump into this in much deeper as this is just for understanding. An understanding of what domain is would be good enough.

Domain Naming Context:
Domain Naming Context is the placeholder in an active directory database (ntds.dit) which contains the information of all active directory objects such as users, their mailboxes, printers, computers and etc.
So whenever a new user object is created and had a mailbox created for it. Exchange Stores the information about the location of that user account, its mailbox location, and everything that appears in User’s properties dialog in Active Directory Users and Computers (ADUC) in Domain Naming Context. This partition also contains the information about domain objects and lot more.

Schema Naming Context:
The Schema naming context holds ClassSchema and AttributeSchema objects that represent the various classes and attributes in Active Directory. If this sounds like a circular definition, it’s meant to be. Unlike some directory services that load the schema in a separate file, the Active Directory schema is completely self-referential.
Every domain controller in a forest hosts a read-only copy of the Schema naming context. Only one domain controller, the Schema Role Master, can make changes to the schema.

Application Naming Context:
A new feature in Windows Server 2003 is the ability to create additional naming contexts that can be placed on specific domain controllers. Microsoft uses this feature to store DNS resource records for Active Directory Integrated zones.
Application Naming Context is currently capable of holding the information of DNS as an application only and this partition was introduced with DS changes in Windows Server 2003, so not available with Windows 2000 Server family products.

Configuration Naming Context:
The Configuration naming context holds information about parameters that control the services that support Active Directory. Every domain controller in a forest hosts a read/write copy of the Configuration naming context. It holds eight top-level containers.
1. Extended Rights
2. Lost and Found Config
3. Partitions
4. Physical Locations
5. Services
6. Sites
7. Well-Known Security Principals
Out of these 8 top level container Services is the one where Active Directory Integrated applications like Exchange store their configuration data. Exchange uses this partition to store the configurations like exchange organization name and all other settings.