Bonus: The Journey of the EMS and NT Directory

“Presenting this as Exchange leveraging the NT Directory might be polite, but it is definitely not accurate.” –Subscriber Don Hacherl, Dev Lead on Exchange Directory from 1991-96, then on Active Dir.

This is a subscriber only bonus post based on a subscriber comment. I’m excited to add this to the work as it shows the value of the community and the opportunity to enhance the work in real-time based on the conversation about the events we all experienced from different perspectives.

---

Don Hacherl (DonH) offered some great commentary and ground truth on the evolution of the EMS (Exchange) Directory and the relationship between it and Windows NT. I thought I’d use this has a chance to describe the challenges of navigating product roadmaps and schedules in the early 1990’s and how difficult that was at the “BillG level”. Was I being too polite? Almost certainly. Was I inaccurate? Yes. Let me explain why I was and why being accurate is a matter of perspective. I think this is a fascinating example of the chaos of product roadmaps at the conceptual and strategy level diverging from what was entirely obvious to those doing the work. In other words, this was classic 1990’s Systems, which I say without judgment and only because that is what it was like.

As discussed in the previous post, in mid-1993 there were three Windows code bases under development. First was Chicago, based on the MS-DOS and Windows 3.1 code, which would go on to become Windows 95, though was slated to ship in 1994. There was Windows NT 3.1 which had just shipped its first version in July 1993. The original plans to build a server product would be fulfilled with the release of Windows NT 3.5 that shipped as both Workstation and Server in September 1994. Third was Cairo, which was colloquially referred to as NT 4.0 as it was based on Windows NT but added a great deal of functionality.

Each of these projects had a different velocity and level of pragmatism when it came to schedule versus reality. Chicago was still early though given the history of Windows 3.1 and the experience of the team was making progress. The scope of the product continued to expand and in general while the project was progressing through milestones M1, M2, M3, etc. (as discussed previously), meeting the goals of competing against Macintosh and more was the high-order bit pushing the schedule beyond the planned mid-1994 date.

Windows NT was a machine. It pioneered the daily (and working) builds along with ongoing performance and quality measurement. The team did an outstanding job of avoiding feature creep, in some ways by laying low in addition to just executing well. They were on a mission. That said, the complexity of Microsoft shipping its first server product with Windows compatibility was a huge job.

Cairo, in contrast, had an ever-expanding charter and feature-set. Even though my friends on a recruiting trip could debate in good faith about Cairo shipping as NT 4.0, there was no way that was going to happen. Almost no one in 1993 thought that would be possible. That reality was not something that would show up on slides or plans. And there’s the rub.

From the ITG memo explaining Microsoft's proposed Domain model for Windows NT rollout from early 1993. Excuse the now unacceptable terminology.

Now back to the directory, or as I referred to it the NT Directory. In late 1992 and early 1993, Microsoft ITG (Information Technology Group) was working diligently with the Windows NT team to architect the new NT networking architecture once Windows NT shipped. This was another example of fantastic dogfood work—Microsoft ITG as the first customer of a new enterprise product from Microsoft. In early 1993, the NT team and ITG met with BillG to go over the architecture. I was at this meeting and saved the memo that was presented because it was so complicated—in fact the only reason I have it was because it is in the small collection of items I displayed on my relight/office window.

The memo described Microsoft's existing management infrastructure. Mail was managed on Xenix at the time. One of the biggest challenges with the NT Directory would be keeping track of permissions each user would have on the network. EMS had a slew of requirements beyond that, many of which appear in the Address Book for Exchange users, for example. Also note the team size required to manage existing work and hint at requiring more.

One of the main points in the memo, and one that derailed the meeting, was that ITG was managing 16,000 email accounts (another point also derailed the meeting which was that ITG had a team of 10 people managing those accounts and thought NT would require far more human effort). This was one of the first points in the memo. Almost immediately that kicked off an entire discussion about the reality that the EMS team (what would become Exchange email) already planned on building an email directory to use for EMS according to ITG. TomEv came to Microsoft with experience across all the directory gateways and infrastructure that the world of telephone companies had been pondering (crazy acronyms like X.500 and so on). It is obvious that EMS would need a list of all the email accounts. It was equally obvious that if everyone had an NT networking account that should also be their email account, and there should be one copy of that for the network.

For fun I just wanted to include this to capture the connectivity Microsoft deployed in 1993 and how just managing the NT Domain Controller was a big challenge for ITG.

It was almost literally at that moment that it became clear from Bill’s perspective that it would be crazy to have the NT product have a directory of email accounts for ITG to manage and for the future email product to also have a directory of actual email accounts. To be specific, the common language for the NT Directory was at the time a domain controller and with that there were primary and backup domain controllers (PDC, BDC). The domain was in effect the directory. It was sold to customers as a directory. Customers thought of it as a directory. It competed (poorly) as a directory.

The problem was when would all this come together and how.

Windows NT was not built to be an email directory. In fact, the NT team did not really want customers (or ITG or BillG) to think of the domain system as a directory in the sense it would compete in the market as a directory, but rather just a feature of NT. This was the kind of internal posturing teams routinely did to avoid being dragged into a strategic battle while trying to ship.

The NT team was hardcore about shipping and had no intention of taking on the requirements (whatever those might be) from the EMS team in order to build whatever EMS thought it needed. Making this conversation extra difficult was the fact that the EMS team was just getting started building the product, though if you would have told them in early 1993 that they would not ship their first version for three more years (March 1996) they would have balked. They needed a directory and needed one soon.

To make this even more difficult, since the NT team was in no position to deliver any features for EMS, they did what everyone was doing at the time—they told the team to work with Cairo and get what they needed to be added to the Cairo schedule. Rather officially, Cairo was building the future “Directory” which was an enhanced version of the “NT Directory” that was shipping with Windows NT 3.5 and being architected by ITG. Everyone was on board with pushing any work for a Directory to the Cairo team. Even the Cairo team seemed fine about that.

What ended up happening for a while was this game of schedule chicken. Except it really wasn’t schedule chicken because everyone was doing whatever it took to ship. The NT train just continued to build what they needed to keep the rollout of NT going for ITG. EMS architected what they needed, though they were a longer ways off from dogfood. And Cairo just kept adding more to their feature list pushing out shipping even more.

As EMS became more solidified and Cairo less solidified, it became abundantly clear there was no real plan for the enhanced directory that could be the one to be used by both NT and EMS. There simply was no Windows NT Directory beyond what was already shipping (designed to identify a finite number of users, computers, and printers and manage permissions across those that already had capacity limits that would be hit just by internal usage in ITG).

This had zero impact on all the meetings and discussions going on with BillG. The strategy from his perspective was already set and every time the topic of “Directory” came up the assumption was that it was the Windows NT architecture and that two things were true. ITG was not going to implement anything new or make any big changes to what they had to roll out whatever was needed for email. And whatever EMS was using was the Windows Directory.

The reality on the ground was rather different as Don commented.

Speaking as the dev lead for the Exchange Directory (1991-1996) and later on Active Directory (1996-2005), there's a lot wrong with this chapter. NT's approach to functional directory services in the early 90's was "wait for Cairo. they're building one", which meant that we in Exchange had to build our own directory service. When Cairo collapsed (late 1995) Exchange and NT struck a deal so that once Exchange 4.0 shipped (April 1996) one of my developers and I brought a copy of the Exchange Directory source code over to Windows, and we built Active Directory out of that. Exchange in no way "bet on" the NT Directory; we essentially built the replacement for it in order to get the features we needed. Ask me if you need details.

Don correctly points out that the solution to this problem rested with him (and the people he worked with). Instead of adding features to the Windows NT product, essentially a new Directory would get built by the EMS team (which Don was a member). The EMS team built the directory it wanted for email and then essentially slipstreamed that into the Windows NT project after the first version of EMS shipped (Exchange 4.0 was the first version, that shipped in 1996).

In his comment he makes that seem like it happened in the dark of night (“struck a deal”) without everyone knowing, but he did way more than that and deserves much more credit. The team took on the work of shipping code with Windows NT which as I personally experienced was a herculean task fraught with challenges. This is the kind of cross-team heroics that exemplified what it took to make things happen in the face of strategy that was not always connected to reality.

The EMS directory then became the new (superset) directory for Windows NT, christened Active Directory. All of this happened essentially transparently to any executives outside the immediate managers. In fact, as Don explained to me further, it literally happened without some key executives aware or approving. To be honest, this is how Systems operated. Through one lens this is a “do the right thing” culture. Through another it amounts to “insubordination”. In reality, it is just a level of chaos and lack of coherency in a plan. As mentioned in the previous post, a key element of plans in Systems was not which team was doing what when, but simply which team got something done first and everything after that sort of faded away. Whatever finished first would become the new foundation and new things would get built on top of that.

This is all consistent with my recollection and experience. Yet there was disagreement over what I wrote because of the semantics over the bet on Windows NT. Don is of course correct—there was literally no directory proper to bet on. At the same time, Don’s architecture for EMS used aspects of the NT Directory (as much as it was) to implement the EMS directory (in particular, EMS did not handle passwords and authentication, deferring to NT). This was both expeditious for EMS and advantageous, and also on strategy.

However, and I do stand by this, from a strategy perspective the bet was that EMS would use the Windows directory and not ship a separate one. The fact that Don and team ended up writing a new Windows NT directory and that directory got its start from EMS, while exactly correct, does not change the strategic outcome or the execution of the strategy going back to that early 1993 meeting that set the wheels in motion. It took until Exchange 2000/Windows 2000 for the full unification to be complete, though the 4 years leading up to that were definitely sold to the market as a unified strategy (as was typical).

The fact that all of this was done without anyone articulating it to BillG or announcing it as some formal plan is kind of entertaining. It is a great example of the necessary work happening in spite of any management or oversight, I suppose. I definitely did not intend to leave out the ordering or specifics of Don’s contributions, who by any account was instrumental in the creation of both Exchange and Active Directory which together proved to be the lynchpin in Microsoft’s success in the enterprise (as I wrote). Two of Microsoft’s most important products owed a great deal to a sort of organizational gymnastics and the team navigated three major products to simply ship their part.

What a story!