072. Notes on Tablet PC Innovation
072. Notes on Tablet PC Innovation
"Microsoft's Tablet PC [is] the first Dynabook-like computer good enough to criticize." –Alan Kay (Xerox PARC) in Newsweek, 4/29/01
Few products have captured as much attention as Microsoft’s Tablet PC (except perhaps Xbox, which coincidently launched the same year). The company’s history working to deliver the form factor goes back to the earliest days of Windows. BillG’s intense focus on handwriting recognition led to one of the first extensions to Windows, Windows for Pen Computing (1992), and a bitter lawsuit with then perceived leader, GO Corp. Subsequently, handwriting recognition was among the very first groups chartered in the new Microsoft Research organization. The small Windows-derived handheld devices, PocketPC, were pen-centric. Then finally in the early 2000s Microsoft began in earnest (meaning in Windows proper) the technology for pen and handwriting that made it into a special edition of Windows XP for Tablet PC and a new type of computer. This arc took place a decade before the iPad was released (purists do love to point out the Newton from Apple came and went in the 1990s). Famously, Steve Jobs tells the story of the iPhone as one that began with a tablet form factor, but that was abandoned in favor of the phone. With such a long story, one could easily write a book on just the evolution of the tablet and the Wikipedia history page represents the industry search for a paradigm-defining product. In this section, we’ll start with the launch of the modern Tablet PC from Microsoft and detail the much more difficult (in my view) challenges of building software for the form factor in the context of Windows.
Back to 071. Resolving NetDocs v. Office
I should have been prepared for what transpired following the unveiling of our Tablet PC. I was not.
The line to enter Bill Gates’s keynote at the November 2001 COMDEX snaked through the hotel and required metal detectors and body searches following the events of September 11, 2001. Themed Digital Decade 2001-2010, it was set to be a highlight of the week—discussing bringing digital innovations across industries and emphasizing increases in productivity, richness of business infrastructure, and significant changes in home entertainment.
Previously at Fall COMDEX 2000, Microsoft unveiled the Tablet PC, showing a prototype tablet created by Microsoft, highlighting BillG’s personal favorites of pen-based handwriting capabilities and online document reading. At the time printing was still the standard for sharing business information. The prototype was how we hoped to convince OEMs to build similar PCs.
A recurring theme for the Windows business was to use big tradeshows to demonstrate exciting new PCs across the PC ecosystem. JeffR joined BillG on stage to show off multiple innovative PCs running the new Windows XP Tablet OS, many of which were to be available later in 2002 with the tablet-capable version of Windows XP. As with all things Windows, the strength in bringing the product to market was emphasized by a broad array of hardware of different sizes, shapes, and price points. Building off the first party prototype demonstrated the previous year, the keynote brought the full strength of the PC ecosystem to this new form factor with support in new variant of Windows XP.
Alan Kay, a pioneer in visualizing and prototyping the concept of a tablet while at Xerox PARC, conceived of the DynaBook, perhaps the original tablet. The DynaBook was envisioned as a “personal and portable information manipulator" described in his amazing paper, A Personal Computer for Children of All Ages (1972). Of the launch of Microsoft’s Tablet PC in 2001, he told Newsweek’s Steven Levy, "Microsoft's Tablet PC [is] the first Dynabook-like computer good enough to criticize." In PARC-speak, that was a high form of praise, not a back-handed compliment.
The quest for a PC tablet was not new and perhaps dates back in our memories to Captain Kirk on the bridge of the starship Enterprise with his tablet and pen. In the 1980s and 90s the PC industry was buzzing with innovative large screen computers based on tablets and pens from the likes of IBM, GRiD, Momenta, NCR, Compaq, and Asian companies like NEC, Samsung, and Toshiba. Small screen computers arrived (and departed) as well. Palm had a breakthrough with its Pilot, while Apple failed with its Newton. Windows CE devices were blessed with a stylus as well.
What Microsoft envisioned to do differently was to build a new class of computer. The Tablet PC would be a full notebook sized computer, but better. It was a notebook PC in power with the convenience of an actual notebook and pen. It was also fully capable of running the latest Windows and Windows software. This was not only the core of the Tablet PC strategy but the core of everything Microsoft was doing with the “scalable Windows architecture”, or “one Windows, with many implementations”.
Microsoft’s new Tablet PC group, led by Alex Loeb (AlexLoeb), reported to JeffR’s division (rather than Windows) to streamline delivery of a killer productivity solution. As the original manager of the pen computing effort a decade or more earlier, JeffR had a longtime passion for pen input (Some personal trivia is that my first trade show booth duty was showing off C++ support for Windows 3.1 for Pen Computing 1.0 at Spring COMDEX 1992). Jeff was an early fan of PocketPC devices, never missing a chance to use the stylus to jot down notes or show off the latest financials in Pocket Excel.
While the connection to information work was clear, this organization structure was a tacit admission that the Tablet PC might require more end-to-end design and implementation than a typical new Windows PC. To build a complete experience required integrating product design and engineering efforts across Windows, Office, research groups where the ink and handwriting technology was being developed, as well as hardware engineering for digitizers and working with PC makers. Coordinating marketing and partnering with OEMs introduced another aspect of the end-to-end effort.
Technology for digital ink progressed significantly, primarily as a result of the increasing focus on screens and digitizers—the technology that picks up a signal of some form from a pen and converts it to a series of datapoints representing what is drawn on the screen. Flat panel displays for notebooks were progressing quickly. The Tablet PC was timed well to capitalize on these innovations as display and touch/ink sensor-equipped panels proved to be the limiting factor for building even a marginally acceptable experience.
From its earliest days, Microsoft Research was working hard to make handwriting recognition a reality. Handwriting recognition had long been one of the fundamental linguistic technologies (along with language translation and bi-directional speech) that were always 5 years away (since the mid-1950s). Our state-of-the-art approach was fascinating in hindsight. Improving digitizers were able to capture many more points (coordinates) as the pen moved across the screen, even when moved quickly. These points were used to connect dots, which could then be smoothed out to look like a continuous line of ink. Meanwhile, the starting point, ending point, direction, and other data were used to guess the letter being drawn. Recognition was obtained by comparing the series of captured points along with direction and speed of the pen to a library of pre-collected samples. This was done letter by letter, building up recognition based on pairs or triads of letters commonly occurring together. Then when a collection of letters was detected, something like spellcheck was used to recognize an entire word. It was rocket science, the kind of rocket science BillG loved because no one could possibly duplicate the effort to develop the technology. The researchers managed to attain some almost useful level of accuracy, perhaps 90 to 95 percent, enabling ink to be searched as though it were plain text or to be selected and pasted into Word as plain text. This physics-based approach was used for decades with slow progress, but state of the art in 2001.
Fast-forward 10 years and handwriting recognition was upended by an ingenious use of machine-learning image recognition advanced by research at AT&T Bell Labs in 1989 led by Yann LeCun. In a heartbeat 50-odd years of attempts turned into something that worked 99 percent (or more) of the time and was vastly easier to implement. The advances in machine learning represent the most fundamental improvement in computer science I’ve seen in my career.
A most exciting step with a new Windows release are new form factors. An even more (super) exciting hardware innovation was dubbed the “convertible,” which was a laptop that looked at first glance like a normal clamshell notebook, though smaller and thinner than most available at the time. Through a presto-change-o flip, the screen rotated and covered the keyboard to become a full-fledged tablet that resembled what Captain Kirk used on Star Trek (likely in size and weight too as the 8.5x11x0.8” device weighed 4lbs/1.8kg). In tablet converted mode, using a pen became the primary way of interacting, instead of a keyboard and mouse or integrated pointer. Techies have always loved and continue to love transformer PCs—presto-change-o is always a crowd pleaser.
The convertible form factor was particularly attractive to typical Office customers who craved mobility and could opt to use a keyboard and mouse when required for “full productivity”. This contrasts with a slate form factor, which (like an iPad today) is a screen-only computer, requiring a pen for all input and manipulation. Perhaps surprising, the idea of using a touch interface was not pondered primarily because the existing screen technology did not work with a finger, nor did handwriting for input. As we will see, even when rumors of Apple building a tablet surfaced (a pun!) the biggest question on BillG’s mind was how they acquired handwriting technology, or did they build their own solution which was bound to be inferior.
The dual personality modality was a significant source of tension across JeffR’s Information Worker division—does the design assume a convertible in which case Office was fine as it was, or do you design for a pen user interface? How productive did it need to be relative to how frequently the pen would be used. Every demo of a Tablet PC created questions (or concerns) about whether Office was participating in this future. There was no using Office without a keyboard and a mouse. Full stop. Microsoft’s DNA was such that a new OS required apps to push the OS to succeed, and the lack of Office was readily apparent. Worse, Office did not commit to “full support,” whatever that might imply.
Program managers across Tablet PC, Office, and Office applications spent months of meetings, prototypes, and discussions trying to understand each other. What did full support look like? In the best case, how should Excel work with a pen? Where and how would one input numbers and formulae? Did the Office user-interface work well when navigated with a pen instead of a mouse? We were in a circular platform-apps debate—the platform said it needed input from apps to finish the platform, while apps said it needed guidance on what the platform was enabling. The dividing line between platform and app is always tested when the organization represents that split (the same dynamic happens with front-end and back-end in web applications).
The platform believed it was enabling a particular scenario while the apps don’t value the scenario or had a thousand reasons why the scenario is way more complicated, and the platform should go back to the drawing board so to speak. In the case of supporting a pen as a replacement for keyboard and mouse, the list of what was impossible to do in a standard Windows app seemed impossibly long. There was a seemingly irrational insistence that every place one could type could also accept ink input, converting that to text. Being able to ink into the Excel formula bar was both awkward and a technical nightmare, with little benefit.
The only way to have pen support, I believed, was to build a pen-capable app from scratch. Unfortunately, the implication of that was that this new platform did not get Excel and there can’t be a new platform without Excel. Except this new platform was also just plain Windows and ran Excel perfectly, so long as there was a mouse and keyboard. Though requiring a convertible PC was out of the question because of the svelte attractiveness of the screen-only slate form factor.
Regretfully, the perception became that Office was stuck in the mud or resistant to change or influence. This was at least equally, if not more so, a problem of a platform searching for validation from Office without clarity for how apps should work. There seemed to be a great deal more work to do on the very basics of user-interaction before validating that with the most complex applications around. As would prove to be the case for a host of innovations around Windows, adding something on top of, or on the side of, Windows was not a sound method of creating either a new product or market, no matter how much we wanted the new market to be based on Windows, and more importantly to have compatibility with existing Windows applications, especially Office.
Beyond the technical integration was the ever-present go to market challenge. Was the new version of Windows for tablets a separate SKU and if so, was it more expensive, or were the tablet features simply extra features that would light up if the hardware supported them and likewise was there a special version of Office or did Office just light up with new features. We should not forget the ever-present tension over putting everything in the enterprise bundle versus monetizing new innovations.
The desire for optionality, both for Microsoft and OEMs, almost always pulled the product towards a strategy where features would be active if hardware supported them. That way third party developers could always assume the APIs were available on any Windows PC, making it safe to use without concern for system requirements. On the other hand, this also reduced the incentive to use those features (APIs) because doing so introduced complexity into a product where it had to test if features were available or not and behave appropriately. These challenges are why BillG always believed in the magic solution where developers used one and only one API and the conditional or variable implementation was hidden away in the API. Making this concrete it might mean a place in a product expecting typed input would magically transform into a place where a pen could be used for input. No one knew how to accomplish this in practice except for extremely limited scenarios that were more demos than reliable approaches. This was reflected in another classic tension point, which is how much of an application is simply a repackaging or use of Windows capabilities versus creating new capabilities in the application code? Recall earlier discussions over the role of text input and creation where Windows was woefully deficient which pushed Office to build significant capability in how text was entered. This would be repeated across every major component of the user experience—Office simply did not use much of what Windows had added over the years. The implication was that even if such magical transforming APIs were invented in Windows, Office would need to reinvent them for much more sophisticated and capable features that existed only in Office, including text input.
The disappointing truth would prove to be that innovation didn’t move from the platform to the applications, at least when it came to Office. Innovation was flowing from Office to Windows, while Office was no longer looking to Windows for direction. The dynamic of leading applications scaling to a level where they are both dependent on but separate from the platform was a sign of platform and app maturity. It would be a while before we’d learn how problematic that was for both Windows and Office. Right now it was a Windows problem.
Collectively, these lessons failed to contribute to evolving Windows for other markets including mobile phones and home entertainment. It would prove to be an amazing struggle for me in a few years when I worked on Windows.
In this case, Office, at the core, used a mouse and keyboard and did so in every single facet of design. Equally at the core, the pen was designed as an alternative to a mouse not as a reimagined interaction model. It was a pointer, like a mouse, and the fact that it also wrote on the screen was added to the side. In any existing Windows software, writing was done in a small window that popped up and allowed for a few short words to be written at a time. Those words were converted to typed text and inserted into the application. That’s how ink worked in all existing Windows products. Switching between ink for text input and typing was awkward, while ink remained inefficient.
The question was how to make the pen when used with apps like Excel work more like ink on paper. No one really knew because the first requirement was to shoehorn pen computing so as to require as few changes as possible to the existing code base. The theory was, and BillG strongly believed this to be the case, that the pen should just work. If this sounds familiar it is because “should just work” was a common refrain for Bill (This contrasts subtlety with Steve Jobs saying “it just works.”). The right architecture and abstractions should enable whole new implementations to appear above or below a given bit of code and like magic new capabilities appear. Programmers know this is great in theory but almost never works in practice. My rule for this type of capability is that unless the operating system ships having tested a particular type of replaceable layer then it doesn’t exist and if one attempts to use it all the problems will eventually surface. Outlook connecting to internet mail, Word converting to/from WordPerfect, Excel/Access connecting to Oracle database, Windows replacing the file system, and on and on there are too many examples to list.
There were so many challenges, we simply did not know where to start. Some of the problems were hardware related, such as while a typical screen might be 60 or more typed characters across, handwriting was more like eight to ten. All the on-screen places one might type characters, from simple numbers in a date to file names to formulas in Excel, all the way to full pages of text in Word, were too small for ink. Making them all larger was simply the first step in an entire redesign of the product.
While we were debating how to make the pen work, keyboarding was becoming a native skill replacing handwriting in schools. Kids were learning to type at the earliest ages, and handwriting was viewed as optional. Smartphones were not yet ubiquitous but sending SMS by triple-tap captured the imagination of teens around the world and replaced the proverbial handwritten note slipped under a desk. Maybe the opportunity for pen computing was generational. . .a boomer scenario?
Bigtime bosses were very excited by the prospect of pen input. In the EBC and with execs in general, taking an existing typed document and marking it up with ink annotations captured their imagination. They loved this. In fact, I loved it. I reviewed significant memos this way—printed them out and wrote on them in red like a teacher. I took notes at meetings on printed out PowerPoint decks. Yet even I was skeptical that such a scenario justified an entirely new computer. It did not even need a new version of Office. The easy solution could take an image of the document (a PDF!) and support ink on top, exactly what the Tablet PC team’s notetaking app, called Journal, did. It was a fantastic solution. There were still limitations, such as that the ink writing was huge compared to what was easy to read text. In fact, about all that could be done effectively compared to a real pen on real paper were gross annotations like arrows, circles, or crossing out with an occasional “bad idea” scrawled. The technology was not a replacement for the commonly used paper markup scenario. Not even close. It did not help that using a PDF-based solution was completely off the table as far as Bill was concerned. His feelings about PDF had not changed in the intervening 7 years since I first confronted him about the advantages of the format.
Even with those obvious limitations, BillG wanted those annotations and comments to use the semantically rich comment and annotation features that were part of Word, Excel, and PowerPoint—the track changes features used by lawyers. There were many problems with this including training people to use proofreading marks to change text versus typing using the existing keyboard features. The screens weren’t big enough, digitizers weren’t accurate enough, and handwriting recognition not good enough for these features to work.
The Journal app remained the closest we had to a killer app within the Tablet PC team—simply using ink as ink, with occasional translation to text. They made a cool feature where you could print a document to Journal and then mark up on top of it as though there was an acetate layer over a document, spreadsheet, or slide. It had all the limitations above, but the demo was perfect for executives.
Several Microsoft executives were early adopters of the new tablets. While they were effusive about the benefits, there was an aspect of them was difficult to escape. The amount of focus and attention it took to take notes with Journal was excessive. It was a head down, blinders on sort of focus. In meetings with people using the tablet maybe the notes were good, but it was a strain on engagement. As if to emphasize the limitations, executives that liked to print out their notes and file them soon discovered that a full screen of notes in Journal printed out cartoonishly large on standard paper due to the relatively low resolution of screens.
In early tests with the tablet across the company the feedback was uniformly positive, surprisingly so. As we kept digging into it, what was clear was that despite the heft of the early devices they were much lighter than the typical Microsoft issued laptop, which came in at 6lbs or more. It wasn’t necessarily the use of a tablet that people were positive about, but simply having a 4lb laptop. The early tablets were designed as premium PCs, which meant they were light, thin, and very expensive. It wasn’t merely the extra hardware for the pen that made them expensive, or the fancy hinges. The PCs used the latest in chips and displays too.
No matter what limitations, or frankly impossibilities, were raised, it always came back to Office being stubborn or resistant to features from other groups. Microsoft’s inherent bias was always to suggest that the new group with the product that wasn’t done yet was in the right and the existing group was resisting—in many ways this was a correct diagnosis and the right bias to avoid stifling innovation. Such a bias left little room for acknowledging that something new might not yet, or ever, work as intended.
It was clear, by collective body language and explicit direction, that Office was on the hook for something. Aside from adding ink to Word and Excel, what we needed was an Office version of Journal. The idea for a free Journal that came with Windows and a fancy version that was part of Office was exactly like the original strategy of having the mini word processor, Write, with Windows and Word in Office. Could we take a pen-centric approach like Journal and create a new category of productivity app, designed for ink but integrated with Office? Perhaps a Journal that also worked with Word and Excel? Or Word and Excel that worked just like Journal? Should Office build a product specifically for one kind of hardware that would likely sell in small units at first? Would such a product make defining SKUs challenging? We already had a half dozen SKUs and customers routinely expressed frustration with the complexity even though the bulk of our business transitioned to enterprise agreements where SKUs don’t matter much. So many questions about what seemed such a simple request. . .this type of routine program management was difficult and not particularly suited to executive-level strategy discussions.
There were many potential categories for Office to enter as we understood from JeffR’s opportunity map, but none seemed uniquely suited to tablet or convertible devices. Notetaking, however, was perennially on the minds of journalists and reviewers, implying anything we did would get a lot of personally motivated (though potentially nitpicking) coverage. Press encounters were also an opportunity to do research on information work. How did the reporter take notes? What tools were used to organize stories? How did they structure the writing process? By 2000, reporters were mostly using a Windows XP luggable with a big power brick in tow, while using Word to take notes during discussions. At most press events we often set up elaborate strips of outlets (and dangling network cables) attached to tables so the power-draining laptops of the era could plug in and file real-time stories. Invariably, reporters bemoaned the inadequacy of Word and Windows for the sort of juggling they did, working with notes, interviews, emails, documents, and later photos. Office didn’t offer a tool to work with these in one place.
In the old days, this category of software was sometimes called personal information managers, brainstorming tools, or outliners. It was never a big category and fragmented among the many small players and metaphors. It was almost exactly the kind of software we tended to avoid. There didn’t seem to be a winner-take-all strategy, nor did there seem to be a ton of revenue. On the other hand, if we could somehow develop an innovative product that caused people to rethink the category there was potentially a broad and horizontal product that could yield much-coveted organic growth. We discussed the idea that all the Office tools were about producing final and permanent documents, but we lacked a tool for ephemeral information. This seemed to be a potential anchor for a new product. I was under a good deal of pressure to provide innovation in a new category that might lead to new revenue.
Notetaking was definitely worth a try. What’s old was new again.
Trying to navigate the urgency to engage on a pen application and have something consistent with Office, I sent an email to Chris Pratley (ChrisPr), the leader of Word program management, to frame the need for a solution to notetaking. Chris was a Waterloo graduate hired to work on Word products who had previous experience living and working in Japan. He was instrumental in the transformation of Office to an extremely successful worldwide product, especially in Japan. Chris was also one of Microsoft’s earliest contributors to UNICODE standards. Aside from his East Asia experience, Chris was an exemplar of Office program management when it came to defining products and executing.
Our biggest concern was that we would embark on notetaking only to end up with a subset of Word, re-creating the problem of Outlook and Outlook Express, but with the anchor tenant and most used Office app. We wanted to innovate without developing a confusing subset of Word. Customers were already using Word for notetaking but adding a notetaking mode to Word to address missing features would have been a horrible mess and bloat for most customers.
Innovator’s Dilemma would say to create a new team to target new customers, but that almost guarantees a collision with Word (as we saw with Publisher). In our world, any time a new team was chartered they will invariably spend 90% of their initial time revisiting basic user interaction models and duplicating basic infrastructure of the main competing product—such a dynamic was rampant at the company in the early 2000s with every new team creating their own variant of menus and toolbars with a web-like twist under the guise of being easier to use and innovative.
The other option was to ask the Word team itself to develop the product as they would be most sensitive to colliding or overlapping. But would that impossibly constrain the team and create an odd product that spent too much energy not duplicating Word, even if it made sense? Again, so many questions . . .
Most products dedicated to taking notes proved to be subsets of Word, where using Word bullets, numbering, and outlining would suffice. Scenarios were changing, however, and notetaking was expanding to include collecting snippets from the web, links, photos, and even audio and video from new laptops incorporating cameras and microphones. To differentiate notetaking from Word, we needed a novel approach to the problem that went beyond typing (and inking) and basic text entry. Tailor-made for user research, notetaking was the kind of thing researchers loved to study. Suddenly, the hallways were filled with examples of notes, the flow of notes for different authors using different tools of Office, notebooks and ways people organized them, notes for students in classrooms, notes for home and work, and more.
Examples from the Office hallway in building 17 showing notes and notebooks collected from a field study. (Source: Personal)
The team was excited and quickly converged on prototypes and an approach that was ink-centric yet also text-centric and highly differentiated from Word—in other words they took on all the work to design a product that seamlessly worked with ink or with a mouse and keyboard, or both. Peter Engrav (PeterEn) joined to lead software development. PeterEn, a rare Bellevue, Washington, native, was one of the most thoughtful development leaders on the team—he was also a founding member of JonDe’s Office development team working on Escher graphics. Our offices were next to each other during the Office11 project and he and I often discussed the choices he was making into the late hours of the evening. The team picked a code name even though Office tended to shun code names, Scribbler.
ChrisPr took the team through a planning and vision process just for Scribbler. The team had sketches, prototypes, and a vision, Scribbler.doc. Perhaps a most impressive aspect of this process was how closely the concepts and details outlined in the original vision made their way to the final product.
From the outset, Scribbler intentionally did many things differently, as expected. We viewed it as a chance to pioneer some new approaches. While not as freewheeling as it might sound, the team would definitely say they would bump up against the culture of Office more than once—ironic given that PeterEn was a founding member of the Office development culture.
Scribbler built a native XML file format with next-level robustness, as one of those new approaches. Scribbler’s files could have multiple people edit a file at the same time and almost immediately see the changes made by others. Editing by multiple people seemed crazy for a personal notetaking product, but early on the idea of shared notes or group notes became a hallmark of the innovation. To enable shared editing, Scribbler would eventually be able to use SharePoint and, for one of the first times in a Microsoft product, data was stored on the internet (what eventually came to be the cloud). While we abandoned many of the MyOffice internet experiences, Scribbler eventually offered a key demonstration of native internet services.
Scribbler was able to focus on truly differentiated features because it was the first new product to make use of the MSO.DLL, or the Microsoft Office library of shared code. For the five previous years, Office engineered a platform of shared code for Word, Excel, PowerPoint, Outlook, and others, but no new product tried to use this code starting from a clean slate. Remarkably, PeterEn and the development team were up and running in short order by using this code—normally it would take months of work for a new application to take shape. With minimal effort, Scribbler inherited all the basic capabilities of an Office application, such as menus, toolbars, localization, Watson recovery, fancy graphics, enterprise deployment and management, plus engineering and test tools, and much more. This let the team focus on the core task of taking notes. It was exciting to see MSO in action and, most importantly, to show that in middle-age Office reached a significant level of operational maturity. InfoPath, discussed previously, also used MSO.DLL.
The novel experience of Scribbler and the key differentiator for users was the ability to write or type anything anywhere on a page. Just like on paper, one could simply tap the screen with the pen and start writing or click the mouse and type in that spot on the screen—delivering on the promise of the NetDocs universal canvas demo from Forum 2000. If one thinks about each of the apps, Word documents are bottomless, starting at the top and continuing down with a fixed page width. Excel is an endless two-dimensional grid. PowerPoint is a fixed-sized rectangle that allows anything to be put anywhere. Scribbler was bottomless, two-dimensional, and it also let users put anything anywhere simply by clicking and typing or tapping with a pen and writing. Ink and text were seamlessly and effortlessly mixed. Content like photos, videos, and audio could be added anywhere on a page. In a hint at the future of all applications, Scribbler also let users mostly ignore files and the tension-filled process of saving data and simply organize thoughts as one did with a paper notebook, with tabs, but with the advantage of software. Reorganizing and quickly searching across all the notes brought the power of software to note taking.
One area where Scribbler diverged from the way the Tablet group thought was integrating handwriting recognition. ChrisPr and team found in working with early adopters that recognizing ink as text was of little utility—people rarely wanted to convert their ink to text. In fact, people loved leaving ink as ink. Where recognized text was most valuable was in searching through the ink notes. Scribbler constantly recognized the ink converting it to text for search, but rarely did that ink get used in other places. Journal had taken the lead from BillG who was literally obsessed with converting ink to text and was generally against leaving ink as ink. It was a debate he and I had many times when looking to expand the use of ink in Word and Excel.
There were many seemingly small but novel touches in the product such as the ability to create a table by typing, hitting tab, typing, tab, typing, return, and poof, a table appeared. Scribbler also created checklists—list items that could be marked as to-do items, checked when completed, for shared or personal use. Scribbler supported photos, drawing shapes, and a whole range of outlining features.
The most eye-popping demo was when taking notes on a new Tablet PC, whether typing or using a pen, Scribbler could record the audio of a meeting and keep track of what notes were taken during the recording. One could easily hop from a note to the full recording of the meeting or vice versa across pages of notes. In another demonstration, easily drawing diagrams using a pen on a PC with the same ease as on paper was unimaginably cool.
Despite all that it offered, and as important as I thought it could be to the Office franchise as a growth opportunity, Scribbler would become the next innovation to get caught up in the challenges of bringing new revenue-generating capabilities to market with Office. Should Scribbler be a new category of software with dedicated marketing and sales yielding new organic revenue, or should it be added to the Office product enhancing the suite for every user? We faced this challenging decision so many times: Outlook, FrontPage, SharePoint, and now in Office11 with Scribbler and InfoPath.
Scribbler was designed to be a core part of the Office suite, a broadly horizontal product, not for a small or specific segment or narrow selling proposition as we saw with InfoPath, Access, Visio, Project, or Publisher. Yet, in an ironic twist, Scribbler did not make it into the suite and was destined to be a new category.
Normally this would be a great idea. A new category meant a real revenue opportunity. That opportunity was only realized with a dedicated sales effort. A dedicated sales effort for a broad, horizontal product would be nearly impossible because those same resources would be selling the Office suite and upselling to EAs or more expensive suites. Recall, however, that the notetaking category was known to be fragmented and small. Such a category has a low return on investment. Customers are hard to find and the efforts at reaching them don’t scale well. When you do find those customers, because there are so many alternatives, there is little pricing power.
Adding Scribbler to an existing suite would make the product available to everyone, which would be great, but would not alter the sales motion unless it became the key reason to upgrade or sign an EA. There were no such plans. In fact, the formal Office 2003 product guide mentions the new product exactly twice in 170 pages. One mention is a trademark footnote and the other in the SKU table, whereas Outlook is mentioned almost 200 times.
Scribbler was a product for everyone but marketed and sold to no one. The only suite Scribbler was added to was Office for Students and Teachers, used as a way of communicating that the low-priced suite was poorly suited for business enterprises because it included a notetaking product. In fairness to marketing, Scribbler had done some pioneering work with students at the University of Washington to support notetaking in college courses. Students loved Scribbler, which was also a testament to the product design for a version one product.
It would be reasonable to ask why I did not force the issue with marketing. Primarily, my view was that SKUs were entirely a marketing decision and accountability. With a dozen individual applications to mix and match, marketing spent the better part of a year picking and choosing combinations. The development team invested in features to make the production of different suites easy. I would be remiss if I didn’t mention that to consolidate marketing across the entire information worker business, JeffR took on marketing as a direct report as well. Packaging literally wasn’t my job or accountability.
I have exciting news to share: You can now read Hardcore Software by Steven Sinofsky in the new Substack app for iPhone.
With the app, you’ll have a dedicated Inbox for my Substack and any others you subscribe to. New posts will never get lost in your email filters, or stuck in spam. Longer posts will never cut-off by your email app. Comments and rich media will all work seamlessly. Overall, it’s a big upgrade to the reading experience.
Right up until the last minute, we had trouble coming up with the commercial name for the product. Scribbler was a good name the team loved, but we were unable to secure the rights. Eventually, the marketing team settled on OneNote, a perfectly good name but the one that left the product team somewhat bummed. Over the course of the product cycle, Scribbler really grew on the team and came to mean a great deal to them. Most (including the press) had a first reaction when hearing OneNote as it reflected the common expression “one note wonder” as in lacking range. It was a name where common usage reflected poorly, shades of DIM Outlook. In a small form of passive protest, some on the team mockingly referred to the official name as Onay-No-Tay. Fortunately, this name led to a tagline “One place for all your notes” and that made everyone happy. Like every naming exercise I experienced, eventually everyone warmed to it.
The product went on to be much loved by a core set of customers and received many super positive reviews. Ed Mendelson of PC Magazine, a longtime (very longtime) reviewer of productivity tools, called OneNote “breathtakingly well-designed.” Paul Thurrott, also a longtime Microsoft commentator and often a critic (especially of me), said, “In my mind, OneNote is one of the best applications Microsoft has released in years.” He and other reviews bemoaned the fact that OneNote was not included in the typical Office suites they bought.
The Tablet PC group was both happy and disappointed with OneNote. On its own OneNote showcased the new convertible Tablet PCs. Reviewers who loved OneNote were anxious to get their hands on one of the new devices, and certainly all the device makers were anxious to show off the capabilities using OneNote. At the same time, the existence of OneNote was viewed as some sort of cop-out relative to solving the big problem of using ink as a first-class input in Office apps. Perceptions aside, we invested disproportionate engineering effort to implement ink support in Word, Excel, and PowerPoint.
I viewed that criticism as harsh and a way of dodging the real question, which was whether ink was broadly suited to productivity or simply a way of trying to use software to emulate an old way of working. Was it the equivalent of controlling the first motor cars like a horse or was there something deeper about the benefits of ink? Was using a pen and ink something that seemed natural to a generation that had to learn to type later in life? Was handwriting a skeuomorphic answer to human-computer interaction that long outlived its need as a technology bridge? It might be an ironic twist that finally after decades of research and development handwriting, which was always the next big thing just five years out, improved to the point that it worked but the market that might have existed moved on.
We are 20 years through continuous investment with massive improvements in recognition technology and first-party hardware that supports pen input, yet the usage remains de minimis.
I have such a warm place in my heart for OneNote. The team did such a wonderful job. It was a breakthrough product, taking advantage of the latest operating system, while leveraging the latest hardware. Failing to capitalize on it with the field and business left me feeling that I let the team down. Microsoft couldn’t capitalize on a horizontal, pure-play productivity tool—the sales team and Office bundle wanted IT-centric, and enterprise-focused products like InfoPath. OneNote was a distraction.
The good news: There was no shortage of complicated features for IT professionals.
On to 073. **DO NOT FORWARD**
I am glad to see the emergence of the usefulness of MSO.dll in creating new apps. That is how it was envisioned when we started even though the immediate focus was on bringing better UI and feature consistency across the original apps. MSO was also useful anytime a product joined Office (Publisher, FrontPage, PhotoDraw). One very early use to build a completely new app was Hagaki Studio, a Japan specific program for making new year cards. That was in 1998 or 99 I think.
As I was reading about the kind of conflict with respect to integration Scribbler presented, all I could think of was OneNote, and then suddenly, ahah! In a way, OneNote became a kind of concept vehicle for cloud collaboration and that was nifty. The enthusiasm for OneNote was strong among some of my colleagues, although I was slow to make use of it. For me, it has the best browser article clipping mechanism ever, although quality has varied over the years.
I bought my first Tablet PC (XP version) from a little-known company whose name I forget. The fan died and killed the processor, which ran very hot with the fan. My second was a Toshiba Satellite which is still running about 8 feet from where I'm sitting, operating as a local web-development server. I never exploited the tablet capability although I had dreams about that as a developer. I did learn that Toshiba fell short with regard to updates and driver availability, ceasing at warranty expiration time. Still, the Satellite keeps on ticking though confined to the power brick and occasionally needing a reboot to get IIS running again. As a power typist, I never went for the tablet functionality. And these days, I just shrug and buy a Dell PC
when I finally have to upgrade. Well, the Surface Laptop Studio does catch my eye. Maybe when I finally have to work entirely with a laptop.