Port 25: The Open Source Community at Microsoft : sam ramji, Linux

Sam Ramji is leaving Microsoft

billhilf — Thu, 10 Sep 2009 17:47:00 GMT

It's been a while since I made an appearance on Port25. I felt it was important to provide some thoughts to the Port25 community on Sam Ramji's impending departure from Microsoft.

After many years helping to carry the open source software banner for the company, Sam is leaving Microsoft at the end of this month. You may have also heard that he has accepted the position of interim President of the CodePlex Foundation as well as a leadership position at a startup in California. (I'll let Sam and his new company share more details there.)

Sam joined my team three years ago to drive open source technical strategy. I have eagerly supported him as he passionately articulated a vision that Microsoft could coexist - and even thrive - in a heterogeneous IT world.

The perspectives on OSS at Microsoft have evolved to the point where Microsoft's open source strategy is no longer just locked in a single ‘lab' on campus - now OSS is an important part of many product groups and strategies across the company. We have become increasingly clear on where we work with open source - development methodologies, projects, partners, products and communities - and where our products compete with commercial open source companies or platforms. Today, there are engineering and business leaders across the company, myself included, looking at how to drive interoperability for customers and as a lever for new growth.

And, because we recognize the importance of having that strong internal advocate for open source, we are actively seeking someone to fill Sam's shoes at Microsoft.

We will not waver in our commitment to open source.

To my friend Sam: Best of luck to you and your family as you move on to your next great adventure, and THANK YOU for all of your efforts and passion.

Microsoft Releases Device Driver Code to the Linux Community

Peter Galli — Mon, 20 Jul 2009 16:48:00 GMT

In what many may see as a surprising move, Microsoft today released 20,000 lines of device driver code to the Linux community under the popular General Public Licence v2.

The code includes three Linux device drivers, and has been submitted to the Linux kernel community for inclusion in the Linux tree.

The drivers will be available to both the Linux community and customers, and will enhance the performance of the Linux operating system when virtualized on Windows Server 2008 Hyper-V or Windows Server 2008 R2 Hyper-V.

In an article posted to Microsoft's PressPass site, Tom Hanrahan, director of Microsoft's Open Source Technology Center, notes that this is a significant milestone because it's the first time the company has released code directly to the Linux community. "Additionally significant is that we are releasing the code under the GPLv2 license, which is the Linux community's preferred license," he said.

In the same article, Sam Ramji, senior director of Platform Strategy at Microsoft, points out that Microsoft communities and open source communities are growing together, which is ultimately of benefit to customers. An example of this is the Linux community, which has built a platform used by many customers. "So our strategy is to enhance interoperability between the Windows platform and many open source technologies, which includes Linux, to provide the choices our customers are asking for," he said.

Ramji also alluded to the fact that people are often surprised when they hear how much open source community and development work is happening across Microsoft, which is largely due to the fact that these collaborations focus more on getting the work done and engaging with the various communities on a one-to-one basis and less about promoting them.

One example of how Microsoft participates with, and contributes to, open source is its relationship with the PHP Community. The company's involvement includes contributing to the PHP Engine, optimizing PHP 5.3 to perform strongly on Windows, and working to improve the performance of numerous PHP applications on Windows. Then there is the ongoing participation in various Apache Software Foundation projects, such as Hadoop, Stonehenge and QPID.

"In short, we're focused on building sustainable business strategies for open source at Microsoft ... we see open source playing into three key areas, one of which is the use of 'inbound' open source and the open source development model to make our software development processes more efficient."

"Good examples of this include what we did recently with jQuery in Visual Studio 2008, the implementation of OpenPegasus connectors and adaptors into System Center Operations Manager, and work that the Microsoft High Performance Computing team did with the Argonne National Lab (ANL) to source its MPICH2 implementation, which is a portable implementation of the Message Passing Interface (MPI) used in cluster computing and super computers," Ramji said.

We'll be posting a number of other articles on the release of the device driver code to the Linux community over the week, several of which will be penned by Hank Janssen from Microsoft's Open Source Technology Center, so look out for those.

Chatting to Students at Michigan State University

Peter Galli — Mon, 04 May 2009 23:08:00 GMT

As Microsoft continues to engage in active dialogues with a variety of communities, including academic institutions, Sam Ramji - Microsoft's Senior Director of Platform Strategy - talked to a group of Computer Science students at Michigan State University (MSU) on Friday May 1.

Given that Friday was the last day of school for students, the good attendance at the talk underscores the level of interest in hearing about Microsoft's Open Source strategy.

Ramji was invited to give an address as part of the Department of Computer Science and Engineering's Spring Colloquium Series, where he talked about the evolution of Microsoft's Open Source strategy and what an increasingly diversified technology landscape means for future software engineers.

Ramji told the students and academics that the company firmly believes that Microsoft, Open Source companies and developers, computer science students and others, all play an important role in the overall future of information technology.

He also talked about how Microsoft sees heterogeneity as a reality for the business world, and building technology and partnerships to embrace this reality is a part of a companywide commitment to greater openness and transparency.

Ramji also spotlighted some innovative Open Source projects the company is supporting at universities across the world, including PlugBlog, which is focused on Windows Live Writer and is being developed by students from Croatia.

The primary goal of the project is to help developers and companies that run blogging services integrate with Live Writer by providing them documentation, samples, screencasts and Visual Studio templates. The project is also focusing on developing a set of Live Writer plug-ins as well as documentation to enable developers to build plug-ins more easily.

He also talked about the KDE Education Apps, an Open Source project from undergraduate students from computer science, computer information systems and design courses at Sao Paulo State University in Brazil.

This project takes a small initial group of education applications and ports them to run on the Windows version of KDE.

The third project he talked about was OpenMP to MPI, a development framework for writing distributed memory applications, and works very well for developing High Performance Computing (HPC) applications for Windows HPC Server 2008.

Typically, though, developers have had to choose between writing for a single system SMP environment, which is what OpenMP was designed for, or writing for a distributed computing cluster environment , where MPI is the standard.

This project, which is taking place in India, translates OpenMP code to be deployable using MPI, so that application developers can develop using OpenMP but deploy in a cluster environment if they choose to.

Ramji also stressed how Microsoft works in partnership with the worldwide academic community and is committed to utilizing the power of Microsoft software and technologies to help inspire, encourage innovation and expand opportunities for students and educators in a heterogeneous technology world.

He also mentioned the Open Source Technology Center, Microsoft's open source technology research and development organization, which supports and promotes the creation of regional programs, such as the Interoperability Labs in the Philippines, Germany and Brazil.

But, while students were interested in hearing more about how to establish a Microsoft-sponsored Open Source project at MSU, what was clear from their questions was their concern about finding employment in the current tough economic environment.

During, and after his talk, Ramji talked to worried Computer Science students about what Microsoft looks for when recruiting new staff, how to craft a compelling resume, and what the benefits are for working at the company.

The Linux Foundation's Collaboration Summit

Peter Galli — Thu, 23 Apr 2009 14:00:00 GMT

A lot has been written by the press and blogosphere since the Linux Foundation's annual Collaboration Summit was held earlier this month, particularly about the panel that included Microsoft's Sam Ramji, Sun Microsystems' Ian Murdock, and Linux Foundation executive director Jim Zemlin.

The panel was entitled "Why Can't We All Just Get Along," which struck me as not only divisive, but also a little outdated given the level of collaboration that already takes place between proprietary and open source software vendors alike.

For example, Microsoft and Sun already have a long-standing working collaborative relationship; Microsoft also has a technical collaboration agreement with Novell, an agreement with Red Hat to test and validate our respective server operating systems running on one another's hypervisors, and a number of arrangements in place with other open source companies.

The panel discussed this in greater depth, looking at how collaboration, cooperation and competition exist: not just between proprietary and open software vendors, but also between Linux and open source ones.

This prompted panel moderator Zemlin to suggest that the three make an even greater effort come together and collaborate where it makes sense.

Interestingly, the Summit also spurred renewed discussion about whether there need to be more critics in the Linux community, with one blogger taking Zemlin to task for what he described as the "tall claims" he made at the Summit.

Ramji, the Senior Director of Platform Strategy at Microsoft, also used the panel to remind the Linux and open source communities of his offer for them to reach out to him and others within Microsoft and share their frustrations, problems and issues, so that they could be better educators and advocates on this front across the company.

Ramji also, again, stressed that Microsoft's customers want interoperability with open source software, including for PHP on Windows, but that making this happen sometimes took time.

Sun's Murdock seconded this, talking about internal inertia and how Sun also had had to deal with hearing from customers and developers that they wanted interoperability with technologies other than their own.

At Microsoft, there are cross-group, company-wide open source discussions and initiatives underway, with each group given the autonomy to decide for itself how this plays out with regard to their product set and business model.

While Port 25's mission is to be the voice of the open source community at Microsoft, it is far from the only voice on this topic. There have been blogs across the company on open and interoperability initiatives, from groups including security, Live and the Mac Business Unit, to name just a few.

It is also important to remember that Ramji and other executives like Bob Muglia, the president of Microsoft's Server & Tools business, have often said that open source is a journey that Microsoft is on and that much more needs to still be done. Many groups across the company are already responding to that call.

Joining Microsoft's Open Source Effort

Mark Stone — Thu, 12 Mar 2009 14:15:00 GMT

"Open source at Microsoft." My friends still find that phrase surprising. Yet for those of us who have worked so long on open source, if we really believe the principles we have espoused, shouldn't this be the expected outcome?

In 1994 I did my first Linux install. It was an early version of Slackware, running the 1.0.8 kernel. The term "open source" was still several years in the future. While I never really accepted the basic premises behind the ideology of the Free Software movement, the methodology we later called "open source" seemed obvious and sensible. Share knowledge, collaborate with others, expect and encourage others to evolve your ideas and share their innovations. In other domains, we call this the Scientific Method. Without the ability to openly share ideas, the process of scientific discovery would come to a grinding halt, and we'd be stuck in something like the medieval era of alchemy.

So I am pleased, but not surprised, at the progress open source has made in the last 15 years. And I'm happy to have had a front row view to a lot of it. That journey has taken me through O'Reilly, as the executive editor for their open source group, to Editor-in-Chief of the brief-lived Journal of Linux Technology, to a long stint at VA Linux Systems (now SourceForge) initially leading the web arm of their open source evangelism efforts and later running their developer relations program. Along the way I worked with Chris DiBona and others to get a couple of important books out on open source (Open Sources, and Open Sources 2.0).

SourceForge's developer relations program introduced me to a lot of technology companies eager to reach out to SourceForge's community of open source developers. About six years ago one of the companies we worked with was Microsoft. I worked with Stephen Walli (then at Microsoft) and others to help get Microsoft's first open source projects up on SourceForge. At the time this was a big deal. Few at Microsoft had much familiarity with open source licensing, and there was unease about opening up intellectual property in this way. And Micrsosoft had no experience with the long term benefits of "paying it forward" with this kind of investment in the open source community.

Today is a different story. Microsoft has its own open source project hosting site, Codeplex. Codeplex is growing steadily, and hosts about as many projects today as SourceForge did in 2002. Microsoft has OSI approved licenses that are used by many projects. And Microsoft has an entire group under Sam Ramji that works, among other things, to improve open source offerings on top of Windows.

Some view this turn of events at Microsoft with suspicion and hostility. I do not. Indeed, it would be hypocritical for any true open source believer to view Microsoft that way. If we genuinely believe that the collaborative practices inherent in open source are an important part of software development methodology, then we have to believe that (a) the world's largest creator of commercial software would benefit from contributing to open source, and (b) the world's largest creator of commercial software would be smart enough to recognize those benefits. So it should seem natural, not surprising, that Microsoft's evolution has turned in this direction.

Six months ago I was curious -- fascinated even -- watching Microsoft's recent open source efforts from the outside. For the last six months I've had the privilege of working first-hand with Sam's team, and getting an inside view of what open source is like at Microsoft. I've also had the distinct privilege of getting to know some of the developers and projects on Codeplex. Mine has been an unusual journey from SourceForge to Codeplex, but one I'm happy to have made. And I look forward to sharing some of my experiences with these open source projects here on Port25.

A Brief History of Open at Microsoft

Sam Ramji — Fri, 22 Feb 2008 17:06:00 GMT

Four years ago, we started the Linux Lab at Microsoft.
Two years ago, we established the Open Source Software Lab at Microsoft.
One year ago, we initiated the Linux Interoperability Lab at Microsoft.

Yesterday, we announced the broadest change to the way the Microsoft builds software and works with open source communities and developers.

By now you’ve probably read the announcement – “Microsoft Makes Strategic Changes in Technology and Business Practices to Expand Interoperability” and are wondering what it all means, and where it came from. In a nutshell, the documentation for the APIs, document formats, and protocols used in Windows Vista, the .NET Framework, Windows Server 2008, SQL Server 2008, Office 2007, Exchange Server 2007, and Office SharePoint Server 2007 will be made publically available. All developers will be able to access the documentation with no need to sign a license or pay any fee.

We are also announcing the launch of the Open Source Interoperability Initiative – a framework that will let us consistently support community development teams who build implementations of these specs with labs, technical support, plugfests, and joint testing and development.

To me, it’s a logical progression from the work and learning we’ve done with the Mozilla Foundation, the Eclipse Foundation, the Linux Foundation, the Apache Foundation, the Samba Project, MySQL, and PHP. We’ve learned how to make agreements with community projects – including those which lack a legal entity for formal agreements; how to deliver technical support; who to listen to; and how to prioritize our work. We have seen how positively developers and users respond to these kinds of collaborative efforts. This is reflected by the progression of our approach: the creation of the OSP (Open Specification Promise), the IECC (Interoperability Executive Customer Council), the IVA (Interoperability Vendor Alliance), the submission and approval of the Ms-RL and Ms-PL by the OSI, and the PFIF/Samba agreement and ongoing collaboration.

It’s also a major evolutionary step, and significant commitment for our engineering teams. Ray Ozzie says it best: “Customers need all their vendors, including and especially Microsoft, to deliver software and services that are flexible enough such that any developer can use their open interfaces and data to effectively integrate applications or to compose entirely new solutions. By increasing the openness of our products, we will provide developers additional opportunity to innovate and deliver value for customers.”

In order to meet these new and higher standards that we’re setting for ourselves, engineers will need to build public documentation of the new formats, protocols, and APIs they develop as they advance our products. For those of us who write (or have written) code, we realize that this is a significant additional phase to the development cycle: design the feature, specify the feature, implement and test it, then proof and test the documentation of the specification, build user documents and sign off on the whole thing.

Because we are a platform company first and foremost, it will be entirely worth the investment both due to the increased transparency to developers, and due to the expanded range of innovation that can be built on Microsoft technologies.

I think this is a great day not just for Microsoft, but for the software industry. And I thank the people who have helped us learn what it’s taken to get here – most notably Jeremy Allison, Matt Asay, Mike Schroepfer, Andi Gutmans, Jim Zemlin, Mike Milinkovich, Zack Urlocker, Marten Mickos, Andrew Tridgell, Miguel de Icaza and Stephen Walli. We will continue to look to their perspectives and advice as we continue down the open road.

Here are a few of the responses we’ve seen – and I’ll quote from the industry publications and blogs:

LWN.net: “The announcement is sweeping enough to make one check the calendar, but we are still a month and a week early for pranks. Microsoft is making available specifications for APIs and communication protocols for Exchange, Office, SQL Server, SharePoint, and others without requiring a license or royalty payments. They will indicate what patents they believe cover any of the protocols and "will license all of these patents on reasonable and non-discriminatory terms, at low royalty rates." There may be lurking dangers, but it appears to be a sincere effort at providing interoperability.”

Matt Asay (Alfresco/The Open Road): “As a Microsoft admirer, critic, and competitor, I can't help but applaud the depth and breadth of this move ... All in all, a huge day for Microsoft. Will there be gaps in Microsoft's efforts? Undoubtedly. For one thing, it hasn't really made much progress on its covenant not to sue commercial open-source providers, despite what Ina writes. But I'm impressed that it's even bothering to try.”

Andi Gutmans (PHP/Zend): “I believe Microsoft has finally understood that their closed nature has significantly hindered the growth of their eco-system. In many ways the threat of Linux has by many been interpreted as a threat of open-source (wrongly so in my opinion) …. Microsoft is now enabling the open-source community to grow its contributor base around such technologies and significantly improve the delivered quality. As most open-source developers and users live in heterogeneous environments this will benefit many.”

Jeremy Allison (via The Register): "It's definitely a positive step. Doesn't mean any change for us [Samba] as we already had all these docs, and the promise not to sue is only for 'non-commercial' open source, which is a bit meaningless. But that's the same thing we had really (they're listing the patents etc.). At least everyone now gets access to the same info, which I'm very happy about. As for the rest, the devil is in the details. If they can follow through with this, the world will be a better place.”

Zack Urlocker (MySQL/Open Sources): “… even if it was legislated, it's still good for the industry. And it’s good for Microsoft customers. And ultimately, it's probably good for Microsoft to be more open. If Microsoft wants to attract the next generation of developers and users, they should take the hint: Open works.”

We recognize that the communities’ judgments of the significance of this announcement will be entirely based on the actions that follow. The optimistic undertone that I’ve seen so far suggests that we can make real progress.

We will report back frequently on the progress and details of this work – especially on the Open Source Interoperability Initiative – here on Port 25. This announcement is the starting point of the next phase of Microsoft’s work with open source, and as Port 25 readers know, we are here for the full marathon.

The interoperability principles are posted here: http://www.microsoft.com/interop/principles/default.mspx

Accessibility on Windows and Linux

Sam Ramji — Fri, 09 Nov 2007 01:05:00 GMT

Back in Windows 95, Microsoft made a major contribution to accessibility to computers for people with vision and hearing impairments: MSAA, or Microsoft Active Accessibility. At that time it was an additional download, but from Windows 98 on this technology was built into the OS.

MSAA allows users to run screen readers, Braille devices, and other accessibility technologies that work across multiple desktop programs without requiring custom adapters for each program. Back in 2000, Rob Sinclair, now our Director of Accessibility, published the architecture for MSAA. It continues to be a core part of the OS in Windows Vista (detailed information here: http://msdn2.microsoft.com/en-us/library/ms788733.aspx )

Why am I talking about this? It’s background for some work we’ve been developing with Novell to improve cross-platform accessibility experiences, which we’ve announced today – work by Rob Sinclair and Norm Hodne at Microsoft and Michael Meeks at Novell, along with our legal teams.

Update: See Michael Meeks' blog on the work here: http://www.gnome.org/~michael/activity.html#2007-11-09

The User Interface Automation (UIA) specification is an advanced accessibility framework, and we are releasing this to the community, including an irrevocable pledge of patent rights for anyone implementing the specification. Novell will build a Linux implementation of the UIA and an adapter to make it work well with Linux accessibility projects. This will mean an advance in interoperable accessibility.

We’ve already gotten great responses from the National Federation for the Blind in the U.S. and from Janina Sajka, the head of the Open Accessibility Work Group at the Linux Foundation.

It is great to see the industry coming together with specs, words, and code to build a better world for people with disabilities.

Cheers,
Sam

Silverlight on Linux

Sam Ramji — Wed, 05 Sep 2007 10:45:00 GMT

For those of you who have met Miguel de Icaza, you know right away what I mean when I say that he is one of the most energetic people I’ve ever met. Clearly, the whole Moonlight team (whom I haven’t met) would also qualify for this appellation – in roughly 21 days between May and June, they collectively built an alpha implementation of Silverlight on Linux, based on many pieces of the Mono codebase.

After a great deal of work between the Moonlight and .NET teams, we’re ready to formally announce that we (Microsoft and Novell) will be bringing Silverlight to Linux, fully supported and including application and media codec compatibility.

The expansion of the existing work between Microsoft and Novell to include support for Silverlight on all Linux platforms is a major step in the journey of interoperability that we are on. We’ve heard clearly from the community that a full cross-platform web development solution is not only Windows and Macintosh, but must include Linux. I think this is a big deal. While we’ve licensed media codecs before, this represents a fully heterogeneous implementation of a strategic client technology.

Here’s to a better web and support for all users. Hopefully this will help breed further productive conversations about what developers and users need, and in someone else’s famous words, we can all “just get along.”

Cheers,

Sam

Linux and Windows Interoperability: On the Metal and On the Wire

MichaelF — Mon, 13 Aug 2007 17:10:00 GMT

I had the opportunity to present at both OSCON in Portland and at LinuxWorld in San Francisco in the last three weeks – both O’Reilly and IDG were gracious enough to grant me a session on the work that Microsoft is doing with Novell, XenSource, and others on Linux and Windows interoperability.

Overall our focus is on three critical technology areas for the next-generation datacenter: virtualization, systems management, and identity. Identity in particular spans enterprise datacenters and web user experiences, so it’s critical that everyone shares a strong commitment to cross-platform cooperation.

Here are the slides as I presented them, with some words about each to give context, but few enough to make this post readable overall. I skipped the intro slides about the Open Source Software Lab since most Port 25 readers know who we are and what we do.

Why interoperability?

The market for heterogeneous solutions is growing rapidly. One visible sign of this is virtualization, an “indicator technology,” which by its nature promotes heterogeneity. Virtualization has become one of the most important trends in the computing industry today. According to leading analysts, enterprise spending on virtualization will reach $15B worldwide by 2009, at which point more than 50% of all servers sold will include virtualization-enabled processors. Most of this investment will manifest itself on production servers running business critical workloads.

Given the ever improving x86 economics, companies are continuing to migrate off UNIX and specialty hardware down to Windows and Linux on commodity processors.

So, why now?

First, customers are insisting on support for interoperable, heterogeneous solutions. At Microsoft, we run a customer-led product business. One year ago, we established our Interoperability Executive Customer Council, a group of Global CIOs from 30 top global companies and governments – from Goldman Sachs to Aetna to NATO to the UN. On the Microsoft side, this council is run by Bob Muglia, the senior vice president of our server software and developer tools division. The purpose of this is to get consistent input on where customers need us to improve interoperability between our platforms and others – like Linux, Eclipse, and Java. They gave us clear direction: “we are picking both Windows and Linux for our datacenters, and will continue to do so. We need you to make them work better together.”

Second, MS and Novell have established a technical collaboration agreement that allows us to combine our engineering resources to address specific interoperability issues.

As part of this broader interoperability collaboration, Microsoft and Novell technical experts are architecting and testing cross-platform virtualization for Linux and Windows and developing the tools and infrastructure necessary to manage and secure these heterogeneous environments.

I am often asked, “Why is the agreement so long?” as well as “Why is the agreement so short?” The Novell-Microsoft TCA is 5 years mutual commitment. To put this in context, 5 years from now (2012) is two full releases of Windows Server and 20 Linux kernel updates (given the 2.5 month cycle we’ve seen for the last few years). This is an eternity in technology. What’s important to me is that it’s a multi-product commitment to building and improving interoperability between the flagship products of two major technology companies. This means we can build the practices to sustain great interoperable software over the long term as our industry and customer needs continue to evolve.

This talk covers two major components of the future of Linux and Windows interoperability: Virtualization and Web Services protocols.

On the Metal focuses on the virtualization interoperability work being done between Windows Server 2008 and Windows Server virtualization, and SUSE Linux Enterprise Server and Xen.

On the Wire covers the details and challenges of implementing standards specifications, such as WS-Federation and WS-Management; and how protocol interoperability will enable effective and secure virtualization deployment and management.

These are the key components required for the next-generation datacenter. We know the datacenters of today are mixtures of Windows, Linux, and Unix, x86, x64 and RISC architectures, and a range of storage and networking gear. Virtualization is required to enable server consolidation and dynamic IT; it must be cross-platform. Once applications from multiple platforms are running on a single server, they need to be managed – ideally from a single console. Finally, they must still meet the demands of security and auditability, so regardless of OS they must be accessible by the right users at the right levels of privilege. Hence, cross-platform virtualization demands cross-platform management and identity.

In non-virtualized environments, a single operating system is in direct control of the hardware. In a virtualized environment a Virtual Machine Monitor manages one or more guest operating systems that are in “virtual” control of the hardware, each independent of the other.

A hypervisor is a special implementation of a Virtual Machine Monitor. It is software that provides a level of abstraction between a system’s hardware and one or more operating systems running on the platform.

Virtualization optimizations enable better performance by taking advantage of “knowing” when an OS is a host running on HW or a guest running on a virtual machine.

Paravirtualization , as it applies to Xen and Linux, is an open API between a hypervisor and Linux and a set of optimizations that together, in keeping with the open source philosophy, encourage development of open-source hypervisors and device drivers.

Enlightenment is an API and a set of optimizations designed specifically to enhance the performance of Windows Server in a Windows virtualized environment.

Hardware manfuacturers are interested in virtualization as well. Intel and AMD have independently developed virtualization extensions to the x86 architecture. They are not directly compatible with each other, but serve largely the same functions. Either will allow a hypervisor to run an unmodified guest operating system without incurring significant performance penalties.

Intel's virtualization extension for 32-bit and 64-bit x86 architecture is named IVT (short for Intel Virtualization Technology). The 32-bit or IA-32 IVT extensions are referred to as VT-x. Intel has also published specifications for IVT for the IA-64 (Itanium) processors which are referred to as VT-i; .

AMD's virtualization extensions to the 64-bit x86 architecture is named AMD Virtualization, abbreviated AMD-V.

There are three Virtual Machine Monitor models.

A type 2 Virtual Machine Monitor runs within a host operating system. It operates at a level above the host OS and all guest environments operate at a level above that. Examples of these guest environments include the Java Virtual Machine and Microsoft’s Common Language Runtime, which runs as part of the .NET environment and is a “managed execution environment” that allows object-oriented classes to be shared among applications.

The hybrid model, shown in the middle of the diagram has been used to implement Virtual PC, Virtual Server and VMWare GSX. These rely on a host operating system that shares control of the hardware with the virtual machine monitor.

A type 1 Virtual Machine Monitor employs a hypervisor to control the hardware with all operating systems run at a level above it. Windows Server virtualization (WSv) and Xen are examples of type 1 hypervisor implementations.

Development of Xen and the Linux hypervisor API paravirt_ops began prior to release of Intel and AMD’s virtualized hardware and were designed, in part, to solve the problems inherent in running a virtualized environment on non-virtualization-assisted hardware. They continue to support both virtualization-assisted and non-virtualization-assisted hardware. These approaches are distinct from KVM, or the Kernel-based Virtual Machine, supports only virtualization-assisted hardware; this approach uses the Linux kernel as the hypervisor and QEMU to set up virtual environments for Linux guest OS partitions.

In keeping with the open source community’s philosophy of encouraging development of open source code, the paravirt_ops API is designed to support open-source hypervisors. Earlier this year VMware’s VMI was added to the kernel as was Xen. Paravirt_ops is in effect a function table that enables different hypervisors – Xen, VMware, WSv – to provide implementation of a standard hypercall interface, including a default set of functions that write to the hardware normally.

Windows Server 2008 enlightenments have been designed to allow WS 2008 to run in either a virtualized or non-virtualized environment *unmodified*. WS 2008 recognizes when it is running as a guest on top of WSv and dynamically applies the enlightenment optimizations in such instances.

In addition to a hypercall interface and a synthethic device model, memory management and the WS 2008 scheduler are designed with optimizations for when the OS runs as a virtual machine.

The WSv architecture is designed so that a parent partition provides services to the child partitions that run as guests in the virtual environment. From left to right:

Native WSv Components:

VMBus – Virtual Machine Bus – Serves as a synthetic bus for the system, enabling child partitions to access native drivers.
VSP – Virtual Service Provider – Serves as an interface between the VMBus and a physical device
HCL Drivers – “Hardware Compatibility List” Drivers (standard native Windows drivers that have passed WHQL certification)
VSC – Virtual Service Consumer – Functions as a synthetic device. For example, a filesystem will talk to the VSC controller instead of an IDE controller. This in turn communicates with the VSP to dispatch requests through the native driver.

Interoperability Components:

Linux VSC – Interoperability component that serves as a synthetic Linux driver. Functions like the VSC in a Windows partition. Developed by XenSource and published under a BSD-style license.
Hypercall Adapter – Adapts Linux paravirt_ops hypercalls to WSv

Like the WSv architecture, the Xen architecture is designed so that a special partition, in this case Dom 0, provides services to guest partitions that run in a virtual environment.

Native Xen Components:

paravirt_ops is a Linux-kernel-internal function table that is designed to support hypervisor-specific function calls. The default function pointers from paravirt_ops support running as a host on bare metal. Xen provides its own set of functions that implement paravirtualization.
Native Drivers – standard set of drivers in the Linux kernel
Xen/Linux ABI – having a consistent ABI enables long-term compatibility between guest operating systems and the Xen hypervisor

Interoperability Components:

Xen Virtualized Drivers – Windows synthetic device drivers must be converted to Xen-virtualized drivers. These are developed using the Windows DDK and will be distributed as binary only per the DDK license.
Xen/Windows ABI – The binary interface that integrates Windows with Xen, enabling Windows hypercalls to be executed through Xen instead of WSv. This will be licensed under the GPL and made available when the WSv top-level functional specification is made public.

The slide says it all… I couldn’t figure out a way to put this one in a graphic. ;)

Virtualization interoperability testing is very challenging. While the architecture may look similar at a high level, the devil is in the details – down at the API and ABI level, the technologies are quite different.

From a personnel standpoint, the expertise required to debug OS kernels is hard to find, let alone software engineers with these skills who are focused on writing test code. Microsoft has established a role known as “Software Design Engineer in Test” or “SDE/T” which describes the combination of skills and attitude required to test large-scale complex software rigorously through automated white-box test development.

The problem of testing Linux and Windows OSes across WSv and Xen requires these kernel-level skills, but on both operating systems. It’s a non-trivial challenge.

Next is the technical issue of the test matrix:

Two full operating systems to test (Windows Server 2008 and SUSE Linux Enterprise Server 10)
Single-core, dual-core, and quad-core CPUs
Single-processor, dual-processor, and quad-processor boards
Intel-VT and AMD-V chips
Basic device configuration (NIC, HD, etc.)

To put this in context, we need a minimum of 40 server chassis to test this matrix – for each operating system.

On top of this, the software components that must be tested include:

Linux VSC
Windows PV hardware drivers
Xen/Windows ABI
Linux/WSv hypercall adapter

Since Windows and Linux are general-purpose operating systems, these components must be tested across a range of workloads which will guarantee consistent, high-performance operation regardless of usage (file serving, web serving, compute-intensive operations, networking, etc.).

Finally – and no less a challenge than the skills and technology aspects – is that of building a shared culture between two very different and mature engineering culture. What is the definition of a “Severity 1” or “Priority 1” designation for a defect? How do these defects compete for the core product engineering teams’ attention? How are defects tracked, escalated, processed, and closed across two different test organizations’ software tools? Most importantly, what is the quality of the professional relationships between engineers and engineering management of the two organizations? These are the critical issues to make the work happen at high quality and with consistency over the long term.

WS-Management is an industry standard protocol managed by the DMTF (Distributed Management Taskforce), whose working group members include HP, IBM, Sun, BEA, CA, Intel, and Microsoft among others. The purpose is to bring a unified cross-platform management backplane to the industry, enabling customers to implement heterogeneous datacenters without having separate management systems for each platform.

All Microsoft server products ship with extensive instrumentation, known as WMI. A great way to see the breadth of this management surface is to download Hyperic (an open source management tool) and attach it to a Windows server – all of the different events and instrumentation will show up in the interface, typically several screen pages long.

It is not surprising that the management tools vendors are collaborating on this work – and it’s essential to have not just hardware, OS, and management providers but application layer vendors like BEA as well – but to me the most important aspect of the work is the open source interoperability.

In the Microsoft-Novell Joint Interoperability Lab, we are testing the Microsoft implementation of WS-Management (WinRM) against the openwsman and wiseman open source stacks. This matters because the availability of proven, interoperable open source implementations will make it relatively easy for all types of providers of both management software and managed endpoints to adopt a technology that works together with existing systems out of the box. Regardless of development or licensing model, commercial and community software will be able to connect and be well-managed in customer environments.

So what does this all mean? We’ll see end-to-end interoperability, where any compliant console can manage any conforming infrastructure – and since the specification and the code are open, the barriers to entry are very low. It’s important that this capability extends to virtualized environments (which is non-trivial) so that customers can get the full potential of the benefits of virtualization – not just reducing servers at the cost of increased management effort.

Sometimes people challenge me with the statement “if you would just build software to the specification, you wouldn’t need to all this interoperability engineering!” This is in fact a mistaken understanding of interoperability engineering. Once you’ve read through a specification – tens to hundreds of pages of technical detail – and written an implementation that matches the specification, then the real work begins. Real-world interoperability is not about matching what’s on paper, but what’s on the wire. This is why it’s essential to have dedicated engineering, comprehensive automated testing, and multiple products and projects working together. A good example of this is the engineering process for Microsoft’s Web Services stack. The specifications (all 36 of them) are open, and licensed under the OSP (Open Specification Promise). In the engineering process, Microsoft tests the Windows Web Services implementation against the IBM and the Apache Axis implementations according to the WS-I Basic profile. A successful pass against all these tests is “ship criteria” for Microsoft, meaning we won’t ship our implementation unless it passes.

In the messy world of systems management, where multiple generations of technologies at a wide range of ontological levels (devices, motherboards, networking gear, operating systems, databases, middleware, applications, event aggregators, and so on) testing is complex. Adding virtualization into this mix adds another layer of complexity, necessitating methodical and disciplined testing.

Open ID is a distributed single sign-on system, primarily for websites. It’s supported by a range of technology providers including AOL, LiveJournal, and Microsoft.

WS-Federation is the identity federation web services standard which allows different identity providers to work together to exchange or negotiate information about user identity. It is layered on top of other Web Services specifications including WS-Trust, WS-Security, and WS-SecurityPolicy – many of which are lacking an open source implementation today.

ADFS is Active Directory Federation Services, a mechanism for identity federation built into Microsoft Active Directory.

Cardspace is an identity metasystem, used to secure user information and unify identity management across any internet site.

Project Higgins is an Eclipse project intended to develop open source implementations of the WS-Federation protocol stack as well as other identity technologies including OpenID and SAML.

Samba is a Linux/Unix implementation of Microsoft’s SMB/CIFS protocols for file sharing and access control information. It is widely deployed in Linux-based appliances and devices, and ships in every popular distribution of Linux as well as with Apple’s OS X.

This work is still in early phases, and you can expect more details here in the future. Mike Milinkovich of Eclipse has been a champion for improving the interoperability of Eclipse and Microsoft technologies, especially Higgins. Separately the Bandit Project has made significant progress in building technologies which support CardSpace. I appreciate the work of these teams and look forward to more progress here.

The slide says it all here. We’re committed to long term development and delivery of customer-grade interoperability solutions for Windows and Linux, and we’ll do it in a transparent manner. Tom Hanrahan, the Director of the Microsoft-Novell Joint Interoperability Lab, brings many years of experience in running projects where the open source community is a primary participant. I and my colleagues at Microsoft are excited to learn from him as he puts his experiences at the OSDL/Linux Foundation and at IBM’s Linux Technology Center into practice guiding the work of the lab.

You can expect regular updates from us on the progress and plans for our technical work, and I expect you to hold me and Tom accountable for this promise.

I hope you found the presentation valuable. I felt it was important to get this material out broadly since it will impact many people and essential to be clear about what we are building together with Novell, XenSource, and the open source community.

Justin Steinman and Sam Ramji on the Novell/Microsoft Partnership

Sam Ramji — Fri, 03 Aug 2007 16:51:00 GMT

Back in the Spring, Sam Ramji attended an Olliance event entitled, the Open Source Think Tank. It's a smaller gathering, and well attended by nearly 100 or so executives and influential developer-users of open source software. During one of the sessions, Sam and Justin Steinman took an impromptu moment to answer some tough questions regarding the nature of the Microsoft-Novell partnership. Justin Steinman is Novell's Director of Marketing for Linux and Open Platforms.

Many of the questions had been asked before and in fact have been posited more than once on Port 25. We thought these discussions would be interesting to the community at large - ...so Sam & Justin hopped on the phone recently to answer them in podcast format. Take a listen....I try to emcee - but these are tough guys to keep on one topic :) As always, we welcome feedback and we'll invite Sam & Justin to answer the comments. If you want more information on the Novell partnership - you may want to check out moreinterop.com - home to most announcements, events and information related to the partnership.