Port 25: The Open Source Community at Microsoft : Virtualization, Linux

Microsoft and Novell: Three Years and Going Strong

Peter Galli — Mon, 09 Nov 2009 19:19:00 GMT

Today, Microsoft and Novell marked the third anniversary of the collaboration agreement during a gathering of IT executives at the Society of Information Management SIMPosium 09 conference in Seattle.

As many customers are running heterogeneous IT environments and are looking to their software vendors to address their interoperability needs, over the past three years Microsoft and Novell have been working together to help these organizations future-proof their IT operations by optimizing mixed source IT infrastructures to better support today and prepare for tomorrow.

This collaboration and the resultant joint solutions have already enhanced reliability and efficiency within mixed IT systems environments. These technical solutions help customers centralize or streamline management functions, reduce internal support requirements, and enable greater interoperability without having to dedicate time and resources to devise workarounds.

The Microsoft-Novell Interoperability Lab also remains focused on addressing our customers' priority interoperability concerns, and this technical collaboration has already produced real-world solutions spanning Virtualization, Systems Management, Rich Media and Server Workload Validation.

Those customers who choose to use Windows and Linux together can leverage not only SUSE Linux Enterprise Server from Novell, but also its support offerings, to ensure business continuity, while customers have added peace of mind as a result of the intellectual property (IP) provisions of the agreement between the two companies.

The success of this relationship is also underscored by steady sales, even in the current difficult economic environment: as of July 31, 2009, Novell has invoiced $226 million in certificate revenue, or 91percent of the total $247.5 million investment.

Microsoft and Novell have jointly recruited more than 475 new customers to receive certificates from Microsoft for three-year priority support subscriptions for Novell's SUSE Linux Enterprise Server (SLES) since the agreement was formalized in November 2006.

To date, more than 20 of these joint customers have also opted to take advantage of Novell's SUSE Linux Enterprise Server (SLES) Subscription with Expanded Support, which offers upgrades, updates and technical support for customers' existing paid and unpaid Linux deployments, including Red Hat Enterprise Linux (RHEL), while they transition to Novell SLES. Customers most commonly cite value and quality as key factors driving their choice of Novell over Red Hat.

Migration from Red Hat and other Linux offerings to SUSE Linux Enterprise Server also helps these customers optimize the capabilities of their IT systems for the future by allowing them to leverage real-world solutions like Microsoft Hyper-V and Microsoft System Center Operations Manager.

You can read more about all this in an article published to Microsoft PressPass, while a number of customer case studies are also available.

Virtualizing Free Linux Distributions in Windows Server 2008 R2

Peter Galli — Tue, 11 Aug 2009 00:00:00 GMT

Jason Perlow, a columnist over at ZDNet, has written a comprehensive review on virtualizing free Linux distributions in Windows Server 2008 R2.

In his Tech Broiler column, Perlow notes that the updated Hyper-V bare-metal hypervisor virtualization layer in Microsoft's upcoming Windows Server 2008 R2, which is due to be released August 14th to MSDN and Technet customers, now has support for SUSE Linux Enterprise Server 11 and Red Hat Enterprise Linux 5.3.

"Additionally, Linux support and performance has greatly improved over the initial Hyper-V release. Microsoft also recently released its Hyper-V Linux Integration Components (Linux ICs) under the GPLv2 Open Source License," Perlow says.

The Linux ICs for Hyper-V, which are in Release Candidate status, provide synthetic device drivers that enhance I/O and networking performance when Linux OSes are virtualized under Hyper-V.

"The source code for the Linux IC's were accepted into the Linux Driver Project and should become part of the Linux Kernel within two subsequent releases and code merges - 2.6.32 is expected to be when they will be integrated, and all Linux distributions using that kernel code base going forward should be Hyper-V enabled out of the box. Yes, you heard that correctly, Microsoft is now an official Linux Kernel contributor," Perlow says.

You can read the rest of Perlow's column here.

More on the Hyper-V Linux Integration Components

hjanssen — Mon, 20 Jul 2009 20:24:00 GMT

Well, there is no easy way to say this, so I am simply going to start this blog with the following line.

Microsoft just submitted source code for the Hyper-V Linux Integration Components to the Linux Kernel Community Under GPL v2.

Well, there's a conversation starter! Are you still all sitting in your chairs???

Let me summarize:

Yes, our device driver code was released directly to the Linux Kernel
We released the code under GPL v2
We are working with Greg Kroah-Hartman so it is ready for the next release of the Linux Kernel, version 2.6.32
We will continue to update the driver code to enhance interoperability on an ongoing basis, but it's our hope that other developers in the community will find the code useful and worthy of collaboration.

Fallen off your chair yet?

Microsoft developed the Linux device drivers to enhance the performance of Linux when virtualized on Windows Server 2008 Hyper-V. My team and I were responsible for testing and validating the driver components that were contributed for this first release.

Now, my team and I will be responsible for further developing this code going forward. (Yes, that does mean that I have gone back to leverage my very early roots as a Kernel programmer. Let the world be warned!!!!). Haiyang Zhang has been working on this code with me, and he will continue to work with me on this going forward.

When I joined Microsoft three years ago, the primary reason was to put my money where my mouth was. You see complaining about something is easy, but it becomes a little more complicated when somebody offers you the opportunity to be part of helping change what you have complained about.

So, three years after taking the job that made me put my money where my mouth was (and still often is!), I for one am EXTREMELY happy to see one of the most significant fruits of our work here in the Microsoft Open Source Technology Center (OSTC). But I have to say, even I would have been hard-pressed to think three years ago that we would consider contributing to the Linux Kernel.

As you know, two years ago Microsoft announced a partnership with Novell, and Tom Hanrahan ran the lab on a day to day basis till about 9 months ago. Since then I have had the pleasure of running the technical side of the execution of that lab under Tom Hanrahan for the OSTC. One of the primary tasks for that lab is to make sure Windows runs well on top of XEN and Linux runs well on top of Hyper-V, and we do this in very close cooperation with Novell.

We do most of this work as an extension to Mike Neil's Hyper-V team.

As part of this, we were asked to help develop and maintain a crucial part of this work called the Linux Integration Components. This code is designed so that Linux can run in an "enlightened mode" on top of Hyper-V (enlightened mode is roughly the Hyper-V equivalent of "paravirtualized mode" for the Xen hypervisor). Without this driver code, Linux can run on top of Windows, but without the same high performance levels. It is this device driver code that we are releasing today, directly to the Linux Kernel.

We're not talking a few hundred lines of code here; we're talking about roughly 20,000 lines of code.

Is this a Dump and Run from Microsoft? Absolutely not! We plan to enhance the functionality of this code, and we will continue to work with the Linux Community to support the drivers and to ensure continued interoperability.

As you can imagine, this was the result of a lot of hard work: Hiyang Zhang, who has been co-writing this code; Hashir Abdi, who has been testing all this stuff; as well as Vijay Tewari and Mike Sterling from the Hyper-V team who have been taking care of the Hyper-V side.

And last, but certainly not least, Greg Kroah-Hartman, who has been helping me to make all this code land in the right area in the kernel. He has patiently worked to help me correct my obvious mistakes and to get the code contributed into the kernel.

So where are we today? Well, Greg Kroah-Hartman will make the code visible to the outside world today. (For those who want to get a head start, the code will sit under <your kernel tree>/drivers/staging/hv). After it becomes visible, I will write a few more blogs this week that should help you to understand, build and run them.

The titles I am thinking for these blogs are:

Where do the Linux ICs reside in the kernel tree and how do I build them?

And

How do I install, configure and run the Linux IC's?

I had almost forgotten how wrapped up you can be once you start writing code again. So I have not gotten much sleep this past week, but it has been a joy to get back into coding again!

Microsoft Releases Device Driver Code to the Linux Community

Peter Galli — Mon, 20 Jul 2009 16:48:00 GMT

In what many may see as a surprising move, Microsoft today released 20,000 lines of device driver code to the Linux community under the popular General Public Licence v2.

The code includes three Linux device drivers, and has been submitted to the Linux kernel community for inclusion in the Linux tree.

The drivers will be available to both the Linux community and customers, and will enhance the performance of the Linux operating system when virtualized on Windows Server 2008 Hyper-V or Windows Server 2008 R2 Hyper-V.

In an article posted to Microsoft's PressPass site, Tom Hanrahan, director of Microsoft's Open Source Technology Center, notes that this is a significant milestone because it's the first time the company has released code directly to the Linux community. "Additionally significant is that we are releasing the code under the GPLv2 license, which is the Linux community's preferred license," he said.

In the same article, Sam Ramji, senior director of Platform Strategy at Microsoft, points out that Microsoft communities and open source communities are growing together, which is ultimately of benefit to customers. An example of this is the Linux community, which has built a platform used by many customers. "So our strategy is to enhance interoperability between the Windows platform and many open source technologies, which includes Linux, to provide the choices our customers are asking for," he said.

Ramji also alluded to the fact that people are often surprised when they hear how much open source community and development work is happening across Microsoft, which is largely due to the fact that these collaborations focus more on getting the work done and engaging with the various communities on a one-to-one basis and less about promoting them.

One example of how Microsoft participates with, and contributes to, open source is its relationship with the PHP Community. The company's involvement includes contributing to the PHP Engine, optimizing PHP 5.3 to perform strongly on Windows, and working to improve the performance of numerous PHP applications on Windows. Then there is the ongoing participation in various Apache Software Foundation projects, such as Hadoop, Stonehenge and QPID.

"In short, we're focused on building sustainable business strategies for open source at Microsoft ... we see open source playing into three key areas, one of which is the use of 'inbound' open source and the open source development model to make our software development processes more efficient."

"Good examples of this include what we did recently with jQuery in Visual Studio 2008, the implementation of OpenPegasus connectors and adaptors into System Center Operations Manager, and work that the Microsoft High Performance Computing team did with the Argonne National Lab (ANL) to source its MPICH2 implementation, which is a portable implementation of the Message Passing Interface (MPI) used in cluster computing and super computers," Ramji said.

We'll be posting a number of other articles on the release of the device driver code to the Linux community over the week, several of which will be penned by Hank Janssen from Microsoft's Open Source Technology Center, so look out for those.

Microsoft, Red Hat to Offer Joint Technical Support

Peter Galli — Mon, 16 Feb 2009 16:30:00 GMT

Microsoft and Red Hat announced this morning that they have recently signed agreements to test and validate their server operating systems running on one another's hypervisors.

This is deeply significant as it means that customers will be able to confidently deploy Windows Server and Red Hat Enterprise Linux (RHEL), virtualized on Microsoft and Red Hat hypervisors, knowing that the solutions will be supported by both companies.

In short, Red Hat has joined Microsoft's Server Virtualization Validation Program, and Microsoft is now a Red Hat partner for virtualization interoperability and support.

Microsoft will also be listed in the Red Hat Hardware Certification List once the Red Hat certification process has been completed later this year.

Microsoft will also publish Linux Integration Components for RHEL when the testing and validation is complete and, according to Mike Neil's blog on this news, Red Hat is expected to provide Windows Hardware Quality Labs drivers for a variety of Windows Server versions.

"This means that those customers with valid support agreements will be able to run these validated configurations and receive joint technical support for running Windows Server on Red Hat Enterprise virtualization, and for running Red Hat Enterprise Linux on Windows Server 2008 Hyper-V or Hyper-V Server 2008," Neil says.

So, while Microsoft and Red Hat will continue to compete, customers have asked us to work together on technical support for server virtualization. These agreements respond to that request by giving them a new level of integration between Red Hat Enterprise Linux and Windows Server for their heterogeneous IT environments.

Customers with valid support agreements will now be able to call either Microsoft or Red Hat to have their issues resolved. If the first vendor contacted cannot resolve the issue, they will refer the problem to the other vendor for resolution; assuming the customer also has a valid support agreement with that vendor.

In the event that the second vendor cannot resolve the problem alone, Microsoft and Red Hat will work together to come to a resolution for the mutual customer.

What's more, once RHEL is validated as a guest on Windows Server 2008, Microsoft System Center Operations Manager 2007 R2 - which will include cross platform monitoring - will support RHEL server versions 4 and 5 in the second quarter of this year so that customers can manage the applications and operating systems in the guest VM.

This will allow customers to monitor end-to-end data center applications that are distributed across both Windows Server and RHEL, whether these servers are physical or virtual, thereby improving the visibility organizations have of these distributed applications, and reducing their operational costs by providing a single tool to manage these across operating systems.

Also, to be clear given that questions are going to be asked about how this compares to the existing relationship between Microsoft and Novell, this agreement with Red Hat is specific to joint technical support for our mutual customers using server virtualization. So, in that regard, think of it as one dimensional, whereas Microsoft's partnership with Novell is multi-dimensional.

For more on all this, you can read Mike Neil's blog, the press release here, and watch the public webcast.

Two Years and Counting....

Peter Galli — Wed, 19 Nov 2008 04:16:00 GMT

It is two years this month since Microsoft and Novell struck their ground-breaking technical collaboration agreement, a move that has effectively ensured greater interoperability between Windows Server and SUSE Linux Enterprise Server.

This technical collaboration has already resulted in a number of milestones, including two new offerings announced today: the availability in the first half of 2009 of an Advanced Management Pack for SUSE Linux Enterprise for Microsoft System Center Operations Manager 2007 R2, and a free beta download of Novell's Moonlight, a rich media application.

Some analysts, vendors and enterprises have said the company that develops effective cross-platform management tools will have an advantage and strategic differentiator over its competitors who do not. Microsoft is already doing that.

The Microsoft Operations Manger 2007 Cross Platform Extensions enable the assessment and management of Windows and Linux servers from a single, unified console, eliminating the costs and complexities of having multiple management consoles. The Advanced Management Pack extends this Linux monitoring capability.

Also, given the current tough economic environment, this solution helps reduce training costs since staff only need to be trained on one management tool for both Windows and Linux environments.

Attendees at the Microsoft TechEd EMEA conference in Barcelona earlier this month got to see a technical preview of the Advanced Management Pack, whose release will coincide with that of Microsoft System Center Operations Manager 2007 R2.

A beta of Novell's Moonlight, an open source implementation of Microsoft Silverlight, will also be released going forward as an open source plug-in for the Firefox web browser. Moonlight brings Linux-based users the same high-definition media capabilities currently available for the Windows and Apple environments.

So, expect to see a lot more solutions in the next year that promote interoperability and help ease customer pain-points across their heterogeneous environments.

Virtual Machine Additions for Linux 2.0

jcannon — Tue, 23 Oct 2007 15:41:00 GMT

A quick note to let our community know that Virtual Machine Additions for Linux 2.0 has been released - bringing the version number up to 2.0. For those unfamiliar with Virtual Machine Additions for Linux, it is technology layer designed to improve the usability and interoperability of running Linux operating systems as guests or virtual machines inside of Virtual Server. From the release notes, it looks like this version adds support for SuSE Linux Enterprise Server 10. You can find additional information on running Linux as a guest operating system with Virtual Server on TechNet.

Qualified distributions now include:

Red Hat Enterprise Linux 2.1 (update 6)
Red Hat Enterprise Linux 3 (update 6)
Red Hat Enterprise Linux 4
SuSE Linux Enterprise Server 9
SuSE Linux Enterprise Server 10
Red Hat Linux 7.3
Red Hat Linux 9.0
SuSE Linux 9.2
SuSE Linux 9.3
SuSE Linux 10.0

Check out download details here.

[PostIcon:3352]

Linux and Windows Interoperability: On the Metal and On the Wire

MichaelF — Mon, 13 Aug 2007 17:10:00 GMT

I had the opportunity to present at both OSCON in Portland and at LinuxWorld in San Francisco in the last three weeks – both O’Reilly and IDG were gracious enough to grant me a session on the work that Microsoft is doing with Novell, XenSource, and others on Linux and Windows interoperability.

Overall our focus is on three critical technology areas for the next-generation datacenter: virtualization, systems management, and identity. Identity in particular spans enterprise datacenters and web user experiences, so it’s critical that everyone shares a strong commitment to cross-platform cooperation.

Here are the slides as I presented them, with some words about each to give context, but few enough to make this post readable overall. I skipped the intro slides about the Open Source Software Lab since most Port 25 readers know who we are and what we do.

Why interoperability?

The market for heterogeneous solutions is growing rapidly. One visible sign of this is virtualization, an “indicator technology,” which by its nature promotes heterogeneity. Virtualization has become one of the most important trends in the computing industry today. According to leading analysts, enterprise spending on virtualization will reach $15B worldwide by 2009, at which point more than 50% of all servers sold will include virtualization-enabled processors. Most of this investment will manifest itself on production servers running business critical workloads.

Given the ever improving x86 economics, companies are continuing to migrate off UNIX and specialty hardware down to Windows and Linux on commodity processors.

So, why now?

First, customers are insisting on support for interoperable, heterogeneous solutions. At Microsoft, we run a customer-led product business. One year ago, we established our Interoperability Executive Customer Council, a group of Global CIOs from 30 top global companies and governments – from Goldman Sachs to Aetna to NATO to the UN. On the Microsoft side, this council is run by Bob Muglia, the senior vice president of our server software and developer tools division. The purpose of this is to get consistent input on where customers need us to improve interoperability between our platforms and others – like Linux, Eclipse, and Java. They gave us clear direction: “we are picking both Windows and Linux for our datacenters, and will continue to do so. We need you to make them work better together.”

Second, MS and Novell have established a technical collaboration agreement that allows us to combine our engineering resources to address specific interoperability issues.

As part of this broader interoperability collaboration, Microsoft and Novell technical experts are architecting and testing cross-platform virtualization for Linux and Windows and developing the tools and infrastructure necessary to manage and secure these heterogeneous environments.

I am often asked, “Why is the agreement so long?” as well as “Why is the agreement so short?” The Novell-Microsoft TCA is 5 years mutual commitment. To put this in context, 5 years from now (2012) is two full releases of Windows Server and 20 Linux kernel updates (given the 2.5 month cycle we’ve seen for the last few years). This is an eternity in technology. What’s important to me is that it’s a multi-product commitment to building and improving interoperability between the flagship products of two major technology companies. This means we can build the practices to sustain great interoperable software over the long term as our industry and customer needs continue to evolve.

This talk covers two major components of the future of Linux and Windows interoperability: Virtualization and Web Services protocols.

On the Metal focuses on the virtualization interoperability work being done between Windows Server 2008 and Windows Server virtualization, and SUSE Linux Enterprise Server and Xen.

On the Wire covers the details and challenges of implementing standards specifications, such as WS-Federation and WS-Management; and how protocol interoperability will enable effective and secure virtualization deployment and management.

These are the key components required for the next-generation datacenter. We know the datacenters of today are mixtures of Windows, Linux, and Unix, x86, x64 and RISC architectures, and a range of storage and networking gear. Virtualization is required to enable server consolidation and dynamic IT; it must be cross-platform. Once applications from multiple platforms are running on a single server, they need to be managed – ideally from a single console. Finally, they must still meet the demands of security and auditability, so regardless of OS they must be accessible by the right users at the right levels of privilege. Hence, cross-platform virtualization demands cross-platform management and identity.

In non-virtualized environments, a single operating system is in direct control of the hardware. In a virtualized environment a Virtual Machine Monitor manages one or more guest operating systems that are in “virtual” control of the hardware, each independent of the other.

A hypervisor is a special implementation of a Virtual Machine Monitor. It is software that provides a level of abstraction between a system’s hardware and one or more operating systems running on the platform.

Virtualization optimizations enable better performance by taking advantage of “knowing” when an OS is a host running on HW or a guest running on a virtual machine.

Paravirtualization , as it applies to Xen and Linux, is an open API between a hypervisor and Linux and a set of optimizations that together, in keeping with the open source philosophy, encourage development of open-source hypervisors and device drivers.

Enlightenment is an API and a set of optimizations designed specifically to enhance the performance of Windows Server in a Windows virtualized environment.

Hardware manfuacturers are interested in virtualization as well. Intel and AMD have independently developed virtualization extensions to the x86 architecture. They are not directly compatible with each other, but serve largely the same functions. Either will allow a hypervisor to run an unmodified guest operating system without incurring significant performance penalties.

Intel's virtualization extension for 32-bit and 64-bit x86 architecture is named IVT (short for Intel Virtualization Technology). The 32-bit or IA-32 IVT extensions are referred to as VT-x. Intel has also published specifications for IVT for the IA-64 (Itanium) processors which are referred to as VT-i; .

AMD's virtualization extensions to the 64-bit x86 architecture is named AMD Virtualization, abbreviated AMD-V.

There are three Virtual Machine Monitor models.

A type 2 Virtual Machine Monitor runs within a host operating system. It operates at a level above the host OS and all guest environments operate at a level above that. Examples of these guest environments include the Java Virtual Machine and Microsoft’s Common Language Runtime, which runs as part of the .NET environment and is a “managed execution environment” that allows object-oriented classes to be shared among applications.

The hybrid model, shown in the middle of the diagram has been used to implement Virtual PC, Virtual Server and VMWare GSX. These rely on a host operating system that shares control of the hardware with the virtual machine monitor.

A type 1 Virtual Machine Monitor employs a hypervisor to control the hardware with all operating systems run at a level above it. Windows Server virtualization (WSv) and Xen are examples of type 1 hypervisor implementations.

Development of Xen and the Linux hypervisor API paravirt_ops began prior to release of Intel and AMD’s virtualized hardware and were designed, in part, to solve the problems inherent in running a virtualized environment on non-virtualization-assisted hardware. They continue to support both virtualization-assisted and non-virtualization-assisted hardware. These approaches are distinct from KVM, or the Kernel-based Virtual Machine, supports only virtualization-assisted hardware; this approach uses the Linux kernel as the hypervisor and QEMU to set up virtual environments for Linux guest OS partitions.

In keeping with the open source community’s philosophy of encouraging development of open source code, the paravirt_ops API is designed to support open-source hypervisors. Earlier this year VMware’s VMI was added to the kernel as was Xen. Paravirt_ops is in effect a function table that enables different hypervisors – Xen, VMware, WSv – to provide implementation of a standard hypercall interface, including a default set of functions that write to the hardware normally.

Windows Server 2008 enlightenments have been designed to allow WS 2008 to run in either a virtualized or non-virtualized environment *unmodified*. WS 2008 recognizes when it is running as a guest on top of WSv and dynamically applies the enlightenment optimizations in such instances.

In addition to a hypercall interface and a synthethic device model, memory management and the WS 2008 scheduler are designed with optimizations for when the OS runs as a virtual machine.

The WSv architecture is designed so that a parent partition provides services to the child partitions that run as guests in the virtual environment. From left to right:

Native WSv Components:

VMBus – Virtual Machine Bus – Serves as a synthetic bus for the system, enabling child partitions to access native drivers.
VSP – Virtual Service Provider – Serves as an interface between the VMBus and a physical device
HCL Drivers – “Hardware Compatibility List” Drivers (standard native Windows drivers that have passed WHQL certification)
VSC – Virtual Service Consumer – Functions as a synthetic device. For example, a filesystem will talk to the VSC controller instead of an IDE controller. This in turn communicates with the VSP to dispatch requests through the native driver.

Interoperability Components:

Linux VSC – Interoperability component that serves as a synthetic Linux driver. Functions like the VSC in a Windows partition. Developed by XenSource and published under a BSD-style license.
Hypercall Adapter – Adapts Linux paravirt_ops hypercalls to WSv

Like the WSv architecture, the Xen architecture is designed so that a special partition, in this case Dom 0, provides services to guest partitions that run in a virtual environment.

Native Xen Components:

paravirt_ops is a Linux-kernel-internal function table that is designed to support hypervisor-specific function calls. The default function pointers from paravirt_ops support running as a host on bare metal. Xen provides its own set of functions that implement paravirtualization.
Native Drivers – standard set of drivers in the Linux kernel
Xen/Linux ABI – having a consistent ABI enables long-term compatibility between guest operating systems and the Xen hypervisor

Interoperability Components:

Xen Virtualized Drivers – Windows synthetic device drivers must be converted to Xen-virtualized drivers. These are developed using the Windows DDK and will be distributed as binary only per the DDK license.
Xen/Windows ABI – The binary interface that integrates Windows with Xen, enabling Windows hypercalls to be executed through Xen instead of WSv. This will be licensed under the GPL and made available when the WSv top-level functional specification is made public.

The slide says it all… I couldn’t figure out a way to put this one in a graphic. ;)

Virtualization interoperability testing is very challenging. While the architecture may look similar at a high level, the devil is in the details – down at the API and ABI level, the technologies are quite different.

From a personnel standpoint, the expertise required to debug OS kernels is hard to find, let alone software engineers with these skills who are focused on writing test code. Microsoft has established a role known as “Software Design Engineer in Test” or “SDE/T” which describes the combination of skills and attitude required to test large-scale complex software rigorously through automated white-box test development.

The problem of testing Linux and Windows OSes across WSv and Xen requires these kernel-level skills, but on both operating systems. It’s a non-trivial challenge.

Next is the technical issue of the test matrix:

Two full operating systems to test (Windows Server 2008 and SUSE Linux Enterprise Server 10)
Single-core, dual-core, and quad-core CPUs
Single-processor, dual-processor, and quad-processor boards
Intel-VT and AMD-V chips
Basic device configuration (NIC, HD, etc.)

To put this in context, we need a minimum of 40 server chassis to test this matrix – for each operating system.

On top of this, the software components that must be tested include:

Linux VSC
Windows PV hardware drivers
Xen/Windows ABI
Linux/WSv hypercall adapter

Since Windows and Linux are general-purpose operating systems, these components must be tested across a range of workloads which will guarantee consistent, high-performance operation regardless of usage (file serving, web serving, compute-intensive operations, networking, etc.).

Finally – and no less a challenge than the skills and technology aspects – is that of building a shared culture between two very different and mature engineering culture. What is the definition of a “Severity 1” or “Priority 1” designation for a defect? How do these defects compete for the core product engineering teams’ attention? How are defects tracked, escalated, processed, and closed across two different test organizations’ software tools? Most importantly, what is the quality of the professional relationships between engineers and engineering management of the two organizations? These are the critical issues to make the work happen at high quality and with consistency over the long term.

WS-Management is an industry standard protocol managed by the DMTF (Distributed Management Taskforce), whose working group members include HP, IBM, Sun, BEA, CA, Intel, and Microsoft among others. The purpose is to bring a unified cross-platform management backplane to the industry, enabling customers to implement heterogeneous datacenters without having separate management systems for each platform.

All Microsoft server products ship with extensive instrumentation, known as WMI. A great way to see the breadth of this management surface is to download Hyperic (an open source management tool) and attach it to a Windows server – all of the different events and instrumentation will show up in the interface, typically several screen pages long.

It is not surprising that the management tools vendors are collaborating on this work – and it’s essential to have not just hardware, OS, and management providers but application layer vendors like BEA as well – but to me the most important aspect of the work is the open source interoperability.

In the Microsoft-Novell Joint Interoperability Lab, we are testing the Microsoft implementation of WS-Management (WinRM) against the openwsman and wiseman open source stacks. This matters because the availability of proven, interoperable open source implementations will make it relatively easy for all types of providers of both management software and managed endpoints to adopt a technology that works together with existing systems out of the box. Regardless of development or licensing model, commercial and community software will be able to connect and be well-managed in customer environments.

So what does this all mean? We’ll see end-to-end interoperability, where any compliant console can manage any conforming infrastructure – and since the specification and the code are open, the barriers to entry are very low. It’s important that this capability extends to virtualized environments (which is non-trivial) so that customers can get the full potential of the benefits of virtualization – not just reducing servers at the cost of increased management effort.

Sometimes people challenge me with the statement “if you would just build software to the specification, you wouldn’t need to all this interoperability engineering!” This is in fact a mistaken understanding of interoperability engineering. Once you’ve read through a specification – tens to hundreds of pages of technical detail – and written an implementation that matches the specification, then the real work begins. Real-world interoperability is not about matching what’s on paper, but what’s on the wire. This is why it’s essential to have dedicated engineering, comprehensive automated testing, and multiple products and projects working together. A good example of this is the engineering process for Microsoft’s Web Services stack. The specifications (all 36 of them) are open, and licensed under the OSP (Open Specification Promise). In the engineering process, Microsoft tests the Windows Web Services implementation against the IBM and the Apache Axis implementations according to the WS-I Basic profile. A successful pass against all these tests is “ship criteria” for Microsoft, meaning we won’t ship our implementation unless it passes.

In the messy world of systems management, where multiple generations of technologies at a wide range of ontological levels (devices, motherboards, networking gear, operating systems, databases, middleware, applications, event aggregators, and so on) testing is complex. Adding virtualization into this mix adds another layer of complexity, necessitating methodical and disciplined testing.

Open ID is a distributed single sign-on system, primarily for websites. It’s supported by a range of technology providers including AOL, LiveJournal, and Microsoft.

WS-Federation is the identity federation web services standard which allows different identity providers to work together to exchange or negotiate information about user identity. It is layered on top of other Web Services specifications including WS-Trust, WS-Security, and WS-SecurityPolicy – many of which are lacking an open source implementation today.

ADFS is Active Directory Federation Services, a mechanism for identity federation built into Microsoft Active Directory.

Cardspace is an identity metasystem, used to secure user information and unify identity management across any internet site.

Project Higgins is an Eclipse project intended to develop open source implementations of the WS-Federation protocol stack as well as other identity technologies including OpenID and SAML.

Samba is a Linux/Unix implementation of Microsoft’s SMB/CIFS protocols for file sharing and access control information. It is widely deployed in Linux-based appliances and devices, and ships in every popular distribution of Linux as well as with Apple’s OS X.

This work is still in early phases, and you can expect more details here in the future. Mike Milinkovich of Eclipse has been a champion for improving the interoperability of Eclipse and Microsoft technologies, especially Higgins. Separately the Bandit Project has made significant progress in building technologies which support CardSpace. I appreciate the work of these teams and look forward to more progress here.

The slide says it all here. We’re committed to long term development and delivery of customer-grade interoperability solutions for Windows and Linux, and we’ll do it in a transparent manner. Tom Hanrahan, the Director of the Microsoft-Novell Joint Interoperability Lab, brings many years of experience in running projects where the open source community is a primary participant. I and my colleagues at Microsoft are excited to learn from him as he puts his experiences at the OSDL/Linux Foundation and at IBM’s Linux Technology Center into practice guiding the work of the lab.

You can expect regular updates from us on the progress and plans for our technical work, and I expect you to hold me and Tom accountable for this promise.

I hope you found the presentation valuable. I felt it was important to get this material out broadly since it will impact many people and essential to be clear about what we are building together with Novell, XenSource, and the open source community.