Port 25: The Open Source Community at Microsoft : Interop, Hank Janssen

Another Great Step Forward for Hyper-V, Red Hat Certification

hjanssen — Wed, 07 Oct 2009 17:06:00 GMT

Hello again! It has been a pretty busy couple of months for us, and I wanted to give you an update on what we've been doing.

We just completed the first step in another major milestone for Hyper-V. As you can read from Mike Neil's Blog, Microsoft and Red Hat just completed certification in each other's virtualization program.

This means that customers now can deploy Microsoft Windows Server and Red Hat Enterprise Linux and a range of select applications, virtualized on Red Hat and Microsoft hypervisor virtualization software, knowing that the solutions will be supported by both companies.

This again demonstrates Microsoft's commitment to its virtualization platform, and the Hyper-V team's continuous investment in interoperability and heterogeneity through the datacenter.

And I am very proud that my team has been working very hard on getting the Red Hat certification completed.

But, for my group, this is just the first step: we worked on and got certified for Hyper-V running Red Hat in emulated mode and, now, the next step for us is to get certification for enlightened mode, the mode where the guest OS is Hyper-V aware,and can thus access Hyper-V functionality directly.

So, what is needed for enlightened mode? Well, to get that step, we needed to get the Linux Integration Drivers submitted to the Linux kernel, and then we needed those drivers officially accepted in a mainline kernel.

This has now happened as well, as we have been accepted into the mainline kernel. We are in Linux Kernel release 2.6.32, and that release is currently going through development and testing. Once that one is final and officially released, we can take the next step, which is to get those official Linux Integration Component drivers certified with Red Hat.

The timeline for that is not completely set in stone right now, and I do not know if there is an official expected release date for 2.6.32 as yet. But we should be able to move forward with the enlightened mode certification soon after the 2.6.32 final release.

And we are, of course, continuing to work on the Linux Integration components, adding new features and all that good stuff. But I will write more about those items in the near future.

Introduction to the Linux Integration Components

hjanssen — Wed, 22 Jul 2009 19:02:00 GMT

Well, here is blog number two. The initial shock has worn off a bit I hope.

The feedback I have received so far has been pretty positive. This really all started in October of 2008 in a meeting with Mike Neil (GM of Hyper-V) and Tom and myself from the Open Source Technology Center (OSTC) at Microsoft.

In that meeting I proposed to Open Source the Linux Integration Components and contribute them to the Linux Kernel. And, secondly, to have the OSTC continue contributing to these IC's after they made it to the Linux Kernel. Well after some discussion, we all agreed that this was the right thing to do.

And so the whole process started inside of Microsoft. Hey, what can I say, we like to push the envelope a bit here at the OSTC, and we have a reputation to uphold!

Before I go on, I again wanted to thank the Kernel community (specifically Greg Kroah-Hartman) in helping us with explaining and guiding us through community process. It gave us a very nice jumpstart to get all of this going, and provided the groundwork for a good working relationship with the community.

I have also seen a few patches already submitted by community members, which is excellent! (Moritz Muehlenhoff gets major kudos for the first community contributed patch ) I will start submitting patches myself next week once the initial submission has stabilized a bit.

It is my plan to use the kernel as my primary development area, and of course I will continue to provide Greg with my patches. My first step is to clean up the code to make sure it fulfills all Kernel coding standards and requirements.

So, here is blog number two: what are the Linux Integration Components?

1. Overview of Linux VM with ICs

Linux Integration Components(IC) take advantages of the VMBUS and synthetic devices provided in Hyper-V to enhance the performance and usability of Linux guests running on Windows servers.

Figure: Conceptual Architecture overview of Linux guest & Hyper-V. Linux IC modules are painted in yellow color.

Glossary:

VSP: Virtualization Service Provider.

VSC: Virtualization Service Client.

VMBus: Data channel between VSP and VSC.

2. Linux IC modules -- VMBus and VSCs

Communication with parent partition is done through Linux VMBus
VSCs are the Linux drivers for synthetic devices (SCSI, IDE, and Ethernet) provided by Hyper-V.
They translate between Linux I/O requests and Hyper-V VSC commands
Devices are registered with Linux Driver Model (LDM)
Every VSC module contains two portions:
o Driver Interface Mapper (DIM): Released as open source

This portion of the VSC component interacts with the Linux kernel like a regular Linux device driver.

o VSC Core: Released as Open Source

The core portion of the VSC module is implemented based on the protocol of the corresponding VSP at Hyper-V host. The VSC core interacts with VSP via the VMBus interface.

3. Descriptions for each Linux IC module

3.1 VMBus driver (hv_vmbus.c)

The VMBus driver is a Linux kernel module. It provides both a lightweight bus driver and library functionality. As a bus driver, it registers with Linux Driver Model framework (LDM) to provide simple bus and device integration and device tree integration (sysfs). As a library, it implements the VMBus channel protocol and provide an abstraction of channel to its clients (Disk and Network VSCs).

3.2 StorVSC driver (hv_storvsc.c)

The Storage VSC interacts with the Windows Storage VSP. The "wire" protocol defined by the storage VSP determines how a VSC interacts with it. The Linux Storage VSC (LSVSC) basically abstracts the Linux I/O stack from needing to understand the Storage VSP's protocol. At the upper-edge of the LSVSC, it talks to the Linux SCSI subsystem. The Linux SCSI subsystem sees the LSVSC as a SCSI low-level driver (LLD) in Linux parlance. It passes SCSI requests (scsi_cmnd) to LSVSC which in turn converts them into the "wire" format understood by the Windows Storage VSP (VSTOR_PACKET). The bottom-edge of the LSVSC talks to Linux VMBus (LVMBUS) which in turn talks to the Windows VMBus to route the packets to the Storage VSP.

3.3 BlkVSC driver (hv_blkvsc.c)

BlkVSC (BlockVSC) supports "fast boot" and fast access to IDE disks. To enable enlightened IDE support for enhancing the performance of Linux when virtualized on Windows, a separate BlockVSC component is used as a Linux block device driver. Like StorVSC, the BlockVSC component is comprised of an upper edge wrapper that interfaces with the Linux block layer and a lower-edge through the infrastructure modules. The infrastructure modules with Hyper-V through the Linux VMBus.

3.4 NetVSC driver (hv_netvsc.c)

The network VSC send and receive network traffic between a Linux guest and Hyper-V host which has direct connection to physical network. The mechanism that this is used to accomplish is the Remote NDIS (RNDIS) protocol. Thus the communication that flows between the VSP and the VSC primarily happens over the RNDIS protocol which then is packaged and forwarded as payload over to the other side over NetVSP / VMBus protocol.

4. Linux IC's, Location in the Kernel tree

Hopefully you now have a better idea what they are. But where in the kernel tree can you find them?

Well, you can find sources in linux-next tree in /drivers/staging/hv directory.

And the git repository you can find them in right now is:

git://git.kernel.org/pub/scm/linux/kernel/git/sfr/linux-next.git

Or give this command (assuming your system is set up correctly) to download this repository to your machine:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/sfr/linux-next.git <your local name>

Since the IC's are part of the kernel now, we follow the normal community process of getting this all migrated into Linus mainline kernel.

More on the Hyper-V Linux Integration Components

hjanssen — Mon, 20 Jul 2009 20:24:00 GMT

Well, there is no easy way to say this, so I am simply going to start this blog with the following line.

Microsoft just submitted source code for the Hyper-V Linux Integration Components to the Linux Kernel Community Under GPL v2.

Well, there's a conversation starter! Are you still all sitting in your chairs???

Let me summarize:

Yes, our device driver code was released directly to the Linux Kernel
We released the code under GPL v2
We are working with Greg Kroah-Hartman so it is ready for the next release of the Linux Kernel, version 2.6.32
We will continue to update the driver code to enhance interoperability on an ongoing basis, but it's our hope that other developers in the community will find the code useful and worthy of collaboration.

Fallen off your chair yet?

Microsoft developed the Linux device drivers to enhance the performance of Linux when virtualized on Windows Server 2008 Hyper-V. My team and I were responsible for testing and validating the driver components that were contributed for this first release.

Now, my team and I will be responsible for further developing this code going forward. (Yes, that does mean that I have gone back to leverage my very early roots as a Kernel programmer. Let the world be warned!!!!). Haiyang Zhang has been working on this code with me, and he will continue to work with me on this going forward.

When I joined Microsoft three years ago, the primary reason was to put my money where my mouth was. You see complaining about something is easy, but it becomes a little more complicated when somebody offers you the opportunity to be part of helping change what you have complained about.

So, three years after taking the job that made me put my money where my mouth was (and still often is!), I for one am EXTREMELY happy to see one of the most significant fruits of our work here in the Microsoft Open Source Technology Center (OSTC). But I have to say, even I would have been hard-pressed to think three years ago that we would consider contributing to the Linux Kernel.

As you know, two years ago Microsoft announced a partnership with Novell, and Tom Hanrahan ran the lab on a day to day basis till about 9 months ago. Since then I have had the pleasure of running the technical side of the execution of that lab under Tom Hanrahan for the OSTC. One of the primary tasks for that lab is to make sure Windows runs well on top of XEN and Linux runs well on top of Hyper-V, and we do this in very close cooperation with Novell.

We do most of this work as an extension to Mike Neil's Hyper-V team.

As part of this, we were asked to help develop and maintain a crucial part of this work called the Linux Integration Components. This code is designed so that Linux can run in an "enlightened mode" on top of Hyper-V (enlightened mode is roughly the Hyper-V equivalent of "paravirtualized mode" for the Xen hypervisor). Without this driver code, Linux can run on top of Windows, but without the same high performance levels. It is this device driver code that we are releasing today, directly to the Linux Kernel.

We're not talking a few hundred lines of code here; we're talking about roughly 20,000 lines of code.

Is this a Dump and Run from Microsoft? Absolutely not! We plan to enhance the functionality of this code, and we will continue to work with the Linux Community to support the drivers and to ensure continued interoperability.

As you can imagine, this was the result of a lot of hard work: Hiyang Zhang, who has been co-writing this code; Hashir Abdi, who has been testing all this stuff; as well as Vijay Tewari and Mike Sterling from the Hyper-V team who have been taking care of the Hyper-V side.

And last, but certainly not least, Greg Kroah-Hartman, who has been helping me to make all this code land in the right area in the kernel. He has patiently worked to help me correct my obvious mistakes and to get the code contributed into the kernel.

So where are we today? Well, Greg Kroah-Hartman will make the code visible to the outside world today. (For those who want to get a head start, the code will sit under <your kernel tree>/drivers/staging/hv). After it becomes visible, I will write a few more blogs this week that should help you to understand, build and run them.

The titles I am thinking for these blogs are:

Where do the Linux ICs reside in the kernel tree and how do I build them?

And

How do I install, configure and run the Linux IC's?

I had almost forgotten how wrapped up you can be once you start writing code again. So I have not gotten much sleep this past week, but it has been a joy to get back into coding again!

On the Road in Europe - Take 1

hjanssen — Tue, 28 Oct 2008 16:25:00 GMT

Today is Tuesday - That must mean I am in Mainz. I am on day 12 of my European trip. I was in Rome and Amsterdam last week.

In Rome I attended the Moodle conference, which was pretty cool. It was put on by Roma Tre and was one of many destinations in which Moodle held conferences this past month. I went to talk to Martin Dougiamas, Helen Foster and Petr Skoda as part of our ongoing quest to get PHP on Windows to be the best experience possible.

The deployment numbers that Martin showed in his presentation are quite impressive! I have been digging around for his presentation to give these numbers, but I can't find them. I am sure that Martin must have posted his presentation somewhere, I just have not found it yet.

But the intention was to start a working relationship with the Moodle community, and this was a great start. One of the most interesting presentations was from the Italian Airforce, who described their experiences in finding better educational tools to train their personnel, and settled on Moodle to be a large part of that. It is always interesting to have a General in the audience.

It is really amazing to see how and where Moodle is used. It is a testament to the intention of Moodle and Martin and the Moodle community that is has become so popular.

The other thing I did was meet with a lot of open source influential's/Government/CTO/Journalists etc. Microsoft Rome asked me to give a bunch of presentations and interviews, which I love to do, but it turned into a 17 hour-long gauntlet. And I just want to go on record and say that I cannot be held accountable for what I talked about the last 4 or so hours of that day. It became a little blurry at that point.

The bad thing about these trips is that I am away from home for a long period of time. The really good thing about doing these trips is that I get to meet so many people. It is really cool to see the faces and have the dialogues when I talk about what Microsoft is doing in the OS world. By far it is very positive.

But I get the biggest bang for the buck when we have discussions on what people do, and want, from the OS and from Microsoft. We have been doing more and more in the OS world, but unless we work closely with the community we have no idea if we are on the right track.

One of the questions I asked in Rome in a meeting with government officials and OS influential's was ‘what does open source mean to you?' There were many responses, and most of them followed the same line. Some of the common responses where: sharing knowledge, collaboration, personal recognition, information that is easy to get to, allocation of rights, intend to make communications possible.

Well, for the next few days I will be at the IPC in Mainz. So I will blog more tomorrow. There are a lot of things we are doing right now, so I have a lot of content

Hank Janssen & John Bocharov: Introduction to SQL Server Driver for PHP (SQLPHP)

jcannon — Thu, 08 Nov 2007 18:14:00 GMT

Our own Hank Janssen gives the Channel9 team an update on the work that has been done to provide a native driver to SQL Server for PHP.

"SQL Team Says:
"The SQL Server Driver for PHP (October 2007) Community Technology Preview (CTP) is designed to enable reliable, scalable integration with SQL Server for PHP applications deployed on the Windows platform. The Driver for PHP is a PHP 5 extension that allows the reading and writing of SQL Server data from within PHP scripts. It provides a procedural interface for accessing data in all Editions of SQL Server 2005 and SQL Server 2000 (including Express Edition), and makes use of PHP features, including PHP streams to read and write large objects."

Wow. This is cool. Need to find out more about this. What exactly is this thing? Why did we create it? What are the platform requirments? Is it open source? Who are the folks behind this? You know the C9 drill. Tune in and meet SQLPHP Program Manager John Bocharov and Microsoft Open Source champion Hank Janssen who answer a bunch of questions and provide good context about the thinking behind SQLPHP, history and future. Check it out.

PHP SQL Server drivers, much improved IIS support for PHP - what is this world coming to!

hjanssen — Fri, 19 Oct 2007 22:15:00 GMT

Yes I am still alive!

As I write this, I realize that I have been very delinquent with writing blogs. Time has gone by very fast, and I keep on working on more and more things. And before you know it, you look back and months have gone by without any blogs....

But I wanted to write a few things that have happened here in the last few months. As you might be aware, Microsoft has been working very hard on making PHP on Windows a premier experience. And working in the OSSL, my group and I find ourselves in the middle of most of these efforts. And it gives us a great view of the changes that are happening inside of Microsoft (Sometimes with a gentle shove from us )

So, we assisted the IIS team with their FastCGI effort and GoLive release a few weeks ago. (http://www.iis.net/php ) And no less interesting, something that I am particularly proud of is that Microsoft has released a CTP (Community technical Preview) of a native PHP SQL server driver on Windows.

As a result, I was invited to speak at the PHP Zend conference that was held in San Francisco last week, (www.phpcon.com) I have included the presentation for people to see what we talked about. John Bocharov from the SQL server team co-presented with me on stage and showed some of the bits that are now available for download. You can download them directly from the Microsoft download site. (here is a super long URL with the download details and helpful hints for the driver; http://www.microsoft.com/downloads/details.aspx?FamilyID=85F99A70-5DF5-4558-991F-8AEE8506833C&displaylang=en )

As part of the release, we at the OSSL converted phpbb (www.phpbb.com) to run with the native SQL server driver, and I showed phpbb running natively on SQL Server, on stage - which was pretty cool. We are currently cleaning those changes up and will be submitting them to phpbb code base. And....I am looking to see what other popular PHP applications out there we can contribute support to by writing code & integrating the driver.

All this shows a commitment from Microsoft that I am pretty excited to be a part of. I mean....an IIS 6 version of FasCGI with specific support for PHP, FastCGI build into Server 2008/IIS7 with PHP optimizations and now the SQL server driver for PHP on Windows!

What is this world coming to!

A note on the PHP driver, it is like I said - a community preview. Which means we are very much looking for your feedback on how to improve it, and any bugs you might encounter. So please participate as you can to make this the driver you would like to see it be.

Let's see...what else we have we been up to? We have been working with the Apache Software Foundation more, which has been a lot of fun. And in discussions with them to see how we could better work together, I supplied them with 75 MSDN licenses to help them develop on Windows. I am looking forward to continued cooperation with ASF on many different fronts.

The Lab continues to grow in both manpower and equipment, and we now have over 300 Servers to play with. From the old P3’s (very few still left) to the latest in IBM, DELL and HP Blade systems.

I will do my best to be more responsive and blog more. So keep the comments coming! Until the next time……………..
Thanks,
Hank.

Windows Media Player Plug-in for Firefox

hjanssen — Mon, 16 Apr 2007 18:04:00 GMT

A few weeks ago I wrote a blog called ‘We're Writing Firefox Plug-ins? Interview with Ian Gilman and Thoughts on 10 Months at Microsoft’ (Who came up with that title anyway!)

In which I chronicled my first 10 months on the job. Also in there I mentioned Microsoft writing plug-ins.

Well today I am very proud to announce that we have released another official Microsoft plug-in. It shows another level of interoperability and eagerness in working with the Community to get this released.

This plug-in allows you to use Windows Media Player inside of Firefox. It is designed to support the following Windows platforms;

Windows XP SP2 (x86)
Windows XP SP2 (x64)
Windows Vista (x86)
Windows Vista (x64)

Port25 is hosting it, and you can download it from here; Windows Media Player Firefox Plugin - Download

It was put together by the Windows Media Player team, and special thanks go to Eric Anderson and Thobias Jones. (And all the other people in the Windows Media Player team). I also wanted to thank Mike Schroepfer and the Mozilla Foundation in helping us get this released.

From now on when you go to the plug-in site at Firefox, you will be automatically routed to Port25 to download and install the plug-in.

It is backwards compatible with the old 6.4 Windows Media Player. And among other things has the following new features;

New robust design that addresses all of the known issues with the old plug-in
Windows Media Player will now work with Firefox on Windows Vista
Support for the Windows Media Player OCX scripting interfaces

We will post some links to example code in the next few days to help people use some of the new features.

Install it, and to test it just go to Port25 and check out any of the videos.

Deploying Identity Management in a Mixed Environment: Hank Interviews Dustin Puryear

hjanssen — Thu, 25 Jan 2007 14:50:00 GMT

Whether deploying Windows in a Linux or UNIX environment, or vice versa, identity management can be a challenge. In order to explore this topic, Hank spent some time talking with Dustin Puryear author, consultant, featured speaker at TechX World and owner of Puryear Information Technology, LLC. Dustin specializes in solving integration challenges related to mixed environments and has penned a few books on the topic. Two of his books: Best Practices for Managing Linux and UNIX Servers and "Spam Fighting and Email Security for the 21st Century" are available as free e-books on his site.

In this podcast (Part 1 of 2) Hank and Dustin explore issues related to implementing Identity Management solutions and focus primarily on the importance of policies. While the act of deployment is by default important, without proper policies even the best solution can fall short. In the next podcast the conversation will focus on implementation strategies. If you have particular questions you'd like answered in the next podcast, please feel free to comment below.