Port 25: The Open Source Community at Microsoft : Interop, Open Source

Team Microsoft Sings "Them Incubator Blues" at ApacheCon

Peter Galli — Thu, 12 Nov 2009 22:33:00 GMT

As you probably know, Microsoft was both a sponsor and active participant at ApacheCon 2009 in Oakland, California last week.

But what you might not know is that we also showed our lighter, more fun side, when we participated in the Lightning Talks, which were held on Thursday evening, November 5, during a reception with plenty of popcorn, beer and wine.

As my colleague Jas Sandhu noted in his blog about this - and where you can also find the song's lyrics - the talks are a lively, spontaneous ApacheCon tradition with speakers getting about 5 minutes to poke at each other, the projects, technology, community etc ... and have a bit of fun!

The singing and dancing Microsoft team was led by Kent Brown, the Product Manager for Windows Communication Foundation, who was not only the singer, songwriter and guitar player, but also the author of 'Them Incubator Blues,' which is very tongue-in-cheek and loosely based on his experience participating in the Stonehenge Project and working with the community.

He was joined by me, Microsoft's open source community manager; Tanya Young, our chief cat herder at the conference; and Jas Sandhu, a Senior Technical Evangelist. It was great fun and we hope you enjoy it!

Video courtesy of Mladen Turk from Red Hat.

The Business of Open Source

Sandy Gupta — Thu, 05 Nov 2009 15:25:00 GMT

I am blogging from ApacheCon here in Oakland, where the Apache Software Foundation its celebrating its 10^th anniversary. Congratulations to the Apache community and cheers to the next ten!

Our focus at Microsoft is to make Windows Server a platform choice for both closed source and open source solutions. Many of our customers who run open source on our server platforms pick open source built in the "Apache Way". And so, participation in the Apache communities is important for us and we continue to support the Apache community through our ongoing platinum sponsorship of both the ASF and ApacheCon.

Microsoft also has a sizeable contingent on the ground here at ApacheCon that is delivering technical talks, presenting at BarCamp Apache, giving Lightening talks, participating in MeetUps and, more importantly, learning more about the projects in the Apache community so we can identify opportunities for greater participation.

Yesterday, I participated on a business panel titled: "The Business of Open Source - Power, Prestige, and Propulsion," which was moderated by Sally Khudairi and included Hewlett-Packard's Scott Lamons, Progress Software's Debbie Moynihan, and RedMonk's Michael Coté. The panel was incredibly engaging - in fact, one of the best panels I have ever participated in.

It was very dynamic, there was great audience interaction, and a range of interesting topics were covered. There was consensus on how customers are taking a pragmatic approach and using a mix of closed and open source software based on the value it has for their business and not based on a religious choice.

We also talked about how there is a lot of open source happening on Windows Server, and there is an opportunity to improve on best practices/understanding of open source development on Windows Server and sandbox infrastructure for all Apache projects. We also had a vibrant discussion regarding release cycles: how can open communities make it more attractive to corporations to offer project manager time so as to help move the project along, keep to deadlines, etc. I hope to be able to participate in more panels like this as there is so much to discuss!

As you know, Microsoft is already participating in many ASF projects like HBase, Stonehenge, QPid, and POI, and we are giving demos around these projects at our booth here as well as on the Eclipse plug-ins for Azure and Silverlight announced at the recent Eclipse Summit.

Enabling our customers to run open source solutions on Windows Server is important for us. It is great to see many business groups in Microsoft are now participating in Open Source projects in areas where they see there is a common value for our customers.

Congratulations once again to the entire Apache Community for a great decade, and here's looking forward to the next one!

Microsoft and ApacheCon 2009

Peter Galli — Tue, 03 Nov 2009 19:35:00 GMT

This week marks the 10^th anniversary of the Apache Software Foundation, which is being celebrated at the annual ApacheCon U.S. event. Microsoft is proud to be a platinum sponsor of both the ASF and ApacheCon 2009.

The Apache community is an important one for Microsoft and, as ASF President Justin Erenkrantz noted recently, Microsoft is now contributing to at least four Apache projects: HBase, Stonehenge, QPid, and POI.

"This really continues the significant sea change from within the organization - Microsoft now isn't afraid of having their employees contribute to Apache projects on Microsoft's time. Committers from Microsoft sign the same legal agreements that we require from all of our contributors. Microsoft's involvement in these specific communities range from having their employees being core contributors driving the project, to having folks contributing patches or ideas on our mailing lists, to even commissioning a third-party to contribute to our project as a work-for-hire. In other words, Microsoft is now actively participating within Apache projects in a broad range of way," Erenkrantz said.

A number of Microsoft folks representing the Interoperability, App Plat, DPE, Open Source Technology Center and Platform Strategy teams will be on-site and participating in a number of events.

We will also have a booth and be demoing:

In addition, Sandy Gupta, the Director of Platform Strategy, will participate in a Business Panel today, titled: "The Business of Open Source - Power, Prestige, and Propulsion," while Kent Brown and David Ingham will be giving a technical session on Project Stonehenge and Qpid.

Garrett Serack from the Open Source Technology Center gave two BarCamp sessions earlier this week ("How the heck do I get help from Microsoft?" and "The Road Less Travelled" about the new CoApp he has developed,) while David gave a BarCamp presentation on AMQP and Qpid and Kent did one on Stonehenge.

David also led a MeetUp about Qpid on Windows on Tuesday night; while Kent and the team will give a Lightening Talk on Thursday.

We look forward to meeting those of you attending ApacheCon, and please feel free to stop by the booth.

Moving Forward With Eclipse

Sandy Gupta — Wed, 28 Oct 2009 01:00:00 GMT

It is great to see the work that started last year with the Eclipse Foundation at EclipseCon 2008 in Santa Clara, California continues its exciting journey.

Today at the Eclipse Summit in Germany, the Interoperability team at Microsoft announced a number of updates with our partners Tasktop and Soyatec. Microsoft is providing funding to these projects and architectural guidance.

Here are the goodies:

Modernizing Eclipse on the Windows7 platform
Extend the Eclipse Standard Widget Toolkit (SWT), to include the mapping of new features offered by Windows 7 and Windows Server 2008 R2 to have that "next generation experience"
Windows Azure Tools for Eclipse to enable PHP developers to build and deploy web applications targeting Windows Azure
Windows Azure SDK for Java developers to facilitate the use of Windows Azure component by Java developers
Eclipse Tools for Silverlight plug-in 1.0 is now available.

You can get more details in the blog from Vijay Rajagopalan in Microsoft Interoperability team, who is leading this effort.

Ian Skerrett's blog also mentions a recent survey targeted at Eclipse users, which shows that more than 64% of respondents used Windows Server or Client as their development platform for Eclipse. These users are a mix of Java, PHP, Ruby and C/C++ developers.

This is good data to show why we care to make Windows the platform for choice for these Eclipse users. These are our customers.

Microsoft already participates in projects under both the Eclipse Foundation and Apache Foundation. A couple of weeks back I blogged about our participation and code contribution to the Apache Qpid project. We will continue to serve our developer customers in the open source world.

Part 1: Lessons I Learned as a Project Manager Converting to Agile

saraford — Mon, 19 Oct 2009 19:16:00 GMT

Before I became the Program Manager for CodePlex.com, Microsoft's open source project hosting site, I worked on the Visual Studio team on four different product cycles. Since Visual Studio uses traditional Microsoft product lifecycle releases, I had to learn about Agile development alongside learning about open source development when I joined the CodePlex.com team. Making the switch from releasing every three years to every three weeks didn't happen overnight!

One of the things I discovered is that Agile is used a lot in open source communities. In fact, many of the people who I've worked with personally on learning Agile have strong roots in OSS. Also, the fact that Microsoft is starting to adopt Agile philosophies shows how the company is changing, becoming more transparent, finding more ways to connect with the community, and embracing other schools of thought. And this is why I am here, to be on the inside to push for these cultural changes within Microsoft.

Although Agile is the single greatest thing a team could do to significantly improve the user experience and quality of their website, I believe it can be quite challenging for anyone not in a developer role to get accustomed to. I hope that by sharing my experiences, I can help others in non-developer disciplines on an Agile team.

In this series of blog posts, you'll discover how I learned to program manage an Agile team after six years of waterfall (the traditional method of software development).

Three Major Takeaways

If I could go back in time, here are the three things I would tell myself about Agile.

1. Design and plan for the very next step.

It's not about reaching the moon, but getting out of your front door. Sure, you can design the perfect feature, but if it is going to take you six months to get there, it is useless to an Agile team. It's about designing the journey towards the perfect feature that matters.

2. Break down work into the smallest possible functional sets.

Adding work is fun and rewarding, but removing incomplete work due to a lack of development time is painful and risky. But, you can't deploy a half-written feature either. First, break down the work into the smallest pieces. Then, put together the smallest functional sets that have to be deployed together for the feature to make sense. Your development team will tell you how many sets they can do per release.

3. Design and plan only 80% of the way.

Not designing the full 100% is a true blessing in disguise. Since you have another release right around the corner, you have the time to collect user feedback and incorporate it into the next design. Not only does this solve the remaining 20% (getting you closer to the perfect design with less cost), but also allows your customers to be virtual members of your team.

The Program Manager Release Cycle

To begin, here's an Agile release cycle from the point of view of the Program Manager. For simplicity, this illustration only depicts a single release cycle, without any overlap of previous or future cycles.

Ratings and Reviews: An Example

To further illustrate, consider the ratings and reviews feature on CodePlex.com as an example. Users can rate a release and write a review for projects on CodePlex.

One quick aside about ratings and reviews: CodePlex users rate an individual release instead the entire project. For example, consider Stephen King as an author. What does it mean for me to rate Stephen King 4 out of 5 stars? I find some of his books to be awful, like the Tommyknockers. I want those 7 hours of my life back. 1 out of 5 stars. Yet, for me, some of his other books are incredible, like The Dark Tower. 5 out of 5 stars. Hence, we allow users to rate an individual release to provide more relevant information to potential downloaders of the project.

Let's explore the ratings and reviews feature step-by-step in the Program Manager shoes.

1. Design Phase Part 1: Limit the scope to designing the minimum to make the feature useful and meaningful. For ratings and reviews, the feature must have the following:

a. User can rate a release. User can view the rating.

b. User can write a review. User can read the review.

c. User can sort by highest rated releases in project directory.

2. Design Phase Part 2: Bucket into smallest deployable functional sets. For me, personally, I use sticky notes to illustrate the "must have" pieces for each set.

a. Sticky Note #1: Rate releases / View rating

b. Sticky Note #2: Write a review / read review

c. Sticky Note #2: Sort by highest rated releases

3. Iteration Planning Meeting: To start the development cycle, meet with the development team to discuss costing.

a. In the case of ratings and reviews, my devs said they could do Sticky Notes #1 and #2, but the project directory sorting feature would have to wait for the next release.

4. Dev Cycle: Because the designs are closer to 1-page specifications rather than fully-documented implementations, questions will come up from the dev team. This is where you, the Program Manager, will:

a. Answer any questions about the missing 20% of the specifications / wireframes

b. "Course correct" (more on that later)

c. Add more feature work if time allows

5. Deployment: The release goes live. Now you can collect user feedback and incorporate it into the next development cycle.

a. The very first tweet I saw regarding the ratings and reviews feature was "Sara, is there a way to sort by highest rated?" Here, I was able to ask the user questions about how this feature should work to confirm our designs. Most of the time we don't inform users what's coming up next (we like surprises.) But in this case, it was pretty obvious.

Here's the visual representation of the ratings and reviews feature in the Program Management agile release cycle:

Conclusion of Part 1: The Agile Program Management Cycle

This concludes my first post on Program Managing an Agile team. Since I could go on endlessly writing about topics I'm passionate about (and those who know me will confirm this is not an exaggeration), I'm going to pause here to conclude this initial train of thought.

If you like what you see, let me know! And if you don't like what you see, please don't hesitate to let me know. Seriously, I love discussing my Love / Hate Relationship with Agile development, as depicted on my personal blog.

Up next: The concept of Course Correction.

Another Great Step Forward for Hyper-V, Red Hat Certification

hjanssen — Wed, 07 Oct 2009 17:06:00 GMT

Hello again! It has been a pretty busy couple of months for us, and I wanted to give you an update on what we've been doing.

We just completed the first step in another major milestone for Hyper-V. As you can read from Mike Neil's Blog, Microsoft and Red Hat just completed certification in each other's virtualization program.

This means that customers now can deploy Microsoft Windows Server and Red Hat Enterprise Linux and a range of select applications, virtualized on Red Hat and Microsoft hypervisor virtualization software, knowing that the solutions will be supported by both companies.

This again demonstrates Microsoft's commitment to its virtualization platform, and the Hyper-V team's continuous investment in interoperability and heterogeneity through the datacenter.

And I am very proud that my team has been working very hard on getting the Red Hat certification completed.

But, for my group, this is just the first step: we worked on and got certified for Hyper-V running Red Hat in emulated mode and, now, the next step for us is to get certification for enlightened mode, the mode where the guest OS is Hyper-V aware,and can thus access Hyper-V functionality directly.

So, what is needed for enlightened mode? Well, to get that step, we needed to get the Linux Integration Drivers submitted to the Linux kernel, and then we needed those drivers officially accepted in a mainline kernel.

This has now happened as well, as we have been accepted into the mainline kernel. We are in Linux Kernel release 2.6.32, and that release is currently going through development and testing. Once that one is final and officially released, we can take the next step, which is to get those official Linux Integration Component drivers certified with Red Hat.

The timeline for that is not completely set in stone right now, and I do not know if there is an official expected release date for 2.6.32 as yet. But we should be able to move forward with the enlightened mode certification soon after the 2.6.32 final release.

And we are, of course, continuing to work on the Linux Integration components, adding new features and all that good stuff. But I will write more about those items in the near future.

New Bridge Broadens Java and .NET Interoperability

Peter Galli — Tue, 29 Sep 2009 15:31:00 GMT

Microsoft continues to deliver on its ongoing promise to build bridges between different technologies, and this week jointly announced with Noelios Technologies a new interoperability bridge between Java and .NET through REST.

The Microsoft Interoperability Technical Strategy team has been working with Noelios to build an extension to the Restlet Framework. As such, Noelios has released a new bridge for Java and .NET. It is shipping a new version of the Restlet open source project, a lightweight REST framework for Java.

This includes the Restlet Extension for ADO.NET Data Services, which makes it easier for Java developers to take advantage of ADO.NET Data Services, a set of recently added .NET Framework features that provides a simple way to expose a wide range of data sources, such as relational databases, XML files, and so on, through a RESTful service interface.

This announcement is yet another example of Microsoft's continued commitment to openness and interoperability, and demonstrates the ever increasing use of web standards in our technologies.

Formerly known as Project Astoria, the ADO.NET Data Services defines a flexible addressing and query interface using a URL convention, and supports the usual resource manipulation methods for data sources, including the full range of Create, Read, Update, and Delete operations.

Microsoft Visual Studio 2008 SP1 and the upcoming Visual Studio 2010 fully support ADO.NET Data Services, including the capability to create and consume data services directly from the development environment.

You can find all the technical details of this announcement on the Interoperability team blog here, as well as on the Noelis blog. You can also find a tutorial on this here.

GroundWork Open Source Joins Microsoft's System Center Alliance

Peter Galli — Thu, 24 Sep 2009 13:31:00 GMT

GroundWork Open Source, Inc., a commercial open source company that produces network management software, last week announced the availability of the GroundWork Connector for Microsoft System Center Operations Manager.

The company has also become a member of the System Center Alliance.

GroundWork Monitor, which already has more than 1,500 plugins available, integrates with System Center Operations Manager and extends monitoring and management coverage to non-Windows systems, applications and devices.

The new GroundWork Connector pulls information from System Center Operations Manager and displays it within GroundWork Monitor Enterprise, giving customers a deeper visibility into the availability and performance of all critical infrastructures on a single console. The connector gives insight into applications, databases, virtual machines and network devices that may be running on Linux, Unix, Windows or embedded operating systems.

I talked to David Dennis, the company's senior director of marketing and business development this week about the move, which he feels is a great follow-up to the release of the System Center Cross Platform extensions earlier this year.

That release broke new ground for using System Center in heterogeneous environments. "In the field, we have more and more users asking about how they can integrate the management of Windows with open source tools for managing network infrastructure, Unix, Linux, and the applications that run on top of them," he told me.

The dialog also no longer seems to be about choice between Windows or Open Source but rather "I want both - now how do I make them work together," he says. Even though GroundWork Open Source is an open source company, about half of the operating systems managed by GroundWork Monitor are running Windows.

"The combination of System Center Operations Manager and GroundWork Monitor provides a full-featured alternative to traditional systems management frameworks, but with greater openness and at a much lower price point," Dennis says.

Zend Launches Open Source Initiative to Drive Cloud Application Development

Peter Galli — Tue, 22 Sep 2009 14:00:00 GMT

Today, Zend Technologies announced the Simple API for Cloud Application Services project, which is a new open source initiative that allows developers to use common application services in the cloud, while enabling them to unlock value-added features available from individual providers.

This new project is designed to encourage widespread participation and contributions from the open source community, resulting in the availability of Simple Cloud API adapters for virtually all major cloud providers.

Zend, Microsoft, IBM, Nirvanix, Rackspace and GoGrid are all co-founding contributors to this community project, which aims to facilitate the development of cloud applications that can access services on all major cloud platforms and whose initial goal is to provide a set of programming interfaces for PHP developers to facilitate the development of applications that have basic cloud storage needs.

The first deliverables will include interfaces for file storage, document database, and simple queue services from platforms like Amazon Web Services, Windows Azure, Nirvanix Storage Delivery Network and Rackspace Cloud Files, allowing developers to deploy software applications to access services in these environments without making time consuming and expensive changes to their source code.

As Andi Gutmans, the CEO at Zend Technologies, notes in the press release announcing the project, "cloud computing offers irresistible value to enterprises of all sizes, but the lack of portability across cloud application services for even the most basic operations has been an impediment to broader adoption of cloud services."

An initial Simple Cloud API proposal and reference implementation is already available now for community review and participation, while a technology preview of the PHP client libraries for Windows Azure can be found here.

Microsoft is also contributing Simple Cloud API adapters, along with the official PHP client libraries for Windows Azure storage, to future versions of Zend Framework. These adapters will allow applications to take advantage of many Windows Azure features through the Simple Cloud API interface, while Microsoft's client libraries will put Windows Azure innovations, such as transaction and partial upload support, at the fingertips of cloud application developers.

"The Simple Cloud API is an example of Microsoft's continued investment in the openness and interoperability of its platform. We're excited to see how this project will foster adoption of cloud computing platforms by PHP developers and hope that many of these developers are encouraged to use Windows Azure," Doug Hauger, the General Manager for Windows Azure, notes in the press release.

Microsoft's involvement started a few months ago, through our work with Real Dolmen on a Windows Azure SDK for PHP developers. This SDK has been submitted to the Zend Framework, and it now forms the basis of Microsoft's contribution to the Simple Cloud API project.

As Vijay Rajagopalan, a Principal Architect at Microsoft, notes in his blog, the Zend Adapter for Windows Azure will leverage Microsoft's contribution. PHP developers will now be able to program against Windows Azure - in a way that is consistent with other cloud platforms - by tapping into the main features of Window Azure Storage.

Those PHP developers who need to use specific Windows Azure features not included in the scope of the Simple Cloup API (like transaction), will be able to combine the Zend Cloud Adapter with the dedicated Windows Azure SDK for PHP.

"This will allow developers to use common application services in the cloud, while enabling them to unlock value-added features available from individual providers. Simple API for Cloud also gives PHP developers more choices, and this is a great opportunity for them to think about using Windows Azure," he says.

Tuxera Signs File System IP Agreement with Microsoft

Peter Galli — Wed, 26 Aug 2009 12:21:00 GMT

Tuxera, a Finnish company that develops NTFS drivers, has entered into a file system IP agreement with Microsoft in the area of data portability for devices

The agreement gives Tuxera access to the exFAT specifications and source code, as well as testing and verification tools, and the company will develop exFAT drivers for host device manufacturers. The company will also join the Interoperability Vendor Alliance.

While this is also the first-ever file system IP agreement that Microsoft has signed with an independent software vendor, Tuxera joins a growing number of companies entering into the exFAT Program.

The Extended File Allocation Table (exFAT) is a new file system that is better adapted to the growing needs of mobile personal storage. It allows a common file system to be used across all platform and devices that implement exFAT, which allows for data portability.

It also handles not only large files, such as those used for media storage, but also enables seamless interoperability between desktop PCs and devices like portable media devices, cameras or even kitchen devices so that files can easily be copied between desktop and device.

"Adding exFAT into our product portfolio is the logical step to help our customers to solve their interoperable file systems needs. Microsoft supports exFAT as the first-choice for many Windows interoperability needs including flash memories in consumer devices. exFAT is for example an integral part of SDXC formats, and the driver is good for flash drives, including devices that use SDXC cards," says Tuxera CTO Szabolcs Szakacsits

For his part, Mikko Välimäki, the company's CEO, believes that his company's ongoing collaboration with Microsoft ensures seamless file system interoperability into the future.

"We were the first to offer exFAT drivers based on our first-hand access to specifications and source code.We are also licensing exFAT drivers to device manufacturers, we are joining Microsoft's partner program, and have also announced a new product: Tuxera exFAT for Embedded Systems, which will be first available for Linux, but we can also port the product to proprietary platforms," he says.

The IP agreement the company has signed with Microsoft is consistent with Tuxera's business model, which is to integrate and license commercial, high-performance versions of its file system drivers to customers, as well as to provide support, maintenance, and product updates, Välimäki says.

The PHP Toolkit for ADO.NET Data Services

Peter Galli — Fri, 21 Aug 2009 11:00:00 GMT

Today, the DPE Interop team at Microsoft released a new open source project that bridges PHP and .NET using REST.

Specifically, the PHP Toolkit for ADO.NET Data Services was made available, which makes it easier for PHP developers to take advantage of the ADO.NET Data Services, a set of features recently added to the .NET Framework.

The PHP Toolkit for ADO.NET Data Services is an open source project funded by Microsoft and developed by Persistent Systems. It is available on CodePlex today.

These services, which were previously referred to by the codename Project Astoria, expose a wide range of data sources through a RESTful service interface.

There is full support for ADO.NET Data Services in Visual Studio 2008 SP1 as well as in the upcoming Visual Studio 2010, which includes direct support for both creating and consuming data services directly from the development environment.

Read more about all this on the Interoperability blog, and watch the Channel 9 video interview with Claudio Caldato, Senior Program Manager in the Interoperability Technical Strategy team, and Pablo Castro, software architect of ADO.NET Data Services.

This is yet another example of Microsoft's continued commitment to openness and interoperability as well as of its embrace of Web standards in its technologies.

Releasing the Linux Integration Component Drivers...

Sam Ramji — Fri, 24 Jul 2009 01:38:00 GMT

Microsoft on Monday contributed the Linux Integration Component drivers to the Linux community for the reasons stated in our release. Microsoft chose the GPLv2 license for the mutual benefit of our customers, partners, the community, and Microsoft.

Microsoft's decision was not based on any perceived obligations tied to the GPLv2 license. For business reasons and for customers, we determined it was beneficial to release the drivers to the kernel community under the GPLv2 license through a process that involved working closely with Greg Kroah-Hartman, who helped us understand the community norms and licensing options surrounding the drivers.

The primary reason we made this determination in this case is because GPLv2 is the preferred license required by the Linux community for their broad acceptance and engagement. For us to participate in the Linux Driver Project, GPLv2 was the best option that allowed us to enjoy the tremendous offer of community support. The community's response even within a few hours of posting the code was welcoming and we appreciate it greatly.

We arrived at the decision to release the drivers to the community under the GPLv2 through this process. Both Greg K-H and Jim Zemlin of the Linux Foundation have reiterated that this is the same process that other companies follow when deciding how to release new device drivers to the Linux community.

We are looking forward to the positive collaboration and acceptance that has marked the vast majority of our interactions with customers and community members regarding this important project.

Cheers,

Sam

Updated 7/25/2009 @ 11:54 AM Pacific: Dave Roberts of Vyatta posted a blog entry rebutting recent cloims that we were accused of a licensing violation with some detail on the technical issues.

Introduction to the Linux Integration Components

hjanssen — Wed, 22 Jul 2009 19:02:00 GMT

Well, here is blog number two. The initial shock has worn off a bit I hope.

The feedback I have received so far has been pretty positive. This really all started in October of 2008 in a meeting with Mike Neil (GM of Hyper-V) and Tom and myself from the Open Source Technology Center (OSTC) at Microsoft.

In that meeting I proposed to Open Source the Linux Integration Components and contribute them to the Linux Kernel. And, secondly, to have the OSTC continue contributing to these IC's after they made it to the Linux Kernel. Well after some discussion, we all agreed that this was the right thing to do.

And so the whole process started inside of Microsoft. Hey, what can I say, we like to push the envelope a bit here at the OSTC, and we have a reputation to uphold!

Before I go on, I again wanted to thank the Kernel community (specifically Greg Kroah-Hartman) in helping us with explaining and guiding us through community process. It gave us a very nice jumpstart to get all of this going, and provided the groundwork for a good working relationship with the community.

I have also seen a few patches already submitted by community members, which is excellent! (Moritz Muehlenhoff gets major kudos for the first community contributed patch ) I will start submitting patches myself next week once the initial submission has stabilized a bit.

It is my plan to use the kernel as my primary development area, and of course I will continue to provide Greg with my patches. My first step is to clean up the code to make sure it fulfills all Kernel coding standards and requirements.

So, here is blog number two: what are the Linux Integration Components?

1. Overview of Linux VM with ICs

Linux Integration Components(IC) take advantages of the VMBUS and synthetic devices provided in Hyper-V to enhance the performance and usability of Linux guests running on Windows servers.

Figure: Conceptual Architecture overview of Linux guest & Hyper-V. Linux IC modules are painted in yellow color.

Glossary:

VSP: Virtualization Service Provider.

VSC: Virtualization Service Client.

VMBus: Data channel between VSP and VSC.

2. Linux IC modules -- VMBus and VSCs

Communication with parent partition is done through Linux VMBus
VSCs are the Linux drivers for synthetic devices (SCSI, IDE, and Ethernet) provided by Hyper-V.
They translate between Linux I/O requests and Hyper-V VSC commands
Devices are registered with Linux Driver Model (LDM)
Every VSC module contains two portions:
o Driver Interface Mapper (DIM): Released as open source

This portion of the VSC component interacts with the Linux kernel like a regular Linux device driver.

o VSC Core: Released as Open Source

The core portion of the VSC module is implemented based on the protocol of the corresponding VSP at Hyper-V host. The VSC core interacts with VSP via the VMBus interface.

3. Descriptions for each Linux IC module

3.1 VMBus driver (hv_vmbus.c)

The VMBus driver is a Linux kernel module. It provides both a lightweight bus driver and library functionality. As a bus driver, it registers with Linux Driver Model framework (LDM) to provide simple bus and device integration and device tree integration (sysfs). As a library, it implements the VMBus channel protocol and provide an abstraction of channel to its clients (Disk and Network VSCs).

3.2 StorVSC driver (hv_storvsc.c)

The Storage VSC interacts with the Windows Storage VSP. The "wire" protocol defined by the storage VSP determines how a VSC interacts with it. The Linux Storage VSC (LSVSC) basically abstracts the Linux I/O stack from needing to understand the Storage VSP's protocol. At the upper-edge of the LSVSC, it talks to the Linux SCSI subsystem. The Linux SCSI subsystem sees the LSVSC as a SCSI low-level driver (LLD) in Linux parlance. It passes SCSI requests (scsi_cmnd) to LSVSC which in turn converts them into the "wire" format understood by the Windows Storage VSP (VSTOR_PACKET). The bottom-edge of the LSVSC talks to Linux VMBus (LVMBUS) which in turn talks to the Windows VMBus to route the packets to the Storage VSP.

3.3 BlkVSC driver (hv_blkvsc.c)

BlkVSC (BlockVSC) supports "fast boot" and fast access to IDE disks. To enable enlightened IDE support for enhancing the performance of Linux when virtualized on Windows, a separate BlockVSC component is used as a Linux block device driver. Like StorVSC, the BlockVSC component is comprised of an upper edge wrapper that interfaces with the Linux block layer and a lower-edge through the infrastructure modules. The infrastructure modules with Hyper-V through the Linux VMBus.

3.4 NetVSC driver (hv_netvsc.c)

The network VSC send and receive network traffic between a Linux guest and Hyper-V host which has direct connection to physical network. The mechanism that this is used to accomplish is the Remote NDIS (RNDIS) protocol. Thus the communication that flows between the VSP and the VSC primarily happens over the RNDIS protocol which then is packaged and forwarded as payload over to the other side over NetVSP / VMBus protocol.

4. Linux IC's, Location in the Kernel tree

Hopefully you now have a better idea what they are. But where in the kernel tree can you find them?

Well, you can find sources in linux-next tree in /drivers/staging/hv directory.

And the git repository you can find them in right now is:

git://git.kernel.org/pub/scm/linux/kernel/git/sfr/linux-next.git

Or give this command (assuming your system is set up correctly) to download this repository to your machine:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/sfr/linux-next.git <your local name>

Since the IC's are part of the kernel now, we follow the normal community process of getting this all migrated into Linus mainline kernel.

The Microsoft Live Services Plug-in for Moodle Debuts

Peter Galli — Tue, 21 Jul 2009 11:15:00 GMT

Today, Microsoft announced the Live Services Plug-in for Moodle, a free download released under the General Public License v2 that integrates Microsoft's Live@edu services such as email, calendar, instant messaging and search directly into the Moodle experience.

What's even better is that this new, integrated experience is accessible via a single sign-on, which lets teachers and students access the resources and services they need to efficiently communicate, collaborate and learn.

Moodle is a free open source course management system that teachers use to create online learning websites for their classes, and has some 30 million users in 207 countries.

The plug-in and its feature set was designed as a result of extensive feedback from teachers and institutional IT leaders, and licensed in a way that is consistent with the practices of the open source community - freely under the GPL v2.

The news of the release of the Live Services Plug-in for Moodle under GPL v2 follows hot on the heels of Microsoft's release yesterday of 20,000 lines of device driver code to the Linux community under GPL v2.

This means that teachers and institutions can download the plug-in under a widely used open source license agreement and under the same terms that Moodle itself is licensed.

This approach underscores Microsoft's commitment to interoperability and open standards, as well as to collaboration so as to help customers, partners, educators and students across the world be successful in a heterogeneous technology world.

With the Live Services Plug-in, educators can email class notes and lecture slides to everyone in the class as well as send alerts regarding homework assignments or quizzes - all from within the same environment.

Students can also utilize Bing for search, check their calendar, send an email or just an instant message - without having to manage multiple accounts in multiple systems. They can do it all right within Moodle. They can also check unread emails using advanced features like keyboard shortcuts to check email quickly for example between class periods or just before lectures start.

The Microsoft Live Services Plug-in for Moodle will be part of a growing collection of solutions available from the Microsoft Education Labs.

For more on this news, you can read the blog from L. Michael Golden, Corporate Vice President of Microsoft's Education Products Group, as well as what Moodle founder Martin Dougiamas has to say about the plug-in.

More on the Hyper-V Linux Integration Components

hjanssen — Mon, 20 Jul 2009 20:24:00 GMT

Well, there is no easy way to say this, so I am simply going to start this blog with the following line.

Microsoft just submitted source code for the Hyper-V Linux Integration Components to the Linux Kernel Community Under GPL v2.

Well, there's a conversation starter! Are you still all sitting in your chairs???

Let me summarize:

Yes, our device driver code was released directly to the Linux Kernel
We released the code under GPL v2
We are working with Greg Kroah-Hartman so it is ready for the next release of the Linux Kernel, version 2.6.32
We will continue to update the driver code to enhance interoperability on an ongoing basis, but it's our hope that other developers in the community will find the code useful and worthy of collaboration.

Fallen off your chair yet?

Microsoft developed the Linux device drivers to enhance the performance of Linux when virtualized on Windows Server 2008 Hyper-V. My team and I were responsible for testing and validating the driver components that were contributed for this first release.

Now, my team and I will be responsible for further developing this code going forward. (Yes, that does mean that I have gone back to leverage my very early roots as a Kernel programmer. Let the world be warned!!!!). Haiyang Zhang has been working on this code with me, and he will continue to work with me on this going forward.

When I joined Microsoft three years ago, the primary reason was to put my money where my mouth was. You see complaining about something is easy, but it becomes a little more complicated when somebody offers you the opportunity to be part of helping change what you have complained about.

So, three years after taking the job that made me put my money where my mouth was (and still often is!), I for one am EXTREMELY happy to see one of the most significant fruits of our work here in the Microsoft Open Source Technology Center (OSTC). But I have to say, even I would have been hard-pressed to think three years ago that we would consider contributing to the Linux Kernel.

As you know, two years ago Microsoft announced a partnership with Novell, and Tom Hanrahan ran the lab on a day to day basis till about 9 months ago. Since then I have had the pleasure of running the technical side of the execution of that lab under Tom Hanrahan for the OSTC. One of the primary tasks for that lab is to make sure Windows runs well on top of XEN and Linux runs well on top of Hyper-V, and we do this in very close cooperation with Novell.

We do most of this work as an extension to Mike Neil's Hyper-V team.

As part of this, we were asked to help develop and maintain a crucial part of this work called the Linux Integration Components. This code is designed so that Linux can run in an "enlightened mode" on top of Hyper-V (enlightened mode is roughly the Hyper-V equivalent of "paravirtualized mode" for the Xen hypervisor). Without this driver code, Linux can run on top of Windows, but without the same high performance levels. It is this device driver code that we are releasing today, directly to the Linux Kernel.

We're not talking a few hundred lines of code here; we're talking about roughly 20,000 lines of code.

Is this a Dump and Run from Microsoft? Absolutely not! We plan to enhance the functionality of this code, and we will continue to work with the Linux Community to support the drivers and to ensure continued interoperability.

As you can imagine, this was the result of a lot of hard work: Hiyang Zhang, who has been co-writing this code; Hashir Abdi, who has been testing all this stuff; as well as Vijay Tewari and Mike Sterling from the Hyper-V team who have been taking care of the Hyper-V side.

And last, but certainly not least, Greg Kroah-Hartman, who has been helping me to make all this code land in the right area in the kernel. He has patiently worked to help me correct my obvious mistakes and to get the code contributed into the kernel.

So where are we today? Well, Greg Kroah-Hartman will make the code visible to the outside world today. (For those who want to get a head start, the code will sit under <your kernel tree>/drivers/staging/hv). After it becomes visible, I will write a few more blogs this week that should help you to understand, build and run them.

The titles I am thinking for these blogs are:

Where do the Linux ICs reside in the kernel tree and how do I build them?

And

How do I install, configure and run the Linux IC's?

I had almost forgotten how wrapped up you can be once you start writing code again. So I have not gotten much sleep this past week, but it has been a joy to get back into coding again!