Port 25: The Open Source Community at Microsoft : OSCON

Releasing the Linux Integration Component Drivers...

Sam Ramji — Fri, 24 Jul 2009 01:38:00 GMT

Microsoft on Monday contributed the Linux Integration Component drivers to the Linux community for the reasons stated in our release. Microsoft chose the GPLv2 license for the mutual benefit of our customers, partners, the community, and Microsoft.

Microsoft's decision was not based on any perceived obligations tied to the GPLv2 license. For business reasons and for customers, we determined it was beneficial to release the drivers to the kernel community under the GPLv2 license through a process that involved working closely with Greg Kroah-Hartman, who helped us understand the community norms and licensing options surrounding the drivers.

The primary reason we made this determination in this case is because GPLv2 is the preferred license required by the Linux community for their broad acceptance and engagement. For us to participate in the Linux Driver Project, GPLv2 was the best option that allowed us to enjoy the tremendous offer of community support. The community's response even within a few hours of posting the code was welcoming and we appreciate it greatly.

We arrived at the decision to release the drivers to the community under the GPLv2 through this process. Both Greg K-H and Jim Zemlin of the Linux Foundation have reiterated that this is the same process that other companies follow when deciding how to release new device drivers to the Linux community.

We are looking forward to the positive collaboration and acceptance that has marked the vast majority of our interactions with customers and community members regarding this important project.

Cheers,

Sam

Updated 7/25/2009 @ 11:54 AM Pacific: Dave Roberts of Vyatta posted a blog entry rebutting recent cloims that we were accused of a licensing violation with some detail on the technical issues.

Introduction to the Linux Integration Components

hjanssen — Wed, 22 Jul 2009 19:02:00 GMT

Well, here is blog number two. The initial shock has worn off a bit I hope.

The feedback I have received so far has been pretty positive. This really all started in October of 2008 in a meeting with Mike Neil (GM of Hyper-V) and Tom and myself from the Open Source Technology Center (OSTC) at Microsoft.

In that meeting I proposed to Open Source the Linux Integration Components and contribute them to the Linux Kernel. And, secondly, to have the OSTC continue contributing to these IC's after they made it to the Linux Kernel. Well after some discussion, we all agreed that this was the right thing to do.

And so the whole process started inside of Microsoft. Hey, what can I say, we like to push the envelope a bit here at the OSTC, and we have a reputation to uphold!

Before I go on, I again wanted to thank the Kernel community (specifically Greg Kroah-Hartman) in helping us with explaining and guiding us through community process. It gave us a very nice jumpstart to get all of this going, and provided the groundwork for a good working relationship with the community.

I have also seen a few patches already submitted by community members, which is excellent! (Moritz Muehlenhoff gets major kudos for the first community contributed patch ) I will start submitting patches myself next week once the initial submission has stabilized a bit.

It is my plan to use the kernel as my primary development area, and of course I will continue to provide Greg with my patches. My first step is to clean up the code to make sure it fulfills all Kernel coding standards and requirements.

So, here is blog number two: what are the Linux Integration Components?

1. Overview of Linux VM with ICs

Linux Integration Components(IC) take advantages of the VMBUS and synthetic devices provided in Hyper-V to enhance the performance and usability of Linux guests running on Windows servers.

Figure: Conceptual Architecture overview of Linux guest & Hyper-V. Linux IC modules are painted in yellow color.

Glossary:

VSP: Virtualization Service Provider.

VSC: Virtualization Service Client.

VMBus: Data channel between VSP and VSC.

2. Linux IC modules -- VMBus and VSCs

Communication with parent partition is done through Linux VMBus
VSCs are the Linux drivers for synthetic devices (SCSI, IDE, and Ethernet) provided by Hyper-V.
They translate between Linux I/O requests and Hyper-V VSC commands
Devices are registered with Linux Driver Model (LDM)
Every VSC module contains two portions:
o Driver Interface Mapper (DIM): Released as open source

This portion of the VSC component interacts with the Linux kernel like a regular Linux device driver.

o VSC Core: Released as Open Source

The core portion of the VSC module is implemented based on the protocol of the corresponding VSP at Hyper-V host. The VSC core interacts with VSP via the VMBus interface.

3. Descriptions for each Linux IC module

3.1 VMBus driver (hv_vmbus.c)

The VMBus driver is a Linux kernel module. It provides both a lightweight bus driver and library functionality. As a bus driver, it registers with Linux Driver Model framework (LDM) to provide simple bus and device integration and device tree integration (sysfs). As a library, it implements the VMBus channel protocol and provide an abstraction of channel to its clients (Disk and Network VSCs).

3.2 StorVSC driver (hv_storvsc.c)

The Storage VSC interacts with the Windows Storage VSP. The "wire" protocol defined by the storage VSP determines how a VSC interacts with it. The Linux Storage VSC (LSVSC) basically abstracts the Linux I/O stack from needing to understand the Storage VSP's protocol. At the upper-edge of the LSVSC, it talks to the Linux SCSI subsystem. The Linux SCSI subsystem sees the LSVSC as a SCSI low-level driver (LLD) in Linux parlance. It passes SCSI requests (scsi_cmnd) to LSVSC which in turn converts them into the "wire" format understood by the Windows Storage VSP (VSTOR_PACKET). The bottom-edge of the LSVSC talks to Linux VMBus (LVMBUS) which in turn talks to the Windows VMBus to route the packets to the Storage VSP.

3.3 BlkVSC driver (hv_blkvsc.c)

BlkVSC (BlockVSC) supports "fast boot" and fast access to IDE disks. To enable enlightened IDE support for enhancing the performance of Linux when virtualized on Windows, a separate BlockVSC component is used as a Linux block device driver. Like StorVSC, the BlockVSC component is comprised of an upper edge wrapper that interfaces with the Linux block layer and a lower-edge through the infrastructure modules. The infrastructure modules with Hyper-V through the Linux VMBus.

3.4 NetVSC driver (hv_netvsc.c)

The network VSC send and receive network traffic between a Linux guest and Hyper-V host which has direct connection to physical network. The mechanism that this is used to accomplish is the Remote NDIS (RNDIS) protocol. Thus the communication that flows between the VSP and the VSC primarily happens over the RNDIS protocol which then is packaged and forwarded as payload over to the other side over NetVSP / VMBus protocol.

4. Linux IC's, Location in the Kernel tree

Hopefully you now have a better idea what they are. But where in the kernel tree can you find them?

Well, you can find sources in linux-next tree in /drivers/staging/hv directory.

And the git repository you can find them in right now is:

git://git.kernel.org/pub/scm/linux/kernel/git/sfr/linux-next.git

Or give this command (assuming your system is set up correctly) to download this repository to your machine:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/sfr/linux-next.git <your local name>

Since the IC's are part of the kernel now, we follow the normal community process of getting this all migrated into Linus mainline kernel.

Getting Ready for OSCON 2009

Peter Galli — Tue, 21 Jul 2009 23:13:00 GMT

I'm back at the San Jose at the McEnery Convention Center after attending the Community Leadership Summit this weekend, where we are setting up the Microsoft booth in preparation for the formal opening of the O'Reilly Open Source Convention (OSCON) tomorrow.

In addition to our booth presence, where a number of product groups will be represented, including folk from the Education, External Research, Open Source Technology Center, Interoperability and CodePlex parts of the company, all of whom will be giving technical demos and chatting to attendees, including the Hyper-V Linux Integration Components and the Microsoft Live Services Plug-in for Moodle, both of which are licensed under the GPL v2

The booth will offer a lounge area, where attendees can play any of a number of Xbox games, including Guitar Hero and Halo 3.

In addition to having a booth on the show floor, Tony Hey, the Corporate Vice President for Microsoft External Research, will deliver a keynote address on Thursday July 23, titled "Open Tools and Services on Microsoft Platforms," while Erik Meijer, one of Microsoft's Principal Architects, will also give a keynote talk on Friday July 24 and titled "Fundamentalist Functional Programming."

Following his keynote, Erik is also presenting on using the LiveLabs Reactive Framework to democratize the cloud.

Vijay Rajagopalan, a Principal Architect in Microsoft's Interoperability group, is also giving a talk on Wednesday July 22 in the Product and Services Track, titled "Interoperability - Build Mission Critical Applications in PHP, Ruby, Java and Eclipse Using Microsoft Software & Services."

We are also hosting a rountable panel that will look at Microsoft and PHP, and include IDC analyst Al Hilwa, Garret Serack from the OSTC, Aaron Fulkerson of MindTouch and John Keagy of GoGrid.

So, feel free tostop by the booth, We look forward to talking to you!

The Microsoft Live Services Plug-in for Moodle Debuts

Peter Galli — Tue, 21 Jul 2009 11:15:00 GMT

Today, Microsoft announced the Live Services Plug-in for Moodle, a free download released under the General Public License v2 that integrates Microsoft's Live@edu services such as email, calendar, instant messaging and search directly into the Moodle experience.

What's even better is that this new, integrated experience is accessible via a single sign-on, which lets teachers and students access the resources and services they need to efficiently communicate, collaborate and learn.

Moodle is a free open source course management system that teachers use to create online learning websites for their classes, and has some 30 million users in 207 countries.

The plug-in and its feature set was designed as a result of extensive feedback from teachers and institutional IT leaders, and licensed in a way that is consistent with the practices of the open source community - freely under the GPL v2.

The news of the release of the Live Services Plug-in for Moodle under GPL v2 follows hot on the heels of Microsoft's release yesterday of 20,000 lines of device driver code to the Linux community under GPL v2.

This means that teachers and institutions can download the plug-in under a widely used open source license agreement and under the same terms that Moodle itself is licensed.

This approach underscores Microsoft's commitment to interoperability and open standards, as well as to collaboration so as to help customers, partners, educators and students across the world be successful in a heterogeneous technology world.

With the Live Services Plug-in, educators can email class notes and lecture slides to everyone in the class as well as send alerts regarding homework assignments or quizzes - all from within the same environment.

Students can also utilize Bing for search, check their calendar, send an email or just an instant message - without having to manage multiple accounts in multiple systems. They can do it all right within Moodle. They can also check unread emails using advanced features like keyboard shortcuts to check email quickly for example between class periods or just before lectures start.

The Microsoft Live Services Plug-in for Moodle will be part of a growing collection of solutions available from the Microsoft Education Labs.

For more on this news, you can read the blog from L. Michael Golden, Corporate Vice President of Microsoft's Education Products Group, as well as what Moodle founder Martin Dougiamas has to say about the plug-in.

OSCON 2009

Peter Galli — Tue, 30 Jun 2009 00:29:00 GMT

As Microsoft continues to support and participate in open source communities, the company is again a proud sponsor of the annual O'Reilly Open Source Convention (OSCON), which is being held in San Jose from July 20 to July 24.

In addition to having a booth on the show floor, Tony Hey, the Corporate Vice President for Microsoft External Research, will deliver a keynote address on Thursday July 23, titled "Open Tools and Services on Microsoft Platforms," which will examine the far-reaching changes open research tools and services will have to support every stage of the research process.

Erik Meijer, one of Microsoft's Principal Architects, will also give a keynote talk on Friday July 24 and titled "Fundamentalist Functional Programming."

His talk will argue that fundamentalist functional programming - that is, radically eliminating all side effects from programming languages, including strict evaluation - is what it takes to conquer the concurrency and parallelism dragon.

Following his keynote, Erik is also presenting on using the LiveLabs Reactive Framework to democratize the cloud.

During his presentation, Vijay will talk about how Microsoft has delivered multiple technologies that focus on interoperability with non-Microsoft and Open Source technologies. He will also show how developers can, today, use Eclipse tools to build Silverlight applications that run on PCs and Macs, as well as how they can develop using combinations of PHP, Java and Ruby in addition to the standard Microsoft languages.

In addition to all the talking, we also expect to do a lot of "showing," and a number of product groups will be represented in the Microsoft booth, including folk from the Education, External Research, Open Source Technology Center, Interoperability and CodePlex parts of the company, all of whom will be giving technical demos and chatting to attendees..

An analyst/partner roundtable discussion is also on the cards, as is a broader interoperability discussion. You won't want to miss any of it.

history.forward()

Sam Ramji — Fri, 25 Jul 2008 08:16:00 GMT

I’m writing this from Portland, Oregon where one of the world’s largest Open Source conferences is taking place: OSCON. This year’s event is focused on a theme of “ten years of open source,” referring to 1998 as the year that Eric S. Raymond, Danese Cooper, et al coined the term. The Day 1 keynote theme was the past, Day 2’s theme was the present, and Day 3 (today) is focused on the future.

In my keynote address this morning I’m announcing three areas of contribution:

PHP on IIS + SQL: Microsoft is contributing a patch to ADOdb, a popular data access layer for PHP used by many applications. The patch enables support for SQL Server through the new “native driver for PHP” built by the SQL Server team. ADOdb is licensed under the LGPL and BSD. This is our first code contribution to PHP community projects but will not be the last.

We have tested over 100 community PHP applications and found them to run on IIS with no changes required. Hank Janssen and Garrett Serack of the Open Source Software Lab at Microsoft have been championing this work from the beginning, and I thank them for it.

Open Specification Promise: Microsoft is putting a wide range of protocols that were formerly in the Communications Protocol Program under the Open Specification Promise (OSP). This guarantees their freedom from any patent claims from Microsoft now or in the future, and includes both Microsoft-developed and industry-developed protocols.

We have established a clarification to the OSP that guarantees developer rights to build software of any kind and for any purpose using these specifications, including commercial use.

I am grateful to Andy Oliver, the creator and maintainer of Apache POI, for contacting me back in June with a hope that Microsoft could supply the necessary rights for POI. These include: rights for Office Binary document formats; Open XML; and the right to intentionally subset, have partial implementations, or defects in implementation of these specification. Andy offered his thoughts here.

Apache Software Foundation: Microsoft is becoming a sponsor of the Apache Software Foundation (ASF). This sponsorship will enable the ASF to pay administrators and other support staff so that ASF developers can focus on writing great software.

Jim Jagielski, Chairman of the ASF, had this to say about the sponsorship:

"We thank Microsoft for their generous sponsorship that goes towards supporting The Apache Software Foundation and the over 60 top level projects in use and development within the ASF," said ASF Chairman Jim Jagielski. "The ASF Sponsorship program is an excellent way for companies and organizations to show their commitment and enthusiasm towards the ASF and The Apache Way, and helps to ensure that highly innovative, freely-available and community-based/consensus-developed software can continue to flourish and thrive within one of the most successful and respected communities in Open Source. Microsoft's sponsorship makes it clear that Microsoft 'gets it' regarding the ASF."

It’s critical to understand two things about our sponsorship of the ASF: what it is, and what it is not.

It is not a move away from IIS as Microsoft’s strategic web server technology. We have invested significantly in refactoring and adding new, state-of-the-art features to IIS, including support for PHP. We will continue to invest in IIS for the long term and are currently under way with development of IIS 8.

It is a strong endorsement of The Apache Way, and opens a new chapter in our relationship with the ASF. We have worked with Apache POI, Apache Axis2, Jakarta, and other projects in the last year, and we will continue our technical support and interoperability testing work for this open source software.

I offer my personal thanks in the learning process that has led to today’s announcements to Allison Randall, Jeremy Allison, Andrew Tridgell, Mike Schroepfer, Andi Gutmans, Wez Furlong, Andy Oliver, Jim Jagielski, Brian Behlendorf, Cliff Schmidt, Sally Khudairi, Gianugo Rabellino, and Justin Erenkrantz.

Cheers,

Sam

The OSP and You

Richard Wilder — Fri, 25 Jul 2008 08:15:00 GMT

I am the Associate General Counsel for Intellectual Property Policy at Microsoft, having joined the company about 9 months ago. My role is to work with a variety of constituencies inside the company and outside to help shape the approach we take to intellectual property. I am new to the company and cannot take credit for it, but am very pleased that in recent years, Microsoft has made progress in participating with open source communities. A part of that has been the implementation of the Open Specification Promise (OSP), which was launched in 2006. We think it is a simple and clear way to assure that the broadest audience of developers and customers working with commercial or open source software can implement specifications. We constantly listen to feedback from community representatives and respond to that feedback – through Q&A’s on the OSP page and directly to the community. Andy Oliver made some positive comments in this regard as recently as yesterday. When asked for clarification of the OSP with respect to the activities of Apache POI, we responded. The concerns were about implementations of specifications covered by the OSP that may be less than fully compliant – in particular due to implementation bugs. Such a situation is not explicitly covered by the OSP since it is meant to apply to a wide range of technologies and development models and it is simply not possible to address all specific situations in which it would apply. We addressed this situation in the following manner – and I apologize if the explanation is a bit technical, but I will try to avoid too much legal jargon.

The OSP says that it covers "any implementation to the extent it conforms to a Covered Specification" which addresses the heart of the conformance issue that was raised." To the extent it conforms" means that we do not require an implementation to be perfect; this can be because of implementation bugs or an intentional choice because the requirements of the particular implementation do not actually require full conformance. Under the OSP, implementations can be less than fully compliant. For example, a given implementation that takes a spreadsheet document, extracts information from it, and stores that information in a relational database might not comply with every required part of the spreadsheet document format but such an implementation would still be covered by the OSP. By way of comparison, other promises in the industry may require complete conformance for the promise to apply, and those normally require full compliance as a condition (see IBM's Interoperability Specifications Pledge). Some others make no statement about the subject at all, leaving it an open question as to whether full compliance is required. At Microsoft, we felt that unless we indicated that the OSP is more flexible, some might conservatively assume that complete compliance is required, so we included the “extent it conforms” language in the OSP. We chose to state explicitly that partially conformant implementations are covered, to the extent they are conformant in their individual implementation aspects.

As a result of this clarification, developers can have peace of mind that the specifications covered by the OSP, are, in fact, openly available without ambiguity. This is the kind of conversation and cooperation that marks our intentions with the open source community, and I look forward to continuing this dialogue into the future.

-Richard

Participating Actively

Bryan Kirschner — Wed, 23 Jul 2008 18:39:00 GMT

Now that we’ve had our first “participate” event in conjunction with OSCON here in Portland, I wanted to share a few thoughts. This was a great experience and a great event—or, really, two consecutive events, the morning case study discussion and the afternoon panel.

First I’ll talk about the case study and then build on comments from some folks who’ve “beaten me to the blog.” In the morning Karim Lakhani from Harvard led the group through a case study about a fast-growing company (Threadless t-shirts) built on community contribution and distributed innovation. This was basically like being in a Harvard Business School class with a bunch of super achievers, complete with questions and counter questions (John Wilbanks from Science Commons blogs about it here). Stepping back and taking a look at a whole bunch of concepts and practices that underlie open source in the software domain in another context (t-shirt design), IMO, really opened the floodgates on discussion—a discussion Karim (with regret) had to close as the buzz in the room kept right on going well over time and into lunch…

In the afternoon I was part of a panel discussion and Q and A that started back in the software domain specifically. The one thing I would definitely do differently is to couple the morning case study and the later panel discussion more tightly. Not everyone who could be part of one was part of the other this year, and the real “ah ha’s” for me came from being a part of both. Here’s what I took away overall.

I introduced the morning session by noting that we’re at the ten-year mark since the folks who founded the Open Source Initiative (OSI) rallied around the term “open source.” At last year’s OSCON, Bill Hilf announced we had launched http://Microsoft.com/opensource , our first public, official, company-wide statement of policy and strategy on OSS. So (I said): “If you look at that span of time from 1998 to 2007, no one can accuse us of being precipitous, and no one can flatter us for being first adopters.”
But there’s a benefit to being slow: other people don’t stand still stuff. That includes folks like Karim and another professor on our panel, Siobhan O’Mahony, doing research. I can’t emphasize enough the contributions their work and that of many others of their peers made to our first step in informing and building acceptance of that step into participation in 2007. We read it all.

And this is where I’ll offer a different perspective than Zack—in his blog he said he felt the afternoon session (on which I really appreciated his participation and contributions) he felt a bit like “it was outside looking in” on open source and “academic.” With regard to the first point, one of my goals for next year is definitely to figure out how we integrate the “inside look out” (at another domain) like we did in the morning. With regard to the latter, here’s the interesting thing to me: “academic” can be pejorative when it means “divorced from any substantive decision-making”—that is, you’re just studying for the sake of studying. And I can where Zack is coming from: MySQL is one of the oldest OSS-based businesses. Zack was quite clear he knows how they manage their dev process and a bunch of other things. Unlike the folks at Threadless and perhaps many younger OSS-based companies, Zack and MySQL’s leadership team don’t even have to wonder about what to do if they are offered a big contract or billion-dollar buy out from a big established vendor…they’ve been there, done that. I respect that.

But if like Zack (and Matt Asay, who couldn’t be at partcipate08…Matt, I’ve read your blogs for years, you’re a thoughtful guy, I would bet money you couldn’t help but love the morning session…save a date for 09!) you are encouraging Microsoft to make more code (or whole products) open source: on the Microsoft side “academic” insights are highly relevant and actionable. Siobhan almost literally wrote the book on how established companies work with foundations and communities. Karim’s understanding of distributed innovation spans from the early days of OSS’ popularity through Wikipedia and beyond (we learned on Monday that there is a vibrant online user innovation community around custom granola recipes…).

Their research and practitioners like Allison (and others) abstracting out how what-worked-in-her-experience might apply to another technology or audience are directly relevant to diverse Microsoft teams figuring out how to “go open” in ways that are sustainable because they engage a community and make business sense—there are some great examples (here and here and here). But if there’s one qualification for being the first person in the history of the universe with the title of “Director of Open Source Strategy at Microsoft” (…thanks Bill and Sam…) it is this: the humility to understand it would be foolish to try to figure out how to expand this list company-wide on our own, without learning from everyone who has gone before.

So here is the real a-ha for me: John Wilbanks’ job is a lot harder than mine. He is approaching the Science Commons domain with a far less robust body of knowledge and shared understanding across communities than we have in OSS. Some of that may be ten years of “open source” versus a shorter timeframe for applying these concepts to science—but what I tried to articulate at the end of the panel was this: I believe “open source” has achieved a fascinating and valuable thing. It has achieved a balance as an construct which is not just a reductive, narrow focus on source code licensing (which is a component) nor a vague, fuzzy, wishy-washy platitude or marketing slogan (which is a risk and something I know the OSI worries about). It has enough cohesion, flexibility, and surface tension to be something you can study scientifically and discuss with a shared understanding of how it relates to software or t-shirts or science, and have an intuitive “know-good-practices-when-you-see-them” dimension.

I think the OSI and other leaders in open source contributed to this by striving to maintain fidelity to a core set of values while being flexible rather than doctrinaire. And here at OSCON this strikes me: last year at OSCON 2007 Bill Hilf also announced we were submitting two Microsoft Shared Source licenses to the OSI for approval. This was a milestone I see as not just instrumentally useful to provide clarity to users of these licenses; I see it as fitting as a matter of respect and recognition. And this year we took another step forward with participate08 here at Tim O’Reilly and Allison Randal’s OSCON 2008. I see this as fitting not just instrumentally as a matter of convenience (--lots of the right people happen to be here--) but as a matter of respect and recognition. I hope to be back for participate09.

I am going to close this blog entry on that thought but for one picture that really is worth a thousand words. Once we get the notes and the whiteboard photos assembled I’ll share more about the discussion, but this image will stick with me a theme for why so many folks did come to think hard and contribute as a part of participate08—and why I am grateful they did:

(photo by James Duncan Davidson/O'Reilly Media)

Participate08

Bryan Kirschner — Fri, 18 Jul 2008 15:09:00 GMT

On July 21 I will have the honor and pleasure of being the sponsor, host, and an active participant in participate08. participate08 is a one-day summit held in coordination with the O'Reilly Open Source Conference(OSCON). It is designed to facilitate dialogue about open source and other collaborative communities and help explore opportunities for greater participation in the design, development, and deployment of software in the modern IT environment.

The reasons I think it is cool are mostly personal as well as professional. The work of Harvard’s Karim Lakhani (our facilitator in the morning and moderator in the afternoon) has been one of the biggest influences on my perspective on free and open source software (...that’s kind of a pun…). I haven’t been familiar with panelist Siobhan O’Mahony’s work quite as long, but she is one of, if not “the” leading researcher on how firms work with open source communities. Her work quite literally helps me figure out how to do my job. Panelist John Wilbanks runs the Science Commons project at Creative Commons, an endeavor I think has a good solid foundation in elements of brilliance. Speaking of which, Zack Urlocker is a super smart guy. And Allison Randal has her own standing tagline with me as “one of the most thoughtful people in FOSS.”

Sometimes we have to focus on what I’ll call day-to-day issues: like what if a Microsoft team releases an application under an open source license (the Ms-PL) without making the source code available? (The answer is: the team, whose disconnect with our policy was 100% accidental and unintended—stepped up to strongly affirm their commitment to OSS best practices and voluntarily released it with source code, to their great credit.) These are important. Most of the time (as in this case) things turn out positively. But participate08 is focused on the big picture, or macro level issues—the future of distributed innovation in software and beyond; being a part of that sort of discussion with folks like our panelists is just mind-blowingly cool.

In the morning, we’ll be holding a small group, facilitated “executive session”—in the afternoon, the panel will star in an open session where we hope to have a great dialogue among the panel—and with the audience. If you will be at OSCON I hope you’ll join us in E145 at 1:30 PM!

Microsoft & Sourceforge

jcannon — Thu, 05 Jun 2008 14:36:00 GMT

Over the past four years, we’ve been working with Sourceforge on a number of unique programs to help connect developers with the code and communities that interest them. In 2004, we started the Windows Installer XML (WiX) Toolkit project on Sourceforge, licensed under the Common Public License. WiX is a toolset that builds Windows installation packages from XML source code. In 2005, we started the open source ODF Converter project. Then, in 2007, we launched the ‘World of Choice’ destination to provide helpful connections to free community and software offerings available from Microsoft.

Developers have told us these interactions are important and that it’s important that we continue collaborating with open source communities. We understand that the language of that collaboration is code. As a result, I’m excited to share that this year, Microsoft is a Diamond sponsor of the Sourceforge Community Choice Awards (CCA), joining O’Reilly and the Linux Foundation in supporting the recognition of world-class open source developers and projects.

For the uninitiated, the CCAs are an annual appreciation of open source projects that allow any OSS project to be nominated and voted on by the Sourceforge community. This joint collaboration will result in two important additions to this year’s Community Choice Awards:

1. The CCAs are now open to open source projects that are hosted on Codeplex. This means that projects hosted on Codeplex, and licensed under the MS Public or Reciprocal License, are eligible for nomination, voting and recognition.

2. As a part of our sponsorship, we’ve also worked with Sourceforge to create a new category for ‘Best Project for Educators’: Whether you're working in grade school education, high school, or college, teaching is difficult. Open source can help! This award goes to the project that makes it easier to educate and share knowledge together.

This is an important program for Microsoft – we believe strongly in supporting developer choice and collaboration. Open source is all about choice and collaboration. I can’t think of a better venue to support and cheer the fantastic work that these individuals and communities do every day. Education is another important theme from Microsoft. Worldwide, access to knowledge is a serious social and economic issue. Technology can be a key to closing this gap. We want to recognize development work being done in this area.

So here’s how to get involved: If you’re a project contributor, maintainer, or user – visit http://sourceforge.net/community/cca08/ - Select ‘Nominate’ and get started recognizing your favorite OSS Projects. You can ‘Search’ for projects too – Sourceforge has developed a clever widget that returns results across forges. In the case below, I searched for ‘XNA’ and received multiple Codeplex results.

This summer, winners will be recognized at OSCON in Portland, OR. If you will be in town, you’re welcome to attend.

This is an exciting time to be a developer. Let the recognition begin.

Haskell in the Hallway: Sam Interviews Simon Peyton Jones

Sam Ramji — Wed, 26 Sep 2007 23:35:00 GMT

While at OSCON this year Sam and I got the chance to spend some time with Simon Peyton Jones, researcher at Microsoft Research in Cambridge, England and Honorary Professor of the Computing Science Department at Glasgow University. Earlier in the week Simon had presented a tutorial on Haskell titled: A Taste of Haskell at the O'Reilly event.

Taking advantage of the opportunity to have some time with Simon, we found a (sometimes) quiet hallway to talk about Haskell, functional programming and other topics.

Video: Haskell in the Hallway: Sam Interviews Simon Peyton Jones

Barton George interviews Sam Ramji

Sam Ramji — Thu, 30 Aug 2007 19:57:00 GMT

On the day before OSCON officially kicked off I was heading back from the Oregon Convention Center to downtown and ended up standing on the MAX next to someone who caught my attention for two reasons. First, he was the first person to tell me about the Tim O'Reilly/Eben Moglen conversation from earlier in the morning which I had missed. Second he had a cool flash memory microphone that he used to record podcasts on the fly. Turns out it was Baron George (http://www.blogs.sun.com/barton808/), Group Manager, Free and Open Source Software from Sun.

I had a nice chat with Barton (and we helped each other navigate the heavily under construction Portland downtown) and little did I know he also had a conversation with Sam Ramji that day. The podcast has been posted and you can download it here. Enjoy.

From Barton's Blog:

"Topics: Where the Open Source Software Lab fits within Microsoft; How big is Sam's group; When software technologies compete, you win; What reaction does he get when he turns up at FOSS events; Debating Eben Moglen at OSBC -- no one wants patent Armageddon; Is there a Wubuntu in the works?."

OSCON and Everything After

Bryan Kirschner — Wed, 29 Aug 2007 03:09:00 GMT

When I describe my job as “helping Microsoft and open source to grow together,” I get a broad range of reactions from people outside and inside of Microsoft. These reactions have included sentiments along the lines of “that must be tough,” or “you must be a glutton for punishment” on occasion.

After wrapping up a fairly momentous year* culminating in OSCON (see this and this), I thought the time was right to put some big-picture context around how I feel about my job.

The year 1995 was when we saw the first official public release (0.6.2) of the Apache server, and MySQL AB was founded.

The world was two years shy of the Debian Software Guidelines and three years away from the articulation of the Open Source Definition (OSD) they inspired.

The Open Source Technology Group (OSTG)—by virtue of operating both Sourceforge and Freshmeat today’s largest hoster of public open source project--was about to be founded.

We were at the very beginning of the growth of open source into a significant, enduring part of the IT environment.

So what’s this graph below showing over the course of (roughly) 1995 through 2007?

It’s showing Microsoft’s reported fiscal year revenue, which grew to $51.122B USD in 2007 from $6.075B in 1995 (you can reproduce it with data from here.)

During most of this time, we didn’t have Codeplex.
We didn’t have licenses submitted to the OSI.
We didn’t have Port 25.
We didn’t have Bill Hilf, or Sam Ramji, or the rest of the OSS lab.
And we didn’t have http://microsoft.com/opensource.

And Microsoft and open source did grow, together—coincidentally. In retrospect, this is not surprising. Microsoft technologies supported an ecosystem of passionate developers and an entrepreneurial individuals and companies and tens of millions of end-user programmers and end-users providing peer-to-peer assistance sharing knowledge—and code—with each other.

And we had many people at Microsoft working on (to highlight some of my current favorites) the research and development and product management path to technologies like Silverlight and XNA and Photosynth.

Now we have all those things—plus the opportunity to think every day about the “growing together” that has happened coincidentally from (say) 1995 until July 2007—and how we might work together with others to make it that much more (--food for thought: MySQL’s Community VP Kaj Arno blogged about the WAMP stack just after OSCON here).

There are reasons why my job can be challenging sometimes—but the slightest concern that Microsoft and open source don’t have opportunities to “grow together” by design faster and farther than they have (largely)** “by accident” over the last 10 plus years isn’t among them.

The “official” t-shirt of the Open Source Software Lab at Microsoft says “Open Source Software Lab at Microsoft: Reports of Snowballs Seen in Hell.” This year was another step forward to replacing that slogan with “Open Source Software Lab at Microsoft: Of Course.” Then I’ll get the answer I give back to people when I describe my job: not tough. Cool.

*I have internalized a July-to-June fiscal year calendar. I attribute this to the fact that my wife works in education, so summer forms an annual breakpoint for her, as well as to the fact I worked in Finance during a point in my life when I think I mistook a love of math for an affinity for pain.

**There’s more than enough material for, and reason to do, a separate post about some of the individual “pioneers” at Microsoft, without whom we would not have the resources we have in place today here at Microsoft.

Observations from OSCON and Linux World Expo

hanrahat — Thu, 23 Aug 2007 18:02:00 GMT

I’ve been a regular attendee of the O’Reilly Open Source Conference in Portland and Linux World Expo – San Francisco for several years, but this is the first time I represented Microsoft at them. Between the two conferences, I met a lot of people with whom I’ve worked for many years. I appreciate the encouraging words I received from many of them and I respect the concerns others expressed regarding my decision to join Microsoft.   A lot of our conversations were about what I thought I could accomplish by making the change.

One observation I’ve made while working with companies involved in open source is that every one of them wrestles with the balance of working within the community for the better good and reserving value for their own need to compete successfully for business.   There are few, if any, companies that are purely open-source directed. There are also few that are purely proprietary. Microsoft is in the spectrum of balance between proprietary and open source just like everyone else.

Clearly Microsoft’s balance tends toward the proprietary, but we demonstrated at both conferences that we take participation as a member of the open source community seriously and announced several significant actions. One of these announcements was that Microsoft is submitting both its permissive (MSPL) and community (MSCL) licenses to OSI for certification. Another was John Lam’s announcement of release of Iron Ruby and Iron Python as open source projects and that these are both open to community contributions.   Both of these efforts reflect serious attempts by Microsoft to participate in the development of truly open source software.

What’s also interesting is that the role of individual developers is changing, too. In his presentation at OSCON, “Current State of the Linux Kernel,” Greg Kroah-Hartman made the point that the largest group of contributors to the kernel is composed of “Unknown Individuals” who have no affiliation to a company with respect to their contributions.   Roughly 18% of contributions come from this group, and 13% come from another group called “Amateurs.” But, a member of the audience pointed out that this means the work of nearly 70% of contributors is being sponsored by industry. Of those 70% few are employed to be purely open-source contributors; most have responsibilities to their individual companies to ensure that some value is retained for their own business purposes.

We’re all finding our balance, companies and individuals alike, and that balance is rarely stationary. It frequently changes as we assess our roles in the software development industry. One of the things I want to accomplish is to find ways that Microsoft can adopt open source methodologies and can contribute to the greater good. Two areas I will concentrate on for now are interoperability, through the work we’re beginning with Novell in the areas of virtualization and web services management, and engagement with the SAMBA community to help ensure the quality of interaction between SAMBA and Microsoft products. I hope to attend the CIFS Workshop at Google next month to see where Microsoft can work with the SAMBA community beyond our current level of sharing bug and test data.

One of the first activities I engaged in when I joined Microsoft was to help draft the mission of Microsoft’s Open Source Software Lab. Here in a nutshell is what I hope to accomplish at Microsoft.

Produce mutual respect and understanding between Microsoft and the Open Source community such that both act responsibly together for the sake of better software and human potential & inclusion.

I invite those of you I have worked with over the years and all of you I spoke with at OSCON and LWE to make this our common goal and to join me in the effort.

OSCON 2007 Trip Report

anandeep — Fri, 17 Aug 2007 15:35:00 GMT

My overall impression was that OSCON was lower key than last year. There seemed to be fewer booths in the Exhibition floor and less palpable excitement in the venue. A lot of people were complaining about the quality of the tutorials and the talks. Or it may just be that this was my second time around attending OSCON and it didn’t have the same quality of excitement for me compared to the very first time!

Microsoft’s name was on the bag that they gave us. That was good to see. Also Microsoft’s name was on the marquee as a diamond sponsor. The other sponsors were all big names. Intel was a diamond sponsor as was Zimki. Zimki? Wait a minute, Zimki isn’t a big name! How did an open source web development platform company spend so much money? According to their web site “Zimki is Fotango's flagship product, a JavaScript web application development platform that enables developers and businesses to build and deploy web apps quickly without the need for tools, equipment or hosting. With no set-up fees and subscription charges,users simply pay for the computing resources that they consume on a utility basis.” Their blog also states that they were holding back their plans to roll out Zimki at OSCON . So a small company spent all this money and had no presence at OSCON. Shades of the dot com boom. Hopefully open source web development platforms are not today’s dot coms!

I attended two keynotes (that I remembered an hour after I attended them) – one involved Eben Moglen, the lawyer dude for the FSF, tearing into Tim O’Reilly. Tim O’Reilly was asking Eben questions about whether GPL V3 gave Google a free ride. Eben went into how he wanted to protect freedom and how the Open Source people had “wasted ten years” not making “freedom” the issue. But as persuasive and articulate as Eben is, I think he left the feeling with the audience that the FSF had given in to Google to get GPL V3 passed. Eben even used the words “diplomacy” to describe the process of building GPL V3.

Some dramatic moments such as Eben pointing to Tim and then to the sign behind him and saying,” Take down that sign with YOUR name on it and put “Freedom” there instead”. Tim even went to say to him – “I will ignore the personal attacks”. To which Eben said – “This is not a personal attack, it’s an invitation to diplomacy”. Wow! I could have watched a musical and not had so much drama.

Don’t get me wrong, I admire the FSF’s devotion to its cause. They have been consistently practicing what they preach. I am glad that there are people like them to keep even big companies like Microsoft honest. But I feel let down with their inconsistency with respect to Google.

The other keynote that I enjoyed was Bill Hilf. Bill is our GM, and we know all the stuff he said in speech. We try not to say one thing and practice another (surprise, surprise!). But he said it all so clearly, and so well in front of a largely skeptical audience. It was a masterful and engaging performance. Even when ambushed by Nat Torkington with questions that were not on the agenda, he didn’t lose his verve and kept on emphasizing what we do instead of what we say. It feels great to have a official place in the Microsoft firmament @ www.microsoft.com/opensource . Wait! Or was it nice to be the exclusive open source guys in such a big company? (From a purely selfish point of view)

I attended a few talks. One was Sam Ramji’s talk about our Interop efforts in virtualization, identity and management. To tell you the truth, I went because Sam is my boss. But I stayed because he did a great job of simplifying and presenting the information. I learnt and reinforced a ton about virtualization and the Interop challenges around it. I now have a firm grasp on the subject – not isolated chunks of information unconnected to each other. Ok Sam, when are you doing a talk on High Performance Computing? J

The other significant talk I attended was about Hadoop. Hadoop is an open source software platform that lets one easily write and run applications that process vast amounts of data. Or basically it implements Google’s MapReduce. According to Google’s original paper on MapReduce - “Programs written in the MapReduce style are automatically parallelized and executed on a large cluster of commodity machines. The run-time system takes care of the details of partitioning the input data, scheduling the program's execution across a set of machines, handling machine failures, and managing the required inter-machine communication. This allows programmers without any experience with parallel and distributed systems to easily utilize the resources of a large distributed system”.

Hadoop has been created by Doug Cutting, the creator of Lucene and Nutch. In order for him to do MapReduce effectively he had to do a “Google File System (GFS)” like system called “Hadoop Distributed File System (HDFS)”. HDFS was originally built as infrastructure for the Apache Nutch web search engine project.

Now, Yahoo is using Hadoop and HDFS for its back end. There is now an open source implementation of Google’s Open Source based proprietary stuff. If the community get’s behind it, it may be that the truly open source stuff may outshine the open source but proprietary stuff. Makes your head spin.

Oh, and why the name Hadoop? Doug Cutting’s son’s favorite elephant was named Hadoop. A name that came from the son’s imagination. I love Open Source!