Port 25: The Open Source Community at Microsoft : Interop, Codeplex

Real Mission Critical

Mark Stone — Mon, 01 Jun 2009 16:15:00 GMT

The 1.0 release of WinBioinfTools might seem like a modest event; as of this writing, the project has
44 downloads. High Performance Computing (HPC) is a small community, granted, and the number of HPC
applications for bioinformatics is a small subset of that. Let's not confuse popularity with importance,
however.

We use the phrase "mission critical" very frequently and somewhat casually within software development. In
talking to a friend about the swine flu outbreak, I was reminded that the phrase has its origin in
military history: an aspect of a mission so critical that failure in that aspect would result in the
loss of life. In the developing world where medical infrastructure can be a fragile thing, information about
the origins or genetic makeup of a virus can be vital. It can be mission critical.

Historically, the developing world has been dependent on developed western countries to do their research for them.
Open source is beginning to level that playing field, though. Using a cluster environment and software
projects like CoCoNUT for gene sequencing and comparison, even university research centers with modest x86
server environments can play in the HPC space. This is important because the research priorities for a
university in a developing country may be very different from the research priorities of a major western
research university. At its best, this is exactly the kind of lowering of barriers to entry that open
source should facilitate.

For all its value, CoCoNUT has two significant limitations. Its license is an academic license, not a fully
open source license. And it runs only on Linux/Unix systems. The latter is particularly important. Research
scientists are not IT professionals, and they should not have to care about the underlying platform on which
their software runs. The spirit of open source is to make software as widely available as possible, and there
is no way to meet that spirit without including Windows Server among the target platforms. Mission critical
demands no less.

So WinBioinfTools makes important steps forward on both fronts. The team at Nile University has released a
GPL-licensed project that "contains a number of programs for Bioinformatics running over Windows Cluster running
Windows HPC server 2008. The current version includes the CoCoNUT system for pairwise genome comparison,
parallel global sequence alignment, and parallel BLAST."

This is a great example of a local software community using open source to make their needs a priority,
and delivering a project that will benefit local software communities in other developing countries with
similar needs. WinBioinfTools puts us one step closer to making scientific computing software platform
neutral, and closer to making Windows Server a first class citizen in the open source world of HPC.

Announcing the PHP SDK for Windows Azure

Peter Galli — Wed, 13 May 2009 00:29:00 GMT

Vijay Rajagopalan, a Principal Architect here at Microsoft, is at TechEd India, where he will demo later this week a new set of interoperability projects related to PHP.

These projects include the PHP SDK for Windows Azure, an open source effort for which Microsoft has provided funding, with development by RealDolmen, whose goal is to provide high-level abstractions that enable PHP developers to interoperate readily with Windows Azure.

The PHP SDK for Windows Azure focuses on REST and provides PHP classes for Windows Azure blobs, tables and queue, helper classes for HTTP transport, AuthN/AuthZ, REST and error management, as well as manageability, instrumentation and logging support.

Rajagopalan will also announce the launch of a series of projects that offer samples and a toolkit that enable PHP developers to include Silverlight controls, Microsoft Virtual Earth maps and IE Webslices and Accelerators in PHP web applications; as well as automatically generated a simple "Create, Read, Update, Delete (CRUD)" PHP application from a table in SQL Server.

These projects, for which Microsoft has provided funding and which are available on Codeplex under a BSD license, are yet another proofpoint of the company's commitment to interoperability, and developers will be happy to know that the first batch of these have already been developed by Accenture.

Read Rajagopalan's full blog here for all the details.

The Azure Services Platform has been designed to be open, standards-based and interoperable, and its support for XML, REST and SOAP standards means that any of the Azure services can be called from other platforms and programming languages.

Microsoft has provided funding for two other SDKs that support third party programming languages: Java SDK for Microsoft .NET Services and Ruby SDK for Microsoft .NET Services so as to facilitate interoperability between the Azure Services Platform and non-Microsoft languages and technologies.

The inclusion of FastCGI in Windows Azure's hosting environment was announced at MIX 2009, and the protocol enables developers to run web applications on Windows Azure that were written using third party programming languages, including PHP. This opens up new options for PHP developers to deploy their applications.

A Technology Preview of the PHP SDK for Windows Azure will be released under a BSD license, while a functionally complete version of the SDK, which will support tables and queues, should be available for download by this fall of 2009, but the team is calling on developers to provide feature requests, test the toolkit, and join the user forum.

So, stay tuned, as there's a whole lot more to come!

Helping Facilitate Open Government

Peter Galli — Wed, 06 May 2009 19:03:00 GMT

Microsoft will announce on May 7 an initiative to help government agencies and developers publish and interact with their data in Windows Azure, the company's cloud computing platform.

One of these is the Open Government Data Initiative (OGDI), a cloud-based approach to housing public government data in Windows Azure, making it accessible in a programmatic manner via open standard protocols and application programming interfaces.

The source code for OGDI is being made publicly available through CodePlex, Microsoft's open source hosting site, so that developers may reuse it and provide feedback. Sample code is also being provided for technologies widely used on the Web, including PHP, Python, Flash, JavaScript, and Silverlight.

This initiative helps to provide government with tools focused on increasing responsiveness and access to critical services, streamlined processes and services.

For their part, Microsoft and its partners have developed a robust enterprise architecture approach that enables agencies to meet the technology requirements of government mandates with a familiar set of tools - built on an enterprise-ready, scalable, and easily-managed software-powered architecture.

So, in short, the goal of ODGI is to reduce the cost of publishing government data, and simplifying data access by leveraging cloud computing and open standards.

More information on Microsoft's Open Government Data Initiative can be found here.

To see an implementation of a data service in Windows Azure, using a sample of publicly available government data, visit this reference beta site.

These moves are part of Microsoft's ongoing open government efforts aimed at helping government organizations meet goals of transparency, participation and collaboration, particularly as an ever increasing amount of data becomes necessary and available.

As such, new methods need to be opened up to allow interaction with that data, and Microsoft's OGDI is designed to help public sector entities meet these goals.

This software, which underscores the importance of programmatic access to government data rather than having to download it, will give developers the ability to write programs that access data via Web-friendly programming methods without having to download or host the data; and let them write applications using any technology via open standards.

It also provides easier access to a broad array of government datasets, enabling the building of new and unique applications, while governments will be able to automatically refresh data without having to buy and maintain servers.

Cloud computing is the ideal platform for government data, and the technology is finally available to make it happen, says John Miri, Senior Fellow at the Center for Digital Government.

"The qualities that government looks for in an information management platform - things like flexibility, scalability, security, performance, and cost efficiency - are all better in a cloud model. As we see demands for government to become more transparent, collaborative, and interactive, a shift like this in technology architecture just has to happen, " he says.

For governments to become truly open, citizen access to public data in standards-based and interoperable ways is essential at all levels of government.

Given that most federal, state, local and education entities implement the American Recovery and Reinvestment Act (ARRA), they can meet open government goals of oversight, transparency and accountability through cloud and on-premises solutions such as Microsoft Stimulus360, which helps public sector agencies track, measure, and share information about federal stimulus programs through graphical dashboards and maps.

Better Windows Development Environments - One project at a Time

Mark Stone — Fri, 10 Apr 2009 14:45:00 GMT

Discussions of the PC market usually break down into "desktop" vs. "server", as if these are the only platform categories. However, the developer's dev box -- call it a "workstation" to distinguish it from desktop -- is really a separate platform. Remember, when Linus Torvalds created Linux it wasn't because he needed a better
desktop operating system, or a better server operating system. What he wanted was something that could run the gcc compiler. He wanted a developer workstation he could use at home.

Of course the developer workstation market influences other markets. Developers who develop on a platform are more likely to develop for a platform. So winning developer mind share is often about giving them what they want in the way of development environment. And in this regard, open source developers are something of a
different breed.

Microsoft has a great set of developer tools; I've certainly known developers who swear by Visual Studio. But there's something of a disconnect between graphical tools favored by Windows developers and then command line tools favored by traditional open source developers; I've also known developers whose first order of business
with a new Windows workstation is to GNU-ify it.

Ironically, the Internet has a convergence effect, drawing these two camps together. Put developers online, and they can collaborate. Put developers online, and they can not only develop, but they can build, deploy, and test. The workstation has become not so much a computer as an environment. The developer's toolkit includes version control, build management, automated testing, and the need to do all these things as a team rather than an individual. Developer environments have evolved rapidly to adapt to these changes.

The Apache Software Foundation (ASF) seems to understand this evolution as well as anyone, and a number of ASF projects focus specifically on tools for the developer environment (Ant, Buildr, Continuum, Gump, and Maven come to mind as a few examples).

What's interesting is to see the .NET developers following suit, and wanting these same sorts of tools for their development environment. What's surprising is that this .NET effort is very grass roots driven.

"Panday" is a Filipino word meaning "blacksmith", and can also be a reference to the graphic novel super hero of the same name (the graphic novel is also originates from the Philippines). This provides an appropriate metaphor for the name of the NPanday project on Codeplex.

The NPanday project is one of several affiliated with Microsoft's Open Source Lab in the Philippines, and is part of the effort to bring to .NET some of the capabilities found in other open source development environments. The aim of NPanday is integrate Apache Maven into the .NET development environment.This would enable .NET
developers to take advantage of Maven-compatible development infrastructure.

Projects like NPanday are important because they offer developers more choice of tools in a Windows development environment. The more familiar those tools are to open source developers, the more open source development will be done on and for Windows. NPanday is also an important project for interoperability, making it easier to integrate .NET development with other development done using Maven.

Brazilian Students Set Their Own Course

Mark Stone — Mon, 16 Mar 2009 19:05:00 GMT

I’m going to tell a story that starts in Indiana, but really it’s about Brazil.

Once upon a time “scientific computing” was nearly synonymous with “Fortran”. Today, though, just about any high level language can be used to write High Performance Computing (HPC) applications. These days that language choice also includes C#.

At Indiana University, the Open Systems Lab has pioneered work to implement Message Passing Interface (MPI) support for .Net, so that MPI applications can be written in C#. The project is MPI.Net, and you can find it on Codeplex. It is open source, about three years old, has reached a 1.0 release, and is compatible with two other important open source projects, OpenMPI and Mono. The principle developers behind the project are Andrew Lumsdaine at Indiana University and his former student, Douglas Gregor, who is now on the faculty of Rensselear Polytechnic Institute.

This is the kind of open source work that’s really exciting to see because of the way it expands choices for the developer and the end user. A C# developer should not be closed off from writing HPC applications if that’s what they want to do. And a research scientist should not have to think about whether their lab is running Linux or Windows Server. Both of these individuals are working enough layers above the operating system that somebody else’s operating system choice should not be a constraint.

So I was very excited to learn that students in Brazil at Federal University of Rio Grande do Sul were doing work on MPI, and excited to talk with them about their work. One of their projects is MPI#, also open source and also hosted on Codeplex.

MPI# builds on top of the work of MPI.Net, adding some functionality not yet present in MPI.Net. Specifically, quoting from the project description:

The goals of this project would be to build upon MPI.NET in order to complement it with the features that are missing, mainly regarding collective communication. Either they could benefit from C# native support for such communication, either they could be programmed on top of the provided MPISend/MPIRecv encapsulations. C# and .NET features such as fault tolerance or dynamicity support would be studied, in other to turn the MPI# implementation robust in large, dynamic and heterogeneous platforms.

Two of the students working on MPI# are Ismael Stangherlini and Fernando Afonso. They are graduate students in computer science, working on projects affiliated with the Brazilian Interoperability and Open Source Software Development Nucleous. When I talked to them about their work on MPI# I was curious what their communication with Indiana University had been like. Their response: they had never been in contact with Indiana University; they simply downloaded the code for MPI.Net and started working on their own.

That’s the magic of open source: that they can, in fact, just download the code on their own and start coding against it. They may make an important contribution to MPI.Net. Or their code may be entirely disregarded. Or they may move on to other projects and somebody else may or may not pick up where they left off. At this stage it’s too early to tell. But the fact that all of these scenarios are possible demonstrates why, as a methodology, open source is so nimble and adaptive. A top-down product development process, or a top-down standards development process can only execute on the innovations envisioned by the few at the top, and at the speed of the slowest decision-makers in the process. But a bottom-up open source process enables every innovation that anyone at the grass roots level can see.

Oxite: an Open Source Content Management Platform

Peter Galli — Tue, 09 Dec 2008 19:40:00 GMT

I'm on the road with Robert Duffner, our Senior Director of Platform Strategy, talking to tech press in New York, Boston and San Francisco this week.

So, imagine my surprise when top Microsoft blogger Mary Jo Foley ran an article about a new open source content system known as "Oxite."

The alpha code for Oxite, which is an open source, standards compliant, and highly extensible content management platform that can run anything from blogs to big web sites, and which already runs MIX Online, was recently released on CodePlex, our open source project hosting site.Oxite is licensed under the Microsoft Public License.

While we were surprised by Mary Jo's blog, this is a great thing as it shows just how many open source projects are taking place under the covers here at Microsoft. Having a range of our developers across Microsoft working on a variety of open source projects targeted at different communities of users, is exactly the goal of our group.

Stay tuned, as I expect to be surprised a lot more going forward and, as I do, I'll bring these projects to your attention, however they come to my attention.

Open Source Interoperability Projects

Jean Paoli — Fri, 07 Nov 2008 07:21:00 GMT

Interoperability has always been a focus area at Microsoft. Being a platform company, Microsoft has engaged in interoperability at many levels - product features, participation in standardization bodies, publishing many technologies under open licenses and working closely with customers, governments and partners to understand the heterogeneous IT landscape and discuss practical interoperability solutions.

Earlier this year, these activities were formalized under the Interoperability Principles for all of our high-volume products.

I am the General Manager of Interoperability Strategy at Microsoft, and I have worked across the company on many interop initiatives. I am happy to see many interop projects now coming out of Microsoft and, personally, having many of them based on XML makes me doubly happy.

My team has built several bridging technologies and solutions for many of our products to enable interoperability. These are being run as open source projects and released under a broad BSD license so that our customers and partners can use them in many open and broad scenarios.

Interoperability has been getting enhanced attention at a lot of conferences lately and Microsoft has also upped its participation at many open source conferences such as OSCON, the Eclipse Conference and ApacheCon.

At Microsoft's Professional Developer's Conference last month, the interoperability story was part of almost every announcement and keynote address. As Sam Ramji writes in his latest blog, Microsoft is also participating at ApacheCon and highlighting the interoperability work we are doing. These are indeed exciting times!

On the interoperability front, my team has been working with the WSO2 since the TechEd 2007 Conference to demonstrate interoperability using our StockTrader reference application.

This week, the WSO2 proposed a new Apache incubation project, known as Stonehenge, to further this work. The aim of this project is to set up sample applications to demonstrate interoperability with multiple underlying platform technologies by using currently defined W3C and OASIS standard protocols. We look forward to working with WS02 on the scope of this project, and having discussions with the community.

I also want to highlight some open source interoperability projects that my team has been working on with third parties, companies and members of the community at large, which may be very relevant to the readers of this blog.

Eclipse Tools for Silverlight

Eclipse4sl allows Java developers to develop code for the Silverlight platform within the Eclipse development environment, and contains both an advanced project system for creating Silverlight applications and media experiences as well as a compiler for packaging Silverlight applications for deployment.

Interoperability with the Azure Services platform

Announced at PDC recently, the Azure Services Platform is an internet-scale cloud computing and services platform hosted in Microsoft data centers. It provides an operating system and a set of developer services which can be used individually or together. Microsoft .NET Services is a key component of the Azure Services Platform that offers a set of Microsoft-hosted, highly scalable, developer-oriented services that provide the key building blocks, like, Access Control, Service Bus, and Workflow service.

The Azure Services Platform, built from the ground up to be consistent with Microsoft's commitment to openness and interoperability and in that spirit, we have built two cross-platform SDKs for .NET services - for Java and Ruby.

Information Cards Interoperability

Windows CardSpace is Microsoft implementation of Information Cards on the Windows platform. Information cards are a core part of Identity Metasystem and help both site owners and visitors to manage, control, and exchange digital identities more safely and consistently.

We have also built four open source projects that help Web developers support information cards on diverse platforms:

OpenXML-ODF translators

The goal of this project is to provide translators to allow for interoperability between applications based on ODF (OpenDocument) standard and Office Open XML standard. The translator is based on XSLT transformations between two XML formats, along with some pre- and post-processing, and is available on Sourceforge under a BSD-like license.

OpenXML-UOF translators

The goal of this project is to provide translators to allow for interoperability between applications based on UOF (Uniform Office Format) standard and Office Open XML standard.

UOF is an emerging standard, which is being developed by the Chinese Office Software Work Group (COSWG), led by the China Electronics Standard Institute (CESI), the Ministry of Information Industry (MII), major suppliers of Chinese office software suites, and other academic institutions.The translator is based on XSLT transformations between two XML formats, along with some pre- and post-processing. It is available at SourceForge under a BSD-like license

I would like to hear your comments and feedback on these projects and also welcome open engagement on what Microsoft should be doing for interoperability. Tell us what other interoperability scenarios we should be looking to address.

I also want to thank the multiple third party companies and the community members we cooperate with, as well as the members of my team: Vijay Rajagopalan, Sumit Chawla, Kamaljit Bath, Claudio Caldato, Jean-Christophe Cimetiere and many others for working on these projects and building technical solutions for interoperability with key Microsoft products and technologies.

struct.new("future", :open, :microsoft)

Sam Ramji — Thu, 06 Nov 2008 08:49:00 GMT

I delivered the keynote at ApacheCon in New Orleans today, where I talked about some of the new milestones we have chalked up on the journey inside Microsoft towards greater participation and growth with open source communities, and our strategy of "architecting for participation."

This strategy focuses on four significant themes: community; contribution; partnerships; and choice. Microsoft believes that the next ten years of software will be a time of growth and change where both open source and Microsoft communities will grow together.

We also believe that in an increasingly interconnected world, where more people have a greater opportunity to use more technology to do more things than ever before. We support those choices and are expanding interoperability between open source technologies and Microsoft technologies.

So, on the interoperability front, we have been working with the WS02 since our TechEd 2007 Conference, to demonstrate interoperability using our StockTrader reference application. Today, the WS02 announced they would build an open source version of the sample application under "Project Stonehenge," which hs been proposed as a new Apache incubation project.

WS02 will use the project to set up sample applications that demonstrate seamless interoperability across multiple underlying platform technologies, using currently defined W3C and OASIS standard protocols.

My team has been working closely with that of Jean Paoli, the General Manager of Interoperability Strategy at Microsoft, whose team is driving much of this interoperability work. You can read more about all this in Jean's blog post.

Microsoft has also decided to move the development of protocol parsers for Microsoft Network Monitor - a free protocol analyzer and network sniffer - to an open source model, on CodePlex, which will host the development of parsers for public protocols and for protocols described in our Open Protocol Specifications for Windows.

An updated parser package has been released and a source tree created on Codeplex. We want Netmon to be the best-of-breed tool for network monitoring at Microsoft, not just for Windows.

Microsoft also recently joined the AMQP Working Group as a participant, with the goal of contributing towards the development of the specification and to enable greater customer choice in the marketplace.

At the request of community members, we have now committed to participate in the Apache Qpid project, a widely adopted open source implementation of the AMQP specification that addresses the customer need for choice and improved messaging interoperability.

Our customers are telling us that they would like to see the Apache Qpid project extended to interoperate with Windows, so the next few months of participation will be focused on understanding the community's effort to build Windows based AMQP software.

Participation will give us the opportunity to learn from other project participants, so that we can be in a position to consider making a valuable contribution. But it is important to note that the Apache Qpid project is just one of many AMQP specification implementations, and we are open to supporting additional projects.

You can read an interesting technical research paper from Ohio State University analyzing the performance of the Qpid implementation of AMQP here.

Microsoft also announced, at PDC 2008, our commitment to include "Oslo" - an upcoming set of technologies for modeling - in the Open Specification Promise. This will ensure that the "Oslo" declarative modeling language, codenamed "M", is interoperable with prominent industry standards such as WS* specifications, XML formats, industry protocols, and security standards.

Two of the core focuses for Oslo are integration and interoperability. As such, it will integrate with next-gen Microsoft technologies, including System Center, Visual Studio and BizTalk Sever. We also plan to work with partners and the industry, so as to make Oslo interoperable with important standards and industry protocols.

One of the key ways we think customers will achieve customization for their platforms is through the use of textual and visual DSLs, which can be written uniquely by the developer for vertical industries and specific domains, or they can use pre-existing DSLs in these same scenarios.

The hope is that we will establish a broad and open ecosystem around "M" that will enable customers to bring the power of model-driven applications and systems to their heterogeneous environments.

Finally, on the Live Search front, the Powerset team recently resumed its participation with HBase, which is elated to infrastructural storage technology enabling large scale data processing.

The HBase project receives significant lift from the active community that supports the project, and Powerset's continued participation on HBase could allow us to accelerate the integration of Powerset's technology into Live Search, resulting in improvements to the end-user experience.

So, stay posted. There's a lot more to come!

Cross-Platform Access to Codeplex Compliments of our Friends at Teamprise

MichaelF — Tue, 20 Mar 2007 22:40:00 GMT

Awhile ago we interviewed Martin Woodward from Teamprise regarding their suite of client applications that provide cross-platform access to Visual Studio Team Foundation Server.

Today Teamprise announced that they are providing complimentary licenses for the Teamprise Client Suite to developers who plan to use the license to access Codeplex. This allows projects with developers on the Eclipse IDE along with anyone on the Mac, Linux or Unix platforms to use Codeplex for hosting their projects.

To obtain access to the client go to the sign-up page.

To provide some additional information on this release and Teamprise, Anandeep sat down with Martin to catch up.

Video: Cross-Platform Access to Codeplex Compliments of Teamprise

Martin's Blog about this announcement can be found here.