Port 25: The Open Source Community at Microsoft : Open Source, Windows Server

Windows Cache Extension 1.0 for PHP Released

Peter Galli — Fri, 20 Nov 2009 00:40:00 GMT

The Microsoft Internet Information Services (IIS) team announced today the release of Windows Cache Extension 1.0 for PHP, a PHP accelerator that is used to increase the speed of PHP applications running on Windows and Windows Server. This is a production-ready release that is provided under an open source BSD license, with the source code hosted and maintained here, and the documentation hosted on php.net.

You can find more details on this release on IIS team Product Unit Manager Mai-lan Tomsen Bukovec's blog.

WinCache extension is a significant open source contribution from Microsoft to the PHP on Windows community. The extension code is hosted and maintained on PHP Extensions Community Library (PECL) and is available for everyone to view, branch, compile, and contribute to.

The IIS team also invites the PHP development community to join it in development of this caching extension for PHP on Windows. There have already been some contributions from the community whileWinCache was in the pre-release mode, and IIS team is looking forward to having others join this new PHP on Windows caching project.

Also, in time for this release, an independent PHP company - Ibuildings - has conducted a benchmark test with the WinCache RTW bits and published the results.

The release of this production-ready PHP accelerator for Windows is an important step towards making the Windows operating system an even better platform for hosting PHP applications.

WinCache extension significantly improves performance of PHP applications and lowers CPU load on the server. This, together with the fact that no application code changes are necessary to take advantage of the caching, makes WinCache a must have extension when running PHP on Windows.

More information about the WinCache extension for PHP can be found as follows:

The PECL documentation on WinCache
The PHP on Windows feature team blogs (Kanwal, Ruslan, and Don)
The WinCache community forum
The IIS.NET documentation on WinCache
The WinCache on PECL source code, licensed under BSD
How to file bugs on WinCache in PECL

Team Microsoft Sings "Them Incubator Blues" at ApacheCon

Peter Galli — Thu, 12 Nov 2009 22:33:00 GMT

As you probably know, Microsoft was both a sponsor and active participant at ApacheCon 2009 in Oakland, California last week.

But what you might not know is that we also showed our lighter, more fun side, when we participated in the Lightning Talks, which were held on Thursday evening, November 5, during a reception with plenty of popcorn, beer and wine.

As my colleague Jas Sandhu noted in his blog about this - and where you can also find the song's lyrics - the talks are a lively, spontaneous ApacheCon tradition with speakers getting about 5 minutes to poke at each other, the projects, technology, community etc ... and have a bit of fun!

The singing and dancing Microsoft team was led by Kent Brown, the Product Manager for Windows Communication Foundation, who was not only the singer, songwriter and guitar player, but also the author of 'Them Incubator Blues,' which is very tongue-in-cheek and loosely based on his experience participating in the Stonehenge Project and working with the community.

He was joined by me, Microsoft's open source community manager; Tanya Young, our chief cat herder at the conference; and Jas Sandhu, a Senior Technical Evangelist. It was great fun and we hope you enjoy it!

Video courtesy of Mladen Turk from Red Hat.

Another Great Step Forward for Hyper-V, Red Hat Certification

hjanssen — Wed, 07 Oct 2009 17:06:00 GMT

Hello again! It has been a pretty busy couple of months for us, and I wanted to give you an update on what we've been doing.

We just completed the first step in another major milestone for Hyper-V. As you can read from Mike Neil's Blog, Microsoft and Red Hat just completed certification in each other's virtualization program.

This means that customers now can deploy Microsoft Windows Server and Red Hat Enterprise Linux and a range of select applications, virtualized on Red Hat and Microsoft hypervisor virtualization software, knowing that the solutions will be supported by both companies.

This again demonstrates Microsoft's commitment to its virtualization platform, and the Hyper-V team's continuous investment in interoperability and heterogeneity through the datacenter.

And I am very proud that my team has been working very hard on getting the Red Hat certification completed.

But, for my group, this is just the first step: we worked on and got certified for Hyper-V running Red Hat in emulated mode and, now, the next step for us is to get certification for enlightened mode, the mode where the guest OS is Hyper-V aware,and can thus access Hyper-V functionality directly.

So, what is needed for enlightened mode? Well, to get that step, we needed to get the Linux Integration Drivers submitted to the Linux kernel, and then we needed those drivers officially accepted in a mainline kernel.

This has now happened as well, as we have been accepted into the mainline kernel. We are in Linux Kernel release 2.6.32, and that release is currently going through development and testing. Once that one is final and officially released, we can take the next step, which is to get those official Linux Integration Component drivers certified with Red Hat.

The timeline for that is not completely set in stone right now, and I do not know if there is an official expected release date for 2.6.32 as yet. But we should be able to move forward with the enlightened mode certification soon after the 2.6.32 final release.

And we are, of course, continuing to work on the Linux Integration components, adding new features and all that good stuff. But I will write more about those items in the near future.

Sam Ramji is leaving Microsoft

billhilf — Thu, 10 Sep 2009 17:47:00 GMT

It's been a while since I made an appearance on Port25. I felt it was important to provide some thoughts to the Port25 community on Sam Ramji's impending departure from Microsoft.

After many years helping to carry the open source software banner for the company, Sam is leaving Microsoft at the end of this month. You may have also heard that he has accepted the position of interim President of the CodePlex Foundation as well as a leadership position at a startup in California. (I'll let Sam and his new company share more details there.)

Sam joined my team three years ago to drive open source technical strategy. I have eagerly supported him as he passionately articulated a vision that Microsoft could coexist - and even thrive - in a heterogeneous IT world.

The perspectives on OSS at Microsoft have evolved to the point where Microsoft's open source strategy is no longer just locked in a single ‘lab' on campus - now OSS is an important part of many product groups and strategies across the company. We have become increasingly clear on where we work with open source - development methodologies, projects, partners, products and communities - and where our products compete with commercial open source companies or platforms. Today, there are engineering and business leaders across the company, myself included, looking at how to drive interoperability for customers and as a lever for new growth.

And, because we recognize the importance of having that strong internal advocate for open source, we are actively seeking someone to fill Sam's shoes at Microsoft.

We will not waver in our commitment to open source.

To my friend Sam: Best of luck to you and your family as you move on to your next great adventure, and THANK YOU for all of your efforts and passion.

Virtualizing Free Linux Distributions in Windows Server 2008 R2

Peter Galli — Tue, 11 Aug 2009 00:00:00 GMT

Jason Perlow, a columnist over at ZDNet, has written a comprehensive review on virtualizing free Linux distributions in Windows Server 2008 R2.

In his Tech Broiler column, Perlow notes that the updated Hyper-V bare-metal hypervisor virtualization layer in Microsoft's upcoming Windows Server 2008 R2, which is due to be released August 14th to MSDN and Technet customers, now has support for SUSE Linux Enterprise Server 11 and Red Hat Enterprise Linux 5.3.

"Additionally, Linux support and performance has greatly improved over the initial Hyper-V release. Microsoft also recently released its Hyper-V Linux Integration Components (Linux ICs) under the GPLv2 Open Source License," Perlow says.

The Linux ICs for Hyper-V, which are in Release Candidate status, provide synthetic device drivers that enhance I/O and networking performance when Linux OSes are virtualized under Hyper-V.

"The source code for the Linux IC's were accepted into the Linux Driver Project and should become part of the Linux Kernel within two subsequent releases and code merges - 2.6.32 is expected to be when they will be integrated, and all Linux distributions using that kernel code base going forward should be Hyper-V enabled out of the box. Yes, you heard that correctly, Microsoft is now an official Linux Kernel contributor," Perlow says.

You can read the rest of Perlow's column here.

Microsoft Releases Device Driver Code to the Linux Community

Peter Galli — Mon, 20 Jul 2009 16:48:00 GMT

In what many may see as a surprising move, Microsoft today released 20,000 lines of device driver code to the Linux community under the popular General Public Licence v2.

The code includes three Linux device drivers, and has been submitted to the Linux kernel community for inclusion in the Linux tree.

The drivers will be available to both the Linux community and customers, and will enhance the performance of the Linux operating system when virtualized on Windows Server 2008 Hyper-V or Windows Server 2008 R2 Hyper-V.

In an article posted to Microsoft's PressPass site, Tom Hanrahan, director of Microsoft's Open Source Technology Center, notes that this is a significant milestone because it's the first time the company has released code directly to the Linux community. "Additionally significant is that we are releasing the code under the GPLv2 license, which is the Linux community's preferred license," he said.

In the same article, Sam Ramji, senior director of Platform Strategy at Microsoft, points out that Microsoft communities and open source communities are growing together, which is ultimately of benefit to customers. An example of this is the Linux community, which has built a platform used by many customers. "So our strategy is to enhance interoperability between the Windows platform and many open source technologies, which includes Linux, to provide the choices our customers are asking for," he said.

Ramji also alluded to the fact that people are often surprised when they hear how much open source community and development work is happening across Microsoft, which is largely due to the fact that these collaborations focus more on getting the work done and engaging with the various communities on a one-to-one basis and less about promoting them.

One example of how Microsoft participates with, and contributes to, open source is its relationship with the PHP Community. The company's involvement includes contributing to the PHP Engine, optimizing PHP 5.3 to perform strongly on Windows, and working to improve the performance of numerous PHP applications on Windows. Then there is the ongoing participation in various Apache Software Foundation projects, such as Hadoop, Stonehenge and QPID.

"In short, we're focused on building sustainable business strategies for open source at Microsoft ... we see open source playing into three key areas, one of which is the use of 'inbound' open source and the open source development model to make our software development processes more efficient."

"Good examples of this include what we did recently with jQuery in Visual Studio 2008, the implementation of OpenPegasus connectors and adaptors into System Center Operations Manager, and work that the Microsoft High Performance Computing team did with the Argonne National Lab (ANL) to source its MPICH2 implementation, which is a portable implementation of the Message Passing Interface (MPI) used in cluster computing and super computers," Ramji said.

We'll be posting a number of other articles on the release of the device driver code to the Linux community over the week, several of which will be penned by Hank Janssen from Microsoft's Open Source Technology Center, so look out for those.

Real Mission Critical

Mark Stone — Mon, 01 Jun 2009 16:15:00 GMT

The 1.0 release of WinBioinfTools might seem like a modest event; as of this writing, the project has
44 downloads. High Performance Computing (HPC) is a small community, granted, and the number of HPC
applications for bioinformatics is a small subset of that. Let's not confuse popularity with importance,
however.

We use the phrase "mission critical" very frequently and somewhat casually within software development. In
talking to a friend about the swine flu outbreak, I was reminded that the phrase has its origin in
military history: an aspect of a mission so critical that failure in that aspect would result in the
loss of life. In the developing world where medical infrastructure can be a fragile thing, information about
the origins or genetic makeup of a virus can be vital. It can be mission critical.

Historically, the developing world has been dependent on developed western countries to do their research for them.
Open source is beginning to level that playing field, though. Using a cluster environment and software
projects like CoCoNUT for gene sequencing and comparison, even university research centers with modest x86
server environments can play in the HPC space. This is important because the research priorities for a
university in a developing country may be very different from the research priorities of a major western
research university. At its best, this is exactly the kind of lowering of barriers to entry that open
source should facilitate.

For all its value, CoCoNUT has two significant limitations. Its license is an academic license, not a fully
open source license. And it runs only on Linux/Unix systems. The latter is particularly important. Research
scientists are not IT professionals, and they should not have to care about the underlying platform on which
their software runs. The spirit of open source is to make software as widely available as possible, and there
is no way to meet that spirit without including Windows Server among the target platforms. Mission critical
demands no less.

So WinBioinfTools makes important steps forward on both fronts. The team at Nile University has released a
GPL-licensed project that "contains a number of programs for Bioinformatics running over Windows Cluster running
Windows HPC server 2008. The current version includes the CoCoNUT system for pairwise genome comparison,
parallel global sequence alignment, and parallel BLAST."

This is a great example of a local software community using open source to make their needs a priority,
and delivering a project that will benefit local software communities in other developing countries with
similar needs. WinBioinfTools puts us one step closer to making scientific computing software platform
neutral, and closer to making Windows Server a first class citizen in the open source world of HPC.

Better Windows Development Environments - One project at a Time

Mark Stone — Fri, 10 Apr 2009 14:45:00 GMT

Discussions of the PC market usually break down into "desktop" vs. "server", as if these are the only platform categories. However, the developer's dev box -- call it a "workstation" to distinguish it from desktop -- is really a separate platform. Remember, when Linus Torvalds created Linux it wasn't because he needed a better
desktop operating system, or a better server operating system. What he wanted was something that could run the gcc compiler. He wanted a developer workstation he could use at home.

Of course the developer workstation market influences other markets. Developers who develop on a platform are more likely to develop for a platform. So winning developer mind share is often about giving them what they want in the way of development environment. And in this regard, open source developers are something of a
different breed.

Microsoft has a great set of developer tools; I've certainly known developers who swear by Visual Studio. But there's something of a disconnect between graphical tools favored by Windows developers and then command line tools favored by traditional open source developers; I've also known developers whose first order of business
with a new Windows workstation is to GNU-ify it.

Ironically, the Internet has a convergence effect, drawing these two camps together. Put developers online, and they can collaborate. Put developers online, and they can not only develop, but they can build, deploy, and test. The workstation has become not so much a computer as an environment. The developer's toolkit includes version control, build management, automated testing, and the need to do all these things as a team rather than an individual. Developer environments have evolved rapidly to adapt to these changes.

The Apache Software Foundation (ASF) seems to understand this evolution as well as anyone, and a number of ASF projects focus specifically on tools for the developer environment (Ant, Buildr, Continuum, Gump, and Maven come to mind as a few examples).

What's interesting is to see the .NET developers following suit, and wanting these same sorts of tools for their development environment. What's surprising is that this .NET effort is very grass roots driven.

"Panday" is a Filipino word meaning "blacksmith", and can also be a reference to the graphic novel super hero of the same name (the graphic novel is also originates from the Philippines). This provides an appropriate metaphor for the name of the NPanday project on Codeplex.

The NPanday project is one of several affiliated with Microsoft's Open Source Lab in the Philippines, and is part of the effort to bring to .NET some of the capabilities found in other open source development environments. The aim of NPanday is integrate Apache Maven into the .NET development environment.This would enable .NET
developers to take advantage of Maven-compatible development infrastructure.

Projects like NPanday are important because they offer developers more choice of tools in a Windows development environment. The more familiar those tools are to open source developers, the more open source development will be done on and for Windows. NPanday is also an important project for interoperability, making it easier to integrate .NET development with other development done using Maven.

Brazilian Students Set Their Own Course

Mark Stone — Mon, 16 Mar 2009 19:05:00 GMT

I’m going to tell a story that starts in Indiana, but really it’s about Brazil.

Once upon a time “scientific computing” was nearly synonymous with “Fortran”. Today, though, just about any high level language can be used to write High Performance Computing (HPC) applications. These days that language choice also includes C#.

At Indiana University, the Open Systems Lab has pioneered work to implement Message Passing Interface (MPI) support for .Net, so that MPI applications can be written in C#. The project is MPI.Net, and you can find it on Codeplex. It is open source, about three years old, has reached a 1.0 release, and is compatible with two other important open source projects, OpenMPI and Mono. The principle developers behind the project are Andrew Lumsdaine at Indiana University and his former student, Douglas Gregor, who is now on the faculty of Rensselear Polytechnic Institute.

This is the kind of open source work that’s really exciting to see because of the way it expands choices for the developer and the end user. A C# developer should not be closed off from writing HPC applications if that’s what they want to do. And a research scientist should not have to think about whether their lab is running Linux or Windows Server. Both of these individuals are working enough layers above the operating system that somebody else’s operating system choice should not be a constraint.

So I was very excited to learn that students in Brazil at Federal University of Rio Grande do Sul were doing work on MPI, and excited to talk with them about their work. One of their projects is MPI#, also open source and also hosted on Codeplex.

MPI# builds on top of the work of MPI.Net, adding some functionality not yet present in MPI.Net. Specifically, quoting from the project description:

The goals of this project would be to build upon MPI.NET in order to complement it with the features that are missing, mainly regarding collective communication. Either they could benefit from C# native support for such communication, either they could be programmed on top of the provided MPISend/MPIRecv encapsulations. C# and .NET features such as fault tolerance or dynamicity support would be studied, in other to turn the MPI# implementation robust in large, dynamic and heterogeneous platforms.

Two of the students working on MPI# are Ismael Stangherlini and Fernando Afonso. They are graduate students in computer science, working on projects affiliated with the Brazilian Interoperability and Open Source Software Development Nucleous. When I talked to them about their work on MPI# I was curious what their communication with Indiana University had been like. Their response: they had never been in contact with Indiana University; they simply downloaded the code for MPI.Net and started working on their own.

That’s the magic of open source: that they can, in fact, just download the code on their own and start coding against it. They may make an important contribution to MPI.Net. Or their code may be entirely disregarded. Or they may move on to other projects and somebody else may or may not pick up where they left off. At this stage it’s too early to tell. But the fact that all of these scenarios are possible demonstrates why, as a methodology, open source is so nimble and adaptive. A top-down product development process, or a top-down standards development process can only execute on the innovations envisioned by the few at the top, and at the speed of the slowest decision-makers in the process. But a bottom-up open source process enables every innovation that anyone at the grass roots level can see.

Microsoft, Red Hat to Offer Joint Technical Support

Peter Galli — Mon, 16 Feb 2009 16:30:00 GMT

Microsoft and Red Hat announced this morning that they have recently signed agreements to test and validate their server operating systems running on one another's hypervisors.

This is deeply significant as it means that customers will be able to confidently deploy Windows Server and Red Hat Enterprise Linux (RHEL), virtualized on Microsoft and Red Hat hypervisors, knowing that the solutions will be supported by both companies.

In short, Red Hat has joined Microsoft's Server Virtualization Validation Program, and Microsoft is now a Red Hat partner for virtualization interoperability and support.

Microsoft will also be listed in the Red Hat Hardware Certification List once the Red Hat certification process has been completed later this year.

Microsoft will also publish Linux Integration Components for RHEL when the testing and validation is complete and, according to Mike Neil's blog on this news, Red Hat is expected to provide Windows Hardware Quality Labs drivers for a variety of Windows Server versions.

"This means that those customers with valid support agreements will be able to run these validated configurations and receive joint technical support for running Windows Server on Red Hat Enterprise virtualization, and for running Red Hat Enterprise Linux on Windows Server 2008 Hyper-V or Hyper-V Server 2008," Neil says.

So, while Microsoft and Red Hat will continue to compete, customers have asked us to work together on technical support for server virtualization. These agreements respond to that request by giving them a new level of integration between Red Hat Enterprise Linux and Windows Server for their heterogeneous IT environments.

Customers with valid support agreements will now be able to call either Microsoft or Red Hat to have their issues resolved. If the first vendor contacted cannot resolve the issue, they will refer the problem to the other vendor for resolution; assuming the customer also has a valid support agreement with that vendor.

In the event that the second vendor cannot resolve the problem alone, Microsoft and Red Hat will work together to come to a resolution for the mutual customer.

What's more, once RHEL is validated as a guest on Windows Server 2008, Microsoft System Center Operations Manager 2007 R2 - which will include cross platform monitoring - will support RHEL server versions 4 and 5 in the second quarter of this year so that customers can manage the applications and operating systems in the guest VM.

This will allow customers to monitor end-to-end data center applications that are distributed across both Windows Server and RHEL, whether these servers are physical or virtual, thereby improving the visibility organizations have of these distributed applications, and reducing their operational costs by providing a single tool to manage these across operating systems.

Also, to be clear given that questions are going to be asked about how this compares to the existing relationship between Microsoft and Novell, this agreement with Red Hat is specific to joint technical support for our mutual customers using server virtualization. So, in that regard, think of it as one dimensional, whereas Microsoft's partnership with Novell is multi-dimensional.

For more on all this, you can read Mike Neil's blog, the press release here, and watch the public webcast.

MindTouch, Microsoft, and Social Enterprise Collaboration

Peter Galli — Wed, 17 Dec 2008 01:33:00 GMT

In this, our second video filmed at an open source company, the spotlight falls on MindTouch, whose Deki Wiki enables users to connect teams, enterprise systems, publishing systems, Web services and Web 2.0 applications.

The first video talked to developers at BitRock about their product as well as their thoughts on open source versus proprietary software, and can be viewed here.

The MindTouch Deki, which is built on top of the Mono, the open source .Net implementation, is downloaded some 2,000 times a day.

You'll see and hear Damien, a graphics designer, talk about his one-eyed fish and the disappearance of Phil, the filter feeder, and how he is only going to have community fish in his next tank, fish who like one another and work well together.

Aaron Fulkerson, the CEO and one of the company's founders, talks about his passion for getting the product running on Windows, with Windows Server 2008 and a Microsoft installer. For him, it really doesn't matter what technology the solution is implemented on, as long as the company has delivered a great product to the market that is "free, as in freedom, as in free speech."

While there is a groundswell of community adoption around MindTouch's Linux deployments, lots of large companies are also asking for deployments on Windows. In fact, its core market is those people who like both .Net and open source.

Take a look for yourself, and see what they have to say.

Two Years and Counting....

Peter Galli — Wed, 19 Nov 2008 04:16:00 GMT

It is two years this month since Microsoft and Novell struck their ground-breaking technical collaboration agreement, a move that has effectively ensured greater interoperability between Windows Server and SUSE Linux Enterprise Server.

This technical collaboration has already resulted in a number of milestones, including two new offerings announced today: the availability in the first half of 2009 of an Advanced Management Pack for SUSE Linux Enterprise for Microsoft System Center Operations Manager 2007 R2, and a free beta download of Novell's Moonlight, a rich media application.

Some analysts, vendors and enterprises have said the company that develops effective cross-platform management tools will have an advantage and strategic differentiator over its competitors who do not. Microsoft is already doing that.

The Microsoft Operations Manger 2007 Cross Platform Extensions enable the assessment and management of Windows and Linux servers from a single, unified console, eliminating the costs and complexities of having multiple management consoles. The Advanced Management Pack extends this Linux monitoring capability.

Also, given the current tough economic environment, this solution helps reduce training costs since staff only need to be trained on one management tool for both Windows and Linux environments.

Attendees at the Microsoft TechEd EMEA conference in Barcelona earlier this month got to see a technical preview of the Advanced Management Pack, whose release will coincide with that of Microsoft System Center Operations Manager 2007 R2.

A beta of Novell's Moonlight, an open source implementation of Microsoft Silverlight, will also be released going forward as an open source plug-in for the Firefox web browser. Moonlight brings Linux-based users the same high-definition media capabilities currently available for the Windows and Apple environments.

So, expect to see a lot more solutions in the next year that promote interoperability and help ease customer pain-points across their heterogeneous environments.

Open Source Highlights at Microsoft's Professional Developers Conference

hanrahat — Tue, 04 Nov 2008 17:32:00 GMT

At last week's Professional Developers Conference in Los Angeles, product previews and announcements, particularly of Windows Azure, Windows 7 and Visual Studio Team System 2010, generated a lot of buzz. Throughout the week, an important undercurrent to that story was Microsoft's engagement with the open source community and its support for open source developers.

Certainly one of the open source highlights for the week was Miguel de Icaza's presentation, "Mono and .Net." In spite of arriving on time for the session, I wound up viewing it from the second overflow room. Miguel talked about innovations on which the Mono community has focused recently, including their embedded C# compiler. His game demos of the technology were fun and amazing to watch. You can see them for yourself here.

Members of my team spent the month leading up to the conference developing three demonstrations of Windows Azure's ability to support open source developers and open source applications. We ran all three at the Open Source pedestal in the Microsoft booth and each was highlighted in a session during the conference.

One of the demos shows how developers can use Eclipse to create applications and deploy them as Azure services. It relies on an Eclipse community plug-in "Emonics" for displaying C# syntax and a "proof-of-concept" Azure plug-in we created for building and deploying the application. This demo was highlighted in Steve Marx's presentation, "Developing and Deploying Your First Cloud Service," which you can find here.

The second demo shows how an open source application can access services from Azure. In this one, we chose the popular PHP application Gallery and show how it can store, retrieve and modify photos as binary large objects (BLOBs) in the cloud. To produce this demo we wrote two small modules, one to create wrappers that represent the BLOB REST API as PHP objects and another to create an Azure subclass with inheritance from the Windows NT Platform class.

We created the third demo to illustrate how an open source developer can use OpenID to authenticate users from an Azure service. For this one, we modified a demo blog service (based on BlogEngine.net) and gave users the option of authenticating through either OpenID or Live ID.

Both the Gallery and OpenID demos were highlighted in Daniel Wang and Stefan Schackow's presentation, "Cloud Computing: Programming in the Cloud." You can find Daniel and Stefan's presentation here.

The excitement that PDC produced was remarkable. I moved through a steady stream of developers for four days, all of us sustained it seems by tables of fruit, powerbars and various forms of chocolate. I enjoyed meeting and talking with many of the attendees and as always appreciate how much I learn in those conversations.

I want to thank Steve Marx, Daniel Wang and Stefan Schackow for sharing the stage with us. Kudos to Hank Janssen, Anandeep Pannu, Garrett Serack and Joel Penner for creating the Azure demonstrations we used throughout the week. And a tip of my hat to Miguel for making the week so fun.

Is High Performance Computing naturally Open Source (ie. for tinkerers)?

anandeep — Wed, 18 Jun 2008 13:17:00 GMT

I have always been fascinated by clusters. Some people envision working with desktops or workstations when they think of “working with computers”. For me working with computers was always with a large collection of computers in a back room somewhere. And how cool if you could make all those computers collaborate with each other working to solve cool things like genome mapping, movie special effects, simulations of car crashes or simulations of molecules being formed!

So you can imagine I jumped at the chance to work with the Windows High Performance Computing team. This is the same team that builds Windows HPC Server 2008.

I think most of the people working in the team are from the “large collection of computers in back room somewhere” school. Would be really different in the Mac software division I assume!

I work with the Open Source Software Lab and we are all things “Open Source” to the rest of the company. The HPC Server team wanted us to make sure that their product played nice with Linux infrastructure and vice-versa. The usual suspects like AD, Samba, LDAP. CIFS etc were involved. We had to make sure that these recurrent interoperability themes were addressed in the HPC environment. I also got a chance to dig into ROCKS, OSCAR, MPI stacks and job schedulers etc etc.

This was a very rewarding experience not only for the technology exposure that I got but the pervasiveness of knowledge of Open Source within the team. They were far ahead of the other product groups in this regard and “got” the Open Source ethos. In fact, prior to my interactions with them they had released an open source MPI stack based on Argonne National Lab’s MPI implementation.

The other reason was that a lot of their customers were relentlessly open source! The conventional wisdom is that HPC applications and infrastructure require a lot of tinkering. Of course, there are some applications like FEM and CFD and that are well understood, but the general feeling was that complete control and access to the underlying infrastructure is a must for getting the most performance out of a cluster. And performance is the main thing in “High Performance Computing”.

Linux is seen providing that access by HPC customers and there is a large base of Linux for HPC in academia, the national labs and other institutions that use large clusters for doing their thing.

But is this really true?

I think that HPC has gone through a typical evolution – it starts with a few people who have a pressing need. There is a cross disciplinary team formed that builds software to do their job and a community grows around it. The community reaches critical mass and people start making building tools to make it more convenient. ROCKS is an example of this. Great skill, knowledge and ability is needed to get the job done.

However, these skilled people now become overloaded. The tools and the infrastructure that they created become so popular that everyone, including people who do not have background that was assumed before, wants to use it for their ends. So the community responds and builds standardized, easy-to-use infrastructure pieces that start to fit seamlessly together. Some control is lost, but ease-to-use is the primary focus.

The infrastructure for HPC has reached that stage (ROLLS with ROCKS). Windows HPC Server 2008 is built for this ease-of-use too.

However, the applications have not reached the stage of ease-of-use. They have to be coded with a lot of domain knowledge and have to built from scratch to truly scale while running on clusters. That means that the application writers demand more control of the underlying infrastructure and want more access to it than the users and maintainers want.

I am going out on a limb and making a prediction here – soon end users will be able to specify instead of coding applications, be it genome comparison or physics simulation. This is similar to accountants finding spreadsheets. There will probably be a few different models for different types of applications but that stage will come pretty quickly.

The infrastructure that runs these user-specified applications will be adaptive and will take these specifications and automatically tune them for high performance on the clusters.

This is where the perception of needing control to the lowest levels will be moot. The best adaptive infrastructure will be the one adopted.

Bold enough for you?

Technical Analysis: OpenSSH on Linux using Windows/Kerberos for Authentication

jcannon — Fri, 06 Jun 2008 14:35:00 GMT

Abstract: Secure remote access to UNIX and Linux systems is generally accomplished through SSH. The most frequent implementation of that protocol is OpenSSH, originally written for the OpenBSD project but now ported to a wide variety of platforms. This paper will show how to use OpenSSH with the Kerberos portion of Active Directory to automate authentication.

Download OpenSSH on Linux using Windows/Kerberos for Authentication

Note: This paper represents testing and documentation in a lab environment. User Account Control (UAC) is an essential security component to Windows and Microsoft does not recommend turning off UAC in production environments.