Port 25: The Open Source Community at Microsoft : Server Center, Windows Server

Virtualizing Free Linux Distributions in Windows Server 2008 R2

Peter Galli — Tue, 11 Aug 2009 00:00:00 GMT

Jason Perlow, a columnist over at ZDNet, has written a comprehensive review on virtualizing free Linux distributions in Windows Server 2008 R2.

In his Tech Broiler column, Perlow notes that the updated Hyper-V bare-metal hypervisor virtualization layer in Microsoft's upcoming Windows Server 2008 R2, which is due to be released August 14th to MSDN and Technet customers, now has support for SUSE Linux Enterprise Server 11 and Red Hat Enterprise Linux 5.3.

"Additionally, Linux support and performance has greatly improved over the initial Hyper-V release. Microsoft also recently released its Hyper-V Linux Integration Components (Linux ICs) under the GPLv2 Open Source License," Perlow says.

The Linux ICs for Hyper-V, which are in Release Candidate status, provide synthetic device drivers that enhance I/O and networking performance when Linux OSes are virtualized under Hyper-V.

"The source code for the Linux IC's were accepted into the Linux Driver Project and should become part of the Linux Kernel within two subsequent releases and code merges - 2.6.32 is expected to be when they will be integrated, and all Linux distributions using that kernel code base going forward should be Hyper-V enabled out of the box. Yes, you heard that correctly, Microsoft is now an official Linux Kernel contributor," Perlow says.

You can read the rest of Perlow's column here.

Is High Performance Computing naturally Open Source (ie. for tinkerers)?

anandeep — Wed, 18 Jun 2008 13:17:00 GMT

I have always been fascinated by clusters. Some people envision working with desktops or workstations when they think of “working with computers”. For me working with computers was always with a large collection of computers in a back room somewhere. And how cool if you could make all those computers collaborate with each other working to solve cool things like genome mapping, movie special effects, simulations of car crashes or simulations of molecules being formed!

So you can imagine I jumped at the chance to work with the Windows High Performance Computing team. This is the same team that builds Windows HPC Server 2008.

I think most of the people working in the team are from the “large collection of computers in back room somewhere” school. Would be really different in the Mac software division I assume!

I work with the Open Source Software Lab and we are all things “Open Source” to the rest of the company. The HPC Server team wanted us to make sure that their product played nice with Linux infrastructure and vice-versa. The usual suspects like AD, Samba, LDAP. CIFS etc were involved. We had to make sure that these recurrent interoperability themes were addressed in the HPC environment. I also got a chance to dig into ROCKS, OSCAR, MPI stacks and job schedulers etc etc.

This was a very rewarding experience not only for the technology exposure that I got but the pervasiveness of knowledge of Open Source within the team. They were far ahead of the other product groups in this regard and “got” the Open Source ethos. In fact, prior to my interactions with them they had released an open source MPI stack based on Argonne National Lab’s MPI implementation.

The other reason was that a lot of their customers were relentlessly open source! The conventional wisdom is that HPC applications and infrastructure require a lot of tinkering. Of course, there are some applications like FEM and CFD and that are well understood, but the general feeling was that complete control and access to the underlying infrastructure is a must for getting the most performance out of a cluster. And performance is the main thing in “High Performance Computing”.

Linux is seen providing that access by HPC customers and there is a large base of Linux for HPC in academia, the national labs and other institutions that use large clusters for doing their thing.

But is this really true?

I think that HPC has gone through a typical evolution – it starts with a few people who have a pressing need. There is a cross disciplinary team formed that builds software to do their job and a community grows around it. The community reaches critical mass and people start making building tools to make it more convenient. ROCKS is an example of this. Great skill, knowledge and ability is needed to get the job done.

However, these skilled people now become overloaded. The tools and the infrastructure that they created become so popular that everyone, including people who do not have background that was assumed before, wants to use it for their ends. So the community responds and builds standardized, easy-to-use infrastructure pieces that start to fit seamlessly together. Some control is lost, but ease-to-use is the primary focus.

The infrastructure for HPC has reached that stage (ROLLS with ROCKS). Windows HPC Server 2008 is built for this ease-of-use too.

However, the applications have not reached the stage of ease-of-use. They have to be coded with a lot of domain knowledge and have to built from scratch to truly scale while running on clusters. That means that the application writers demand more control of the underlying infrastructure and want more access to it than the users and maintainers want.

I am going out on a limb and making a prediction here – soon end users will be able to specify instead of coding applications, be it genome comparison or physics simulation. This is similar to accountants finding spreadsheets. There will probably be a few different models for different types of applications but that stage will come pretty quickly.

The infrastructure that runs these user-specified applications will be adaptive and will take these specifications and automatically tune them for high performance on the clusters.

This is where the perception of needing control to the lowest levels will be moot. The best adaptive infrastructure will be the one adopted.

Bold enough for you?

Technical Analysis: Security Considerations for rdesktop and Windows Terminal Services

jcannon — Tue, 17 Jun 2008 16:10:00 GMT

Abstract: Microsoft Terminal Services provides an important set of functionality for remote administration and centralized application management. This service allows administrators to log in remotely and with full access to the system. Similarly, users can log in and run specific applications, which are centrally managed by IT personnel. The standard client for Linux systems is rdesktop. Rdesktop is shipped with many Linux distributions and this paper briefly looks at common security considerations around using this client application in Windows environments.

Download Security Considerations for rdesktop and Windows Terminal Services

Note: This paper represents testing and documentation in a lab environment. User Account Control (UAC) is an essential security component to Windows and Microsoft does not recommend turning off UAC in production environments.

Technical Analysis: Remote Administration of Windows Systems with SSH

jcannon — Wed, 11 Jun 2008 13:22:00 GMT

Abstract: SSH has largely replaced Telnet for remote administration of UNIX and Linux systems, but has not yet been used much on Windows. SSH is generally considered to be more secure than Telnet and the Berkeley remote commands (rlogin, etc). This paper uses SSHWindows, a minimal package of Cygwin and OpenSSH. It is available from http://sshwindows.sourceforge.net. The paper is written such that an average Windows system administrator can get an SSH server up while understanding how to make use of security features.

Download Remote Administration of Windows Systems with SSH (PDF)

Note: This paper represents testing and documentation in a lab environment. User Account Control (UAC) is an essential security component to Windows and Microsoft does not recommend turning off UAC in production environments.

Technical Analysis: OpenSSH on Linux using Windows/Kerberos for Authentication

jcannon — Fri, 06 Jun 2008 14:35:00 GMT

Abstract: Secure remote access to UNIX and Linux systems is generally accomplished through SSH. The most frequent implementation of that protocol is OpenSSH, originally written for the OpenBSD project but now ported to a wide variety of platforms. This paper will show how to use OpenSSH with the Kerberos portion of Active Directory to automate authentication.

Download OpenSSH on Linux using Windows/Kerberos for Authentication

Managing Towards Open

Sam Ramji — Tue, 29 Apr 2008 19:59:00 GMT

I have the privilege of interacting almost every day with technical and business experts who are creating the future of software—including both core engineering teams at Microsoft and thought leaders across a broad spectrum of open source communities. Especially in the last few months, I’ve been able to take more time to articulate where I think this is going – such as writing how open source has influenced Windows Server 2008 and participating in Infoworld’s roundtable on the state of open source.

I think that many people are seeing that the interrelationship between Microsoft and open source is being changed fundamentally (and for mutual benefit).

Today, Bob Muglia and Brad Anderson announced that System Center will have the ability to deliver automated management across heterogeneous IT environments, such as UNIX and Linux. What I see as a best practice for commercial and community engagement with open source technology plays a big part in this.

Specifically, Microsoft will deliver an agent infrastructure and management packs (MPs) for monitoring Linux and UNIX platforms for System Center Operations Manager 2007. Early partners like Xandros and Quest are delivering cross-platform MPs for MySQL and Apache, and Oracle, respectively. Microsoft and Novell are collaborating on the SUSE Linux Enterprise MP.

The agent infrastructure Microsoft is building to interoperate with UNIX and Linux is built leveraging industry standards and open source such as WS-Management and OpenPegasus. Pegasus is an open-source implementation of the DMTF CIM and WBEM standards coded in C++, designed to be portable, and licensed under an MIT license, and work is underway to integrate with the newly DMTF ratified WS-Management standard. Pegasus already ships as part of major Linux and UNIX distros.

It simply makes great technical and business sense to cooperate with the OpenPegasus community to build upon an industry-standards based, cross-platform technology. Just as important, however, is preserving the virtuous cycle of contribution, benefit, and subsequent contribution: Microsoft is joining the OpenPegasus Steering Committee. The agent technology—being built will be contributed back to the community under the Microsoft Public License (MS-PL), an OSI approved open source license.

I greatly appreciate Allen Brown's positive comments (Allen is the President and CEO for The Open Group) and the support and education we’ve received from the sponsors and maintainers of Pegasus. He said:

“We are pleased to have Microsoft join the OpenPegasus Steering Committee and welcome their commitment as a positive step for the global open source development community. Since The Open Group initiated the OpenPegasus project seven years ago, it has been deployed across a wide range of IT platforms worldwide. We look forward to Microsoft’s active participation in the continuing development of the project.”

Today’s announcement and the business and technical decisions made by the System Center team are a great example of the fact that commercial innovation, industry partnerships, and open source participation can all work together to make the whole greater than the sum of its parts. My enthusiasm and excitement—and my applause for the System Center team, partners like Xandros, Quest, and Novell, and the OpenPegasus community—is tempered solely by my conviction this is not the only or last example of the best of Microsoft, partners, and open source growing together. This is a great day – and there are more great days to come.

PHP on Windows

hjanssen — Tue, 04 Mar 2008 13:50:00 GMT

I have this really odd feeling of Dejavu. The last blog I wrote (here), I started by saying that I have been very delinquent with writing blogs. So what do I do? I continue to be very delinquent with my blogs!

If anything, I am consistent! :)

So, why this sudden re-emergence of myself on port25? (Besides the fact that I just realized that it has been months since I last blogged, that is) Well we have been keeping ourselves very busy in the last few months. A lot of that work relates directly to PHP, so I wanted to talk about some of these efforts.

We have significantly increased our work in this area. And my group continues to find itself at the middle of pretty much all of these efforts. The SQL Server driver for PHP is now in its second release (Get the latest bits here). And we continue to make improvements and enhancements going forward to this driver.

We have also been working very closely with the PHP community on improving PHP performance on Windows. This is an effort that will be ongoing and probably accelerating in the months to come.

With the release of Windows Server 2008, I wanted to take a moment and highlight some of the things that have been done to make Windows an excellent PHP Deployment platform.

First of all, Microsoft late last year released FastCGI for IIS6 and IIS7. IIS7 is integrated with Windows Server. IIS7 has as part of its deployment FastCGI included (not an optional download as is the case with IIS6) this in effect means that with IIS7, Microsoft has added out of the box Microsoft-supported software designed to run PHP on IIS.

Equally as interesting, we have been working with Zend to help them certify Zend Core for Windows Server 2008. I think this makes it one of the first PHP products to be certified for WS2008. Which makes Windows PHP ready :). Imagine that, PHP running on Windows Server, fully certified!

To this end we will continue to work with Zend very closely to continue to improve PHP on Windows.

Secondarily (although not directly related to Windows Server) we have also been working with Zend to provide Cardspace functionality in the Zend Framework. You can get it here, and read more about how to use it at the Zend website here. Which is another way in which Microsoft has added support to PHP for Microsoft technologies.

This continues the close relationship with Zend into the future, and I am expecting to work with them on a whole host of other PHP related efforts going forward.

As you could read in earlier blogs by Garrett Serrack we hosted the Apache Software Foundation guys here in Redmond last week. So look for a blog from me later this week on the wrap-up of that visit.

Dominic Sartorio on SpikeSource and Open Source Interoperability

Community Contributor — Wed, 27 Feb 2008 23:33:00 GMT

First, many thanks to Microsoft’s Port 25 Team for the opportunity to post today!

Today SpikeSource announced the availability of five additional PHP-based applications on the Windows Server 2008 platform. Gallery, Mantis, Moodle, PhpBB and WebCalendar are available for free download from www.spikesource.com. We welcome you to take a look and to offer your feedback!

Similar to the previous applications we released on Windows Server (Drupal, Alfresco), SpikeSource delivered these as turnkey “SpikeIgnited” applications, with all components needed to run the application available in a single one-click-install distribution. We have also included a variant of our SpikeNet update service optimized for Windows Server 2008.

“That’s great”, you might say, “SpikeSource has ‘Ignited’ a few more apps, so what’s the big deal?” We believe the big deal is that these applications also run on a stack of other Microsoft products in addition to Windows, namely, IIS/Fast-CGI and SQL Server Express, with which many open source products have historically not interoperated well. Quite a bit of engineering went into assembling this “WISP” stack and building/testing these PHP applications, and this know-how is an important step towards improving the interoperability between the worlds of Microsoft and open source.

Why do this? Because customers want it. Throughout SpikeSource’s history, nearly 50% of our customer’s request Windows versions of our open source applications, and most of them also care about interoperating with IIS, SQLServer, Sharepoint, ActiveDirectory and so forth. Our experience is representative of the industry. Ask any commercial OSS ISV with a server-side application, and they’ll tell you the same thing, with similar numbers.

Also, last December, the Open Solutions Alliance (www.opensolutionsalliance.org) published a report (pdf) summarizing the results of its customer outreach efforts. One of the key findings was that customers want better open source and Microsoft interoperability, and moreover, they felt this was the issue that the industry has collectively done the least to address. While there has been a lot of unfortunate history that has gotten in the way of this, ultimately customers don’t care as much about grudges as they care about everything simply working. Together, SpikeSource and Microsoft’s open source lab are doing something about it.

The release of these five PHP applications is just a first step. By taking five commonly used PHP applications and making them run better on a Windows stack, we took a step towards better interoperability, and we also built some technical expertise that we intend to leverage more broadly and share with the community in the future.

So, stay tuned, and we welcome your input. What other types of open source applications are important to run well on Windows? What specific technical issues do you have that you would like to see us solve? What more can we do? Please send us your feedback!

Dominic Sartorio
Sr Director, Product Management, SpikeSource
President, Open Solutions Alliance

How open source has influenced Windows Server 2008

Sam Ramji — Wed, 27 Feb 2008 12:00:00 GMT

When I think about what works really well in open source development and technology, the following things stand out:

Modular architectures
You can find these wherever you see participation at scale – and often a rearchitecture to a more modular system precedes expanded participation. Great examples of this are Firefox, OpenOffice, and X11 – from both the historical rearchitecture and the increased participation that resulted. The Apache HTTP server and APR are good examples that have been modular for as long as I can recall.
Programming language agnostic
A given project uses a consistent language, but there are no rules on what languages are in scope or out of scope. Being open to more languages means opportunity to attract more developers – the diversity of PHP/Perl/Python/Java has been a core driver in the success of a number of projects including Linux.
Feedback-driven development
The “power user” as product manager is a powerful shift in how to build and tune software – and this class of users includes developers who are not committing code back, but instead submitting CRs and defects – resulting in a product that better fits its end users.
Built-for-purpose systems
Most frequently seen in applications of Linux, the ability to build a system that has just what is needed to fulfill its role and nothing else (think of highly customizable distributions like Gentoo or BusyBox, as well as fully custom deployments).
Sysadmins who write code
The ability of a skilled system administrator to write the “last mile” code means that they can make a technology work in their particular environment efficiently and often provide good feedback to developers. This is so fundamental to Unix and Linux environments that most sysadmins are competent programmers.
Standards-based communication
Whether the standard is something from the IETF or W3C, or simply the implementation code itself, where these are used projects are more successful (think of Asterisk and IAX2) and attract a larger ecosystem of software around them.

So where did we apply these ideas to the development of Windows Server 2008?

Modular architectures was applied in multiple areas, but the one that stands out most to me is Internet Information Server 7 (IIS7). IIS7 has been rearchitected for flexibility as 40 individual modules, enable more to be written by community developers or delivered as out-of-band releases. This has already enabled performance improvements and independent evolution, and I expect to see further enhancements.
Programming language agnostic is something we’ve delivered on with support for PHP on IIS7 and the enhancements to FastCGI (which can be used by any of the P* languages). We set a goal of having PHP certified on Windows Server 2008, and we’ve achieved that. We’ll continue to improve runtime, security, and manageability support for non-.NET languages and the applications that are built on them, as well as testing the full stacks of PHP-based applications running on Windows Server, IIS, and SQL Server.
Feedback-driven development based on developer and customer trials (RDPs, TAPs, and Betas in our process) led to a range of “feature completion” developments that connected different components – like connecting Windows Firewall with Active Directory central policy, and the end-to-end improvements in SMB 2.0. Features like the RODC (Read-Only Domain Controller) have become more and more solid through experience with early alpha and beta customer deployments, and requests to enforce things like BitLocker encryption of user disks from a central authority have achieved full support.
Built-for-purpose systems such as DNS, DHCP, file and web serving can be created through wizard-driven configuration thanks to Windows Server Core. The goal of having a minimum attack surface and a small hardware footprint, inspired by the capabilities mentioned above, yet achievable by a broad base of admins has been achieved. Additionally, this has created an opportunity for Windows admins to become much more knowledgeable about the low-level structure of the operating system.
Sysadmins who write code are first-class citizens in the PowerShell-driven infrastructure. We’ve increased Windows administrators’ opportunity to master the full surface area of WMI and demonstrate that mastery in reusable, low-level scripts. As we evolve this to support multiple language bindings and bash aliasing, this should become a comfortable home for highly skilled sysadmins.
Standards-based communication such as in CardSpace (with support for X.509, SAML, Kerberos tokens, and more) and the Web Services stack (not only are all 38 Web Services standard under the Open Specification promise, but our implementations have achieved a high level of interop with Apache’s Axis web services stack), and beta support for emerging standards like Xen virtualization represent a small subset of the standards built into Windows Server 2008.

Overall, we’ve learned and continue to learn from open source development principles. These are making their way into the mindset, development practices, and ultimately into the products we bring to market.

I’ve focused here on “what Microsoft has learned from Open Source” – and ironically, I’ve agreed to do a panel at OSBC on 3/25 with Jim Zemlin of the Linux Foundation on “what Open Source can learn from Microsoft”. As all of the different organizations in IT continue to evolve, we’ll learn from each others’ best practices and make increasingly better software. As in science, this incremental improvement will move all of us forward.

Technical Analysis: Apache with mod_auth_kerb and Windows Server

jcannon — Fri, 25 Jan 2008 21:58:00 GMT

Abstract: The Apache authentication module mod_auth_kerb allows Apache to authenticate users against a Kerberos KDC including one from ActiveDirectory. Kerberos itself can be fairly complex to set up. This guide will attempt to show the specific steps required to make this possible as well as discuss security limitations specific to the interoperability matters. This guide assumes a basic understanding of Kerberos V and that the Active Directory domain controller is properly configured prior to starting this process.

Technical Analysis: Installing Apache with SSL on Windows

jcannon — Fri, 04 Jan 2008 17:02:00 GMT

Abstract:
Often SSL or TLS is required to secure data from web applications. Sometimes this is just prudent to prevent confidential or sensitive data from being confiscated. Sometimes this is required by regulations like HIPAA or industry bodies, such as the Payment Card Industry. This guide will show how to install Apache with SSL on Windows.

Download Report Installing Apache on Windows

Core

billhilf — Thu, 07 Jun 2007 18:24:00 GMT

We recently announced the addition of IIS7 to the Server Core installation of Windows Server 2008 (formerly known as Longhorn Server). Server Core is an important evolution of our server product and will include a variety of roles, such as print server, media services, Active Directory, DNS, DHCP, and now IIS7 for Web serving. All of these will be able to run in a lightweight, low footprint modes – a server core installation requires about 1GB of physical disk space to install and approximately 2 GB for operations post-install. This means it’s Windows Server but with just the bits you need to run a specific type of server role – which means less disk, less memory, lower attack surface, less stuff to manage, patch, etc. There are also a variety of optional features you can add to server core, such as the subsystem for Unix applications, Bitlocker drive encryption, failover cluster, and others. Of course, Windows Server 2008 can still run as a full featured general purpose server operating system as well. Sam and Hank did an interview with Iain McDonald about Server Core last September, you can see that interview here.

I’ve built and run many Web server farms over that past years and having the ability to roll out small footprint, role-based server configurations is something I found to be an important architectural advantage. In the past, I used Apache on Linux/BSDs to build customized servers. Certainly you can still do this today. What I think is exciting about this announcement of IIS7 on Windows Server 2008 Server Core is that it shows the full spectrum of the Windows Server 2008 capabilities, from very modular, low footprint Web serving to the all-singing all-dancing full featured server.

Additionally, as you may have read here before, we’ve been working hard with the Zend on making PHP run great on Windows Server. With the new FastCGI support (which is now integrated with IIS7 in beta 3), PHP runs extremely well on Server Core. So if you need a tier of streamlined front-end PHP Web servers that require minimal system resources and just the needed bits for doing the job? Now you will be able do this with Windows Server 2008. And those systems can be managed, secured, updated, authenticated, etc. just like any other Windows server machine.

I think this all brings more choice to developers and system administrators. And you can expect this will be something we continue to evolve, adding more customization scenarios and support of other technologies, including .NET. Check out Bill Staple’s blog on this as well – his team is doing all the IIS work. You test drive Windows Server 2008 here or download the latest beta here. The FastCGI Technology Preview can be found here.

Unrelated and Personal (non-work related) tidbit:

Talking about server core and IIS7 reminded me of this quote: “The future you have tomorrow won’t be the same future you had yesterday” from Chuck Palahniuk’s latest book Rant. I think it’s his best so far -I just finished this on my last trip. If you like Chuck, watch this lecture – the ‘little software story’ advice at the end is priceless and motivational for aspiring writers.

What Lies Beneath: Setting up underlying HPC tools

kishi — Thu, 21 Dec 2006 22:34:00 GMT

This blog continues what I started writing about w/ Thinking About HPC Infrastructure and what Frank wrote in about in Overloading Clusters.

After reading thru the previous blogs on HPC, someone might ask “What are some of the core components of HPC ?”. After all, once you’ve seen the outside of a Maserati or a Pantera DeTomaso, you’re not going to be satisfied just by ogling at it. Even after a test drive, the engineer in you will want to pop the hood and see what’s inside. Taking a similar approach let’s uncover some underlying HPC technologies by looking at any basic HPC setup. Once all the provisioning has been completed, the HPC system will be physically deployed with an OS and relevant drivers, utilities etc. Yet, before the actual HPC application can get installed across, there remains a critical step in the process, i.e. configuration of cluster and file system along with any tools and interfaces such as MPI (Message Passing Interface) etc. After peeling through the HPC application layer, its worthwhile to do a “deep-dive” into what really runs the HPC clusters. A broad category of these tools are:

Cluster Management tools e.g. CSM
Job Scheduling tools e.g. SCALI, Maui
Resource Management tools e.g. Torque

If you’re trying to understand the “WHY” behind the existence of these tools and their importance, take a look at Cluster Management for example. Cluster configuration, installation and management can be difficult and requires intimate familiarity with the HPC hardware, OS, underlying architecture etc. Without specific tools that attend to and manage specific underlying HPC sub-components, HPC just won’t be what it is. So, it is worthwhile to understand the unique installation experience of the tools, such as the ones listed above to understand the complexity of HPC systems. Ready – let’s dive in to the installation and function of these tools:

1. SCALI: The SCALI management and MPI software packages provide deployment, monitoring and job scheduling services for a cluster. After you deploy this software, you will be able see all the compute nodes that may have been preconfigured or are configured on your system. Scali will enable you to monitor the systems and run jobs using the SCALI graphical interface. In order to license the SCALI software, you must utilize the scainstall command to produce a license request file. This file can then be sent to SCALI to receive a permanent key. For those that need some hand-holding through this, luckily SCALI provides very comprehensive documentation on their website. A large portion of the SCALI Manage User’s Guide is dedicated to pre-setup planning and configuration of the cluster and the network. The documentation provides detailed recommendations about how you can set up their Ethernet-based network environment and out-of-band management network. The documentation also provides a general overview about how to install and configure higher performance interconnects, including bonded Ethernet, Infiniband, Myrinet and SCI. The SCALI Manage interface provides simple tools to assist in configuring and testing DET, Infiniband, and Myrinet devices for use with the SCALI MPI implementation. The SCALI MPI software supports multiple Infiniband stacks including Mellanox, Topspin, Voltaire and Infinicon.

2. HP-MPI: HP-MPI is Hewlett-Packard’s Linux-based implementation of the Message Passing Interface (MPI). Many of the utilities distributed with HP-MPI are similar to other common MPI utilities such as MPICH - e.g. mpicc, mpirun, etc. In order to utilize the HP-MPI software, a license is required for each CPU core in the cluster. To obtain a license file you are required to obtain the MAC address from each node (typically eth0) and input that information into a form at licensing.hp.com. The resulting file can then be copied to the compute node. The HP-MPI software is non-functional until licensing files are generated for the nodes

3. CSM (Cluster Systems Management): The CSM software suite is designed to automate the deployment and management of cluster nodes. Nodes can be remotely installed with an operating system as well as the CSM software for later monitoring. The CSM software supports RedHat and Novell on multiple platforms. In order to obtain and install the CSM software one must register with IBM’s website and download the required RPMs. In order to configure CSM, it can remotely install the operating system and/or the CSM software on the compute nodes. Much like Platform ROCKS, CSM makes use of PXE functionality and RedHat’s kickstart or the autoyast software to remotely install the operating system. The CSM software provides multiple methods for defining the nodes that should be deployed and managed:

a. The first method involves creating a hostname mapping (hostmap) file, which is a colon-delimited file that defines a number of attributes of each node
b. The second method also involves manually creating and editing a “node definition” (nodedef) file. This is the method suggested by the documentation for use with small clusters

Proper remote power and remote console capabilities greatly ease the administration and deployment of the compute nodes, however according to the CSM FAQ remote power management is not absolutely required. All the compute nodes must be rebooted (remotely or manually). They are then PXE booted and installed with RHEL4 using the kickstart installation system.

4. Maui and Torque: Both Torque and Maui are free software which must be compiled from the source distribution on the head node. Maui is an open-source job scheduler for compute clusters. It supports a number of task management features not found in other parallel batch processing software including policy-based scheduling and prioritization of tasks. Torque is an open-source resource manager for managing compute nodes and scheduled jobs. It can integrate with Maui to provide additional features for scheduling and managing scheduled tasks. Installation of Torque can be done using the guidance available in the Torque 2.0 Admin Manual .

5. Platform Rocks: Platform Rocks is a cluster deployment software that facilitates the deployment of various software stacks (“rolls”) onto the compute nodes. The software is capable of deploying the base operating system and utilities required for cluster administration, management and scheduling. The software can also manage configuration and updates to ensure consistency throughout the cluster. Platform Rocks is a suite of utilities that are packaged together as separate installable rolls. One of the main goals of the software is to allow for easy installation and integration of third-party rolls and applications. One unique aspect to the Platform Rocks installation approach is that the software installs an operating system on the head node, and also installs all the required rolls at the same time. The software can also automatically set up the subsystem required to install an operating system and other packages on the compute nodes (such as management agents, etc).

That about does it for a quick “deep-dive”. Let me insert a gentle reminder that these are not the only cluster or resource management technologies out there in the HPC space but rather the ones most prevalent. If you have additional tools that you have worked with, we’d like to hear from you and thank you for tuning in to Port 25. HAPPY HOLIDAYS!

Windows Unified Data Storage Server

MichaelF — Wed, 06 Dec 2006 00:36:00 GMT

Today Microsoft announced the availability of the Windows Unified Data Storage Server 2003 (WUDSS). Built for mixed environments this solution is highly interoperable, without loss of performance in both NFS and CIFS environments. Remote administration is also provided both through Active X RDP and Java RDP Client Support.

This sounded interesting so we sent Hank to talk with Tres Hill, Sr Product Manager, to find out why this announcement is significant for IT professionals running heterogeneous environments.

Video: Tres Hill on Windows Unified Data Storage Server

Windows and Linux Integration: Printer Sharing between Windows and Linux

MichaelF — Mon, 16 Oct 2006 22:13:00 GMT

Last Thursday we posted an interview with Jeremy Moskowitz in which we mentioned the book he authored: Windows and Linux Integration.

As promised, today we have posted a complete chapter from his book dealing with: Printer Sharing between Windows and Linux

Please respond with any questions or comments, we'll see if we can get Jeremy back on to answer your feedback.