Port 25: The Open Source Community at Microsoft : Technical Analysis, Windows Server

Technical Analysis: Apache with mod_auth_kerb and Windows Server

jcannon — Fri, 25 Jan 2008 21:58:00 GMT

Abstract: The Apache authentication module mod_auth_kerb allows Apache to authenticate users against a Kerberos KDC including one from ActiveDirectory. Kerberos itself can be fairly complex to set up. This guide will attempt to show the specific steps required to make this possible as well as discuss security limitations specific to the interoperability matters. This guide assumes a basic understanding of Kerberos V and that the Active Directory domain controller is properly configured prior to starting this process.

Note: This paper represents testing and documentation in a lab environment. User Account Control (UAC) is an essential security component to Windows and Microsoft does not recommend turning off UAC in production environments.

Technical Analysis: Installing Apache with SSL on Windows

jcannon — Fri, 04 Jan 2008 17:02:00 GMT

Abstract:
Often SSL or TLS is required to secure data from web applications. Sometimes this is just prudent to prevent confidential or sensitive data from being confiscated. Sometimes this is required by regulations like HIPAA or industry bodies, such as the Payment Card Industry. This guide will show how to install Apache with SSL on Windows.

Download Report Installing Apache on Windows

What Lies Beneath: Setting up underlying HPC tools

kishi — Thu, 21 Dec 2006 22:34:00 GMT

This blog continues what I started writing about w/ Thinking About HPC Infrastructure and what Frank wrote in about in Overloading Clusters.

After reading thru the previous blogs on HPC, someone might ask “What are some of the core components of HPC ?”. After all, once you’ve seen the outside of a Maserati or a Pantera DeTomaso, you’re not going to be satisfied just by ogling at it. Even after a test drive, the engineer in you will want to pop the hood and see what’s inside. Taking a similar approach let’s uncover some underlying HPC technologies by looking at any basic HPC setup. Once all the provisioning has been completed, the HPC system will be physically deployed with an OS and relevant drivers, utilities etc. Yet, before the actual HPC application can get installed across, there remains a critical step in the process, i.e. configuration of cluster and file system along with any tools and interfaces such as MPI (Message Passing Interface) etc. After peeling through the HPC application layer, its worthwhile to do a “deep-dive” into what really runs the HPC clusters. A broad category of these tools are:

Cluster Management tools e.g. CSM
Job Scheduling tools e.g. SCALI, Maui
Resource Management tools e.g. Torque

If you’re trying to understand the “WHY” behind the existence of these tools and their importance, take a look at Cluster Management for example. Cluster configuration, installation and management can be difficult and requires intimate familiarity with the HPC hardware, OS, underlying architecture etc. Without specific tools that attend to and manage specific underlying HPC sub-components, HPC just won’t be what it is. So, it is worthwhile to understand the unique installation experience of the tools, such as the ones listed above to understand the complexity of HPC systems. Ready – let’s dive in to the installation and function of these tools:

1. SCALI: The SCALI management and MPI software packages provide deployment, monitoring and job scheduling services for a cluster. After you deploy this software, you will be able see all the compute nodes that may have been preconfigured or are configured on your system. Scali will enable you to monitor the systems and run jobs using the SCALI graphical interface. In order to license the SCALI software, you must utilize the scainstall command to produce a license request file. This file can then be sent to SCALI to receive a permanent key. For those that need some hand-holding through this, luckily SCALI provides very comprehensive documentation on their website. A large portion of the SCALI Manage User’s Guide is dedicated to pre-setup planning and configuration of the cluster and the network. The documentation provides detailed recommendations about how you can set up their Ethernet-based network environment and out-of-band management network. The documentation also provides a general overview about how to install and configure higher performance interconnects, including bonded Ethernet, Infiniband, Myrinet and SCI. The SCALI Manage interface provides simple tools to assist in configuring and testing DET, Infiniband, and Myrinet devices for use with the SCALI MPI implementation. The SCALI MPI software supports multiple Infiniband stacks including Mellanox, Topspin, Voltaire and Infinicon.

2. HP-MPI: HP-MPI is Hewlett-Packard’s Linux-based implementation of the Message Passing Interface (MPI). Many of the utilities distributed with HP-MPI are similar to other common MPI utilities such as MPICH - e.g. mpicc, mpirun, etc. In order to utilize the HP-MPI software, a license is required for each CPU core in the cluster. To obtain a license file you are required to obtain the MAC address from each node (typically eth0) and input that information into a form at licensing.hp.com. The resulting file can then be copied to the compute node. The HP-MPI software is non-functional until licensing files are generated for the nodes

3. CSM (Cluster Systems Management): The CSM software suite is designed to automate the deployment and management of cluster nodes. Nodes can be remotely installed with an operating system as well as the CSM software for later monitoring. The CSM software supports RedHat and Novell on multiple platforms. In order to obtain and install the CSM software one must register with IBM’s website and download the required RPMs. In order to configure CSM, it can remotely install the operating system and/or the CSM software on the compute nodes. Much like Platform ROCKS, CSM makes use of PXE functionality and RedHat’s kickstart or the autoyast software to remotely install the operating system. The CSM software provides multiple methods for defining the nodes that should be deployed and managed:

a. The first method involves creating a hostname mapping (hostmap) file, which is a colon-delimited file that defines a number of attributes of each node
b. The second method also involves manually creating and editing a “node definition” (nodedef) file. This is the method suggested by the documentation for use with small clusters

Proper remote power and remote console capabilities greatly ease the administration and deployment of the compute nodes, however according to the CSM FAQ remote power management is not absolutely required. All the compute nodes must be rebooted (remotely or manually). They are then PXE booted and installed with RHEL4 using the kickstart installation system.

4. Maui and Torque: Both Torque and Maui are free software which must be compiled from the source distribution on the head node. Maui is an open-source job scheduler for compute clusters. It supports a number of task management features not found in other parallel batch processing software including policy-based scheduling and prioritization of tasks. Torque is an open-source resource manager for managing compute nodes and scheduled jobs. It can integrate with Maui to provide additional features for scheduling and managing scheduled tasks. Installation of Torque can be done using the guidance available in the Torque 2.0 Admin Manual .

5. Platform Rocks: Platform Rocks is a cluster deployment software that facilitates the deployment of various software stacks (“rolls”) onto the compute nodes. The software is capable of deploying the base operating system and utilities required for cluster administration, management and scheduling. The software can also manage configuration and updates to ensure consistency throughout the cluster. Platform Rocks is a suite of utilities that are packaged together as separate installable rolls. One of the main goals of the software is to allow for easy installation and integration of third-party rolls and applications. One unique aspect to the Platform Rocks installation approach is that the software installs an operating system on the head node, and also installs all the required rolls at the same time. The software can also automatically set up the subsystem required to install an operating system and other packages on the compute nodes (such as management agents, etc).

That about does it for a quick “deep-dive”. Let me insert a gentle reminder that these are not the only cluster or resource management technologies out there in the HPC space but rather the ones most prevalent. If you have additional tools that you have worked with, we’d like to hear from you and thank you for tuning in to Port 25. HAPPY HOLIDAYS!

Recovering remote NT/W2K/XP desktops with a network boot CD/DVD

admin — Thu, 27 Apr 2006 21:46:00 GMT

Question:
re: Welcome to Kishi's Korner
Monday, April 17, 2006 8:04 AM by Les Kobiernicki

When can we expect to see M$ produce a composite Win/Linux real boot recovery CD/DVD like Bart PE & Ultimate Boot CD for Win ? I have many legacy systems to keep up & running til we get the scheduled Tech Refresh that gets put back further & further. A network boot CD/DVD with multicasting server ability to recover remote NT/W2K/XP desktops would be most helpful. The answer is not necessarily always new technology, but more precisely targetted troubleshooting tools for what we already have deployed out there ..

Answer:
If you are looking for a good Open Source solution for Imaging and recovery one way to do this is by using: g4u (Ghost for UNIX) http://www.feyrer.de/g4u/ . Based on NetBSD, G4u is a bootable floppy/CD for cloning and imaging hard disks and partitions.

If you have a mixed environment, which most of us do, you might wonder what file or operating systems it can handle. The answer is all of them. G4u reads the disks bit by bit starting with byte #0. This includes any MBR, boot record, partition table and the partitions themselves. G4u can as easily clone a Windows XP disk as a Linux or Solaris/X86 disk. By moving the hard disks to a PC, g4u can even deploy or image operating systems for non-PC based SCSI machines such as HP-UX, Solaris, Irix, and AIX. You can image a drive or partition locally, IE disk to disk, or have the image uploaded to an ftp server. The cloned images can be compressed to save space, however the compression isn’t nearly as good as some of the commercial alternatives so make sure your ftp server has plenty of space! If space is a concern, be sure to check out the FAQ on G4u’s website. http://www.feyrer.de/g4u/#hints

Here is a quick example. I recently imaged my Fedora Core 5 laptop to a local ftp server here in my office.

Once I booted my laptop up with the g4u CD, I was at the main menu and the command prompt.

g4u>

The laptop only has one hard disk. I used the ‘disks’ command to see it.

g4u> disks
wd0: at atabus0 drive 0: <FUJITSU MHT2060AT PL>
wd0: drive supports 16-sector PIO transfers, LBA addressing
wd0: 57231 MB, 116280 cyl, 16 head, 63 sec, 512 bytes/sect x 117210240 sectors

I wanted to image the IDE disk (wd0) to my ftp server (192.168.1.1) using the ftp account ‘images’. I typed the following command.

g4u> uploaddisk images@192.168.1.1 fc5laptop.gz wd0

I entered in my password when prompted.

This took a while on a 100MB connection, a couple hours or so. I think I went and got coffee while it was running. Ok, so now on my ftp server I have the file fc5laptop.gz.

$ ls –l
-rw------- 1 images images 20259936597 Apr 18 12:18 fc5laptop.gz

To recover the image I booted again with the g4u CD and at the command prompt typed:

g4u> slurpdisk images@192.168.1.1 fc5laptop.gz wd0

Again I entered my password for the ftp server when prompted and went for coffee (anytime is a good time for coffee J )

After about an hour my laptop was restored. I ejected the g4u cd and rebooted.

G4u doesn’t try to do everything but what it does do, it does very well.

Managing the Lab: SMS and our mixed environment

admin — Fri, 31 Mar 2006 15:00:00 GMT

Getting the Open Source Software Lab up and running presented a number of challenges – not the least of which was how we were going to manage fifty Linux distributions, fifteen versions of UNIX, and multiple Windows instances deployed across literally hundreds of physical and virtual servers. This is quite a job for any management solution. Being the pragmatists we are, we decided to use this to test the viability of SMS (Microsoft Systems Management Server) using VMX (Vintela Management Extensions) in a mixed environment.

We deployed the solution and found it to be capable of handling our environment. Currently a large part of the lab is managed by SMS and VMX. When we describe this to people we are often asked, "Why does Microsoft supports this kind of solution? Why do we care about mixed environments?"

We asked Bill Anderson, Lead Program Manager on the Windows Management Team, and here is what he had to say:

Bill Anderson
Not really, but his lab is less camera-shy.

The first question I always get asked is, “what really was the catalyst for SMS to seek out a partner to provide extensions to OSS/Linux? Simple – our customers demanded it. Our existing SMS customers are managing both desktops and servers, and have a multitude of platforms in production in those environments and wanted to extend the success they have with SMS on Windows to those additional platforms. And, as we’re driving SMS into new customers, it has become one of the top requirements for customers – an integrated solution to manage all their critical platforms.

Now, the second driver was the WAY in which the market was doing cross-platform management. It’s, well, “suboptimal”. You either take 2 management systems (Windows mgmt, non-Windows mgmt) with their own array of servers, agents, and databases – and join the databases, or you try to take one agent that runs on all platforms, and you can then only join the things that are the same/similar. You either get a bunch of extra infrastructure with no leverage of skillsets, or you get a lowest common denominator management experience.

What we did was option 3 – build a single shared infrastructure that was extensible at the protocol, data, and UI layer, and then take the 2 leaders in the field to build from that same plumbing. So, we optimized our agents for the work on Windows, and we worked with the Vintela team as the experts in managing OSS/Linux to really optimize their experience for that platform. So, what does a customer get? 1 database, one UI, one protocol, and agents unique to each platform. Low operational cost, leveraged skillsets, and the opportunity for each vendor to really highlight the best they could do on each platform. Some of the things that Vintela can surface and manage on the Linux platform, using SMS as a pipeline, are pretty amazing! They’ve extended our UI to really expose all the remote functions available on Linux from the different vendors like Red Hat, SuSE, HP, and Sun. My challenge to them was to make Linux look BETTER in SMS than Windows does. We’ll try to make Windows more manageable by adding more, not by restricting. And the results are pretty compelling. As Andi put it in Network World's Network/Systems Management Newsletter: “Yes, you read that correctly - Microsoft tools can make Linux management easier. To its credit, Microsoft has made this easier through partnerships and programs like its Dynamic Systems Initiative - a commitment from Microsoft and its partners to deliver self-managing dynamic systems…(snip). This allows enterprises to leverage their investment in native Windows tools to make them a very effective management platform for diverse networks. “

How the Vintela solution works is really pretty simple. They take a WBEM based agent (they are the project maintainer for OpenWBEM) that runs on the major OSS platforms, that points at a URL that is our Management Point role. They extend our MMC based UI and voila – instant management for Linux! No database schema changes required, no separate middle infrastructure, etc. Initially, there was an ISAPI.dll “gateway” they had built to convert their agent protocol to ours at the Management Point, but we’ve worked to even eliminate that as they are now using our native protocols. As you can see, this is a slam dunk for a customer using SMS already to manage Windows that wants to extend it to manage Linux. It’s amazing to walk to an SMS admin, open their admin UI, have them see machine collections based on Linux versions/vendors, and be able to send software to a group of Linux machines in about 3-4 clicks. But, we’re even seeing customers use THIS as a solution for managing Linux only! Vintela has done a great job of really just using the SMS UI, database, and pipes as their engine, and leveraging all the

manageability on the native Linux platform to provide a great stand-alone tool for managing Linux. Inventory, software distribution, patch management and remote tools – all in one single UI and infrastructure. The other key is really leveraging the OpenWBEM work to provide consistent management on different Linux versions. The Vintela team has done a great job of driving consistency via OpenWBEM, but still leverage all the extra tools and functions provided by each Linux vendor. If I were managing Linux systems (not a lot of that around here by the way!) I’d definitely use it!

For more info, I’d also suggest taking a peek at the EMA document they did for Vintela at http://www.vintela.com/products/vmx/docs/Managing_Heterogeneous_IT_with_SMS_EMA.pdf.