Port 25: The Open Source Community at Microsoft : Hank Janssen, Community

Another Great Step Forward for Hyper-V, Red Hat Certification

hjanssen — Wed, 07 Oct 2009 17:06:00 GMT

Hello again! It has been a pretty busy couple of months for us, and I wanted to give you an update on what we've been doing.

We just completed the first step in another major milestone for Hyper-V. As you can read from Mike Neil's Blog, Microsoft and Red Hat just completed certification in each other's virtualization program.

This means that customers now can deploy Microsoft Windows Server and Red Hat Enterprise Linux and a range of select applications, virtualized on Red Hat and Microsoft hypervisor virtualization software, knowing that the solutions will be supported by both companies.

This again demonstrates Microsoft's commitment to its virtualization platform, and the Hyper-V team's continuous investment in interoperability and heterogeneity through the datacenter.

And I am very proud that my team has been working very hard on getting the Red Hat certification completed.

But, for my group, this is just the first step: we worked on and got certified for Hyper-V running Red Hat in emulated mode and, now, the next step for us is to get certification for enlightened mode, the mode where the guest OS is Hyper-V aware,and can thus access Hyper-V functionality directly.

So, what is needed for enlightened mode? Well, to get that step, we needed to get the Linux Integration Drivers submitted to the Linux kernel, and then we needed those drivers officially accepted in a mainline kernel.

This has now happened as well, as we have been accepted into the mainline kernel. We are in Linux Kernel release 2.6.32, and that release is currently going through development and testing. Once that one is final and officially released, we can take the next step, which is to get those official Linux Integration Component drivers certified with Red Hat.

The timeline for that is not completely set in stone right now, and I do not know if there is an official expected release date for 2.6.32 as yet. But we should be able to move forward with the enlightened mode certification soon after the 2.6.32 final release.

And we are, of course, continuing to work on the Linux Integration components, adding new features and all that good stuff. But I will write more about those items in the near future.

Introduction to the Linux Integration Components

hjanssen — Wed, 22 Jul 2009 19:02:00 GMT

Well, here is blog number two. The initial shock has worn off a bit I hope.

The feedback I have received so far has been pretty positive. This really all started in October of 2008 in a meeting with Mike Neil (GM of Hyper-V) and Tom and myself from the Open Source Technology Center (OSTC) at Microsoft.

In that meeting I proposed to Open Source the Linux Integration Components and contribute them to the Linux Kernel. And, secondly, to have the OSTC continue contributing to these IC's after they made it to the Linux Kernel. Well after some discussion, we all agreed that this was the right thing to do.

And so the whole process started inside of Microsoft. Hey, what can I say, we like to push the envelope a bit here at the OSTC, and we have a reputation to uphold!

Before I go on, I again wanted to thank the Kernel community (specifically Greg Kroah-Hartman) in helping us with explaining and guiding us through community process. It gave us a very nice jumpstart to get all of this going, and provided the groundwork for a good working relationship with the community.

I have also seen a few patches already submitted by community members, which is excellent! (Moritz Muehlenhoff gets major kudos for the first community contributed patch ) I will start submitting patches myself next week once the initial submission has stabilized a bit.

It is my plan to use the kernel as my primary development area, and of course I will continue to provide Greg with my patches. My first step is to clean up the code to make sure it fulfills all Kernel coding standards and requirements.

So, here is blog number two: what are the Linux Integration Components?

1. Overview of Linux VM with ICs

Linux Integration Components(IC) take advantages of the VMBUS and synthetic devices provided in Hyper-V to enhance the performance and usability of Linux guests running on Windows servers.

Figure: Conceptual Architecture overview of Linux guest & Hyper-V. Linux IC modules are painted in yellow color.

Glossary:

VSP: Virtualization Service Provider.

VSC: Virtualization Service Client.

VMBus: Data channel between VSP and VSC.

2. Linux IC modules -- VMBus and VSCs

Communication with parent partition is done through Linux VMBus
VSCs are the Linux drivers for synthetic devices (SCSI, IDE, and Ethernet) provided by Hyper-V.
They translate between Linux I/O requests and Hyper-V VSC commands
Devices are registered with Linux Driver Model (LDM)
Every VSC module contains two portions:
o Driver Interface Mapper (DIM): Released as open source

This portion of the VSC component interacts with the Linux kernel like a regular Linux device driver.

o VSC Core: Released as Open Source

The core portion of the VSC module is implemented based on the protocol of the corresponding VSP at Hyper-V host. The VSC core interacts with VSP via the VMBus interface.

3. Descriptions for each Linux IC module

3.1 VMBus driver (hv_vmbus.c)

The VMBus driver is a Linux kernel module. It provides both a lightweight bus driver and library functionality. As a bus driver, it registers with Linux Driver Model framework (LDM) to provide simple bus and device integration and device tree integration (sysfs). As a library, it implements the VMBus channel protocol and provide an abstraction of channel to its clients (Disk and Network VSCs).

3.2 StorVSC driver (hv_storvsc.c)

The Storage VSC interacts with the Windows Storage VSP. The "wire" protocol defined by the storage VSP determines how a VSC interacts with it. The Linux Storage VSC (LSVSC) basically abstracts the Linux I/O stack from needing to understand the Storage VSP's protocol. At the upper-edge of the LSVSC, it talks to the Linux SCSI subsystem. The Linux SCSI subsystem sees the LSVSC as a SCSI low-level driver (LLD) in Linux parlance. It passes SCSI requests (scsi_cmnd) to LSVSC which in turn converts them into the "wire" format understood by the Windows Storage VSP (VSTOR_PACKET). The bottom-edge of the LSVSC talks to Linux VMBus (LVMBUS) which in turn talks to the Windows VMBus to route the packets to the Storage VSP.

3.3 BlkVSC driver (hv_blkvsc.c)

BlkVSC (BlockVSC) supports "fast boot" and fast access to IDE disks. To enable enlightened IDE support for enhancing the performance of Linux when virtualized on Windows, a separate BlockVSC component is used as a Linux block device driver. Like StorVSC, the BlockVSC component is comprised of an upper edge wrapper that interfaces with the Linux block layer and a lower-edge through the infrastructure modules. The infrastructure modules with Hyper-V through the Linux VMBus.

3.4 NetVSC driver (hv_netvsc.c)

The network VSC send and receive network traffic between a Linux guest and Hyper-V host which has direct connection to physical network. The mechanism that this is used to accomplish is the Remote NDIS (RNDIS) protocol. Thus the communication that flows between the VSP and the VSC primarily happens over the RNDIS protocol which then is packaged and forwarded as payload over to the other side over NetVSP / VMBus protocol.

4. Linux IC's, Location in the Kernel tree

Hopefully you now have a better idea what they are. But where in the kernel tree can you find them?

Well, you can find sources in linux-next tree in /drivers/staging/hv directory.

And the git repository you can find them in right now is:

git://git.kernel.org/pub/scm/linux/kernel/git/sfr/linux-next.git

Or give this command (assuming your system is set up correctly) to download this repository to your machine:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/sfr/linux-next.git <your local name>

Since the IC's are part of the kernel now, we follow the normal community process of getting this all migrated into Linus mainline kernel.

More on the Hyper-V Linux Integration Components

hjanssen — Mon, 20 Jul 2009 20:24:00 GMT

Well, there is no easy way to say this, so I am simply going to start this blog with the following line.

Microsoft just submitted source code for the Hyper-V Linux Integration Components to the Linux Kernel Community Under GPL v2.

Well, there's a conversation starter! Are you still all sitting in your chairs???

Let me summarize:

Yes, our device driver code was released directly to the Linux Kernel
We released the code under GPL v2
We are working with Greg Kroah-Hartman so it is ready for the next release of the Linux Kernel, version 2.6.32
We will continue to update the driver code to enhance interoperability on an ongoing basis, but it's our hope that other developers in the community will find the code useful and worthy of collaboration.

Fallen off your chair yet?

Microsoft developed the Linux device drivers to enhance the performance of Linux when virtualized on Windows Server 2008 Hyper-V. My team and I were responsible for testing and validating the driver components that were contributed for this first release.

Now, my team and I will be responsible for further developing this code going forward. (Yes, that does mean that I have gone back to leverage my very early roots as a Kernel programmer. Let the world be warned!!!!). Haiyang Zhang has been working on this code with me, and he will continue to work with me on this going forward.

When I joined Microsoft three years ago, the primary reason was to put my money where my mouth was. You see complaining about something is easy, but it becomes a little more complicated when somebody offers you the opportunity to be part of helping change what you have complained about.

So, three years after taking the job that made me put my money where my mouth was (and still often is!), I for one am EXTREMELY happy to see one of the most significant fruits of our work here in the Microsoft Open Source Technology Center (OSTC). But I have to say, even I would have been hard-pressed to think three years ago that we would consider contributing to the Linux Kernel.

As you know, two years ago Microsoft announced a partnership with Novell, and Tom Hanrahan ran the lab on a day to day basis till about 9 months ago. Since then I have had the pleasure of running the technical side of the execution of that lab under Tom Hanrahan for the OSTC. One of the primary tasks for that lab is to make sure Windows runs well on top of XEN and Linux runs well on top of Hyper-V, and we do this in very close cooperation with Novell.

We do most of this work as an extension to Mike Neil's Hyper-V team.

As part of this, we were asked to help develop and maintain a crucial part of this work called the Linux Integration Components. This code is designed so that Linux can run in an "enlightened mode" on top of Hyper-V (enlightened mode is roughly the Hyper-V equivalent of "paravirtualized mode" for the Xen hypervisor). Without this driver code, Linux can run on top of Windows, but without the same high performance levels. It is this device driver code that we are releasing today, directly to the Linux Kernel.

We're not talking a few hundred lines of code here; we're talking about roughly 20,000 lines of code.

Is this a Dump and Run from Microsoft? Absolutely not! We plan to enhance the functionality of this code, and we will continue to work with the Linux Community to support the drivers and to ensure continued interoperability.

As you can imagine, this was the result of a lot of hard work: Hiyang Zhang, who has been co-writing this code; Hashir Abdi, who has been testing all this stuff; as well as Vijay Tewari and Mike Sterling from the Hyper-V team who have been taking care of the Hyper-V side.

And last, but certainly not least, Greg Kroah-Hartman, who has been helping me to make all this code land in the right area in the kernel. He has patiently worked to help me correct my obvious mistakes and to get the code contributed into the kernel.

So where are we today? Well, Greg Kroah-Hartman will make the code visible to the outside world today. (For those who want to get a head start, the code will sit under <your kernel tree>/drivers/staging/hv). After it becomes visible, I will write a few more blogs this week that should help you to understand, build and run them.

The titles I am thinking for these blogs are:

Where do the Linux ICs reside in the kernel tree and how do I build them?

And

How do I install, configure and run the Linux IC's?

I had almost forgotten how wrapped up you can be once you start writing code again. So I have not gotten much sleep this past week, but it has been a joy to get back into coding again!

PHP|Tek in Chicago

hjanssen — Thu, 28 May 2009 00:29:00 GMT

Last week I got the perfect excuse to get out of the Planning and Budget process that we are going through right now, attending PHP|Tek, which was a welcome escape as planning and budgeting in any company is usually enough fun to make a grown man cry!

So last week I went to PHP|Tek in Chicago to speak and meet folk from the PHP community. As always, I greatly enjoy meeting the people who write and use PHP, and I have been to and spoken with enough of the speakers at past events that I know a lot of the core people by first name.

Kind of funny that we now have gotten to the point inside of Microsoft that we are almost old hats at Open Source conferences

There were two days prior to the conference where a group of core PHP developers and community people talked about the state - past, present and future - of PHP. It was super cool to be invited to that one!

Unfortunately I was only able to join one of those two days: amazing that flying from Seattle to Chicago takes the better part of a day!

The discussions there where very wide ranging, from whether there will be a PHP 5.4, what 6.0 will bring, which bugs are current show stoppers, where PDO is going, etc. etc.

For me PHP|Tek remains a very nice ‘community' conference, where the focus is on the community of PHP and not the business/vendors of PHP.

These kinds of conferences are the best way to network, and it would take too long to talk about all the people I spoke to. But Elizabeth Smith and I talked about us writing documentation for php.net (I have been wanting to write the ‘how to build PHP for Windows' part) so hopefully look for more documentation written by Microsoft for php.net soon.

As always I talked to a lot of the usual suspects: Scott MacVicar, Andrei Zmievski, Derick Rethans, Sebastian Bergmann, Chris Shiflett, Cal Evans and others.

Oh, and if you are really bored, check out the latest May issue of php architect, which has a bunch of really cool articles about PHP and Windows. Some of them were even co-written by me, which gives you an idea how far php | architect has sunk to have people write articles for them

I just checked out the Website, and the May issue is not posted yet. But everybody who attended PHP|Tek got a copy of that issue in their goodies bag.

I always enjoy giving sessions and the session I did give at PHP|Tek was ‘PHP 5.3 The best PHP on Windows Yet' , and I got some really good feedback. I think I had about 40+ people in my session. People are always surprised to see Microsoft's involvement with PHP and what we have done with the community so far.

It is a talk I have given before. It starts with describing what the organization I belong to (the Microsoft Open Source Technology Center) does and how we work inside of Microsoft. After that I go into some detail about why PHP 5.3 is the best PHP on Windows.

Did you know that, for example, with PHP releases prior to 5.3, the code was build with libraries that were more than 10 years old and for which nobody really had any idea where the source code went? So it was built - linked rather - with object files that were more than 10 years old.

It makes it really hard to fix/improve stuff that you do not have the source code for

Well, pretty much all the issues of the past are now gone. I will make sure I write a blog about what truly went into PHP 5.3 for Windows soon, if the budgeting and planning process doesn't kill me before that point. In the meantime, here is a link to phpfreaks where, a few weeks ago, I posted a bunch of what we have been doing.

One really interesting thing is that there were a lot of Microsoft people at this conference, specifically from the DPE (Developer Platform Evangelism) side of Microsoft.

These are the people who are very much field and customer focused. From my conversations with them, they enjoyed the conference and were glad to get the opportunity to speak with a lot of the OS crowd. It is amazing how much we all have in common once we talk about technology.

Thanks to the people who put on the conference: of course Marco Tabini, the man behind PHP|Tek, but especially Elizabeth Naramore, who is the unsung hero that is the real driver behind keeping PHP|Tek running smoothly!

On the Road in Europe - Take 1

hjanssen — Tue, 28 Oct 2008 16:25:00 GMT

Today is Tuesday - That must mean I am in Mainz. I am on day 12 of my European trip. I was in Rome and Amsterdam last week.

In Rome I attended the Moodle conference, which was pretty cool. It was put on by Roma Tre and was one of many destinations in which Moodle held conferences this past month. I went to talk to Martin Dougiamas, Helen Foster and Petr Skoda as part of our ongoing quest to get PHP on Windows to be the best experience possible.

The deployment numbers that Martin showed in his presentation are quite impressive! I have been digging around for his presentation to give these numbers, but I can't find them. I am sure that Martin must have posted his presentation somewhere, I just have not found it yet.

But the intention was to start a working relationship with the Moodle community, and this was a great start. One of the most interesting presentations was from the Italian Airforce, who described their experiences in finding better educational tools to train their personnel, and settled on Moodle to be a large part of that. It is always interesting to have a General in the audience.

It is really amazing to see how and where Moodle is used. It is a testament to the intention of Moodle and Martin and the Moodle community that is has become so popular.

The other thing I did was meet with a lot of open source influential's/Government/CTO/Journalists etc. Microsoft Rome asked me to give a bunch of presentations and interviews, which I love to do, but it turned into a 17 hour-long gauntlet. And I just want to go on record and say that I cannot be held accountable for what I talked about the last 4 or so hours of that day. It became a little blurry at that point.

The bad thing about these trips is that I am away from home for a long period of time. The really good thing about doing these trips is that I get to meet so many people. It is really cool to see the faces and have the dialogues when I talk about what Microsoft is doing in the OS world. By far it is very positive.

But I get the biggest bang for the buck when we have discussions on what people do, and want, from the OS and from Microsoft. We have been doing more and more in the OS world, but unless we work closely with the community we have no idea if we are on the right track.

One of the questions I asked in Rome in a meeting with government officials and OS influential's was ‘what does open source mean to you?' There were many responses, and most of them followed the same line. Some of the common responses where: sharing knowledge, collaboration, personal recognition, information that is easy to get to, allocation of rights, intend to make communications possible.

Well, for the next few days I will be at the IPC in Mainz. So I will blog more tomorrow. There are a lot of things we are doing right now, so I have a lot of content

Hank Janssen & John Bocharov: Introduction to SQL Server Driver for PHP (SQLPHP)

jcannon — Thu, 08 Nov 2007 18:14:00 GMT

Our own Hank Janssen gives the Channel9 team an update on the work that has been done to provide a native driver to SQL Server for PHP.

"SQL Team Says:
"The SQL Server Driver for PHP (October 2007) Community Technology Preview (CTP) is designed to enable reliable, scalable integration with SQL Server for PHP applications deployed on the Windows platform. The Driver for PHP is a PHP 5 extension that allows the reading and writing of SQL Server data from within PHP scripts. It provides a procedural interface for accessing data in all Editions of SQL Server 2005 and SQL Server 2000 (including Express Edition), and makes use of PHP features, including PHP streams to read and write large objects."

Wow. This is cool. Need to find out more about this. What exactly is this thing? Why did we create it? What are the platform requirments? Is it open source? Who are the folks behind this? You know the C9 drill. Tune in and meet SQLPHP Program Manager John Bocharov and Microsoft Open Source champion Hank Janssen who answer a bunch of questions and provide good context about the thinking behind SQLPHP, history and future. Check it out.

PHP SQL Server drivers, much improved IIS support for PHP - what is this world coming to!

hjanssen — Fri, 19 Oct 2007 22:15:00 GMT

Yes I am still alive!

As I write this, I realize that I have been very delinquent with writing blogs. Time has gone by very fast, and I keep on working on more and more things. And before you know it, you look back and months have gone by without any blogs....

But I wanted to write a few things that have happened here in the last few months. As you might be aware, Microsoft has been working very hard on making PHP on Windows a premier experience. And working in the OSSL, my group and I find ourselves in the middle of most of these efforts. And it gives us a great view of the changes that are happening inside of Microsoft (Sometimes with a gentle shove from us )

So, we assisted the IIS team with their FastCGI effort and GoLive release a few weeks ago. (http://www.iis.net/php ) And no less interesting, something that I am particularly proud of is that Microsoft has released a CTP (Community technical Preview) of a native PHP SQL server driver on Windows.

As a result, I was invited to speak at the PHP Zend conference that was held in San Francisco last week, (www.phpcon.com) I have included the presentation for people to see what we talked about. John Bocharov from the SQL server team co-presented with me on stage and showed some of the bits that are now available for download. You can download them directly from the Microsoft download site. (here is a super long URL with the download details and helpful hints for the driver; http://www.microsoft.com/downloads/details.aspx?FamilyID=85F99A70-5DF5-4558-991F-8AEE8506833C&displaylang=en )

As part of the release, we at the OSSL converted phpbb (www.phpbb.com) to run with the native SQL server driver, and I showed phpbb running natively on SQL Server, on stage - which was pretty cool. We are currently cleaning those changes up and will be submitting them to phpbb code base. And....I am looking to see what other popular PHP applications out there we can contribute support to by writing code & integrating the driver.

All this shows a commitment from Microsoft that I am pretty excited to be a part of. I mean....an IIS 6 version of FasCGI with specific support for PHP, FastCGI build into Server 2008/IIS7 with PHP optimizations and now the SQL server driver for PHP on Windows!

What is this world coming to!

A note on the PHP driver, it is like I said - a community preview. Which means we are very much looking for your feedback on how to improve it, and any bugs you might encounter. So please participate as you can to make this the driver you would like to see it be.

Let's see...what else we have we been up to? We have been working with the Apache Software Foundation more, which has been a lot of fun. And in discussions with them to see how we could better work together, I supplied them with 75 MSDN licenses to help them develop on Windows. I am looking forward to continued cooperation with ASF on many different fronts.

The Lab continues to grow in both manpower and equipment, and we now have over 300 Servers to play with. From the old P3’s (very few still left) to the latest in IBM, DELL and HP Blade systems.

I will do my best to be more responsive and blog more. So keep the comments coming! Until the next time……………..
Thanks,
Hank.

Apache Conference 2007: Day 2

hjanssen — Thu, 03 May 2007 21:01:00 GMT

Here we are, day two of the Apache Conference in Amsterdam.

I have been attending less tracks today, I seem to be ending up talking to a lot of people.

It is very enjoyable to see the reaction when I tell people that I am from Microsoft, and I work at the open source software lab at Microsoft.

So far nothing but positive reactions to me being there.

I had the pleasure of talking with, among others, Lars Eilebrecht, Roy Fielding and William Rowe. They are of course very active in the core foundation. Very enjoyable, and there seems to be synergy for future collaborations.

Okay, before I go into what all took place today, I wanted to finish up yesterday’s events. And I am going to severely reduce my long winded writing (yeah right).

Two tracks I went to that were of interest yesterday were ‘ mod_rewrite’, which finally had some more technical content in it. I would love to see more of these talks. How and when to use which mod_*.

The second one was given by Rebecca Hansen of Sun Microsystems. She talked about ‘Best practices for incorporating open source code in Commercial Production’. I did not think she spent that much time on what the subject seems to imply. Much more time was spent talking about how Open Source is now viable and you can and should switch to it because large companies are now going to provide you support and services for it; so you will be safe using it.

She also said that companies are much more willing to pay for support to get what they want instead of paying for a license and being stuck with a product.

I have to say that these comments where met with some skepticism from the audience. And the questions that followed clearly showed this.

General audience response was that they are very well aware that OSS exists because of a community, not because of a company. So without the community there is no product/service. Which made the statement that you now can switch to open source because large companies will provide you service on the community software is kind of odd. Several people I spoke with afterwards seemed to share my views of it.

I think there is a place for service orientated opportunities for companies. But they better realize that without a healthy community for the projects they are trying to provide service to there is no business opportunity. Community comes first.

Okay I will write some more about what happened today. But I ended up talking to a lot of people and did not attend all the tracks I set out to. And since it is late here on the other side of the planet, I am stopping here for today.

Till tomorrow.

Well, at least it is a little shorter this time :)

Apache Conference 2007: Part One

hjanssen — Wed, 02 May 2007 21:11:00 GMT

So here I am, Amsterdam May 2nd 2007. At the Apache Conference. (A Microsoft person at an Apache Conference, what is this world coming to??)

I am going to blog from the Conference until it is over.

So, today the conference started in earnest with all the tracks kicking off. The first day was one of technical training. But this second day is where all the sessions started.

It started all with Sander Striker President of the Apache Foundation. He described very high level what was to be expected in the next few days, and he talked about the following.

He describes describes ASF, Est. June 1999. Non profit 501(R )(3) charity

He talked about how ASF is much more about community than about code, ASF manages communities, not code.

As with most projects, Open Source or otherwise, there is a tendency of burnout. He wants to make sure people stick around at the ASF by making sure there is an environment of Healthy community through: respect, open discussion, shared views and direction.

Today there are 43 Top level Projects (6 more than last Apachecon, October Austin - 2006.). There are also 31 projects in the Incubator (compared to 38 at last Apachecon). Overall he expressed his belief in that the future is looking bright and ASF being very healthy.

Also, today there are 1500 Committers worldwide, 220 Members. Membership is about the individual. Not corporations.

He closed by saying that People have a tendency to burn out in the infrastructure portion. It is a tough job to keep doing.

Picture 1: Here is a shot of the attendance during the keynote and introductions

Being notoriously bad at guesstimating the total number of attendants at any event, I am guessing that there are about 250 to 300 people here.

A question from the audience resulted in a very interesting answer. The question was how do you become a member. The response from Sander was:

Become a Committer first, and provide good quality work. If you keep contributing you might be proposed as a member. This will be subject to a vote.

But the description of a clear path to become a member is somewhat unclear from my point of view. I would think this path is more defined for those people wanting to become way more involved.

Next up was the key note delivered by Steven Pemberton, Researcher at the Center of Math and Computer Science. His keynote was called:

Abstraction and extraction: in praise of

He talked about abstractions of programming languages. And then went into how complicated these abstractions still are today. Yet daily interaction with objects can lead us to confuse the concrete with the abstract.

One of the nice things about programming languages is that they abstract away detail, like how data structures are implemented, how procedures are called. Etc.

He described a talk by Kernigan and Ritchie that he went to in the 70s where they were talking about Unix and C. This gave me a nice flashback and I am starting to feel pretty old! Thanks!

Some of the things we are struggling with today where the result of mistakes that were made when UNIX/C came to be. He talked that in his view that UTF-8 today is the result of the way they conflated characters worth units of store.

The intention of his talk was to speak more about usability, and designing for usability.

I have taken many notes when he spoke and I am trying to compose them back into his keynote. Bear with me while I try to reconstruct my notes. :)

He stated that you shouldn't confuse usability with Learnability. They are distinct and different. What he means with that is that if you want your software to be used by a large audience, you need to make is usable. Emacs (Still my personal favorite) is a powertool, you can do great things with it. But it is not what I would call usable. (powerful? Yes, Easy to learn? Not so much.)

What are the features of websites that you go back to regularly. The thing that differentiate them from other websites with the same purpose that you don't go back to.

Forrester research found 4 reasons for this.

Good content 75%
Usability 66%
Speed 58%
Frequency of updating 54%

The rest was noise: 14% and lower.

Yet Usability is usually the first thing scrapped when web sites are built. This seems to be applied to the design of software as well.

Eric Raymond, stated that making good software requires a lot of money to make sure it is usability tested and designed. This takes a large company with a large amount of money. OSS has not solved this problem yet.

Programmers like the command line, they are much more intuitive. ("Sensories" like much more graphical design). OSS programmers are intent with their use of the interface, yet the rest of the world is not. The rest of the world is much more Sensory.

A Dutch Magazine places GIMP last in it's review because of it's poor interface.

US Department of Defense discovered 90% of cost of SW production is debugging.

For example AJAX empowered page is a lot of work, Google maps, poster child of Ajax generation is more than 200k of code. He asks if it truly have to be this hard?

He made a really funny comment, while preparing his presentation he checked to see how much processor usage was going on on his machine. Then realized that his machine had dual core. And discovered that his computer is now twice as idle as it used to be. :)

Centre of his talk was really about usability. Much more so as it relates to languages. And I will give a plug here, it is basically the same argument he made as I did in my blog a few months ago. (He probably was more elegant in describing it) A link to the blog I wrote can be found here; Languages are becoming way too easy. In there I make the argument that languages are becoming easier yet they and the operating systems they run on have not kept up. (Meaning both have a really hard time protecting the programmer from the outside world :))

Some more data he gave that I found interesting: Computers have become 40 times faster in 25 years, Programmers managed to become 2 to 3 times faster maybe over that same time period. Which is because you still need to do to many things in languages. The example he gave was source code he found to display a clock. The clock part was only a few lines of code. But the rest of the 1000+ lines were taken up by setting up the framework. Making sure redraws and sizing are handled etc etc.

I will leave it at this for now. There is a lot more to write in the next few days, and I need to start reducing my blogs, they are becoming way to long!

Stay tuned, more to come in the next few days.

We're Writing Firefox Plug-ins? Interview with Ian Gilman and Thoughts on 10 Months at Microsoft

hjanssen — Wed, 28 Feb 2007 21:06:00 GMT

When I started working at Microsoft in May of 2006 I wanted to chronicle my adventures here. So my first blog was posted on June 7th 2006 Titled What is a guy like me doing in a place like this, I had every intention writing frequently about my experiences. As you can see, I have not been very consistent with that. Something that I will try to improve in the future.

If you would have told me 1 year ago that I would work at Microsoft I would have laughed. I still walk around looking with amazement at my badge, and when I go to other MS buildings I shake my head when I have to swipe my badge on the reader. When I talk to people I continue to refer to them and us (them being Microsoft, Us being the rest of the world )

I am happy to report that I continue to be the department's skeptic , something I will continue to be.

So I wanted to take this opportunity to talk about a bunch of my experiences since a lot has happened in the last 10 months (more about this later)

First of all, contrary what people believe, I do not know of a greater Microsoft plot to take over the world and destroy Linux and OSS. If there is such a thing, we at our level are unaware of it. And since this department in many ways is on the front lines working with OSS and Linux, I would have expected to see some evidence.

There is no helicopter pad (not on campus anyway) where Microsoft stores its black helicopters. There are no dispensers of Microsoft Kool-Aid. (They might have some dispensers in the water coolers though.) And the articles, blogs and posts that I read on what is going on here are most of the time completely off the mark.

Is Microsoft competing with Linux and OSS?? You bet they are. Just like every other company is competing against other companies/people/products that create similar products.

Is Microsoft working to better interface with some of the Linux and OSS products? You bet they are too! We are frequently working on those things as well.

I am not being censored or restricted in any way. I actually have access to a very wide array of things. More so than I thought I would when I started.

The department has a unique position inside of Microsoft. We get to talk to and work with a very wide swath of Microsoft product lines. Just to highlight a typical week that took place a few weeks ago; in the same week I spoke to the Robotics guys, people from the embedded department, People from IIS, SQL server department, the Powershell developers and the cardspace group. And this is a typical week. I am not sure how may other places in Microsoft have the same breadth.

And more and more groups are becoming aware of what we do and contact us to work with us.

Is Microsoft changing?? Yeah, I think it is. In some places it is going very fast, in other places not so much.

Yet if I look back over the last 10 months, I have seen some great changes happen. To name a few:

My Boss Sam Ramji invited the Mozilla foundation guys out to Redmond to help them get their products to work better inside of Vista. I spent the entire time they where here with them. It was met with great skeptism, the Slashdot comments ranged from ‘they are hiring the mozzilla folks’ to other helpful hints such as ‘fly on a different planes’. But I can say that when we send out feelers inside of the company of people would like to talk to them, people at Microsoft where literally lining up to do so. Read my blog for more on it.
Microsoft’s announcement of working with Zend to make PHP work better on windows, The department works with the Zend guys to assist them wherever we can.
We have continued to write and post many technical docs to Port 25, and will of course continue to do so in the future. We are always looking for topics.
Sam interviewed Miguel De Icaza and Steve Wozniak and many other people.
We continually help other parts of Microsoft when they have any questions about OSS, and help them when they want to Open Source things. (Powertoys for example)
The department is working with Novell on implementing the partnership. The interoperability lab that is being talked about with Novell is not smoke and mirrors.
There are many more items. I will leave it as an exercise to the reader to read port25.technet.com for some of the other stuff we have been doing.

But we have been touching a lot of items people never thought a few years ago would be likely. Getting Mozilla people on site for one. Another one that would have been considered impossible is Microsoft writing plugins for Firefox. Here is a cool one for example Photosynth, and you can listen to my podcast in which I interview Ian Gilman one of the Photosynth developers. Here is a link to the blog metioned in the podcast: http://labs.live.com/photosynth/blogs/

Just think about that for a second, Microsoft writing Firefox plugins!!!

I will leave you all with a few more observations:

Microsoft does listen to OSS people. And we do work with those people. More and more all the time.
There is still an awful lot of work to be done. Both internally and externally.
When the OSSL was created at Microsoft people thought it to be another Microsoft marketing ploy. Well I can tell you that that is certainly not the case here at OSSL.
The other guys in the group Sam, Anandeep, Kishi, Michael and Jamie are a blast to work with.

There seems to be a perception that we are not moving fast enough. But I believe we have been able to move at a pretty good speed! And, there are quite a few more things that we are working on that will show up in the future.

Looking back over the past 10 months I have come to the realization that I am really enjoying the job. There certainly are frustrating times, if you are on the front lines like we seem to be you are likely to get smacked every once and a while. If you are not, than you are not doing your job . But we are seeing noticeable change on all fronts. And it is a blast to be able to work with so many groups inside and outside of Microsoft.

So I will close with the following, I am not drinking the Kool-Aid, quite the opposite; I continue to question everything that is going on inside of Microsoft. And I will continue to be a voice for Open Source inside of Microsoft.

Powershell in Action! Hank interviews Bruce Payette

MichaelF — Fri, 09 Feb 2007 02:00:00 GMT

We get quite a few requests to provide more information about Powershell so Hank and I decided to go straight to the person who just finished writing the book on Powershell: Powershell in Action, Bruce Payette. Bruce was one of the founders of the Powershell team here at Microsoft and is an expert on not only Powershell but according to Jeff Snover, on any number of popular and obscure languages.

In this interview Hank and Bruce discuss Powershell and Bruce gives us a demo. While we couldn't show everything here, if there is interest, we can go back and have Bruce show us specific scenarios and examples. Let us know if there is anything you'd like to see...

Check out the link above to see Bruce's book and read a couple of sample chapters.

Video: Powershell in Action! Hank interviews Bruce Payette

Web 2.0... Or: The Web that Wasn't

hjanssen — Tue, 09 Jan 2007 01:05:00 GMT

Locate soapbox -
Place soapbox –
Make sure I do not fall off or thru said soapbox –
Stand on top of soapbox –

Start with a small cough, take a deep breath and begin………………….

The people I work with in the OSSL group know one of the easiest thing they need to do to push my buttons. The mere mention of Web 2.0 results in tirades from me that usually result in comments that are not fit for print.

Yet, I have decided to write a blog about this latest phenomenon called ‘Web 2.0’ or as I like to say, ‘the web that wasn’t.’

When I started this blog, I was sitting in the Barcelona airport on my way back from a presentation I gave at TechED 2006. The talk was about what we do here at the OSSL. But that really has no relevance to this blog.

While sitting in the lounge at the airport I was reading an article in USA today that was left by somebody on the chair next to me. (I wonder if that makes me cheap?) The article was written by Kevin Maney. (I honestly have never met him) And it was called ‘Packed Tech Summit With Vats of Yahootinis Ring Bubble Warning Bells’ (The USA Today in question was from November 16, 2006 – Page 9A)

Not really a title that seems to have anything to do with Web2.0, but if you can get a hold of the article, I highly recommend reading it. It very much describes the Web 2.0 phenomenon, and he draws comparisons with the big telecom 2000 tech bubble. I will not get into the article here; I will leave that as an exercise to the reader of this blog. But it was the catalyst for me to finally write a blog on what has been bugging me with Web 2.0.

Having worked with OSS since the very first Linux kernels came out, and with Unix at AT&T for quite a few years prior to that, I have seen a lot of great and sometimes not so great changes happen. But one of the things OSS allowed you to do is take it and for the most part do with it (or to it) what you want. To make it serve your purpose. It is a very evolutionary way of creating software. The strong and useful survive; the weak and useless do not.

For me there is no such thing as Web 2.0. There never was, and there never will be. The whole mistaken concept of what Web2.0 is is something that actually completely flies in the face of what I believe OSS has stood for. OSS has always prided itself for its independence, its freedom. Not being able to put a label on. Where the web or the OSS movement is today is in large part due to natural evolution. And the great thing about natural evolution is that you never really know where it ends up. It is always changing. So putting a label on something, a label that basically is used by the ‘establishment’ does the concept of OSS a disservice. And when that label seems to imply a version of something I really get uncomfortable. Anandeep Pannu send me an interesting link to a cartoon that very nicely sums up a lot of my feelings on this subject, you can check it out here:

http://twopointouch.com/wp-content/uploads/2006/07/userfriendlyweb20.1.gif

The cartoon seems to be done by the same person who does the cartoon in Linux Journal, It has the same characters and identifiers on the cartoon that Linux Journal seems to have.

And if Web 2.0 truly was such a thing that the OSS community was actively working towards can somebody than please tell me what Web 1.0 was? Or more importantly, please let me know what web 3.0 is _before_ it takes place. Not define something after it has happened.

When I explain to people my description of Web 2.0, I describe it as a big bus that has all kinds of developers on board (OSS and Commercial ones) they where all writing really cool stuff, and the never really did it with the idea of putting together what is now termed Web 2.0. Than there where the people that where running outside of the bus trying to slap stickers on it with Web 2.0 printed on it. It was something that was done not by people on the bus, but people that where running outside of it trying to keep up with it. Actually the people on the bus where not even aware that it was happening.

I have spoke to a lot of OSS developers, at conferences and thru contacts that I have, and often Web 2.0 comes up. And I am struck by the similarities of their view on it compared to my views.

For me one of the really cool things that has happened with the web in the last few years is that a lot of it was written by people who had a passion for solving or accommodating a vision of what they had (Both OSS and Commercial developers). The fruit of this labor then was merged and used by end users in such a way that the original creators never thought of. Which is the great thing about it!! Technology was adapted/used/applied by people around the world to solve or create things they really wanted. A really cool way of developers and end users getting together without there ever having been a plan to do so.

Often software is created for a specific purpose; a lot of technical innovation goes into it. And frequently we forget about the people who will end up using what we create. What is happening on the web (and I am starting to see it in other software areas as well) is that other developers or in a lot of cases end users have put things together that where never thought of to be put together.

If we could agree on calling it something other than Web 2.0, which for me invokes a clearly defined software release. Something it certainly is not. Maybe call it something like ‘The unintended web collaboration framework’ (Can you tell I do not work in Marketing, my slogans would not sell water to a dehydrated person in the desert!)

The web is an evolution; I for sure am not smart enough to know where it is going. But I am looking forward with great interest and enthusiasm how everything will look like a year or more down the road.

Takes a small cough –
Steps down from his soapbox –
Puts soapbox away –

Languages Have Become Too Easy...

hjanssen — Mon, 18 Dec 2006 17:40:00 GMT

I have finally found a way to write more blogs!!! When I am in the office I have so much work that I rarely get enough time to sit down and concentrate on a blog. When I get home (My wife tells me normally later than she wants me to) I do not always have the desire to write a blog. But I am flying for work this week and I am finding all kinds of time!

What for me the line is that epitomizes the fact that I must have turned into my parents is “When I was Young”. Yet I am finding myself starting this blog with exactly that.

First a let me describe he catalyst for this blog;

A few months ago I attended OSCON 2006, one of the sessions I went to was called ‘PHP Security Hoedown’ given by Ed Finkler (http://homes.cerias.purdue.edu/~coj/phpsecinafo)

Basically, what this session was about was talking about PHP security. The session was a response to security problems people have been finding with PHP. Specifically the installations and running of PHP.

He stated that a large part of the Security problems that PHP seems to be suffering from can be summed like this (I have taken some liberty to paraphrase some of the things that where said, but check the above link to his original presentation.);

PHP has a fairly shallow learning curve. Because it is a shallow learning curve, there is a lot of variety of people that are wide in range of skill sets. Basically almost anybody can get started in PHP and get something running pretty quickly.
There are really only a small percentage of top level people who could be considered ‘experts’ in the language.

So, now we are getting to the part that I warned about. ‘When I was Young’.

Many moons ago, now more than I am willing to legally admit to, I started my career with Philips/AT&T who at the time had a joint venture, they developed very complex digital telephone switches. The 5ESS line. This was a very sophisticated telephone system that was almost completely written in C.

When I started my programming career with AT&T (Now over 20 years ago) you had to go through a lengthy process of learning the language C. Carrier grade software was and still is of very complex nature. Since people that have ever written in C know, it is a very powerful language that provides you with a very large gun to shoot yourself in almost every body part you can if you are not careful. So we where trained very well before we where let loose writing switching code. One of the other things that was required, if you wanted to make the jump into C++ (Mind you this was when there was no C++ compiler yet, but only CFront which was a pre-compiler/parser), you where not allowed to write in C++ unless you have been programming C for at least 3 years consistently.

There really where not that many higher level languages as there are today.

For the last few years I have seen more and more computer languages born, and in some cases die. And they all try to fix what their authors thought where missing in the languages that came before it. Another trend has been to make languages more accessible and easier to use to people who want to program of all walks of life. Imagine that! A language that does not require a 4 year degree to work in!

Some of these languages for example PHP and Ruby (They sure are not limited to these languages I might add!). They allow people with limited computing background to make in fairly decent programs in a small amount of time.

But this is where some of the security issues are showing up. The languages are becoming easier to use. But a lot of the operating systems they run on really have not become easier. So, many of these programs are now used without the realization on the part of the installer or programmer what the effect and impact of running their programs are on the operating systems. This seems to be a problem on both Linux and Windows platforms.

Although I applaud making programming languages easier for the more casual user, I do see that we are forgetting in many cases to make the environments these programs need to run in safer and easier as well.

I have seen so many times programs that write their files in ‘interesting’ and unsecured places. The presence of multiple libraries that might or might not support the application (heck, I am not sure what makes the thing run, so I will just copy all kinds of libraries in an attempt to make the application work).

File permissions that are set incorrectly, readable by the world. Incorrect owners etc.

And these are just some of the issues that seem to be present. And unfortunately a lot of these problems are easily fixed.

But I think that we need to do more as developers and system architects. Some of the suggestions that come to mind are:

Provide Security and architecture primers as part of the languages that are being developed. This should make it easier for the end app developer to have an appreciation of the program they wrote and what environment it will run in. (Tips and tricks documents, do’s and don’t documents etc)
Keep up with the development of the operating systems to make if safer/easier to deploy these new languages. UML in Linux might be a step in the right direction, and so is the new security mode that Internet Explorer runs in on Vista. But more needs to be done.
Have experts in the language provide more support in the area of the interaction with the OS and application programming for the target audience.
Make installers easier to use and smarter. Taking a lot of the work of deployment out of the hands of those who want to write code without needing a masters in the OS they are deploying on. WIX for Windows does a very nice job. And there are a few on Linux as well (rpm for example) but I would say they have some way to go so that they are easy and safer to use.
Have ‘self check’ modes on the languages that are being developed. E.g. Start the program the end user just wrote and the language will have a mode that will warn/comment/suggest things to the app developer. (Such as there was lint in Unix. But it should be part of the execution of the application program. And it has to be user friendly. Lint at times was downright sadistic in trying to decipher J)
Force files to be created in safe areas.
A lot of OSS software comes with ‘configure’, which is a very old and robust way of building make files and their dependencies. Now create something called ‘deploy’ that will do the same thing for the completed applications the end programmer just created. The things it should check for example are:
o    Are the libraries it needs in the correct place
o    Set up the environment variables if needed
o    Does it follow the language authors best practices for deployment. (Make application programs go to /usr/local/bin instead of /bin)
o    Make sure that the directories it gets deployed in are not owned by the wrong owner/groups
Have more interaction with the OS developers and the Language developers to help each other build better languages and safer deployments on the OS.

It seems to me that languages need to be developed more with the end user in mind regarding deployment and the OS’s they will be running in. A language can have all the cool features you ever thought off, but if on deployment you create system issues of worse a bad security hole, than it all will have been just a hobby.

I can equate it to getting your drivers license, getting your license is fairly easy (at least in the US it is). And you can get it without knowing anything at all about cars. Car manufacturers have realized this and have made their cars tell the driver what is wrong with it. Now if you keep on driving your car with the ‘check engine light’ on, well than you are on your own.

If we want languages to be adopted and thrive, we better find a way to build in a ‘check program’ light.

Firefox Explorer (Or: Mozilla is Here)

MichaelF — Wed, 04 Oct 2006 11:47:00 GMT

So, as people are finding out, we at OSSL are doing things a little differently. One of those things Sam Ramji’s invite to the Mozilla guys to come up and help them making their products work on Vista better.

This was met with great criticism (Slashdot for example had people make all kinds of wild assumptions that started with suggestions all Mozilla folks flying on different planes to MS offering everybody jobs. And everything in between) and surprise.

Well, I am glad to report that the Mozilla guys are here. I have spent a really cool day in all kinds of meetings with the following guys from Mozilla; Robert Strong, Mike Schroepfer, Doug Turner, Vladimir Vukicevic, Scott Macgregor. (If I slaughtered anybody’s name I apologize for it in advance!!)

They are meeting with all kinds of people here, and it seems that both sides are learning greatly from each other. One thing that I noticed right away is that both the MS side and the Firefox side just love the technology and are hitting it off nicely.

The things we talked about today for example have been;

Cardspace; Mike Jones and Garrett Serack gave an overview of cardspace. Cardspace is a carrier of identity tokens. But you can read about it here as a nice starting point http://blogs.msdn.com/garretts/ . We spend some time talking about what it is providing, and that it is an effort from the industry to provide a way to provide identity management. Vista will include it out of the box, and .Net 3.0 will include the cardspace technology. Which will make it run on XP and 2003.

And we are currently looking to writing a plugin to make cardspace work inside of Firefox. (Like I said, we do things differently here at OSSL )

IE Lower Integrity Marked: IE will be running in a Low Integrity Marked setting, resulting in a greater security environment IE will run in. But it limits a whole bunch of places it can write to. (which is good) We talked with the IE people to the benefits and pitfalls of doing this, Mozilla is interested in checking into doing this for Firefox as well.

MS WPFE: the developers from the plugin had a conversation to make it work inside Firefox better. (I am not an expert on this, but this should give a better understanding of what it is; http://msdn.microsoft.com/winfx/reference/presentation/default.aspx )

MSI: MSI developers talked about the changes in and advantages of using the new MSI way of installing software in Vista. One of the things that came up was that some vendor have problems with their plugins (xpi files) such as Adobe. (Problem relating to global plugins) XPI can only be installed per user in Vista. I will talk more about this in a later blog

Windows Vista Shell: we went through a high level overview of the Windows shell in Vista.

If you want to know what the Mozilla folks thought of yesterday’s meeting, check out Vladimir’s blog http://blog.vlad1.com/

Well, I wanted to keep this short. But still let you all know what is going on. There is another day with them coming up tomorrow.

Btw, the chips have been implemented into the brains of all the Mozilla folks that have shown up here. And it looks like from Firefox 2.0 on it will be released on Windows exclusively.

Hank.

OPEN/SHARED SOURCE AT Microsoft

MichaelF — Tue, 12 Sep 2006 21:25:00 GMT

It has been a while since I posted a blog, and I really have no other excuse than that I have been very busy. I have had a whole bunch of blog ideas percolating in the back of my mind, and I will be writing them down soon.

When we started port25 and the OSSL it was met with great skepticism. But there have been a lot of changes going on around us here at Microsoft. And one of those I wanted to bring to your attention.

A few years ago the mere thought of Open Source at Microsoft was ridiculed both inside and outside of the company. But I am starting to see small and sometimes not so small changes. This blog describes a very positive change.

As you might all know, I went to the 2006 OSCON conference in Portland. And there I met another Microsoft employee, Sara Ford. She works in the Visual Studio and Power Toys area. She has been a very active blogger in the past (unlike myself, working on it though!) And you can find her page here.

We got to talking at the conference and I have worked with her a little since then and found her to be a very energetic person greatly interested in OSS. But why is this interesting??? Well she attended a session at OSCON given by James Howison. (See his OSCON session info here ) And his presentation was on open source communities.

She was so impressed by it that she is currently working on Open Sourcing the Power Toys. I had the pleasure to sit in the training she gave the team, you can see more of the training she gave (unfortunately I was there as well and probably messed up the whole video by opening my mouth. So ignore me!) here.

In any case, who would ever thought Microsoft would open sourcing anything. But it is happening, and in future blogs I will give you all more insight on my first 4 or so months here and the changes I am seeing both internal and external.

Till then!

Hank.