Port 25: The Open Source Community at Microsoft : Community, Dev Center

Part 2: Lessons I Learned as a Project Manager Converting to Agile

saraford — Tue, 20 Oct 2009 21:11:00 GMT

In my first post for this series, I stated that "Agile is the single greatest things a team could do to significantly improve the user experience and quality of their website" and "not designing the full 100% is a true blessing in disguise." By putting these two concepts together, you can fully embrace the power that is Agile: Course Correction.

Making Design Changes in Waterfall

In the traditional waterfall, a Program Manager writes a specification, in which all members of the team and the associated business owners review and approve. Then, the development cycle begins to code to the specification word for word. Once code complete, the test team takes over reviewing the code to ensure it matches the specification.

But, what if half way through, someone on the team realizes the specification is wrong? Or more practically, what if the requirements change sometime during development? In my Microsoft experiences, this is called a Design Change Request (or a DCR), and it is extremely costly. New designs need to be made and reincorporated with the rest of the specification. Dev needs to re-code the feature. And the test team has to restart any prior testing, especially for regressions. In other words, DCRs are not good things.

Making Design Changes in Agile

One of the light bulb moments for me on CodePlex.com was accepting the fact that Agile is really just a series of mini waterfalls. Instead of designing a waterfall release cycle that will span the course of several years, you're only designing for a waterfall process of a few weeks. There's a Planning Phase, a Development Phase (which encapsulates the Test Phase, but is beyond the scope of this blog post), and the Deployment Phase. It's waterfall, but waterfall moving at an incredibly fast pace.

Now let's reconsider those Design Change Requests, but on an Agile team. The work is very, very scoped, which implies less ambiguity. (Yes, you could have a perfect specification that isn't ambiguous at all, but if that spec is 60 pages, the risk of human error is higher. Trust me, I've seen this, where I was the human in error.) And, recall that you're only doing 80% of the work. The cost of the design change is already accounted for in the remaining 20% of the development cycle. In other words, it's in the spirit of Agile that you're going to make changes along the way to better the product, or "course correct," as it was initially described to me. In agile, DCRs are good things.

Regardless whether you discover that you need to make a course correction during the Development Phase or the Deployment Phase, it's okay because Agile by its very nature can adapt and respond to these changes quickly.

An Example of Course Correction

Below is a screenshot from the original ratings and reviews designs. Notice how I circled the downloads count.

During implementation, a developer realized that the download count could be confusing. Did it mean total downloads for the entire project, total downloads for the release, or just recent (past 7 days) downloads for the release? We realized that given these designs we couldn't really convey the information we felt necessary to provide the right user experience to our visitors. So, we went back to the design and made a few tweaks. Again, we weren't trying to completely redesign the project directory. Even if we wanted to, there was no time to do anything radical. At the time the developer noticed this issue, we were going to finalize the build for deployment in a little over a week.

What we did instead was "course corrected" by making slight changes to the project metadata on the left-hand side to finish the current feature (or user story, if you're more aware of that terminology). Circled below is the metadata we added. We also added the * next to the number of page views and downloads, where we state at the bottom of the page "in the past 7 days."

Aside: CodePlex shows past 7 days data as a way of reinforcing release early, release often, which we continue today in the project directory. The screenshot below was taken at the time of this writing.

Conclusion of Part 2: Course Correction

This concludes my second post on Program Managing an Agile team.

If you like what you see, let me know! And if you don't like what you see, please don't hesitate to let me know. Seriously, I love discussing my Love / Hate Relationship with Agile development, as depicted on my personal blog.

Up Next: Putting it All Together - How the CodePlex team Builds Software

Part 1: Lessons I Learned as a Project Manager Converting to Agile

saraford — Mon, 19 Oct 2009 19:16:00 GMT

Before I became the Program Manager for CodePlex.com, Microsoft's open source project hosting site, I worked on the Visual Studio team on four different product cycles. Since Visual Studio uses traditional Microsoft product lifecycle releases, I had to learn about Agile development alongside learning about open source development when I joined the CodePlex.com team. Making the switch from releasing every three years to every three weeks didn't happen overnight!

One of the things I discovered is that Agile is used a lot in open source communities. In fact, many of the people who I've worked with personally on learning Agile have strong roots in OSS. Also, the fact that Microsoft is starting to adopt Agile philosophies shows how the company is changing, becoming more transparent, finding more ways to connect with the community, and embracing other schools of thought. And this is why I am here, to be on the inside to push for these cultural changes within Microsoft.

Although Agile is the single greatest thing a team could do to significantly improve the user experience and quality of their website, I believe it can be quite challenging for anyone not in a developer role to get accustomed to. I hope that by sharing my experiences, I can help others in non-developer disciplines on an Agile team.

In this series of blog posts, you'll discover how I learned to program manage an Agile team after six years of waterfall (the traditional method of software development).

Three Major Takeaways

If I could go back in time, here are the three things I would tell myself about Agile.

1. Design and plan for the very next step.

It's not about reaching the moon, but getting out of your front door. Sure, you can design the perfect feature, but if it is going to take you six months to get there, it is useless to an Agile team. It's about designing the journey towards the perfect feature that matters.

2. Break down work into the smallest possible functional sets.

Adding work is fun and rewarding, but removing incomplete work due to a lack of development time is painful and risky. But, you can't deploy a half-written feature either. First, break down the work into the smallest pieces. Then, put together the smallest functional sets that have to be deployed together for the feature to make sense. Your development team will tell you how many sets they can do per release.

3. Design and plan only 80% of the way.

Not designing the full 100% is a true blessing in disguise. Since you have another release right around the corner, you have the time to collect user feedback and incorporate it into the next design. Not only does this solve the remaining 20% (getting you closer to the perfect design with less cost), but also allows your customers to be virtual members of your team.

The Program Manager Release Cycle

To begin, here's an Agile release cycle from the point of view of the Program Manager. For simplicity, this illustration only depicts a single release cycle, without any overlap of previous or future cycles.

Ratings and Reviews: An Example

To further illustrate, consider the ratings and reviews feature on CodePlex.com as an example. Users can rate a release and write a review for projects on CodePlex.

One quick aside about ratings and reviews: CodePlex users rate an individual release instead the entire project. For example, consider Stephen King as an author. What does it mean for me to rate Stephen King 4 out of 5 stars? I find some of his books to be awful, like the Tommyknockers. I want those 7 hours of my life back. 1 out of 5 stars. Yet, for me, some of his other books are incredible, like The Dark Tower. 5 out of 5 stars. Hence, we allow users to rate an individual release to provide more relevant information to potential downloaders of the project.

Let's explore the ratings and reviews feature step-by-step in the Program Manager shoes.

1. Design Phase Part 1: Limit the scope to designing the minimum to make the feature useful and meaningful. For ratings and reviews, the feature must have the following:

a. User can rate a release. User can view the rating.

b. User can write a review. User can read the review.

c. User can sort by highest rated releases in project directory.

2. Design Phase Part 2: Bucket into smallest deployable functional sets. For me, personally, I use sticky notes to illustrate the "must have" pieces for each set.

a. Sticky Note #1: Rate releases / View rating

b. Sticky Note #2: Write a review / read review

c. Sticky Note #2: Sort by highest rated releases

3. Iteration Planning Meeting: To start the development cycle, meet with the development team to discuss costing.

a. In the case of ratings and reviews, my devs said they could do Sticky Notes #1 and #2, but the project directory sorting feature would have to wait for the next release.

4. Dev Cycle: Because the designs are closer to 1-page specifications rather than fully-documented implementations, questions will come up from the dev team. This is where you, the Program Manager, will:

a. Answer any questions about the missing 20% of the specifications / wireframes

b. "Course correct" (more on that later)

c. Add more feature work if time allows

5. Deployment: The release goes live. Now you can collect user feedback and incorporate it into the next development cycle.

a. The very first tweet I saw regarding the ratings and reviews feature was "Sara, is there a way to sort by highest rated?" Here, I was able to ask the user questions about how this feature should work to confirm our designs. Most of the time we don't inform users what's coming up next (we like surprises.) But in this case, it was pretty obvious.

Here's the visual representation of the ratings and reviews feature in the Program Management agile release cycle:

Conclusion of Part 1: The Agile Program Management Cycle

This concludes my first post on Program Managing an Agile team. Since I could go on endlessly writing about topics I'm passionate about (and those who know me will confirm this is not an exaggeration), I'm going to pause here to conclude this initial train of thought.

Up next: The concept of Course Correction.

Expanding the interoperability of eclipse4SL: Mac support comes to the Eclipse for Silverlight project

Vijay Rajagopalan — Wed, 18 Mar 2009 18:15:00 GMT

In October 2008, when Microsoft announced the general availability of Silverlight 2.0, and, as part of Microsoft’s ongoing commitment to interoperability, Microsoft and Soyatec unveiled the Eclipse Tools for Silverlight (Eclipse4SL).

Today, Microsoft and Soyatec are expanding our interoperability collaboration by releasing a Community Technology Preview (CTP) of the eclipse4SL project that includes support for the Macintosh platform. Mac and Windows developers can now collaborate on Silverlight projects using the Eclipse or Visual Studio IDEs. We have also added several new features to the project that should please all developers like C# code generation and improved XAML auto-completion (check out our Interoperability Team blog for more details and a demo)

As an open source initiative sponsored by Microsoft (funding and architectural guidance) and led by Soyatec (development), the eclipse4SL project is released under the Eclipse Public License Version 1.0 on SourceForge.net and was submitted by Soyatec it to the Eclipse Foundation as an open Eclipse project. Since its inception the project has received lots of feedback and made significant progress. eclipse4SL has been among the “Top Rated” projects on www.eclipseplugincentral.com for weeks:

(Screenshot taken on 03/16/2009)

For more information, visit the Eclipse for Silverlight web site, or join the discussion at http://www.eclipse4sl.org/community/.

Vijay Rajagopalan, Principal Architect in the Interoperability Strategy Team at Microsoft.

Moonlight 1.0 Hits the Street

Peter Galli — Wed, 11 Feb 2009 19:42:00 GMT

Moonlight 1.0 is now available.

Moonlight is an open source project that gives Linux users access to Microsoft Silverlight content, and is available for all major Linux distributions, including openSUSE, SUSE Linux Enterprise, Fedora, Red Hat, and Ubuntu. This milestone release is part of the technical collaboration between Novell and Microsoft.

Microsoft has worked with the Moonlight team and Novell to enable interoperability between Windows and Linux platforms and extend the high-quality interactive Web and video experience for the benefit of the Linux community, said Scott Guthrie, corporate vice president of Microsoft's .NET Developer Division.

Microsoft has provided Novell with access to its test suites for Silverlight, and provides Linux end users of Moonlight with free access to the Microsoft Media Pack, a set of licensed media codecs for video and audio that bring optimized and licensed decoders to every Linux user using Moonlight. Windows Media Video (.wmv), Windows Media Audio (.wma) and MP3 files are supported through the Microsoft Media Pack.

A pre-release of Moonlight was made available on January 19, 2009 to allow Linux users to stream Barack Obama's Inauguration, and more than 20,000 Linux users downloaded Moonlight to watch that Silverlight broadcast.

"Microsoft Silverlight offers the most comprehensive and powerful solution for the creation and delivery of rich internet applications and media experiences, and is used by hundreds of thousands of developers worldwide," Guthrie said.

For his part Miguel de Icaza, the founder of the Mono project founder and vice president of Developer Platforms at Novell, said Moonlight brings the benefits of Silverlight's popular multimedia content to Linux viewers. "This first release delivers on the goal of breaking down barriers to multimedia content and creating parity in the user's viewing experience regardless of whether the user is on Windows or Linux."

Welcome Snakebite, the Newest Open Network in Town!

Peter Galli — Tue, 10 Feb 2009 13:38:00 GMT

There's a new Open Network in town, known as Snakebite, which is the brainchild of Trent Nelson, a committer to Python.

Snakebite is a network of some 37 servers of different shapes and sizes, spread over three sites and specifically geared towards the needs of open source projects like Python.

In short, and according to its Website, Snakebite is a network to provide open source developers unrestricted access to as many of the different platforms, operating systems, architectures, compilers, devices, databases, tools and applications that they need to optimally develop their software.

"Why do we develop open source software on closed networks? Why do open source developers only have access to a fraction of platforms that their software will eventually run on? And why the *@&# are the Windows buildbots always red?! Snakebite was created out of a desire to try and address problems like these faced by open source projects," the Website says.

Some months back, Nelson realized that while buildbots were fine when everything was running smoothly, nothing compared to actually having access to a system when a developer is trying to debug something.

"So, I thought to myself, why not buy a couple of clunky old boxes off eBay and donate them to the Python Software Foundation, such that all developers had access to them ... Ten months, seven trips to Michigan State University, six blown fuses and about $60,000 later, I'm proud to introduce you all to Snakebite: The Open Network!," he said.

This means that every CPython, Jython, IronPython and PyPy committer will have access to every development server on the network, Nelson says, adding that he has already extended the offer to prominent Python projects like Django and Twisted.

The end-goal is to ultimately invite other open source projects like Apache, Subversion, MySQL and Postgres, among others, but given that this network is Nelson's "gift to All Things Python, first and foremost," Python projects will always get preferential treatment.

Nelson also has big aspirations for Snakebite going forward, which you can read all about in this email he sent to the Snakebite list.

Nelson was also blown away by the level of support he received for the initiative: "Microsoft jumped on board and provided unlimited MSDN licenses in less time than it took me to write them an e-mail asking for stuff. Having the support of Microsoft from very early on has been a huge boost, and the MSDN licenses have already been invaluable ," he said.

An email to HP asking them for a Tru64 license and 2GB of RAM for the Itanium box he bought off eBay, also resulted in the company shipping two quad Itanium 2 RX-5670s, full of 73GB 15k disks and 78GB of RAM between the two servers; 32GB in one and 46GB in the other.

Sun, Google and Canonical have also expressed interest in the project, but Nelson has stopped asking for hardware as they have run out of space to host it all.

But, in a recent post to the Python Committers mailing list, Nelson notes that it will probably be a few weeks before users can start logging in and doing stuff.

"The HPCC/CSE server room at MSU is about to have walls knocked in and ramps built in order to accommodate a giant PDU that has been sitting outside it for about six months; the Snakebite rack is going to get shuffled around a bit so I figure there's not much point going live before that's taken care of," he said.

Web Sandbox Source Now Available Under Apache License 2.0

Peter Galli — Tue, 27 Jan 2009 02:48:00 GMT

Microsoft has released more source code under an OSI-approved license: this time it has made the source code for the Web Sandbox runtime available under the Apache 2.0 open source license.

The Web Sandbox project explores how to advance the web platform to improve security, isolation, quality of service and extensibility capabilities for web developers and website users.

More information on the licensing details, as well as comprehensive documentation for experimenting and integrating with the Web Sandbox, can be found here.

But, while developers are being encouraged to help define and refine the Web Sandbox, it is not recommended for those developers creating production sites as it is still under development.

The Web Sandbox was created in response to limitations found in the current web platform, and is designed to explore potential solutions. Having a more secure and robust architecture as a foundational building block will help drive the next wave of Web innovation.

The Sandbox is a framework that works on most modern browsers that support the"ECMA-262, 3^rd Edition" (JavaScript) standard, and provides the same features in all modern web browsers. No browser add-ons or changes are required to leverage this technology. Beyond security, the Web Sandbox normalizes the different browsers and provides consistent W3C DOM support.

Since the initial release of Web Sandbox at PDC 2008, the team has received a lot of useful feedback from the web security community, and has also been collaborating with a number of customers, partners and the standards communities, all of whom want to adopt the technology when it is ready.

The goal? An open and interoperable standard that will help foster interoperability with complementary technologies like script frameworks and drive widespread adoption of the Web Sandbox.

This move is good news for Microsoft and the open source communities. But, it is important to note that while an Apache license is being used, the Web Sandbox project is not an Apache Software Foundation project and is not sponsored or endorsed by the ASF.

Microsoft does, however, already have an active relationship with the ASF. In fact, last year the company announced it had become a sponsor of the ASF so as to help enable the Foundation pay administrators and other support staff so that its developers can focus on writing great software.

Sam Ramji, the senior Director of Platform Strategy at Microsoft, also delivered a keynote address at ApacheCon in New Orleans last November.

Microsoft's Interoperability Technical Strategy Team already participates as a code contributor to the Apache Stonehenge incubator project; the company has also contributed a patch to ADOdb, a popular data access layer for PHP used by many applications and which is licensed under the LGPL and BSD; while Microsoft's Powerset team contributes to HBase, an open-source, column-oriented, distributed database written in Java.

Update: Stonehenge Incubation Project

Peter Galli — Mon, 19 Jan 2009 20:00:00 GMT

As Microsoft continues the drive for interoperability between different implementations on various platforms, the Interoperability Technical Strategy Team is, for the first time, participating as a code contributor to an Apache project: the Stonehenge incubator project.

Microsoft first talked about Stonehenge at ApacheCon 2008, which was held in New Orleans last November. Since then, it has been approved as an incubator project within Apache Software Foundation, and WSO2 and Microsoft have already contributed code for a web-services based sample application, known as StockTrader, to this effort.

That code can be found here, along with the contributions from WSO2.

Stonehenge has attracted some very prominent committers so far, Kamajit Bath, a Principal Program Manager in the Interoperability Technical Strategy Team and the lead for Microsoft's participation in Stonehenge, says in a recent blog post.

"I hope that the momentum will be sustained, and I am looking forward to seeing code contributions from other folks and seeing the StockTrader sample application enhanced with new features. I also hope that new sample applications will be developed to cover other areas of the WS-* standards that are not best represented by the StockTrader application. I look forward to participating in this discussion with the Stonehenge community," he says.

There are three Microsoft committers on the Stonehenge incubator project: Greg Leake, who wrote the original StockTrader application; Drew Baird, who worked to get it ready for contribution to Stonehenge; and Mike Champion, who will play an active role in this effort.

In a recent blog post, Champion says Microsoft have heard from customers that they want sample applications based on real-world scenarios and challenges, as these will help them realize the potential of these technologies that have been developed and standardized for the last 8 years or so.

The initial response from the Apache community has also been quite favorable, and "I have a personal commitment to invest in helping make Stonehenge a success, and look forward to digging in," he says.

Champion also notes that Stonehenge is being championed by Paul Fremantle, co-founder and CTO of WSO2, "which has been a great partner in helping to improve and demonstrate the interoperability of the WS-* standards across platforms."

He cites, as an example of this, TechEd 2008, where Jonathan Marsh of WSO2 and Greg Leake of Microsoft demonstrated how separate WSO2 and Microsoft components implementing a mutlti-tier stock trading application can interoperate and be substituted for one another, he says in the blog.

StockTrader is also just the starting point for the broader goals of Stonehenge, which aims to develop a set of sample applications to demonstrate seamless interoperability across multiple underlying platform technologies by using currently defined W3C and OASIS standard protocols.

Stonehenge can also help wire up the ‘last mile' between the standardized web services infrastructure that is now implemented across key platforms, and a new generation of service oriented applications that will span them, he says.

Existing WS-* interoperability work such of the sort done by WS-I and in our "plugfests" will continue to solidify the platform-level interoperability. The new work, exemplified by Apache Stonehenge, should attract a wider community of users who can exploit the hard standardization and platform interoperability work without having to wallow in as many nasty details as in the past.

For Bath, projects like Stonehenge are important in enhancing interoperability between different software implementations. While standards organizations do a great job and the roll out of various WS-* standards is a testimonial to the fact that they can work efficiently, interoperability work doesn't stop at the end of the standardization process but, rather, that is where it really starts, he says.

SMB2: a Complete Redesign of the Main Remote File Protocol for Windows

Peter Galli — Mon, 08 Dec 2008 16:24:00 GMT

SMB (Server Message Block) is a remote file protocol commonly used by Microsoft Windows clients and servers that dates back to 1980's.

Back when it was first used, LANs speeds were typically 10Mbps or less, WAN use was very limited and there were no Wireless LANs. Network security concerns like preventing man-in-the-middle attacks were non-existent at that time.

Obviously, things have changed a lot since then. SMB did evolve over time, but it did so incrementally and with great care for keeping backward compatibility. It was only with SMB2 in 2007 that we had the first major redesign.

In this blog Jose Barreto, a senior technical evangelist in Microsoft's Storage Solutions Division, explains some of the history behind the protocol and outlines important improvements in SMB2, particularly in regards to reduced complexity, pipelining and compounding.

Introduction

SMB (Server Message Block) is a remote file protocol commonly used by Microsoft Windows clients and servers that dates back to 1980's. Back when it was first used, LANs speeds were typically 10Mbps or less, WAN use was very limited and there were no Wireless LANs. Network security concerns like preventing man-in-the-middle attacks were non-existent at that time. Obviously, things have changed a lot since then. SMB did evolve over time, but it did so incrementally and with great care for keeping backward compatibility. It was only with SMB2 in 2007 that we had the first major redesign.

A History of SMB and CIFS

When it was first introduced to the public, the remote file protocol was called SMB (Server Message Block). SMB was used, for instance, by Microsoft LAN Manager in 1987 and by Windows for Workgroups in 1992. Later, a draft specification was submitted to the IETF under the name Common Internet File System (CIFS). The CIFS specification is a description of the protocol as it was implemented in 1996 as part of Microsoft Windows NT 4.0. A preliminary draft of the IETF CIFS 1.0 specification was published in 1997. Later, extensions were made to address other scenarios like domains, Kerberos, shadow copy, server to server copy and SMB signing. Windows 2000 (released in 2000) included those extensions. At that time, some people went back to calling the protocol SMB once again. CIFS/SMB has also been implemented on Unix, Linux and many other operating systems (either as part of the OS or as a server suite like Samba). A few times, those communities also extended the CIFS/SMB protocol to address their own specific requirements.

One important limitation of SMB was its "chattiness" and lack of concern for network latency. It would take a series of synchronous round trips to accomplish many of the most common tasks. The protocol was not created with WAN or high-latency networks in mind and there was limited use of compounding (combining multiple commands in a single network packet) or pipelining (sending additional commands before the answer to a previous command arrives). This even led to products created to address the specific issues around SMB WAN acceleration. There were also limitations regarding the number of open files, shares and users. Due to the large number of commands and subcommands, the protocol was also difficult to extend, maintain and secure.

Introducing SMB2

The first major redesign of SMB happened with the release of SMB2 by Microsoft. SMB2 was introduced with Windows Vista in 2007 and updated with the release of Windows Server 2008 and Windows Vista SP1 in 2008.

SMB2 brought a number of improvements, including but not limited to:

Reduced complexity, going from over 100 commands and subcommands to just 19 (see details below)
General mechanisms for data pipelining and credit-based flow control (see details below)
Request compounding, which allows multiple SMB requests to be sent as a single network request(see details below)
Larger reads and writes make better use of faster networks, even with high latency
Caching of folder and file properties, where clients keeps local copy of information on folders and files
Durable handles allow an SMB2 connection to transparently reconnect to the server if there is a temporary loss of network connectivity
Message signing improved (HMAC SHA-256 replaces MD5 as hashing algorithm) and configuration/interoperability issues simplified
Improved scalability for file sharing (number of users, shares and open files per server greatly increased)
Protocol works well with Network Address Translation (VC count is gone)
Extension mechanism (for instance, create context or variable offsets)
Support for symbolic links

It is important to highlight that, to ensure interoperability, SMB2 uses the existing SMB1 connection setup mechanisms, and then advertises that it is capable of a new version of the protocol. Because of that, if the opposite end does not support SMB2, SMB1 will be used.

The SMB2 protocol specification was published publicly by Microsoft and you can find the link at the end of this post.

Reduced Complexity

One of the ways to showcase the reduced complexity in SMB2 is to make a comparison to the commands and subcommands in the old version.

Here is the complete list of the 19 opcodes (or commands) used by SMB2 in the message exchanges between the client and the server, grouped in three categories:

Protocol negotiation, user authentication and share access (NEGOTIATE, SESSION_SETUP, LOGOFF, TREE_CONNECT, TREE_DISCONNECT)
File, directory and volume access (CANCEL, CHANGE_NOTIFY, CLOSE, CREATE, FLUSH, IOCTL, LOCK, QUERY_DIRECTORY, QUERY_INFO, READ, SET_INFO, WRITE)
Other (ECHO, OPLOCK_BREAK)

When you try to get a similar list for the old SMB, things get a little more complex. I tried to make a list of all commands and subcommands using only the documents linked below and came up with over 100:

Protocol negotiation, user authentication and share access (NEGOTIATE, SESSION_SETUP_ANDX, TRANS2_SESSION_SETUP, LOGOFF_ANDX, PROCESS_EXIT, TREE_CONNECT, TREE_CONNECT_ANDX, TREE_DISCONNECT)
File, directory and volume access (CHECK_DIRECTORY, CLOSE, CLOSE_PRINT_FILE, COPY, CREATE, CREATE_DIRECTORY, CREATE_NEW, CREATE_TEMPORARY, DELETE, DELETE_DIRECTORY, FIND_CLOSE, FIND_CLOSE2, FIND_UNIQUE, FLUSH, GET_PRINT_QUEUE, IOCTL, IOCTL_SECONDARY, LOCK_AND_READ, LOCK_BYTE_RANGE, LOCKING_ANDX, MOVE, NT_CANCEL, NT_CREATE_ANDX, NT_RENAME, NT_TRANSACT, NT_TRANSACT_CREATE, NT_TRANSACT_IOCTL, NT_TRANSACT_NOTIFY_CHANGE, NT_TRANSACT_QUERY_QUOTA, NT_TRANSACT_QUERY_SECURITY_DESC, NT_TRANSACT_RENAME, NT_TRANSACT_SECONDARY, NT_TRANSACT_SET_QUOTA, NT_TRANSACT_SET_SECURITY_DESC, OPEN, OPEN_ANDX, OPEN_PRINT_FILE, QUERY_INFORMATION, QUERY_INFORMATION_DISK, QUERY_INFORMATION2, READ, READ_ANDX, READ_BULK, READ_MPX, READ_RAW, RENAME, SEARCH, SEEK, SET_INFORMATION, SET_INFORMATION2, TRANS2_CREATE_DIRECTORY, TRANS2_FIND_FIRST2, TRANS2_FIND_NEXT2, TRANS2_FIND_NOTIFY_FIRST, TRANS2_FIND_NOTIFY_NEXT, TRANS2_FSCTL , TRANS2_GET_DFS_REFERRAL, TRANS2_IOCTL2, TRANS2_OPEN2, TRANS2_QUERY_FILE_INFORMATION, TRANS2_QUERY_FS_INFORMATION, TRANS2_QUERY_PATH_INFORMATION, TRANS2_QUERY_PATH_INFORMATION, TRANS2_REPORT_DFS_INCONSISTENCY, TRANS2_SET_FILE_INFORMATION, TRANS2_SET_FS_INFORMATION, TRANS2_SET_PATH_INFORMATION, TRANSACTION, TRANSACTION_SECONDARY, TRANSACTION2, TRANSACTION2_SECONDARY, UNLOCK_BYTE_RANGE, WRITE, WRITE_AND_CLOSE, WRITE_AND_UNLOCK, WRITE_ANDX, WRITE_BULK, WRITE_BULK_DATA, WRITE_COMPLETE, WRITE_MPX, WRITE_MPX_SECONDARY, WRITE_PRINT_FILE, WRITE_RAW)
Other (ECHO, TRANS_CALL_NMPIPE, TRANS_MAILSLOT_WRITE, TRANS_PEEK_NMPIPE, TRANS_QUERY_NMPIPE_INFO, TRANS_QUERY_NMPIPE_STATE, TRANS_RAW_READ_NMPIPE, TRANS_RAW_WRITE_NMPIPE, TRANS_READ_NMPIPE, TRANS_SET_NMPIPE_STATE, TRANS_TRANSACT_NMPIPE, TRANS_WAIT_NMPIPE, TRANS_WRITE_NMPIPE)

I make no claim that the list above for SMB is exact or complete, but it does make a point. As an interesting exercise, check the lists above to verify that, while SMB2 has a single WRITE operation, there are 14 distinct WRITE operations in the old protocol.

SMB2 also requires TCP as a transport. SMB2 no longer supports NetBIOS over IPX, NetBIOS over UDP or NetBEUI (as SMB version 1 did).

Pipelining

A key improvement in SMB2 is the way it makes it easy for clients to send a number of outstanding requests to a server. This allows the client to build a pipeline of requests instead of waiting for a response before sending the next request. This is especially relevant when using a high latency network.

SMB2 uses a credit based flow control, which allows the server to control a client's behavior. The server will start with a small number of credits and automatically scale up as needed. With this, the protocol can keep more data "in flight" and better utilize the available bandwidth.

This is key to make a large transfer go from hours (in SMB) to minutes (in SMB2) in a "long and fat pipe" (high bandwidth, high latency network).

For an example of how pipelining in SMB2 can improve performance, check out this blog post.

Compounding

When you look at the command set for the new SMB2 protocol, you notice that they are all simple operations. The old SMB1 protocol had some complex commands and subcommands that combined a set of simple operations as required in specific scenarios.

One of the important changes in SMB2 is the ability to send an arbitrary set of commands in a single request (single network round trip). This is called compounding and it can be use to mimic the old complex operations in SMB1 without the added complexity of a larger command set.

For instance, an old SMB1 RENAME command can be replaced by a single request in SMB2 that combines three commands: CREATE (which can create a new file or open an existing file), SET_INFO and CLOSE. The same can be done for many other complex SMB1 commands and subcommands like LOCK_AND_READ and WRITE_AND_UNLOCK.

This compounding ability in SMB2 is very flexible and the chain of commands can be unrelated (executed separately, potentially in parallel) or related (executed in sequence, with the output of one command available to the next). The responses can also be compounded or sent separately.

This new compounding feature in SMB2 can be used to perform a specific task in less time due to the reduced number of network round trips.

Conclusion

I hope this post has helped you understand some of the important improvements in SMB2, particularly in regards to reduced complexity, pipelining and compounding.

Reference

Below is a list of important links that document SMB2, SMB and CIFS, including the latest protocol specifications published by Microsoft:

Microsoft at AJAXWorld

Peter Galli — Mon, 20 Oct 2008 22:05:00 GMT

The AJAXWorld Conference and Expo got underway in San Jose today, under the broad theme of "Rich Web Technologies for Enterprise Web 2.0 & Social Web."

Scott Guthrie, a corporate vice president at Microsoft, delivered the keynote address today, while Brad Abrams, the Product Unit Manager for the AppFx team here at Microsoft, will also be presenting a couple of sessions over the next few days.

While Microsoft is not announcing anything new at the show, Abrams will point to the cumulative effect of all the work that has been done so far. He will talk about how JQuery will ship with future versions of Visual Studio, as well as how standards based JavaScript, CSS, and HTML are all now supported in Visual Studio 2008.

Attendees will also get to see how JQuery interoperability and usage is now on the same page with ASP.NET Ajax, as well as demos of the cross-browser, cross-platform Silverlight plug-in, which works on Firefox, Safari and Internet Explorer, and on both Macs and Windows machines.

Abrams tells me he also plans to show off a couple of Silverlight sites running on Linux with the Novell Moonlight implementation of Silverlight, as well as a demo of its Silverlight tools for Eclipse, which underscores how developers can use the tools they want to leverage Silverlight.

He has also posted a lot more detail on his talk, which was titled "Building a Great Ajax application from Scratch," in this blog post.

Also, Bryan Kirschner of the platform strategy group, will be delivering a keynote address on how Microsoft participates in a world of choice at GOSCON, the Government Open Source Conference, in Portland tomorrow.

In addition, two other colleagues from Microsoft will also be participating. Stuart McKee, the National Technology Office for the U.S., will be on the Government Open Collaboratives Panel with Brian, and Kathleen Connor from Microsoft's Health Solutions Group, will also be speaking.

CodePlex launches support for TortoiseSVN

jcannon — Tue, 16 Sep 2008 17:24:00 GMT

From our very own Sara Ford, word comes of Codeplex now offering server support for SvnBridge. SvnBridge enables TortoiseSVN to talk to Team Foundation Server. From Sara, "Support for Subversion has been our number one requested feature, and by hosting SvnBridge, our users can now use their favorite Subversion client with any project." Read the full blog here.

A picture says a thousand words:

Code Parallel or Die

Frank Chism — Mon, 14 Apr 2008 17:22:00 GMT

“What we have here is a failure to communicate.”
- Luke in “Cool Hand Luke”

A fist full of cores in a server rich environment.
If you haven’t noticed that the world has changed, you had better wake up and smell the coffee. The era of killer apps written in serial code is ending. Like the norm in species extinction, the reason many ISVs will not survive this change is that the environment in which we develop code has changed rapidly and not all species can adapt to this new world. The change was preceded by a hint of ‘The Shape of Things to Come’ in the Beowulf and compute cluster revolution. This revolution created the first generation of commodity clusters based on microprocessor nodes connected by commodity networks. An evolutionary message from this generation is that if you are building a cluster you really ought to use server class nodes. You _can_ build a cluster from desktop parts, but the higher you scale your node count the more you want the convenience and reliability of server parts. I remember seeing a look of outrage on the face of an Intel marketing person when I commented that I thought their then brand new Itanium Tiger platform was a really nice part for building a cluster. He thought of the new platform as a High Performance Server, not as a ‘part’. Well get used to it. We don’t build computers out of discreet components like transistors any more (which is good for him because Intel makes really big Integrated Circuits, not transistors) and we have just moved up to the point where the level of integration is the server, not the processor.

And Then There Were Cores
So into this environment where servers are parts and many high performance computing applications were written to a distributed memory communications aware parallel programming model, the microprocessor has changed in the most radical way I can remember. Our equilibrium has been punctuated (see: "Punctuated equilibria: an alternative to phyletic gradualism" (1972) pp 82-115 in "Models in paleobiology", edited by Schopf, TJM Freeman, Cooper & Co, San Francisco. Eldredge, N. & Gould, S.J.) and we can expect rapid change to a whole new kind of parallelism. The change I refer to is the introduction and near ubiquity of multiple core processors. For years software developers have been able to count on dramatic increases in CPU speed and processing power on a Moore’s Law schedule. But, Moore did not say that the processing speed and power of CPUs would double every eighteen months, he said that the number of gates on a single chip would double every eighteen months. So, all was good for a long time because that doubling of gates meant smaller _and_ faster gates and greater architectural complexity and thus ‘faster’ CPUs. So by about 2005, just about every good idea for a better computer processor architecture ever thought of was incorporated into the major microprocessors on the market. So, when 2006 rolled around the obvious thing to do was to use the next doubling of gates to double the number of ‘processors’ on a microprocessor. Dual core was here and the trend will continue for at least a couple of doublings. Who knows what strange beasties will emerge when we get to the point where ‘many core’ processors become ‘too many core’ processors. Until then, though we can be sure: Multicore killed the serial star....

More on this tomorrow.

Pilot

jcannon — Wed, 19 Dec 2007 11:49:00 GMT

I spend a significant amount of my time crisscrossing Microsoft, looking for (and advocating) interesting ways that our research and development teams are adopting open approaches in their work. It’s not terribly difficult - and, in fact, increasingly easy to find areas where sharing code, participating in community and collaborating with the commercial open source industry are part of what we do every day. As part of my role as an open source community & platforms lead, growing this list is core to my job. And in blogging more actively on Port 25, I'm excited about discussing and expanding this list out in the open.

That said, once in awhile something different comes along...and in my crisscrossing this weekend, something different popped for me. As a former web developer, I have a special place in my heart for the developer tools we offer to build web experiences. I can recall using FrontPage '97 to develop simple WYSIWYG websites (*WYSIWYG, admittedly, was on a journey then...still is) and patching together Imagemaps with hand-calc'd coordinates; the joy of Photoshop and installing Kai's Power Tools for the visual fun of it, or the pain of Paint Shop Pro and transparent GIFs circa 1996. Dreamweaver was doing some interesting things and Visual Studio was starting to get more and more web-centric. Ahhh, the days of Web 1.0.

Additionally, as a former IS major, I also have a special place in my heart for development on a budget. Those dispositions keep me acutely aware of what goes on with our Express products - and it caught my eye when I started skimming across the refreshed Express site.

A lot of folks aren't aware that Microsoft offers a free-as-in-beer line of development tools for application, game & web development. We call these 'Express' - and there are four editions: Visual Web Developer 2008, Visual C++ 2008 Express, Visual Basic 2008 Express and Visual C# 2008 Express. There is even a free version of SQL Server 2005 in Express which provides complimentary database services during development. They've been available for a few years, but they were recently rev'd with enhancements from Visual Studio 2008.
With my job at Microsoft focused on open source so acutely, I figured I would take them for a test drive & see how easy it would be to get started. The real test, though? How easy would it be to start using this app if I was a Ruby, or PHP developer (Truth be told - I am not, nor was I during my development days) – but these are different times and those are popular choices. It’s a quick litmus test I’m thinking of using more often…because I do get asked frequently about Microsoft’s support of programming languages and frameworks beyond .NET.

Setup is straightforward - download, run the wizard & choose additional (free) documentation and development options like SQL Express. Click Next & you're set on your way. Note: you do need Windows XP or later to run an Express tool.

After VS Express is installed, the welcome center offers multiple ways to get started via community sample code, beginning programming resources, guided videos and relevant articles from a variety of online communities.

So I tried a quick test. In two-clicks, I went to "Help" and "Search" - and typed in 'PHP'. Here are the automatic & dynamically returned search results:

I immediately get source code samples to begin programming (from MSDN); SQL Server samples for data access and more from the Codezone Community and direct links into community forum posts. Right away, these are pretty useful tools to get started with.

Ruby is a very popular language - so let's try this one more time with 'Ruby':

Again, that’s not bad for a quick and dirty search. I get relevant programming articles, opinion pieces, sample code and more. It’s all hyperlinked out into the web, so I can easily jump-off and explore at my own discretion, or as I’m inclined to do…lose myself in a trail of links, only to recall my original point hours later. (Case in point )

The point I walked away with, in all seriousness, is that the spirit of Port25 is spreading at Microsoft & the proof is in the programming. It's exciting to see these offerings baked in from day one & I encourage you to take one of the Express offerings for a spin & post your feedback - what are your impressions?

On Port 25, I'll continue to highlight examples like this, and expand on my role at Microsoft and how we're working on growing the role of open source within Microsoft's DNA. For now, though, I have to dust off my copy of Jakob Nielsen's Designing Web Usability and wax nostalgic with some old friends.

[PostIcon:3352]

Hank Janssen & John Bocharov: Introduction to SQL Server Driver for PHP (SQLPHP)

jcannon — Thu, 08 Nov 2007 18:14:00 GMT

Our own Hank Janssen gives the Channel9 team an update on the work that has been done to provide a native driver to SQL Server for PHP.

"SQL Team Says:
"The SQL Server Driver for PHP (October 2007) Community Technology Preview (CTP) is designed to enable reliable, scalable integration with SQL Server for PHP applications deployed on the Windows platform. The Driver for PHP is a PHP 5 extension that allows the reading and writing of SQL Server data from within PHP scripts. It provides a procedural interface for accessing data in all Editions of SQL Server 2005 and SQL Server 2000 (including Express Edition), and makes use of PHP features, including PHP streams to read and write large objects."

Wow. This is cool. Need to find out more about this. What exactly is this thing? Why did we create it? What are the platform requirments? Is it open source? Who are the folks behind this? You know the C9 drill. Tune in and meet SQLPHP Program Manager John Bocharov and Microsoft Open Source champion Hank Janssen who answer a bunch of questions and provide good context about the thinking behind SQLPHP, history and future. Check it out.

Releasing the Source Code for the .NET Framework Libraries

jcannon — Wed, 03 Oct 2007 21:47:00 GMT

Some news from Scott Guthrie's blog this morning - we'll get more from the .NET and Shared Source team over the next few months:

"One of the things my team has been working to enable has been the ability for .NET developers to download and browse the source code of the .NET Framework libraries, and to easily enable debugging support in them.

Today I'm excited to announce that we'll be providing this with the .NET 3.5 and VS 2008 release later this year.

We'll begin by offering the source code (with source file comments included) for the .NET Base Class Libraries (System, System.IO, System.Collections, System.Configuration, System.Threading, System.Net, System.Security, System.Runtime, System.Text, etc), ASP.NET (System.Web), Windows Forms (System.Windows.Forms), ADO.NET (System.Data), XML (System.Xml), and WPF (System.Windows). We'll then be adding more libraries in the months ahead (including WCF, Workflow, and LINQ). The source code will be released under the Microsoft Reference License (MS-RL).

You'll be able to download the .NET Framework source libraries via a standalone install (allowing you to use any text editor to browse it locally). We will also provide integrated debugging support of it within VS 2008.

Integrated Visual Studio 2008 Debugging Support

The final release of VS 2008 will support the ability to configure the debugger to dynamically download the .NET Framework debugger symbols (and corresponding source code) from a web server hosted by Microsoft. You'll be able to configure the .NET Framework symbols to be downloaded all in one shot, or manually retrieved on demand:

Read the rest of the post at Scott Guthrie's Blog....

PHP on IIS

jcannon — Mon, 24 Sep 2007 20:01:00 GMT

Just about one year ago, Bill Hilf announced Zend and Microsoft's partnership to enhance PHP performance on Windows Server's IIS web server. This technical collaboration has focused on enhancing the reliability and performance of PHP on Windows Server 2003, and Windows Server 2008. As part of this collaboration, the IIS product group has been working on a new component for IIS6 and IIS7 called FastCGI Extension which will enable IIS to much more effectively host PHP applications.

Today, we're excited to announce the release of the Go Live version of Microsoft FastCGI Extension for IIS 5.1/6.0 (FastCGI Extension) as a free download. There's a ton more information on various community sites, so let's get right to it:

Download the FastCGI Extension (You'll need a running install of Windows Server. If you don't already, you can grab Windows Server 2003 R2 here.)
Detailed release information on IIS.NET and configuration documentation.
Join the PHP community discussion on IIS.NET

Would like to hear feedback from our community on this release and will ensure it is routed back to the OSSL and the IIS product team.

- Jamie