Port 25: The Open Source Community at Microsoft : .NET Development, Dev Center

Expanding the interoperability of eclipse4SL: Mac support comes to the Eclipse for Silverlight project

Vijay Rajagopalan — Wed, 18 Mar 2009 18:15:00 GMT

In October 2008, when Microsoft announced the general availability of Silverlight 2.0, and, as part of Microsoft’s ongoing commitment to interoperability, Microsoft and Soyatec unveiled the Eclipse Tools for Silverlight (Eclipse4SL).

Today, Microsoft and Soyatec are expanding our interoperability collaboration by releasing a Community Technology Preview (CTP) of the eclipse4SL project that includes support for the Macintosh platform. Mac and Windows developers can now collaborate on Silverlight projects using the Eclipse or Visual Studio IDEs. We have also added several new features to the project that should please all developers like C# code generation and improved XAML auto-completion (check out our Interoperability Team blog for more details and a demo)

As an open source initiative sponsored by Microsoft (funding and architectural guidance) and led by Soyatec (development), the eclipse4SL project is released under the Eclipse Public License Version 1.0 on SourceForge.net and was submitted by Soyatec it to the Eclipse Foundation as an open Eclipse project. Since its inception the project has received lots of feedback and made significant progress. eclipse4SL has been among the “Top Rated” projects on www.eclipseplugincentral.com for weeks:

(Screenshot taken on 03/16/2009)

For more information, visit the Eclipse for Silverlight web site, or join the discussion at http://www.eclipse4sl.org/community/.

Vijay Rajagopalan, Principal Architect in the Interoperability Strategy Team at Microsoft.

Moonlight 1.0 Hits the Street

Peter Galli — Wed, 11 Feb 2009 19:42:00 GMT

Moonlight 1.0 is now available.

Moonlight is an open source project that gives Linux users access to Microsoft Silverlight content, and is available for all major Linux distributions, including openSUSE, SUSE Linux Enterprise, Fedora, Red Hat, and Ubuntu. This milestone release is part of the technical collaboration between Novell and Microsoft.

Microsoft has worked with the Moonlight team and Novell to enable interoperability between Windows and Linux platforms and extend the high-quality interactive Web and video experience for the benefit of the Linux community, said Scott Guthrie, corporate vice president of Microsoft's .NET Developer Division.

Microsoft has provided Novell with access to its test suites for Silverlight, and provides Linux end users of Moonlight with free access to the Microsoft Media Pack, a set of licensed media codecs for video and audio that bring optimized and licensed decoders to every Linux user using Moonlight. Windows Media Video (.wmv), Windows Media Audio (.wma) and MP3 files are supported through the Microsoft Media Pack.

A pre-release of Moonlight was made available on January 19, 2009 to allow Linux users to stream Barack Obama's Inauguration, and more than 20,000 Linux users downloaded Moonlight to watch that Silverlight broadcast.

"Microsoft Silverlight offers the most comprehensive and powerful solution for the creation and delivery of rich internet applications and media experiences, and is used by hundreds of thousands of developers worldwide," Guthrie said.

For his part Miguel de Icaza, the founder of the Mono project founder and vice president of Developer Platforms at Novell, said Moonlight brings the benefits of Silverlight's popular multimedia content to Linux viewers. "This first release delivers on the goal of breaking down barriers to multimedia content and creating parity in the user's viewing experience regardless of whether the user is on Windows or Linux."

SMB2: a Complete Redesign of the Main Remote File Protocol for Windows

Peter Galli — Mon, 08 Dec 2008 16:24:00 GMT

SMB (Server Message Block) is a remote file protocol commonly used by Microsoft Windows clients and servers that dates back to 1980's.

Back when it was first used, LANs speeds were typically 10Mbps or less, WAN use was very limited and there were no Wireless LANs. Network security concerns like preventing man-in-the-middle attacks were non-existent at that time.

Obviously, things have changed a lot since then. SMB did evolve over time, but it did so incrementally and with great care for keeping backward compatibility. It was only with SMB2 in 2007 that we had the first major redesign.

In this blog Jose Barreto, a senior technical evangelist in Microsoft's Storage Solutions Division, explains some of the history behind the protocol and outlines important improvements in SMB2, particularly in regards to reduced complexity, pipelining and compounding.

Introduction

SMB (Server Message Block) is a remote file protocol commonly used by Microsoft Windows clients and servers that dates back to 1980's. Back when it was first used, LANs speeds were typically 10Mbps or less, WAN use was very limited and there were no Wireless LANs. Network security concerns like preventing man-in-the-middle attacks were non-existent at that time. Obviously, things have changed a lot since then. SMB did evolve over time, but it did so incrementally and with great care for keeping backward compatibility. It was only with SMB2 in 2007 that we had the first major redesign.

A History of SMB and CIFS

When it was first introduced to the public, the remote file protocol was called SMB (Server Message Block). SMB was used, for instance, by Microsoft LAN Manager in 1987 and by Windows for Workgroups in 1992. Later, a draft specification was submitted to the IETF under the name Common Internet File System (CIFS). The CIFS specification is a description of the protocol as it was implemented in 1996 as part of Microsoft Windows NT 4.0. A preliminary draft of the IETF CIFS 1.0 specification was published in 1997. Later, extensions were made to address other scenarios like domains, Kerberos, shadow copy, server to server copy and SMB signing. Windows 2000 (released in 2000) included those extensions. At that time, some people went back to calling the protocol SMB once again. CIFS/SMB has also been implemented on Unix, Linux and many other operating systems (either as part of the OS or as a server suite like Samba). A few times, those communities also extended the CIFS/SMB protocol to address their own specific requirements.

One important limitation of SMB was its "chattiness" and lack of concern for network latency. It would take a series of synchronous round trips to accomplish many of the most common tasks. The protocol was not created with WAN or high-latency networks in mind and there was limited use of compounding (combining multiple commands in a single network packet) or pipelining (sending additional commands before the answer to a previous command arrives). This even led to products created to address the specific issues around SMB WAN acceleration. There were also limitations regarding the number of open files, shares and users. Due to the large number of commands and subcommands, the protocol was also difficult to extend, maintain and secure.

Introducing SMB2

The first major redesign of SMB happened with the release of SMB2 by Microsoft. SMB2 was introduced with Windows Vista in 2007 and updated with the release of Windows Server 2008 and Windows Vista SP1 in 2008.

SMB2 brought a number of improvements, including but not limited to:

Reduced complexity, going from over 100 commands and subcommands to just 19 (see details below)
General mechanisms for data pipelining and credit-based flow control (see details below)
Request compounding, which allows multiple SMB requests to be sent as a single network request(see details below)
Larger reads and writes make better use of faster networks, even with high latency
Caching of folder and file properties, where clients keeps local copy of information on folders and files
Durable handles allow an SMB2 connection to transparently reconnect to the server if there is a temporary loss of network connectivity
Message signing improved (HMAC SHA-256 replaces MD5 as hashing algorithm) and configuration/interoperability issues simplified
Improved scalability for file sharing (number of users, shares and open files per server greatly increased)
Protocol works well with Network Address Translation (VC count is gone)
Extension mechanism (for instance, create context or variable offsets)
Support for symbolic links

It is important to highlight that, to ensure interoperability, SMB2 uses the existing SMB1 connection setup mechanisms, and then advertises that it is capable of a new version of the protocol. Because of that, if the opposite end does not support SMB2, SMB1 will be used.

The SMB2 protocol specification was published publicly by Microsoft and you can find the link at the end of this post.

Reduced Complexity

One of the ways to showcase the reduced complexity in SMB2 is to make a comparison to the commands and subcommands in the old version.

Here is the complete list of the 19 opcodes (or commands) used by SMB2 in the message exchanges between the client and the server, grouped in three categories:

Protocol negotiation, user authentication and share access (NEGOTIATE, SESSION_SETUP, LOGOFF, TREE_CONNECT, TREE_DISCONNECT)
File, directory and volume access (CANCEL, CHANGE_NOTIFY, CLOSE, CREATE, FLUSH, IOCTL, LOCK, QUERY_DIRECTORY, QUERY_INFO, READ, SET_INFO, WRITE)
Other (ECHO, OPLOCK_BREAK)

When you try to get a similar list for the old SMB, things get a little more complex. I tried to make a list of all commands and subcommands using only the documents linked below and came up with over 100:

Protocol negotiation, user authentication and share access (NEGOTIATE, SESSION_SETUP_ANDX, TRANS2_SESSION_SETUP, LOGOFF_ANDX, PROCESS_EXIT, TREE_CONNECT, TREE_CONNECT_ANDX, TREE_DISCONNECT)
File, directory and volume access (CHECK_DIRECTORY, CLOSE, CLOSE_PRINT_FILE, COPY, CREATE, CREATE_DIRECTORY, CREATE_NEW, CREATE_TEMPORARY, DELETE, DELETE_DIRECTORY, FIND_CLOSE, FIND_CLOSE2, FIND_UNIQUE, FLUSH, GET_PRINT_QUEUE, IOCTL, IOCTL_SECONDARY, LOCK_AND_READ, LOCK_BYTE_RANGE, LOCKING_ANDX, MOVE, NT_CANCEL, NT_CREATE_ANDX, NT_RENAME, NT_TRANSACT, NT_TRANSACT_CREATE, NT_TRANSACT_IOCTL, NT_TRANSACT_NOTIFY_CHANGE, NT_TRANSACT_QUERY_QUOTA, NT_TRANSACT_QUERY_SECURITY_DESC, NT_TRANSACT_RENAME, NT_TRANSACT_SECONDARY, NT_TRANSACT_SET_QUOTA, NT_TRANSACT_SET_SECURITY_DESC, OPEN, OPEN_ANDX, OPEN_PRINT_FILE, QUERY_INFORMATION, QUERY_INFORMATION_DISK, QUERY_INFORMATION2, READ, READ_ANDX, READ_BULK, READ_MPX, READ_RAW, RENAME, SEARCH, SEEK, SET_INFORMATION, SET_INFORMATION2, TRANS2_CREATE_DIRECTORY, TRANS2_FIND_FIRST2, TRANS2_FIND_NEXT2, TRANS2_FIND_NOTIFY_FIRST, TRANS2_FIND_NOTIFY_NEXT, TRANS2_FSCTL , TRANS2_GET_DFS_REFERRAL, TRANS2_IOCTL2, TRANS2_OPEN2, TRANS2_QUERY_FILE_INFORMATION, TRANS2_QUERY_FS_INFORMATION, TRANS2_QUERY_PATH_INFORMATION, TRANS2_QUERY_PATH_INFORMATION, TRANS2_REPORT_DFS_INCONSISTENCY, TRANS2_SET_FILE_INFORMATION, TRANS2_SET_FS_INFORMATION, TRANS2_SET_PATH_INFORMATION, TRANSACTION, TRANSACTION_SECONDARY, TRANSACTION2, TRANSACTION2_SECONDARY, UNLOCK_BYTE_RANGE, WRITE, WRITE_AND_CLOSE, WRITE_AND_UNLOCK, WRITE_ANDX, WRITE_BULK, WRITE_BULK_DATA, WRITE_COMPLETE, WRITE_MPX, WRITE_MPX_SECONDARY, WRITE_PRINT_FILE, WRITE_RAW)
Other (ECHO, TRANS_CALL_NMPIPE, TRANS_MAILSLOT_WRITE, TRANS_PEEK_NMPIPE, TRANS_QUERY_NMPIPE_INFO, TRANS_QUERY_NMPIPE_STATE, TRANS_RAW_READ_NMPIPE, TRANS_RAW_WRITE_NMPIPE, TRANS_READ_NMPIPE, TRANS_SET_NMPIPE_STATE, TRANS_TRANSACT_NMPIPE, TRANS_WAIT_NMPIPE, TRANS_WRITE_NMPIPE)

I make no claim that the list above for SMB is exact or complete, but it does make a point. As an interesting exercise, check the lists above to verify that, while SMB2 has a single WRITE operation, there are 14 distinct WRITE operations in the old protocol.

SMB2 also requires TCP as a transport. SMB2 no longer supports NetBIOS over IPX, NetBIOS over UDP or NetBEUI (as SMB version 1 did).

Pipelining

A key improvement in SMB2 is the way it makes it easy for clients to send a number of outstanding requests to a server. This allows the client to build a pipeline of requests instead of waiting for a response before sending the next request. This is especially relevant when using a high latency network.

SMB2 uses a credit based flow control, which allows the server to control a client's behavior. The server will start with a small number of credits and automatically scale up as needed. With this, the protocol can keep more data "in flight" and better utilize the available bandwidth.

This is key to make a large transfer go from hours (in SMB) to minutes (in SMB2) in a "long and fat pipe" (high bandwidth, high latency network).

For an example of how pipelining in SMB2 can improve performance, check out this blog post.

Compounding

When you look at the command set for the new SMB2 protocol, you notice that they are all simple operations. The old SMB1 protocol had some complex commands and subcommands that combined a set of simple operations as required in specific scenarios.

One of the important changes in SMB2 is the ability to send an arbitrary set of commands in a single request (single network round trip). This is called compounding and it can be use to mimic the old complex operations in SMB1 without the added complexity of a larger command set.

For instance, an old SMB1 RENAME command can be replaced by a single request in SMB2 that combines three commands: CREATE (which can create a new file or open an existing file), SET_INFO and CLOSE. The same can be done for many other complex SMB1 commands and subcommands like LOCK_AND_READ and WRITE_AND_UNLOCK.

This compounding ability in SMB2 is very flexible and the chain of commands can be unrelated (executed separately, potentially in parallel) or related (executed in sequence, with the output of one command available to the next). The responses can also be compounded or sent separately.

This new compounding feature in SMB2 can be used to perform a specific task in less time due to the reduced number of network round trips.

Conclusion

I hope this post has helped you understand some of the important improvements in SMB2, particularly in regards to reduced complexity, pipelining and compounding.

Reference

Below is a list of important links that document SMB2, SMB and CIFS, including the latest protocol specifications published by Microsoft:

Featured Project: Family.Show on Codeplex

jcannon — Tue, 27 May 2008 16:09:00 GMT

Over the course of the past year, we've highlighted various community and open source projects on Codeplex. This morning, I wanted to do the same with a very cool open source project that I came across over the weekend called Family.Show. Family.Show is a genealogy project that visualizes family mapping and relationships. The project was released in July 2007 and it's currently at version 2. While the project is backed by Vertigo Software, it is Open Source and licensed under the Microsoft Public License.

Here's a small screen grab from the project home page:

Over the weekend, I used some free time & a family BBQ to start building my own family tree. Frankly, while I'm impressed with the application and it's ease-of-use, I was more amazed at the interest and enjoyment everyone in my family experienced using it. Everybody from my 83 year old grandmother to my 19 year old sister were excited to explore and add to the family tree. It gives new meaning to the collaborative potential of open source :>

See above, the Cannon family tree (yikes) - but infinitely fascinating to self-described family historians. It's worth noting that Family.Show also supports the import/export of GEDCOM standard files, so your work can interchange with other genealogy software packages as well.

I encourage anyone interested to download & tinker with it. Additionally, here are some additional resources:

Visit the Codeplex project home page
Family.Show at MIX07
Watch a Channel9 Video with the core development team

-Jamie

Supernova

Sam Ramji — Wed, 19 Mar 2008 16:22:00 GMT

I’m writing this from EclipseCon in Santa Clara, California, where I’m going to announce the beginning of Microsoft’s collaborative work with the Eclipse Foundation.

This started about a year ago when I met Mike Milinkovich at an open source event (the Open Source Software Think Tank 2007) where we were seated at the same table, and assigned to discuss “key issues inhibiting the growth of open source”. We found we had pretty similar ways of looking at problems – I found Mike to be very pragmatic and straightforward in his thinking. That discussion led to a conversation about what we could do to help Eclipse developers building software for Windows.

At the same time, the CardSpace team at Microsoft was already working actively with the Higgins Project to establish a secure, interoperable framework for user identity on the web – an architecture known as the Identity Metasystem. Since the inception of Higgins, the CardSpace team has worked very closely with the Higgins team, providing them the protocol documentation they needed to be able to build an identity selector that is interoperable with CardSpace, as well as placing those protocol specifications under the OSP so that they knew that it was safe to do so. We share a commitment to building a user-centric, privacy-preserving, secure, easy-to-use identity layer for the Internet.

Currently, Higgins, Microsoft, and dozens of other companies and projects are in the midst of the third OSIS-sponsored user-centric identity interop, where we all try our code together, providing the data needed to improve both our implementations and the interoperability between them.

Among a range of other opportunities (which we’re still working on), we discovered that Steve Northover (the SWT team lead) had gotten requests to make it easy for Java developers to write applications that look and feel like native Windows Vista. He and a small group of developers built out a prototype that enables SWT to use Windows Presentation Foundation (WPF). We’re committing to improve this technology with direct support from our engineering teams and the Open Source Software Lab, with the goal of a first-class authoring experience for Java developers.

This is exciting to me – as a Java developer in my prior life (as well as the first technical marketing manager for BEA’s WebLogic Workshop, now Apache Beehive) it just makes sense to enable Java on Windows. We started a collaborative effort with JBoss two years ago that continues to this day. At the end of the day, it’s all about the developer. There will be more to come from the conversations that Eclipse and Microsoft have begun, and I look forward to announcing those in the future as we have demonstrable technology results.

Cheers,
Sam

Releasing the Source Code for the .NET Framework Libraries

jcannon — Wed, 03 Oct 2007 21:47:00 GMT

Some news from Scott Guthrie's blog this morning - we'll get more from the .NET and Shared Source team over the next few months:

"One of the things my team has been working to enable has been the ability for .NET developers to download and browse the source code of the .NET Framework libraries, and to easily enable debugging support in them.

Today I'm excited to announce that we'll be providing this with the .NET 3.5 and VS 2008 release later this year.

We'll begin by offering the source code (with source file comments included) for the .NET Base Class Libraries (System, System.IO, System.Collections, System.Configuration, System.Threading, System.Net, System.Security, System.Runtime, System.Text, etc), ASP.NET (System.Web), Windows Forms (System.Windows.Forms), ADO.NET (System.Data), XML (System.Xml), and WPF (System.Windows). We'll then be adding more libraries in the months ahead (including WCF, Workflow, and LINQ). The source code will be released under the Microsoft Reference License (MS-RL).

You'll be able to download the .NET Framework source libraries via a standalone install (allowing you to use any text editor to browse it locally). We will also provide integrated debugging support of it within VS 2008.

Integrated Visual Studio 2008 Debugging Support

The final release of VS 2008 will support the ability to configure the debugger to dynamically download the .NET Framework debugger symbols (and corresponding source code) from a web server hosted by Microsoft. You'll be able to configure the .NET Framework symbols to be downloaded all in one shot, or manually retrieved on demand:

Read the rest of the post at Scott Guthrie's Blog....

ASP.NET AJAX Released!

MichaelF — Thu, 01 Feb 2007 21:34:00 GMT

I wanted to take a moment to let the Port 25 community that Microsoft has officially released ASP.NET AJAX to the web under the Microsoft Permissive License (Ms-PL). Under this release developers are free to modify the Microsoft AJAX Library Scripts and can distribute derivative works per the terms of the Ms-PL.

As part of the release some improvements have been made:

Performance and scalability have been improved for shared hosting scenarios
Globalization fixes to ScriptManager and ScriptResource handler to support date and number parsing and UI culture fallback

This week we also released the ASP.NET 2.0 AJAX Extensions source code under the Microsoft Reference License (Ms-RL). This release was intended to help the community with debugging, maintenance and interoperability challenges with the additional hope that the transparency helps establish better coding patterns and guidelines.

You can find ASP.NET AJAX here.

Check back tomorrow when we'll post an interview with Steve Marx, Technical Evangelist for ASP.NET AJAX who talks about the release and shows us a demonstration of the technology.

From Atlas to ASP.NET AJAX: Sam Interviews Brad Abrams

MichaelF — Thu, 30 Nov 2006 19:31:00 GMT

Brad Abrams, Group Program Manager for the .NET Framework, sits down with Sam to discuss all things AJAX including: the Open AJAX Alliance, Atlas, the Microsoft AJAX Library and cross browser compatability. Brad also does a quick demo near the end of the video.

Video: From Atlas to ASP.NET AJAX: Sam Interviews Brad Abrams

Related Links:

ASP.NET AJAX homepage: http://ajax.asp.net
Shanku Nyogi's (Product Unit Manager for UI Framework and Services Team) blog: http://www.shankun.com/Atlas_Php_2.aspx
Brad's Blog: http://blogs.msdn.com/brada

Video: From Atlas to ASP.NET AJAX: Sam Interviews Brad Abrams

Java and .NET Interoperability: JNBridge

MichaelF — Tue, 19 Sep 2006 20:57:00 GMT

Update: We thought we would try our videos on YouTube...would love to hear feedback on this implementation.
Sam interviews Wayne Citrin to discuss work his company, JNBridge, has done to provide interoperability between .NET and Java.

Wayne Citrin is the CTO at JNBridge. He is the architect of JNBridgePro, and has been engrossed in Java and .NET interop issues since .NET's beta days. Previously, Wayne was a leading researcher in programming languages and compilers, and was on the Computer Engineering faculty at the University of Colorado, Boulder. He was a researcher at IBM's research lab in Zürich, Switzerland, and has a PhD from the University of California, Berkeley, in Computer Science.

The JNBridge blog can be found here.

Alternate Video Formats
- Windows Media
-Download MP4 Video