File descriptor abstraction in Windows... - Port 25: The Open Source Community at Microsoft

File descriptor abstraction in Windows... by admin on May 24, 2006 06:11PM

Submitted by: Alexandre Ferrieux

Question:

Hello,

I'd like to describe what is my highest frustration at the unix-Windows boundary: the lack of 'file descriptor abstraction' in Windows.

Consider:

In unix everything is a file descriptor, on which you simply use
read(), write(), and select() regardless of the underlying reality (files,pipes,sockets,devices,pseudoterminals).

In windows you have a set of API for every new type, with a few bridges here and there with limited support (not even talking of Windows CE).

Here is my point: this may look like just aesthetic considerations (the sheer beauty of having fewer syscalls is irrelevant to the end user). But there is one catch: when it comes to *mixing* all these things together, complexity explodes in the Windows case, and in my case there are true show-stoppers.

More precisely: let's try single-threaded, event-driven programming with select()/poll()/WaitForMultipleObjects(). In unix it amounts to giving a list of file descriptors. In Windows it is superficially the same (with handles), but it is *not*, because many handle types are just not waitable. To circumvent that, of course there is overlapped IO. But it is only possible when you open the handle yourself (to allow overlapped mode), not for one you inherit from the parent (like stdin).

Then the only possible workaround is to spawn extra threads doing their blocking IO or type-specific wait. But when it comes to resource-challenged environments (like WinCE), spawning an extra thread is sometimes not an option.

Questions:

(1) Are MS aware of such limitations in WinCE and even XP ?

(2) Any smart workaround to save me right now ?

TIA,

-Alex

Answer (Jeffrey Snover, Architect: Administration Experience Platform):

I concur!

PowerShell provides a similar abstraction on top of the OS and are working with feature teams to get providers. We call these namespace providers. It is slightly different than the UNIX model but we think it is more powerful. Our design center was admin scripting so we need to provide these abstractions against the Registry, WMI, AD, SQL, CERT STORES, etc.

If you download PowerShell and explore the concept of DRIVES (Get-PSDRIVE) then the concept will become a little clearer.

I'll monitor the comments here if you have any additional questions/feedback.

Jeffrey Snover

Email Digg It! Del.icio.us SlashDot It!

Comments RSS

MaxZinal said:

Mr. Snover,

All these shiny new and great features of PowerShell (e.g. Monad) are not like a solution to this problem. The original post was about an API-level thing, low-level syscalls and libraries just above them. Asynchronous programming for Win32 is a real headache due to limitations iof WaitForMultipleObjects and friends.

It's a pity that higher-level systems (like PowerShell) have to provide additional level of abstraction (and complexity) to achive something that I should call basic functionality.

posted at 01:23PM 05/28/2006
Alex said:

Though it's not obvious from the header, I'm the original poster.
And yes, MaxZinal's right in that PowerShell is completely off-topic.

I'm focusing on limitations of the Win32 API, for resource-limited, optimized C code; these limitations strike hard when it comes to porting stuff that works in a few elegant lines of small-footprint code in unix. Even creating an extra thread to circumvent MS's patchy grasp of asynchronism is *not* an option in some situations, including mine.

Instead of blind ads for additional layers of complexity built on sandy grounds, I'd hoped you'd point me to smart tricks daily used by Microsoft in-house developers to survive anyway ? Or are there any ?

-Alex

posted at 10:31AM 05/29/2006
jsnover said:

Your point is well taken. Providing these functions in PowerShell is not the same as providing them as a core OS primative. I didn't mean to imply that. That said, there are a lot of admins whose needs will be fully met by providing this in PowerShell. Also, a number of PowerShell concepts are being picked up and used throughout the company so perhaps the same thing will happpen here in the Post Vista timeframe.

Jeffrey Snover

jps

posted at 11:23PM 05/30/2006
fluke said:

Q: "Are MS aware of such limitations in WinCE and even XP [when developing applictions]?"

A: "there are a lot of admins whose needs will be fully met by providing this in PowerShell [for Windows 2003 Server]"

Q: I am an Eskimo and with my previous system of trapping fish, I could monitor multiple fishing holes all at the same time using the "select()" style of fishing. Using your product, I have to "thread" myself multiple times to do the same thing which seems less efficient.

A: There are a lot of people whose needs will be fully met by Freezer. Just stop ice fishing all together and use Freezer to keep the fish you already have longer.

Q: Is there anyone interested in discussing the question actually asked?

I have an application that runs on ARM CPU based hardware that monitors sensors. The sensors are character devices which are read using normal file descriptors on *nix. Commands can be sent to retrieve data from the sensors via serial port (also a character device) and TCP connection. Also, there is the option for the customer to run their own appliction directly on the hardware which can send commands to our application via named pipes. All of the input methods (sensors device files, serial port, TCP connection and named pipes) can be monitored via a single select() call. Since Win CE runs on ARM, there should be the possiblity of switching to it for the sensor monitor hardware. However, porting the application runs into the same problem that Alexandre Ferrieux pointed out.

While the cost of Win CE licenses is reasonable for this project, telling the customers to replace all the ARM CPU with x86 hardware is not a reasonable option. Also, licensing Windows 2003 Server for every single hardware sensor is not an option. (SUA also does not seem to lend itself as an embedded solution). So, is there a solution to this problem or is all we going to get is "I concur" followed by some sort of cure all named PowerShell that really sounds more like snake oil when it comes to addressing porting my application?! Even if I rewrite the app in PowerShell, how does that get it working on WinCE for ARM?

I'm sorry if I come off as anti-Microsoft, but I am finding this supposed answer to be very frustrating! It really isn't helpful at all.

posted at 12:02PM 06/07/2006