Port 25: The Open Source Community at Microsoft : Bryan Kirschner, OSS Research

What We Do Every Day

Bryan Kirschner — Sun, 20 May 2007 21:53:00 GMT

I just read Bill and Sam’s “Business as Usual” post. It made me think about the fact Port 25 was established in part to apply the idea that “transparency increases trust” to the work we do with the lab. So I’m sitting down to do a blog entry that’s a bit longer than usual, but will provide transparency about why “business as usual” for me.

I previously blogged about a project we were starting to look at usability, human-computer interaction (HCI) and design rationale in open source development. I want to share how that came about and what I work on every day, over a period of about 12 months.

Part 1: Andrew Ko at Carnegie-Mellon (hi, Andrew) and folks from Microsoft Research (you rock, HIP) have done fascinating work on “Information Needs in Collocated Development Teams:” (emphasis added):

[In] a two-month field study of software developers at Microsoft. We took a broad look, observing 17 groups across the corporation, focusing on three specific questions:
What information do software developers’ seek?
Where do developers find this information?
What inhibits the acquisition of such information?
In our observations, we found several needs. The most difficult to satisfy were design questions: for example, developers needed to know the intent behind code already written and code yet to be written.

Code itself was a poor conductor—let’s call it bad currency, for reasons that will become apparent later—for transmission of design knowledge. From the MSR paper:

code did not look like design; intent could rarely be inferred from code; programming languages only allowed a single, structural perspective on code, yet there were many other perspectives on which developers reasoned about code

As a result, “the knowledge was primarily stored in the minds of developers. Consequently, developers relied on each other for design knowledge.” A common way to do this was face-to-face contact. Another way to do this was through email.

Part 2: Flore Barcellini (hi, Flore) is a research at INRIA (France) who has done a fascinating analysis of “Thematic Coherence and Quotation Practices in OSS Design-Oriented Online Discussions.” The implication is that traversing threads may be a lot more “lossy” than one might think because the “tree” you can build following transmission of knowledge using quotes can differ (from the abstract):

We show how quotation practices can be used to locate design relevant data in discussion archives. OSS developers use quotation as a mechanism to maintain the discursive context. To retrace the thematic coherence in the online discussions of a major OSS project, Python, we follow how messages are linked through quotation practices. We compare our quotation-based analysis with a more conventional analysis: a thread-based of the reply-to links between messages. The advantages of a quotation-based analysis over a thread-based analysis are outlined.

All but a few open source projects do not receive investment from vendors and do not have material revenue streams—for these “community-driven” projects, face-to-face contact would obviously be prohibitively expensive. So in reliance on code and email to transmit design knowledge, they would seem to be dependent on a lossy medium (code-as-currency) and a lossy mechanism (mail threads).

Part 3: David Nichols and Michael Twidale (hi, Michael) have done research identifying usability & HCI challenges in open source development, thoughtfully articulating some of the issues and possible ways to evolve distributed development.

Part 4: I was left with the impression this is a scenario that is really not good for community-driven OSS—and, by implication, for any resource-constrained distributed development process (something applicable to end-user developers collaborating online, and perhaps small ISVs, communities large in both number and importance to Microsoft’s business).

After reaching this conclusion I contacted the Codeplex team (meet the team) to talk about Microsoft taking a role in developing new functionality that might help this scenario. But first we needed to establish a research program to figure out whether this was a good path to go down, and what to do. That led to contact with Jack Carroll, Paula Bach, and the current project.

The first public session we held on this was a special interest group (Usability and Free / Libre / Open Source Software) at the recent CHI 2007 conference. Jack, Paula, and I moderated. I’ll let notes mostly from Paula sum up one aspect of a great discussion that gave me ideas I’d never thought of before:

About 40 people (1/3 to 1/ 2 of whom were involved in open source projects as contributors or researchers) attended the CHI Special Interest Group (SIG) on Usability and Free/Libre/Open Source Software (FLOSS). The group raised many issues including the “code as currency” issue. In essence, if “code is the only currency’ can there be a “benevolent HCI dictator?” The currency problem arises when HCI people who don’t write code work on FLOSS projects, potentially preventing the common mechanism of the “benevolent dictator” who can arbitrate conflicts over coding from emerging in the design and HCI domain. An interesting benevolent HCI dictator experiment would be to have HCI people design and initiate an open source project (it could even be a rapid prototyping tool that could be used as currency between FLOSS HCI people and developers) and have developers work on the project with an HCI person as the leader. This would be interesting in terms of social dynamics and to see who prevails as the benevolent dictator: would the HCI person remain or would a developer move into the leadership position once code writing began?

This is what we do every day. I hope this provides a bit of a view over time into our daily work to be center of excellence for (1) understanding and (2) finding opportunity with open source: ways for Microsoft and open source to “grow together.”

(Speaking of growing together, if you were one of the companies invited to the Microsoft Open Source ISV Forum before OSBC, I hope to see you there.)

Barry Wellman, SD Clark Professor of Sociology, on Social Network Analysis and Community

Bryan Kirschner — Fri, 15 Dec 2006 19:46:00 GMT

Web 2.0. Enterprise 2.0. Open Source 2.0. All the latest expectations for major revs of a good chunk of the information technology world seem to be heavily based on excitement about the possibilities for new forms of social networking and collaboration.

Nobody has more to say about how this can be done right—or wrong—than Barry Wellman.

Dr. Barry Wellman is the S.D. Clark Professor of Sociology at the University of Toronto and is the director of Netlab, a scholarly network studying computer networks, communication networks, and social networks. To quote from an introduction to a tribute event, Barry “pioneered innovative approaches to three fields:

Social network/structural analysis,
Personal community and social support analysis, and
The study of cybersociety (which he calls "living networked in a wired world").”

He has authored 3 books and more than 200 journal articles. He is, to use images from social network analysis, perhaps the biggest “hub” of folks—students, former students, and industry and academic collaborators—who study online and offline communities (including open source communities) there is.

He’s also a really nice guy and was kind enough to take a few minutes to talk with us while he was at the University of Washington’s ISchool talking about “What is the Internet Doing to Community and Vice Versa?.”

Video: Barry Wellman on Social Network Analysis

It's Like This...Or Maybe Like That...(Part 1.2)

Bryan Kirschner — Tue, 12 Dec 2006 17:24:00 GMT

It’s been just over a month since I last blogged on the law-and-open-source –analogy, and, despite a cool, unrelated entry in the middle, I feel my blog karma is running dangerously low… But—proving either that life is a journey of continuous learning and joyful surprise, or, more simply, that good things come to schlubs who drag their feet—last week not only did NPR run a story on legal apprenticeship programs, I also heard a speaker who thoughtfully referenced Foucault in a talk on fostering innovation.

The first line of reasoning was about rather than being a case where the “source code” of the domain (law) was “closed,” law is a case where it is really, really “open:”

legal documents are almost universally public as well, so you can seek an example of someone else’s filing, brief etc from among literally millions of such documents—from the lowliest pleading to the most momentous Supreme Court argument. If a situation where the full text of millions of legal artifacts available freely (or for the price of distribution) aren’t like open source code…I’m not sure what is!

The implication, going way, way back to the first blog in this chain, is that it seems you can make a lot of knowledge (qua legal artifacts) “more like open” without new artifacts becoming “cheap” (lawyers still are, or at least feel, expensive).

There is a different angle, though: the second topic was about restricting access to the profession itself. Is law an example where the “openness” of knowledge is counterbalanced by extreme restrictions on being able to function as a lawyer—specifically multiple years of (often expensive) law school? In this blog entry I am going to take a stab at articulating three views on this: let’s call them “the bad scenario,” the “not bad but not great scenario,” and the “pretty cool scenario.”

(Since we’re comparing to open source development, I am going to make one simplifying assumption: having to pass a test—the bar exam—is entirely compatible with being “open.” I say this because I see it as analogous to many OSS communities: to become (say) a committer you basically show up and start doing some work to demonstrate your skills; the test is analogous. If the bar exam is broken somewhere in terms of content or form factor, I see that as a tweak as opposed to fundamental to the show-up-and-demonstrate-your-ability analogy.)

First “the bad scenario” (this is where Foucault comes in). To be somewhat painfully reductive, Foucault observed there is a relationship between the structures of power and (ostensibly objective) knowledge, and that a characteristic of the modern world was the application of “discipline.” Discipline, broadly speaking, is defining and conditioning a state of behavior that creates non-egalitarian power relationships such that it becomes “normal” and this dynamic becomes invisible within a formally egalitarian, “fair” and “open” environment. The real rubber-hits the road example here would be a cycle that works like this: based on a scientific and meritocratic rationale, three years of law school becomes a standard; as a consequence, most lawyers who pass through this system view it as both feasible(since they did it) and meritocratic (since they by definition did well, or at least well enough, in it). In tandem, it also means lawyers are relatively scarce and expensive and have similar billing rates and incomes, because they all ponied up fairly homogeneous investments. Thus, they have psychological and material incentive to perpetuate the system, because in rejecting it they would not only repudiate their own accomplishments, but possibly introduce competitors who could undercut them. In the end, the system justified its own perpetuation without controversy…because it seems to comport with the accepted paradigm of a modern profession.

On to the “not bad but not great” scenario. This is where Spence and Signalling Theory come in. The Wikipedia entries linker above are concise so I recommend reading them: the upshot is that for education to be used as a “signal” to help employers choose more valuable employees, it is not necessary for education to have any intrinsic value. The reason this is “kind of OK“—despite the fact that investing in education even if it doesn’t increase productivity seems intuitively perverse—is that at least solves a (communication) problem, and can be economically efficient. By contrast, the’ bad scenario” is quite likely to cause the behavior of the system will become very, very inefficient relative to a truly open consideration of all the options.)

Finally, the “pretty cool” scenario. Seven states enable people to become residents without attending law school through some type of apprenticeship program :

In Vermont, participants don't need a college degree, but they must have completed three-quarters of their undergraduate course work. Then they have to spend 25 hours a week for four years studying alongside a licensed attorney.

In Washington state, a “law clerk”

…shall study for 4 calendar years. Each calendar year shall consist of 12 months, with a minimum of 120 hours of study each month, including the time spent in performing the duties of a law clerk. The tutor shall give personal supervision to the law clerk averaging at least 3 hours each week. "Personal supervision" is defined as time actually spent with the law clerk for the exposition and discussion of the law, the recitation of cases, and the critical analysis of the law clerks written assignments.

(In both cases these positions can be paid jobs.) This suggests a radically different paradigm for entry into the profession (one might, and Foucault might agree, a “throwback” to a previous era), and a lovely mentoring dynamic. What’s particularly interesting is that women outnumber men in apprenticeship programs, and the typical age of the participants is older than law students. And while at least according to NPR the likelihood of programs like these increasing, among law schools there is an increasing incorporation of paid internships and flexible schedules.

I think parallels to each scenario can be made to the open source development domain, and thinking about the “balance” of scenarios in both the legal and software development domains will be a fascinating discussion. That blog will be along in less than 30 days, I promise….

There's a Vendor in my OSS

MichaelF — Mon, 23 Oct 2006 14:34:00 GMT

Taking a brief detour from the thread about OSS and its similarities (or not) to law to take note of a couple recent publications, both of which discuss the interaction between traditional IT vendors and OSS:

In MIS Quarterly (September) (link) Brian Fitzgerald (University of Limerick—one of the must-read researchers on OSS, IMO) provides a comprehensive survey of what he calls “The Transformation of Open Source” with expectations for “Open Source 2.0.” He expects IT vendors—including Microsoft—to play significant roles in “OSS 2.0.”

In Communications of the ACM (October) (link) Pamela Samuelson (UC Berkeley) discusses “IBM’s Pragmatic Embrace of Open Source.” (The title pretty much speaks for itself as a summary.)

I highlight these because they reflect what seems to me (qualitatively) to reflect a trend in the literature. We’ll work on getting a better sense of what the trend is and researchers’ perspectives on it to bring back to Port25…because (to bring things back to analogy and metaphor), they introduce the question: Is a vendor in my OSS more a fly in the ointment or chocolate in my peanut butter?

It's Like This...Or Maybe Like That... (Part 1.1)

MichaelF — Thu, 28 Sep 2006 20:31:00 GMT

In my last blog I started talking about the power of analogy and metaphor, and dove into a discussion of the first analogy of my collection, asking what if the practice of law, rather than being like a domain suffering the consequences of a “failure of openness,” was more like an example of a domain with a great deal of openness. I promised to offer some ideas for analogies that helped make sense of the situation in my next blog.

I was subsequently intrigued by where some of the comments to the post (unexpectedly) led me; thus, I postpone those (original) ideas in order to share some of where those comments led me on the path toward new ones. In this post, I will address the first of two substantive issues: the analogy made by CDarklock between the legal profession and open source software development (--in his view, to the disadvantage of OSSD).

Comparing “do-it-yourself lawyering” with “do-it-yourself usability,” he implies a lack of sensitivity among developers to the risks of the latter relative to the legal profession’s diligence with respect to the former.

Surely enough, when pointed to Groklaw by stats for all, front and center is a warning of the risks of the former:

IANAL. I am a journalist with a paralegal background, so if you have a legal problem and want advice, please hire an attorney.

This prompted me to think about some quick phenomenological test to determine if similar evidence for this type of distinction made by journalists commenting on law (Groklaw) might in fact be recognized with equal alacrity by open source developers commenting on usability: in fact, in defense of OSS developers, there are indeed cases of IANAUE (“I Am Not A Usability Engineer”)—although AFAIK it has not made Wikipedia yet, in contrast to variants of the apparently fecund IANAL which has spawned IANYL, TINLA, and IAAL. (WTF?)

At this point you’re probably wondering why I was so engaged on this topic—the first reason is that, IMHO, challenges with consistent and effective usability practices are endemic and impactful. (I will never forget my introduction to the usability disipline: I was helping usability engineers build a stochastic model on top of their user observational testing of a web experience which (thankfully) has since been improved. The model was pretty cool as a quantification of how much customer time and effort things like ambiguous terms and redundant links actually wasted—but nothing seared the importance of usability into my brain like watching a test subject (a middle-aged, tech-savvy woman with, as I recall, a PhD—kind of hard to blame the user) actually start to cry in frustration as she tried to complete a task.)

The second reason is that I had never considered the possibility of an analogy between “ legal self-representation” and “developer self-usability” as conceptually similar problems to be solved. This analogy offers a different (and interesting) way to think about why it occurs and what to do about it in OSSD, in contrast to traditional corporate development where the origins of usability challenges nor their resolution seem to me to be fairly straightforward: does a company (or development group) recognize the value of good practices, resource for it, make it a priority, test against user interaction metrics etc.

In fact that there is a paper (Nichols & Twidale, online at First Monday) which provides a comprehensive assessment of usability in OSSD, with suggestions for remediation that come awfully close to echoing ideas for what folks in the legal profession would call increasing access to justice (--like academic volunteerism and corporate involvement).

It’s an interesting line of thinking both because it is just in time for CSCW 2006 (Computer Supported Cooperative Work—anybody going?) and because folks in the OSS community (just like in the legal community) are doing some “out of the box thinking.” I’ve been trading mails with a PhD candidate at Penn State who has outlined a very thoughtful research agenda on OSS and Usability—I’m pleased to say we’ll be bringing her to Port25 for an interview near the end of October.

With that said, the next point release on our path to part 2 will address an issue raised by ssjdrn who surfaces (in my words) a tension between two principles I have always taken for granted: efficient signaling and disciplinary control through normalization that is keeping me up at night (literally—when Spence and Foucault, respectively, aren’t jibing for me, I can’t sleep. You can ask my wife.)

And yes, it all does come back to an even richer –than-anticipated analogy between law and open around making successful software.

It's Like This...Or Maybe Like That...

MichaelF — Fri, 08 Sep 2006 22:19:00 GMT

I am compiling a list and analysis of all the analogies and metaphors that have been used to characterize open source software development and its social, technical, and business implications. I think it is unlikely this will be the next DaVinci Code-style best seller, so I don’t expect to give up my day job, but my interview of Professor Alfonso Fuggetta where he talked about the distinction between specification and implementation reminded me of why I started this project.

Analogy and metaphor are fundamental to not only how we understand something, but also to how we form our opinions about it; in an influential paper on artificial intelligence and cognition, the authors emphasize the centrality of analogy to not only how we make sense of the world but its powerful and often unrecognized implications:

When people make analogies, they are perceiving some aspects of the structures of two situations—the essences of those situations, in some sense—as identical…Furthermore, not only is analogy-making dependent on high-level perception, but the reverse holds true as well: perception is often dependent on analogy-making itself. The high-level perception of one situation in terms of another is ubiquitous in human thought. In the large or the small, such analogical perception—the grasping of one situation in terms of another—is so common that we tend to forget that what is going on is, in fact, analogy.*

The authors use a few contemporaneous controversial situations from US politics in the late ‘80s – early ‘90s to illustrate this point—what struck me about their emphasis on the power of analogical reasoning to shape opinion was the fact that analogies to “Vietnam” and “Hitler” were used in those examples in their paper and, 20 years plus later, those same historical events are in the news as analogies today. The two examples highlight how acceptance of the analogy carries with it a strong bias to shape your attitudes and your behavior (even manipulate your emotions): analogies don’t necessarily just frame the argument—sometimes they are the argument.

The analogy that’s top of mind for me today is from Open Sources 2.0 (Sam blogged about it here), and was laid out by Matt Asay (who frequently has interesting things to say on his blog or, for that matter, here on Port25). It’s one of those that I think warrants some discussion because, as it stands, I think it actually raises much more interesting questions than what Matt initially describes.

In his chapter “Open Source and the Commodity Urge: Disruptive Models for a Disruptive Development Process,” he is arguing open source reduces prices by facilitating the option to “do it yourself.” After hiring out their landscaping, including cement work, he goes on to relate the unfortunate story of having to pay (high) legal fees to resolve a dispute with his (bad) cement contractor. The price of legal assistance was high because of the “skill set involved and the artificial licenses set up by the legal profession to keep would-be attorneys in would-be land.” And necessary because (emphasis added) ”I am effectively barred from accessing the “source code” of the legal…profession, which drives up the price I must pay.”

Here’s the interesting thing: as much or perhaps more than any other domain, we are not barred from accessing the “source code” of the legal profession.

True, there are hurdles you have to go over to practice as an attorney and represent others—but that is not relevant to the “do it yourself option.” In fact, you are entirely able to represent yourself if you want to, and some venues (small claims court and many administrative processes) are designed to accommodate self-representation. So, first—relatively speaking—law is an areas where you have a lot of latitude to “do it yourself” (by comparison, “do it yourself law enforcement” or “do it yourself public highway repair” by contrast, is actively discouraged.)

Second, the “how-to” of underlying knowledge is perhaps more widely available in the legal domain than in any other: there are many public law libraries and a large (although its practitioners would say not big enough) network of volunteer assistance organizations.

Finally, legal documents are almost universally public as well, so you can seek an example of someone else’s filing, brief etc from among literally millions of such documents—from the lowliest pleading to the most momentous Supreme Court argument. If a situation where the full text of millions of legal artifacts available freely (or for the price of distribution) aren’t like open source code…I’m not sure what is!

This analogy was powerful enough to lead me to think a long time about the contra case: not is lack of openness inflating prices, but what if a high degree of openness does not reduce prices (--or at least “reduce enough” relative to expectations). Thus the question: are there analogies that help make sense of this possibility? I have some ideas that I’ll offer in my next blog.

*High-Level Perception, Representation, and Analogy: A Critique of Artificial Intelligence Methodology; Chalmers, French, Hofstadter 2001

Open Standards, Open Formats, and Open Source: Bryan Kirschner Interviews Alfonso Fuggetta

MichaelF — Thu, 24 Aug 2006 22:47:00 GMT

Alfonso Fuggetta is a Professor of Software Engineering at Politecnico di Milano in Italy, CEO of CEFRIEL and Faculty Associate for the University of California, Irvine's Institute of Software Research. Among his many activities, Alfonso advises European Policy Makers on Information Technology Issues.

A paper, written by Professor Fuggetta: Open Standards, Open Formats, and Open Source is widely circulated at Microsoft as a helpful guide in thinking about Open Source, Standards and Formats.

In this interview Bryan and Professor Fuggetta discuss his views on Open Source, Standards and Formats as well as why he chose to write this valuable paper.

Video: Bryan Kirschner Interviews Alfonso Fuggetta“”<

Ants and Software Development: Thoughts from the ASA Conference

MichaelF — Wed, 16 Aug 2006 20:12:00 GMT

I’ve been surrounded by people who want to study us like bugs—and they intend that as a compliment.

I just got back from the attending sessions of Communication and Information Technology Section of the American Sociological Association (CITASA) at the ASA conference in Montreal. This group of researchers studies:

The social aspects of computing, the Internet, new media, computer networks, and other communication and information technologies. This includes online communities, knowledge management, the digital divide, labor markets, workplaces, and how the Internet fits into everyday life [and] the design and use of technology [including] developing and analyzing new kinds of software, and thinking about the implementation of technologies for teaching, research, and the real world.

Myself and Prof. James Witte (Clemson), Chair of CITASA

The thought of bugs struck me when I sat down to recap my experience because, I thought—as a systems-minded person myself—an individual bug is not that interesting. But put a lot of bugs together, and the humble ant, bee, or termite, can, by acting in concert, create spectacular feats of engineering. Sound like building software? –That’s interesting.

What’s particularly important is that these practitioners focus on the criticality of social dynamics to any endeavor—the relevance to distributed, voluntary open source development is obvious, but these dynamics are important to closed-source development, diffusion and use of technology, information dissemination, bridging the “digital divide”…

I know I won’t do justice to everything here and now: suffice to say there are folks working on things like how building your reputation on Slashdot works (got your attention?); how to predict the level of documentation that will be produced by developers given different levels of social reinforcement for contributing code versus good documentation (…a challenge as applicable to proprietary as open source development, in my experience); and how different newsgroup communities use data to measure and control their “community health” (one group dedicated to quilting (yes, like with needles and fabric) seem to be particularly aggressive about this, going to show that traditional geek stereotypes may be becoming victims of the ubiquity of the Internet.) I’ll just promise that we will work hard on bringing some of the most interesting and relevant information, and interviewees, to Port25 over the next year.

Before I go nurse my jet lag, two quick notes:

First, since it is always nice to agree with the Boss (--which holds true for both Bill Hilf and Bruce Springsteen, IMHO), I will pile on Bill’s blog post about closed (or open) mindedness. If you happened to click through to CITASA’s website, you might have noticed something: our lab is a sponsor, Microsoft Research (MSR) is a participant (and a long-time participant and creator of some of the leading software for conducting social network analysis, I might add)—and the site is hosted by the Clemson Linux User Group. Bernie Hogan, one of the sociologists at the conference, said it best when the group turned to discussing “moral panic” (their words) or what one might also call hysteria (my word) about MySpace and “Internet Child Predators” right now: researchers have a critical role to play in “explaining what’s really happening.” I take as a point of pride that we share across our Port25 team, CITASA, MSR, and (I hope) you the reader a commitment to understand what’s really true—empirically measured, tested, challenged, tire-kicked…if everyone was focused on finding the demonstrably best solution to well-articulated scenarios, the world would be, IMHO, a better place…and the perceptions Bill referred to would take care of themselves expeditiously.

The second point is that my bug metaphor, above, has a basis in published research: Valverde, et al, from the University Pompeu Fabra, Barcelona, Spain, published “Self-organization patterns in wasp and open source communities” in IEEE Intelligent Systems, March-April 2006 (Volume 21, Issue 2). In this “comparative study of how social organization takes place in a wasp colony and OSS developer communities” they found “both systems also define interacting agent networks with similar common features that reflect limited information sharing among agents.”

And if you didn’t think that I really do care about tracking down every single bit of knowledge available to understand “what’s really going on”….I read the whole thing.

Hank Just Blogged About Critical Thinking

admin — Mon, 17 Jul 2006 20:00:00 GMT

Hank just blogged about critical thinking. If I had to state my own concise definition of what lies at the heart of critical thinking, it would be a personal commitment to finding the right solution to any problem, regardless of whether or not figuring it out and the subsequent implications are easy or comfortable (in practice, this usually means being the resident skeptic right at the point everyone else is getting excited.) This is not necessarily a comfortable thing to do: we tend to have a psychological affinity for propositions that confirm or reinforce, rather than challenge, our existing beliefs. And there are many sources of social and institutional pressure that militate against “naysayers” and “people who make things more complicated than they need to be.” Moreover, no matter how many tools the hardcore skeptic has in his or her toolbox—years of accumulated experience and wisdom, technical savvy, statistics and operations research skills, or sharp psychological intuition—the context for bringing those tools to bear is seldom ideal.

As an example, consider a team trying to lock on a project plan under some challenging time constraints: success means leaving the room with (1) agreement on a plan; (2) shared conviction that the plan can achieve the team’s goals, if every contributor executes against their commitments; and (3) an accurate understanding of the probability of achieving the goals. From experience, you probably know (1) and (2) are difficult enough—now think about being the one person in the room who says “you know, if you bracket our point estimates for milestones with sensitivity limits and run a Monte Carlo, I think our point-based-looks-like-99%-chance-of-success looks a lot more like 20% chance of success…”. (I use this example deliberately, thinking of the Carnegie-Mellon Software Engineering Institute (SEI)’s Capability Maturity Model Index (CMMI)—“level 2” project management entails items (1) and (2) from my example—the kind of quantitative management becomes institutionalized at “level 4.” Whether or not you are a CMMI fan, it does provide a benchmark indicating how few organizations would find our data-driven critical thinker’s suggestion easy, comfortable, or routine.)

On Port 25 we’ve had discussion of some hard problems: about the tough choice between MS building product capabilities versus partners and ISVs and about licensing and shared source project requirements. A big reason Port25 exists is because our point-of-view in the lab is that the best possible answers to these and other questions are unlikely to be easy or comfortable . (To use some extreme touchpoints, in my view the position that MS could answer these questions optimally by discounting the phenomenon of open source development and its history is just as almost certainly incorrect as the position that MS should answer these questions optimally by discounting the phenomenon of commercialized software development and its history in favor of “opening everything.”) If this is true, a dialogue among diverse perspectives is essential to continuously push the thinking of everyone involved away from the personally easy or comfortable—hence, the Port25 dialogue. There’s a new empirical study that I hope drives this point home and offers you the same motivation to continue to read and post to Port25 that it offers me.

Kevin Boudreau at MIT’s Sloan School of Management took an empirical approach to the question “Does Opening a Platform Generate More Innovation?” Cleverly, he looks at handheld computers (PDAs)—an area with multiple software and hardware players—and does a deep analysis of innovation measures in relation to “openness” measures. (What’s also clever, IMO, is that he includes different types of openness (ranging from licensing to SDK’s and documentation)—and innovation (differentiating between lots of incremental deltas and big breakthroughs)). What he finds overall is:

… setting an optimal open strategy may be a far more nuanced problem when trying to promote innovation, in contrast to managing the traditional trade-off between promoting adoption (by opening) versus retaining appropriability (by closing). In this traditional perspective, intermediate levels of openness might have been understood as a means of achieving some middle ground between two relatively simply opposing interests. In the case of opening to promote innovation, an intermediate level of openness might in fact be in the best interest of promoting innovation; there may not be so severe a trade-off with maintaining appropriability. However, the decision to open a little or a lot, and precisely how, will also likely involve trade-offs across multiple dimensions of innovation. (p. 25)

“Openness” was non-monotonic in relation to innovation, meaning (depending on the particular constructs used in different analyses) the curve peaked and then declined at some point. And the type of openness appeared to promote different types of innovation—lots of incremental or imitative innovations as opposed to a few breakthroughs:

Openness therefore should not only affect the rate of technical change, but also its direction. Therefore, these findings offer another explanation of why we observe so few “perfectly” open strategies in practice and why there might plausibly be place for heterogeneity of open strategies, insofar as there is space for heterogenous innovations and differentiation in a product market.” (p. 26)

Why is this exciting? Because it provides compelling empirical data from one technology domain that the question of optimal “openness” is an empirical question, not an ideological one. And that “openness” in relation to a traditional software business model is not a zero-sum, oppositional game. That means not only empirical research but also the Port25 dialogue are essential—even determinative. The bottom line: what we discuss here matters. If you download the paper, let me know your reaction. - Bryan

Kudos to Open Source Developers

jcannon — Mon, 03 Jul 2006 16:30:00 GMT

I see my last couple posts were about ambiguity, so I thought today I’d blog about something, IMO, that is not ambiguous at all—and the topic would be a fitting hat tip to Sara and Korby and all the folks involved with CodePlex.

Brief background: We had to buy our own combination padlocks on our lockers in my high school. I used to forget the combination all the time (—I still have nightmares about that). I finally solved this by writing my combination in hex on the back of the lock. (I figured there was only one other kid in my class who would know what 0F was in base-10, so if anything was ever missing, I’d know where to look. )

I tell this little anecdote because it made me think about the lack of a community of folks with similar interests in my little world back then. The only reason I knew hex* went back years earlier to a similar lack of community: I couldn’t get a game I was writing on my Commodore 64 to do some things fast enough in BASIC, so I asked my Dad what else I could do and he explained what Assembly language was, and from then on there were lots of nights when I was supposed to be asleep, sitting there in my pajamas, banging away in 6502 Assembly land—by myself.

This was long before the concept of a home modem would have ever occurred to us, never mind the modern Internet’s enablement of community and collaborative development--but I can’t help but wonder what a difference it might have made to me (never mind the quality of that game!) if there had been a more readily accessible community of folks interesting in collaborating and mentoring at that time.

What does this have to do with praising open source developers? This week, inspired by CodePlex, I was looking back at two of the most important studies of the motivations of open source developers. In the two studies (Ghosh in 2002 and Lakhani (PDF) in 2004—both are available online), although slightly different sets of questions were asked, by a notable margin the leading responses were “Learn and develop new skills” and “Share knowledge and skills” (Ghosh) and “Code for project is intellectually stimulating to write” and ‘Improve programming skills” (Lakhani). What’s even more striking about this is comparing these types of motivations—about learning and sharing—with more “confrontational” motivations. Developers could choose multiple answers in both studies, and, for example, in the Lakhani study four times as many volunteer developers chose “Improve programming skills” as a reason for joining an open source community than “Dislike proprietary software and want to defeat them.”

To be clear, anybody’s reason is valid to them--but I am a person who would rather learn than win. That’s true when I write code, it’s true when I play soccer; I think that is a good way to view the world—and from all the research I’ve seen, the evidence is compelling that folks who voluntarily participate in open source development communities place very high value on learning and sharing their knowledge with others. I don’t have comparable data at hand, but I’m willing to believe it is well higher than the average person in the population at large. And for that—kudos. I think that means there are far more opportunities for kids like I once was not just because of technological advances, but because of people—maybe people like you reading this post.

*I actually can’t remember if I stumbled across hex first in Traveller, where, as I recall the descriptive strings for character attributes and planets where in hex—come on, don’t snicker, you know you played it too…

How Open? Open How? It Depends…

admin — Tue, 20 Jun 2006 19:51:00 GMT

Not long after I blogged about “disambiguating open” as a research issue, a debate erupted on Slashdot about “How Open Does Open Source Need to Be?” Three different criteria for deciding whether something could legitimately call itself “open source” seemed to me to dominate the discussion: the level and terms of source code availability; whether or not there was a community organized around the code; and compliance (or not) with the Open Source Initiative (OSI) definition.

What struck me about this debate was, first, the fact that it was happening because “open source,” in this case, was regarded as a favorable characterization (and thus in the interest of a software producer to apply to their bits) and that favorability was rooted outside of any producer’s application of the term (in contrast, it’s unlikely such a debate would occur if one of the parties had chosen to label their product “cromulent ,” for example.) Put another way, the operative assumption is that calling your product “open source” piggy backs on a store of perceived value among potential customers built up by others, which may be in a technical or moral sense be “undeserved” and (at worst) might actually result in dilution of that store of value.

What I thought was missing, however, was some empirical perspective on the ways in which states of the three criteria above might in fact add to or erode that perceived store of value when the so-called (or not-so-called) “open source” product was actually consumed by customers. This reminded me of Perspectives on Open Source Software (2001) from Carnegie-Mellon’s Software Engineering Institute, where the authors do a great job of describing why the answer to “how open” software needs to be will almost always come down to “it depends...”—if we’re thinking of “need” relative to optimizing for the resulting customer experience (as opposed to the relative ease of calling something “open” for marketing purposes or conforming to a check-the-box definition.)

The report describes “open source software” as

“at the most basic level simply[meaning] software for which the source code is open and available.”

Which they define as:

“open--The source code for the software can be read (seen) and written (modified). Further, this term is meant to promote the creation and distribution of derivative works of the software…available--the source code can be acquired either free of charge or for a nominal fee”

And they contrast this with the Open Source Definition (OSD) from the Open Source Initiative (OSI) noting:

“Under OSI (strictly speaking) a software product is in fact open source if and only if it conforms to the OSD…Upon reviewing the complete text of the OSD, it is interesting to point out that the definition does not pertain specifically to the source code itself, but rather to the license under which the source code is distributed. Therefore, in strict conformance to the OSD written by the OSI, a software product that conforms to only eight of the nine criteria is not OSS.”

They raise a few points relevant to distinguishing meaningful customer requirements for the software :

An example of a license (Sun’s JDK 1.3) that violates OSD criteria #6 (“No discrimination against fields of endeavor”) by stating the software “… is not designed, licensed or intended for use in the design, construction, operation or maintenance of any nuclear facility." (For most folks for whom an OSI cert might be a desirable characteristic, IMO this violation is probably likely to be something of a trivial one in terms of relevance to actual usage scenarios…)
The importance for customers who intend to treat OSS as a “black box”—meaning not themselves making any modification or even reviewing code—of the OSS product’s community, since the customer will in this case plans to be dependent on the community: “If the community is small and stagnant, it is less likely that the software will evolve, that it will be well tested, or that there will be available support."
Conversely, they note the importance for customers who intend to treat OSS as a “white box”—meaning changing code and managing the derivative work themselves—of characteristics of the code itself rather than the community: “sometimes the source is the only documentation that is provided. Some consider this to be enough. Linus Torvalds, the creator of Linux, has said, "Show me the source." Yet if this were the case, there would be no need for Unified Modeling Language (UML), use cases, sequence diagrams, and other sorts of design documentation. Gaining competency in the OSS component without these additional aids can be difficult.”

The authors of the report conclude that:

OSS [is] a viable source of components from which to build systems. However, we are not saying that OSS should be chosen over other systems simply because the software is open source. Rather, like COTS [commercial off-the shelf software] and CSS [closed-source software], OSS should be selected and evaluated on its merits…Caveat emptor (let the buyer beware): the [software] product should be chosen based on the mission needs of the system and the needs of the users who will be the ultimate recipients.

To me, reading the discussion thread and then mulling over the SEI’s examples bring me right back to what gets me out of bed every (work) day: there is rich field of inquiry about what makes software successful for its developers and users. Neither any one commercial vendor nor organization has it all figured out and trademarked yet—if caveat emptor is good advice to keep in mind, salus populi suprema lex esto (“the welfare of the people shall be the supreme law “) is a good motto to live by.

(Incidentally, SEI authors Chuck Weinstock and Scott Hissam revisit some of the findings and analysis in the SEI report in a chapter in a great new book Perspectives on Free and Open Source Software (MIT Press) –I’ll blog more about the book soon.)

Research Strategy Corner: Disambiguating “Open”

admin — Tue, 06 Jun 2006 17:30:00 GMT

Research Strategy Corner: Disambiguating “Open”

Disambiguate (transitive verb): to establish the true meaning of an expression, regulation, or ruling that is confusing or that could be interpreted in more than one way

I’m a Research Strategist with the Open Source Lab here at Microsoft. When folks ask what that means , I usually tell them the second-best definition of “strategist” I’ve ever head is “a researcher who gets to make stuff up”—but the first-best is “someone who establishes a series of steps to achieve a goal.” The latter is what my job is all about—in the process, because it involves synthesizing both technical information and insights from computer science, organizational science, and sociology research, I sorta get to make stuff up—as long as the math works (which is probably a little bit different from what the average civilian bystander thinks of as “making stuff up.”)

What’s the goal? The title of this post summarizes it concisely: disambiguating “open.” When the phrase “open source” is used for example, it could represent a simple descriptive statement of fact about code visibility (read any good mash-ups lately?); it could also be referring to software artifacts available under a fairly wide range of license types…or it could be intended to refer to something compliant with a very specific set of criteria like the Open Source Initiative’s ten-point definition (http://opensource.org/docs/definition.php). It could be referring to one of tens of thousands of single-developer projects on SourceForge—or to highly coordinated efforts like FreeBSD; and on the other hand altogether, it could be in marketing materials from big corporations like IBM or Novell (if this animation is still up on Novell’s site you can experience a dizzying array of suggestions for what “open means to your enterprise:” http://www.novell.com/solutions/?sourceidint=hdr_productsandsolutions ).

“Open” is one of those words that today in the software domain is increasingly becoming probabilistically uninformative…the word as applied to an endeavor (like a software development project) or an artifact (like a piece of software) less and less enables you to more accurately predict attributes of the endeavor or artifact—because attributes of the endeavor (like who built it and how) and the artifact (like architecture, coupling, interaction paradigm) may actually help you better predict what will happen now and further on down the road if you choose to take a dependency on something.

I don’t care how the characterization “open” makes you feel, whether it is nervous or giddy with excitement: my objective function is the ability to use one bit of information to reliably predict other bits of information. In this space I’ll share our efforts to go about doing that with regard to “open” and what we find in the lab and in the world of academic research—but first I’ll give you some visibility into how I start out structuring lines of inquiry.

There are a few different approaches you can take to things: for our purposes here, an analytic approach is like an argument from first principles—the position of Free/Libre software advocates is essentially analytic, as their argument is software should be a certain way given the set of principles they start from, and there really isn’t any evidence you could collectout in the world that would change their minds. This discussion isn’t particularly relevant to my day job. An empirical approach is all about data and probability: if you know foo you predict bar better. This is entirely relevant but is exactly what we don’t know enough about yet. A phenomenological approach (http://plato.stanford.edu/entries/phenomenology/ if you really want to know) starts with experience as it is experienced--and this is useful for starting to disambiguate open source. Here’s why: I don’t want to argue about what’s open and what’s not or about whether things should be a certain way—I want to build an informative base of data that lets me characterize and analyze endeavors and artifacts underneath this fuzzy term “open”. So we can start by asking: what would I—and, if there’s enough established shared meaning, other people—experience as a phenomenon that is certainly “like open source” versus certainly “not like open source.” I suggest two sets of statements along an axis shown in figure 1, below. Once we have this down, we have a starting for the collection of empirical data that ( if our starting point is right) will position endeavors and artifacts somewhere along a continuum between the two extremes.

Figure 1: Phenomenological approach to characterizing endeavors

I won’t start into operationalized definitions here, because to some extent that would defeat the purpose of capturing a top of mind response to the statements themselves. So what do you think: top of mind, as you experience these collections of statements, is the essence of like and not like open source represented? Do they raise questions? Controversy? Let me know and we can dive in to where some of these come from (yes, I said my experience-as-experienced, but remember, when I make stuff up the math has to work…there’s lot of great research out there that can help tune these characterizations).

Disambiguation: Because what you don’t know you don’t know probably will hurt you