Port 25: The Open Source Community at Microsoft : Community, OSS Research

More Open Source Goodness

Peter Galli — Wed, 20 May 2009 20:05:00 GMT

Tony Hey, the corporate vice president of Microsoft Research's External Research group, used the Open Repositories Conference to announce today the public availability of Zentity and the second version of the Article Authoring Add-in for Word 2007, both of which will be released as open source.

Zentity, previously called Research-Output Repository Platform and code-named Famulus, is a platform that allows institutions to store all of their digital scholarship: papers, lecture, presentations, videos-anything that might be collected by the university as part of the digital output of their researchers and scholars.

Over the past nine months two betas for this have been released, which refreshed the user interfaces and added new controls, and complement the services provided in the package.

The second version of the Article Authoring Add-in for Word 2007 includes new functionality, including the ability to upload directly into a repository - Microsoft's or those of others - via the SWORD [Simple Web Operation for Repository Deposit] protocol.

Support for authoring Object Reuse and Exchange resource maps within the Word environment has also been added, as well as the ability to perform literature searches and to import the bibliographic information in Word with one click, which makes it very simple to quickly add citations into a paper.

A key element of the Microsoft External Research vision is to support the scholarly communications lifecycle with software and services so that data and information flow in a coordinated and seamless fashion.

With regard to the plan to open source these tools, Lee Dirks, the director of the Education and Scholarly Communication team, said that "first and foremost, we're releasing the binaries, but soon thereafter, we'll release both of these as open source. Once they are available, our big push over the next 12 to 18 months will be to build a worldwide community around these assets."

You can find a lot more information on these announcements here.

These moves also follow the March release by Microsoft and the Creative Commons of an add-in for Microsoft Word 2007 that enables authors to easily insert scientific hyperlinks or ontologies as semantic annotations to their documents and research papers.

Microsoft is also making the source code available for the Creative Commons Add-in for Word 2007 free of charge to open source communities on CodePlex through the OSI-approved Microsoft Public License, which lets developers tailor it for specific industries using domain-specific language.

Bridging Chasms

Paula Bach — Mon, 18 Aug 2008 19:57:00 GMT

I have blogged previously about interdisciplinary and multidisciplinary research. Now I want to turn to disciplinary chasms in software development. Social aspects such as how people communicate, collaborate, and coordinate interest me because as Andy Begel in the HIP group in MSR found, software development teams when coordinating and communicating with other teams, are like dysfunctional families. I won’t go into details, but software development is socially complex. Software development teams generally consist of software engineers, to state the obvious. But in the last ten years or so, many software development businesses began to implement user-centered design because they realized that software could be frustrating for users. A new discipline arose where people were either trained in Human Computer Interaction, or learned on the job. Professional organizations like UPA and SIGCHI are thriving. But software engineers and usability experts are from different communities of practice. Although they both work on creating software products, their goals, values, approaches, culture, and well, their work practices differ tremendously.

A theoretical approach called communities of practice helps to describe and explain the social nature of learning and interacting in communities. But the approach does not yet incorporate multidisciplinary communities or how two communities of practice come together to accomplish their goals, like designing and building software.

In my experience, through being a designer practicing user-centered design and a researcher studying user-centered design in software development practices, about half of any design practice is communicating your ideas across disciplines or communities of practice and actively listening and working to understand different perspectives. Other practitioners echo this observation as well: Gitta Salomon of swimstudio.com, an interaction design firm, states that “One of the biggest challenges is remembering that half of what we do is the design work and the other half is the communication of that design work.” (Quoted from the book Interaction Design, by Preece, Rogers, and Sharp. This book is an excellent guide, both theoretically based and practical, most people trained in HCI will know about 80% of what is in this book and apply aspects everyday.)

Bridging software development and user-centered design could be investigated by looking at communities of practice and paying attention to communication could help bridge the chasm between the two disciplines.

CodePlex project developers wanted

Paula Bach — Mon, 30 Jun 2008 14:14:00 GMT

I would like to invite CodePlex developers to participate in research. If you have been reading my blog posts here on Port25, then you will know that I have been investigating how to integrate usability into open source software development. Now I am seeking research volunteers.

I am looking for developers who are working on projects hosted on CodePlex. The projects could be in the planning stage, alpha, beta, or stable. As a volunteer, you would be asked questions about your activities performed on CodePlex. The interview should not take longer than 60-90 minutes.

Data from the research will be used to help design support for usability on CodePlex and analyzed as part of my dissertation at the College of Information Sciences and Technology at Penn State University. I work with Dr. John M. Carroll at Penn State and Rob DeLine at Microsoft Research. One part of the study data collected from CodePlex users will inform the design of new usability support features. The other part of the study is to understand how developers work on open source projects. This research has been approved by The Pennsylvania State University Institutional Research Board, IRB #27804. As such, data will be used for the above research purposes only.

If you would like to participate, (participation is voluntary) or have any questions, please email codeplexresearch@live.com.

Go Hybrid

Paula Bach — Mon, 09 Jun 2008 19:29:00 GMT

I am back in Redmond. For those of you who don’t know, I spent last summer here in the open source software lab conducting research on integrating usability into open source. My last blog talked about how I have made some changes to my research program. One change is to situate the research argument within the broader scope of how open source has been changing. My favorite paper discussing this is titled “The transformation of Open Source Software” by Brian Fitzgerald and is only available if you have a subscription to the MISQ journal, but you can download an audio mp3 version read by Fitzgerald himself here. He coins the term ‘open source 2.0’ and characterizes software development in open source and compares it to proprietary, and shows that open source software development has elements of proprietary software development and proprietary has elements of open source. Any of the big open source projects with paid developers, and my favorite example, Mozilla, with paid UX professionals, is an example of the former, while Microsoft is an example of the latter. When I first met Bryan Kirschner (now director of open source strategy at Microsoft) he introduced me to the idea that API developer communities are a lot like open source communities.

The most interesting part of my research is that it is situated right in the middle of open source hybridization. A hybrid open source software development model combines a business model, either open source or proprietary, and open, two-way community input. The basis of my argument for the research is as follows: open source software development has been so successful that proprietary companies have been paying attention to incorporating open source strategies into their business model and very successful open source projects have had business models created around them. Both of these phenomena share some characteristics of software development, but taking a well-developed model of usability and transplanting it into a hybrid software development environment will be challenging because the hybridization landscape is still being cultivated. Because Microsoft has been successful with integrating usability activities into its production of software, it makes an interesting case to investigate how one of their hybridization strategies, CodePlex, integrates usability.

I am working with the CodePlex team to develop usability support for CodePlex. This means that the CodePlex community will have a say in how we design the support. Traditionally, open source projects are challenged for usability resources so the support has to range from being able to support code-centered and usability-interested developers to the possibility of usability professionals. The project addresses three main challenges for usability in open source: merit and trust, chasm between work activities, and incommensurable tools and methods. If you have a project on CodePlex and are interested in participating in this research, then please contact me: codeplexresearch at live dot com.

Passing Without Talking

Paula Bach — Wed, 06 Feb 2008 00:29:00 GMT

My last blog was about me traveling to Limerick and Toronto. I have now defended my dissertation proposal and passed. (Yay!) Here is a funny story. A week before the proposal defense I created my presentation and rehearsed it every day until the third day before when I began to get a sore throat. I don’t think it was from rehearsing the presentation or nerves or anything like that. Instead it was just a bug that was going around. Lots of students are sick at the end of fall semester. Anyway, two days before my defense I was getting a froggy voice, so I did not talk all day long. The day before the defense my voice was really raspy. The night before I worked on saving my larynx by gargling with salt water and any remedy I could find online. I woke up at 4AM the day of the defense and tried to speak a word. Nothing but a squawk came out. I had lost my voice. Because it is really difficult to get committee members together, the show had to go on. So at 9AM I stood up in front of my committee and a few fellow graduate students and began to squawk my way through the well-rehearsed presentation. It was not fun to look at the audience trying not to look disturbed at the sound of my voice.

Finally, after about the fifth slide, one committee member stopped me and asked the rest if they could just go into the discussion and skip the presentation. Everyone agreed and I listened to 5 professors, all of whom I respect a great deal discuss the merits and faults of my research. It was really an enlightening experience because I cannot think of another time when I will get five really smart people in one room discussing my research to make it better. In the end I came out with some ideas to rework my plan. The committee agreed that I was trying to do too much and advised that I choose one of the two parts. The first part, understanding FLOSS usability in general through the survey, observations, and interviews is almost done, and I have learned a lot, but the second part, designing a tool for CodePlex to support usability activities is not only more interesting, but also part of the agreement between IST and Microsoft. I came up with a new direction based on more literature I have gathered. The first exciting addition is the use of a theory to guide the design and research. I will use activity theory because it can handle people, both from an individual and social level, and artifacts. It also considers context and the dynamics of activities. Other HCI theories, for example, distributed cognition, handle people at the individual and social levels, and artifacts, but does not specifically take into account context and dynamics of activities. I am also using a methodological approach called action research. Action research is a practical approach to research where solving problems leading to intervention is a collaborative act between researcher and practitioner. I am a practical kind of researcher so this approach suited the project and me best.

I will be working with Microsoft UX people and the CodePlex team to integrate usability support for the CodePlex community site. I will also be working with a few projects hosted on CodePlex to help with the design.

[PostIcon:4037]

A Socio-Cultural Exploration: Geekus Unixus Microsoftus

jcannon — Fri, 28 Sep 2007 17:09:00 GMT

Frequent visitors to Port 25 may be familiar with Paula Bach’s HCI and ICT work from her blogging over the summer. What may not be familiar to many is the collaborative work we’ve done with Tracy Kennedy at the University of Toronto around the intersection of technology and communities. Open source is clearly a manifestation of this intersection.

Tracy’s work is impressive, and while her doctoral thesis examines the integration of the Internet into Canadian households, and how pervasive household internet use has led to its domestication – she does find time for lighter fare. One such example – the ‘Geekus Unixus Microsoftus’ (pdf) was published by Tracy in March of 2007. Through humor, Tracy uses primary research and empirical data obtained through interviews with employees to examine the culture of a UNIX expert working at Microsoft - and how they relate to other technology 'clans'. An excerpt follows:

“In uninhabited areas of web a new clan of hybrid technologists have been spotted: the Geekus Unixus Microsoftus (GUM). As the prevalence of interoperability between platforms and between commercial and open source software continues to grow, this report provides a socio-cultural overview of the GUM clan – a hybrid group of UNIX individuals working at Microsoft. An investigation of cultural habits, social customs, and personal experiences in a previously uncharted terrain is documented. It is hoped that technologists (from whatever platform) may better understand this new clan and endeavor to co-exist peacefully with them so that we can all benefit from their initiatives.”

We're excited to post it for Port 25 readers to enjoy. We welcome people's thoughts on the paper & we'll certainly invite Tracy to join the discusson.
- Jamie.

More About Tracy Kennedy:
Tracy Kennedy is a PhD Candidate in Sociology at the University of Toronto. Her doctoral thesis examines the integration of the internet into Canadian households, and how pervasive household internet use has led to its domestication. Tracy is also a research consultant in virtual and physical worlds. She has organized several virtual world events such as the 2007 Second Life Conference for the Communication & Information Technology section of the American Sociology Association, and a blended reality event at Vancouver’s Centre for Digital Media in British Columbia that featured an Open House for the new campus in both worlds. Tracy recently returned from an internship with Microsoft in Redmond, Washington where she worked closely with the Community Technologies Group and Games User Research Group to examine gaming networks, women’s online gaming experiences on Xbox Live, and the issues the industry faces in attracting non-traditional gamers. Tracy is also a lecturer at Brock University and the University of Toronto on the subjects of media, culture, ICTs, gaming & virtual environments, higher education and gender. Check out Tracy’s blog: http://netwomen.ca/Blog/

Pondering the preliminary results of OSS usability research - Part 2

Paula Bach — Mon, 10 Sep 2007 18:59:00 GMT

Developing software has been an engineering discipline with formal methods. The evolution of software methods has ranged from the now outdated waterfall method to formal specification languages with precise semantics. Despite having methodologies, software engineering continues to be difficult. Yet despite having what seems a lack of software engineering methodology, open source software development can produce stable, useful software. In 2002, Fred Brooks gave a talk at CMU discussing the design of software. You may remember Fred Brooks from such publications as The Mythical Man Month and No Silver Bullet. In the talk at CMU, Brooks focuses on the social issues surrounding software engineering.

Likewise, Microsoft Research (MSR) recently initiated a new group called Human Interactions in Programming (HIP). This group studies the social aspects of software engineering. Their joke is that they “ build tools as if software were made by people … working together. This quote, taken from the technical report, outlines some social issues software developers experience in their day-to-day work.

Another way to look at my dissertation project is as an extension to looking at social issues in software development. While the HIP group at MSR studies human interactions among software developers, I extend that and study the human interactions among software teams (or project members): developers, project managers, and usability experts in both proprietary and open source software development environments.

As I mentioned in my last blog, I am studying the role of usability expertise in both software environments through surveys, interviews and observations. I have previously reported on the open source software survey and observations. In this blog, I am reporting on the interviews I conducted internally at Microsoft. I spoke to eleven employees who are working on various projects at MSFT in various roles including program managers, developers and user experience researchers and designers. I spoke to junior and senior staff and well as leads. Program managers are responsible for the feature that their team designs and builds. Developers write code. User experience designers create mockups and give feedback in design meetings and user experience researchers collect field data and conduct usability studies.

Travel Interlude

I wrote the above section before I left Redmond and now I am back on campus at Penn State. I had to hurry back and drove straight through from Redmond to Minneapolis. We left Redmond in the afternoon and stopped in Spokane for dinner and left at dusk. Spokane looks like it is growing and as such has some money injected into its economy. We drove through the night passing through Idaho and through Montana the next day. We passed through North Dakota late in the afternoon and stopped in Fargo for dinner. After dinner was the biggest rain storm I have ever seen – and I am from Vancouver, Canada where it rains a lot. I was driving southeast to Minneapolis on I-94 and slowed down to 10MPH because the rain was pelting down and blowing so hard across the road that it was like a whiteout. I could barely see five feet ahead. We made it to Minneapolis (avoiding the I-35W bridge area) at about 1AM and checked into our hotel. The next morning we awaked late, had breakfast, did some grocery shopping at a favorite natural food coop called the Wedge. I used to live in Minneapolis when I worked at Unisys and Promedicus--a startup that made decision support systems for physicians that died with many other dotcom startups. It was nice to be back to The Cities.

After our deserved travel break we went to Madison, WI to visit an old college buddy of my husband’s. They used to play in a band together. His buddy Frank still plays. We stayed there way longer than planned, but it was fun to catch up. We ended up reaching our next destination really late. We ended up in South Bend, Indiana (home of Notre Dame) when the sun started to rise and the birds began chirping. We were so tired the next day that we forgot things in the hotel, including a credit card! We did not notice it missing until we got to Toledo. Luckily it was the last leg of our trip and we made it back to State College, PA around 11PM and slept in the next morning ready to move back into our townhouse on campus. End of Travel Interlude.

Now that I am back on campus, I have had some time to reflect on the interviews. They uncovered a variety of interesting things. Overall the eleven people I interviewed were very enthusiastic about the research and most wanted to see results. Again, I have not analyzed anything formally yet, but all of the people I interviewed mentioned that communicating design changes was very challenging especially when it comes to usability issues. It seems like the biggest challenges relate to power relationships (not their words) among the team members and the ability of the person with usability expertise and training to gain trust with decision makers. A prevailing problem is that some people tend to think they are usability experts even when they are not trained and if they are more pushy or otherwise in a position to make the final decision, usability might be compromised.

Of course many other factors weigh into the usability of a product, but overall it seems that the usability experts are being heard one way or another. In comparison to usability in open source, a large proprietary software company has more resources for bringing usability expertise into products, but the social dynamics appear to be as complex as in open source. The only difference may be the characteristics of the dynamics. In my observations online of open source usability discussions, most of the interactions seemed to be devoid of such social dynamics, except for one group about one issue. So in comparison, open source might not have the same kinds of power relationships because the roles are not as differentiated. As I continue to investigate the characteristics of usability expertise I will see what open source interviews turn up. Stay tuned.

Pondering the preliminary results of OSS usability research, Part 1

Paula Bach — Wed, 01 Aug 2007 17:35:00 GMT

In my last blog I talked about interdisciplinarity and multidisciplinarity and a little bit about my research this summer. In part 1 and 2 of this blog I am going to talk more about the research I have been doing here at Microsoft. Over the last few months I have been looking at a phenomenon called usability expertise. Anybody who has had difficulty using a product has some experience with usability expertise. Usability expertise is knowledge about how to design an artifact to ensure users experience product effectiveness, efficiency, and satisfaction in a specified context of use. Even if people are not experts in Human Computer Interaction (HCI), they can experience a lack of usability expertise in the design of the product.

HCI experts are actually quite rare because the field is young and underdeveloped. The field of HCI is newer than computer science. HCI grew out of computer science about fifteen years after the software engineering crisis in the sixties and although Human Factors is about fifty years old, it has not necessarily been linked to software engineering like HCI has. Software development has included a user interface role to design and develop the human-computer interface, and although some companies still employ user interface developers, HCI experts include UI designers, Usability Engineers, User Experience Researchers and Designers, and Interaction Designers—roles that go beyond the interface and include field research, visual design, and lab studies, for example.

Although the obvious place to look for usability expertise is in the knowledge of HCI experts, I am interested in what role this expertise plays in software development. Just having HCI experts available is not enough to ensure good usability. I want to know who has usability expertise, how it is communicated among project members, and how it is used to make decisions. To find these things out the research looks at both proprietary and open source software development settings. What I am reporting here is an overview, or summary, of preliminary findings. I am still analyzing the data and will publish “official results” in the next year and a half while I work on and finish my dissertation. The research seeks to understand the role of usability expertise in software development and takes that understanding to inform the design of a feature or tool on CodePlex that will support usability expertise for projects interested in making sure their software is usable by their intended user base.

Usability expertise in the context of design is related to design rationale, or more specifically usability design rationale. Design rationale is the "the capture, representation, and use of reasons, justification, notation, methods, documents, and explanations involved in the design of an artifact" (from the book Design Rationale by Moran and Carroll). Since design rationale is a well defined concept that has many details, its presence in real design discussions may be fragmented. This fragmentation might be better understood as usability expertise. So a rough definition of usability expertise might be the “stuff” needed to talk about and make decisions about usability during software development. The “stuff” could be the elements in design rationale or something people have not talked before. In this sense my discoveries made while investigating the role of usability expertise could be groundbreaking or they could be well known in the software development communities. Either way reporting the findings of the role of usability expertise should be interesting. In fact, several people, both at Microsoft and the open source communities I surveyed have already stated that they would like to see the findings, so this is encouraging.

I am collecting data in a number of ways: surveys, interviews, and observations. I surveyed people at Microsoft who are part of the software development process of a project, namely usability experience researchers and designers, developers, and program managers. In the open source world I posted the survey to major projects who met criteria for overtly caring about usability, namely that they had a usability list and at least one person listed as a usability expert. The Microsoft usability expertise survey is still collecting responses, and although I am still working with the data on the open source survey, I can mention a few things.

In the open source survey, fatigue affected about half of the 125 respondents with 56 making it to the last question. The survey had two open ended questions asking about the importance and challenges of usability in open source. Usually open ended questions are best saved for the end after other more important questions are answered. The tradeoff was that the open ended questions were important and that the survey could have biased the open ended responses if they were at the end because the survey included questions that asked about specifics with the importance of and challenges with usability.

Data clustered around categories of ease of use, simplicity, and consistency for usability importance, with each category claiming about a quarter of the responses. About 10% of the respondents stated that issues related to system performance were important for usability. Usability challenges included about a quarter of the respondents reporting that challenges with usability in open source software development were developer based. This included not valuing usability, not having usability expertise other than self-referential (based on own experience), and communication problems related to common ground. Common ground is when two people reach a mutual understanding such that one person knows that the other person knows that the first person knows. Common ground is more difficult to reach in computer-mediated environments than in face-to-face environments because not as many channels exist to help with understanding—in face-to-face you can use people’s expressions and gestures to help you understand what they are saying. Other categories included lack of resources and lack of process (both at about 10% of the responses). Other questions I am asking the data include the following:

1. Who has usability expertise?

2. How is usability expertise communicated?

3. How is usability expertise used to make decisions?

4. Who cares about usability expertise?

5. How available is usability expertise?

The data may not be able to answer the above questions in full, but it will get me closer to asking different questions that may be more relevant to the data. I am conducting interviews which may also be able to address the questions and get at depth surveys cannot.

I have been scheduling and conducting interviews with Microsoft people and will report on those preliminary findings in the next blog. I will conduct the open source interviews via video conference when I get back to Penn State. The open source usability people I am going to talk to are all over the world: US, Canada, Germany, Australia, and France.

I have also been observing three open source projects looking at email lists and other interesting things like conversations in the bug tracker, how a usability issue is handled in the bug tracker, and reading UI specifications. I chose three ‘big’ open source projects that attend to usability. I wanted diversity in the projects and a wide user base. I spent 8 weeks observing the workings of usability in Firefox, KDE, and OpenOffice.org. The discussions on the email lists vary considerably. Some are short and polite with a developer inquiring about the usability of a particular design change or feature he is thinking about. Others are heated and get users, developers and usability people involved trying to hash out the merits of a feature.

The most often used design rationale, or type of usability expertise, is self-referential. The people on the lists, and in the beginning mostly users or user/developers respond to the feature proposal, speculate about the usability of the change based on their own experience. Since most of the users on the lists are advanced or power users, this might not be representative of the main user base, at least for the three projects I was studying. I don’t know if they have any data about the user base, but it may be that the email lists are only one input to the decision making about usability of those projects. Despite the openness of the discussion list and other aspects of the development, there are other decisions that are made ‘behind the scenes’. Possibly, the ‘behind the scenes’ usability expertise that contributes to decision making about which usability fixes to include in the next release is similar to how proprietary usability expertise is used in decision making. This is something I will consider when investigating the role of usability expertise in both environments.

Greetings from the Open Source Software Lab

Paula Bach — Wed, 13 Jun 2007 00:46:00 GMT

Bryan has previously blogged about the project partnership between the Penn State University (PSU) College of Information Sciences and Technology (IST) and the Open Source Software Lab (OSSL). I am at the OSSL here at Microsoft this summer and next as a research intern. The project, which started in May 2007 and will last two years, is my dissertation research. I work with Jack Carroll in the Center for HCI at Penn State. I am a third year PhD candidate and I study HCI in open source software development.

In this blog I want to talk about interdisciplinarity and multidisciplinarity. Broadly speaking, the information society is like the Wild West and many challenges as well as opportunities, especially with information technologies, have arisen. So for example, the Internet is like the Wild West of the information society. Challenges and opportunities in a new frontier are exciting for business and academia at once. Understanding the challenges and opportunities, however, needs new ways of investigating. A single discipline can address some of the challenges and opportunities, but complex problems, especially ones involving the intersection of information, people, and technology can benefit from expertise from multiple approaches. This is where a multidisciplinary or interdisciplinary approach can be helpful. Rogers et al (http://rizzo.media.unisi.it/page2/assets/Rogers_Scaife_Rizzo.pdf) make the distinction between interdisciplinary and multidisciplinary:

Interdisciplinary usually means “the emergence of insight and understanding of a problem domain through the integration or derivation of different concepts, methods, and epistemologies from different disciplines in a novel way.” Multidisciplinary can be characterized as “a group of researchers from different disciplines cooperate by working together on the same problem towards a common goal, but continue to do so using theories, tools, and methods from their own discipline, and occasionally using the output from each other’s work.” The characterizations differ in whether elements of a discipline are coupled or decoupled.

Although both terms have been used interchangeably, the subtle differences in problem solving depend both on the kind of problem a team of collaborators is solving and on the investigatory skills of the team members. The OSSL takes both approaches to both the challenges and opportunities inherent in understanding the open source and where Microsoft fits in. This broad approach is inherent when comparing Microsoft’s past and current missions: A computer on every desktop and in every home running Microsoft software compared to To enable people and businesses throughout the world to realize their full potential. The missions shifted from technology-centric to people and organization-centric. This new approach includes a global perspective on key aspects of the information society: people, information, and technology. This new approach is also exemplified by a new type of academic unit called information schools, or iSchools. The joint project, looking at HCI in open source software development, is interesting from a number of perspectives in the space of information, technology, and people. My approach is interdisciplinary, taking a number of concepts and methodologies and combining them in using different epistemological perspectives. Please contact me if you would like details on the interdisciplinary nature of the study of HCI expertise in open source software development—it would be too long to expound on here.

Bryan and I recently went to the iSchool at University of Washington to talk to graduate students and faculty about the project. The research conversation, as it is called, was well attended especially for a sunny Friday afternoon at the end of the spring semester. (The iSchool dean even showed up!) We talked about the challenges of studying the open source community and about doing interdisciplinary research in an iSchool.

The most interesting aspect of my experience so far as part of this joint partnership is that I am doing interdisciplinary academic work in a business unit studying open source software development at Microsoft – all of which are normally ”separate worlds” (academic/business and Microsoft/open source software). My summer here will entail collecting data and analyzing results of HCI expertise in open source software development as well as looking at HCI expertise in software development internally at Microsoft as a basis for comparison. In this summer series, look for my blog entries as I ponder results from the studies.

What We Do Every Day

Bryan Kirschner — Sun, 20 May 2007 21:53:00 GMT

I just read Bill and Sam’s “Business as Usual” post. It made me think about the fact Port 25 was established in part to apply the idea that “transparency increases trust” to the work we do with the lab. So I’m sitting down to do a blog entry that’s a bit longer than usual, but will provide transparency about why “business as usual” for me.

I previously blogged about a project we were starting to look at usability, human-computer interaction (HCI) and design rationale in open source development. I want to share how that came about and what I work on every day, over a period of about 12 months.

Part 1: Andrew Ko at Carnegie-Mellon (hi, Andrew) and folks from Microsoft Research (you rock, HIP) have done fascinating work on “Information Needs in Collocated Development Teams:” (emphasis added):

[In] a two-month field study of software developers at Microsoft. We took a broad look, observing 17 groups across the corporation, focusing on three specific questions:
What information do software developers’ seek?
Where do developers find this information?
What inhibits the acquisition of such information?
In our observations, we found several needs. The most difficult to satisfy were design questions: for example, developers needed to know the intent behind code already written and code yet to be written.

Code itself was a poor conductor—let’s call it bad currency, for reasons that will become apparent later—for transmission of design knowledge. From the MSR paper:

code did not look like design; intent could rarely be inferred from code; programming languages only allowed a single, structural perspective on code, yet there were many other perspectives on which developers reasoned about code

As a result, “the knowledge was primarily stored in the minds of developers. Consequently, developers relied on each other for design knowledge.” A common way to do this was face-to-face contact. Another way to do this was through email.

Part 2: Flore Barcellini (hi, Flore) is a research at INRIA (France) who has done a fascinating analysis of “Thematic Coherence and Quotation Practices in OSS Design-Oriented Online Discussions.” The implication is that traversing threads may be a lot more “lossy” than one might think because the “tree” you can build following transmission of knowledge using quotes can differ (from the abstract):

We show how quotation practices can be used to locate design relevant data in discussion archives. OSS developers use quotation as a mechanism to maintain the discursive context. To retrace the thematic coherence in the online discussions of a major OSS project, Python, we follow how messages are linked through quotation practices. We compare our quotation-based analysis with a more conventional analysis: a thread-based of the reply-to links between messages. The advantages of a quotation-based analysis over a thread-based analysis are outlined.

All but a few open source projects do not receive investment from vendors and do not have material revenue streams—for these “community-driven” projects, face-to-face contact would obviously be prohibitively expensive. So in reliance on code and email to transmit design knowledge, they would seem to be dependent on a lossy medium (code-as-currency) and a lossy mechanism (mail threads).

Part 3: David Nichols and Michael Twidale (hi, Michael) have done research identifying usability & HCI challenges in open source development, thoughtfully articulating some of the issues and possible ways to evolve distributed development.

Part 4: I was left with the impression this is a scenario that is really not good for community-driven OSS—and, by implication, for any resource-constrained distributed development process (something applicable to end-user developers collaborating online, and perhaps small ISVs, communities large in both number and importance to Microsoft’s business).

After reaching this conclusion I contacted the Codeplex team (meet the team) to talk about Microsoft taking a role in developing new functionality that might help this scenario. But first we needed to establish a research program to figure out whether this was a good path to go down, and what to do. That led to contact with Jack Carroll, Paula Bach, and the current project.

The first public session we held on this was a special interest group (Usability and Free / Libre / Open Source Software) at the recent CHI 2007 conference. Jack, Paula, and I moderated. I’ll let notes mostly from Paula sum up one aspect of a great discussion that gave me ideas I’d never thought of before:

About 40 people (1/3 to 1/ 2 of whom were involved in open source projects as contributors or researchers) attended the CHI Special Interest Group (SIG) on Usability and Free/Libre/Open Source Software (FLOSS). The group raised many issues including the “code as currency” issue. In essence, if “code is the only currency’ can there be a “benevolent HCI dictator?” The currency problem arises when HCI people who don’t write code work on FLOSS projects, potentially preventing the common mechanism of the “benevolent dictator” who can arbitrate conflicts over coding from emerging in the design and HCI domain. An interesting benevolent HCI dictator experiment would be to have HCI people design and initiate an open source project (it could even be a rapid prototyping tool that could be used as currency between FLOSS HCI people and developers) and have developers work on the project with an HCI person as the leader. This would be interesting in terms of social dynamics and to see who prevails as the benevolent dictator: would the HCI person remain or would a developer move into the leadership position once code writing began?

This is what we do every day. I hope this provides a bit of a view over time into our daily work to be center of excellence for (1) understanding and (2) finding opportunity with open source: ways for Microsoft and open source to “grow together.”

(Speaking of growing together, if you were one of the companies invited to the Microsoft Open Source ISV Forum before OSBC, I hope to see you there.)

It's Like This...Or Maybe Like That...(Part 1.2)

Bryan Kirschner — Tue, 12 Dec 2006 17:24:00 GMT

It’s been just over a month since I last blogged on the law-and-open-source –analogy, and, despite a cool, unrelated entry in the middle, I feel my blog karma is running dangerously low… But—proving either that life is a journey of continuous learning and joyful surprise, or, more simply, that good things come to schlubs who drag their feet—last week not only did NPR run a story on legal apprenticeship programs, I also heard a speaker who thoughtfully referenced Foucault in a talk on fostering innovation.

The first line of reasoning was about rather than being a case where the “source code” of the domain (law) was “closed,” law is a case where it is really, really “open:”

legal documents are almost universally public as well, so you can seek an example of someone else’s filing, brief etc from among literally millions of such documents—from the lowliest pleading to the most momentous Supreme Court argument. If a situation where the full text of millions of legal artifacts available freely (or for the price of distribution) aren’t like open source code…I’m not sure what is!

The implication, going way, way back to the first blog in this chain, is that it seems you can make a lot of knowledge (qua legal artifacts) “more like open” without new artifacts becoming “cheap” (lawyers still are, or at least feel, expensive).

There is a different angle, though: the second topic was about restricting access to the profession itself. Is law an example where the “openness” of knowledge is counterbalanced by extreme restrictions on being able to function as a lawyer—specifically multiple years of (often expensive) law school? In this blog entry I am going to take a stab at articulating three views on this: let’s call them “the bad scenario,” the “not bad but not great scenario,” and the “pretty cool scenario.”

(Since we’re comparing to open source development, I am going to make one simplifying assumption: having to pass a test—the bar exam—is entirely compatible with being “open.” I say this because I see it as analogous to many OSS communities: to become (say) a committer you basically show up and start doing some work to demonstrate your skills; the test is analogous. If the bar exam is broken somewhere in terms of content or form factor, I see that as a tweak as opposed to fundamental to the show-up-and-demonstrate-your-ability analogy.)

First “the bad scenario” (this is where Foucault comes in). To be somewhat painfully reductive, Foucault observed there is a relationship between the structures of power and (ostensibly objective) knowledge, and that a characteristic of the modern world was the application of “discipline.” Discipline, broadly speaking, is defining and conditioning a state of behavior that creates non-egalitarian power relationships such that it becomes “normal” and this dynamic becomes invisible within a formally egalitarian, “fair” and “open” environment. The real rubber-hits the road example here would be a cycle that works like this: based on a scientific and meritocratic rationale, three years of law school becomes a standard; as a consequence, most lawyers who pass through this system view it as both feasible(since they did it) and meritocratic (since they by definition did well, or at least well enough, in it). In tandem, it also means lawyers are relatively scarce and expensive and have similar billing rates and incomes, because they all ponied up fairly homogeneous investments. Thus, they have psychological and material incentive to perpetuate the system, because in rejecting it they would not only repudiate their own accomplishments, but possibly introduce competitors who could undercut them. In the end, the system justified its own perpetuation without controversy…because it seems to comport with the accepted paradigm of a modern profession.

On to the “not bad but not great” scenario. This is where Spence and Signalling Theory come in. The Wikipedia entries linker above are concise so I recommend reading them: the upshot is that for education to be used as a “signal” to help employers choose more valuable employees, it is not necessary for education to have any intrinsic value. The reason this is “kind of OK“—despite the fact that investing in education even if it doesn’t increase productivity seems intuitively perverse—is that at least solves a (communication) problem, and can be economically efficient. By contrast, the’ bad scenario” is quite likely to cause the behavior of the system will become very, very inefficient relative to a truly open consideration of all the options.)

Finally, the “pretty cool” scenario. Seven states enable people to become residents without attending law school through some type of apprenticeship program :

In Vermont, participants don't need a college degree, but they must have completed three-quarters of their undergraduate course work. Then they have to spend 25 hours a week for four years studying alongside a licensed attorney.

In Washington state, a “law clerk”

…shall study for 4 calendar years. Each calendar year shall consist of 12 months, with a minimum of 120 hours of study each month, including the time spent in performing the duties of a law clerk. The tutor shall give personal supervision to the law clerk averaging at least 3 hours each week. "Personal supervision" is defined as time actually spent with the law clerk for the exposition and discussion of the law, the recitation of cases, and the critical analysis of the law clerks written assignments.

(In both cases these positions can be paid jobs.) This suggests a radically different paradigm for entry into the profession (one might, and Foucault might agree, a “throwback” to a previous era), and a lovely mentoring dynamic. What’s particularly interesting is that women outnumber men in apprenticeship programs, and the typical age of the participants is older than law students. And while at least according to NPR the likelihood of programs like these increasing, among law schools there is an increasing incorporation of paid internships and flexible schedules.

I think parallels to each scenario can be made to the open source development domain, and thinking about the “balance” of scenarios in both the legal and software development domains will be a fascinating discussion. That blog will be along in less than 30 days, I promise….

There's a Vendor in my OSS

MichaelF — Mon, 23 Oct 2006 14:34:00 GMT

Taking a brief detour from the thread about OSS and its similarities (or not) to law to take note of a couple recent publications, both of which discuss the interaction between traditional IT vendors and OSS:

In MIS Quarterly (September) (link) Brian Fitzgerald (University of Limerick—one of the must-read researchers on OSS, IMO) provides a comprehensive survey of what he calls “The Transformation of Open Source” with expectations for “Open Source 2.0.” He expects IT vendors—including Microsoft—to play significant roles in “OSS 2.0.”

In Communications of the ACM (October) (link) Pamela Samuelson (UC Berkeley) discusses “IBM’s Pragmatic Embrace of Open Source.” (The title pretty much speaks for itself as a summary.)

I highlight these because they reflect what seems to me (qualitatively) to reflect a trend in the literature. We’ll work on getting a better sense of what the trend is and researchers’ perspectives on it to bring back to Port25…because (to bring things back to analogy and metaphor), they introduce the question: Is a vendor in my OSS more a fly in the ointment or chocolate in my peanut butter?

It's Like This...Or Maybe Like That... (Part 1.1)

MichaelF — Thu, 28 Sep 2006 20:31:00 GMT

In my last blog I started talking about the power of analogy and metaphor, and dove into a discussion of the first analogy of my collection, asking what if the practice of law, rather than being like a domain suffering the consequences of a “failure of openness,” was more like an example of a domain with a great deal of openness. I promised to offer some ideas for analogies that helped make sense of the situation in my next blog.

I was subsequently intrigued by where some of the comments to the post (unexpectedly) led me; thus, I postpone those (original) ideas in order to share some of where those comments led me on the path toward new ones. In this post, I will address the first of two substantive issues: the analogy made by CDarklock between the legal profession and open source software development (--in his view, to the disadvantage of OSSD).

Comparing “do-it-yourself lawyering” with “do-it-yourself usability,” he implies a lack of sensitivity among developers to the risks of the latter relative to the legal profession’s diligence with respect to the former.

Surely enough, when pointed to Groklaw by stats for all, front and center is a warning of the risks of the former:

IANAL. I am a journalist with a paralegal background, so if you have a legal problem and want advice, please hire an attorney.

This prompted me to think about some quick phenomenological test to determine if similar evidence for this type of distinction made by journalists commenting on law (Groklaw) might in fact be recognized with equal alacrity by open source developers commenting on usability: in fact, in defense of OSS developers, there are indeed cases of IANAUE (“I Am Not A Usability Engineer”)—although AFAIK it has not made Wikipedia yet, in contrast to variants of the apparently fecund IANAL which has spawned IANYL, TINLA, and IAAL. (WTF?)

At this point you’re probably wondering why I was so engaged on this topic—the first reason is that, IMHO, challenges with consistent and effective usability practices are endemic and impactful. (I will never forget my introduction to the usability disipline: I was helping usability engineers build a stochastic model on top of their user observational testing of a web experience which (thankfully) has since been improved. The model was pretty cool as a quantification of how much customer time and effort things like ambiguous terms and redundant links actually wasted—but nothing seared the importance of usability into my brain like watching a test subject (a middle-aged, tech-savvy woman with, as I recall, a PhD—kind of hard to blame the user) actually start to cry in frustration as she tried to complete a task.)

The second reason is that I had never considered the possibility of an analogy between “ legal self-representation” and “developer self-usability” as conceptually similar problems to be solved. This analogy offers a different (and interesting) way to think about why it occurs and what to do about it in OSSD, in contrast to traditional corporate development where the origins of usability challenges nor their resolution seem to me to be fairly straightforward: does a company (or development group) recognize the value of good practices, resource for it, make it a priority, test against user interaction metrics etc.

In fact that there is a paper (Nichols & Twidale, online at First Monday) which provides a comprehensive assessment of usability in OSSD, with suggestions for remediation that come awfully close to echoing ideas for what folks in the legal profession would call increasing access to justice (--like academic volunteerism and corporate involvement).

It’s an interesting line of thinking both because it is just in time for CSCW 2006 (Computer Supported Cooperative Work—anybody going?) and because folks in the OSS community (just like in the legal community) are doing some “out of the box thinking.” I’ve been trading mails with a PhD candidate at Penn State who has outlined a very thoughtful research agenda on OSS and Usability—I’m pleased to say we’ll be bringing her to Port25 for an interview near the end of October.

With that said, the next point release on our path to part 2 will address an issue raised by ssjdrn who surfaces (in my words) a tension between two principles I have always taken for granted: efficient signaling and disciplinary control through normalization that is keeping me up at night (literally—when Spence and Foucault, respectively, aren’t jibing for me, I can’t sleep. You can ask my wife.)

And yes, it all does come back to an even richer –than-anticipated analogy between law and open around making successful software.

It's Like This...Or Maybe Like That...

MichaelF — Fri, 08 Sep 2006 22:19:00 GMT

I am compiling a list and analysis of all the analogies and metaphors that have been used to characterize open source software development and its social, technical, and business implications. I think it is unlikely this will be the next DaVinci Code-style best seller, so I don’t expect to give up my day job, but my interview of Professor Alfonso Fuggetta where he talked about the distinction between specification and implementation reminded me of why I started this project.

Analogy and metaphor are fundamental to not only how we understand something, but also to how we form our opinions about it; in an influential paper on artificial intelligence and cognition, the authors emphasize the centrality of analogy to not only how we make sense of the world but its powerful and often unrecognized implications:

When people make analogies, they are perceiving some aspects of the structures of two situations—the essences of those situations, in some sense—as identical…Furthermore, not only is analogy-making dependent on high-level perception, but the reverse holds true as well: perception is often dependent on analogy-making itself. The high-level perception of one situation in terms of another is ubiquitous in human thought. In the large or the small, such analogical perception—the grasping of one situation in terms of another—is so common that we tend to forget that what is going on is, in fact, analogy.*

The authors use a few contemporaneous controversial situations from US politics in the late ‘80s – early ‘90s to illustrate this point—what struck me about their emphasis on the power of analogical reasoning to shape opinion was the fact that analogies to “Vietnam” and “Hitler” were used in those examples in their paper and, 20 years plus later, those same historical events are in the news as analogies today. The two examples highlight how acceptance of the analogy carries with it a strong bias to shape your attitudes and your behavior (even manipulate your emotions): analogies don’t necessarily just frame the argument—sometimes they are the argument.

The analogy that’s top of mind for me today is from Open Sources 2.0 (Sam blogged about it here), and was laid out by Matt Asay (who frequently has interesting things to say on his blog or, for that matter, here on Port25). It’s one of those that I think warrants some discussion because, as it stands, I think it actually raises much more interesting questions than what Matt initially describes.

In his chapter “Open Source and the Commodity Urge: Disruptive Models for a Disruptive Development Process,” he is arguing open source reduces prices by facilitating the option to “do it yourself.” After hiring out their landscaping, including cement work, he goes on to relate the unfortunate story of having to pay (high) legal fees to resolve a dispute with his (bad) cement contractor. The price of legal assistance was high because of the “skill set involved and the artificial licenses set up by the legal profession to keep would-be attorneys in would-be land.” And necessary because (emphasis added) ”I am effectively barred from accessing the “source code” of the legal…profession, which drives up the price I must pay.”

Here’s the interesting thing: as much or perhaps more than any other domain, we are not barred from accessing the “source code” of the legal profession.

True, there are hurdles you have to go over to practice as an attorney and represent others—but that is not relevant to the “do it yourself option.” In fact, you are entirely able to represent yourself if you want to, and some venues (small claims court and many administrative processes) are designed to accommodate self-representation. So, first—relatively speaking—law is an areas where you have a lot of latitude to “do it yourself” (by comparison, “do it yourself law enforcement” or “do it yourself public highway repair” by contrast, is actively discouraged.)

Second, the “how-to” of underlying knowledge is perhaps more widely available in the legal domain than in any other: there are many public law libraries and a large (although its practitioners would say not big enough) network of volunteer assistance organizations.

Finally, legal documents are almost universally public as well, so you can seek an example of someone else’s filing, brief etc from among literally millions of such documents—from the lowliest pleading to the most momentous Supreme Court argument. If a situation where the full text of millions of legal artifacts available freely (or for the price of distribution) aren’t like open source code…I’m not sure what is!

This analogy was powerful enough to lead me to think a long time about the contra case: not is lack of openness inflating prices, but what if a high degree of openness does not reduce prices (--or at least “reduce enough” relative to expectations). Thus the question: are there analogies that help make sense of this possibility? I have some ideas that I’ll offer in my next blog.

*High-Level Perception, Representation, and Analogy: A Critique of Artificial Intelligence Methodology; Chalmers, French, Hofstadter 2001

Ants and Software Development: Thoughts from the ASA Conference

MichaelF — Wed, 16 Aug 2006 20:12:00 GMT

I’ve been surrounded by people who want to study us like bugs—and they intend that as a compliment.

I just got back from the attending sessions of Communication and Information Technology Section of the American Sociological Association (CITASA) at the ASA conference in Montreal. This group of researchers studies:

The social aspects of computing, the Internet, new media, computer networks, and other communication and information technologies. This includes online communities, knowledge management, the digital divide, labor markets, workplaces, and how the Internet fits into everyday life [and] the design and use of technology [including] developing and analyzing new kinds of software, and thinking about the implementation of technologies for teaching, research, and the real world.

Myself and Prof. James Witte (Clemson), Chair of CITASA

The thought of bugs struck me when I sat down to recap my experience because, I thought—as a systems-minded person myself—an individual bug is not that interesting. But put a lot of bugs together, and the humble ant, bee, or termite, can, by acting in concert, create spectacular feats of engineering. Sound like building software? –That’s interesting.

What’s particularly important is that these practitioners focus on the criticality of social dynamics to any endeavor—the relevance to distributed, voluntary open source development is obvious, but these dynamics are important to closed-source development, diffusion and use of technology, information dissemination, bridging the “digital divide”…

I know I won’t do justice to everything here and now: suffice to say there are folks working on things like how building your reputation on Slashdot works (got your attention?); how to predict the level of documentation that will be produced by developers given different levels of social reinforcement for contributing code versus good documentation (…a challenge as applicable to proprietary as open source development, in my experience); and how different newsgroup communities use data to measure and control their “community health” (one group dedicated to quilting (yes, like with needles and fabric) seem to be particularly aggressive about this, going to show that traditional geek stereotypes may be becoming victims of the ubiquity of the Internet.) I’ll just promise that we will work hard on bringing some of the most interesting and relevant information, and interviewees, to Port25 over the next year.

Before I go nurse my jet lag, two quick notes:

First, since it is always nice to agree with the Boss (--which holds true for both Bill Hilf and Bruce Springsteen, IMHO), I will pile on Bill’s blog post about closed (or open) mindedness. If you happened to click through to CITASA’s website, you might have noticed something: our lab is a sponsor, Microsoft Research (MSR) is a participant (and a long-time participant and creator of some of the leading software for conducting social network analysis, I might add)—and the site is hosted by the Clemson Linux User Group. Bernie Hogan, one of the sociologists at the conference, said it best when the group turned to discussing “moral panic” (their words) or what one might also call hysteria (my word) about MySpace and “Internet Child Predators” right now: researchers have a critical role to play in “explaining what’s really happening.” I take as a point of pride that we share across our Port25 team, CITASA, MSR, and (I hope) you the reader a commitment to understand what’s really true—empirically measured, tested, challenged, tire-kicked…if everyone was focused on finding the demonstrably best solution to well-articulated scenarios, the world would be, IMHO, a better place…and the perceptions Bill referred to would take care of themselves expeditiously.

The second point is that my bug metaphor, above, has a basis in published research: Valverde, et al, from the University Pompeu Fabra, Barcelona, Spain, published “Self-organization patterns in wasp and open source communities” in IEEE Intelligent Systems, March-April 2006 (Volume 21, Issue 2). In this “comparative study of how social organization takes place in a wasp colony and OSS developer communities” they found “both systems also define interacting agent networks with similar common features that reflect limited information sharing among agents.”

And if you didn’t think that I really do care about tracking down every single bit of knowledge available to understand “what’s really going on”….I read the whole thing.