Port 25: The Open Source Community at Microsoft : Paula Bach

Bridging Chasms

Paula Bach — Mon, 18 Aug 2008 19:57:00 GMT

I have blogged previously about interdisciplinary and multidisciplinary research. Now I want to turn to disciplinary chasms in software development. Social aspects such as how people communicate, collaborate, and coordinate interest me because as Andy Begel in the HIP group in MSR found, software development teams when coordinating and communicating with other teams, are like dysfunctional families. I won’t go into details, but software development is socially complex. Software development teams generally consist of software engineers, to state the obvious. But in the last ten years or so, many software development businesses began to implement user-centered design because they realized that software could be frustrating for users. A new discipline arose where people were either trained in Human Computer Interaction, or learned on the job. Professional organizations like UPA and SIGCHI are thriving. But software engineers and usability experts are from different communities of practice. Although they both work on creating software products, their goals, values, approaches, culture, and well, their work practices differ tremendously.

A theoretical approach called communities of practice helps to describe and explain the social nature of learning and interacting in communities. But the approach does not yet incorporate multidisciplinary communities or how two communities of practice come together to accomplish their goals, like designing and building software.

In my experience, through being a designer practicing user-centered design and a researcher studying user-centered design in software development practices, about half of any design practice is communicating your ideas across disciplines or communities of practice and actively listening and working to understand different perspectives. Other practitioners echo this observation as well: Gitta Salomon of swimstudio.com, an interaction design firm, states that “One of the biggest challenges is remembering that half of what we do is the design work and the other half is the communication of that design work.” (Quoted from the book Interaction Design, by Preece, Rogers, and Sharp. This book is an excellent guide, both theoretically based and practical, most people trained in HCI will know about 80% of what is in this book and apply aspects everyday.)

Bridging software development and user-centered design could be investigated by looking at communities of practice and paying attention to communication could help bridge the chasm between the two disciplines.

CodePlex project developers wanted

Paula Bach — Mon, 30 Jun 2008 14:14:00 GMT

I would like to invite CodePlex developers to participate in research. If you have been reading my blog posts here on Port25, then you will know that I have been investigating how to integrate usability into open source software development. Now I am seeking research volunteers.

I am looking for developers who are working on projects hosted on CodePlex. The projects could be in the planning stage, alpha, beta, or stable. As a volunteer, you would be asked questions about your activities performed on CodePlex. The interview should not take longer than 60-90 minutes.

Data from the research will be used to help design support for usability on CodePlex and analyzed as part of my dissertation at the College of Information Sciences and Technology at Penn State University. I work with Dr. John M. Carroll at Penn State and Rob DeLine at Microsoft Research. One part of the study data collected from CodePlex users will inform the design of new usability support features. The other part of the study is to understand how developers work on open source projects. This research has been approved by The Pennsylvania State University Institutional Research Board, IRB #27804. As such, data will be used for the above research purposes only.

If you would like to participate, (participation is voluntary) or have any questions, please email codeplexresearch@live.com.

Go Hybrid

Paula Bach — Mon, 09 Jun 2008 19:29:00 GMT

I am back in Redmond. For those of you who don’t know, I spent last summer here in the open source software lab conducting research on integrating usability into open source. My last blog talked about how I have made some changes to my research program. One change is to situate the research argument within the broader scope of how open source has been changing. My favorite paper discussing this is titled “The transformation of Open Source Software” by Brian Fitzgerald and is only available if you have a subscription to the MISQ journal, but you can download an audio mp3 version read by Fitzgerald himself here. He coins the term ‘open source 2.0’ and characterizes software development in open source and compares it to proprietary, and shows that open source software development has elements of proprietary software development and proprietary has elements of open source. Any of the big open source projects with paid developers, and my favorite example, Mozilla, with paid UX professionals, is an example of the former, while Microsoft is an example of the latter. When I first met Bryan Kirschner (now director of open source strategy at Microsoft) he introduced me to the idea that API developer communities are a lot like open source communities.

The most interesting part of my research is that it is situated right in the middle of open source hybridization. A hybrid open source software development model combines a business model, either open source or proprietary, and open, two-way community input. The basis of my argument for the research is as follows: open source software development has been so successful that proprietary companies have been paying attention to incorporating open source strategies into their business model and very successful open source projects have had business models created around them. Both of these phenomena share some characteristics of software development, but taking a well-developed model of usability and transplanting it into a hybrid software development environment will be challenging because the hybridization landscape is still being cultivated. Because Microsoft has been successful with integrating usability activities into its production of software, it makes an interesting case to investigate how one of their hybridization strategies, CodePlex, integrates usability.

I am working with the CodePlex team to develop usability support for CodePlex. This means that the CodePlex community will have a say in how we design the support. Traditionally, open source projects are challenged for usability resources so the support has to range from being able to support code-centered and usability-interested developers to the possibility of usability professionals. The project addresses three main challenges for usability in open source: merit and trust, chasm between work activities, and incommensurable tools and methods. If you have a project on CodePlex and are interested in participating in this research, then please contact me: codeplexresearch at live dot com.

Passing Without Talking

Paula Bach — Wed, 06 Feb 2008 00:29:00 GMT

My last blog was about me traveling to Limerick and Toronto. I have now defended my dissertation proposal and passed. (Yay!) Here is a funny story. A week before the proposal defense I created my presentation and rehearsed it every day until the third day before when I began to get a sore throat. I don’t think it was from rehearsing the presentation or nerves or anything like that. Instead it was just a bug that was going around. Lots of students are sick at the end of fall semester. Anyway, two days before my defense I was getting a froggy voice, so I did not talk all day long. The day before the defense my voice was really raspy. The night before I worked on saving my larynx by gargling with salt water and any remedy I could find online. I woke up at 4AM the day of the defense and tried to speak a word. Nothing but a squawk came out. I had lost my voice. Because it is really difficult to get committee members together, the show had to go on. So at 9AM I stood up in front of my committee and a few fellow graduate students and began to squawk my way through the well-rehearsed presentation. It was not fun to look at the audience trying not to look disturbed at the sound of my voice.

Finally, after about the fifth slide, one committee member stopped me and asked the rest if they could just go into the discussion and skip the presentation. Everyone agreed and I listened to 5 professors, all of whom I respect a great deal discuss the merits and faults of my research. It was really an enlightening experience because I cannot think of another time when I will get five really smart people in one room discussing my research to make it better. In the end I came out with some ideas to rework my plan. The committee agreed that I was trying to do too much and advised that I choose one of the two parts. The first part, understanding FLOSS usability in general through the survey, observations, and interviews is almost done, and I have learned a lot, but the second part, designing a tool for CodePlex to support usability activities is not only more interesting, but also part of the agreement between IST and Microsoft. I came up with a new direction based on more literature I have gathered. The first exciting addition is the use of a theory to guide the design and research. I will use activity theory because it can handle people, both from an individual and social level, and artifacts. It also considers context and the dynamics of activities. Other HCI theories, for example, distributed cognition, handle people at the individual and social levels, and artifacts, but does not specifically take into account context and dynamics of activities. I am also using a methodological approach called action research. Action research is a practical approach to research where solving problems leading to intervention is a collaborative act between researcher and practitioner. I am a practical kind of researcher so this approach suited the project and me best.

I will be working with Microsoft UX people and the CodePlex team to integrate usability support for the CodePlex community site. I will also be working with a few projects hosted on CodePlex to help with the design.

[PostIcon:4037]

10th European Computer-Supported Cooperative Work (ECSCW) Conference - Limerick, Ireland

Paula Bach — Wed, 07 Nov 2007 17:22:00 GMT

I’ve been on the road..

In September, I went to Limerick, Ireland for the 10th European Computer-Supported Cooperative Work (ECSCW) conference. Computer-supported cooperative work is a sub-sub-discipline of computerscience and a sub-discipline of Human Computer Interaction (HCI). CSCW researchers look at groupware, how people collaborate, and tools that support collaboration. They also do ethnographies to find out how people collaborate. Jonathan Grudin in MSR has researched extensively in CSCW and came up with an early idea called critical mass, which refers to the success of groupware. One of the tenets for groupware to succeed is for it to reach critical mass. This means that many people use the system. If only a few people use the groupware system, after a while, it will fail.

I participated in the doctoral colloquium. This venue is a common event at academic conferences where graduate students get informal feedback about their research from established researchers in the field. I received helpful feedback including advice about characterizing the HCI and OSS communities and finding out what the different members of OSS communities think usability is. The nice thing is that I have this information in the OSS survey data.

I had not been to Ireland before, and have not been in Europe since 1989. Limerick and Ireland in general are experiencing an economic boom, mostly because of the IT industry. Young folks have nice clothes, nice cars, and do lots of drinking. The conference dinner was held at at the Bunratty Castle. It was really nice and full of history.

I had an interesting conversation with Volker Wulf. He has written two books that are interesting for my work, but in the conversation I had with him, I was describing my research to him and he asked me about the tool that I will be designing for CodePlex. I mentioned that the project is sponsored by Microsoft and is a joint effort with CodePlex and MSR. He said, and I quote, “Micosoft doesn’t do open source.” And I emphatically replied, “yes they do!” I talked about CodePlex, Port25 and the open source website.

Now Europe knows.

Last week I was at the Free Open Source Software Symposium in Toronto, Canada. I like visiting the homeland, even though I am from the other side of Canada. The annual symposium is an effort put on by Seneca College School of Computer Studies in the greater Toronto area. The applied program offers courses in open source software development. The courses are in partnership with the Mozilla Foundation. Several Mozilla developers come to the college to talk about open source. Because of this partnership and because of their focus on open technologies, they have established a niche program dedicated to open systems. Bryan Kirschner (of Port25 fame), Mike Beltzner (of Firefox User Experience fame), Bob Young (of Red Hat fame), and many other key players in open source were there. I presented some findings from my open source survey and received good feedback. It was the first timeI had looked at the data in a while and there are some interesting things going on. One is that it appears that most of the people who responded to the survey call themselves usability advocates. It will be interesting to see how advocacy plays out in terms
of usability expertise.

I interviewed Mike Beltzner and a couple of people from the Fluid Project. I got some more names of people to interview and after I get my dissertation proposal written and defend it, I will go full steam ahead with interviewing. I can’t wait. I am in consultation with a couple of statisticians for ways to analyze the survey data from both the Microsoft and OSS surveys. I am working with some other graduate students to look at the role of usability expertise in Microsoft, using the data from the internal studies. This will be interesting to see the difference between the two software development environments with respect to the role of usability expertise.

Well that’s the news from Penn State.

Pondering the preliminary results of OSS usability research - Part 2

Paula Bach — Mon, 10 Sep 2007 18:59:00 GMT

Developing software has been an engineering discipline with formal methods. The evolution of software methods has ranged from the now outdated waterfall method to formal specification languages with precise semantics. Despite having methodologies, software engineering continues to be difficult. Yet despite having what seems a lack of software engineering methodology, open source software development can produce stable, useful software. In 2002, Fred Brooks gave a talk at CMU discussing the design of software. You may remember Fred Brooks from such publications as The Mythical Man Month and No Silver Bullet. In the talk at CMU, Brooks focuses on the social issues surrounding software engineering.

Likewise, Microsoft Research (MSR) recently initiated a new group called Human Interactions in Programming (HIP). This group studies the social aspects of software engineering. Their joke is that they “ build tools as if software were made by people … working together. This quote, taken from the technical report, outlines some social issues software developers experience in their day-to-day work.

Another way to look at my dissertation project is as an extension to looking at social issues in software development. While the HIP group at MSR studies human interactions among software developers, I extend that and study the human interactions among software teams (or project members): developers, project managers, and usability experts in both proprietary and open source software development environments.

As I mentioned in my last blog, I am studying the role of usability expertise in both software environments through surveys, interviews and observations. I have previously reported on the open source software survey and observations. In this blog, I am reporting on the interviews I conducted internally at Microsoft. I spoke to eleven employees who are working on various projects at MSFT in various roles including program managers, developers and user experience researchers and designers. I spoke to junior and senior staff and well as leads. Program managers are responsible for the feature that their team designs and builds. Developers write code. User experience designers create mockups and give feedback in design meetings and user experience researchers collect field data and conduct usability studies.

Travel Interlude

I wrote the above section before I left Redmond and now I am back on campus at Penn State. I had to hurry back and drove straight through from Redmond to Minneapolis. We left Redmond in the afternoon and stopped in Spokane for dinner and left at dusk. Spokane looks like it is growing and as such has some money injected into its economy. We drove through the night passing through Idaho and through Montana the next day. We passed through North Dakota late in the afternoon and stopped in Fargo for dinner. After dinner was the biggest rain storm I have ever seen – and I am from Vancouver, Canada where it rains a lot. I was driving southeast to Minneapolis on I-94 and slowed down to 10MPH because the rain was pelting down and blowing so hard across the road that it was like a whiteout. I could barely see five feet ahead. We made it to Minneapolis (avoiding the I-35W bridge area) at about 1AM and checked into our hotel. The next morning we awaked late, had breakfast, did some grocery shopping at a favorite natural food coop called the Wedge. I used to live in Minneapolis when I worked at Unisys and Promedicus--a startup that made decision support systems for physicians that died with many other dotcom startups. It was nice to be back to The Cities.

After our deserved travel break we went to Madison, WI to visit an old college buddy of my husband’s. They used to play in a band together. His buddy Frank still plays. We stayed there way longer than planned, but it was fun to catch up. We ended up reaching our next destination really late. We ended up in South Bend, Indiana (home of Notre Dame) when the sun started to rise and the birds began chirping. We were so tired the next day that we forgot things in the hotel, including a credit card! We did not notice it missing until we got to Toledo. Luckily it was the last leg of our trip and we made it back to State College, PA around 11PM and slept in the next morning ready to move back into our townhouse on campus. End of Travel Interlude.

Now that I am back on campus, I have had some time to reflect on the interviews. They uncovered a variety of interesting things. Overall the eleven people I interviewed were very enthusiastic about the research and most wanted to see results. Again, I have not analyzed anything formally yet, but all of the people I interviewed mentioned that communicating design changes was very challenging especially when it comes to usability issues. It seems like the biggest challenges relate to power relationships (not their words) among the team members and the ability of the person with usability expertise and training to gain trust with decision makers. A prevailing problem is that some people tend to think they are usability experts even when they are not trained and if they are more pushy or otherwise in a position to make the final decision, usability might be compromised.

Of course many other factors weigh into the usability of a product, but overall it seems that the usability experts are being heard one way or another. In comparison to usability in open source, a large proprietary software company has more resources for bringing usability expertise into products, but the social dynamics appear to be as complex as in open source. The only difference may be the characteristics of the dynamics. In my observations online of open source usability discussions, most of the interactions seemed to be devoid of such social dynamics, except for one group about one issue. So in comparison, open source might not have the same kinds of power relationships because the roles are not as differentiated. As I continue to investigate the characteristics of usability expertise I will see what open source interviews turn up. Stay tuned.

Pondering the preliminary results of OSS usability research, Part 1

Paula Bach — Wed, 01 Aug 2007 17:35:00 GMT

In my last blog I talked about interdisciplinarity and multidisciplinarity and a little bit about my research this summer. In part 1 and 2 of this blog I am going to talk more about the research I have been doing here at Microsoft. Over the last few months I have been looking at a phenomenon called usability expertise. Anybody who has had difficulty using a product has some experience with usability expertise. Usability expertise is knowledge about how to design an artifact to ensure users experience product effectiveness, efficiency, and satisfaction in a specified context of use. Even if people are not experts in Human Computer Interaction (HCI), they can experience a lack of usability expertise in the design of the product.

HCI experts are actually quite rare because the field is young and underdeveloped. The field of HCI is newer than computer science. HCI grew out of computer science about fifteen years after the software engineering crisis in the sixties and although Human Factors is about fifty years old, it has not necessarily been linked to software engineering like HCI has. Software development has included a user interface role to design and develop the human-computer interface, and although some companies still employ user interface developers, HCI experts include UI designers, Usability Engineers, User Experience Researchers and Designers, and Interaction Designers—roles that go beyond the interface and include field research, visual design, and lab studies, for example.

Although the obvious place to look for usability expertise is in the knowledge of HCI experts, I am interested in what role this expertise plays in software development. Just having HCI experts available is not enough to ensure good usability. I want to know who has usability expertise, how it is communicated among project members, and how it is used to make decisions. To find these things out the research looks at both proprietary and open source software development settings. What I am reporting here is an overview, or summary, of preliminary findings. I am still analyzing the data and will publish “official results” in the next year and a half while I work on and finish my dissertation. The research seeks to understand the role of usability expertise in software development and takes that understanding to inform the design of a feature or tool on CodePlex that will support usability expertise for projects interested in making sure their software is usable by their intended user base.

Usability expertise in the context of design is related to design rationale, or more specifically usability design rationale. Design rationale is the "the capture, representation, and use of reasons, justification, notation, methods, documents, and explanations involved in the design of an artifact" (from the book Design Rationale by Moran and Carroll). Since design rationale is a well defined concept that has many details, its presence in real design discussions may be fragmented. This fragmentation might be better understood as usability expertise. So a rough definition of usability expertise might be the “stuff” needed to talk about and make decisions about usability during software development. The “stuff” could be the elements in design rationale or something people have not talked before. In this sense my discoveries made while investigating the role of usability expertise could be groundbreaking or they could be well known in the software development communities. Either way reporting the findings of the role of usability expertise should be interesting. In fact, several people, both at Microsoft and the open source communities I surveyed have already stated that they would like to see the findings, so this is encouraging.

I am collecting data in a number of ways: surveys, interviews, and observations. I surveyed people at Microsoft who are part of the software development process of a project, namely usability experience researchers and designers, developers, and program managers. In the open source world I posted the survey to major projects who met criteria for overtly caring about usability, namely that they had a usability list and at least one person listed as a usability expert. The Microsoft usability expertise survey is still collecting responses, and although I am still working with the data on the open source survey, I can mention a few things.

In the open source survey, fatigue affected about half of the 125 respondents with 56 making it to the last question. The survey had two open ended questions asking about the importance and challenges of usability in open source. Usually open ended questions are best saved for the end after other more important questions are answered. The tradeoff was that the open ended questions were important and that the survey could have biased the open ended responses if they were at the end because the survey included questions that asked about specifics with the importance of and challenges with usability.

Data clustered around categories of ease of use, simplicity, and consistency for usability importance, with each category claiming about a quarter of the responses. About 10% of the respondents stated that issues related to system performance were important for usability. Usability challenges included about a quarter of the respondents reporting that challenges with usability in open source software development were developer based. This included not valuing usability, not having usability expertise other than self-referential (based on own experience), and communication problems related to common ground. Common ground is when two people reach a mutual understanding such that one person knows that the other person knows that the first person knows. Common ground is more difficult to reach in computer-mediated environments than in face-to-face environments because not as many channels exist to help with understanding—in face-to-face you can use people’s expressions and gestures to help you understand what they are saying. Other categories included lack of resources and lack of process (both at about 10% of the responses). Other questions I am asking the data include the following:

1. Who has usability expertise?

2. How is usability expertise communicated?

3. How is usability expertise used to make decisions?

4. Who cares about usability expertise?

5. How available is usability expertise?

The data may not be able to answer the above questions in full, but it will get me closer to asking different questions that may be more relevant to the data. I am conducting interviews which may also be able to address the questions and get at depth surveys cannot.

I have been scheduling and conducting interviews with Microsoft people and will report on those preliminary findings in the next blog. I will conduct the open source interviews via video conference when I get back to Penn State. The open source usability people I am going to talk to are all over the world: US, Canada, Germany, Australia, and France.

I have also been observing three open source projects looking at email lists and other interesting things like conversations in the bug tracker, how a usability issue is handled in the bug tracker, and reading UI specifications. I chose three ‘big’ open source projects that attend to usability. I wanted diversity in the projects and a wide user base. I spent 8 weeks observing the workings of usability in Firefox, KDE, and OpenOffice.org. The discussions on the email lists vary considerably. Some are short and polite with a developer inquiring about the usability of a particular design change or feature he is thinking about. Others are heated and get users, developers and usability people involved trying to hash out the merits of a feature.

The most often used design rationale, or type of usability expertise, is self-referential. The people on the lists, and in the beginning mostly users or user/developers respond to the feature proposal, speculate about the usability of the change based on their own experience. Since most of the users on the lists are advanced or power users, this might not be representative of the main user base, at least for the three projects I was studying. I don’t know if they have any data about the user base, but it may be that the email lists are only one input to the decision making about usability of those projects. Despite the openness of the discussion list and other aspects of the development, there are other decisions that are made ‘behind the scenes’. Possibly, the ‘behind the scenes’ usability expertise that contributes to decision making about which usability fixes to include in the next release is similar to how proprietary usability expertise is used in decision making. This is something I will consider when investigating the role of usability expertise in both environments.

Greetings from the Open Source Software Lab

Paula Bach — Wed, 13 Jun 2007 00:46:00 GMT

Bryan has previously blogged about the project partnership between the Penn State University (PSU) College of Information Sciences and Technology (IST) and the Open Source Software Lab (OSSL). I am at the OSSL here at Microsoft this summer and next as a research intern. The project, which started in May 2007 and will last two years, is my dissertation research. I work with Jack Carroll in the Center for HCI at Penn State. I am a third year PhD candidate and I study HCI in open source software development.

In this blog I want to talk about interdisciplinarity and multidisciplinarity. Broadly speaking, the information society is like the Wild West and many challenges as well as opportunities, especially with information technologies, have arisen. So for example, the Internet is like the Wild West of the information society. Challenges and opportunities in a new frontier are exciting for business and academia at once. Understanding the challenges and opportunities, however, needs new ways of investigating. A single discipline can address some of the challenges and opportunities, but complex problems, especially ones involving the intersection of information, people, and technology can benefit from expertise from multiple approaches. This is where a multidisciplinary or interdisciplinary approach can be helpful. Rogers et al (http://rizzo.media.unisi.it/page2/assets/Rogers_Scaife_Rizzo.pdf) make the distinction between interdisciplinary and multidisciplinary:

Interdisciplinary usually means “the emergence of insight and understanding of a problem domain through the integration or derivation of different concepts, methods, and epistemologies from different disciplines in a novel way.” Multidisciplinary can be characterized as “a group of researchers from different disciplines cooperate by working together on the same problem towards a common goal, but continue to do so using theories, tools, and methods from their own discipline, and occasionally using the output from each other’s work.” The characterizations differ in whether elements of a discipline are coupled or decoupled.

Although both terms have been used interchangeably, the subtle differences in problem solving depend both on the kind of problem a team of collaborators is solving and on the investigatory skills of the team members. The OSSL takes both approaches to both the challenges and opportunities inherent in understanding the open source and where Microsoft fits in. This broad approach is inherent when comparing Microsoft’s past and current missions: A computer on every desktop and in every home running Microsoft software compared to To enable people and businesses throughout the world to realize their full potential. The missions shifted from technology-centric to people and organization-centric. This new approach includes a global perspective on key aspects of the information society: people, information, and technology. This new approach is also exemplified by a new type of academic unit called information schools, or iSchools. The joint project, looking at HCI in open source software development, is interesting from a number of perspectives in the space of information, technology, and people. My approach is interdisciplinary, taking a number of concepts and methodologies and combining them in using different epistemological perspectives. Please contact me if you would like details on the interdisciplinary nature of the study of HCI expertise in open source software development—it would be too long to expound on here.

Bryan and I recently went to the iSchool at University of Washington to talk to graduate students and faculty about the project. The research conversation, as it is called, was well attended especially for a sunny Friday afternoon at the end of the spring semester. (The iSchool dean even showed up!) We talked about the challenges of studying the open source community and about doing interdisciplinary research in an iSchool.

The most interesting aspect of my experience so far as part of this joint partnership is that I am doing interdisciplinary academic work in a business unit studying open source software development at Microsoft – all of which are normally ”separate worlds” (academic/business and Microsoft/open source software). My summer here will entail collecting data and analyzing results of HCI expertise in open source software development as well as looking at HCI expertise in software development internally at Microsoft as a basis for comparison. In this summer series, look for my blog entries as I ponder results from the studies.