Mutual Assessment in the Social Programmer Ecosystem

Developers use social media sites to communicate, collaborate, connect with each other, and even for competition. These sites and their users create a social programmer ecosystem with dynamics that are the subject of ongoing research.

Recently, websites have appeared that create profiles from developers’ content and activities on other sites — they are developer profile aggregators. Examples are Masterbranch and Coderwall. Among other mechanisms, both services award achievement badges to developers for specific accomplishments — such as being the most active committer in a project, or having a project forked by others.

With colleagues from Brazil and Canada, we studied users of these sites, providing us with a window into the social programmer ecosystem.

Update: our paper has been accepted for publication by CSCW 2013!

Our Study

We did a survey with members of Masterbranch and Coderwall, and conducted interviews with some of the survey respondents as well as some recruiters and hiring managers we found on our own. This gave us access to some interesting perspectives regarding the use of social media by software developers.

We found that developers are often assessing each other to evaluate whether other developers are interesting, worth following, or worth collaborating with. To get a feel what another developer is about, they look at what an interviewee called one’s coder footprint: a combination of many hard and soft signals, including the developer’s technical niche and its popularity in the community; their diversity; their passion for technology; their standing in the community. Grasping a developer’s coder footprint seems to be greatly helped by the developer profile aggregators.

On the other hand, developers are self-conscious about being assessed by others, and thus manage their public images. They value passion for software development, new technologies, and learning — and they know that other developer have similar priorities. Therefore, they try to show others just how passionate, how diverse they are.

This helps forging new relationships, weak and strong, between developers — and such ties provide inspiration:

On Twitter, I follow a few prominent software developers. For example, Kelly Sommers from Canada, she’s constantly trying new things. I don’t think she ever sleeps. She’s a great source of inspiration. [D11]

Some recruiters participate in the ecosystem and use it to find candidates for hiring. Often, these have a technical background. This a quote from a developer who is building his own small in team inside of a startup, and is therefore involved in hiring:

I used to look for that manually in GitHub repositories, but Coderwall and others make it easier now [D4]

Other recruiters struggle with the interpretation of signals and issues of trust. A hiring manager who had no technical background told us:

Well, I don’t know all these achievements. They look fun! But they don’t really help me. [R12]

This mutual assessment is changing how software engineers collaborate and how they advance their skills. Two developers told us how they had found work, helped by their Masterbranch profile. A team coordinator told us that their company-internal dashboard constantly displays the Coderwall leaderboard to motivate (and amuse) the team. Yet, this is just a small glimpse — in our findings, we discuss reasons, strategies, impacts, challenges, and risks of participation in the social programmer ecosystem.

Our exploratory study revealed a rich ecosystem of passionate, innovative software developers and many interesting dynamics. Several of the effects we found warrant future research. For example, using social media and gamification mechanisms to motivate software developers is a very current topic that might have great benefits for developers and companies. The data that is publicly accessible about the interactions of software developers enables further research into how and why the social programmer ecosystem works. Finally, our study revealed several potentially problematic issues — problems of trust especially for non-technical recruiters, stress in developers that always chase after the latest technologies, or the isolation of actors who cannot keep up. We plan to investigate these aspects in future work.

Read our Report

We provide a preprint of our paper: “Mutual Assessment in the Social Programmer Ecosystem: An Empirical Investigation of Developer Profile Aggregators”. Leif Singer, Fernando Figueira Filho, Brendan Cleary, Christoph Treude, Margaret-Anne Storey, Kurt Schneider.

Epilogue

This research was a collaborative effort of researchers from three different universities: Hannover in Germany, Natal in Brazil, and Victoria in Canada. The fact that the work day started in Victoria when mine in Germany came to its end was highly useful: my colleagues would be busy at work while I was sleeping, and I could build on their results when I received them in the morning.

Of course, the geographical distribution also brought its challenges. These are the tools we used to overcome them:

Google Forms for conducting our survey
Skype for voice and video calls (daily, at times) and for conducting the interviews
Dropbox for exchanging files and having a shared database of literature
Google Docs for collaboratively sketching early versions of the text
LaTeX for writing it all up cleanly, hosted in a private git repository on bitbucket
E-mail — I cannot overstate how important it is for communication and collaboration over distance
airplanes — I spent three weeks in Victoria, living with a pit bull called Diesel and a cat named Nixon, getting to know my remote colleagues (not associated with said pets)

And finally, I worked with a great team — a big thank you to all of you!