Social science and systems development

A couple of weeks ago, I wrote a blogpost on Agile and pseudo-science, which led to an interesting discussion on my Facebook wall about the lack of science and data in systems development.

As I wrote in my post, there is an unfortunately tendency of systems developers to rely on pseudo- or popular science, rather than proper science. A lot of decisions are made on assumptions for which there is no real evidence, but which somehow has become embedded in the world of systems development - be it project management, programming, testing, or some other aspect.

We tend to think of systems development as off shot of computer science with some cognitive psychology thrown in. While computer science and cognitive psychology certainly are important for creating systems, they are very much related to the "systems" part of systems development, while they hardly add to the processes, methods etc. that adds up to the "development" part.

A comment in the before-mentioned Facebook discussion suggested that we should look at social science when talking about science and systems development.

This seems like a good idea to me. A lot of what goes on in systems development is about humans interacting with each other, rather than about math, algorithms, cryptography or other aspects of computer science.

I once worked on a project where we had a horrible high turnover, which is definitely a well-known sign of a doomed team. Yet this team was one of the best teams you could imagine - continuously over-performing in the sense of delivering over scope, under budget and without flaws. From everything we know about team dynamics, this team should not have performed like this, but it did, and it could have been good to have had someone qualified there to analyse how this could be the case.

Equally it would be good to have someone look at the opposite type of teams - the ones which in theory should perform well, but which continuously under-perform. The ones where good developers suddenly seem unable to actually finish anything, where simple requirements turn into hideous bloated things, and where the only cure seems to be to kill of the team, never to let them work together again.

In both cases we know the anomalies relate to people, the interaction between them, and the culture of the team and the organization surrounding it. We just don't seem to be able to figure out how to create the first type of team, while avoiding the second type.

Given how many systems development projects fail, maybe it is worth looking at these things? And if we do, maybe we should try to find the proper tools?

Social science would seem like a good fit, and it would seem like a good idea for large organizations with many systems development projects, to take a look at how the research methods of social science could be applied to systems development, in order to become better at it.