Social science and systems development

A couple of weeks ago, I wrote a blogpost on Agile and pseudo-science, which led to an interesting discussion on my Facebook wall about the lack of science and data in systems development.

As I wrote in my post, there is an unfortunately tendency of systems developers to rely on pseudo- or popular science, rather than proper science. A lot of decisions are made on assumptions for which there is no real evidence, but which somehow has become embedded in the world of systems development - be it project management, programming, testing, or some other aspect.

We tend to think of systems development as off shot of computer science with some cognitive psychology thrown in. While computer science and cognitive psychology certainly are important for creating systems, they are very much related to the "systems" part of systems development, while they hardly add to the processes, methods etc. that adds up to the "development" part.

A comment in the before-mentioned Facebook discussion suggested that we should look at social science when talking about science and systems development.

This seems like a good idea to me. A lot of what goes on in systems development is about humans interacting with each other, rather than about math, algorithms, cryptography or other aspects of computer science.

I once worked on a project where we had a horrible high turnover, which is definitely a well-known sign of a doomed team. Yet this team was one of the best teams you could imagine - continuously over-performing in the sense of delivering over scope, under budget and without flaws. From everything we know about team dynamics, this team should not have performed like this, but it did, and it could have been good to have had someone qualified there to analyse how this could be the case.

Equally it would be good to have someone look at the opposite type of teams - the ones which in theory should perform well, but which continuously under-perform. The ones where good developers suddenly seem unable to actually finish anything, where simple requirements turn into hideous bloated things, and where the only cure seems to be to kill of the team, never to let them work together again.

In both cases we know the anomalies relate to people, the interaction between them, and the culture of the team and the organization surrounding it. We just don't seem to be able to figure out how to create the first type of team, while avoiding the second type.

Given how many systems development projects fail, maybe it is worth looking at these things? And if we do, maybe we should try to find the proper tools?

Social science would seem like a good fit, and it would seem like a good idea for large organizations with many systems development projects, to take a look at how the research methods of social science could be applied to systems development, in order to become better at it.

What sounds interesting at GOTO Copenhagen 2015?

The GOTO Copenhagen conference is still a couple of months away, but I have taken a look at the talks, and spotted a few that I think will be interesting.

Generally speaking, I don't go for technology specific talks - by which I mean that I tend to go for talks about processes (e.g. Agile), concepts (e.g. Big Data) or general architecture, rather than talks about specific languages, or even worse aspects of languages, or about specific programs (e.g. a specific NoSQL database). I don't mind examples at a talk being language specific, but I prefer to be able to apply the concepts from the talk broadly.

Having said that, I am as prone to follow hype as everybody else, so if something new and exciting is presented, I might very well go there, even if it breaks my general preferences for talks.

When you see my list, you'll notice that I haven't filled my schedule. This is quite common for me - I rarely know what I want to see at a conference when I begin my day. Instead I like to hear the talks getting presented in the morning, and then decide what to go to.

If you tend to skip those talk presentations, I think you're really missing out. Some of the most exciting talks I've listened to was not remotely on my schedule until I heard them presented at the start of the day.

So... onwards to my list.

The first talk I noticed was Richard Lander's How to train your corporation to prefer open source.
Microsoft has gone through an incredible change in regards to their stance on Open Source in the last few years, to the degree where now they have Open Sourced most of the .NET platform. This is hardly something one would have guessed a few years ago (even though Microsoft has never been as anti-OS as some people think it has), so I definitely want to hear how this happened.

I don't usually go to IOS-related talks, but I might just go to Jorge D. Ortiz Fuentes' Test Driven Development (by controlling dependencies).
There is no description of the talk, but I am always interested in seeing how people handle dependencies, as I think this is an overlooked problem-area in my software development Projects. Given it is in the IOS and Swift track though, there is a high likelihood of me giving the talk a pass.

Other than the mentioned talks, I will probably spend the rest of the first day listening to talks in either the Game Changing Methods and Practices track or the Fast and Continous Delivery track.

On the second, and final, day of the conference, there are four tracks that all sounds interesting - Reactive Architectures, The State of Data, Front-end: The Bleeding Edge, and Security, Safety and Privacy.

In the first track, Reactive Architectures, I am probably going to Jonas Bonér's The Sadness at the End of the Happy Path and Dave Farley's Reactive Systems: 21st Architecture for 21st Century Systems.
Building resilient and high-performance systems is hard to do, and any ideas on how to do it better, are definitely welcome.

If I am not going to Dave Farley's talk, I will certainly be going to Dave Thomas' The State of Data 3.
I have seen Dave Thomas talk a number of times, and his talks are often quite amusing, but also, more importantly, highly informative. Big data is his field, so I expect the talk to be as great as always.

A talk that sounds interesting, is Phil Winder's Modern Fraud Prevention using Deep Learning, which is at the same time slot as  Andreas Halberg's Secure Coding Patterns - another talk that sounds useful for when developing systems.

As always, when at multi-track conferences, there are often several talks I find interesting at the same time, and which I won't choose between until the last minute. Again, much depends on how the talks are presented, and whether I have heard the speaker before. Usually there are a few speakers that are "must-go-to" for me at the GOTO conferences, but this doesn't seem to be the case at this conference. This is not a bad thing - it gives me the opportunity to get to know new great speakers.

Disclaimer: As a blogger who blogs from the conference, I get a free ticket (like I have done for the last couple of years). The deal comes with no strings attached, except for an agreement on me writing a certain number of blogposts about the conference. The conference has no say over the content of the blogpost.

Agile and pseudo-science

Back before I started blogging about IT-related stuff, I used to write about pseudo-science on my other (very quiet) blog Pro-science, and before I had that blog, I used to comment a lot on a number of science and skeptic blogs.

First commenting and later blogging about science and pseudo-science has helped enable me to be better at detecting woo and pseudo-science in my daily life. Woo is a term used in the skeptic community to indicate something which is based on anti-science and have no scientific validity or data to back it up.

Woo and pseudo-science exists everywhere, and I have found that the field of IT is absolutely no exception. Rather, there are certain areas which is rampant with pseudo-science, often bordering on woo.

One such area is the field of Agile.

The field of Agile is not a well-defined field, but is a loosely collection of methodologies, techniques, and tools, used for systems development. It is commonly understood as having formed as a result of the agile manifesto, but in reality, it started developing before then, and would probably have come into existence as a field, even if the manifesto hadn't been written.

As a software developer, I use Agile methodologies, techniques, and tools on a daily basis. I also go to conferences and meetups about Agile frequently.

Due to that, I can't help noticing that there is not a whole lot of science behind Agile. Not only that, many of the arguments people use to promote Agile is not exactly based in science or data, but rather on pseudo-science, often in the form of popular science.

This summer, I went to a conference which had a talk on Agile and NLP. I didn't go to the talk, so I can't say anything about the actual content, but let's be perfectly clear: NLP is pseudoscience (.pdf) - see also Thirty-Five Years of Research on Neuro-Linguistic Programming. NLP Research Data Base. State of the Art or Pseudoscientific Decoration? by Tomasz Witkowski (.pdf).

If you do a search on the words "Agile" and "NLP", you'll find articles like NLP Patterns and Practices for High-Performance Teams and Achievers by J.D. Meier. If you look at the blogpost, and are in any way familiar with the writing of people pushing pseudo-science, you'll notice the patterns there. There are appeals to effectiveness and popularity, but no actual data to back those claims up. 

It is highly likely that Meier actually believes that NLP helps, but there is no science to back it up - neither in the form of an explanation of how the NLP mechanisms works or in the form of data, backing up the claims of improvement.

This is, unfortunately, widespread in the field, and not just when it relates to NLP.

Agile (as all things related to systems development) is very much dependent on people, and since most realize this, there is a lot of focus on this aspect. 

This results in many speakers on the subject on Agile, will pepper their talk with references to neuroscience, but rather than looking at the actual science papers, they will get their knowledge from popular science books, by people like Malcolm Gladwell.

In 2012, Steven Poole wrote an article for The New Stateman on the subject of popular science books on neuroscience, called Your brain on pseudoscience: the rise of popular neurobollocks. While he, in many peoples' opinion, painted with too broad a brush, there was an important message in the article, which we'll do well remembering - popular science books often have an agenda, and they are simplistic, and, in many cases, get the science completely wrong.

E.g., as a rule, one should expect anything written by Malcolm Gladwell to be wrong.

I understand why speakers and authors tend to use popular science books as sources, rather than the real science articles. With popular science books, there is a good chance that the audience has already read them, and the science is presented in a form which is easy to digest and regurgitate.

Just a pity that the science is often presented simplistic at best and wrong at worst.

Some speakers, such as Linda Rising and Kevlin Henney, instead read the original science papers, and refer to them instead. Given that those two speakers are among my favorite speakers, I hardly think it takes anything away from their talk. It just makes them more accurate.

Having addressed the use of pseudo-science and popular science, I think it is also important to address the fact of data on which to evaluate things.

This is obviously connected to the other problems. Lack of data, leads to bad conclusions.

A simple example of the lack of data, is this: How do we know Agile works? 

Or put in another way: On what basis do we conclude that Agile works better than what was there before?

As far as I've been able to find out, there is no real data to support such claims.

Yes, there have been studies that indicate that Agile works better, but those are somewhat doubtful, since there was no baseline to compare to, no clear definition of what went before Agile, or even what Agile is.

People tend to compare Agile to Waterfall, but as Laurent Bossavit wrote in his excellent book The Leprechauns of Software Engineering, there is no clear evidence that Waterfall was ever used - at least not in the form that people commonly think of when talking about the Waterfall method. 

Personally, I believe there is great value in using Agile, but I am not willing to make any claims without data to back me up, and in order to have that, we need to try to not base the field on pseudo-science and misrepresentations in popular science, and instead try to use real science, and to collect and evaluate data.

Not everything fit into the scientific method, and in a field that is so heavily dependent on people, there will be times where personal preferences will lead the way, but this doesn't mean that we can't do better.

Think of it this way: We would not accept a new UX design without an AB test or some other empirical data to back it up. So why would we accept it when it comes to our methodologies, techniques, and tools?