Not-done-here syndrome is mental laziness

The provocative title of this post, is one of the many great points made by Jessica Kerr at the morning keynote at today's GOTO Copenhagen. The keynote was called "Forget Velocity, Let's Talk Acceleration"

Jessica Kerr gave an energetic talk, which covered a lot of ground - enough for several blogposts, which I hopefully will get around to write in the next few days. This blogpost, however, will focus on the point expressed in the title.

Most of us have come across developers, some of them quite good, who suffered from the not-invented-here syndrome, i.e. they'll rather spent the time it takes to write the code themselves, instead of using existing code written elsewhere.

There are probably cases where it makes sense to write it yourself, rather than reuse or modify code made elsewhere, but it is rarely the case. Usually, it takes a lot of time and effort, and it ends up being not as good as the code you could have gotten elsewhere.

I've seen it happen from ORM over logging frameworks to even an entire Ajax framework.

I have always assumed that this happened because of the arrogance of the developers in question, who thought they were better than everyone else.

Not so, says Jessica Kerr - it is due to mental laziness.

This is based on the principle that programmers find it easy to invent new things, but hard to analyze existing code. This means that it is mentally easier to write a framework yourself, rather than take in a framework written by someone else, and get to understand it.

Or put differently, the mental energy you have to spent on writing the framework yourself is less than the mental energy you have to spent on learning a new framework.

Thus, people who suffer from a not-invented-here syndrome, are mentally lazy.

This resonates with me.

One of the followup points that Jessica Kerr made, was that if you look at something, and think that you should rewrite it, then you should stop up, and make sure that it isn't just mental laziness that is speaking.

It is often necessary to rewrite existing code, but rarely to the extend that developers and architects wants to do so. Rewriting the code, however, spares the developers and architects for the work they need to put into analyzing and understand the current code beyond the basic level.

So, in conclusion, rewrite code if necessary, but make sure it is really necessary, and not just done to avoid the hard work of analyzing the existing code.

Disclosure: This blogpost mentions the GOTO Copenhagen 2017 conference. As a blogger who blogs about that conference, I get a free ticket from the organizers. The organizers don't dictate what I write about, and don't have any say about the content of the posts.  

Progress is slow if you ignore the past

I have just returned from spending my Sunday at the 1st day of the 21st Danish GOTO conference (not all of them were named GOTO, there has been some name changes over the years).

As expected, it was a great day, with some great talks - if you want to see my running commentary on those talks, go look at my Twitter stream @kriswager

Most of the talks I attended were great, but I especially enjoyed the keynote Engineering You by Martin Thompson, and Dan North's talk Beyond Developer, both of which focused on how software engineers/developers should expand their knowledge and become better programmers.

Both of these talks started out by looking back at the roots of the field. Dan North took us all the way back to Ada Lovelace:

Taking us back to the roots. @tastapod starts his talk with going back to Ada Lovelace #Gotocph pic.twitter.com/exmt9OcroE

— Kristjan Wager (@kriswager) October 1, 2017

Martin Thompson, on the other hand, only took us back to 1967, where "software engineer" was coined. He spent some time on the 1968 Software Engineering conference sponsored by the NATO Science Committee, where many of the issues raised, and experiences expressed, would fit well into a conference today.

It is interesting how little we have learned in 50 years. Words on slide is from meeting in 1968 #Gotocph pic.twitter.com/XV1AxclpmQ

— Kristjan Wager (@kriswager) October 1, 2017

The above tweet only shows one of Martin Thompson's examples from the conference.

While it is easy to despair over how little progress that has been over the last 50 years, I think it is probably better to think about it this way: while our methodologies change, the problems we face are the same, and it makes sense to look at what people did in the past, to see if they have solved problems we face today.

A lot of people tend to think of pre-Agile days as a period of endless, mostly failing, waterfall projects, but the truth is, that this generally wasn't the case in the early days, and probably wasn't as common later, as legend makes it sound.

It is worth remembering that back in the early days of the field, people were not only building programs, they were also building the tools (e.g. compilers) they needed in order to build programs. These tools have evolved since then, but they are still based on the same solutions that we base our tools on today. Why should solutions to organizational problems be any different?

Disclosure: This blogpost mentions the GOTO Copenhagen 2017 conference. As a blogger who blogs about that conference, I get a free ticket from the organizers. The organizers don't dictate what I write about, and don't have any say about the content of the posts. 

My schedule at GOTO Copenhagen

GOTO Copenhagen 2017 is just around the corner, and I have taken a look at the talks that I definitely want to go to this year.

Unlike earlier years, GOTO Copenhagen starts on a Sunday this year, which makes some demands to the quality of the speakers, trying to lure people in early Sunday morning.

Luckily, the conference delivers - there are two keynotes on at 9:00, and while both looks interesting, I am going for Martin Thompson's talk. I have heard him speak a number of times, and every time the talk have been engaging and informative.

After the keynote, there are some interesting talks, but the next talk I really look forward to, is Dan North's Beyond Developer talk. Not only does the talk sound interesting and relevant, I also know that anything Dan North presents is going to be fun.

Skipping to the talks on Monday, I have spotted the following talks I'd be interested in:

• Eoin Woods: Secure by Design – the Architect’s Guide to Security Design Principles 
• Emma Arfelt: Privacy in Software

The many security breaches in recent months, shows that security is still a hugely important topic, which is all too often ignored or neglected. These two talks addresses different, but equally important, aspects of security.

Tuesday, there is a track called Herding Cats: The Human Factor, where I plan to spend most of my time, for reasons I explained in my last blog post

Other than that track, I am looking forward to the end keynote by Linda Rising. Linda Rising is one of the most engaging and brilliant speakers I have ever heard, and it is a pleasure to go to one of her talks.

Disclosure: This blogpost mentions the GOTO Copenhagen 2017 conference. As a blogger who blogs about that conference, I get a free ticket from the organizers. The organizers don't dictate what I write about, and don't have any say about the content of the posts.

The importance of peoples tracks at IT conferences

Let's talk about the importance of peoples tracks at IT conferences.

Some IT conferences only focuses on technical issues, but some conferences, such as the QCon conferences and the GOTO conferences always include a peoples track.

I generally go to these tracks if I have the chance, since I think they are important.

Last month, James Damore published his now infamous open letter, where he wrote a screed against diversity in Google and the tech sector as a whole. The screed was not only incredibly scientific illiterate, but also clearly demonstrated the need for people tracks at conferences.


In the upcoming GOTO Copenhagen 2017 conference, there is a people track that illustrates why I think they are important.

The track contains five talks:

• The Engineering-Manager Transition: How to take great engineers and make them great technical leaders
• Stress and depression – a taboo in our time
• Scaling Engineering Teams
• The 2D Kitten Problem
• Build the right thing: how to survive the accelerating rate of business change through experimentation

While these talks are all about people, they are quite diverse, allowing the listeners to learn from the experiences of others, and to see new perspectives.

In the above list of talks, it is especially the second and the fourth talk that I think makes the track worth my time.

The second talk,  Stress and depression – a taboo in our time by Gitte Klitgaard, raises a subject which is all too rarely discussed, and which most of us are affected by - either directly or indirectly.

The fourth talk, The 2D Kitten Problem by Laura Laugwitz, is described thus:

"Diversity" is one of those buzzwords that alternates between being supercharged and sadly hollow. While many tech companies like to boast about their diversity programs, the numbers regarding their employees don't change much. But that's no reason to give up on the concept of diversity just yet! However, we need to re-examine what actually constitutes this term in order to make it sustainable: Diversity is more than hiring a few "different" people, it's about empowering oneself and others to create positive change.

This talk will give back some meaning to the term diversity while illustrating why it's still important. There will be some interaction, some tangible examples and more than a few cats to help you through the more challenging parts of theories and self-reflection.

While it is highly unlike that James Damore would ever go to such a talk, it is talks like this that would have allowed Damore to understand the importance of diversity.

It is not possible to reach Damore and his irk, but there are other people out there who are not aware of the importance of diversity, who can be reached through such talks. This is why I think people talks are important, and why I hope conferences keep having them.

Disclosure: This blogpost mentions the GOTO Copenhagen 2017 conference. As a blogger who blogs about that conference, I get a free ticket from the organizers. The organizers don't dictate what I write about, and don't have any say about the content of the posts.

Tearing down the walls

It is pretty widely accepted that one of the major challenges for IT projects is communications between the different parties involved.

The above figure demonstrates how a traditional IT project works, and why communication is such a big issue.

The business have some wishes, which they give to the requirement people, who turns them into requirements, which are handed over to the development team. The development team then develops the system, and hands it over to testing. When the system is tested, it is handed over to operations, which puts it into production.

Such a process is characterized by each group being walled off from each other, often in a real physical sense. It is also characterized by a lack of communications between the groups.

This means that it can take a long time to find misunderstandings, and that the requirements tend to become outdated before they reach production.

SCRUM and other Agile methods have tried to address this in different ways, but even so, communications problems still exists.

The above figure illustrates an IT project using SCRUM. Here the developers and testers form one team, and the requirements specification is handled by a PO who represents the business.

Due to the processes in SCRUM, the communication barrier between the PO and the SCRUM team is reduced, but there is still a barrier. Also, in most organizations, the barrier between the business and the PO is unfortunately as great as the barrier between business and requirements people in traditional projects.

There is also a communication barrier between the SCRUM team and operations.

The fairly recent concept of DevOps have tried to address the communication barrier between the SCRUM team and the operations people.

As the above figure illustrates, this is handled by embedding the operations people in the SCRUM team (this is obviously a simplified description).

Last November I was at a conference in Orlando, where one of the speakers were the PO for the SCRUM process, and he told us that there is a focus on adding DevOps to the SCRUM methodology.

Unfortunately, he also said that they are not going to do anything about adding requirements specification to the methodology.

This is, in my opinion, unfortunately, as it would help reduce the barrier between the business and the rest of the project, and between the PO and the SCRUM team.

So, what can be done instead, in order to remove the barrier between the business and the rest of the project?

Well, I think it will be hard to completely remove the barrier, but I think there are different ways of reducing the communication barrier.

These are, in random order:

• Embedding a business person in the SCRUM team
• Regular meetings
• Techniques such as story mapping

These can obviously be combined.

Embedding a business person in the SCRUM team

This is an idea that was introduced with eXtreme Programming, though it probably existed before that as well.

It is simply getting someone from the business sitting together with the SCRUM team.

This is often hard to do, as it can be hard to get people released from their normal position in order to place them in an IT project, even if they continue to work on what they usually work with. This is partly due to the, probably reasonable, concern that the person will have much less time to do their job. But it is also often partly due to the fact that many bosses don't like not having people sitting apart from their usual team. On top of that, it might be hard for the business to see the value of embedding the business person in the SCRUM team.

If you manage to get a business person embedded with the SCRUM team, it is worth regularly evaluating whether the business person is the right person for the role, or whether there should be a rotation of the business people, in order to keep the focus fresh.

Regular meetings

A simple, but effective way of enabling communication between the business and the rest of the project, is to plan frequent meetings.

The sprint reviews in the SCRUM methodology is one type of frequent meetings, but there should be other meetings, such as requirements meetings between the PO and the business.

These meetings can be used for presenting user stories before they are taking into a sprint, in order to get the input from the business, and clarify any unclear requirements. It can also be used to prioritize the requirements, based on the current situation, and to adjust the scope of the project, based on any developments.

Techniques such as story mapping

Requirements in SCRUM projects are often written as user stories, and there are some techniques that can be used while writing these, allowing the business to get involved.

One of these methods is user story mapping.

No matter what technique is used, it is important that to ensure that the business are willing to get fully involved. If they are not committed, no technique will help reduce the communication barrier.

In conclusion, it is unlikely that it is completely possible to reduce the communication barriers completely in an IT project, but there are methodologies, concepts, and techniques that can greatly reduce them, allowing for a greater likelihood of the successful development of a system which fulfills the needs of the business.

One aspect I haven't touched on in this post, is the extra complications if the project isn't aimed at the internal business, but rather at customers outside the organization. External customers obviously adds an entirely extra complexity to communications, and is a subject for an entirely different blogpost.

Impressions from the first day of GOTO Copenhagen 2016

I have just returned home after spending the whole day at GOTO Copenhagen 2016, listening to great speakers talking about software development.

As I wrote in my last post on GOTO Copenhagen, I had planned to spend the day watching the Effective Delivery track, and this is what I did.

For someone like me, who works with projects in the public sector, this was a fantastic track.

The first talk was Tony Grout's Hand to hand with a gorilla won't work, about how you introduce agile in a very large organization.

Basically, it takes a lot of effort, and some cunning, and it won't be pure, but as Grout said, "good luck with pure". Sometimes you have to be pragmatic in order to reach your goal.

One interesting thing that Grout said, was that a fairly simple, yet very effective tool, is an ordered list across the organization, allowing everyone to know what they should focus on first. Introducing such a list in Skype, increased the productivity ten-fold.

Next up was Jez Humble's When DevOps Meets Regulations: Integrating 'Continuous' with 'Government'

This talk was about the efforts of introducing DevOps in government projects in the US. A big barrier to this are the regulations they have to follow when making "information systems".

A partial list of regulations that @jezhumble have to remember when launching government system in the US. Seems similar to Denmark #gotocph pic.twitter.com/DZxepcDubP

— Kristjan Wager (@kriswager) October 3, 2016

As I write in my tweet, the list doesn't seem overly long to me, compared to what we experience in Denmark. As the talk progressed, however, I realized that the US process for fulfillment is much more cumbersome than the Danish one, and it definitely needs an overhaul.

One cool thing that Jez Humble introduced us for, is an open source cloud platform for government projects in the US. This seems like a great idea, and I'd love to see something similar in Denmark.

After the US, the turn came to the UK, in the form of Stephen Forshew-Cain's Building an effective delivery culture, where he talked about the Digital Service and its culture.

I am planning on writing a longer blogpost about Jez Humble's and Stephen Forshew-Cain's talks and how they relate to the Danish situation, so I won't go further into the talk here.

Having spent some time on government projects, the time came for talking about effective teams - this came in the form of Camille Fournier's Building a High-Performance Team is Everyone's Job. This was the title in the program, but she had given it a different name at the time of the talk (the title is unfortunately on a photo in my phone which has run out of power).

I had never heard Camille Fournier speak before, but I will most certainly do so again, if I ever get the chance.

It was a very amusing, and highly informative talk, where she spoke about her experiences, and what she had learned from them, when it came to leadership.

"I came from finance where it was OK to yell at people. Apparently that's not the case everywhere" @skamille on early mistakes #gotocph

— Kristjan Wager (@kriswager) October 3, 2016

last, but not least, on the track, Emily Webber gave her talk Communities of practice, the missing piece of your agile organisation, in which she talked about how to create communities of practice inside your organisation.

All of the talks were great, and I certainly got a lot out of them. Hopefully the same will be the case tomorrow, when I spend my time at the Tactics for better Teams track.

It is time for Danish politicians to form an opinion on IT projects in the public sector

As in most other countries, public IT projects have a bad reputation in Denmark. It seems that most people think that the public sector is horrible at doing IT projects (much worse than the private sector), and that the majority of public sector IT projects fail.

This perception is caused by a number of very big IT projects failing, causing not only the loss of the money put into them, but also the loss of state revenue, due to necessary IT infrastructure not being in place in order to e.g. ensure that tax refunds are paid correctly.

In my opinion, the public perception is wrong on two areas:

1. The public sector is no worse than the private sector at doing IT projects.
2. The majority of public sector IT projects go well.

The reason for both of these misperceptions is that people don't hear about projects that fail in the private sector, nor about IT projects in the public sector that does well.

But even with this caveat, there is no doubt that there is uncomfortable large number of IT projects that fail in the public sector, having a negative impact on the Danish economy.

Given this, I think it is remarkable how little the Danish political parties seem to care about IT projects in the public sector.

The politicians generally seem to care only on the most superficial level, saying that the public sector need to get better at doing IT. If they go beyond that, they generally focus on ensuring that laws are not getting in the way of implementing new IT systems and work rutines.

I think it is great that they are willing to take a look at how the laws in a given area might get in the way of eg. digitalization, but this is not enough. The head of the Danish Business Authority, which has just gone through a fairly succesful five year modernization program, has stated that it is frequently not laws that gets in the way, just interpretations and existing workflows.

It is often not clear that the interpretations and workflows will get in the way of development of the new systems, before the actual development work starts.

So, in other words, it is not enough that politicians look at the laws, they also have look at fostering an environment, where it is not only OK, but pretty much required, for public servants to reevaluate interpretations and workflows. For this to be possible, it is necessary to get them directly involved in the IT projects, not only as end-users, but also as sparring partners for the developers. This requires that resources are put aside for this.

Another area where I'd like politicians to focus, is creating an environment where different ministries, departments, and agencies learn from one another. There are some great attempts at this on the local level (eg. some departments look around at other departments to see what works before they start up), but it has to be structured better, and the responsibility anchored in one place.

There are a lot of experience spread across the different ministries, departments and agencies, but all too often it cannot be easily located, and each IT project is cause of re-learning everything from scratch.

A good attempt of drawing on the experiences of others, is the use of IT-projektrådet to evaluate the risks of IT projects larger than 10 million kroner and to follow up on high risk projects.

This is only done to federal projects however, and not on projects on the municipality level or regional level, even when they are much larger than 10 million kroner.

Obviously there are other areas where politicians could, and should, form an opinion on IT projects in the public sector, but I think that the areas mentioned above are some of the more important ones.

What I want to see at GOTO Copenhagen 2016

I have taken a look at the schedule of GOTO Copenhagen 2016, trying to figure out what I want to see.

On Monday the 3rd, there are a few interesting tracks for me.

First of all, the description of the Effective Delivery track sounds very intriguing:

Anyone with less than 15 years in IT has never known a time when there wasn’t Scrum or XP, or cross-functional teams or automated tests. They take these things for granted, but they also believe this is the state-of-the-art. Mainstream agile methods are around 20 years old now, and the world has moved on. Planning every 2 weeks makes sense if you have a 3 month release window, but what about if you are releasing every week, or even several times a day? Servers are now an inexpensive commodity, and cloud infrastructure means my expensive data center is now someone else’s pay-per-use model. How are modern practitioners taking advantage of this? What about challenges like large-scale delivery, regulated government departments, or heavyweight corporates? And how can you navigate a career when half the companies on your CV no longer exist? This track passes a critical eye over the current delivery landscape, and hopefully gives you some tools and techniques for navigating the modern world of Effective Delivery.

I like that the track takes for granted that we understand the basics of Agile development, as most of us have worked with it our whole career, and build upon this.

It is highly likely that I will spend most of the day at this track, and there are a couple of talks I definitely won't want to miss. One of these is Jez Humble's When DevOps Meets Regulation: Integrating 'Continuous' with 'Government'. As someone that works primarily with projects related to the public sector, this certainly sounds like something that I can use in my daily work. The same is probably true for Stephen Foreshew-Cain's talk Shepherding Government towards Effective Delivery, but there is no description of this talk yet.

If I am not at the Effective Delivery track, I'll probably be at the Deep Learning Analytics track, where there are several interesting talks, including Michael Hunger's How the Investigative Journalists of the ICIJ used modern Open Source Technologies to unearth the stories of the Panama Papers, where Neo4j will play a major part of the talk. Neo4j is a technology I have followed for years, and I would love to hear about how it was utilized by news organizations, so they could map the data.

On Tuesday the 4th, there is no doubt that I will be spending my day at the Tactics for better Teams track. The track is described thus:

Let's get practical. Often small simple changes have a huge positive effect on team performance and work-life satisfaction. This track presents a set of simple tactics that you can bring home to your team and start using today to make a better team. The track will present both technical ideas like software visualisation as well as more process oriented initiatives.  

The reason I'll be spending my day at this track, is partly that it is the track that is most related to my daily work, but mostly because of the speakers on the track. Of the speakers, the only one I don't know if I have seen before is Corey Haines. The rest of the speakers of the tracks are people that I don't only know I have seen before, but also know that I love - especially Linda Rising, who is a brilliant speaker.

Hadn't that been the case, I'd probably have spent some time on the Microservices track, trying to figure out what all the hype is about.

 

The Dangers of Conferences

In just under a month, it is time for GOTO Copenhagen 2016, which is the 20th anniversary of the GOTO conferences (which started out as JAOO). I am looking forward to listening to some great speakers and network with other participants and the vendors.

When talking with other people in the field of IT, I often come across people whose employers won't pay for conferences, and sometimes even won't let them take time of to go there.

I have in the past tried to make the case for allowing employees to go to conferences, but I think it is time to also acknowledge and address the dangers of allowing employees to go to conferences.

The rest of this post will list the major explanations I've heard, and try to address them.

1. The company only work with legacy systems, and all the new stuff mentioned at the conferences is irrelevant for us.

A conference like GOTO doesn't only present new languages and technology, but also have sessions on methods, architecture and many other things that are several layers of abstraction away from actually systems development.

Even if your company doesn't make use of the newest technology, conferences present a rich source for re-evaluating current practices, and address inefficiencies in the current setup. Many, if not all, of the speakers have not only worked on cutting edge greenfield projects, but also on brown-field projects, that have turned into big balls of mud, so they understand the need for taking such projects into account.

2. When my employee go to a conference, they will met people from other companies, and be tempted to go there.

There is an old anecdote about a the leaders of a company talking to a consultant, and asking "what if we train our employees, and they all leave?" To which the consultant answers, "what if you don't train them, and they stay?".

If you are afraid that the employees will leave if they hear about conditions at other companies, maybe you should address the conditions at your company, rather than letting them get worse, by not allowing people to go to conferences.

Also, have you considered the idea that the employee might convince other people to check out your place as a potential workplace?

3. If I allow one employee to go to a conference, everyone wants to go.

First of all, I don't think that is true. In most companies, there is a large group of employees who don't particularly like going to conferences, and thus won't ask for it.

But even if that's the case, then I don't really think it is a problem. There might be an economic limit to how many can go, but that can probably be solved by creating a turnus, where employees get to go on alternate years.

If a large group of employees go, then they can either use the conference as a team event (if they work together), or as a way to get to know people in other parts of the organization better (if they don't work together). Either way, it makes for a closer knit mass of employees.

4. Conferences are expensive

"Expensive" is relative.  IT employees are usually highly paid, and hard to recruit. For many good IT people, conferences are a factor when considering whether to stay or change jobs.

If someone changes jobs, the cost of replacing that person is much, much higher than the cost of allowing them to go to a conference.

5. My employees will get frustrated over hearing about new tech that they can't use

Yes, this is a real risk, but there are ways around that. Maybe allowing them to slowly introduce new tech in their current projects, or by allowing them some "play time" to build tools and other programs that can help them in their daily life.

Trying to keep them away from learning new tech is only going to work for the most incurious of employees. If you are fine with your employees not wanting to keep up to date on their field, that is fine, but if that's not the case, then you have to find a different solution than keeping them from konferences.

6. My employees will try to implement the newest fad after the conference

Somewhat related to point 5. Yes, this is a risk, but you can limit the risk by ensuring that the employee get an outlet for their creativity.

Is microservices the new SOA?

Within the last couple of years, Microservices has become all the rage of systems development.

Microservices, like many other terms in systems development, doesn't have an exact definition, but I found a good working description at an article by Martin Fowler and James Lewis:

The term "Microservice Architecture" has sprung up over the last few years to describe a particular way of designing software applications as suites of independently deployable services. While there is no precise definition of this architectural style, there are certain common characteristics around organization around business capability, automated deployment, intelligence in the endpoints, and decentralized control of languages and data.  

Looking at the description, there is nothing that doesn't reasonable, and reading the article linked above, it becomes clear that Microservice Architecture is a valid way of building systems, and one that should be taken into consideration when designing a system.

Unfortunately, it seems like people don't really consider it, but rather just choose it by default.

A decade ago Service Oriented Architecture was all the rage, and all systems had to be based on SOA. Unfortunately a lot of people didn't understand SOA and thought it was all about having a servicebus as the central component through which all communication went through. This was so widespread that I and many others started to say that SOA stood for Servicebus Oriented Architecture.

It seems like Microservice Architecture is moving down a similar path.

Many systems supposedly based on Microservice Architecture are not really based upon it, but rather on a misunderstanding of what Microservices are all about, creating systems which mimics the properties of Microservice Architecture, but don't implement the actual characteristics of Microservices.

Some symptoms of this misunderstanding of Microservices, could be:

• Code dependencies between services. If code changes to one services means that you also have to change the code of other services, then the architecture doesn't fulfill the concept of independently deployable services.
• Functionality dependencies between services. If the system always have to call service A before service B, and only call service A when service B needs to be called, then it seems like the lines of the system hasn't been drawn the right place.
• The need for a service portal (or even a servicebus).

There could be good reasons for each of these symptoms (though I am hard pressed to think of one for the first one), but they are definitely something that should make you consider whether you have really created a Microservice Architecture, or if you have just peppered your system with services.