The Incubation Effect

A colleague recently mentioned to me an interesting psychological phenomenon called the Incubation Effect, which is an observation that people do better at problem solving if they put the problem aside for a while before coming back to it.

This effect is certainly not counterintuitive. Many people, myself included, have experienced it. On more than one occasion I found that, after wrestling with a complicated algorithm or a tough bug all day, I’d wake up the next morning with the solution suddenly occurring to me. I had always assumed that the key to it was good night’s sleep, but that turns out not to be the case.

Elements of Incubation

There are a few ingredients that make the incubation effect possible:

First, the mind needs to be primed. The person trying to solve the problem needs to have a good enough understanding of it to actually make progress. So, you need to spend some time studying the problem before pausing.

Second, you need to pause for a specific amount of time. It can’t be too short or too long and the duration of the pause should be proportional to the duration of the priming. Apparently, a ratio of 10:1 is beneficial (i.e. for a problem which takes 1 hour to prime for, the interruption should last about 10 hours).

Third, what you do during the interruption matters, although exactly how is a bit murky. There is some evidence that doing something unrelated to the original problem helps improve results. On the other hand, there is also evidence that doing something related can reduce the required duration of the interruption in some people.

There is a lot more to this effect than what I just described (things like variations by individual, types of problems being solved, mechanisms that make this effect possible, etc). If you want to learn more, check out this study from the University of Sydney and this paper from Carnegie Mellon.

Taking Advantage of Incubation

It’s interesting to consider whether the incubation effect can be exploited in software development projects. Some implications are obvious: if you run into a challenge, move on to something else and come back to it in a day or two (maybe even put up an “In Incubation” section on the Task Board).

Other implications are more subtle. For example, should interruption occur sometime during project’s inception phase? After all, the team is brainstorming ideas about how the feature should work, how major architectural elements are to be implemented, and so on. Is there major insight to be had if the team pauses for a few days after the initial discussion? And is that insight worth it, especially considering the logistics of pausing a team for a few days?

Final Thought

I’m sure there are other ways to exploit incubation for our benefit, but I just can’t think of any right now. Perhaps I should take a break and come back to this in a few days.