Skip to main content

Hayek on Complexity and Knowledge

In Software Development, knowledge acquisition and transformation on one hand, and internal and external complexity, on the other hand, are two interconnected and inseparable aspects of the same endeavour.

F. Hayek, a towering figure in economics, and a major member of the Austrian School of Economics, put it simply.

“𝑻𝒉𝒆 𝒄𝒖𝒓𝒊𝒐𝒖𝒔 𝒕𝒂𝒔𝒌 𝒐𝒇 𝒆𝒄𝒐𝒏𝒐𝒎𝒊𝒄𝒔 𝒊𝒔 𝒕𝒐 𝒅𝒆𝒎𝒐𝒏𝒔𝒕𝒓𝒂𝒕𝒆 𝒕𝒐 𝒎𝒆𝒏 𝒉𝒐𝒘 𝒍𝒊𝒕𝒕𝒍𝒆 𝒕𝒉𝒆𝒚 𝒓𝒆𝒂𝒍𝒍𝒚 𝒌𝒏𝒐𝒘 𝒂𝒃𝒐𝒖𝒕 𝒘𝒉𝒂𝒕 𝒕𝒉𝒆𝒚 𝒊𝒎𝒂𝒈𝒊𝒏𝒆 𝒕𝒉𝒆𝒚 𝒄𝒂𝒏 𝒅𝒆𝒔𝒊𝒈𝒏. 𝑻𝒐 𝒕𝒉𝒆 𝒏𝒂𝒊𝒗𝒆 𝒎𝒊𝒏𝒅 𝒕𝒉𝒂𝒕 𝒄𝒂𝒏 𝒄𝒐𝒏𝒄𝒆𝒊𝒗𝒆 𝒐𝒇 𝒐𝒓𝒅𝒆𝒓 𝒐𝒏𝒍𝒚 𝒂𝒔 𝒕𝒉𝒆 𝒑𝒓𝒐𝒅𝒖𝒄𝒕 𝒐𝒇 𝒅𝒆𝒍𝒊𝒃𝒆𝒓𝒂𝒕𝒆 𝒂𝒓𝒓𝒂𝒏𝒈𝒆𝒎𝒆𝒏𝒕, 𝒊𝒕 𝒎𝒂𝒚 𝒔𝒆𝒆𝒎 𝒂𝒃𝒔𝒖𝒓𝒅 𝒕𝒉𝒂𝒕 𝒊𝒏 𝒄𝒐𝒎𝒑𝒍𝒆𝒙 𝒄𝒐𝒏𝒅𝒊𝒕𝒊𝒐𝒏𝒔 𝒐𝒓𝒅𝒆𝒓, 𝒂𝒏𝒅 𝒂𝒅𝒂𝒑𝒕𝒂𝒕𝒊𝒐𝒏 𝒕𝒐 𝒕𝒉𝒆 𝒖𝒏𝒌𝒏𝒐𝒘𝒏, 𝒄𝒂𝒏 𝒃𝒆 𝒂𝒄𝒉𝒊𝒆𝒗𝒆𝒅 𝒎𝒐𝒓𝒆 𝒆𝒇𝒇𝒆𝒄𝒕𝒊𝒗𝒆𝒍𝒚 𝒃𝒚 𝒅𝒆𝒄𝒆𝒏𝒕𝒓𝒂𝒍𝒊𝒛𝒊𝒏𝒈 𝒅𝒆𝒄𝒊𝒔𝒊𝒐𝒏𝒔 𝒂𝒏𝒅 𝒕𝒉𝒂𝒕 𝒂 𝒅𝒊𝒗𝒊𝒔𝒊𝒐𝒏 𝒐𝒇 𝒂𝒖𝒕𝒉𝒐𝒓𝒊𝒕𝒚 𝒘𝒊𝒍𝒍 𝒂𝒄𝒕𝒖𝒂𝒍𝒍𝒚 𝒆𝒙𝒕𝒆𝒏𝒅 𝒕𝒉𝒆 𝒑𝒐𝒔𝒔𝒊𝒃𝒊𝒍𝒊𝒕𝒚 𝒐𝒇 𝒐𝒗𝒆𝒓𝒂𝒍𝒍 𝒐𝒓𝒅𝒆𝒓. 𝒀𝒆𝒕 𝒕𝒉𝒂𝒕 𝒅𝒆𝒄𝒆𝒏𝒕𝒓𝒂𝒍𝒊𝒛𝒂𝒕𝒊𝒐𝒏 𝒂𝒄𝒕𝒖𝒂𝒍𝒍𝒚 𝒍𝒆𝒂𝒅𝒔 𝒕𝒐 𝒎𝒐𝒓𝒆 𝒊𝒏𝒇𝒐𝒓𝒎𝒂𝒕𝒊𝒐𝒏 𝒃𝒆𝒊𝒏𝒈 𝒕𝒂𝒌𝒆𝒏 𝒊𝒏𝒕𝒐 𝒂𝒄𝒄𝒐𝒖𝒏𝒕.”

It is true and insightful on multiple levels. To illustrate: take the way software architectures are generally built: there's an hidden assumption that the future is made of a sequence of static "images" and as a result the architecture/structure reflects that perspective. Unfortunately, there will always be an event or a stressor happening in the environment that the architecture was not designed for, forcing you to make a decision that is not completely under your control. Generally, when this hits, it is painful, and will likely break your architecture (or put your system out of balance).

The reality is, as alluded by Hayek, the environment your software will be deployed is socio-economically complex and unpredictable.

This is what makes building software very hard (and very human):

⭕Embedding flexibility in the functional space (I called this forward thinking mode) to accommodate change

Evolving the architecture based on unpredictable usage patterns (backward thinking) by stacking residues (see Residuality Theory by Barry O'Reilly).

No advanced AI can help here, sorry. Apart from the desired efficiency gain we demand from technology.


Comments