Natural Selection in Technology

Let me preface this article by saying that I am in a position reliant upon the evolving technology market and I believe it’s my job to contribute to that market. I’m in a very unique position in that I see so many transactions and technology choices across the Chicago market that I get to generate some interesting theories. So let’s explore this one together.

So, Natural Selection – an idea pioneered by Charles Darwin about favorited evolutionary traits leading to the progression of a species. This is how we get the fastest antelopes, the strongest lions, the most unique natural camouflage, etc. While the human race has, in a way, evolved out of the need for natural selection through communication, medicine, psychology, and so on, technology has not. The need for natural selection in technology has never been more important given how quickly technology choices are made. I’ve studied several technology markets and am now looking to discuss my findings.

This idea started when I started noticing Python/Django shops start moving to Flask. Why was this transition happening? As most engineers out there probably know, Django has been the dominant web framework of choice for Python shop everywhere. Since it’s an open-source framework, many people were contributing to it given that engineers were using it heavily day to day and all wanted their own customizations to solve the problems at hand. While this is why open-source frameworks are so attractive, ultimately it led to a very heavy framework which became slow. More code was contributed to add new features/customizations/tools without older stuff being deleted (which tends to happen). So Django became heavy and engineers looked at Flask as a lightweight web framework they could start over with and bring over the good elements of Django. This is a fantastic example of why natural selection in technology is so essential. The open-source market is lucky because there are so many contributors out there and so much usage of every tool that problems and benefits arise quickly and can be diagnosed. 

This isn’t the only area where natural selection happens, so let’s explore an enterprise example: Microsoft.  Now I’m making quite a claim here, but that’s the way technology seems to evolve; through discussion, arguments, and ideas. Microsoft built a suite of technology used in the enterprise world for years. It’s not a cheap solution, but obviously you get a lot of support from the Microsoft team and it’s worth it if you have the money. However, it seems in the recent years that Microsoft tools were not progressing as quickly as open-source software in terms of new versions, new tools, everything developers need to stay challenged, interested and constantly solving problems. The reason this happened – I believe – is because there were so many more users of open-source technologies, so many more companies trying to do something never done before, and so many contributors collaborating via co-working spaces, Github, and meetups that the technologies got a lot more attention and were put through the ringer so many more times. Microsoft/.NET didn’t need to evolve as quickly. Open-source tools did.  Never before was a company seeing completely random, and massive, web traffic, but then Twitter came along and had to evolve quickly. But Microsoft is smart.  They knew what was going on and had to force evolution. Now Microsoft will offer BizSpark – free access to developer tools, support and even education on how to launch your startup. This is a great move for Microsoft to start seeing their tools take on the startup market and be used by some of the smartest engineers (which used to all go work for Microsoft out of college, but are now seeking startups (insert article link). Microsoft did not necessarily choose this path out of interest, they were forced to based on how the markets were naturally evolving. 

Now, let’s discuss another technology sector and what we think might happen: Javascript MVC. The Javascript market is very unique and it’s also the technology sector I hear the most demand for. Much like when Ruby on Rails became so popular, it was empowered by a new framework, not a new language. Like Ruby, Javascript has been around for a long time. Previously, it was used in limited capacity and the reason for the massive change and demand was the emergence of a new, powerful framework. But Javascript did not become popular because of just one framework (as Ruby did because of Rails), Javascript became popular because all of a sudden there was a suite of frameworks to choose from based on the problem at hand.  Never before have there been so many framework choices for engineers to choose from, which, in turn, is evolving the role of an engineer. Rather than just coding what they’re told, engineers are increasingly relied upon to analyze the business problem prior to deciding the technology stack. I always associate this with Civil Engineers building a bridge: before choosing what material/metal to build the bridge with, you need to understand the purpose of this bridge. How much tonnage will pass over it? What are the winds like? Are there traffic lights with cars sitting for a while? How long does it need to be?  Once the business problem is understood from all angles, then you choose the metals and architecture of the bridge. Engineers do the same thing depending on what they’re trying to build. 

Today, there are more technology problems being solved than ever before so we need natural selection to continue driving technology. From this article, I also hope technology can welcome ideas from sociology, biology, communication, etc. to drive the evolutions and become the first industry reliant on many theories. Economics are already at play defining the languages engineers first start working with, and people’s willingness to pay thousands of dollars to join a bootcamp to learn Ruby on Rails. Most new engineers are attracted to Ruby on Rails because of the quick path to high salaries (based on the low supply, high demand) and the uniqueness of ownership over a large web system so early in their career.

Now, I’m not an expert in technology, but this was a unique trend that I wanted to share with my community. Pretty interesting right?