orion3/Articles

SUBSCRIBE TO MY UPDATES

Get notified when I post a new article, software update or release. All messages will be sent in plaintext only. To subscribe, just send an email to

Fine-grained subscriptions

You can subscribe for notifications of a certain type only. Add the following words, separated by spaces to the subject field of your initial message.

  • articles — A new article is posted.
  • releases — Software updates or releases are rolled out.
  • other — Something else, which may include things ranging from like "hey, this site is closing down", to "I never sell things to my subscribers, but when I do...", to "the site is changing its domain name".
No unwanted messages, optional PGP
Your email will be kept safe out of in any DB on a 3rd party-server (given that, you trust ProtonMail at least), but rather it'll be kept in an encrypted text file that I will be accessing manually. If you'd like notifications to be encrypted, attach your public PGP-key, or if you're using ProtonMail, it'll be encrypted be default.

How to unsubscribe

The address you shall use to unsubscribe is the same, only the keyword is different: add STOP to the subject field. You may also unsubscribe from the certain categories of notifications only, in which case make subject field look something like: STOP updates other. At the same time, send STOP all or STOP will unsubscribe you from all notifications.

Programming, 2022

I went on Twitch and clicked on channels that came up for the search term "programming". It's really sad. Part of me knew this was already going on with programming, but Twitch only reconfirmed my concerns. I realize Twitch is somewhat more popular with game devs, but I have not been clicking on those channels. I clicked on those, where people would be writing things in Javascript (web-dev seems to be the only other topic with significant representation there). But regardless, what I saw was not just disappointing - it invoked a feeling of inevitable doom.

I must prepend the rest of the article by saying that I don't believe that the young people I've observed coding and trying to work on their projects were stupid. On the contrary — they seemed to be very bright and receptive to the few suggestions that I had made, but, I figured, I cannot make too many. In fact, I doubt that even the ones I made would help with the underlying problem: we're a drowning in layers and layers of software that cannot even be classified as "abstraction" layers. It's vanity layers.

The story goes like this: somewhere along the way from year 2002 to 2022 we were all collectively led to believe — with different degrees of certainty — that what's happening in how we approach writing software is the right way. This process has been happening not just in one area with which I'm intimately familiar — web applications — but in other areas too, be it smartphones, Desktop and server operating systems, crypto-currencies... you name it.

But first, let me just give you a very straight and specific example of what I saw on one of the channels on Twitch.

The person streaming this one channel was working on a webapp, building it with React & Firebase (first time I heard about Firebase!). Now, I'll set aside my own reservations towards React, because we used to burn people on fires in early 2000s for mixing Javascript with HTML and CSS — but I don't think that would serve any justice to my attempt at describing the real issue. My opinion about React is my opinion and just that. But I inquired about the nature of the app they were trying to build and upon receiving the description, I asked my next question: why would you need Javascript, let alone React, to build it? Because, at the time I asked that question, I was looking at this person's screen where they were trying to figure out the Sign Up / Sign in window (or pop-up, or whatever you may call it). The answers that I got to my question from that person can be separated into to major categories:

  • The first were statements along the lines of "I only know how to do it in React and it would also help me build my portfolio to get a job" - that's more or less reasonable, fine.
  • But the second category were false statements: for instance, the person wrongly assumed that without Javascript users would have to sign in every time they visited the website (as if cookies depended on Javascript). Or that without Javascript, it'd be impossible to make API calls — which was also clearly wrong, because an API call is an HTTP request (most of the time) and authenticating users doesn't imply asynchronous requests and pop-ups with username/password fields brought up dynamically by Javascript. All of this, as well as many other things, can be done with plain HTTP requests to the server which would then send the rendered page back (also commonly known as server-side rendering these days — another term, that wasn't even necessary in 2000s, because all rendering was done in the backend).

You get the picture: it's layers of software that is actually much harder to work with, and which by mere introduction of the extra code increases attack surfaces of webapps using them.

And the situation with backend frameworks isn't any better. Those frameworks usually implement their own routers, which assume a single point of entry and the whole framework being held in memory — often multiple instances of it, so that it can process more than 1 request at a time. But routing is something that Nginx or Apache or any other web-server were designed to do — that would in fact be one of their primary jobs.[1]

It would be a good moment now to say that I'm not anti-framework. I've used quite a few frameworks and wrote my own JavaScript frontend framework. There is a deeper underlying issue here and frameworks are not necessarily to blame for it. [2]

But I'd like to return to the original point of this article: I'm quite sad about what the younger generation of engineers have on their minds — not by choice, but out of lack of the ability to even make one. My impression is that if someone decides they want to learn how to create a webapp and hadn't had any prior experience with programming, they would go google this question and then all they'd generally get would be something like: React, Vue, {insert your backend framework here}. You see, even though I really started programming in high-school without even having a computer of my own — and it was Visual Basic believe it or not — the actual start of my interest in programming came with the discovery of CGI & Perl. Then, of course, it was PHP for a while, and I also somehow made this strange turn towards web-design and Photoshop for a few years before coming back to programming with the emergence of Ruby and Ruby On Rails. So in that sense, young folks I saw on Twitch aren't much different from the younger me.

Of course, these years were also very helpful with two things: I learned HTML and CSS well and have been keeping my knowledge of all the new stuff in it up to date to this day. But I don't have a computer science degree or similar (many programmers don't, ask Larry Wall, the creator of Perl). So I'm not one of the people claiming you're not a real programmer if you can't program a bubble sort algorithm, if you don't know much about data-structures or you don't know C. In fact, I barely ever had the need to know anything about those things over my 20 year career and only learned some either when I truly needed, say, to optimize things so that user experience wouldn't suffer or simplify things for developers using my code — these would be the very few cases where I needed fancy data-structures.

On the other hand, my interest was always directed towards things that most people shunned. When every Ruby On Rails programmer had a MacBook, I used Linux — that helped me set up my own server at no additional time I'd spend learning how to do it. I've been using Vim for almost the entirety of my career too. And I enjoyed the command line interface.

What's different now from the days I was starting out is that the layers on top aren't abstractions making goals newcomers have easier to achieve. I was watching that one person struggling with that sign-up page and React documentation all the while thinking to myself: "who was the smart-ass behind all this mess who managed to make a problem that would take 20 minutes to code — registration and authentication that is — into a task that can take a day to figure out even if it wasn't a novice programmer doing this? I know there are many many complexities to authentication — and you come up with complexities (sometimes necessary) for many of the common general problems. But it was a prototype, "an app for the portfolio" the person claimed. You don't need a solid attack-proof solution for a prototype. You need it to: a) work b) learn the underlying mechanism c) not spend too much time accomplishing one task.

The underlying issue, thus, seems to be the following: this new layer of software had been invented by the people who seem to be decent engineers by themselves. But they made a horrible decision to release and promote it (for which many of them had the means, since this software came out of large companies). There wasn't any profit in it either (at least, I cannot see it immediately) — it's all free and opensource. And so they somehow convinced the masses — or maybe it was just an unfortunate trend — that this is what you need to learn to be able to do this or that. You don't learn programming, you don't reinvent the wheel — you program in frameworks. Worst of all, those frameworks could hardly be called "another level of abstraction" as they don't seem to abstract so much as to obstruct learning about how things actually work — how to program, in other words.

This issue pervaded software engineering from all sides, not just webapps. A young guy I recently talked to, was learning game development through the Unity engine. He knew some C++, yet didn't seem to know the term "cli-arguments". But the most unexpected symptom of the problem I discovered was this: the young folks didn't know what the term "cargo cult" meant upon me mentioning it and after googling and reading up on it, they still struggled to understand why the word "cargo" was even there.

Programming in 2022, it seems, very much resembles what game developer Jonathan Blow described in his talk "Preventing the Collapse of Civilization". His point was that as time goes by and new layers of abstractions are invented, there are less and less people left who actually understand the lower levels — and when you run out of people who know how to make CPUs or write assembler, who's going to make those computers that run React for you?

It's worse than that though: the real problem is that degradation isn't just the process of losing people with knowledge of lower-level fundamentals — I myself don't program in C, even — I think that's fine. The degradation is happening at higher levels, where these abstractions become layers of obfuscation and aren't helpful at all. What happens when React (inevitably) perishes? What are all these young developers, who have next to none grip on the basics of programming in any language — a higher level language, even — what are they going to do?

I would add to Blow's concern of not having enough people knowing the lower levels yet another issue which only exacerbates the problem: many people these days only learn this useless "obfuscation" layer I talked about and are completely lost when it comes to even the basic ability to program outside of a particular framework or an ecosystem. To be fair, this concern was around even when Ruby On Rails was just released — it was constantly repeated by many that people should take time and learn Ruby first, then move on with using Rails. Yet a lot of folks ignored this advice and I don't think they got anywhere with their chosen career path. Maybe they eventually became project-managers, but certainly not programmers.

Someone on Twitch suggested the problem was "low entry barrier" — an argument I heard even when I was starting out and which always felt bonkers to me. It's good if barriers are low — let people learn. Unfortunately, the barriers seem to actually be higher. You need more powerful CPUs and GPUs to run all these layers of crap. If you want to be developing smartphone or Desktop apps, you need hardware and licences from Apple, Windows or Google. Same goes for any other gatekeepers. Even webapps are expensive: the wonderfully crafted extortion business called Cloudflare emerged and became widespread by DDoS-ing the crap out of your tiny website first and then forcing you to pay them for protection. This "captcha" company, as I like to call them, literally sees and stores 80% of the internet traffic, because they don't just sign the SSL certificates, oh no. They issue them (same goes for smaller Cloudflare competitors). But this is another topic and I'll leave it out for now, as it has more to do with privacy.

Frankly, I don't see any way out of this mess. You can't be "yelling at the cloud" (in some cases, literally), you can't be trying to convince anyone, even the young, as this wastes your own energy. I feel like there is now a great divide, which has manifested itself on multiple levels.

  • Divide 1 (within the group of inexperienced or young engineers):
    • The first group of youngsters, even though they're young, they know stuff deeply or have the interest in learning things that are fundamental. Or, at least, learning to distinguish what is fundamental and important from what isn't. And then studying the former. They are the minority that isn't vocal as they see no reason to speak up and waste their energy.
    • Then there are the ones who are "lost in the cloud" of software-development nonsense. They sometimes even seem to know a lot, but can do very little with it.
  • Divide 2 (within the experienced group or the older generation):
    • Here, the first group is represented by experienced programmers who see absolutely nothing wrong with the emergence of this pile of bloatware. Be it frontend stuff, or, say, this Kubernetes thing. They see nothing wrong with spending time studying AWS documentation instead of learning how to set up your own Linux of BSD server and learning how to set up your own virtualization (which you'd admittedly need) with QEMU & KVM or bhyve respectively.
    • Then there's another group which I seem to belong to. That group may indeed not have the knowledge or the desire to learn the complex trendy stuff such as Kubernetes, the multitude of AWS services... or how to start using that Webpack thing without attempting suicide at least twice in the process. But this is not because we're incapable of learning it. In fact I did learn some of the above due to the demands imposed at work — only to discover this knowledge was to become irrelevant very soon. But having that experience once or twice seemed to be enough. After that, the people belonging to this group start to feel like it's more important to learn things that can get the job done even if AWS servers are nuked and half the internet is offline and you no longer have StackOverflow to rely upon. And that's not because that's what we expect, but because "nuking" most of the time is, in fact, a slow decay and happens slowly over time as these tools become irrelevant or change drastically. Either way, you cannot possibly keep up with all that without losing sleep.

So this is almost like a class divide. I'm not sure what to do about it or whether anything needs to be done. I'd sometimes browse through job openings only to confirm my concerns — to my horror — the kinds of requirements companies have these days. That lame stuff from 10 years ago such as "we want a rockstar developer with 10 years of experience who is excited at whatever we're going to throw at him" was child's play. These days it's something worse, but I cannot quite pin it down just yet. Perhaps in one of the next articles.

One thing I am more or less certain about is that the divide is very real. And this is something new and most definitely not one of those tales as old as time. It's not me being rigid. We are being attacked, deliberately or not, by absurdity.

I bought a book on Perl 5 recently, in paper, 4th edition — that would be from 2015. I first came across Perl around year 2000. So I read this book and almost nothing had changed. The things that changed were mostly very pleasant, such that I'm pretty sure Ruby, in comparison, has literally nothing to show for itself anymore. Looking forward to learning Perl 6 (renamed to Raku — horrible name, to be honest) and seeing what's there.

And here's what I realized: I can build just as fast and make it just as good with no framework at all. Give me a language: Perl, Javascript, PHP even. Hook it up to the CGI in Nginx or Apache and you're good to go. I'll admit that by learning frameworks I now have the knowledge of the multitude of things those frameworks took care about for me. But I hardly needed all of the features. And I think that for a small project one should not be concerned with stuff that's irrelevant. No one's hacking or DDoS-ing you until you start making money (or too much noise, for that matter) — which means you then have the luxury to actually learn programming, frontend stuff and system administration. And when the time comes and extortion folks show up, you deal with them. But let us not worry about them while they're busy with larger targets.

Each individual programmer, whether their aspiration is to build their own business or simply get an interesting satisfying job, shall be concerned first and foremost with how they spend their own time and which things they're learning — not with things such as "business development" or "passing an interview". These latter things reduce you to a mere craftsman. I realized I'm not willing to be one. Being an artist is much more fulfilling. And art can emerge from almost any skill and field of human activity. That distinction people have in their heads about "artistic" professions vs "craftsmanship" is just wrong. I think this might actually be the very point of this article.

References

[1] This topic deserves a separate article. The "microservice paradigm" obviously emerged as a reaction to the unmanageable monolithic frameworks. What the authors of the paradigm may have forgotten was that CGI was the original microservices implementation. User would make an HTTP request to the server and the server would execute a particular file and send back whatever output that file produced. You would not be holding the whole app — or even parts of it — in memory, simply because it'd be fast enough to load and execute on demand. Yes, eventually, you'd hit a bottleneck even with that approach, but that bottleneck would come much further down the road, and because you would had designed your app this way, it wouldn't be much of a problem to break it down and twist it any way you wish to resolve the scalability issue. And you wouldn't have this long and stressful "transition" period from a monolith to a microservice. We've being going in circles all these years.

[2] Being the CTO at the time, I could afford that luxury of imposing my own software, picked or written by me, but I also wrote an extensive documentation for the framework, so the team wouldn't struggle. In our case the framework was justified, because on some pages of the website we had some heavy user interaction and there was need for both websockets, Ajax-requests, client-side encryption and morphing the DOM according to the response from the backend. But the framework was also designed in such a way that only certain parts with the required functionality could be loaded onto the page and it never forced anyone to make the choice of proclaiming their webapp to be "a single-page-application". As well as not forcing anyone to use a particular backend. It simply didn't care about the backend — it would be up to you how to interact with the backend. It cared about other things.