Mis-plantation
Nature is really strict when it comes to conservation of resources, always rerouting them according to the demand and cutting costs whenever it's possible.
That is why we have evidence of such impossible feats like Mitsutaka Uchikoshi surviving more that three weeks in a state of "incidential hibernation", that happened due to his body slowing down his metabolism to such extent to prevent starvation and dehydration. You can find a plethora of such stories, serving as anecdotal evidence of this law being applied for our own good and You may think that this effect is always with the "our best interest" from the point of view of survival, but the truth is much more grounded - it is pure physics, and there is no intent here. Moreover, there is also a mirror phenomenon, one of many, that shows this ruthless efficiency.
During my third year of materials engineering studies I had an awesome set of courses dedicated to biomaterials and their use in medicine. Teeth fillings, endoprosthetics, totally recreated parts of bodies, small stents made from specific metal alloys - sometimes this stuff sounded like taken straight out of Shirow's New Port City.
One of my favorite lectures was about hip implants and tricks used to make the prosthesis more acceptable to human body i.e. make it more likely to accept a totally foreign object inside of it. After a couple of those presentations, we also investigate what could happen if the properties, like tensile strength and other characteristics, differ greatly from those of its surroundings, skewing the intricate balance of forces normally experienced by our bones more towards being carried by the implant. Letting something do the body's original work is not always a good idea, since this gives is only false signals and fires up these automatic rerouting mechanisms mentioned at the beginning.
It is highly rational if You think about it - if there is no need to fuel the useless, now relieved by prothesis, then the resources can be moved elsewhere. Over time - the atrophy proceeds, tissue deteriorates and the original seating of the implant loosens, making a cascading effect of negative forces acting upon its surroundings. Thus, it is important to make the implant and the "implantee" to work augmenting their resemblances.
Kind of a long intro, but what in the magical world of biomechanics is called as this "Use It or Lose It" mechanism, in the overall landscape of post-GPT-3.5 world can be found in the effects of most commonly seen way of working with generative AI models.
Brainrot
The spooky word of 2024 (according to Oxford University Press), used originally in the context of gobbling up endless hours of Subway Surfers gameplay interlaced with elaborate overviews of latest dramas or trends popping up in the corners of the Web. Or abstract stories of bomber-jet-morphic crocodille interacting with an espresso cup with legs. Or both. Multiple streams of unchallenging content, that takes the cognitive load off the mind and just "is", just "happens". This issue by itself became a part of our modern lives, hence the subsequent indentification of phenomena like "doomscrolling".
So, in general, the "cognitional discount" seems to be an appealing aspect of the Wide Web 3.0 World, and its effects can be seen even in more serious areas of our series of tubes. With raise of the demand due to the sheer throughput needed to keep these streams of low-quality content going, shortcuts are also taken on the side of producing it and making it available for consumers. This is kind of the "chicken or the egg problem", because it is hard to pin-point the cause and effect in the relationship between the sudden boom in popularity of LLMs or the AI slop fed via the various social media algorithms straight into our palms.
Now after couple of years, this effects spreads from the general area of entertainment into our professional work, and the first doomsday cult certainly appeared within the AI sector, with the tales of massive exodus of developers from companies that can splurge on the Pro Business Enterprise plans from their chosen GenAI provider.
Truth be told, while nothing like that happened on a massive scale, some cases of bigger layoffs have been noticed (with negative repercussions for the chosen companies, which jumped the gun) and LLMs became the "your Mom's best friend's son" to fresh IT juniors, making the recruitment more difficult. However, the negative effect can be felt in the day-to-day job when the already hired developers start trusting this best friend too much.
Story of a hollowed mind
When Google first hit the market and became our official all-knowing Uncle, the ancestors of the contemporary AI doomsday cult preached about the incoming idiotification of the masses and dystopian landscapes full of people who traded their hard-earned general trivia for easy-to-access and nicely indexed content.
While some loss of long-term knowledge has been noted and branded officially as digital amnesia i.e. the Google effect, the value of the lost knowledge is a very subjective matter, because it pretty much boils down to a question of what is the value of the things we know, given our everyday lives? Would my life worsen in any way if I suddenly forgot what was the date of the Battle of Grunwald? Would my value deteriorate if I list this knowledge? And to which eyes, in what context? I can say confidently, though that it wouldn't impact my day-to-day job in any shape or form (well, maybe I would have to use some other placeholder dates in my mock data for some projects). In other cases, I can always look it up online, so this part of my knowledge has been practically delegated, or even orphaned, to the omnipotent Cloud.
However, this is just a process of fracturing away portions of static knowledge, which lays dormant and its only functionality or purpose is to just exist to be retrieved some time later in an appropriate moment. It may leave space for other, more relevant information, which can be actually used to my benefit and also does not really diminish my cognitive capabilities - the core mechanism of grabbing that portion of knowledge in an efficient manner is still there, I can still successfully remember things and I do their selective storage consciously. I learn what I need, for anything else - I know where to look.
The other aspect is the side-effect, which can appear due to my constant interaction with the Web as another "storage" or source of knowledge I might use to solve my everyday tasks. This is really a matter of diligence and being thorough, because "looking up things in Google" can look different based on the type of person who performs the actual search. I am inquisitive enough that I try investigate a little bit longer, find couple of sources of the information I looks for and then synthesize quickly an answer to my original question for that. If I am wrong in my final conclusion - I might relay that false positive of the truth to other people. However, I still performed these operations on my own, or quoting the classic, the task failed successfully, since I looked through and churned some data to produce this result - the fault can maybe be looked after in my critical thinking, but the raw cogs and wheels of research and synthesis parts worked okay.
When we start to look into the relationship of current developers and commonly used GenAI tools, the problem resembles a hollowed mind, but however drastic this effect may sound, the actual process starts on a microscopic scale and in general is very inconspicuous. Profesionally, every one of us has some small "holes" in the way of our thinking. For example, some of us can be "algorithmically impaired" and, even after finishing 1000s of LeetCode exercises, they still need extra time to invert that binary tree when caught off-guard. Others may not have that skill of high-level recon of the infrastructure they are working on. Whatever it may be, it is one of the reasons why teams exist, with leaders slowly accumulating the knowledge about those weak spots over time.
After the generative AI tooling has been widely adopted in the IT sector, its users, mainly developers, started patching up these imperfections with in-promptu "fillings", similar to how it is done when a prosthetic is introduced into a faulty body part. However, the computational capabilities of rows of NVIDIA GPU-filled server racks sitting somewhere in the North Pacific region are unfathomably bigger than our minds (duh), so there is a mismatch between properties of the implant and "implantee" and You can probably guess what happens next. The same energy-saving mechanism happens, but here the atrophy is cognitive, the "brainrot" becomes functional.
Generally, if we would take the whole duo of "me + GPT" and measure it in terms of this analytical performance - it starts fine, we can even say that the initial state of the mind is "augmented", "enhanced" and other transhumanistic buzzwords, but if we look at the same metric, while taking the human factor into the account - the truth hurts. Use it or Lose it. This is not only my opinion, and while it may have some "anti-tech vibes for being contrarian sake", it has actually been a subject of serious research done by business world folks at Microsoft with collaboration with Carnegie Mellon University (CMU) and also scientists from MIT, that serve as a cautionary tale about traps a developer can fall into if the distribution of cognitive power shifts to the side of the computer, instead of the human.
For the brevity sake, I will try to outline three examples of skills, that I've noticed are crucial in everyday software development and often pop up routinely in discussions with my colleagues. With each one of them I will also outline how I see the constant use of AI influencing it.
...And improper fillings
So, first of all, what are those skills?
- Lateral thinking
- Vertical thinking
- Algorithmic intuition
There are of course much more abstract and more social things that are good to have and sometimes are more crucial in the context of working in a team (and which can also get highly affected by the prolonged use of LLMs), but I've wanted to first approach these low-hanging fruits to make my point across and, from my initial research, seem to have the highest ratio of being influenced by the aforementioned athrophy, to the effects it has on the day-to-day work quality of a developer.
Mental schematics and project memory
Navigating an IT project, especially that which greatly predates You career in a given softdev team, can be an art on its own and really requires a lot of time to sink into the caveats of multiple libraries and/or dependencies used there.
At the beginning it can be reminiscent of an investigation with red threads spread between AWS us-east-1, janky on-site servers hosting some reverse proxies and networking labyrinths set up by the mix of DevOps and SysAdmin teams.
You learn the most important settings for the pieces of Your event-driven puzzle, know which library should not be bumped due to that critical memory runoff bug introduced in the newest version etc. etc.
Over time, You can even sprinkle in the domain-specific knowledge of why certain decisions have been made - why some services are rate-limited, why do we care about storing the particular dataset generated by one-shot ETL pipelines and, in general, which parts of the code translate to net gain or loss of money within Your company.
This particular set of knowledge comes from the hours of Confluence/GitHub(Lab) Pages/Google Drive surfing and marathons of ad-hoc meetings with people on several levels of the corporate ladder.
So, what may happen if a developer starts to use the AI tooling in such projects? It really depends on how much additional data can be squeezed into the context of used prompts, but my gut instinct tells me that nobody really tries to attach recordings of all conversations relevant to the current chat's topic together with all technical documentation, for all building blocks of the most-recent version of architecture (not that it would be also doable from SecOps perspective when using externally hosted models).
This gives us something like one or two PDFs would be attached and, if we are feeling super unsure about what the AI will produce on its own - a deep search option enabled, for good measure. Now imagine that you would be presented with the same amount of resources. What would you be able to deduce, in practice? And here, You may even have a ace up Your sleeve in form of creativity and experience from previous, similar projects. The only far-fetched thinking from AI that is available would be a set of loosely connected hallucinations, that may look convincing on the first sight. This interference in navigating the project's scope may grow bigger and bigger over time, with the original knowledge about its intricacies muddied with unnecessary noise.
But that is not the main point of this section. According to the MIT Media Lab research, in the fourth experimental session the users were disconnected from the "genie in the Cloud" and asked to perform creative work on their own, with their previous experiences with GenAI possibly a boon, due to absorbing the new, more efficient ways of working then and being able to translate this methodology into future.
The kicker here is this - they have performed worse than the control group. When they were asked to recollect what they were working on previously with LLMs, they couldn't, meaning that this knowledge has effectively vanished. This can be an extreme inverse of the effect that You may recognize from Your high-school/college days.
Usually, when You make notes during lectures, You tend to learn more than from just listening. You have to engage with the presented content, "grab and organize" words, abbreviate something, maybe condense two sentences using some reference to previous courses ("just like with the...", "similar to the...") - the mind's internal ETL pipeline is bombarded with data. Now, in case of the GenAI content and the MIT's experiment, You don't even have to do anything. Content is created, it is written down on the computer's screen, the most of the engagement comes from some subsequent grooming prompts ("okay, but make it more XYZ") and copy-and pasting the stuff somewhere. It has a very low chance of leaving any mental mark on its "creator" (?). It stays external.
In short, over time more and more cognitive debt is accrued that can be another layer of obstruction between developer and solving the underlying technical debt, because if this lateral overview of the project is lost, then "the coarse system scan" becomes difficult and, by the proxy, the location of where to go deeper and face the issue is hard to locate. The initially powerful augmentation of the troubleshooting process leads to general deterioration of its crucial part - Big Picture thinking.
Dull bit drilling
Society is built on interfaces. You take a complex thing, put it inside a sturdy box, and put some simple buttons on the box so that people can use the thing inside. The box makes it easier to use and prevents people from breaking it. For example, you can take the machinery of a clock, put it in a box, and put two hands on the outside along with a knob for winding it. Take all the machinery of a car, hide it behind a dashboard, and give people two pedals and wheel. Take all the circuits of a computer, put them in a box, and give people a monitor and a keyboard.
Interfaces receive input and produce output, and that's all we need to know. The clock gets wound, and its hands show the time. Input and output. As far as the user needs to know, what happens inside the box is magic. This allows stupid and ignorant people to use complicated things, as long as the interface inputs and outputs are simple.
Toyota uses millions of kilograms of steel every year. Does the CEO of Toyota know how to make steel from scratch? If he wanted to beat a guy up, could he go digging in the ground for some ore and whip himself up a batch of steel to make a pipe? No. He uses interfaces to get steel. He buys steel from an steelmaking company. Except he doesn't personally go down to the steelmaking company with a bag full of Yen, saying, "How much for a million kilos?" He uses a bank. Except he doesn't even personally go to the bank. He has a subordinate who does it for him. All these people and institutions are interfaces he can use. He employs a system of layered interfaces, both metaphorical and literal, to control things he doesn't really understand. We all do. The point is this: don't go messing with the CEO of Toyota. I assure you, he could get his hands on a steel pipe if he wanted.
The word "interface" refers to the input and the output, but it also refers to the box. We think of interfaces as existing in order to give us access to things, but they are also there to hide things from us. The idea is that some things are better off hidden. Everything will go along fine so long as a certain input produces the expected output. But when this stops happening, we have to open up the box and see what's inside. Sometimes we don't like what we find.
-- What happens inside the Box is Magic _9MOTHER9HORSE9EYES9
How deep into the black can our black boxes be before we lose sight of them in dark of the rising tech debt? The truth is similar to the Toyota's CEO and his steel bars bought through a series of convenient proxies, but now the allure of knowing the true inside-outs of the production-ready solutions is close to none.
Sometimes developers can spare to be sophisticated in their solutions, reach down to the primordial techniques like manual memory allocation, optimize close to bare metal and tinker with the numbers to add the "-est" suffix to whatever they polish. But the speed man, this seems to be the usual metric, and as with each metric that becomes THE goal over time - it starts to lose sense. The "gimme fuel, gimme fire" tempo quickly brings its drawbacks, since recklessness during creation is often paid with time spent fixing. And who/what is great at rapidly producing new "prototypes" (which are quickly promoted to staged version of the project and baptized with the beta tags)? <insert the LLM name from Your chosen provider>
This also goes outside the low-level boundaries of how 0's and 1's dance around the heart of the machines we communicate with, but spills over the abstractions we use every day - the language itself. There are two, actually investigated tricks our minds use to let the AI take over our syntactic functions during the writing of our code.
Automation bias + cognitive miserliness = cracks in production
Usually, we know that we shouldn't trust "strangers", especially when they try to aggressively pitch in their two cents into our carefully sculpted works of programming art. We often treat those people as those who cross some threshold of accepted involvement in what we are currently working on, sometimes even harshly summing up that fact with "they don't get it". So, there is a natural boundary between those who are "in" and those who are "out".
However, things get much more complicated in case of introduction of GenAI into our workflow. In 2023, team from Harvard Business School conducted a research on the "jagged technological boundary" or, in other words, cases where we let LLMs temporarily diffuse through this barrier of gatekeeping, normally reserved for our fellow humans. When compared to group who have worked without the use of AI, they have scored relatively worse. Why though? Well, as often it is in our everyday relations, it is a matter of trust.
What is the usual roadmap of the input given to us by a human "outsider" in the context of our work? What is usually the initial state of that comment under the MR that You have prepared, an often knee-jerk reaction to the notification about it? It starts as somebody else's opinion. Objectively, it could be a very much grounded piece of technical information and should be an automatic inclusion into the project, but when presented by a fellow human it must be first accepted and digested, AND THEN it can be promoted to the status of a fact, after conscious admission from our side. Is it due to our natural or learned competitiveness? Or the way we handle such situations also in everyday life? Dunno. But the existence of this barrier is noticeable, especially when significant differences of professional experience are present. There are seniors who practically make such diffusion of information reserved only for developers starting from middle positions. There are juniors who disregard such imbalances and are "always right", just because. And also - You can just straight up don't like somebody, and this feeling can cloud Your judgement.
When it comes to automated systems and other colleagues of synthetic kind - the trust is higher, and we are much more welcoming to sources "outside of the frontier" of our current work. The professional tone and ability of AI to quickly tuck in its tail when even slightly corrected make us more likely to categorize its new creations as fact, simply because it comes from "the machine", from something which is strictly utilitarian. Ignoring such spicy areas such as roleplaying bots and other dark recesses of our Web3 society, GPTs and Geminis don't really have personality per-se, so the "human-like noise" coming from it is zero. We don't look at the notification about the newly synthesized report and thing to ourselves, "great, what's is this asshole problem, NOW?". It just appears as new information and the boundary between "us and them" becomes blurred or, as the Harvard team puts it, jagged.
Persnoality adjacent factors aside, we also intuitively understand (as "the tech tribe") that what we get is the result of a simply stochastic process, so we have a knowledge of a natural error margin in the back of our minds. According to Stack Overflows analysis of its 2025 Developer Survey:
...66% of developers say they are spending more time fixing "almost-right" AI-generated code..."
But what is one of the questions hidden behind this statistic? "Why do they accept it in the first place?" Well, because it works, at least in the beginning. Over time, things like reckless memory usage or security vulnerabilities start popping up, which previously were probably just ignored in the beginning or shadowed by the blazingly fast speed of the delivery of a new feature and put aside for later "optimizations" or "grooming" of the project (which often doesn't happen if its not plopped down as an actual task in Jira). Sadly, this effect is inversely proportional to the experience held by the person accepting these inclusions.
Together with the structural integrity of the codebase, we also start to lose this connection to the bare metal and syntactic intricacies of the language. We can become dulled down to "reviewers" of autosuggested patches, and our vertical penetrability of several abstraction layers that are in place, because we have effectively hidden ourselves behind the most fuzzy of all of them - LLM.
Moreover, once again, let's go back again to more primal and human things.
As we are part of the nature surrounding us, we observe the patterns happening within it and are often eager to adopt it as part of our approaches to some aspects of life. The uniform optimization of the energy spent on various tasks is also pretty convenient for us, so any time we can copy-and-paste instead of writing stuff, we choose the path of least resistance. Pretty similar to what happens in the case of the gradual degredation of project-wide reasoning skills, we train our brain, through these periods of chosen lower neural activity (once again, according to the EEG measurements done during the aforementioned MIT study), to hand over the analysis and inquisitiveness to the external provider.
We basically outsource our analytical processing to an entity, which may be good at dissecting raw data and performing very dry analysis of it, but lacks more out-of-the-box thinking. What if somebody previously came up with a new, yet undocumented computational method? Should we treat it as non-existent, because LLMs did not have a chance to train on it or its applications in the wild? Because there is a high chance that an automation will do that, seeing it just as noise when compared to thousands of other scraped StackOverflow posts.
Instead of deep processing of both the newly generated content and the place when it would be placed in, we practically loose our natural FAFO way of being and fallback to surface-level visual verification of what is produced to us. Because it costs us less energy, which "can be spent elsewhere" and, again, similar to the tale of the automation bias, those previous moments are dedicated to fixing bugs - at which we may be worse than previously, due to the advancing "rot".
How can we shortly sum it up? We delegate the thinking to something else because we are naturally lazy, down to the unconscious level, and we become more tightly coupled to the artificially introduced processing - the implant filled the hole successfully, but its surroundings started to become more and more inflamed. The architecture nuts and bolts start to become impenetrable to us, and we become mere operators of the high-level commands, losing the ability to drill down into the depths of the code we are working on.
Sucking at puzzles
Both of the aspects discussed up to this point had one thing in common - solving problems. With our Okham's razor chipped and general sense of direction blurred, we can have a hard time coming up with novel solutions to previously unencountered problems when disconnected from the outside help. This is kind of an answer to a question why do we need to learn so much maths up until the college and why making fun of people spending time to honestly go through the libraries of coding exercises is not really beneficial.
You may not remember how to perform a bubble sort, especially when You have You mind is occupied with trying to develop an asynchronous integration with some sketchy API at work, but the feeling of seeing something similar before, the way of reasoning applied there, tricks and shortcuts. Listening to the algorithmic, using a consequent method of separating those hunches from observations and reshaping hypotheses into more tangible relationships between cause and effect. A very specific muscle memory of the most important one of those in our bodies.
Vibe coding won't birth another Fast InvSqrt()
This is a simple combination of limitations of the domain available to LLMs i.e., (and I am being generous here) all of the historical human knowledge indexed on the Internet and nothing more, together with our, now, algorithmically impaired mind.
Okay, but what about the business value?
Ah, the good old question coming from the heavens above - how does it translate to money?
Well, this is the point where this post's underlying vivid imagery, inspired by one of my favs from Internet stories, comes in: if we make those previously shown boundaries become unstructured, what do we really become in practice? What is our role if we put the generative AI models between us and the systems with which we previously tried to interact in a conscious manner? The more trust and dependency is shoved into this relationship between human and inherently, not-yet generally intelligent thing, the more another barrier starts to become porous - between the roles assigned to either of those as receivers and producers.
At some point, we become (wait for it) flesh interfaces for the AI on its road of interacting with the architecture that we are the original creators of. Garbage comes in, garbage comes out - in this case, we might start acting as middlemen, who take the generated stuff, do whatever equivalent of CTRL+C CTRL+V of our operating systems, and put the garbage out into the world.
After "implement" the hot new feature, we start asking Gemini to take huge chunks of information and synthesize it into nicely written documentation. If that moment hits the sweet spot of our cognitive laziness (now nicely pre-trained by becoming dependent on that very same tooling), we, again, do the magic keyboard combo, content pops up in the Confluence, and we could possibly "fakenews" our team about the nature of the newly introduced feature.
And where does this bug come from that You are trying to solve for the last two hours? The spectrum of possible causes is now widened to not only who has written it, but also who and how might have just autogenerated it. Did they choose the new Codex or the locally hosted bootleg model found on a Reddit post? To what extent of content was used to lay an appropriate foundation of context for such a task?
One person's AI-generated treasure is another person's unfixable trash. Sadly, also according to findings published by Dinesh Elumalai on DZone, most likely it is the most experienced developers who have to deal with this kind of problem, which translates to real costs for the company since avoidable hours of development work hurt. But we can do such funny pranks to ourselves, which has also been reported by METR (Model Evaluation & Threat Research), visible first in the abstract of its July article:
Before starting tasks, developers forecast that allowing AI will reduce completion time by 24%. After completing the study, developers estimate that allowing AI has reduced completion time by 20%. Surprisingly, we find that allowing AI actually increases completion time by 19%--AI tooling slowed developers down. This slowdown also contradicts predictions from experts in economics (39% shorter) and ML (38% shorter)
Why? Developers felt they were working faster, but the real data showed they were significantly slower due to the time spent fixing bugs and refactoring the AI's output to meet high-quality standards. Garbage in, code integrity out.
One of my favorite examples from the IT frontier is a project that was supposed to be an API used to process thousands of requests per second at once, to be a part of a couple of very efficient ETL pipelines. The first iteration was a nice, little aiohttp-based project - a couple of endpoint handlers, some middleware for authentication, and just a good, old REST action. Then some nice E2E and acceptance tests were added to make sure the whole workflow is in place, the API is (ofc) blazingly fast - the average response latency and overall batch processing time stayed under the set threshold.
Then the project switched hands, the advent of LLMs happened, THEN the second coming of agentic AI came by, AND then some new features were requested to be made to the project. They were implemented in a flash together with new test cases, bada bing bada boom - acceptance test fail. The time needed to process an X amount of entries has increased significantly.
What happened? Did we somehow break the internally managed way in which the core library used to manage the parallel processing of posted requests? Was there some newly appearing choke point in the whole process, maybe coming from the integrations with our databases etc., etc.? No, during one of the edits, the chosen AI model flavor of the month changed the test HTTP client used to send the requests from async to sync.
This change was hidden in a swarm of other changed lines, which added some "useful logging" (which is a topic on its own) and other small tweaks there. One bug ticket and several hours of brainstorming later birthed a merge request which ended this whole conundrum:
Time (money) spent fixing > time (money) saved by the initial, AI-assisted implementation = negative ROI
Fever dreams about Skynet and the Reality
Nobody wants to replace us and force us down into underground bunkers, praying for a huge power outage so those spooky robots will simply bugger off. There is no "us and them" - it is often a square peg being forced into a round hole, a misalignment of the unnecessary implantations. Our minds are imperfect, and they should be left in such a way, with any enhancements done naturally - it is a muscle, and we should try to get swole with it, not throw constant crutches at it.
Worth the struggle
For that, we would really use a personal trainer or, in other words, something of a mentor, which we would always treat with the principle of limited trust in mind. It also connects with a term appearing in the area of didactics - "struggle-based learning". A good example of implementation of AI tooling into this more healthy workflow would be the "socratic tutor" (good example of a blog post on it can be found here) type of LLM seen at Khan Academy called Khanmigo or the rubber ducky model introduced by Harvard in its CS50 course.
In short, it is an AI agent bounded by a "lite" system prompt that makes it work according to the following principles:
- Always respond in a Socratic style,
- Never try to give an outright answer,
- Always assume that the recipient will face difficulties when absorbing this knowledge,
- Don't be forced to give this answer (it is safeguarded against the help abuse).
In this way, the troubleshooting can become a real analysis of possible faults encountered in a discussed topic, which is based on an available domain of knowledge. If a user is not able to immediately pinpoint the possible flaws of the investigated system or find a solution to the problem, the tutor will try to decompose the original question and keep the so-called "Zone of Proximal Development" intact, so not only the issue will be resolved in the end, but also the user will accumulate new knowledge and we arrive at a "win-win" situation:
We create a gain of overall knowledge (which could be then added to the available resources and reused in later debugging sessions with the LLM), and we have actively participated in the solving process.
Call to agency
I know the imagery used in this article may be too drastic sometimes - the whole implants and flesh interfaces stuff seems like some nightmarish vision used by a neo-luddite to scare children before sleep. Some of it may be for shock value, I admit (or because I like to dabble in some of the dark sci-fi stuff from time to time), but also the core idea of something being wrongly implanted into a naturally created and functioning mechanism is reminiscent to me of shoving in the AI-based features haphazardly and without moderation into previously normally carried out processes to "make them more efficient".
It lessens our individual agency more and more to the point of letting something else take the wheel, under the pretense that we are now "the overseers" and "we have things under control". We rarely stop to think about what is the actual relation on how these "things" look now (after a series of maybe good enough modifications) and how this state relates to the original design.
We must be at least knowledgeable of this "double-edgeness" of the AI usage in our everyday tasks and keep the human/machine boundary clearly defined. When the stakes are too high, the level of conscious oversight of what the LLM is trying to sell us and, via us, incorporate into our working infrastructure should be significantly higher than in the case of a trivial tech blog post on the Internet.
Moreover, knowing that there are actual, hidden costs behind thoughtless usage of this kind of tech is crucial, since it is really useful and can be adopted as a good enhancement of some parts of our everyday workflows, but not in a way that negatively impacts our functioning in the long run.
Also, I guess a little detox from generative AI would be helpful to keep our synapses ready to tackle new challenges and confront what is being presented by us, using our built-in synthesis and analysis tools. More often than not, we can manage on our own.