Bamboo Agile | Custom Software Development Company
Bamboo Agile is an Estonia-based custom software development company that crafts bespoke solutions for telecom, education, healthcare, finances, other sectors.
In Part 1 of this article, we explored the benefits of adopting AI in software development, highlighting the most popular tools based on insights from our technical experts, industry research, and an analysis of feedback from 2,000 Reddit developers.
In Part 2, we’ll examine the potential downsides of using AI tools in development, including data security risks, concerns about system transparency, and their impact on DevOps metrics.
Introduction
Despite the widely discussed advantages of AI tools across many areas of software development, recent research reveals a steadily growing wave of skepticism. According to the 2024 study “I Don’t Use AI for Everything,” the majority of surveyed developers consider AI tools unreliable, citing poor reasoning capabilities, high hallucination rates, and a tendency to “memorize” rather than “generalize”.
The 2024 Stack Overflow global survey expands on this sentiment: only about 43% of developers say they trust their output. The percentage of those with a generally positive attitude toward AI tools also dropped, from 77% in 2023 to 72% in 2024. Additionally, 45% of professional developers believe AI tools don’t perform well on complex tasks.
Analysts and research institutions share these assessments. While AI can accelerate development and reduce cognitive load in low- to mid-level tasks, its uncontrolled use, especially without architectural constraints or clear guidelines, often leads to mounting technical debt and other downstream issues. This risk is particularly pronounced in scalable systems or when AI-generated code is deployed without rigorous human oversight.
Many developers share these concerns on Reddit, citing real-world issues with the quality, maintainability, and long-term impact of AI-generated code.
In the following sections, we’ll take a closer look at their assessments.
Considering AI but have doubts?
Contact us – we can turn early AI experiments into stable, production-ready software
Software insecurity from overlooked code errors and data leak risks
In 2024, nearly 80% of the 1,000 CIOs surveyed by Gartner reported concerns about security risks associated with adopting AI in software development. The shift in sentiment is significant: just a year earlier, only 23% were classified as AI skeptics.
Among the most pressing security issues cited in the new report were the challenges of managing AI use responsibly (72%) and data privacy (71%) – the latter being particularly relevant in light of the well-known Samsung case. Many also flagged a growing shortage of qualified talent capable of using AI effectively.
A research team at Stanford University, meanwhile, found that participants using AI assistants were significantly more likely to write insecure code than those in the control group in four out of five assigned tasks. Most of the AI users also overestimated the security of the code they produced.
Meanwhile, Deloitte reports that current DevSecOps frameworks show a marked lack of preparedness for AI integration, particularly in regulated and risk-sensitive domains such as healthcare and finance.
To illustrate, Reddit user ShelBulaDotComposted a strongly worded comment about AI-powered low-code development. She described the ongoing need to fix security flaws in such products, including numerous cases where passwords were stored in plain text on the client side. One real-world example involved a Facebook login button that didn’t actually authenticate users but instead collected usernames and passwords for over a year. “High time for data theft, though!” the commenter concluded.
“I have yet to see a copilot spit out stuff that doesn’t need to be examined carefully. About 50% of the time, it needs correcting,” wrote another user, tazebot, in a discussion titled “AI Is Making Us Worse Programmers.” “The biggest problem I see is that it produces code that at first appears very credible, tempting inexperienced coders to just ‘take it’.”
The imbalance between simplicity and complexity in AI-generated code
AI tools can produce code that is either overly simplistic or excessively complex, inconsistent, and non-standard. Attempting to automate such “difficult” code rapidly can lead to both technical debt and overengineering, which may weigh down the codebase in the long term. As these codebases grow exponentially, they risk compromising system stability under increased load, ultimately leading to financial losses.
Overly simplistic AI-generated code, lacking the necessary “margin of safety,” also poses hidden risks that tend to surface under stress. For example, code that performs well on local machines may fail in production environments under high user loads and frequent API calls, potentially resulting in outages.
According to Deloitte’s report, the success rate of AI-generated code is closely tied to prompt complexity: simple prompts succeed about 90% of the time, while complex ones drop to around 42%. Less experienced developers are less likely to recognize this imbalance compared to their more seasoned peers. Several studies have also noted a decline in the effectiveness of AI tools, particularly ChatGPT, in generating functional code over time. Citing IEEE research, Deloitte reports that earlier versions of ChatGPT trained on 2021 data outperformed those trained on 2023 datasets, especially on non-standard tasks. Success rates dropped from 89% to 52% for simple tasks and from 40% to just 0.66% for complex ones, making manual code review a necessity, not an option.
Lack of standardization and unpredictability in AI output
While LLMs and similar tools can generate the expected templated output with precise prompts, it’s common for team members on the same project to receive up to 20 distinct variations of what is essentially the same response. This variability complicates the testing and evaluation of generative AI systems.
Trained prompt engineers may help address the issue in the future, though achieving fully standardized outputs is unlikely, and such expertise is still in short supply.
Reddit user kibblerzshared his own experience using AI for code generation. Despite his skepticism, he tried “vibe coding” under pressure from a former manager who criticized him for relying on traditional dev tools instead of AI-based solutions.
The post’s author shared that trying to build a React drawer menu took 50 cents in credits and “it was highly problematic.” Any libraries that had changed after the model’s training data collection were “a mess” and made for “altogether a very bumpy process.” According to kibblerz, all their routine work could have been automated using a few VM Macros, Grep, and Regex “in a sequential fashion that’s under full control.” In the end, the user called this vibe coding “a nightmarish dystopia.”
“In my view, a major issue is that AI doesn’t understand the coding standards specific to a company or project. Code isn’t written in a vacuum; it’s written for real clients and often follows concrete agreements. For example, a project might require all variable names to begin with an abbreviation that reflects the project name. So, while the AI-generated code might not be inherently bad, it has no awareness of such conventions.
Another issue is the high number of false positives when AI tools review code. They might report ten issues, and three of them aren’t actually errors but intentional deviations from generic standards with a specific purpose.”
Architecture turns into a black box and loses the “human touch”
As GenAI becomes increasingly integrated into the workflows of product and engineering leaders, helping codify business requirements, standardize design, support knowledge transfer, and even guide architectural standards, experts are warning about the risks of erosion of flexibility and creative thinking. There is growing unease that teams may end up stuck in a “vacuum” of repetitive patterns and decisions – the very ones on which AI models were trained.
On the business side, this might lead to inefficient but fixable strategies. On the engineering side, however, the consequences can be more serious: the architecture may gradually become a black box; it works, but no one can clearly explain how it does or predict what might happen if it’s changed. As a result, maintenance, scaling, and auditing become significantly riskier.
It’s worth mentioning that even cybersecurity experts have begun voicing similar concerns, warning that junior developers are increasingly struggling with deep system-level understanding – a trend they view as potentially hazardous for the field. “Many can generate functional code snippets but struggle to explain the underlying logic or defend them against real-world attack scenarios,” notes Om Moolchandani, co-founder and CISO/CPO at Tuskira, in a comment to CSO.
Analyzing 211 million lines of code from 2020 to 2024, GitClear researchers confirmed that since the introduction of coding assistants powered by AI, code quality, in measurable terms, had been on the decline. In 2024, for the first time on record, copied lines outnumbered moved lines – indicating a decline in refactoring practices. The share of moved code fell from 24% in 2020 to 9.5%, while copied code rose from 8.3% to 12.3%.The study also reported an eightfold increase in code blocks containing five or more duplicated lines. Of these, 57% of co-edited clones were associated with software defects, linking duplication to declining system stability. These findings align with Google’s DORA 2024 data report, discussed in the section below.
Interdependence of multimodal AI systems
Many companies now rely on publicly available large LLMs, which may increase the need for multimodal systems – solutions where multiple models or agents interact and share responsibilities, such as text generation, data processing, or visualization, depending on which agent performs each task most effectively.
In these setups, a critical error from one agent can propagate to another, potentially leading to system-wide failures or compromising data integrity.
According to Deloitte, organizations pursuing such strategies are already raising concerns about internal AI governance (62%), workforce training (60%), and auditability and compliance with regulations (55%).
DevOps: the paradoxical effect of AI on software delivery
Despite cautious optimism about AI in DevOps, some data points reveal a paradox: under certain conditions, AI adoption can reduce key performance indicators in areas where improvements were expected.
The Google DORA 2024 report, for instance, found that every 25% increase in AI usage was associated with a 7.2% drop in stability and a 1.5% decline in throughput.
These findings were unexpected, as improvements in areas such as documentation, code quality, and review speed have historically been correlated with higher performance. While these local aspects improved with AI (documentation quality rose by 7.5% and code quality by 3.4%), they had no measurable effect on overall delivery stability.
One possible explanation, the authors suggest, is a shift in the nature of releases themselves. Code autogeneration accelerates development but breaks with one of DevOps’s core principles: small, incremental iterations. The issue lies in the large code batches, which can increase the risk of failures even without factoring in typical AI-related errors and reduce deployment resilience.
It’s worth noting that AI demonstrates a positive impact at both the individual and team levels. According to the report, developer productivity increased by 2.1%, job satisfaction rose by 2.2%, team effectiveness improved by 1.4%, and overall company performance grew by 2.3%. However, product-level metrics, such as usability, latency, and security, did not show a statistically significant improvement.
As a result, the DORA authors conclude that while AI can optimize specific parts of the development process, it doesn’t yet guarantee improved DevOps performance overall, especially when its core principles are compromised. In particular, some developers on Reddit point out that in areas like orchestration, hallucinations and errors still make it hard to choose “1,000 hours of prompt engineering” over “a couple of hours and some coffee with an experienced engineer.”
Still, the report’s authors emphasize that the industry is only at the beginning of its journey with AI in DevOps and that many of its potential benefits (and drawbacks) have yet to fully emerge.
Maxim Leykin, CTO at Bamboo Agile
“I still have high hopes for AI in DevOps. Why? Because DevOps is highly standardized, it relies on a limited set of tools and technologies that can be assembled into a CI/CD pipeline, much like Lego blocks. Setting all of this up is often a repetitive routine that takes a lot of time: you forget something here, misconfigure something there, link Jira to GitHub incorrectly, and so on. These are exactly the kinds of tasks AI should be good at handling. This is exactly where I see AI making a real impact – and we’re already seeing early signs of it with tools like Copilot, GitHub Actions, and others. Eventually, I expect we’ll reach a point where you simply feed AI the input data and get a configured CI/CD pipeline in return.”
Conclusion
While AI tools will undoubtedly continue to unlock broader opportunities for accelerating and optimizing specific stages of development, their more systematic yet poorly thought-out adoption poses serious risks, from data security threats and architectural imbalance to the not yet fully understood impact on DevOps metrics.
In the next part of the article, we’ll broaden the scope and take a closer look at the social, ethical, and legal aspects of AI adoption – from tensions in professional culture and the risk of fragmentation within the tech community to concerns about research freedom and corporate monopolization.
And in the fourth part, we’ll present a practical guide on how to avoid the pitfalls of AI integration and strike the right balance between efficiency, control, and trust in AI tools.
Partner with a team that knows where AI helps – and where it hurts
Aliaksei Patapau is a writer with over 12 years of experience in digital communications and content production. For the past two years, he has been covering software-related topics such as product development, automotive and connected car technologies, and engineering team management.
He works closely with the company’s tech experts, grounding their insights in credible research and real-world case studies. This approach enables him to produce materials that are both analytical and practical – yet engaging to read.
Aliaksei’s work is published across the blogs and social media channels of Bamboo Group, Bamboo Agile, and Bamboo Apps.
We use cookies to analyze user behavior and improve the website for you. Check our Privacy Policy for more information.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
