How will AI affect software development?

Logistics, pharma, climate research, HR–AI is everywhere. Will it soon eliminate software engineers? Here is the answer!

The AI landscape is bursting with competition, and coding-oriented solutions are proliferating. These can be roughly categorized into two groups: Chatbots and coding assistants.

While the former (like OpenAI’s ChatGPT or Google Bard) enables anyone to transform text to code or vice versa, the latter integrates into a code editor and suggests snippets while you type.

GitHub Copilot is the most popular among them. It integrates into leading code editors/IDEs and supports over a dozen frontend, backend, and infrastructure programming, markup, and templating languages.

Under the hood, it uses the OpenAI Codex–the same LLM (Large Language Model) as ChatGPT, which was trained chiefly on codebases stored in GitHub.

We first took GitHub Copilot for a test drive in the summer of 2022 when it was still in beta. Overall, more than 1.2 million people participated in Copilot’s year-long technical preview, trying to determine if it’s truly a game-changer that redefines the industry or just a glorified autocomplete add-on.

Since then, our Software Engineering team has regularly used AI, experimenting with potential use cases and assessing its maturity in a business environment. The findings indicate that developers can safely integrate ChatGPT into their toolbox with the necessary supervision.

Let’s dive in.

Coding challenges

To warm up, we checked if ChatGPT could pass our tech interview. We asked theoretical and practical questions designed for senior engineers and found that it answered them perfectly, albeit more verbose than expected.

ChatGPT also completed one of our coding challenges spectacularly, documented the iterations it created, and came up with good variable naming (which we all know is the biggest challenge in coding).

However, the unit tests we asked for had mixed results, requiring further human intervention and prompt refinement: In one of the more advanced tasks, we had to explicitly tell ChatGPT to include a critical step it omitted, simplify the code, and optimize performance issues.

The most complicated test assignment resulted in buggy code; after requesting ChatGPT to fix it, it hallucinated and generated new bugs instead of fixing the original one. The test was still useful, though, because refactoring the faulty solution was much quicker than writing it from scratch.

The final challenge was feature development. Our prompt was straightforward: Create a JavaScript command line script that calls the Instagram API of a user and stores the data in a database table.

ChatGPT provided detailed answers, describing a robust database model and anticipating potential pitfalls.

Santa's Little Helper

Our ongoing collaboration with the AI resulted in greater efficiency, particularly when it comes to creating tests and repurposing boilerplate code.

The prompt affects the quality of the generated output: ChatGPT is probabilistic, meaning prompt iterations produce distinct results, and the more detailed the instructions, the better.

We’ve also learned that current AI tools are inherently vulnerable–to hallucinations, lack of context, and poor training data. Since the code quality decreases as the complexity rises, knowing what you need is critical, and reviewing the output to ensure it’s free of bugs, performance issues, or overly complicated code is mandatory.

Here are a few things you can already use AI for today:

• Add unit tests to a project
• Explain and audit legacy code
• Document new and existing codebases
• Create quick MVPs

The future of software engineering

Software engineering will undoubtedly metamorphose as AI evolves–and it does. Fast. Professional programmers would move from writing code to reviewing, optimizing, and maintaining it.

AI will become instrumental in the development workflow, and tools that are well-integrated with IDEs will grow more popular, helping developers generate tests, document and explain code, and save time wasted searching external sources like Google or Stack Overflow.

Software developers who will take the time to adopt these new capabilities and adapt to this new paradigm will gain an unexpected benefit: As coding gets easier and more accessible, programmers could spend less time in their IDE and shift their focus to developing other indispensable skills, like communication, empathy, and teamwork.