While GenAI has been dominating the tech news cycles for a while, does all the AI hype translate into usage for professional developers? We tapped the Stack Overflow community for answers (I hear they like answering questions). Over 1,700 people told us what code assistant tools they are using, how they feel about them, and whether they feel more productive as a result. We also asked when and how often CodeGen tools fall short, what challenges developers face with these tools, and what they are doing with all of the free time these tools purport to offer.
We found that most of those using code assistant tools report that these assistants are satisfying and easy to use and a majority (but not all) are on teams where half or more of their coworkers are using them, too. These tools may not always be answering queries accurately or solving contextual or overly specific problems, but for those that are adopting these tools into their workflow, code assistants offer a way to increase the quality of time spent working.
The majority of respondents (76%) let us know they are using or are planning to use AI code assistants. Some roles use these tools more than others amongst professional developers: Academic researchers (87%), AI developers (76%), frontend developers (75%), mobile developers (60%), and data scientists (67%) currently use code assistants the most. Other roles indicated they are using code assistants (or planning to) much less than average: data/business analysts (29%), desktop developers (39%), data engineers (39%), and embedded developers (42%). The nature of these tools lend themselves to work well when trained well; a tool such as GitHub Copilot that is trained on publicly available code almost definitely shall be good at JavaScript for frontend builders and never so good with enterprise and proprietary code situations that enterprise analysts and desktop builders face usually.
Once we requested what CodeGen instruments obtained improper, builders relayed they battle with context, complexity, and obscurity. For instance, one person wrote that code assistants have hassle with “[m]ost high-level or architectural questions. Or questions concerning area of interest elements, or assets behind NDAs”. One other person wrote that their coding device of selection supplied incorrect responses when working with lesser recognized “programming instruments (or instruments of [a] particular language).”
There are extra code assistant instruments being launched on a regular basis, although two are dominating the area to this point: ChatGPT and GitHub Copilot. Skilled builders and people studying to code are equally prone to be utilizing ChatGPT, however these studying are much less prone to be utilizing GitHub Copilot (29% vs. 49%). On condition that ChatGPT gives a well-liked free possibility and GitHub Copilot gives a time-limited free trial, this is sensible. Visible Studio IntelliCode is extra standard with these studying (16% vs. 11%), as there’s a free model of this IDE for these not utilizing an enterprise license.
Past simply grinding out code, GenAI instruments could also be a method to simply really feel good. The instruments which are extra satisfying to make use of additionally rank excessive for being straightforward to make use of: the top-box scores for best to make use of are Codeium (84%), GitHub Copilot (76%) and ChatGPT (61%) respectively and the identical three so as have been ranked for top-box satisfaction scores (86%, 72%, 65% respectively). In our 2023 Developer Survey, one third of respondents agreed that the top benefit of AI was increased productivity. Almost a yr later, do these straightforward, good time CodeGen instruments translate to precise productiveness? Microsoft reports that power users of AI tools at work are more likely to make use of them to spice up creativity and give attention to higher-level, strategic work. Somewhat assist from a CodeGen device may very well be what turns an exhausting day into an empowering day, even when the identical quantity of labor will get executed.
The truth of code assistants is that accuracy stays a difficulty: 38% of builders report code assistants present inaccurate data half of the time or extra. The inaccuracy doesn’t have an effect on satisfaction with code assistant instruments, however the productiveness reported for every does have a optimistic relationship with satisfaction and most customers (95% of these answering each questions) report “a bit” to “an awesome deal” of extra productiveness. In fact, productiveness stays an elusive metric: respondents indicated that almost all don’t know or are uncertain how their group measures productiveness. This lack of readability was extra pronounced for builders working in smaller organizations: 77% are uncertain of what productiveness metrics are for these in corporations of lower than 500 individuals, 72% in corporations of 500-4,999, and 62% in corporations for five,000+.
The method of adopting AI instruments into a fancy and considerate workflow will in the end have an effect on how helpful and satisfying these instruments shall be. We requested code assistant customers to inform us what’s stopping their workforce from adopting these instruments and respondents indicated an incapability to deal with the next degree of complexity and lack of belief within the instruments as the highest two causes. These on groups with decrease adoption charges cite the shortage of utilization as a self-perpetuating cycle: 11% of customers on groups with 50% or extra adoption say workforce members not utilizing code assistants is a problem for his or her workforce/firm to combine AI, whereas the next 19% of these on groups with lower than 50% adoption say lack of teammates utilizing AI is a problem. What’s extra, practically three quarters of our respondents (73%) point out they don’t know or are uncertain if their corporations have an AI coverage, which may very well be additional slowing adoption of code assistants inside groups as effectively. Within the meantime, these on groups with decrease adoption are indicating extra time is being spent on high-level technique or private initiatives whereas these on larger adoption groups are spending extra time on administrative duties or job-related coaching. The processes and methods should want time to catch up, and people ready to make use of these instruments with no course of in place can take pleasure in a bit extra time for creativity of their job.
The character of working as a developer is complicated and dynamic. Even when productiveness isn’t but clearly articulated as a enterprise KPI, builders are extremely glad with AI instruments which are straightforward to make use of and make them really feel productive, and groups are slowly starting to include these instruments into their workflows. Complexity and inaccuracy stay challenges in terms of widespread adoption and use of code assistants, however maybe builders on groups with decrease adoption charges may very well be utilizing the time they acquire from coding assistants to have interaction in additional inventive work. This additionally may very well be a turning level the place the long-term funding of an elevated expertise at work turns into extra vital than near-term enhancements on productiveness metrics.
We have now extra questions on AI and programming traits for builders that you could be be focused on weighing in on – Take our annual developer survey here while it is still open!
