Four months ago, we asked Are LLMs making Stack Overflow irrelevant? Data at the time suggested that the answer is likely “yes:”
You must log in or register to comment.
Ever ask a question on SO? I tell my students to search there but never, ever ask a question. The unmitigated hostility is not what new developers need or deserve. ChatGPT won’t humiliate you for asking a question that someone else has already asked.
Problem being that someone else asked the question 10 years ago and the answer is now irrelevant due to version changes. People with high scores are just early adopters who answered all of the easy questions. Hostile users generally can’t understand the question. The issue with llms answering your question is that they are going to be stuck in the current time period. In the future their answers will also be irrelevant due to version changes.
Earlier today I googled how to toggle full screen in dosbox-x and the AI-generated answer said to use alt+enter. Tried it and it didn’t work, so I look in the documentation and it turns out that they changed it to F12+f a while ago (probably to avoid interfering with actual dos input).
This is definitely already a problem.
I mean that is already a problem, if you ask a question you have to be ready for the answer to be a mismatch of version conflicts.
But that is ok. ChatGPT is a tool that can either help you or hurt you. I like to think of it like a power hammer. If you are doing a roofing job, it can help you get things done faster compared to a manual hammer, but you still need to know how to build a roof to get started.
ChatGPT is great at helping you organize your thoughts or finding an answer to some error message buried in some log file, but you still need to know what questions to ask and you need to be ready for it to give you a stupid answer and how to get around that.
I see this hot take often, and it isn’t entirely without merit, but it is mitigated by moderation; in some Stack communities better than others. I’ve been an active member for many years, and in my view it goes like this.
If you contribute a question without reading the rules and How to Ask a Good Question, you don’t provide minimal reproducible steps with code, post images of code, etc. you may get flamed out of town. And that may feel bad and it may be mean if the questioner didn’t know to read those. But they are there for you.
If, however, you ask a thoughtful question, give examples, show what you’ve tried, etc. you definitely can get quality, courteous help.
Doesn’t change that video killed the radio star here. The show is over.
Beginners are the least likely to ask thoughtful questions. We include slides in lectures about how to ask a question, but when there’s an assignment deadline and you’re inexperienced, it’s more likely you’re going to just blurt out “help me!” rather than provide a detailed explanation that doesn’t require repeated prompting. It takes time to learn how to work through an issue yourself before asking. Students are often facing time pressure and that can drive bad behavior. Correcting them is important, just don’t do it in a way that crushes their spirit.
100% understood and agreed. I don’t want to defend the bad behavior. It is out there among questioners and in the experienced community alike. Just saying it is possible to find quality help there.
I’ve asked questions on S.O. I’ve answered some too.
What I’ve found works well on s-o is
- Researching a bit first
- Asking a question properly*
- Including that search attempt to prove you’ve done some due-diligence
I’ve found even a dick like me can get a lot of leeway by showing I’ve put in the effort and asked properly.
*Same as Usenet
Make no mistake. LLMs aren’t killing stackoverflow. LLMs just arrived to finish it off. The stuff that was killing it are the regular posters there, and their passive aggressive bullshit
I never asked a question, despite using it daily. Too afraid of being berated 😅
Yup. I once decided to spend an afternoon answering questions on a framework I was expert in, as a kind of profile-building exercise to help with job hunting, and after around the third smug self-satisfied comment picking me up on some piece of irrelevant bullshit I deleted my account.
I’m not convinced that the number of questions asked is the correct metric. In the end the point is not to have a constant flow of questions, rather constant flow of answers found.
There is a point in proficiency in language/library/whatever after which it is faster to find the answer in the code/documentation/test example than to wait until another person on even higher level will come and answer your question.
Maybe we simply filled out what was needed to be asked in the beginner-bug found-intermediate space and, apart from questions stemming from new versions etc, SO does not need more questions?Expectation for everything to constantly grow is unrealistic
As more and more libraries are open source on GitHub or gitlab or sourceforge or whateverthefuck, asking questions on the libraries themselves (as an issue) is often the right thing to do, too… Less centralised than SO but also the only people who care about how to do things in a lib are people using the lib, so…
Sucks because I prefer stack overflow in searches because I get more of a human explanation and wisdom. With llm i have to figure out what it’s_trying to do_ , debug it, and god forbid you want various ways of doing the same thing. I hate LLMs for coding. I hate clients for trying to force me to use it when most of the time now they admit they’re hiring me because AI failed in the first place
It’s not dead until I stop getting 10 year old outdated answers in my searches!
This is interesting because a huge amount of AI “knowledge” comes from stack exchange.
Now I’ll go read the other comments and article to see if that’s already been mentioned :)
Anyone remember experts-exchange?
I remember when it didn’t have a dash. Until people started making fun of the old URL…
So easily avoided too
Not terribly surprising, Google would often direct me to StackOverflow threads as I was googling for an answer to a question. And as often as not, either the question was closed; or, instead of anyone providing an answer, the commenters would spiral off into questioning everything about the original question asker’s life choices. While I do get the whole XY Problem, this sort of thing seemed to be over-used on SO.
Granted, I don’t know if AI answers are any better. Sure, they can answer a lot of the simple questions, but I’ve not seen them be useful on hard, more obscure questions. Probably because those questions don’t have ready answers on SO.
I’ve lost count the number of times where I try to find something in SO, and it’s just someone posting the exact same example code as the answer. Or someone suggesting you just google it. Then I ask ChatGPT… and I get an answer.
Like it or hate it (personally I prefer the latter, posting there I felt like a middle schooler with a PUNCH ME sticker on my face) it was a great source of indexable data on programming.
I wonder how will this affect future search and llms, now that all similar questions are being asked in private llm threads.
I never once actually asked a question there. Partly because most of the time, the question I was asking had always been asked.
However, I have found the correct answer to 100s of questions there. Usually through google/ddg/kagi searches.
It will endure as long as the LLM’s on there know how to misinterpret the question and fire back snarky unhelpful answers about how clueless you are for asking in the first place.
SO is a collaborative encyclopedia of technical discussion that tries to be relevant, be practical, and to not constantly repeat topics.
LLMs can’t provide that structure, they just shit out answers.
Most people think SO is a help desk and don’t appreciate the structure and just want it to shit out an answer.
Maybe SO isn’t dying so much as a cancerous growth is being treated.
Ai haters out there, doesn’t this give a valid use case for LLM?
Not necessarily directly, many people may have abandoned learning programming because of LLMs, rather than Stack Overflow specifically.
I don’t think such trend would be so big. And anyone who has used any LLM for programming learns very quickly that those are very far from replacing anyone
For real. You can tell how good a programmer someone is, by how good they think an LLM is at programming.
I use it to bounce ideas around with or get it to direct me in the right direction if I am stumped for further research, but it will be a cold day in Hell before I have it write more than the most gruntiest of grunt boilerplate code. It just can’t do it to a useful standard without a lot of oversight.
Same, it’s largely doing pretty much as the article implies, replacing StackOverflow for when I need the correct runes to do something specific.
You’re pulling this out of your ass. That is completely made up.
