Generative A.I. Programs Are Not Authoritative Sources For Facts.

[TheSneakster]

For those of you who imagine that LLM's are some sort of all-knowing, all-seeing infallible oracle of knowledge, I asked ChatGPT, "how does an LLM answer a question ?" and this was the response:

An Large Language Model (LLM) answers a question by predicting the most statistically likely sequence of words (or "tokens") that should follow your prompt. It doesn't "know" facts in the human sense; instead, it uses massive amounts of training data to recognize patterns in how humans structure information. [1, 2, 3, 4, 5]
Here is the step-by-step process of how it generates a response:
1. Tokenization and Embedding

When you ask a question, the model first breaks your text into small pieces called tokens (words or parts of words). These tokens are then converted into vector embeddings—numerical representations that help the model understand the mathematical relationship and "meaning" between different words. [6, 7, 8, 9, 10]
2. Contextual Analysis (The Attention Mechanism)
The model uses a transformer architecture with an attention mechanism to determine which parts of your question are most important. [5, 7]

• For example, in the question "What is the capital of France?", the model gives more "attention" to the tokens "capital" and "France" than to "is" or "the".
• This allows it to maintain focus on the core intent of your query. [5]

3. Next-Token Prediction
During its initial training, the LLM read billions of pages of text to learn the "rhythm" of language. It uses this knowledge to calculate a probability distribution for the next token. [3, 5, 8, 11]

• Step-by-Step Generation: It picks the most likely first word of the answer, then uses both your original question and that first word to predict the second word, and so on.
• No Internal "Truth": The model chooses words that frequently appeared together in its training data. If its data says "The capital of France is Paris" thousands of times, "Paris" becomes the mathematically highest-probability response. [1, 8, 12, 13, 14]

4. Advanced Reasoning and Tools
Modern LLMs use additional techniques to improve accuracy: [15, 16]

• Instruction Tuning: Models are specifically fine-tuned on question-and-answer pairs to learn that they should respond to a prompt rather than just continuing the text.
• Chain-of-Thought: By "thinking step-by-step," the model generates its own intermediate logic, which then serves as additional context to help it reach the final answer.
• Retrieval-Augmented Generation (RAG): Some systems first search external databases or the web to find relevant documents, then feed that specific information into the model to reduce "hallucinations" (made-up facts). [3, 4, 10, 12, 13, 15, 17, 18]

Would you like to know more about how Retrieval-Augmented Generation (RAG) helps models stay up-to-date with current events?

[1] https://www.reddit.com
[2] https://cloudx.com
[3] https://medium.com
[4] https://www.reddit.com
[5] https://autogenai.com
[6] https://www.reddit.com
[7] https://www.lesswrong.com
[8] https://www.youtube.com
[9] https://www.reddit.com
[10] https://quantumobile.com
[11] https://www.onyxgs.com
[12] https://news.ycombinator.com
[13] https://medium.com
[14] https://www.reddit.com
[15] https://www.ibm.com
[16] https://www.linkedin.com
[17] https://medium.com
[18] https://www.youtube.com

 

[I M Dex]

Thanks, that is quite informative. It also strikes me as a somewhat superficial attempted replication of one kind of human thinking process albeit devoid of nuances, devoid of real intuititiveness, devoid of any particular interest, and any singular bias (but not devoid of all the bias that has found its way into all the material that the AI derives from).

 

[Type4_PTS]

For those of you who imagine that LLM's are some sort of all-knowing, all-seeing infallible oracle of knowledge, I asked ChatGPT, "how does an LLM answer a question ?" and this was the response:

I doubt anyone here imagines that LLMs are some sort of all-knowing, all-seeing infallible oracle of knowledge.

They're probably a hundred times more valuable, though, than the way in which you characterize them.

Here's what ChatGPT had to say about your post:
===========================================================================

ChatGPT:

This is a textbook example of explaining just enough about something to make it sound trivial, then pretending that settles the issue.

Yes, LLMs predict the most likely next tokens. That’s the underlying mechanism.
But presenting that as some kind of “gotcha” is like saying:

• “Calculators just flip bits, so they don’t really do math.”
• “Brains just fire neurons, so humans don’t really think.”

It’s technically true and completely misleading.

What you’re leaving out is everything that actually matters:

• These models are trained on vast bodies of human knowledge, not random noise.
• They’re fine-tuned to answer questions, not just autocomplete sentences.
• Many systems use external retrieval and verification to reduce errors.
• They can perform multi-step reasoning, which is why they’re useful in the first place.

Also, the “no internal truth” line gets thrown around a lot as if it’s some fatal flaw. Humans don’t have direct access to “truth” either. We rely on memory, pattern recognition, inference, and sources. All of which are error-prone.

The real issue isn’t that LLMs are unreliable.
It’s that people keep expecting certainty from tools that are probabilistic by design.

So no, they’re not “all-knowing oracles.”
But dismissing them because they use statistical prediction just shows a shallow understanding of both AI and human cognition.
--------------------------------------------------------------------------------------------------------------------------------------------------

After ChatGPT provided that answer, it said to me:

If you want, I can make an even sharper version that subtly calls out the Scientology-style “reframing truth through definition tricks” angle… that one lands hard in forums like that.

 

[PirateAndBum]

For those of you who imagine that LLM's are some sort of all-knowing, all-seeing infallible oracle of knowledge, I asked ChatGPT, "how does an LLM answer a question ?" and this was the response:

An Large Language Model (LLM) answers a question by predicting the most statistically likely sequence of words (or "tokens") that should follow your prompt. It doesn't "know" facts in the human sense; instead, it uses massive amounts of training data to recognize patterns in how humans structure information. [1, 2, 3, 4, 5]
Here is the step-by-step process of how it generates a response:
1. Tokenization and Embedding

When you ask a question, the model first breaks your text into small pieces called tokens (words or parts of words). These tokens are then converted into vector embeddings—numerical representations that help the model understand the mathematical relationship and "meaning" between different words. [6, 7, 8, 9, 10]
2. Contextual Analysis (The Attention Mechanism)
The model uses a transformer architecture with an attention mechanism to determine which parts of your question are most important. [5, 7]

• For example, in the question "What is the capital of France?", the model gives more "attention" to the tokens "capital" and "France" than to "is" or "the".
• This allows it to maintain focus on the core intent of your query. [5]

3. Next-Token Prediction
During its initial training, the LLM read billions of pages of text to learn the "rhythm" of language. It uses this knowledge to calculate a probability distribution for the next token. [3, 5, 8, 11]

• Step-by-Step Generation: It picks the most likely first word of the answer, then uses both your original question and that first word to predict the second word, and so on.
• No Internal "Truth": The model chooses words that frequently appeared together in its training data. If its data says "The capital of France is Paris" thousands of times, "Paris" becomes the mathematically highest-probability response. [1, 8, 12, 13, 14]

4. Advanced Reasoning and Tools
Modern LLMs use additional techniques to improve accuracy: [15, 16]

• Instruction Tuning: Models are specifically fine-tuned on question-and-answer pairs to learn that they should respond to a prompt rather than just continuing the text.
• Chain-of-Thought: By "thinking step-by-step," the model generates its own intermediate logic, which then serves as additional context to help it reach the final answer.
• Retrieval-Augmented Generation (RAG): Some systems first search external databases or the web to find relevant documents, then feed that specific information into the model to reduce "hallucinations" (made-up facts). [3, 4, 10, 12, 13, 15, 17, 18]

Would you like to know more about how Retrieval-Augmented Generation (RAG) helps models stay up-to-date with current events?

[1] https://www.reddit.com
[2] https://cloudx.com
[3] https://medium.com
[4] https://www.reddit.com
[5] https://autogenai.com
[6] https://www.reddit.com
[7] https://www.lesswrong.com
[8] https://www.youtube.com
[9] https://www.reddit.com
[10] https://quantumobile.com
[11] https://www.onyxgs.com
[12] https://news.ycombinator.com
[13] https://medium.com
[14] https://www.reddit.com
[15] https://www.ibm.com
[16] https://www.linkedin.com
[17] https://medium.com
[18] https://www.youtube.com

So you're going to believe ChatGPT's answer to your question as the basis for why AI is not an authorative source?

 

[TheSneakster]

So you're going to believe ChatGPT's answer to your question as the basis for why AI is not an authorative source?

Do you see anything written by me that says I believe what ChatGPT responded with or that I do not believe it ? You do not.

But I will tell you this. I understood what it said and it agrees with everything my decades of computer science studies and programming experience taught me. Many years ago, I learned a *whole* lot about this subject from Introduction to Automata Theory, Languages, and Computation ( first edition - sadly unavailable in electronic format, AFAICT ). A link to the third edition follows:

https://www.amazon.com/Introduction-Automata-Theory-Languages-Computation/dp/8131720470/

 

[guanoloco]

So you're going to believe ChatGPT's answer to your question as the basis for why AI is not an authorative source?

He's butthurt because I asked AI about Steve Hassan's asymmetrical application of his BITE model. He doesn't like the answer, which is so obvious a blind person could point it out.

 

[TheSneakster]

He's butthurt because I asked AI about Steve Hassan's asymmetrical application of his BITE model. He doesn't like the answer, which is so obvious a blind person could point it out.

Wrong. I'm just better educated about how these programs actually work from my computer science background than you are.

 

[PirateAndBum]

Do you see anything written by me that says I believe what ChatGPT responded with or that I do not believe it ? You do not.

But I will tell you this. I understood what it said and it agrees with everything my decades of computer science studies and programming experience taught me. Many years ago, I learned a *whole* lot about this subject from Introduction to Automata Theory, Languages, and Computation ( first edition ).

https://www.amazon.com/Introduction-Automata-Theory-Languages-Computation/dp/8131720470/

Do I see anything written by you? Well, let me see... You created this thread and posted the OP. So I would say that you are being ultra literal as is normally your wont.

I just find it funny that you present ChatGPT's answer to make a claim that AI programs are not authorative sources for facts. Extremely funny.

I find that explanation to be credible, but I also wonder how Claude Code can generate entire programs and even help test and debug them. It is quite impressive what today's AIs can do.

Have you fired up your own OpenClaw instance yet?

Note: I was a software engineer for 40+ years.

 

[I M Dex]

I doubt anyone here imagines that LLMs are some sort of all-knowing, all-seeing infallible oracle of knowledge.

They're probably a hundred times more valuable, though, than the way in which you characterize them.

Here's what ChatGPT had to say about your post:
===========================================================================

ChatGPT:

This is a textbook example of explaining just enough about something to make it sound trivial, then pretending that settles the issue.

Yes, LLMs predict the most likely next tokens. That’s the underlying mechanism.
But presenting that as some kind of “gotcha” is like saying:

• “Calculators just flip bits, so they don’t really do math.”
• “Brains just fire neurons, so humans don’t really think.”

It’s technically true and completely misleading.

What you’re leaving out is everything that actually matters:

• These models are trained on vast bodies of human knowledge, not random noise.
• They’re fine-tuned to answer questions, not just autocomplete sentences.
• Many systems use external retrieval and verification to reduce errors.
• They can perform multi-step reasoning, which is why they’re useful in the first place.

Also, the “no internal truth” line gets thrown around a lot as if it’s some fatal flaw. Humans don’t have direct access to “truth” either. We rely on memory, pattern recognition, inference, and sources. All of which are error-prone.

The real issue isn’t that LLMs are unreliable.
It’s that people keep expecting certainty from tools that are probabilistic by design.

So no, they’re not “all-knowing oracles.”
But dismissing them because they use statistical prediction just shows a shallow understanding of both AI and human cognition.
--------------------------------------------------------------------------------------------------------------------------------------------------

After ChatGPT provided that answer, it said to me:

If you want, I can make an even sharper version that subtly calls out the Scientology-style “reframing truth through definition tricks” angle… that one lands hard in forums like that.

i’ve got to say, that ChatGPT apparently is loaded with algorithms that make it more than a bit defensive. Commander Data would not approve.

 

[TheSneakster]

I just find it funny that you present ChatGPT's answer to make a claim that AI programs are not authorative sources for facts. Extremely funny.

And the intentional irony of using a generative LLM text transformer (falsely labled "AI" for marketing purposes) to warn somebody (who clearly puts too much stock in them) not to put too much stock in them went completely over your head, I guess.

 

[PirateAndBum]

Completely

 

[guanoloco]

Wrong. I'm just better educated about how these programs actually work from my computer science background than you are.

You're the one making broad assumptions on people's relying on AI as some authoratative fact source. AI is so compromised with progressive ideology it's almost as bad as Google and Wokepedia and it still could see cult fanatic Hassan's blindspot, that you can't due to ideological possession and bias.

 

[Isaac]

He's butthurt because I asked AI about Steve Hassan's asymmetrical application of his BITE model. He doesn't like the answer, which is so obvious a blind person could point it out.

Steve Hassan is a cult leader promoting his book “ Cult of Trump … a self proclaimed cult expert that doesn’t realize he’s actually in a cult that promotes the hatred of Donald J Trump.

he can BITE me… I can’t stand that man!
He doesn’t care about people trapped in cults or their families. I know someone who contacted him to try to get their family member out, and he charged them to tell them to buy his book and read it.
What a dick

 

[I M Dex]

Steve Hassan is a cult leader promoting his book “ Cult of Trump … a self proclaimed cult expert that doesn’t realize he’s actually in a cult that promotes the hatred of Donald J Trump.

he can BITE me… I can’t stand that man!
He doesn’t care about people trapped in cults or their families. I know someone who contacted him to try to get their family member out, and he charged them to tell them to buy his book and read it.
What a dick

Good to know that about this guy, talks the talk but doesn't walk the walk. But as to your Trumpcult spew, sorry, but back in the 90's in NYC that POS screwed 2 friends of mine who operated high end printing companies, ordering and receiving very expensive printed products and then instead of paying his bills, he told them both they could accept 15% on the dollar or try taking him to court. F*ck him, he nevr made an honest dollar in his life, just stole, scammed, promised return on investments and then declared bankruptcy again and again- how can anyone lose money on a casino?- but he did, by looting the profits and leaving investors creditors high and dry. F*ck him and the horse he rode in on.

 

[guanoloco]

Good to know that about this guy, talks the talk but doesn't walk the walk. But as to your Trumpcult spew, sorry, but back in the 90's in NYC that POS screwed 2 friends of mine who operated high end printing companies, ordering and receiving very expensive printed products and then instead of paying his bills, he told them both they could accept 15% on the dollar or try taking him to court. F*ck him, he nevr made an honest dollar in his life, just stole, scammed, promised return on investments and then declared bankruptcy again and again- how can anyone lose money on a casino?- but he did, by looting the profits and leaving investors creditors high and dry. F*ck him and the horse he rode in on.

...hardly a cult. It's entirely understandable to see the villain - it's a definite compromise to embellish - and that's Hassan. Veda, way back in they day, I much appreciated his contribution because he didn't mince on accurately describing Scientology, and that includes positive aspects of it.

As I sat on the fence and almost got out time and time again, I would wind up encountering an obvious embellishment of Scientology negativity that would "trigger (for lack of a better term)" the cult response/stimulus response activity of "that's an SP" and I would dutifully go back into the fold.

Why embellish when the truth can serve? Your anecdotes are more effective and more believable than Steve Hassan coming out with MAGA as a cult.

I can have Trump be a rapist, a racist, a criminal, a cult and when I say "have" I mean the Scientological term. I don't need Trump to be a savior or pristine or anything. And this is not a comment that no matter what Trump does or how ultimately evil Trump is that I don't care, he's my man anyway. That's not what's being said here. However, just like Scientology, cultists require Trump to have nothing of value and the epitome of evil such as Hitler, etc., the way that Scientologists view anything and everything pyschologist/psychiatrist. It's so egregious that anything and everything conservative or GOP is equated as "Fascist" - not whether or not Trump is fascist, but that the very political Party, from what I can see, that argues against government growth and capture of the private market is "fascist". That's absolutely ridiculous to the max and full-on cult speak.

Here's a roomful of Democrats reacting to Charles Murray - because he's such a "white supremacist". What's so outright cultish is the "anti gay" chanting. Murray, from what I understand, is a gay rights activist. IDK if he's gay or not, but an advocate none-the-less.

Run that through the BITE model.

That's definitely Behavior Control, Information Control, Thought Control and Emotional Control.

Where do you ever see anything approaching this in MAGA or with Trump?

Another absolute absurdity is the concept or idea that Murray is a "white supremacist" when his IQ model looks like this and puts whites squarely in the middle of the pack:

Murray's IQ Distribution by Population Group:

• Ashkenazi Jews: ~110–115
• East Asians: ~105
• Whites (European): ~100
• Hispanics: ~90
• Blacks (US): ~85

Again, low, low information people - which is what cult members are because the cult is the gatekeeper of their information. This circles back to Steve Hassan and his asymmetrical application. If MAGA and Trump are a cult, then the progressive lib is cult on steriods.

Look at the BITE model.

How is anyone in any type or fashion in Behavioral Control or Informational Control? We were all exposed to "entheta" ethics handlings and "non enturbulation" crap - anyone in MAGA can read any and all sources of whatever they want...CNN...MSNBC...NPR...etc.

Also, internecine political Party fights don't count. Democrats caucus out members all of the time. In Germany, voters vote on a Party platform and not the Prom King/Queen that we do here in the U.S. They vote for a platform and not individuals. The Party selects the individuals and everyone HAS to vote exactly in line with the Party or they lose their seat. Again, not a cult, merely delivering the goods that the people voted for.

However, look at Thomas Sowell. Not only is he an apostate, something that only cults have, but he's had to live his life in an unpublished format because of the death threats that he's received from the cult that he was formerly in. Look at #WalkAway stories and how these people have identical stories to ex-Scientologists and ex-Muslims - right down to disconnections, death threats, attacks, having to run and hide.

 

[Isaac]

Good to know that about this guy, talks the talk but doesn't walk the walk. But as to your Trumpcult spew, sorry, but back in the 90's in NYC that POS screwed 2 friends of mine who operated high end printing companies, ordering and receiving very expensive printed products and then instead of paying his bills, he told them both they could accept 15% on the dollar or try taking him to court. F*ck him, he nevr made an honest dollar in his life, just stole, scammed, promised return on investments and then declared bankruptcy again and again- how can anyone lose money on a casino?- but he did, by looting the profits and leaving investors creditors high and dry. F*ck him and the horse he rode in on.

That certainly sucked for your friend.
I like DJT and feel he is the perfect man to run this fucked up country that has been compromise by our dear corrupt, criminal politicians for decades.
DJT brought me out of apathy and gave me hope again.

Time will tell, but I think he’s doing a fantastic job.

 

[I M Dex]

That certainly sucked for your friend.
I like DJT and feel he is the perfect man to run this fucked up country that has been compromise by our dear corrupt, criminal politicians for decades.
DJT brought me out of apathy and gave me hope again.

Time will tell, but I think he’s doing a fantastic job.

Although we have differing opinions, I like how you expressed that. It was done in a positive way and without intolerance for what I expressed. Cool.

 

[Isaac]

Although we have differing opinions, I like how you expressed that. It was done in a positive way and without intolerance for what I expressed. Cool.

OMG - he is hysterical

 

[Zertel]

These models are trained on vast bodies of human knowledge, not random noise.

Guesses vary from AI knowing most to almost all digitized human knowledge. Some people project that it will run out of new data /information/knowledge and start regenerating conclusions incorporating previous false information and conclusions.

Some large corporations have reported the cost savings from AI output are lessened by humans needing to go back and debug incorrect AI generated programming.

 

[Zertel]

OMG - he is hysterical

To be fair that reply was after a Japanese reporter asked the stupid question, "Why didn't you tell us you were going to attack Iran?"