A marketing director reads a confident, well-written paragraph produced by a large language model and assumes it must be accurate because it reads so smoothly. A data scientist on her team explains that fluency and factual correctness are not the same thing for this kind of model.
What is the most fundamental reason a large language model can produce text that reads fluently but is factually wrong?