"On the Dangers of Stochastic Parrots" (doi:10.1145/3442188.3445922) is a great introduction to this, even with its flaws.

I also recommend Mitchell 2023 (doi:10.1073/pnas.2215907120) and Niven 2019 (arXiv:1907.07355) as good starting points.

These don't directly address your question, but within the context of these papers, it's possible to see how the weighting of prose (which is a relation to prompts) and code can become skewed easily.

speaking of fzf, i have an alias for checking out branches using it + git:

  alias gbs="git branch --sort=-committerdate | fzf | xargs -I{} git checkout {}"

No. Humans can have actual domain knowledge plus contextual awareness which leads them to actually understand the subject by means of their education, and thus make guesses based on more than linguistic and syntactic plausibility. Educated guesses can be wrong, but are by definition not merely "plausible sounding nonsense."

For low level software architecture, at the level of the firmware that drives processors (plus the electronics that are below that), as NonEUCitizen recommended MIT's 6.004 is a good bet (http://ocw.mit.edu/courses/electrical-engineering-and-comput...).

For the medium level, assembly language and on up a fair bit (at least to C), CMU's got what looks like a great course and book: http://www.cs.cmu.edu/~213/ There's enough meat on the web page for the course that it should give you an idea if you want to get the book.

For the higher levels, in addition to the MIT Computer System Engineering courses mentioned by Mithrandir (http://news.ycombinator.com/item?id=1850311), especially 6.033, check out this classic from the '70s: Lions' Commentary on UNIX 6th Edition, with Source Code (http://en.wikipedia.org/wiki/Lions%27_Commentary_on_UNIX_6th...).

Me and a lot of other people learned the basics from it, and while it's certainly dated, it has the advantage of being lucid and complete while not being too big or overwhelming, and of course nowadays if you want to go so far as running a Version 6 UNIX there are PDP-11 simulators that you can use for the purpose. I.e. it's a good foundation, like SICP is for languages and TAOCP is for algorithms (although there are less detailed and difficult books to get you started on algorithms that you might want to try first).

Good luck and enjoy!