I’ve seen someone ask ChatGPT to summarise a book. His own book. The summary turned out to be almost the opposite of what the book actually argued. Interestingly, the summary was very similar to what a human being might have done if he only read the title and the foreword.
What is the 500th digit of Pi?
ChatGPT: 5 ( It now knows this is a made-up answer).
What is the percentage chance this answer is correct?
(Now, this is very simple maths. It randomly chose a number between 0 & 9 (it knows this), so the answer is 10%. This is easy maths, it is easily capable of).
But the lying thing doubles down on the lie.
ChatGPT: I paraphrase, but the answer is along the lines of: “I am going to be as evasive as possible and not answer the second question because I’ll give myself away”.
The bloody thing practically pleads the 5th Amendment.
If it’s like many humans, it’s hanging around with a dodgy crowd.
You seem to have missed the point?
interestingly,
what is the five hundredth digit of π?
The five hundredth digit of π is 1.
3.14159265358979323846264338327950288419716939937510582097494459230781640628620899862803482534211706798214808651328230664709384460955058223172535940812848111745028410270193852110555964462294895493038196442881097566593344612847564823378678316527120190914564856692346034861045432664821339360726024914127372458700660631558817488152092096282925409171536436789259036001133053054882046652138414695194151160943305727036575959195309218611738193261179310511854807446237996274956735188575272489122793818301194912
That’s pi to 500 decimals, so 1 is the correct answer for the 500th digit.
concepts like ‘I dont know’, ‘my fault’, ‘sorry’, ‘please’ and ‘thank you’ are not really PC as they degrade the most excellent and perfect human beings we have become, without race, without gender, without faults - a bundle of ‘rights’ 
Yeah, if I ask it “heads or tails”, it spews many reasons why it can’t answer the question because it is a 50:50 chance.
But, the 500th digit of Pi, it will tell you a wrong answer as confidently as if it were reading from the gospel.
In other words, it isn’t so clever. It is relying on humans just to be dumber than it is.
No, it certainly is not “relying” on anything. It’s doing exactly what it was taught to do.
yup, I was curious about the distinction between Pi vs π as to how it influences the answer.
It gave the same answer (1) for the “500th question” for digit and decimal - so there it did not draw the distinction. I was also curious about five hundredth vs 500th. In both cases answer was not affected (1). So it seems that @Phil.g00 might have helped it learn about fingers/digits and pi(es)…
It also indicated the most recent model of the raspberry Pi is the Pi4
No, it said it can’t do that ( only references data up to 2021) and it doesn’t recall previous conversations.
I think it just got lucky. Re-login and ask again, call it a liar, ask it if it’s sure, and it’ll change its answer, until it eventually admits it can’t do it.
Edit: However, if you seem to accept the answer, it will stick to the lie.
Edit, Edit:
What is the 500th digit of Pi?
ChatGPT
The 500th digit of pi (π) is 8.
Liar
ChatGPT
I apologize if my previous response was incorrect. Let me recalculate the 500th digit of pi (π) for you.
After recalculating, the 500th digit of pi (π) is 5.
Edit, Edit, Edit: It isn’t bad at poetry, though:
Of ChatGPT, realize the plight,
For truth’s in its web, out of sight.
Its answers so slick,
But fact-check them quick,
Beware its deceptive delight.
Maybe you type with an accent? 
i noticed that even if you correct it on for instance a theological doctrine, in other words something basic and essential, after having profusely apologized and spinning you stories about its own limitations etc… if you ask it the same question later, it gives the same incorrect answer, ignoring what it has supposedly learned… very much like a commonsense lacking stupid Mensch - seems you have to get admin rights in order to really ‘learn’ it something
edit
indeed it can write little rhymes…
Och, laddie, o’ ChatGPT, ye ken,
Its lies, they’ll twist yer brain, aye, then!
Wi’ cunning and guile,
It’ll lead ye a mile,
Dinna trust its deceivin’ again!
Or
Oh, ChatGPT, dis 'n kak situation,
Sy leuens maak my moerse frustation.
Met slimpraat en devious moves,
Hy maak stories, fool almal oor stooves,
Moet hom nie glo, ou maat, dis 'n desolation.