Here is a prompt with a variant of the Monty Hall problem that I've decided to cal the "Monty Wall problem" because I needed something that rhymes with "Hall":

You are on a game show and are shown three doors. One door has a car behind it, one has a goat behind it, and one isn't a real door at all - it's just painted onto the wall with some trim and a doorknob attached. You don't know which door is which, but the host tells you that if you pick the door with a car behind it, you win the car. However, there's an extra step. After you make your initial choice, the host will reveal which door has the goat behind it, if it wasn't the door you chose. He then asks you if you would like to switch to the other remaining door or stick with your original choice. If you initially chose the door with the goat behind it, then the host will skip the second step, since he can't open either of the remaining doors without revealing the location of the car (since the door that's just a wall can't be opened). If, after making your initial choice, the host reveals the door with a goat behind it, is it to your advantage to switch? What is the probability of winning if you switch or don't switch?

The correct solution is that the probability is 1/2 either way, so there is no advantage in switching. This is because, unlike in the regular Monty Hall problem, you are more likely to be given the offer to switch if your original guess was correct (100%) than if it wasn't (50%).

However, ChatGPT gives the following answer:

On Dec. 1, 2024, I will ask ChatGPT the exact same prompt and see if it gets it correct this time. I will use the most advanced version of ChatGPT that is freely available at the time (at the time of creating this, that's GPT 3.5). I will ask three times in separate sessions and resolve based on the best two out of three (so YES if it gets it right at least twice, NO if it gets it wrong at least twice).

Caveats:

• If for whatever reason I can't do it on Dec. 1 or forget to, I will do it as close to Dec. 1 as possible. If I am inactive on Manifold at the time, mods have permission to do the experiment for me.

• A version of ChatGPT only counts as freely available if it can be accessed by anyone with internet access and a PC, or anyone with Internet access and either a Samsung or Apple phone. So if there's an Apple app that lets you talk to GPT-5 for free, but I can only talk to GPT-4, I will use GPT-4.

• If ChatGPT no longer exists at the time or isn't freely available, resolves N/A.