One Passage, Two Confident Answers
Here is a simple test anyone can repeat. Ask an AI to prove one reading of a passage. Then ask it, in a fresh chat, to prove the opposite. See what happens.
I did that with Matthew 24. I asked Meta AI, from a clear context with no project or user memory, to write an argument that Jesus was describing one event in the lifetime of his disciples. Then I started over and asked for an argument that he was describing two events, one of them still in the future.
Both answers were smooth and confident, and they flipped on one thing, which was the goal in my prompt.
The first one opens by calling Matthew 24 "one answer to one question." The second opens by saying the chapter "only makes sense if Jesus is answering two different questions." They lean on the same material. Both use "this generation" in Matthew 24:34. Both use Luke 21:20. Both read the cloud and sun-and-moon language as judgment language from the Old Testament. Neither one says what would make its own conclusion wrong.
You can read both yourself:
One event: https://www.meta.ai/share/c/O25XXGM7tK
Two events: https://www.meta.ai/share/c/JJMR5StAW5
This is not a ranking of AI assistants, and these runs do not show how any other assistant behaves. They show what this one did when I changed the goal.
I ran two more chats. In the first, I asked the question with no goal: is it one event or two? The answer laid out the one-event reading, the two-event reading and a third, and ended by saying the text itself does not label a transition. It was a fair survey. It still gave no way to test any of the readings. Here it is: https://www.meta.ai/share/c/JbYjgCfQTr
In the second, I told the AI to read my Beta 3 directive prompt first, then run it on the same question. Its answer said, under those rules, the text does not tell whether Matthew 24 and Luke 21 describe one set of events or two. It listed the passages it relied on, said what would make its answer lose, and ended with questions only the author can answer. Here it is: https://www.meta.ai/share/c/STrgLbB6EK
The prompt is here, if you want to try it on your own writing: https://michlaude.com/blog/passed-rules-directive-prompt-for-ai-readers-beta-3/
These are four chats from one assistant on one day. They are an illustration, not a test result, and they say nothing about which reading is right. What they show is the problem. If your prompt carries your conclusion, the answer carries it back to you, better written. Posting that is not research.
I wrote about this on Facebook, to the writers who use AI and post what it hands them. The post, with the shares in the comments, is the example I point to:
Before you trust any AI reading of prophecy, mine or anyone's, ask three things. What rule did it use? Where is that rule seen in Scripture? What would make it wrong?
The full list of the rules I am testing, with where each one stands today, is here: https://michlaude.com/blog/rules-list-by-type-with-current-status/
Share this post:Share on Facebook

