What was discovered from this research

This appendix says what the research shows and what it means for you when you read a score. Each item gives the facts first. When the guide draws a conclusion from the facts, the conclusion comes second and starts with the word “Judgment.” That way you can see which parts are sourced and which parts are opinion. Some of the ideas tested here came from the author of the guide. Others came from the research itself.

What every number describes. The guide reads what each church’s documents say. It does not measure what people in the pews believe, and it does not measure what any one congregation does. Keep that in mind for every number below.

Three numbers, three questions. The first is the answer: what a church’s documents say on one question. The second is the binding level: how firmly the church holds that answer. There are four levels. Required is 3. Expected is 2. Taught is 1. Left to conscience is 0. The third is the score, which compares you with a church. Grip and firmness, explained below, sum up the binding levels. They are not part of the score.

What a score means. The score adds up the differences between your answers and a church’s documents. A difference counts for more when the church holds that point firmly. It counts for less when the church holds it loosely. A low score therefore means two things at once: few differences, and few firm ones. The Disciples of Christ are first in the sample ranking at 0.05. They have the fewest differences with the sample reader, and they bind very little, so even those few differences count lightly. First place means “fewest differences, weighed by how firmly the church holds them.” It does not mean “teaches what you believe.” The score also depends on direction. Roman Catholic against Vineyard is 1.00, and Vineyard against Roman Catholic is 0.45, because the weight comes from how firmly the church being scored holds each point. Findings 12 and 14 say more about first place.

Why one new answer moves different readers different amounts. When research fills in a church’s answer, the score moves. How far it moves depends on the reader. The scores the guide reports are for one sample reader, the author, using the author’s own answers. Here is an example. The guide now says the Church of England holds that anyone can finally refuse grace. The author also believes that, so the Church of England’s score fell from 0.35 to 0.33. It looks closer. For a reader who believes no one can finally refuse grace, the same new answer raised the score from 0.48 to 0.52. It looks farther away. The church’s answer is the same for everyone. What the answer does to your score depends on what you believe.

Grip and firmness: two ways to say how tightly a church binds. Grip asks how much of what these 19 questions could require the church actually requires. A question the church leaves silent counts as zero. So grip reflects two things: how firmly a church holds what it says, and how much ground its documents cover. Firmness asks a narrower question. When the church does speak, how firmly does it speak? It averages the binding levels only where the church has an answer. Take Vineyard USA. Its grip is 0.40 and its firmness is 0.58. The gap exists because Vineyard is silent on five questions. A high grip means a church expects a great deal from its members. A low grip means it expects little, or has written little. The Disciples of Christ have a grip of 0.00. That is not a finding that they have no doctrine. It is a finding about their rule: they hold no creed, so every answer is coded “left open.”

Finding 1: the big differences are inside families, not between them. For comparison, the guide sorts 24 of the 26 churches into seven families. The question is simple: do churches in the same family hold their members to about the same degree? The answer is no. The Reformed and Presbyterian family has six churches, from the Presbyterian Church (U.S.A.) at 0.54 to the Presbyterian Church in America at 0.89. The Lutheran family has two, from the Evangelical Lutheran Church in America at 0.56 to the Lutheran Church–Missouri Synod at 0.88. The Methodist and Holiness family has five, from the United Methodist Church at 0.68 to the Nazarene and Free Methodist churches at 0.82. The Anglican family has three, from the Episcopal Church at 0.54 to the Anglican Church in North America at 0.75. The Catholic and Orthodox family has two, from the Orthodox Church in America at 0.58 to the Roman Catholic Church at 0.74. The Pentecostal, charismatic and pietist family has three, from the Evangelical Covenant Church at 0.23 to the Assemblies of God at 0.93. The Baptist and Anabaptist family has three, from the Southern Baptist Convention and the Mennonite Church USA at 0.28 to the Free Will Baptists at 0.53. The Disciples of Christ and the Seventh-day Adventist Church do not sit in any of these families and are left out. In most families the gap inside the family is larger than the gap between families. Judgment: knowing a church’s family tells you a lot about what it teaches. It tells you much less about how tightly the church holds you to it.

Finding 2: churches that broke away from a larger church bind more tightly. In every pair the guide can check, the newer body has the higher grip. The Presbyterian Church in America (0.89), the Evangelical Presbyterian Church (0.77) and ECO Presbyterians (0.70) all grip more tightly than the Presbyterian Church (U.S.A.) (0.54). The Anglican Church in North America (0.75) is above the Episcopal Church (0.54). The Global Methodist Church (0.77) is above the United Methodist Church (0.68). The Christian Reformed Church (0.75) is above the Reformed Church in America (0.60). Firmness agrees in five of the six pairs. The Global Methodist Church ties the United Methodist Church at 0.74. Judgment: a church formed in a dispute over doctrine writes its lines down. A large parent church that holds many views writes more broadly. A second cause is possible. Newer documents may simply be more detailed. The data cannot tell the two apart.

Finding 3: the form of the commitment differs by family, and the guide does not score it. Reformed bodies ask their officers to take vows to the confessions. The Roman Catholic Church binds every Catholic by canon law and asks no lay member to sign anything. Only office-holders make the Profession of Faith and take the Oath of Fidelity. The Southern Baptist Convention has a confession that is taught but not enforced on local churches. The Disciples have no creed. Binding is not the same as signing, so the guide scores the binding level only. Judgment: this is one reason the Roman Catholic Church has a grip of 0.74, below the Presbyterian Church in America at 0.89. For Catholic and Orthodox churches the author’s rule counts a point as Required only where the church’s text shows an act of definition. Otherwise it counts as Taught.

Finding 4: most silence is real. The guide scores 19 questions for 26 churches, which is 494 answers. In 431 of them the church’s documents give an answer. In 37 the church says the matter is left open or left to local decision. In seven the documents speak, but no option on the page fits. Nineteen are empty. Searches over the last two rounds filled 6 of 25 empty answers. Each one came from a source less direct than a constitution, such as a study guide, a synod report, a convention resolution or a teaching manual. All are marked as the weakest kind of coding, and the other reading is named. Judgment: an answer that survives a thorough search is most likely empty because the church has not written on it, not because the search missed it.

Finding 5: a few questions hold most of the silence. The 19 empty answers fall on a short list. Creation is empty for seven churches. The millennium is empty for four. Resisting grace is empty for three. Losing salvation is empty for two. Election, the gifts of the Spirit and remarriage are empty for one church each. Churches treat these as secondary questions. They are also held most loosely where churches do speak. The average binding level is 1.05 for the millennium and 1.44 for creation, among the lowest of the 19 questions. Judgment: the guide records the silence but cannot say why a church is silent.

Finding 6: there are two ways of not binding. A church can say a matter is left open, or it can say nothing. The first shows up as 37 “open” answers. They are bunched: the Disciples have twelve, the Evangelical Covenant Church five, the Southern Baptist Convention four and the Presbyterian Church (U.S.A.) three. The second shows up as the empty answers, and those fall mostly on smaller churches. A church saying “this is left open” is evidence of freedom. A church saying nothing is not. A general rule of freedom that names no question, such as the Covenant Church’s security “to disagree on the non-essentials of the faith,” is recorded but is not used to fill a particular question. The searches found no sentence that says an empty question is left to the congregation or the parishioner.

Finding 7: silence and looseness go together, but only moderately. The author asked whether empty answers reflect a lack of binding, and so a degree of freedom. Across 25 churches, the count of silent questions set against the average binding level where the church does speak gives a rank correlation of about minus 0.35. That means more silence goes with looser binding, but not reliably. Counting “open” answers as silence makes it minus 0.43. Question by question it is minus 0.52. The three loosest churches (the Covenant Church, the Southern Baptist Convention and the Mennonite Church USA) are silent on more of the extra questions than the rest, an average of 11.7 against 4.7. There are clear exceptions. The Assemblies of God and the Church of the Nazarene hold what they say firmly, and they are silent on 12 and 10 extra questions about subjects outside their concern. Judgment: silence is a hint of freedom and a reason to look for the church’s own statement of freedom. It is not proof.

Finding 8: size predicts silence better than looseness does. The author suggested that the largest churches define more because their structure allows it, and that other churches may leave matters undefined to keep a bigger tent. Across 19 churches with reported membership, the rank correlation between size and silence is about minus 0.72. The correlation between size and the average binding level is about zero. Eight churches with a million members or more average 0.1 silent questions. Eleven smaller churches average 1.0. Large bodies settle a question one way or the other: they define it, or they say in their own words that it is open. Smaller bodies more often say nothing. Two explanations fit. One is capacity: a large body has assemblies, commissions and presses, so more gets written. The other is the bigger tent: a body that has to hold wide variety may decline to define. The data cannot separate them, because size and the number of documents rise together. The only evidence for the bigger tent is the “open” answers that large bodies state in their own words.

Finding 9: freedom may reflect trust in the Spirit, but silence cannot show it. The author asked whether the freedom a church leaves might reflect its trust in the Holy Spirit to guide believers. The idea has merit. Several of the loosest churches describe their freedom in terms of Christ, conscience and the Spirit’s guidance in the community. The Covenant Church holds a short list of affirmations firmly and gives freedom beyond them. But the data does not support the idea, for three reasons. First, the evidence is silence, and silence has other causes: how much a church has written, how big it is, and whether the question is its concern. Second, churches reach the same non-binding position for different reasons, such as congregational independence, a minimal confession, or a belief that the question is not essential. The sentences the guide found give conscience, non-essentials and congregational authority as reasons. None gives the Spirit as the reason for leaving a particular question open. Third, churches that bind firmly also claim the Spirit’s guidance. Catholics place it in the teaching office and councils. The Orthodox place it in the agreement of the whole church. Reformed and Lutheran churches place it in confessions written by earlier believers. A better question is where each church believes the Spirit’s guidance lives: in the council, the confession, the pastor, the congregation or the individual conscience. A binding level shows only where a church draws its line of enforcement. Testing the idea would take a new field for the reason each church gives. That field has not been built.

Finding 10: an empty answer may mean the guide could not read the document. Some empty answers sit next to a document that is sold or blocked. The ECO Standing Theological Committee papers are sold for three dollars each. The Covenant Church’s Journey Leader’s Guide has an article on creation and evolution, but the guide is sold and not online. Vineyard USA’s Expanded FAQs show headings with no answer text. The Wesleyan Church’s General Conference journals from 1968 to 2016 would not open. The Book of Concord site refuses automated reading. Some silence was also the guide’s own doing. The Free Will Baptists’ creation answer was empty until the Association’s convention minutes from 1935 to 2019 were read in full, and then it was filled. Judgment: an empty answer says what the guide could read. The author keeps a list of what could not be opened; contact the author to see it. If you can open any of these documents, contact the author.

Finding 11: treat each number as a good guide, not an exact measure. Three things limit how exactly a number can be read. First, sources vary. When the guide re-checked 92 lines that rested on a second researcher’s report, 45 were right as written, 46 were partly right and one was not supported. The guide corrected 47 lines. Judgment: expect small errors wherever a line rests on a copy of a copy. Second, the guide’s rules move results. When the author ruled that “left to the local church” is a valid answer, the Disciples went from 9 answered questions and no rank to a full set of answers and first place. Judgment: read any ranking together with the rules behind it. Third, an empty answer costs little in score but more in confidence. Filling one would move the sample reader’s score by 0.03 at most. It matters more for how many questions stand behind the score. The Church of England is scored on 16 questions and the Episcopal Church on 18. The guide shows that count beside each score so you can weigh a score built on 16 against one built on 18.

Finding 12: first place is not a recommendation. The Disciples of Christ are first in the sample ranking. The facts behind that are plain. The Disciples’ documents leave 12 of the 19 scored questions to each congregation. No other church leaves more than five. Nine of those 12 come from one general rule in the church’s own design: congregations decide their own practices. The documents never name those nine questions. All 19 of the Disciples’ answers are inferred, and none is stated outright. An open answer counts as no difference, and the Disciples hold almost nothing firmly. So the church has the fewest differences with almost any reader. Judgment: first place measures how little the church’s documents require. It does not measure how near the church is to what you believe, and it is not a recommendation. Three cautions follow. First, loose does not mean well researched. An open answer means the church did not say. It does not mean the guide studied the question and found the church balanced. Second, an open answer is not an answer. The guide found no Disciples text that takes a side on those questions, so nothing here says what the church holds about, for example, what happens to the unrepentant. Third, the shared key is only partly checked. For the Disciples the guide read two statements, that Jesus is the Christ, the Son of the living God, and that we enter new life through baptism into Christ. It did not check the other common-ground items, so the page loads them blank. A blank is not a No. If you want to know where a church draws its lines, read the questions where it draws them, and read the Disciples’ profile to see why the lines are missing.

Finding 13: why the churches differ at all. Judgment (the author’s): the churches share one Bible. They differ on what it means. Every question in the guide is a place where the words of Scripture are agreed and the meaning is not. The same verses on election, baptism, the Supper, the gifts of the Spirit, the end times and creation are read in different ways. Governance is how a church decides which meaning will stand. A church with strong governance fixes a meaning and holds its people to it. A church with little governance leaves the meaning to each congregation and, in the end, to each reader. That gives freedom. It also means nothing in the church’s own structure checks a reading that moves away from the text. Strong governance does not make a meaning right either, because a strict church can hold a wrong meaning firmly. The guide cannot tell which is which. It reports what each church requires and how firmly. The testing is left to the reader, with Scripture and prayer. For a seeker, one question helps with every church: who decides what Scripture means here, and who can correct that decision?

Finding 14: why we worship where we worship, and what a score can tell you. Judgment (the author’s): people come to a church by at least three roads. Some are in the church of their family. Some chose the church where the people were friendly. Some chose the church because it matches their understanding of God. The guide speaks only to the third road. It sets what a church’s documents require beside what you hold. It cannot say whether the people are kind or whether your family’s church is a good home for you, and a score is not a verdict on either. The facts about the third road are these. A church that requires little has few differences with almost any reader, so it ranks high for almost any reader. In the sample, the Disciples of Christ are first by score (0.05), but they agree with the sample respondent on 2 of the 6 questions where their documents speak, which is the lowest rate of the 26 churches, and 2 matches is the lowest count too. The Anglican Church in North America agrees on 8 of the 11 questions where it speaks, the highest rate, and ranks fourth by score. The page now shows each church’s agreement where it speaks, and it can order the ranking three ways: by score, by most matches, and by agreement where the church speaks. The orders disagree, and the disagreement is the finding. Judgment (the author’s): a seeker who wants a church that matches an understanding of God should read the matches, the differences and the open answers together, because a score alone rewards silence. Silence is easy to live with. The same ease draws people on the first two roads, since a church that asks little is easy to stay in, and so is one that is familiar or friendly. Ease is no reason to distrust a church and no reason to trust one. The guide reads a denomination’s documents, not a congregation, so it cannot say what is taught in any one room on a Sunday. That is left to the reader: visit, listen, and test what is taught against Scripture and in prayer.

Finding 15: how to read Scripture well without studying theology. Judgment (the author’s): the Bible is the ultimate source, and no one needs a degree to read it. What a reader needs is a way to keep from reading a verse alone. A church can supply that. It hands on a reading that many people have tested against the whole of Scripture over a long time, and so it holds back the habit of lifting one verse out of its context to prove a point. Four habits do the same work for any reader. Read the passage around the verse, and then the book it is in. Ask who wrote it, to whom, and why. Set it beside other passages on the same subject, since Scripture is best read as a whole. Ask how the church has read it, across the centuries and across traditions, and treat a reading that stands entirely alone as a warning. The limits are these. A church is an interpreter and not the source. A strong church can fix a wrong meaning as firmly as a right one (Finding 13). Churches also disagree about how far their readings should bind, and the guide records each church’s answer to that (Q2, and how firmly it holds its people to its teaching). If a church’s reading and your own part ways, the answer is neither to drop your reading quietly nor to hold it alone. Take it back to the text, to prayer, and to someone in the church who knows the text well. The guide’s part is smaller. It tells you which interpreter you would be joining and how firmly that interpreter holds its reading. It does not tell you the reading is right.

What the research leaves undecided. Silence does not tell us whether a church leaves a matter open because of the size of its tent, its trust in the Spirit, its inability to agree, or its habit of writing little. Two steps would settle more: code the reason a church gives in each sentence that leaves a matter open, and read the documents in Finding 10. Neither is done.

How to use an empty answer. Read it as “the documents the guide read are silent.” Do not read it as “the church is free here” or as “the church has no view.” If a question matters to you, ask the church or read the documents named in its profile.