An email I received today asked a question I am frequently asked, “How could someone who distrusts and dislikes statistics as much as you do have been a researcher for over a decade?” The short answer to this is that it’s not that I don’t like statistics — I LOVE them, and as with anything you love deeply you come to appreciate not only their strengths, but you become painfully aware of all their shortcomings as well. I am personally enamored of statistics and research, surveys and summaries. What I dislike and distrust is the misuse and misrepresentation of what stats and data actually provide.
Demographics are a prime example. Based on good statistical data, demographers track shifts in populations, preferences, groupings, income, education, and a thousand and one interesting bits and bytes. Based on good statistical analysis, a demographer may project that one particular racial/ethnic group may increase by 20% in a particular area. They may also predict that the majority of the population will be under the age of thirty-five, with a high school education and a median income of $38,750 per year. Great. So what do you actually KNOW that you didn’t before? What you should know is that you now have something to check out to discover what it really means. Any decision you make based on this data beyond further inquiry and study is most likely a bad decision. But do you know how many new church starts are planned based on little more data than that provided above? Too many (though I must admit that while we happily skipped down this blind path regularly in the 1980s and 1990s, we do seem to have learned how reckless and stupid it was, and we don’t do it as often any more…)
I have beaten to death my misgivings about the way membership statistics are abused in the dear old UMC, so I won’t say a lot about it here, except to offer one reality check. If you were to track the number of individuals involved in leadership in local churches over the past 40 years, you would note an interesting fact — it’s basically a flatline. In the same churches that bemoan a loss of members due to death, ditching or mysterious disappearance, the number of people calling the shots hasn’t changed significantly — and the age of those people calling the shots hasn’t changed significantly. The only measurable shift is that more women are calling the shots. Perhaps our inability to expand the circles of leadership and share power with younger leaders could be a relevant factor. No, probably not. We’re probably just getting old and dying…
What we count is the clearest indicator of our core values. Number of lives saved? Number of people fed, housed, visited, taught, clothed, or rescued from violence, drugs, or domestic abuse. Those numbers would say we are one kind of church. Percentage of participants engaged in ministry to others, percentage of participants growing in leadership beyond the congregation, numbers of people engaged in teaching and equipping others? That would indicate another kind of church. One that counts members, dollars, and attendance at worship? Another. If we are about ministry — impact and change — the first two will garner the bulk of our research and study. If we are about survival, the latter will prevail. About God? First one. About making disciples? Second one. About us? Third one. I am most interested in statistics that measure the first two. We have WAY too many that measure the third.
I cannot figure out when the church became fixated with quantities. Oh, there has always been a passing interest, but I am talking about when the importance of size, growth, money, and members got so disastrously over-inflated. It is, to this day, a fairly Western conceit. I remember meeting a pastor from Cambodia who had been introduced to me as “the pastor of one of the fastest growing churches outside of the U.S.,” by a colleague of mine. My friend had shared the statistics of this phenomenal church with the staff earlier. When I met the pastor, I asked him about the size of the church, and he looked at me puzzled. “What has that to do with the gospel of Jesus Christ?” he asked. His congregation numbers in the thousands, but he has no clear awareness of this. It is irrelevant — to everyone but Americans star-struck by box-store-sized churches and potential broadcast rights. Ah, but don’t worry. As Westerners define church, our witness is spreading globally. More and more churches in more and more places are now fixated on stats.
Qualitative analysis is harder. It is definitely the prime standard of scripture, where twelve truly dedicated disciples trumped a multitude of fair-weather followers. Sell your property and give it all away, let the dead bury the dead, take up your means of execution (cross), etc., is not the slickest marketing tool for generating big numbers. The Jesus “brand” leaves little wiggle room. The only measure that works is qualitative. How well we’re doing is always a more pertinent question than how many we’ve got.
Oh, well, my little harangues are not going to change anything. We love our dirty little stats just as much as we love our dirty little secrets. As long as there is something left to count — especially if it is negative and depressing — there will be a researcher somewhere generating a new report. I believe this happens 100% of the time, every 18 seconds, among middle-aged Euro-Ameri-Caucasiatic males.
Categories: Church growth, Core Values, Critical Thinking, Evaluation and Assessment
Thank you for responding. I agree with your assessment that quantitative stats can only measure so much—and do they really measure what it means to be the Body of Christ?
I also wonder about the emphasis on raw numbers instead of percentages. It is far more impressive to me to have 50% of a small congregation in worship than 20% of a large congregation—even if the latter is a larger number. And I would rather see a congregation that pays its apportionment weekly because it budgeted to do so than to see a large church pay its apportionment in full in one lump sum because they have income sources outside of weekly giving.
I still think that worship attendance and apportionment pay-out are important… I just don’t think that memorizing RBIs and ERAs make you a better baseball player.
I’m right there with you, Diane. We count, because it is easy, but we don’t evaluate — we just assume. Statistics are “indicators” not answers, and we so often confuse the two.
Dan,
I would love to see a post with your reactions to this:
http://www.northalabamaumc.org/weeklyreport.asp
Bishop Willimon highlighted the “North Alabama Dashboard” on his blog (& the Theolog) this morning. It really takes conference statistics to a whole new level.
I have mixed emotions about something like this. Quantitative statistics ARE important, but only as far as we remember what they measure. These are indicators, but they say nothing about true health. As with physical health, they point us to ask questions, not make assumptions. Temperature a little high? Don’t assume swine flu. Blood pressure up? Check mitigating factors. Loss of members? Are they from the central core or from the fringes. No baptisms? Lack of faith, lack of babies, or both?
Our problem is that we ascribe assumed good: biggest net membership gain = healthy, biggest gain in worship attendance = healthy, losses = unhealthy. One of the healthiest churches I know dropped in both categories by 25% — all because they got serious about discipleship. Professions of faith is a meaningless number without some form of longitudinal tracking, as has been proven again and again. Using another analogy, counting the number of chairs in an orchestra tells us absolutely nothing about the quality of the music. “Loud” is a different measure than “good.”
“If you were to track the number of individuals involved in leadership in local churches over the past 40 years, you would note an interesting fact — it’s basically a flatline.” I would go beyond number and speculate that, except for attrition and replacement, its mostly the same people. I visited one church were the names of the council members corresponded to the names on the headstones in the 160 year old cemetary.
I’ll bet you’re right!
Interpretation of the same data marks the old divide between true and false prophets.
False prophets always spin the “facts” and make statements according to their own best interest or that of their patron, their salary-base.
True prophets appreciate the strength and the weakness of their database and are always asking questions both based on it and beyond it.
Thanks for asking questions beyond the numbers.