|
|||||||||||||||||||||||||
|
|||||||||||||||||||||||||
|
This page revised and Copyrighted: Theon Doxazo |
|||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||
 |
|||||
 |
|||||
|
04.1.3.0 |
|||||
|
Why am I taking all this trouble to describe and advocate for Psychometrics? Because, as one deals with issues of Psychological Theory, one is adrift on a lonely and desolate ocean. There are precious few landmarks one can use to decide which theory is better than another and which theories one should become adept at using with patients. Psychometrics provides answers to just such problems.
You may recall that I bemoaned the current state of research in Psychology [as well as the other social sciences]. It seems that the literature is infested with 'junk science.' People claim that their empirical results demonstrate that one approach is better than another. This is exactly the way Psychology [and the other social sciences] is supposed to work. Empirical studies drive the adoption of one theory or another. The battle ground that is Psychology is ruled by Empirical results. Yet, if the 'science' that demonstrates those results is flawed junk, then how can one decide correctly which theory is to be accepted and which rejected? Psychology becomes 'faddish', a discipline ruled by the 'social influencer' and the forces of popularity. This is NOT the way things are supposed to be!
It is in this context that Psychometrics comes to our rescue. Psychometrics provides the rigor and the discernment to allow one to decide between otherwise attractive options. How? Psychometrics studies the way tests and measures work. It provides a mathematical foundation for using measures to discern Empirical Truth. It lets us know if what we are measuring is reliable and valid. Once we know that our measures are trustworthy, then it becomes the job of Research Design to apply those measures in ways that will identify the Empirical Truth present in a given theory. The only way we can feel confident in our 'research' results is if our measures give us good information. Otherwise, as the information processing experts would say, it's 'garbage in, garbage out.' All the sophistication of research design will be of no use if the measures one uses don't produce accurate readings. You'll be lost; adrift on an ocean without landmarks.
It is at this point that the first, and most crucial of Psychometric's assumptions comes into play. If you are a Christian, it should sound eerily familiar and comforting. The most basic doctrine of Psychometrics is that: 'All measures are flawed.' I say that this should be 'familiar and comforting' for Psychologists with a Christian commitment, when one remembers the Churches' Doctrine that: 'All have sinned and fallen short of the glory of God.' It SOUNDS like we are both beginning at a very similar place! We may differ in the scope of our interests, but the main theme is very similar.
I say that this basic premise is 'comforting' to Christians because Psychometrics then goes on to say that these flawed measures MAY, if properly understood, provide helpful information, even in spite of their flawed nature. You may have heard of the controversy of intelligence testing across cultural lines. This is an area that is practically 'guaranteed' to produce an increase in errors. Measuring a White, middle class subject versus an American Indian on the reservation is an extreme case in point. I undertook two separate practica where I did just that. The dangers were inherent to the situation. Yet, the Bureau of Indian Affairs recognized the NEED for testing in order to qualify candidates for governmental payments and special programs. They saw the need and researched how assessments done on the 'Indian on the reservation' could be modified to more accurately reflect the clear cultural differences from the group the IQ measures were normed on. With those adjustments IQs could provide useful information and could be used to qualify clients for governmental services.
I spent the time on this Illustration of testing on the Reservations in order to describe the 'comforting' nature of Psychometrics. We, as human beings, are ALL flawed. We are ALL sinful. YET, with proper help, we can grow and develop to become more like our Savior. We will NEVER be just like Jesus, barring His Glorification of us when we reach heaven. BUT, we will be 'more' like Him, and that's the goal to be striven for. We may never be 'perfect' here on earth, but we can be closer to Him as we continue to walk in His Spirit. And by doing so, we will be useful to His work, even now, even here, even flawed as we are, we can still be helpful.
Looking at this situation from a Psychometric point of view, Reliability is an asymptotic function. Like Chi Square, the curve moves upward as if moves to the right. It never reaches the far right limit, but as it does so it extends upward dramatically. It is asymptotic in that it never reaches the right-most limit, but it can get infinitely close. Simultaneously, as it gets closer to the right, it gets 'larger'. As the limit is approached, the 'size' increases dramatically. As one gets infinitely close to the limit, so one becomes 'infinitely' large. It's Asymptotic. [I got a B in Calculus.]
It's at this point that I am about to inflict on you another of my 'stories'. I taught for years. One of the classes I taught was 'Tests and Measures' [of course!] One year I asked the class to develop their own test, just to see how it's done. After some back and forth, they decided to develop what we later called the 'Horror Film Appreciation Scale'. Basically, they brainstormed dozens of true/false questions related to Horror Films and associated themes. Each student was assigned the task of writing a dozen [?] questions, which were later 'weeded' and compiled into a questionnaire. This 'test' was administered to the Introduction to Psychology class the following year. The data was repeatedly 'crunched' using Chronbach's Alpha as the criterion to produce a best fit measure. This test was then re-administered to the Intro to Psych class the following year. The resulting Reliability estimate was something like α = 0.9998! This result was simply Amazing! I had never seen a Reliability estimate that was that high. It was a truly 'asymptotic' result! Clearly, the measure was not 'perfect', but it was HIGHLY Reliable, and thus useful! Now, you may question that. What possible use could a measure of 'Horror Film Appreciation' possibly have for studies involving Faith or development generally? Well, it turns out that one of the options for demonstrating Validity involves giving two tests and seeing how they 'relate'. The direction of the relationship is unimportant and driven by theory, but the SIZE of the relationship is crucial. The one problem with this measure of Validity is that the usefulness of this approach is reduced by the PRODUCT of the Reliabilites of the two measures. Thus, if I am trying to demonstrate the Validity of one measure, that demonstration is hampered by the Reliability of the other measure. The Horror Film Appreciation Scale comes to our rescue at this point, in that, having an almost perfect Internal Consistency Reliability, the Validity of the other measure is made dramatically more clear when compared to the Horror Film Appreciation Scale! I added the Horror Film scale to the research protocol I used with the Intro to Psych students and now have years of data, across hundreds of students, and a half-dozen different religiously-oriented measures. It's sitting, unanalyzed, in my files, just waiting for the time and energy to be analyzed. Sigh.
Just because I feel the need, let's look at the characteristics of the Horror Film scale a bit. One important factor in it's abnormally high Reliability is the fact that it was created and studied in a relatively homogeneous environment. The vast majority of the students involved were of college-age, White, and working to upper-middle class. Lots of measurement error was eliminated by this environmental situation. Further, the items were created by the students in this population. They asked questions that were drawn from their own experiences of then currently available Horror Films. This would, likely, be true for all of the students involved. A major force delivering the high Internal Consistencies was also the lack of an impact by Social Desirability. If one liked Horror Films, that was seen as OK. If one didn't like Horror Films, that was also seen as OK. As a result, there was no need to try to 'fake good' on the test because there was no perceived 'good' to try to fake. A final factor that increased Reliabilities was one's personal response to Horror Films generally. Whether one liked them or not, there was generally a clear and strong emotional reaction to the films that drove the opinion. There were few 'wishy washy' or 'undecided' opinions about Horror Films. To the degree that these same variables can be 'crafted' into the measures one wishes to develop, it may prove helpful in improving the Reliability of said measures.
Taken together, this Psychometric methodology provides the opportunity to hone one's measures and thereby improve them. Good Psychometrics will reduce error variance in one's research, even as good Research Design will. Taken together we can approach the Second Peter Theory with the hope that it may be shown to be helpful in understanding Christian Spiritual Development.
|
|||||