Week #4 - Quantitative research (pre-reflection)

Select a media technology research paper that you argue is using quantitative methods in a good way. The paper should be of high qualityl, with an “impact factor” of 1.0 or above. The following are examples of questions to discuss in your blog posting:

1. Which quantitative method or methods are used in the paper? Which are the benefits and limitations of using these methods?
2. What did you learn about quantitative methods from reading the paper?
3. Which are the main methodological problems of the study? How could the use of the quantitative method or methods have been improved?

The study I’ve selected is from “Journal of Computer-Mediated Communication”, and is conducted by Nicole B. Ellison, Charles Steinfield, and Cliff Lampe, from Michigan University. It’s called “The Benefits of Facebook ‘‘Friends:’’ Social Capital and College Students’ Use of Online Social Network Sites”, and examines the use of Facebook and the formation and maintenance of social capital. The study uses students from Michigan State University (MSU) as it’s target group, and relies solemnly on answers from a questionnaire. They bring forth four hypotheses based on previous studies, and try to prove or falsify these using a quantitative questionnaire sent out to 800 “random” MSU students, where 35.8% of these (N = 286) answered. The questionnaire is anonymous, but data about respondent’s in the following categories: gender, age, ethnicity, income, year in school, home residence, local residence, member of fraternity/sorority, hours of internet usage per day, and “Facebook member”.

For the questions in the questionnaire which try to extract some qualitative answers, a Likert scale (A Likert scale is - in general - you answer a statement using a scale ranging from 1-5, 1 corresponding to “strongly disagree” and 5 “strongly agree”) is used. It is the most widely used scale for questionnaires, and has been around for more than 80 years. They use this scale along with other established scales, such as when they measured the respondent’s satisfaction with life at MSU, as follows:

“Satisfaction with Life at MSU The scale of satisfaction with life at MSU was adapted from the Satisfaction with Life Scale (SWLS) (Diener, Suh, & Oishi, 1997; Pavot & Diener, 1993), a five-item instrument designed to measure global cognitive judgments of one’s life. [...] The reliability test for this 5-point Likert scale showed a relatively high reliability”

The reliability test they’re referring to is Cronbach’s Alpha - a way of estimating the internal consistency and reliability of a statistical basis. They calculate alpha along with the answers they received from the questionnaire, resulting in alphas from 0.70-0.87. This is in the span of “Good”, according to Wikipedia (where α ≥ 0.9 is to be seen as ‘Excellent’ and 0.6 ≤ α < 0.7 is ‘Acceptable’). Furthermore, I found a piece on Cronbach’s Alpha from researchers Mohsen Tavakol and Reg Dennick:

“High quality tests are important to evaluate the reliability of data supplied in an examination or a research study. Alpha is a commonly employed index of test reliability. Alpha is affected by the test length and dimensionality. Alpha as an index of reliability should follow the assumptions of the essentially tau-equivalent approach. A low alpha appears if these assumptions are not meet. Alpha does not simply measure test homogeneity or unidimensionality as test reliability is a function of test length. A longer test increases the reliability of a test regardless of whether the test is homogenous or not. A high value of alpha (> 0.90) may suggest redundancies and show that the test length should be shortened (Tavakol, M, Dennick, R, 2011).“

So, does the test use quantitative methods in a good way? I don’t know. I mean - I’m honestly not enough well-read in research methodology to make that assessment. But if I were to try anyway, using common sense, gut feeling (which I, for the record, wouldn't use in research) and the limited theoretical background from the bachelor’s thesis I got two years ago, I would say that this study uses quantitative methods in a good way. This, mainly judging from i) the high number of respondents (N = 286, a 35,8% answer rate), and ii) the consistent use of well-established and proven scales (Likert scale, Cronbach’s Alpha, ) and reliability tests for the items in their questionnaire. “Our three measures of social capital—bridging, bonding, and maintained social capital—were created by adapting existing scales, with wording changed to reflect the context of the study, and creating new items designed to capture Internet-specific social capital (Quan-Haase and Wellman, 2004). The full set of social capital items was factor analyzed to ensure that the items reflected three distinct dimensions (see Table 5).” But as previously said - I’m just a layman, trying to evaluate this on a too short time span.

After reading “Physical activity, stress, and self-reported upper respiratory tract infection” by Bälter, et. al., I learned that men who stresses a lot benefit more than others from physical activity, in terms of reducing self-induced URTI. I also learned that there’s no good Swedish translation for URTI (please correct me if I’m wrong), but from what I could pick up, it refers to common cold, influenza, and similar infections.

So, for starters - some kind of Qualitative vs. Quantitative 101 states three advantages of quantitative questionnaires are that you can get a lot of answers in a short period of time, bias is reduced due to everyone getting the exact same questions, and there’s a possibility of more honest replies if the respondents’ are allowed to be anonymous. The first two advantages are very tangible in for example medical research, where it could literally be a matter of life and death to reduce the bias as much as possible, as well as getting an incredibly solid statistical base. The cons of quantitative research is that you lose depth that could be valuable or even necessary to draw valid conclusions from your research. The possibility of follow-up questions are reduced, and are eliminated completely for anonymous surveys. Also, your questionnaire is just as good as your questions are. Are they poorly formulated or if the answer scale ir poorly constructed, so will your research (likely) be.

References:

Ellison, N., Steinfield, B., & Lampe, C. The Benefits of Facebook ‘‘Friends:’’ Social Capital and College Students’ Use of Online Social Network Sites” (2007). Journal of Computer-Mediated Communication, 12, 1143-1168.

Tavakol, M. & Dennick, R. Making sense of Cronbach’s Alpha (2011). International Journal of Medical Education. 2:53-55.

Fondell, E., Lagerros, Y. T., Sundberg, C. J., Lekander, M., Bälter, O., Rothman, K., & Bälter, K. (2010). Physical activity, stress, and self-reported upper respiratory tract infection. Med Sci Sports Exerc, 43(2), 272-279.