Quantitative and Qualitative Research
On Language Learning
By
Mohammad Adnan Latief
University of Pittsburgh
StateUniversity of Malang
2009
Validity of Research Data on Language Learning
By Mohammad Adnan Latief
Abstract: Thisarticle presents the differences in the paradigms between quantitative and qualitative research. The discussion covers (a) verifying vs. generating theory, (b) formulating research problems, (c) numerical vs. verbal data, (d) sample vs. informant, (e) one vs. many kinds of data sources, (f) one vs. many techniques in data collection, (g) naturalvs. non-natural setting, (h) deductive vs. inductive analysis, (i) linear vs. circular procedure, (j) snow balling technique, (k) process vs. product, (l) human vs. non-human instrument. This article does not claim that one approachis more scientific than the other. Each approach has its own scientific value if used for the right kind of data. A researcher does not have to bother about contrasting or combining the two approaches. A researcher, though, mayuse the two approaches at the same time for different kindof data, not combining the two approaches for the same kind ofdata
Key Words: quantitative, qualitative, deductive, inductive, generating, verifying, linear, circular.
We can not expect every body to have the same way of thinking nor the same way of solving problems. The difference naturally happens because we are exposed to different experience. Those who share the same experience, though, may agree on the same way of thinking. In research, the different way of thinking can be classified, among others, based on quantitative and qualitative paradigms. This article discusses different ways of thinking especially in research.
EXAMPLES OF TWO DIFFERENT WAYS OF THINKING
Everyday we always make decisions to make choices in our life. From the time we wake up from bed in the morning, to the time we go to bed again; we make decisions to choose what time to wake up, what to eat for breakfast, to go to school (for students), to work (for those who have a job) or to stay at home, to teach what subject and how (for teachers), to spend weekend at home or to go somewhere, etc. Each decision of choices leads to different results. The best choice for us is of course decided based on our knowledge and experience. We will not make a decision to choose to do something that we know will not lead us to our success. Following are some examples of different decisions in choosing an action.
Indeveloping students’ good behavior, some teachers believe in behaviorist’stheory of stimulus-response, which requires teachers to give rewards for students' good behavior and punishment for bad behavior. They believe that students will have good behavior in their life if they get used to good habits and are never allowed to experience bad habits in their school. Other teachers believe in constructivist’s theory which requires the teachers to develop students’ self awareness of the need to have good behavior. Students will behave well if they know that it is good for themselves to do so. Students who misbehave do not realize that what they are doing is wrong (Degeng, 2000:5).
In teaching English skills to students, some teachers believe in structural theory that drilling the students is the best way, because drilling develops students’ accuracy of language use.They emphasize accuracy of the students’use of English. Others believe in communicative theory that practicing communication in English is the best way because English can be learned by actually using the language in the real communication.They emphasize fluency of students’ use of English.
In some organizations, leaders are elected based on voting, those who get the majority of the votes are considered to be the best leaders for them. In other organizations leaders are selected based on some criteria. Those who get the highest score on the criteria-based selection are considered to be the best leaders for them.
In deciding students’ scores at the end of a semester, some teachers use norm-referencing technique, which require the teachers to give high scores to few of the students, middle scores to the majority of the students, and low scores to another few of them. Proportion of students who get high, mid, and low scores is always of interest to the teachers. Other teachers use criterion-referencing,which requires the teachers to use a certain standard of scoring. In this scoring technique, the percentage of students who get high, mid, and low scores is not of interest.
There are still many other different ways in solving the same problems. What is important is that each approach is selected with certain reasoning and is believed to be the best choice.
QUANTITATIVE AND QUALITATIVE RESEARCH
As each research is trying to discover the underlying system of the research objects, a researcher has to decide the appropriate strategy for that purpose.The strategy is decided based on either quantitative or qualitative approach. The different approaches dictate different ways in the process of research;in defining data, in selecting data collection instrument, in selecting the right sources of data, in collecting data, and in data analysis. Following are differences between quantitative and qualitative research paradigms.
Verifying vs. Generating Theory
Research questions ask the underlying system of the objects of the study. In quantitative research, a researcher predicts the answer to the research problems based on theory. This theoretical prediction is called research hypothesis. The goal of the quantitative research is to verify this theoretical hypothesis using empirical data. The empirical data are analyzed to verify the theoretical hypothesis. So the goal of the quantitative research isto verify theory. In qualitative research, researchers try to understand a research object without making any theoretical prediction. The existing theory does not dictate what a researcher should do to understand a research object. Anything resulted from the observation and analysis will be accepted as the findings. Or the theory is generated freely from the result of observation of the research object and analysis of the result of observation (Bogdan & Biklen 1998:38).
Fixed vs. Flexible Research Problems
Inquantitative research, research problems are usually developed cautiously based on the purpose, the design to be used, the sources of data to be selected, the instruments to be developed, and the hypothesis to predict the finding. So it usually takes a lot of thinking before a research problem can finally be formulated. But once the research problem is formulated, the research problem becomes fixed and the researcher has to stick to that research problem. The only thing the researcher needs to do is to proceed with the process of research to get the answer to the problem. The researcher does not need to worry about changing or modifying the research problem that has been well developed. In qualitative research, research problems are usually defined in a general way. The more focused research problems are usually made after the research has been started, when some collected data have been analyzed (Bogdan & Biklen, 1998:2). So while research problems in quantitative research are made fixed from the beginning, research problems in qualitative research are developing into more focused during the process of research.
Numerical Data vs. Verbal Data
In quantitative research, data collected are represented numerically. Data representing achievement, for example, are recorded in scores. Even data representing sex are recorded in numbers, 1 for male and 2 for female. The numbers are needed for statistical analysis. Charles, C.M. (1995:118) states that: “Quantitative data are analyzed mathematically, and the results are expressed in statistical terminology.”In qualitative research, the data analysis does not use statistics that requires numerical data. Therefore, the data are collected and recorded in description, not symbols or numbers (Bogdan & Biklen, 1998:5). Data representing students’ language achievement, for example would be the description of what language performance students can show to represent their language competence. Charles, C.M (1995:118) states that: “Qualitative data are analyzed through logical-inductive analysis, a process of grouping, regrouping, and matching data with research questions. The results are expressed as verbal statements”.
Representative vs. Authoritative
In quantitative research, sources of data are assumed to be heterogeneous, having different variations, each of which has equal right to represent the group. Since the population of the sources ofdata is usually big and so only a sample can be taken, then the best sample is the one that is most representative to the population. The sample is usually called respondents or the research subjects (when the source is human beings). In qualitative research, sources of data are assumed to be homogeneous, having no variation. Therefore, the trustworthy source of data does not come from representation of different groups of the source, but selected based on certain criteria to find the most authoritative one. The source in qualitative research is usually called informats (of course when the source is human being)
The Same Source vs. Many Different Sources of Data
In quantitative research, data are collected from the same group of sources. When data on personality are collected from students of the English Department of StateUniversity of Malang, for example, only scores resulted from personality tests of those students are collected. The interpretation of the students’ personality is mainly based on the scores resulted from the personality test done by the students.In qualitative research, when personality is involved as one of the variables, the data on personality are assessed from as many different sources as possible; from their parents, from their neighborhood, from their diaries, etc.In short, in quantitative research, only one kind of sources is used while in qualitative research, many different kinds of sources are used.
One Technique vs. Many Different Techniques of DataCollection
In quantitative research, data are collected using one data collection technique. When data on personality are collected from students of the English Department of StateUniversity of Malang, for example, only scores resulted from personality tests of those students are collected. Another data collection technique is not needed. The interpretation of the students’ personality is mainly based on the scores resulted from the personality test done by the students. In qualitative research, when personality is involved as one of the variables, the data on personality are assessed using as many as possible different data collection techniques, like interview to the students themselves, interview to their parents, examining their neighborhood, examining their historical biography, etc. In short, in quantitative research, only one technique of data collection is used, while in qualitative research, various techniques of data collection are used.
One Data Collection Instrument vs. Many Different Instruments
Since the sources of data representing population in quantitative research tend to be in big number, or otherwise not representative, only one data collection instrumentis used. This one instrument has to be developed correctly because the validity of the data collected are mainly relied on the quality of this instrument. If the data collected suffers from validity problems, it is the instrument which is to blame. The quantitative researchers believe, though, that if data collection instrument is correctly developed, then researchers can trust the well-developed instrument toresult in data with strong validity. So, it is important that the data collection instrument be prepared with high caution.
In qualitative research, data are taken from many different sources, but each source is in a small number because it is not the representativeness of a sample that counts, but the authoritativeness that comes from the source selected based on certain criteria. Therefore, different data collection instruments are used. To collect data on personality of a certain student, for example, an interview guide to the student, an interview guide to their parents, an interview guide to their friends and relatives, an observation sheet to observe their behavior, a note to write anything related the explanation of their personality, etc. are used as the data collection instruments. To collect data on the correct use of high level Javanese Structure, to cite another example, a tape recorder to record the speech of an authoritative Javanese speaker is used and an interview guide to interview the Javanese speakers and a guide to examine Javanese literature in Javanese are used.
Non-Human Instrument vs. Human Instruments
Since quantitative researchers assume that well-developed instruments can be used to collect valid data, they can rely so much on their instruments. They have to develop their instruments in their best possible way to meet their purpose so that the instruments can be administered by anybody. That’s why this instrument is called non-human instrument, an instrument that can run by itself to collect data.
In qualitative research, the researcher collects and anlyzes data simulatneously to draw a temporary conclusion and repeats the cycles several times, deciding what data needs to be collected again to verify their temporary conclusion. Therefore, the researcher has to be involved himself in the process of data collection using all kinds of necessary instruments. The instruments used by the researchers themselves are called human instruments.
The qualitative researchers believe that many advantages can be obtained more in the process of data collection with the human instruments than using non-human instruments.When the subjects do not understand the questions raised by the researcher, an explanation can be offered by the researcher to clarify the questions. Qualitative researchers can dig more information from the subjects byrephrasing the questions or by asking other additional related questions. Qualitative researchers can decide the right time when to ask, to stop, to continue, or to repeat asking questions to the subjects. Qualitative researchers who are involved in the life or in the community of the subjects can understand what is not said by the subjects, or when the subjects are not telling the truths. The researchers can recheck the information to confirm the reliability of the information that has been obtained, and to cross check the information with some other evidence to verify the validity of the information collected.
Qualitative researchers have to be very closed to the subjects’ life and or the subjects’ community so that they can see the information objectively but are also reminded not to be too closed to the life of the subjects so that they are not carried away and do not become biased.
Linear vs. Circular
In quantitative research, all steps of research are done in a linear order, one step is followed with another step, or one step becomes the pre-requisite for the following step. The whole process of research is planned from the very beginning to the end. Quantitative researchers assume that when a plan is well prepared then the whole project will run successfully. Therefore, once the plan is launched, the researcher has to stick to the plan, the rest of the activities are done following the plan. There is no need to question the plan or to go back and forth from one step to the previous steps. In qualitative research, the research plan does not dictate the research steps as rigidly as the plan in quantitative research does. The plan may be modified during the process of research. The assumption is that many things may show up to be better than what we have planned. And this has to be accommodated in the research. Therefore, the process in qualitative research is not linear but circular, going back and forth from one step to the previous step in a circular way.
Manipulated Setting vs. Natural Settings.
Quantatitative research data are usually resulted from a formal assessment, like a test, questionaire, structured interview, etc. This kind of assessment is usually done in a special maniputed setting where data can be collected efficiently. An achievement test, for example, is administered when the students are not learning. An interview to an English teacher can be done in an office when he/she is not teaching. In an experimental research, settings can even be manipulated, assigning one group of studentsto an experimental group to be given an experimental treatment, and another group into a control group and are given a control conventional treatmnet. This is very different from the way qualitative researchers collect data. Qualitative researchers collect data using naturalistic observation or authentic assessment. Qualitative research data on students’ language achievement, for example, is collected through authentic classroom assessment at the time when students are using the language while studying in a classroom or at the play ground when they are playing using the language we are assessing. Research data on language teachers’ teaching behavior is observed when the language teacher is in class teaching the language.Charles, C.M. (1995) states that a study on patterns of personal interaction, dominance, and submission among children requires a naturalistic observation done at the play ground while the children are playing and interacting. A study on changes, over time, in language patterns by young learners learning Englishrequires naturalistic observation.