van de Vijver, F. J. R. (2002). Types of cross-cultural studies in cross-cultural psychology. In W. J. Lonner, D. L. Dinnel, S. A. Hayes, & D. N. Sattler (Eds.), Online Readings in Psychology and Culture (Unit 2, Chapter 6), Center for Cross-Cultural Research, Western Washington University, Bellingham, Washington USA

This material is copyrighted by the author(s), who have kindly extended to the Center the right to use the material as described in the Introduction to this collection and the form entitled "Agreement to Extend License to Use Work."

UNIT 2, CHAPTER 6

TYPES OF CROSS-CULTURAL STUDIES IN CROSS-CULTURAL PSYCHOLOGY

Fons J. R. van de Vijver
Tilburg University
The Netherlands

ABSTRACT

From a methodological perspective cross-cultural studies in psychology differ in three dimensions. First, cross-cultural psychological studies can be exploratory or test specific hypotheses. Second, some cross-cultural studies compare countries or ethnic groups while other cross-cultural studies relate specific characteristics of a country or ethnicity (e.g., socialization patterns or religiosity) to psychological variables. Third, studies can compare either constructs (e.g., do Chinese and Kenyans mean the same when they say that a person is intelligent?) or score levels (e.g., are Americans more extravert than Italians?). A classification of cross-cultural psychological studies, based on the three dimensions, is presented and examples are given.

INTRODUCTION

Types of Cross-Cultural Studies

The Argentinean author Jorge Borges once proposed a taxonomy of animals that he claimed to have found in a Chinese encyclopedia (http://www.multicians.org/thvv/borges-animals.html):

1. those that belong to the Emperor,
2. embalmed ones,
3. those that are trained,
4. suckling pigs,
5. mermaids,
6. fabulous ones,
7. stray dogs,
8. those included in the present classification,
9. those that tremble as if they were mad,
10. innumerable ones,
11. those drawn with a very fine camelhair brush,
12. others,
13. those that have just broken a flower vase,
14. those that from a long way off look like flies.

The wonderful creativity of the taxonomy provides a good introduction for the central topic of the present chapter: How can we categorize cross-cultural studies? The classification by Borges clearly illustrates that there are various ways of classifying and that some are more useful than others. The same issue plays a role in categorizing cross-cultural studies. Many categorizations can be envisaged, but not all of them are equally consequential.

Common distinctions are between cross-national and intranational studies; the former involves different countries, while in the latter different cultural groups are studied, that live in a single country. Examples of the latter are the numerous studies in which European Americans and African Americans or Hispanics are involved. In European countries intranational studies often compare majority group members and migrants or refugees or examine topics that are specific for migrants, such as acculturation processes. Examples of cross-national studies are the numerous comparisons of American and East Asian countries, such as Japan and China. Another categorization system can be based on the various psychological disciplines in which cross-cultural studies are carried out, such as social psychology, personality, and developmental psychology. An application of this perspective is particularly useful if one wants to identify areas of interest in cross-cultural psychology; social behavior is the most frequently examined behavioral domain in cross-cultural psychology. Still another perspective refers the distinction between cultural and cross-cultural psychology; the latter is culture comparative (a journal mainly devoted to comparative studies is the Journal of Cross-Cultural Psychology, see http://www.ac.wwu.edu/%7Epsych/jccp.htm), while the former is more focused on an in-depth analysis of cultural phenomena in a specific culture (journal: Culture and Psychology, see http://www.sagepub.co.uk/frame.html?http://www.sagepub.co.uk/journals/details/j0160.html). The present chapter only deals with comparative studies.

A Methodological Classification of Cross-Cultural Studies

The perspective to categorize studies of the current chapter is methodological. Three dimensions are introduced. The first dimension refers to the question as to whether contextual factors are included in a study. Contextual factors refer here to a wide variety of variables that could influence the cross-cultural differences observed; these variables may involve either participant characteristics (such as socioeconomic status, education, and age) or culture characteristics (such as a country's affluence and institutions). Many cross-cultural studies do not include contextual factors; these studies are interested in the comparison of countries. Examples are the large cross-national comparisons of educational achievement, such as the TIMMS project (http://timss.bc.edu), in which the performance of secondary school students in mathematics and science is compared across many countries. One of the most essential parts of reports of such studies is a table in which the scores of the countries are ranked. In other cross-cultural studies, however, more attention is paid to cultural factors and countries are deliberately chosen because of some characteristic they have. An example is the currently popular distinction between individualistic and collectivistic countries (see the chapter on individualism and collectivism on this website).

Both types of studies have their own strengths and weaknesses. If studies include many countries, they almost always have an intrinsic interest for cross-cultural psychologists. The studies increase our insight in the cross-cultural differences and similarities across these countries. Historically, they are often the precursors of more focused studies in which country differences are seen as related to differences in underlying dimensions. The latter type study is more precise than studies in which no contextual factors were examined; therefore, they are often easier to interpret. When a cross-cultural study involves only a few countries, problems of interpretability of differences are often large. As an example, suppose that a self-esteem questionnaire has been administered to adults in the USA and Iran, and that the mean score of the Americans was higher. The seemingly obvious conclusion would be that American adults on average have a higher self-esteem than Iranian adults. The conclusion might be valid, but various alternative explanations can be envisaged. In many Islamic countries there is a norm to be humble and not to brag about one's personal qualities; as a consequence, Iranians may show lower scores. Also, unintended sample differences may account for the difference in scores; the Americans may have had more education (which is known to be positively related to self-esteem). It may well be that if one would examine samples from the two countries that have the same average educational level, the differences in scores on the questionnaire may become smaller or may even disappear altogether. Problems of interpretability are more salient in studies in which no cultural factors are included than in studies that include contextual factors. If a measure of the educational level of the participants as well as a questionnaire on the perceived norm about humility would have been administered in both countries, statistical tools, such as regression analysis or covariance analysis, could have been used to evaluate their influence on the cross-cultural score differences observed.

The second dimension to classify cross-cultural studies refers to the distinction between exploratory and hypothesis-testing studies that is commonly described in introductory textbooks to methods of research. Exploratory studies attempt to increase our understanding of cross-cultural differences by assuming a perspective that is as open and unprejudiced as possible about the nature and size of cross-cultural differences; no prior ideas are formulated about where these differences and similarities are to be expected. Researchers often want to stay "close to the data" and are not inclined to make large inferential jumps. Exploratory studies are helpful in initial stages of a research paradigm in which it is not yet clear to what extent a theory, model, or instrument "works" in another culture.

After the initial stage of exploratory studies, a researcher may feel more confident about what to expect in other cultures. In these cases hypothesis-testing studies can be carried out. In such studies theories or models about the relationship between psychological and cultural phenomena are specified at beforehand and tested for accuracy.

Both types of studies have their own strengths and weaknesses. The power to detect differences and similarities in a large variety of domains in a single study is a strength of exploratory studies. A broadband approach to cross-cultural differences provides an efficient means to collect much information in an efficient way. The openness of the exploratory approach also constitutes its weakness: exploratory studies can easily become "fishing trips" in which the researcher wants to "catch" as much as possible. In their most extreme form such studies address a multitude of cultural differences and similarities without providing any overarching framework for the patterning of the similarities of differences. In sum, exploratory studies are usually good at identifying cross-cultural differences and similarities, but poor at providing a framework to interpret these differences. The latter is the stronghold of hypothesis-testing studies, which combine theoretical precision (testing specific cultural aspects) and statistical rigor.

The third dimension to classify cross-cultural studies refers to the kind of research question addressed in a study. A distinction is made between structure-oriented and level-oriented studies. As an example of a structure-oriented study, one could ask whether the nature of intelligence differs across countries; the question is not how much samples from various countries differ in intelligence, but whether intelligence is different across countries. For example, it has been argued that Westerners tend to approach problems in an analytic way, which means that a problem is reduced to its constituent parts and solving the problem amounts to successfully dealing with all the parts in succession. Easterners, on the other hand, are said to opt more frequently for a holistic type of reasoning, in which the relations between the parts of a problem rather than the parts themselves form the essence of a problem. Whatever the validity of the latter claim, it is helpful to illustrate the issue that in many cases cross-cultural researchers are not interested in quantitative differences, but in qualitative similarities and differences. These structure-oriented studies focus on relationships among variables and attempt to identify similarities and differences in these relationships across cultures.

Level-oriented studies examine the size of cross-cultural differences. Examples are the numerous studies in which the level of individualism and collectivism are compared across countries and the studies in which the school performances of American pupils and Eastern Asian pupils are compared.

Structure- and level-oriented studies are complementary and often follow each other in time. Studies that examine the similarity of structures across countries are often done first. They pave the way for a second wave of studies in which scores are compared across countries. Although the two kinds of studies often do not have the neat temporal separation suggested here, it is important to realize that they address different questions and that a numerical comparison of scores requires that an instrument measures the same in each cultural group considered. Take the cross-cultural study of depression as an example. Depression has a somatic component (e.g., sleeplessness and loss of appetite) as well as a psychological component (e.g., feeling down and being pessimistic). There are indications that individuals from different cultures with depressive symptoms show more agreement in their somatic symptoms than in their psychological complaints. To some extent this may be a consequence of differences in norms about expressing personal feelings to others. A comparison of depression scores obtained in different cultures can show misleading results if the symptoms (or at least the tendency to report these) are not identical across cultures.

Examples

The three classification dimensions (i.e., contextual factors included or not included; exploratory vs. hypothesis-testing; structure-oriented vs. level-oriented) produce a total of 8 (= 2 x 2 x 2) studies, as can be seen in Table 1.

Table 1. Types of Studies in Cross-Cultural Psychology (after van de Vijver & Leung, 1997).

Orientation more on

Consideration of contextual factors

Exploration

Hypothesis testing

 

(a) Structure-oriented

No

(1) Structure-oriented psychological differences

 (2) Structure-oriented generalizability

Yes

(3) Structure-oriented ecological linkage

 (4) Structure-oriented contextual theory

 

(b) Level-oriented

No

(5) Level-oriented psychological differences

 (6) Level-oriented generalizability

Yes

(7) Level-oriented ecological linkage

(8) Level-oriented contextual theory

Table 2.

Type of study

Source and description

Structure-oriented psychological differences

Source: Russell, J. A., & Sato, K. (1995). Comparing emotion words between languages. Journal of Cross-Cultural Psychology, 26, 384-391.

 

Description: The authors studied the meaning and equivalence of emotion words among English speaking, Japanese speaking, and Cantonese speaking individuals. A set of 14 photographs of faces was shown to the subjects and they were asked to judge to what extent the face shown in the picture was an expression of each of 14 emotion words. For any two language groups, a correlation index for an emotion word can be calculated based on the ratings of these groups on the 14 photographs. The higher the correlation, the more similar is the meaning of the emotion word across the two languages. Three comparison groups could be formed: English/Japanese, English/Cantonese, and Japanese/Cantonese, and these three groups could be compared on the correlations of the 14 emotion words. Results showed that the correlations were similar across the three comparison groups for 12 of the 14 emotion words.

Structure-oriented generalizability

Source: Church, A. T., & Katigbak, M. S. (1989). Internal, external, and self-report structure of personality in a non-Western culture: An investigation of cross-language and cross-cultural generalizability. Journal of Personality and Social Psychology, 57, 857-872.

 

Description: The authors studied the generalizability of a well-known Western model of personality, the Five-Factor Model of personality, in a non-Western culture.

Structure-oriented ecological linkage

Source: Van de Vijver, F. J. R., & Poortinga, Y. H. (2002). Structural equivalence in multilevel research. Journal of Cross-Cultural Psychology. PAGES TO BE ADDED.

 

Description: They compared the meaning of Inglehart’s concept of postmaterialism across more than 30 countries. It was found that the concept does not have an identical meaning in countries with low and high Gross National Product.

Structure-oriented contextual theory

Source: Sidanius, J., Pratto, F., & Rabinowitz, J. L. (1994). Gender, ethnic status, and ideological asymmetry. Journal of Cross-Cultural Psychology, 25, 194-216.

 

Description: Based on social dominance theory, the authors proposed that for members of high-status ethnic groups, social dominance orientation (i.e., the desire to establish hierarchical social relationships among social groups) should be positively related to group salience and differential group closeness. Group salience refers to the experienced salience of one's ethnic group membership, and differential group closeness refers to the emotional closeness of one's ethnic group to other ethnic groups. The stronger the social dominance orientation, the more salient is one's ethnic identity and the closer one feels toward one's ethnic group. For members of low-status groups, however, the relationship between social dominance orientation and group salience and differential group closeness should be weaker. This prediction was tested in the US with a group of whites, the high-status ethnic group, and a group of Blacks and Hispanics, the low-status groups.

Level-oriented psychological differences

Source: Guida, F. V., & Ludlow, L. H. (1989). A cross-cultural study of test anxiety. Journal of Cross-Cultural Psychology, 20, 178-190.

 

Description: The authors examined cross-cultural differences in test anxiety between American and Chilean school children. The latter group was found to display higher levels of test anxiety. No attempt was made to evaluate causal antecedents for these differences in this study.

Level-oriented generalizability

Source: Amir, Y., & Sharon, I. (1987). Are social psychological laws cross-culturally valid? Journal of Cross-Cultural Psychology, 18, 383-470.

 

Description: The authors replicated a number of well-known Western social psychological studies with Israeli subjects. The authors were interested in the generalizability of findings from experimental social psychology obtained among Western subjects to an Israeli context. Significant main effects could often be replicated but interaction effects did not travel well.

Level-oriented ecological linkage

Source: Van Hemert, D. D. A., Van de Vijver, F. J. R., Poortinga, Y. H., & Georgas, J. (2002). Structure and score levels of the Eysenck Personality Questionnaire across individuals and countries. Personality and Individual Differences.

 

Description: Differences in country scores on the three personality dimensions in Eysenck’s theory (psychoticism, neuroticism, and extraversion) have been reported. The question was addressed to which country-level variables these differences were related.

Level-oriented contextual theory

Source: Van de Vijver, F. J. R. (1997). Meta-analysis of cross-cultural comparisons of cognitive test performance. Journal of Cross-Cultural Psychology, 28, 678-709.

 

Description: A meta-analysis was carried out in which different models that could presumably explain cross-cultural differences in scores on mental tests were tested. As an example, support was found for the hypothesis that differences in GNP and educational expenditure (per head) between cultural groups are positively related to differences on cognitive test scores.

The eight possibilities are illustrated on the basis of a fictitious set of studies (Table 2 briefly presents real examples, which are not further discussed here). Suppose that we have a theory of emotions according to which each human emotion is a combination of two, independent components: valence (positive and negative emotions) and intensity (low and high intensity) and an instrument that has shown this structure in samples of British psychology students. Each emotion is then seen a point in a two-dimensional space. In the first type of study, structure-oriented psychological differences studies (the names and order of Table 1 are followed here), the researcher may develop a new instrument for a culture in which the instrument has not yet been administered; the development should start from a thorough knowledge of the specific culture. The newly developed instrument is then administered and the researcher examines whether the two-dimensional structure is also present in the new sample.

In second first type of study, structure-oriented generalizability studies, one would accumulate data from various countries with the instrument and check to what extent the two-dimensional structure is found in each of these. In other words, the generality of the structure elsewhere is addressed.

The third type of study, a structure-oriented ecological linkage study, could be used if the two-dimensional structure would not be replicated everywhere. It is the challenge for the researcher to determine which contextual factors influence the poor replicability (e.g., two other factors have been found in some countries). We can investigate whether the countries in which the two factors were not found, differ from the countries in which the two-dimensional British structure was found in country indicators, such as average income, educational level, or extraversion.

In ecological linkage studies one often needs country indicators. The Internet is a rich source of country-level data. Examples of interesting sites are http://www.un.org and http://www.oecd.org, to mention a few. Anthropologists have built a large database, the Human Relations Area File (HRAF; http://www.yale.edu/hraf), with information on a large number of cultural characteristics, ranging from birth practices to death rites. The HRAF database differs in important respects from the country-level databases compiled by supranational institutions like the United Nations. Whereas in the latter institutions there is an emphasis on economic indicators, the HRAF database contains more data that are presumably more familiar to psychologists, such as socialization practices. Furthermore, the HRAF base contains data per ethnic group while databases from the United Nations have country as unit. Finally, in the HRAF database there is an emphasis on subsistence societies of hunters and gatherers, while the databases of the supranational institutions involve nation states.

The fourth type, structure-oriented contextual theory-based studies, tests theories about cross-cultural differences (or similarities) in structure. In particular when the generalizability studies just described (type 2) would show that the two-dimensional structure does not hold in all cultures examined, the need will arise to learn more about the background of the differences. Structure-oriented contextual theory-based studies could test to what extent the two-dimensional model of emotion fits better in countries with a higher level of formal education, collectivism, higher proportions of religious people, with less stringent socialization patterns, to name but a few (arbitrary) examples; in general, in structure-oriented contextual theory-based studies a researcher tests hypotheses that could explain the differences in fit.

Level-oriented psychological differences studies test the presence of cross-cultural differences, often using a t test or analysis of variance. These studies are popular in the literature. Suppose, that we administer a questionnaire measuring our two emotion dimensions in different cultures. A level-oriented psychological differences study could test the presence of differences in valence and intensity across cultures. In such studies the researcher typically does not have prior ideas about where to expect cultural differences on any dimension, but employs well-established statistical techniques (e.g., a t test) to determine if the score differences observed reflect real differences or are mere sample fluctuations that are so small that they can be safely ignored.

Level-oriented generalizability studies usually build on studies in Western countries and examine to what extent differences observed there can be generalized to other cultures. Suppose that we have asked participants to indicate the valence and intensity of emotions experienced during the last week and that we consistently find that in Western countries women are more expressive than men, as indicated that women show a higher variation in reported emotions. Level-oriented generalizability studies would be studies in new cultures that address the generality of the Western gender differences.

If these studies would find that the gender differences are not universal, the next question would be to examine which country factors could be held responsible for the difference. A level-oriented ecological linkage study could address this question by linking the gender differences observed in the various studies to various country indicators, such as gross national product and average level of education.

Finally, level-oriented contextual theory-based studies test a theory of such differences. For example, Williams and Best (1990; reference: Williams, J. E., & Best, D. L. (1990). Measuring sex stereotypes: A multination study. Beverly Hills, CA: Sage) have argued that "national development may be accompanied by a reduction in the degree in which women and men are viewed as 'psychologically different'" (p. 253). Based on this model, we would predict a negative correlation between gender differences in standard deviation of the valence and intensity of reported emotions on the one hand and some indicator of national development (such as Gross Domestic Product per head) on the other hand.

Conclusion

Hundreds of cross-cultural studies are published each year. These studies can be seen as belonging to different types: they are exploratory or test hypotheses, they include or do not include contextual variables, and they focus either on the structure of psychological phenomena or they compare score levels obtained in different cultures. The rank numbers of the eight types of study in Table 1 should not be seen as rankings going from less to more (or from more to less) valuable studies. Rather, depending on the level of theory and availability of data, co-researchers from other countries, resources and various other issues, each of the eight cells can be appropriate. In each of the cells of Table 1 good and bad studies can be carried out.

The dimensions underlying Table 1 may help to think about existing and new studies. The dimensions may help researchers to appreciate the strengths and weaknesses of studies and may help to think about design and analysis prior to the data collection, which tends to improve the quality of a study.

About the Author

Fons van de Vijver is professor of cultural psychology at Tilburg University. He obtained a PhD from the same university in 1991. The study dealt with cross-cultural differences and similarities in inductive reasoning in Zambia, Turkey, and the Netherlands. He has published on various topics in cross-cultural psychology. His main interests are in cognition, acculturation, and methodological aspects of cross-cultural studies (how can we design and analyze cross-cultural studies so as to maximize their validity?). He has written a book on the latter topic with Kwok Leung from Hong Kong (1997, Sage; see suggested readings). He is the current Editor of the Journal of Cross-Cultural Psychology
Webpage: http://www.kub.nl/webwijs/english/show.html?anr=199788&lang=en
E-mail address: fons.vandevijver@kub.nl

Suggested Readings

Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago: Rand McNally.

Harkness, J. A., Van de Vijver, F. J. R., & Mohler, P. Ph. (Eds.) (2002). Cross-cultural survey research. New York: Wiley.

Jahoda, G. (1982). Psychology and anthropology: A psychological perspective. London: Academic Press.

Lonner, W. J., & Adamopoulos J. (1997). Culture as antecedent to behavior. In J. W. Berry, Y. H. Poortinga, & J. Pandey (Eds.), Handbook of cross-cultural psychology (2nd ed., vol. 1). Chicago: Allyn & Bacon.

Lonner, W. J., & Berry, J. W. (Eds.) (1986), Field methods in cross-cultural research. Newbury Park, CA: Sage.

Naroll, R. & Cohen, R. (Eds.) (1970). A handbook of cultural anthropology. New York: American Museum of Natural History.

Poortinga, Y.H. (1997b). Towards convergence? In J. W. Berry, Y. H. Poortinga & J. Pandey (Eds.), Handbook of cross-cultural psychology (2nd ed., vol. 1). Boston: Allyn & Bacon.

Van de Vijver, F. J. R., & Leung, K. (1997). Methods and data analysis for cross-cultural research. Newbury Park, CA: Sage.

Van de Vijver, F. J. R., & Leung, K. (2000). Methodological issues in psychological research on culture. Journal of Cross-Cultural Psychology, 31, 33-51.

Related Websites

All links listed here are referred to in the main text, where their relevance is further explained.

1. Journal of Cross-Cultural Psychology: http://www.ac.wwu.edu/%7Epsych/jccp.htm
2. OECD (source of country indicators): http://www.oecd.org 
3. Culture and Psychology journal: http://www.sagepub.co.uk/frame.html?http://www.sagepub.co.uk/journals/details/j0160.html
4. TIMMS study, international project to compare educational achievement: http://www.timss.bc.edu
5. Site United Nations (source of country indicators): http://www.un.org
6. Site Human Relations Area Files: http://www.yale.edu/hraf

Questions for Discussion

1. Suppose that you want to carry out a study in which you want to test the theory that holds that cultures that are more religious tend to be employ more authoritarian patterns of socialization. How would you proceed to test such a theory? Which data sources would you use?
2. Various authors have administered the same questionnaire in dozens of countries (examples are the studies by Hofstede, 1980, 2001, and Schwartz, 1992). In each country the questionnaire was administered to samples of at least a few hundred participants. What are the strengths and weaknesses of such large-scale studies?
3. Which kind of studies, exploratory or hypothesis-testing, are more important for advancing our knowledge in cross-cultural psychology? Explain your answer.
4. Which types of studies or research designs do you think are more valuable in advancing psychology? Explain your answer.
5. What are the advantages and disadvantages of studies involving two ore three countries as compared to studies involving ten or more countries?
6. Look up the summaries of a recent issue of the Journal of Cross-Cultural Psychology and indicate where you think each of the articles would fit in Table 1.
7. Discuss how the Internet can be useful in designing, carrying out, analyzing, and writing up cross-cultural research (the distinction between the various types of studies can be helpful here).
8. Can you think of other types of studies that are not included in the classification given in Table 1?

Go to top of page | Home | Brief Table of Contents | Complete Table of Contents