Research Design and Research Strategies
Jeffery C. Johnson
We need a powerful mode of argumentation, a mode that ensures
we can represent our representations in credible ways. In such
worlds, a systematic argument enjoys a
star-spangled legitimacy. We need a way to argue what we know
based on the process by which we came to know it. That's what
I seek, not as the only possible representation that our field can offer, but
as an essential lever to try and move the world. Michael A.
Agar (1996:13)
Introduction
In a complex world of competing arguments, who is to be
believed or trusted? Are data themselves, independently of
how they were conceived and collected, proper evidence for making a case? Although some may be swayed by the elegance of a well-written essay, for many it's crucial to know something
about the author , his or her motivations,
experiences, skills, methods of investigation, and so on before passing
judgment on the conclusions . In
Agar's statement above, we get the impression that a credible argument should
be systematic and based on a process that
informs us about how researchers came to know
what they know.
It
is the articulation of this "process by which we came to know it"
that reflects the elements of research
design. For Stinchcombe (1987:23), the observations produced
by how a study was designed are fundamental
to the proper assessment of empirical evidence: "We always want to reject
evidence if it can be explained by the design of the research or by a large number of small,
unorganized causes. " Some things, like perceptual
errors, that hinder our observation may be beyond our control. Some
things, like site selection, sampling, measurement, and recording are at least
partly within our control. The value of empirical evidence
can only be properly evaluated by understanding the details of how the research was conducted.
According
to Pelto and Pelto (1978:291): "Research
design involves combining the essentials of investigation into an effective problem -solving sequence. Thus
the plan of research is a statement that
concentrates on the components that must be present in order for the objectives
of the study to be realized. "
This statement illustrates at least two important elements of research design.
First,
research design involves an a priori plan or strategy for all phases of the research (such as data collection and analysis)
including, for some researchers , the
production of the final product (like an ethnography
). By definition, a plan cannot deal with the unanticipated
or unknown realities of research , such as
tragedies or acts of nature that disrupt fieldwork
. A good understanding of the research
problem and the research
site allows us to plan for some contingencies, but there is no research design crystal ball. In
fact, chance factors often lead to great discoveries or unexpected findings . Still, while luck
plays a role in research , planning for such
luck is not within the realm of research
design (Kirk and Miller 1986).
Second,
an idealized plan gives guidelines for linking theory
to the methods of data collection and analysis that yield either valid or
"defensible" results. I use "defensible"
in addition to "valid," which I normally use, to make readers aware
that I am broadening the traditional
application of research design to include the
variety of research strategies found
in anthropology today. Interpretive,
hermeneutic, and postmodern approaches make
little explicit reference to ethnographic
design issues, but well-written examples
from ethnography may provide "moral
evidence" to deal with current social problems , moving people (including
politicians) in ways that numerical facts can't (Seidman 1994:134). Nevertheless, a well-articulated project design helps "to
promote the effective conduct of research
," whether one starts from a positivist or humanist perspective (Ellen
1984:158).
On
a practical level, good research design is
essential in the competition for research
grants and contracts. There is much variation in what funding
agencies and foundations expect regarding research design. One agency
may require a detailed description of the proposed project paying attention to
the research design logic of science
(for example, validity, reliability , hypotheses , etc. , see also
Plattner [In press]); others may require a description of the
research problem
and site but require less detail about the methods of data collection and analysis.
All funding agencies expect a well-organized outline of the
proposed project-one that meets the design expectations of peer reviewers and
agency personnel.
A
distinction needs to be made between what's
sometimes called the laundry-list component of research
and research design. The
laundry-list component is important. It involves details
about getting into and out of the field situation, travel arrangements, getting
proper government permissions, making contacts at the field site, arranging for
living accommodations, and so on. Design, on the other hand,
involves the methodological and analytical
details that contribute to the credibility, validity, believability, or
plausibility of any study . In
this chapter I concentrate on elements of design related to the production of
valid results or a believable ethnographic
account.
The
Need for Design
Evidence for the power of research
design is all around us. The invention of the simple
control/treatment design of clinical trials allowed researchers in this century to evaluate
competing therapies and to select the ones that worked best. One
result is that infectious childhood diseases that killed thousands of young
people a century ago are today only a memory in industrialized countries. The lessons learned from controlled experimentation are applied today to the policy
arena where groups are in conflict over resources or because of social inequalities (Johnson and Pollnac 1989; Porter
1995). Members of such competing groups-such as large-scale
commercial producers, commodity producers, environmental groups, and real
estate developers-believe strongly in their positions. They
have evidence, often anecdotal, that their positions are credible. Without some unbiased means for assessing the evidence, the
truth is only be a matter of who has the most political clout.
The
outcry for a ban on nets in tuna fishing is
a famous recent example. Environmental organizations launched
campaigns to ban nets in tuna fishing because
dolphins are often caught incidentally in that fishery. Media
campaigns in the U.S. showing pictures of dolphins being caught in nets
(generally not in U.S. waters), contributed to Florida's totally banning fishing nets-even though no marine mammals were
threatened by the use of nets in Florida
waters. Thus, policy emerges from interactions between groups
of differing political, ideological, social
, and economic backgrounds.
There
has been similar concern over the incidental catch of harbor porpoises by net fishers in New England (Schneider 1996). This case led to a systematic
test of a technology that might ameliorate
the problem . Wildlife
conservationists petitioned the U.S. federal government in 1991 to declare
harbor porpoises a threatened species. In response, the fishing
industry proposed the voluntary use of "pingers" an underwater
acoustic device-to keep porpoises from their nets. The
effectiveness of the device, however, was in question, and there was no firm
evidence in the literature about it. Fishers petitioned the federal government to
fund a study of pinger effectiveness. The study used the classic
control/treatment design in which catch rates for a set of nets with pingers
were compared to catch rates for set of nets without pingers.
In
the first experiment , the control net
caught 10 porpoises while the treatment net caught none. Some
conservationist groups claimed the study was
biased in that the treatment nets were placed in areas known not to have large
numbers of porpoises. So another study was conducted placing experimental treatment and control nets in the
same proximity. This time, the treatment net caught only 1
porpoise while the control net caught 32. Some environmental
groups were still concerned that evidence with more statistical power was needed . Lobbying efforts by fishers yielded more funds for a larger, more
comprehensive study involving more than
10,000 fishing nets. Both
control and treatment nets were outfitted with pingers, but only the pingers on
treatment nets would activate once placed in the water. Thus,
fishers were blind as to which nets were
control and which were treatment-a classic double-blind experimental design. Again
the evidence was impressive: The treatment nets caught 2 porpoises (1 was
thought to be deaf), while the control nets caught 25.
The
issue is still under debate, but this series of studies
illustrates how the elements of research
design help muster evidence in light of competing beliefs and philosophies. In each successive study ,
investigators tried to control for as many extraneous variables as possible so that the hypothesized
effect could be assessed (that is, the effectiveness of pingers compared to not
using pingers). The logic
of the research design contributed to the
production of credible results.
Although
the power of experimental design is evident,
concern for its application in anthropology
-particularly cultural anthropology -has
been limited . Some early
exceptions include Brim and Spain 's (1974) book on hypothesis -testing
designs, Pelto and Pelto's (1978) book on research
methodology in cultural anthropology , and Naroll and Cohen's (1973) A
Handbook of Method in Cultural Anthropology
, which has several chapters that address issues in research design (LeVine
1973; Sechrest 1973; Spindler and
Goldschmidt 1973). Bernard (1994) has elaborated in more
detail on issues of design, but his treatment is necessarily limited , given his task of describing the
range of methods available to anthropologists
.
If
research design gets relatively little
attention from anthropologists , other social scientists
have written volumes about it. What should we make of this apparent dearth of specific
treatments of research design in cultural anthropology ? I don't think
we should make too much of it because the important elements of research design-reliability
, informant accuracy, validity, objectivity, and operationalization of theoretical concepts-have been present in the writings of cultural anthropologists even before Boas .
Boas , Malinowski, and Research Design in the Scientific Tradition
Boas and most of his students advocated a natural science logic
in the collection of ethnographic materials
and a true concern for the collection of reliable
data that could lead to the production of valid theory
. Yet, despite his concern for scientific
method, Boas was more explicit about his
methods of data analysis than about his methods of fieldwork and data collection (Ellen 1984; Boas 1920). Malinowski
was also concerned with the aims of science
and with methodological rigor. His earliest contributions, however, were more a demonstration
of the value of ethnographic writing -his "unusual literary sense"
(Lowie 1937:231)--rather than of methodological
details of proper ethnographic fieldwork (Ellen 1984).
A
good example of this tension between the stated early concerns for the methods
of science and the actual use of such
methods in ethnography comes from
correspondence between Boas and his student Margaret
Mead during her first fieldwork in Samoa
. As Orans (1996)
describes it, Mead wrote to Boas
with her concerns about possible violations of scientific
principles in the data she had collected to that point. She wrote of her doubts about the comparability of
cases and about her ability, or even the need, to do a quantitative comparison of the similarity of attitudes among
the adolescent girls in her study . She had concerns-and I believe she thought her mentor, Boas , would feel similarly-as to whether a
valid comparison of this type could be made
given the selection process for her sample of girls.
The
constraints of field research may lead one to
stray from the idealized prescriptions of a research
design, but Mead was attempting to exert her
authority without necessarily following the research procedures advocated by Boas and others. Orans says: "What she wants is permission
to present data simply as `illustrative material' for the representativeness of
which one will simply have to take her word" (p. 127). What
is most surprising is Boas 's response to Mead . He writes :
I am very decidedly of the opinion that a statistical treatment of such intricate behavior as the one that you are studying , will not have very much meaning and
that the characterization of a selected number of cases must necessarily be the
material with which you operate. Statistical work will require the tearing out
of its natural setting, some particular aspects of behavior which, without that setting may have
no meaning whatever. A complete elimination of the subjective use of the
investigator is of course quite impossible in a matter of this kind but
undoubtedly you will try to overcome this so far as that is all possible. (from Orans 1996:128)
This response is important for at least two reasons. First, it demonstrates the differences between the stated scientific objectives of ethnographic work as advocated by Boas and the actual practice of ethnographic research
. There appears to be a perception that a systematic treatment of the data will have to
be abandoned to preserve context and meaning. Ironically,
this concern for context and meaning over methodological
rigor, particularly for those in search of theoretical
foundations (that is, the Boasian idea of
data leading to the construction of theory
), would ultimately hinder the comparability of data from different ethnographic sources (see Moran [1995] for a
recent discussion of this issue and see Ember
and Ember, this volume).
Second,
Boas 's concern for contextual meaning over
the statistical analysis of data was
prophetic. Rightly or wrongly, the preeminence of
contextualization has been a consistent issue in ethnographic
research and has often clouded issues in research design. The idea
that quantification detracts from context and meaning in the ethnographic endeavor-evident even in the time
of Boas -and a failure to understand that systematic methods-whether quantitative or
qualitative-help minimize the subjectivity of the investigator have impeded the
development of well-delineated research strategies in anthropology
.
It's
tempting to explain this as the consequence of the intensely personal nature of
fieldwork , and the complexity of a holistic
approach. However, this debate has its parallel in sociology
where schools such as ethnomethodology and symbolic interactionism developed in
response to the largely quantitative macro-level focus
of the discipline. These micro-level approaches are attempts
to get at a better understanding of meaning in everyday life (Cook 1994).
Boas 's final sentence in his response to Mead illustrates that even at this early stage
the issue of the subjectivity of ethnographic
research was of concern. There
was a faith, however, that awareness of the potential biases associated with
the subjectivity of the investigator could be dealt with in some reasonable
way. A further irony is that the one thing that might have
lessened potential subjectivity biases-the use of standardized methods-was
rejected outright because meaning might be compromised. Mead 's position on these various elements of research design provided fuel for the continuing
discussions about the validity of her
original findings (Brim and Spain
1974; Freeman 1983; Orans 1996).
Thus,
while early British and U.S. anthropologists
advocated the scientific method in ethnographic research
, there is little evidence that they considered appropriate design issues when
they actually did the research . As Urry (1984) sees it:
In Britain the claims that anthropology
not only studied a distinctive body of data
but also that it possessed a sophisticated methodology
to collect these data, was an important factor in the establishment of anthropology as a discipline. This
was less necessary in America where, by the late nineteenth century, anthropology was already established in
universities, museums and government agencies. But in spite
of claims to scientific methodology , particularly in the British tradition , there are surprisingly few details
about actual methods anthropologists used in
the field, beyond a few first principles and illustrative anecdotes. There was a wide belief among British anthropologists that fieldwork could not be taught to new recruits,
but could only be experienced by individuals
in the field. In the American tradition
texts provided what was regarded as an objective body of data, whereas the
British tradition was more a matter of
subjective experience . It
is a strange paradox in the development of field methods that the scientific study
of other cultures has been built upon such a foundation
. (p. 61)
There is much anecdotal evidence for a belief, across the
British and U.S. traditions , in a
trial-by-fire method of training for ethnographers
. This belief supports the current lack of formal training
in methods and research design in anthropology . Agar (1980)
and Bernard (1994) relate stories about Kroeber's recommendations regarding the
teaching and conduct of ethnographic research . In the stories,
one concerning Wagley's teaching of a field methods course and one concerning a
graduate student at Berkeley asking for
advice before going to the field, Kroeber's response was a terse, one liner
that reflected the attitude of the times. Even in the late
1960s, when concern for methodological rigor
was probably at its peak in anthropology ,
many treatments of research methods and
design in the literature played down the need for more systematic methods and design detail,
particularly with respect to hypothesis -testing approaches (LeVine 1973). A good example
of this is a book by Thomas Rhys Williams (1967) published in the Spindlers's
series on field methods. Williams writes :
I believe that only someone wholly involved and fully
immersed in fieldwork can really communicate
the essence of cultural anthropology to students or general readers. And
since I have indicated here that research in
culture involves a great deal of unique personal experience
for the anthropologist , I have taken the
position that it is probably unlikely there can be a rigorous, systematic , and formal presentation of methods
in the study of culture like those of the
natural sciences and that there are overriding concerns among many
sociologists, psychologists, and economists. I find this stance comfortable, for it is my
conviction that so long as prime theoretical
concerns in the study of culture are an
attempt to record and understand the native's view of his culture and the
objective and historical realities of culture, then methods for field study will have to reflect the end purpose of
making a whole account of a part of the human experience
. (pp. 64-65)
LeVine (1973) and others
(Johnson 1990) make the point that the
nature of fieldwork , in terms of its
requisite huge investments in time and geographical focus , has often limited
the attractiveness of more formal research
designs because of its commitment to studying
specific problems in a specific way. The realities of fieldwork
often dictate the need to change the problem
focus or, finding
that the proposed hypotheses are
inappropriate to the cultural setting under study
, the need to somehow salvage the research .
Laboratory
and survey researchers have some flexibility
to change the problem focus and study
populations in light of emerging problems , but field workers are limited in their ability to do so. Thus, the idea of researchers
"putting all their eggs in one basket" may have limited the a priori formulation of problems in fieldwork
(LeVine 1973:184). Further,
the huge investment in time and resources limited
another important goal of science , that of
replication, since an ethnographer couldn't
realistically be expected to replicate someone else's work. The
"my natives" or "my village" mentality of some and the fact
that careers were made by discovering new theories
or describing exotic less well-known cultures has certainly inhibited
replication efforts (Johnson 1990).
Contemporary
Design Issues in Cultural Anthropology
There
is an ongoing debate in cultural anthropology
concerning science and its role in
contemporary research . A discussion of the basic arguments as related to
epistemology, objectivity, reality, authority
, and the like are beyond the scope of this chapter (see Schweizer in this
volume). Suffice to say that traditionally
, research design and its logic have been associated with science and an underlying belief in objectivity
and explanation . The
historical tension between interpretive and scientific
approaches in anthropology has given way to
an outright rejection by some anthropologists
of science and its logic of design. To say that
the research design logic of science
has been replaced by something that is recognizable as the research design logic
of, say, postmodernism would, I think, be
misleading. It is not that interpretive approaches lack some
form of research plan; but
the term "design" itself smacks of the very formalism that is being
rejected. A more appropriate term that would encompass the
diversity currently found in cultural anthropology might be "research strategy
. "
Figure I is a taxonomic
characterization of the different types of research
strategies found
in contemporary cultural anthropology . The figure distinguishes
between strategies within the realm of
interpretive studies and those using systematic strategies
that have more of the elements of science . This is a highly simplified representation. Many
examples of research in anthropology fall within the two extremes of
the continuum. Under the systematic
distinction are the two primary categories of exploratory and explanatory approaches, each entailing a
specific design strategy . The
light line connecting the two categories indicates their complementarity and
interrelatedness in that a design may include both within an overall research design framework. These
approaches are by no means mutually exclusive in approaching a research problem
(see section on Research Design in Systematic Research
, below).
In
its most extreme form, systematic strategies tend to involve the search for explanations of phenomena and the pursuit of theoretical foundations
. In searching for such foundations
, there is a need for objectivity, replication, and control over possible
sources of error leading to a valid assessment of a given theory . Epistemologically, systematic work is objectivist. Its
practitioners are ultimately interested in research
findings that approximate an external truth.
As a result, the assessment of any theory involves research
designs more heavily concerned with the means-the research
process, rather than simply the way the study
was written or argued-since the validity of study results depends on the scientific soundness of the research design. For any
given research problem
, it is the purpose of research design to
ward off as many threats to validity as
possible. This leads to designs that involve concern for a
higher degree of methodological and
analytical detail, whether quantitative or qualitative. In
this line of thinking, the researcher is a
field-worker-as-writer .
Figure 1. Types of anthropological research
strategies and their features.
Exploratory:
Exploratory approaches are used to develop hypotheses
and more generally to make probes for circumscription, description, and
interpretation of less well-understood topics. This is
similar to the grounded theory ideas of
Glaser and Strauss (1967), where exploratory descriptive research leads to the development of more
meaningful theory and measures . Exploratory research can be the primary focus of a given design or just one of many
components.
Explanatory : Explanatory
approaches generally involve testing
elements of theory that may already have
been proposed in the literature or that have been informed by exploratory research . Research designs in this mode are determined a
priori and their primary purpose is to eliminate threats
to validity, where validity is concerned with whether things are what they
appear to be or are the best approximation to the truth (Cook and Campbell
1979). In this enterprise, explanation
can involve a general search for causality or prediction.
Interpretive
strategies , on the other hand, differ from systematic approaches in that they question a researcher 's ability to maintain objectivity,
particularly in the ethnographic context
where the ethnographer is often the
instrument of measurement. A variety of names are used in
the lexicon of social scientists that can be associated to varying
degrees with an interpretive strategy . Phenomenology, hermeneutics, symbolic anthropology , interpretive anthropology , interpretive interactionism,
deconstructionism, postmodernism , and
constructivism, to name a few, question, in one way or another, some or all of
the ontology, epistemology, and methodology
of systematic approaches. Although
some of the older interpretive strategies
that emerged from the scientific tradition in the social
sciences, such as early interpretive anthropology
, still adhered to some logical empiricist methodology and maintained a degree of belief in
ethnographic authority
, more recent approaches, such as postmodernism
and constructivism, are more radical in their sweeping rejection of scientific method and design logic (see Schwandt 1994). In
contrasting Geertz and early interpretive anthropology
with some of the later postmodern turns of
such ethnographic writers as James Clifford, Rabinow (1986)
observes:
At first glance James Clifford's work, like that of others
in this volume, seems to follow naturally in the wake of Geertz's interpretive
turn. There is, however, a major difference. Geertz
(like the other anthropologists ) is still
directing his efforts to reinvent an anthropological
science with the help of textual mediations.
The core activity is still social
description of the other, however modified by new conceptions of discourse, author , or text. The other
for Clifford is the anthropological
representation of the other. This means that Clifford is
simultaneously more firmly in control of his project and more parasitical. He can invent his questions with few constraints; he must constantly feed off others' texts. (p.
242)
There is a fundamental belief that the intersubjective,
everyday meanings and how they are produced, maintained, and changed in any
given context often defy objective study and
explanation . Practitioners
of almost all interpretive paradigms are searching in one way or another for
some understanding (verstehen) rather than for some explanation of social
phenomena. However, some interpretive work is more similar
in nature to the exploratory or descriptive strategies
found under the systematic side of Figure
1 than to some of the more radical forays into, for example, postmodernism . Thus, the
rather simple characterization of research strategies found
in Figure 1 attempts to recognize the
variation inherent in the range of work found
in contemporary anthropology by placing
"interpretive anthropology "
adjacent to "exploratory/descriptive" (see, for example, the work of
Zabusky 1995). Discussions
about this debate can be found in Seidman
(1994), on the one hand, and Faia (1993), on the other, and, more specifically
for anthropology , by Kuznar (1997).
An
important implication here is that scholars who follow this line of inquiry are
searching for local rationales rather than nomothetic theory or universal foundations and may be more interested in
conveying a moral tale of some type rather than a value-free account (Seidman
1994). Further, the purpose of research
strategies under these interpretive paradigms
is more focused on the production of a believable or plausible account or story
rather than a single depiction of the truth, since it is thought that there are
a multitude of plausible accounts rather than just a single true story. Epistemologically, interpretive paradigms are subjective, with findings that are value mediated or even
created. Thus, there is less focus
on the means of research , such as methods of
data collection and analysis as found in the
systematic strategies
, and more on the ends of research -the ethnographic or literary product. In contrast to the field-worker-as-writer , we find
the writer -as-field-worker (Denzin and
Lincoln 1994).
For
scholars like Geertz, analysis of ethnography
has less to do with the methods of observation and description than the
inscriptions and writings concerning the
meaning of human action. In many ways, this blurs the
distinction between what is anthropological
and what is literary. More extreme forays into experimental ethnography
have blurred this distinction even further, and there is more of a focus on writing
strategies that include such approaches as
montages, evocative representations, polyvocal texts, and even ethnographic fictions (Denzin and Lincoln
1994). While systematic
analytical paradigms are primarily concerned with threats
to validity, recent interpretive paradigms are focused more on threats to believability -as in "Do you
believe my story? " (Tyler 1991:85)-or, in critical theory , threats
to trustworthiness (Kincheloe and McLaren 1994). If we talk
of an interpretive method, particularly with regard to postmodernism , it more than likely involves
both the researcher 's immersion into the
cultural context of the actor(s) and some means, usually literary, for
conveying the understanding gained from such an immersion.
As
stated, many interpretive studies are closer
in character to exploratory and descriptive research
in the systematic mode than to some of the
more extreme postmodern studies . A good example of
this is Zabusky's (1995) ethnographic study of cooperation in European space science that she admits "took the form of
mutual exploration rather than unidirectional examination" (p. 46). She contrasts her study with
research on cooperation by "experimental " psychologists, emphasizing
the cultural and social orientation of her
work and the importance of considering context (social
, cultural, political, etc. ) in her analysis. Following in the "thick description" tradition of Geertz, Zabusky clearly believes in
some kind of ethnographic authority . In a short methodology section, she discusses the challenge
of conducting participant observation research
in this rather complex, geographically dispersed, cross-cultural setting. She also discusses the rationales for selecting the site and
the group she studied , problems of working in a linguistically and
technically diverse social milieu, the use
of semistructured and unstructured interviews, and the effect of her role as ethnographer on informant relations and data
quality. Although Zabusky doesn't talk specifically about
design or about concerns for potential threats
to validity, there is implicit concern for such issues throughout the ethnography .
In
contrast to Zabusky, there is a body of interpretive work in anthropology that is more extreme in its
rejection of systematic design issues. Ramos (1995), for example, has recently published an ethnography based on a rewrite of her 1972 ; : dissertation, with additional ethnographic
insights. She rejects the "anthropological `j austerity" of her
original work in favor of an "intersubjective understanding" that
captures the "flavor" of her ethnographic
encounter with the Yanomami. To her, the original work was
"old-fashioned and theoretically
unsophisticated" and had to be replaced by a more reflexive work. This contrast between the old and the new reflects the
increased variation in epistemological emphasis in the field that has developed
over the last 30 years. As Ramos sees it, "I found myself making forays into the
self-conscious meanderings of reflexive anthropology
in order to shift the axis of analysis from the skeleton-like dissertation to
the flesh and blood of ethnography "
(p. 6).
Along
with this shift came the freedom not to be concerned with issues of bias and
validity or with the need for working systematically, thus allowing for a less
restrictive ethnographic narrative. Although Ramos discusses informant interviewing and various
sources of data, her introduction is largely devoted to discussions of her reliance
on her own memory in writing the ethnography and the shift in the narrative
between synchrony and diachrony. Thus, there is little discussion of research
design and methods of data collection as might be found
in work in the systematic tradition . Instead, Ramos
emphasizes the emergent and reflexive nature of data and the literary strategies used in producing the ethnographic product. Other
examples in this vein include Panourgia's (1995) use of we and they in her
"Athenian Anthropography" and Behar's (1993) use of montage in her
collaboration with a single woman in the telling of that woman's life story. Behar discusses the multiplexity of roles, in that she was
variously involved as "priest, interviewer, collector, transcriber,
translator, analyst, academic, connoisseur, editor, and peddler" (p. 12).
The
idea of a montage as an organizing principle was also central to Taussig's
(1987) historical and ethnographic account
of shamanism, colonialism, and terror in South America. This
work is important in at least two ways. First, it is
representative of the genre that rejects explanation
in favor of conveying a moral tale. Its purpose is not a traditional attempt at explanation where facts are considered real,
but political interpretation and representation of facts, independent of their
"realness. " Second, Taussig uses the
"principle of montage" as a means, at least in his view, for better
relating the lessons of history. As he states:
As against the magic of academic rituals of explanation which, their alchemical promise of
yielding system from chaos, do nothing to ruffle the placid surface of this
natural order, I choose to work with a different conflation of modernism and
the primitivism it conjures into life-namely the carrying over into history of
the principle of montage, as I learned that principle not only from terror, but
from Putumayo shamanism with its adroit, albeit unconscious, use of the magic
of history and its healing power. (p. xiv)
These examples offer only a brief glimpse of the range of
possible strategies in use by interpretivists
in anthropology . For
some, interpretive work is an exploratory enterprise with an implicit concern
for methodological issues. For
others, interpretive work is concerned more with the strategies and methods of ethnographic presentation and with the
reflexive character of the ethnographic
enterprise. Thus, traditional
methods sections are replaced by discussions
on how to read the work or on the particular methods used in writing the ethnography
itself (see, for example, Panourgia's discussion
on the use of the parerga).
In
the following pages, I focus primarily on research designs in systematic research
. For further discussion
of research strategies
in the interpretive mode, see Fernandez and Herzfeld (this volume).
Research Design in Systematic
Research : The Challenge of Making a Case
In some social science disciplines, like psychology, the
design of research is driven by features of
the analysis. Analysis-of-variance models and multigroup comparisons (factorial designs) may dictate the
whos, whats, and wheres of a given project. In sociology,
multiple regression models, structural equation models, and path analytic
models (all related analytical techniques) have influenced the design of survey
research . Ethnography , referred to as the anthropological method by William Foote Whyte
(1984), has influenced the nature of design in anthropology
, but in profoundly different ways.
Whereas
the analytical techniques most often used in psychology, sociology, and
economics often led to rather standard designs, in anthropology the eclectic nature of ethnography leaves the design of research more open ended. There
are generally no ethnographic
"analytical techniques" driving the design, although ethnography has been variously associated with
a number of qualitative methods. The good news is that ethnographic research
is amenable to a wide range of research designs,
including the use of multiple designs within a single ethnographic context. This
allows for flexibility, multiple tests of a theory , increased chances for various types of
validity, triangulation, and the potential for high levels of innovation and
creativity. The bad news is that the open-ended character of
ethnography contributes to a less
well-focused discussion of research design issues in ethnographic approaches. Part
of the confusion stems from a lack of consensus on what ethnography really is (Johnson 1990). To some, it
is both a process and a product (Van Maanen 1988). Although
this process might be equated to a method, it's better to think of ethnography as a strategy
in which a variety of methods can be used in the quest for knowledge (Pelto and
Pelto 1978). Thus, ethnography
should involve multiple methods, both qualitative and quantitative, and may
involve applying more than one research
design. This is particularly true today, given the large number
of computer analytical packages available for analyzing text (see Bernard and
Ryan, this volume). Currently, the qualitative analysis of
text and discourse is no longer: restricted to either interpretive or
exploratory approaches, but can also be used in hypothesis
testing and explanatory
research .
Figure 2 illustrates the relationship between
exploratory and explanatory approaches
within the ethnographic context. This contrast between explanatory
and descriptive or exploratory approaches is commonly made in nonexperimental
disciplines in both the natural and social
sciences. Community ecologists, for example, similarly
distinguish between exploratory or descriptive studies
that seek to describe and determine patterns in ecological data and those studies that specifically seek to predict or test hypotheses
. As with research in
community ecology, ethnographic research can be purely exploratory or
descriptive involving a research process
focused on producing better theory -or
purely explanatory , although this is
usually not the case. Rather, the most common model has
exploratory research informing and
complementing explanatory research . As we will see in
the examples to come, exploratory research is
often an essential component of the explanatory
research process. Exploratory
research may contribute to the production of reliable and valid measures , provide information essential for
constructing comparison groups, facilitate
construction of structured questions or questionnaires, or provide information
necessary for producing a sound probability or nonprobability sample.
The
figure shows that the overall research process is more than just a matter of study design. There is no
substitute for a good theory , and there is
a critical need to link theory , design,
data collection, analysis, and interpretation in a coherent fashion. Design, however, is the foundation
of good research . No
amount of sophisticated statistics, computer intensive text analysis, or
elegant writing can salvage a poorly
designed study . Hurlbert
(1984) emphasizes this in a classic paper on the design of field experiments in ecology. "Statistical analysis and interpretation,"
he says, "are the least critical aspects of experimentation
, in that if purely statistical or
interpretive errors are made, the data can be reanalyzed. On
the other hand, the only complete remedy for design or execution errors is
repetition of the experiment " (p.
189). Redoing an experiment
because of fundamental design errors is one matter; redoing
a year-long ethnographic field study because of such errors is quite another.
Figure 2 shows that the research process involves a simultaneous concern
for the development of empirical statements from theory
(for example, hypotheses ), the
operationalization of theoretical concepts
(for example, meaningful and reliable measures ), design (for example, groups to be studied ), data collection (for example,
qualitative versus quantitative), and data analysis (for example, multiple
regression and text analysis). Theoretical
knowledge is derived either from earlier studies
or from exploratory work. The levels at which theoretical concepts are measured (for example, nominal or ordinal), the
types of sampling strategies used, and the
application of appropriate types of analysis must all be considered as a part
of the design. For example, the particular structure of an
empirical statement or hypothesis will
partially determine the manner in which theoretical
concepts are operationalized and eventually analyzed. (Stinchcombe
[1987] provides an excellent discussion of
how empirical statements are derived from theory
. )
Thus, research design is
more than just methods of data collection and analysis. It
involves constructing a logical plan that
links all the elements of research together
so as to produce the most valid assessment possible of some theory , given some set of realistic
constraints (for example, cost, scope, geographical setting, etc. ). The purpose of research design is to ward off as many threats to validity as possible and to help one
eliminate competing hypotheses . It requires careful attention to detail and, often, an
admission concerning the potential weakness of a given design. Outside
the laboratory, a multitude of influences can threaten
the validity of any conclusions . In natural settings, particularly fieldwork
, there is no perfect design that can control for all possible extraneous
effects at once. A recognition of limitations doesn't
invalidate a study 's results. Rather it creates an open forum that can contribute much to
important theoretical and methodological debates. Without
such attention to good design and methodological
detail, researchers leave themselves open to
one of the worst criticisms of all-of being "not even wrong" (Orans 1996). In other words,
a lack of design and methodological detail
makes it next to impossible to fairly and adequately assess the validity of any
study 's conclusions
such that "rightness" or "wrongness" may not even be
debatable.
True
experiments involve random assignment and
afford the best chances for controlling for things like: the effects of
extraneous factors (that is, unmeasured variables
that might affect the dependent variable); the effects of
selection (that is, comparison groups differ
because of the way they were selected and not due to the treatment); the effects of reactive measurement (that is, the measurement
procedure itself caused a change in the dependent variable); or
interaction effects involving selection (that is, when selection interacts with
other factors to create erroneous findings
). These and other sources of error are all potential rival hypotheses and randomized experiments are best at eliminating the threats of rival explanations
. Designs of this type, however, are often impossible in anthropological fieldwork
. Nevertheless, the principles of experimentation are instructive and are a guide
for understanding potential sources of error, even in a nonlaboratory setting. I borrow terminology from Kleinbaum et al. (1982)
in constructing a typology of research
designs. Included are experiments
, quasi -experiments
, observational study designs, and what I refer to as natural experiments .
Experiments involve the random allocation of
subjects to groups and afford the most control over distorting effects from
extraneous factors. Random allocation produces equivalent comparison groups, and artificial manipulation
of independent variables (also known as explanatory variables
or study factors), with all other variables or factors controlled for, allows for
the most valid assessment of the causal relationship between the independent
and dependent variables or response variables . What separates quasi -experiments
from true experiments is the lack of random
assignment of group members. Random assignment maximizes the
probability that experimental groups are
equivalent on key variables prior to the
introduction of an intervention. Nonrandom assignment lays
an experiment open to validity threats and reduces our ability to make causal
inferences. Observational
studies involve neither random assignment of
members to comparison groups nor the
manipulation by the observer of independent variables
.
This
distinction between experimental and observational approaches is similar to one in
ecological field studies . Hurlbert
(1984) distinguishes between two classes of experiments
. He terms the first manipulative experiments . These are
basically true experiments involving random
assignment, multiple comparisons (for
example, treatment versus control), and the manipulation of independent variables . He refers to the
second as mensurative experiments , which
involve simply the measurement of variables
in space and time and among a number of comparison
groups, without random allocation and the manipulation of experimental factors.
The
primary distinction lies between that of sampling versus allocation. In manipulative experiments
, analytical units are randomly allocated to comparative groups, whereas in mensurative
experiments selection of units is based on
some probability or nonprobability sampling scheme. While
random assignment aids in controlling for confounding variables by producing homogeneous comparative
groups, random sampling of units produces comparison
groups that are representative of such groups. Random
sampling meets the restrictions of some statistical
tests , but it does not afford the same protection
as does random assignment of group members against the potential effects of
extraneous factors. Mensurative designs, then, are observational and characteristic of the types
of comparative designs found in field studies in anthropology
.
Finally,
natural experiments are similar to quasi -experiments
except that the manipulation of independent variables
occurs naturally or is unplanned rather than artificial or directed. Thus, comparison groups may
be chosen on the basis of different levels of exposure to some naturally
occurring or human-induced phenomena (for example, natural disaster, war, or
the building of a dam). Cook
and Campbell (1979) make a similar
distinction but refer to these kinds of natural experiments
as "passive-observational studies . " Anthropologists involved in development and
evaluation research are most likely to use
this design.
True
experiments are, of course, rare in anthropology (but see Harris et al. [1993] for an example of a true experiment
in a field setting). Even in quasi
-experiments , it's often difficult to
manipulate independent variables directly. Howevert, with careful attention to design and ethnographic context, quasi -experimental
and natural experimental designs can be
applied to anthropological field settings,
particularly in evaluation research and
development research . Johnson and Murray (1997), for example, used a quasi -experimental
design to evaluate the use of fish
aggregation devices (FADS) in small-scale fisheries development projects. Two fixed fishing structures
(piers) were pretested for differences in catch rates. Then,
FADS, umbrella-like units suspended in the water column, were alternately
placed at the piers and individual fishers
were interviewed simultaneously during randomly selected times at both the
treatment (the pier with the FADS) and the control (the pier without the FADS)
piers. Johnson and Murray
compared and determined catch rates.
From
a statistical standpoint, designs that don't
involve random assignment including quasi -experiments -are considered observational (Cook
and Campbell 1979). It is
important, though, to contrast quasi -experiments to what Kleinbaum et al. (1982) refer to as observational
studies . The most common
designs used traditionally by anthropologists have been observational in nature. Designs
of this type lack direct control over independent variables
and, thus, have more potential problems with
various types of internal validity and with the ability to assess time order
effects and causality. However, if done properly, such
designs can have increased external validity and generalizability.
Due
to their predominance in anthropology , the
examples that follow are comparative observational
designs. Most research
designs in the explanatory mode, like true experimental designs, are comparative (for
example, control versus treatment). Table 1 describes examples from observational and quasi
-experimental study
designs discussed by Kleinbaum et al. (1982) and Cook and Campbell (1979). More
details can be found in these and other sources
(for example, Robson 1993). In anthropological
fieldwork , these designs and others can be
used in tandem to test or explore components
of a theory (such as combinations of time
series and repeated measures designs
particularly applicable to long-term fieldwork
). For example, in their study
of preschool children, Johnson et al. (1997) used a cross-sequential design, which involved
cross-sectional research on a cohort of
children carried out over time.
When
one is interested in explanation , the
importance of comparative thinking in ethnographic
work cannot be overemphasized. Discussing
"common sense knowing" in evaluation research
, Campbell (1988) gives an important
critique of ethnography . His
idea is that "to know is to compare" is fundamental to explanatory work in anthropology :
The anthropologists have
never studied a school system before. They have been hired after (or just as) the experimental program has got under way, and are
inevitably studying a mixture of the old and
the new under conditions in which it is easy to make the mistake of attributing
to the program results which would have been there anyway. It
would help in this if the anthropologists
were to spend half of their time studying
another school that was similar, except for the new experimental program. This
has apparently not been considered. It would also help if
the anthropologists were to study the school for a year or two prior to the
program evaluation. (This would be hard to schedule, but we
might regard the current school ethnographies
as pre-studies for new innovations still to
come. )
All knowing is comparative, however phenomenally absolute it
appears, and an anthropologist is usually in
a very poor position for valid comparison ,
as their own student experience and their secondhand knowledge of
schools involve such different perspectives as to be of little comparative use.
(p. 372; emphasis added)
While the purpose of experimental
design is to ward off as threats to validity,
there are several types of validity-face, construct, statistical conclusion
, internal, external, etc. In one way or another, various study designs, in combination with other
considerations such as the operationalization of theoretical
constructs and sampling, are better or worse at dealing with each. Here, I stress the importance of thinking through how validity threats have influenced and will influence
observations or data (for a more in-depth discussion
of how these types of validity can impact study
conclusions , see Cook and Campbell
1979). Potential errors and bias creep in at various steps
in the research process. It's
your job to contain these errors. In research design, forewarned is forearmed.
Tables 2 and 3 give examples of threats to internal and external validity as discussed in Cook
and Campbell (1979) for quasi -experimental
designs. Internal validity is concerned with the
approximation to the truth within the research
setting. External validity is concerned with the
approximation to the truth as expanded to other settings-that is, with the
generalizability of research findings . The threats in Table
2 deal with extraneous factors that may account for the presence or absence of
a hypothesized effect (that is, contrast validity with invalidity). In the quasi -experimental case, this means changes between
pre- and posttest, but this way of thinking can be expanded to include
hypothesized effects dealing with differences, similarities, or associations
whether diachronic or synchronic. Cook and Cambell (1979) detail how each of the quasi -experimental
designs in Table 1 are better or worse at
dealing with each of the threats to validity
that are found in Tables 2 and 3. For example,
the pretest/posttest nonequivalent groups design controls for some internal threats to validity, but it's problematic with
respect to controlling for changes due to how groups members were selected
(selection maturation), changes due to how individuals were tested (instrumentation), changes due to the
selection of individuals with extreme pretest measures
leading to regression toward the mean (regression), and changes due to local
events not a part of the study (history). Each of these threats may
hamper a researcher 's ability to assess the
contribution of a hypothesized effect to any changes observed. Similarly,
threats to external validity, such as problems stemming from biased samples or research in atypical or unique settings, can
hamper the generalizability of one's findings
. Kleinbaum et al. (1982) offer a similar
discussion of the strengths and weaknesses of
observational designs in terms of
controlling for threats to both internal and
external validity.
Other
sources of potential bias include sampling error (that is, chance),
nonresponse, the use of imprecise measures ,
data recording errors, informant inaccuracies, and interviewer effects (see
Pelto and Pelto 1978; Bernard 1994). Careful
attention to sampling, whether probabilistic (Babbie 1990) or nonprobabilistic
(Johnson 1990), is essential. Measurement, operationalization of theoretical concepts, and type of analysis used
are other important factors. How reliable are your measures
in terms of precision, sensitivity, resolution, and consistency? Are they valid, particularly with respect to accuracy and
specificity, in that they are actually measuring
what they are intended to measure ? Attention and concern with all the potential sources of error,
whether stemming from how the study was
designed, how the data were collected (for example, face-to-face interviews or
mail-out surveys), or how the data were analyzed (for example, statistical conclusion
validity), will help lead to the production of solid evidence.
Some
Comments on Sampling
Many probability and nonprobability sampling designs are
available for any given research problem . These include systematic sampling, stratified random
sampling, cluster sampling, and multistage sampling. The
selection of any of these designs or the development of some hybrid design
depends on the overall design of the research
itself. The nature of the groups or characteristics to be
compared-in terms of such things as the size of the comparison groups in the overall population , the frequency of characteristics
of interest in the population , the
availability of a sampling frame, the ability to identify members of the population (for example, hidden or clandestine populations )-all influence the choice of a
sample design. But it's not always easy to know who or what
you want to sample and to know enough about these sampling units to derive a valid
sample.
The
selection of units of analysis, whether settings, events, times, households, or
people, is important for understanding a variety of internal and external threats to validity, but it is particularly
important for increasing external validity. We mostly think
of selection in terms of some type of sample units. To
generalize to a target population , the
sample has to be representative of the population
of interest. This is essential if we are to generalize to a
whole population and is generally, though
not always, a requirement for classical statistical
tests .
When
generalization to a target population is the
objective, you should strive to define a sampling universe or frame using a
selection procedure with known error limits and one that represents the population of interest. This
usually entails a random sample of some kind. There is a
vast literature on sampling theory and
random sampling procedures, including discussions
of sample sizes (see, for example, Bernard [1994] for a summary and Babbie
[1990] for detailed discussion of sampling
issues).
Cook and Campbell
(1979) discuss two sampling models for increasing external validity in quasi -experiments
. These models don't necessarily involve random selection
and are consequently less powerful than are random samples. In
one approach, the model of deliberate sampling for heterogeneity, target
classes of units, whether classes or categories of persons, places, times, or
events, are deliberately chosen to represent the range of such classes found in the population
. Thus, testing for a
treatment effect across a wide range of classes in the set of all possible
classes (including both extremes and the modal class) in the population allows the researcher to say something about how the effect
holds in a variety of settings. While this might not be
generalized to the population as a whole, it
does inform the researcher if an effect holds
across wide ranging classes within the population
. The logic behind this
model can be extended beyond the quasi -experimental case to observational studies
. Kempton et al. (1996) used a
static-group comparative design sampling across a range of groups that varied
with respect to their values on environmental issues. Kempton
et al. interviewed members of Earth First (a radical
environmentalist group) and dry cleaning shop owners (who depend on toxic
chemicals for their business).
For
some populations , it may be impossible to
develop a sampling frame from which to draw a sample. In
these cases, there are a variety of solutions, including intercept sampling,
snowball sampling, random walks, quota sampling, and purposive sampling. Each of these approaches has potential problems , and most do not allow for
generalizations about a population since
they involve elements of unknown error even if the method involves some form of
random selection criteria (for example, random selection of locations in which
to intercept respondents).
Nonprobability
sampling methods have come to be associated with qualitative approaches or for
the selection of ethnographic informants,
particularly key informants or consultants (Werner and Schoepfle 1987; Johnson 1990; Miles and Huberman 1994). In some cases, a researcher may not be interested in generalizing
to a population but may just want to know
whether two subgroups obtained from a snowball sample differ with respect to
some variable of interest. In that case, much of the bias in
the sample is a matter of the logic used in
the original selection of sample seeds and any statistical
analysis of the data must be concerned about violations of assumptions for the
particular statistical test to be employed (for example, independence
of observations or random sample from a population
). Such matters are particularly germane for observational designs using various social network approaches (see Johnson [ 1994] for a review).
How
samples are chosen is an important element of any research
design. If you are interested in generalizing to a given population , random sampling of some kind is
essential. If generalization is not a primary goal, then
sampling requirements may be relaxed. In most cases, if you
can use a random sample, do it! No matter what the sampling
method, you should be explicit about how you chose the sampling units. This increases the chances of detecting potential bias and also
makes replication feasible. Replication is extremely
important to external and other types of validity, such as construct validity. Random sampling has been a primary requirement in the proper
application of parametric statistics. If you don't use
random sampling, pay careful consideration to possible violations of
assumptions for a given statistical test .
Recent
developments in randomization and computer-intensive methods of statistical analysis involve less restrictive
assumptions concerning the data (for example, assumption of a random sample
from a population or skewed, sparse, or
small sample sizes), opening the way for the development of new test statistics particularly suited for the problem at hand (Noreen 1989; Johnson and Murray 1997). These
new approaches seem particularly well suited for the imperfect world of ethnographic research
, where the rather restrictive assumptions of parametric analysis are often
difficult to meet. But it is critical to remember the
connection between theory , design
(including sampling), and data analysis from the beginning, because how the
data were collected, both in terms of measurement and sampling, is directly
related to how they can be analyzed. The next section shows
how concern for the elimination of potential errors and bias through design and
attention to methodological detail applies to
discussions about the findings of Margaret
Mead and Derek Freeman
in Samoa .
Mead Versus Freeman
: Research Design as Mediator
Derek Freeman 's (1983)
criticism of Margaret Mead 's work and her findings in Samoa
has led to reactions from anthropologists
who come from different epistemological traditions
. Some have defended Mead
(Shankman 1996); others have pointed to the biases and flaws
in Freeman 's argument (Marcus 1983; Ember 1985). The criticisms and
counter-criticisms are difficult to assess, given the time between Mead 's and Freeman
's studies , the differences in locations of
their work, and the differences in their ideological positions (Ember 1985). Freeman contended that some
of Mead 's informants lied to her and that Mead 's commitment to a particular ideological
position caused her to evaluate evidence incorrectly. We
certainly cannot hold Mead to the design
standards available today. Still, it is instructive to
review her work through a contemporary design lens, noting how slight
modifications in design and method could have thwarted later criticisms.
Mead used what can be referred to as a static
group comparison design with a conjectural
treatment group. The comparison
group, Samoan adolescent girls, was compared to a conjectural treatment group,
American adolescent girls, to test the
proposition that exposure to Western civilization increases adolescent trauma. Implicit in this proposition is the overall theoretical notion that culture is the major
factor contributing to human behavior . Brim and Spain (1974) recognized several problems in the design that could have affected
Mead 's ability to draw valid conclusions .
There
were no equivalent measurement procedures for the two groups. In
her use of a conjectural treatment group, Mead
assumed some things about American adolescents without collecting comparable
data. Mead relied mostly
on herself as an instrument to measure the variables of interest. There
were possible problems with interaction
between selection and the effects of extraneous variables
. That is, any observed difference between the two groups
with respect to the dependent variable, adolescent trauma, might have been due
to one or several extraneous (unmeasured) factors and might have had nothing to
do with the independent variable, exposure to Western culture. In
lieu of the between-culture comparisons , Mead could have made a within-case comparison that would have suffered less from problems with possible sources of error. She could have chosen comparison
groups that were as similar as possible in order to rule out the effects of
unmeasured variables as much as possible. For example, Mead could have
compared girls living in the households of native pastors to those who did not.
She could then have tested
the proposition that exposure to competing standards of sexual morality leads
to higher levels of emotional distress in adolescents.
More
recently, Martin Orans did fieldwork in Samoa
. Some of his experiences were incongruent with Mead 's descriptions. Orans (1996) reanalyzed Mead 's field notes and correspondence and once
again found that her depiction of Samoa as a halcyon society was at odds with his
own impression of Samoa as much more
agonistic. If Samoa was
not, during Mead 's day, a halcyon society,
then her conclusions might have been flawed.
Orans 's work was, of
course, many years after Mead 's, and he worked
at different field sites than did Mead . But, in common with Brim and Spain , Orans
found itemized problems
with Mead 's research
design.
There
was a lack of comparisons between various
sources of data that were crucial to Mead 's
argument. For example, Mead
made an assertion concerning the relationship between the size of residential
units and adolescent troubles. She did not, however, make
any systematic comparisons
among the different units. Similar to the observation by Brim and Spain
, Orans points out that Mead made no comparison
of sexual behavior between girls living in a
native pastor's household and girls living with their own family. There is a lack of well-defined samples for both people and
events. Mead makes
assertions about the rarity of events without any knowledge of the frequency
distribution of all such events. In addition, she had a
tendency to understate the population and
overstate the proportion of girls in her study
. There is a lack of specificity in the development and
operationalization of key concepts. For example, there was
no measurement on which to compare differences in stress experienced by adolescents in Samoa and the United States.
For
Brim and Spain
, and for Orans , Mead 's research
design limited her ability to draw the conclusions she did. More
attention to issues of research design and
methods would have improved her chances to make valid claims and possibly limited later criticism of her work. In ethnographic research , no matter the mix of methods, the
design of the study should allow for an ethnographers ' hypotheses
or hunches to be rejected as well as confirmed (Campbell
1975).
Research Design in Anthropological
Practice: Systematic Research
Strategies
The following examples illustrate some of the issues discussed so far. Several
examples are reviews of studies that
incorporate comparative designs of various types in which nonequivalent groups
are constructed in order to control for as many extraneous factors as possible,
and the manipulation of independent variables
is a function of how comparative groups were chosen. These
examples show how, even for less powerful designs, the interplay of exploratory
and explanatory approaches can aid in
guarding against threats to validity (Robson
1993).
Field
Experiments
In field experiments ,
the experimenter has little control over all
possible extraneous factors and the experiment
may not involve random assignment of subjects to groups. Nevertheless,
field experiments can be quite informative
and, if carefully constructed, can provide formal tests
of hypotheses derived from and complementary
with ethnography . In his
work on "colonizing the night," Melbin (1987) theorized that the
night was a frontier, not unlike the western United States in the nineteenth
century. Frontiers have certain features in common. Among other things, they provide escape and opportunity,
tolerate a wider range of behaviors ,
consist of isolated settlements, have fewer status distinctions, involve novel
hardships, have decentralized authority ,
involve lawlessness and peril, have a reputation for helpfulness and
sociability, lag in the development of policies to exploit and regulate, and
involve a variety of interest group conflicts.
Melbin
conducted four tests of the feature relating
to helpfulness and sociability. He designed a clever experiment in which keys were placed at similar
locations during each two-hour field visit over a 24-hour period covering day
and night. The idea was to see if there were a difference in
key-returning behaviors among the different
times. According to Melbin, "To find a key is to come across an implied need
for help" (p. 75). The hypothesis
was that residents of the night would return keys on average more often than
those of the day. The keys had the request to "Please
Return," with an address encased in plastic (keys dropped in the mailbox
were delivered by the U.S. Postal Service to the address on the keys with
postage due). Each of the keys were coded so they could be
identified as to what time of day they were picked up and from what location.
In
all, 326 keys were picked up, of which 220 were returned. Returned
keys were also scored for the manner in which they were sent. One
point was given for keys dropped unwrapped in the mail, two points for keys
returned wrapped in an envelope, and three points if the envelope contained a
personal note. Contrary to expectations, night-timers were
not more amiable than day-timers in their key-returning behaviors ; in fact, they
were the least "helpful. " However, Melbin's three
other tests supported the hypothesis of more sociability and helpfulness
at night.
Melbin
speculates that the variation in results may have been due to the fact that the
other three experiments involved direct
personal contact among the subjects, while the key experiment involved no such interactions. This example illustrates nicely the importance of not relying
on a single test , but having multiple tests and measures
(Stinchcombe 1987). Had Melbin conducted only the key experiment , he may have come to very different
conclusions regarding the helpfulness and
sociability of night-timers. This example also shows how
readily multiple tests can be incorporated
into a research design within a field
setting.
In
all research , but particularly in field experiments like the one described above, there
should be a concern for ethics and the well-being of experimental participants. Unlike
studies where informed consent is obtained
prior to participation, in experiments like
Melbin's, individuals often participate without knowing about it. The ramifications and consequences of experimental outcomes must be considered
thoroughly before any experimental design is
implemented.
Control
and Treatment in a Two-Community Comparative Design
One of the central concerns of medical anthropologists has been to better understand
the relationship between health-related behaviors
and native perceptions about illness. Young and Garro's
(1982) investigation of treatment choice in two Mexican communities is an
example of a static-group comparison where
the presence or absence of the treatment is based on selection criteria not
directly under the control of the researchers
. One of the primary purposes of the research design was the elimination of competing
hypotheses -the hallmark of good research design-and the testing of the primary hypothesis is an example of descriptive
inference, as opposed to statistical
inference. Descriptive inference is an approach highly suited
for much anthropological research .
An
important issue in this area of research
concerns the factors influencing the use of Western treatments among
non-Western populations . One
explanation views use tied to congruence
between a client's medical beliefs and scientific
medical theory : the higher the congruence,
the more likely the client will choose a physician's treatment. Termed
the "conceptual-incompatibility" hypothesis
, a number of studies have suggested that
such a congruence was the primary determinant of treatment choice among Third
World peoples. Young and Garro took a different stance,
stressing physician accessibility as the most important determinant of
physician use. An important element of this position is that
traditional medical beliefs are not a barrier
to choice of physician treatment.
The
research design included the comparison of two Mexican communities that were
similar in terms of cultural traditions and
economies but varied in terms of access to Western medical services. The town of Pichátaro had restricted access (a 20-minute bus
ride from Uricho), while the town of Uricho had easy access. From
a random sample of approximately 10% of the households in each of the towns,
Young and Garro collected data on the number of illnesses that had occurred
during the previous two months and the treatment each had received. Later, the researchers
collected triads data and what they call term-frame data on informants'
perceived similarity of illnesses.
Young
and Garro tested the two main hypotheses in sequence. They
had to establish differences in treatment choice behavior
in the two communities before they could assess any hypotheses concerning differences in beliefs. Using a standard chi-square test
, the authors found
a significant difference in the frequency distribution of treatment
alternatives between the two towns, with the exception of folk curers. Thus, the two communities seemed to differ in their use of
Western medical services. This established, Young and Garro
could then test the second hypothesis relating to the similarity in beliefs
between the two communities. Ironically, in statistical terms, the authors have more interest in the null hypothesis of no difference in beliefs than in
the alternative hypothesis of a difference in
beliefs between the two communities. Using multidimensional
scaling, Young and Garro (1982) compared the belief data and found striking similarities in the medical
beliefs of communities. They conclude:
On the basis of the data from the triads study and the term-frame interviews, we see
little reason to reject the "null hypothesis
" of no significant differences between the responses of the two groups of
informants. This leads us to the conclusion that the substantial variation
apparent in the use of a physician's treatment between the two samples, a
consequence of differential access to such treatment, occurs without
corresponding degrees of variation in resident's attitudes and beliefs about
illness. (p. 1462)
The authors ' careful
attention to research design and analytical
issues contributed to the production of impressive evidence that casts doubt on
the validity of the "conceptual incompatibility" hypothesis . Note that the
analysis used to test the hypothesis concerning similarities in beliefs
involved descriptive inference, not statistical
inference. Despite the authors
' claims of finding no "significant
difference," there was no real way, at least when the study was conducted, to assess the extent to
which any differences were significant in the sense of statistical probability. Recent
developments in statistical procedures allow
us to assess the similarities in aggregated judged-similarity matrices between
the two communities (see Handwerker and Borgatti, this volume, and Hubert
1987). In Young and Garro's case, a visual inspection of the
graphical representations of the data could lead to no other conclusion than that there was little or no
difference in beliefs between the two communities (see Figure 3). This distinction
is important, particularly with regard to anthropological
research , in that hypothesis -testing
research can be done without narrowly
restricting it to analytical methods using statistical
inference.
There
are, of course, threats to validity in this study . Because respondents
weren't randomly assigned into comparison
groups, it's difficult to know the influences of confounding variables on physician utilization and beliefs
about illness. It is unrealistic to suppose that Young and
Garro could have randomly assigned community members to the different comparison groups in order to control for
confounding variables and then subject their
informants to the treatments of interest. Given a lack of
pretest observations, we can only assume that beliefs were similar prior to the
availability of physicians in Uricho. In lieu of
equalization through randomization, Young and Garro, through extensive ethnographic background research , produced groups that, although
nonequivalent in the quasi -experimental sense, shared similarities with regard to a number of
important characteristics. This isn't perfect, but a greater
in-depth exploratory understanding and an explicit discussion
of design can enhance our chances for the production of valid explanations .
Comparative
Design and Ethnobiology
Ethnobiologists have long debated whether folk biological
classifiers are natural historians who compare animals on the basis of their
morphological characteristics or pragmatists who compare on the basis of the
utility of organisms. Boster and Johnson (1989) explored this issue in an
ethnobiological study of fish . Were individual
informants classifying organisms on the basis of form or function? Boster and Johnson used a
static group comparison design to compare
several groups of expert fishermen with a group of novice fishermen. This is analogous to treatment and control groups without the
random assignment of subjects to experimental
units and where the treatment is implied rather than researcher directed (that is, natural
differences in experience with fish ). In the comparison , both culture and language were held
constant while experience with fish was varied. Four
groups-from North Carolina, East Florida, West Florida, and Texas-were sampled
to examine the effects of different kinds of experience
since there are regional variation in species abundance.
To
ensure that experts were, in fact, experienced
recreational fishermen, the rosters of sport fishing
clubs in each region were sampled at random. The selection
of control group subjects, by contrast, involved a purposeful selection
procedure in which potential subjects were screened for recreational fishing experience
. Using a questionnaire to gain background information, 15
college undergraduates who had the least amount of recreational fishing experience
were selected from two introductory anthropology
classes. These students
were the control group. Each of the four expert groups
comprised 15 subjects chosen at random from a larger sample of recreational
fishermen. Thus the groups to be compared consisted of five groups
of 15 subjects, four consisting of experts and one of novices.
All
the groups were shown cards with artists' renderings and the common names of 43
marine species commonly found from North
Carolina to Texas. Individuals were asked to perform an
unconstrained judged similarity of the fish
-a free pile sort (see Weller, this volume, and Weller and Romney 1988). Further, beliefs about the use and functional characteristics
of the fish obtained from extensive ethnographic interviews were turned into a
sentence-frame completion task described by Weller and Romney (1988). Finally, a measure of
morphological similarities was determined, using taxonomic distances between
pairs of fish . Boster
and Johnson used statistical and graphical methods to evaluate
whether experts' and novices' judgments of fish
, at the aggregate and individual levels, were closer to the morphological
characteristics of fish (taxonomic distance)
or the uses of fish (beliefs about use). Using statistical and
descriptive inference, the authors concluded
that whether informants use form or function for classification depends on the
knowledge base of the informants and the methods used to test their knowledge (see Figure 4).
Some
of the criticisms of the Young and Garro study
apply to this example as well. Lack of random assignment of
subjects to treatment and control groups and pretest observations limit the
ability to make causal inferences. But the in-depth ethnographic background research , the particular structure of the hypothesis , and the overwhelming reliability of informant responses make for
more confidence in the possible validity of the study
's conclusions . Exploratory
Research and the Development of Cultural
Models
Often the primary objective of research
design is more a matter of discovery and exploration than the testing of hypotheses
. Although such designs are less driven by an established theoretical framework, there still is need to
pay careful attention to a number of design details in the proper development
of new theories and models. An
example of research in this mode is Naomi
Quinn's (1996) development of Americans' cultural models concerning marriage.
There
is a body of literature that views the interaction of culture with the
individual as so deeply unique and personal as to not be researchable in terms of cultural universals,
coherence, or even sharing . In
contrast, Quinn views culture as being shared
-that there are cultural models for a variety of domains that are widely held
in common, and that these models can be developed from the discourses of
cultural members.
Based
on in-depth interviews with 22 informants, Quinn (1996) attempted to build a
cultural model of Americans' reasoning about marriage. Because
she was interested in a model that was shared
, it was crucial to interview a wide range of couples who, although of the same
culture, were not just from one region of the country or of only one ethnicity,
religion, or social class. As
she puts it:
All of my interviewees were residents of the same
middle-sized southeastern city or its immediate environs; all
were native-born Americans who spoke English as a first language; and all were married during the period of their interviews, all
in first marriages. Beyond these constancies of cultural and
marital experience , they were selected to
maximize diversity with regard to such obvious differences as their occupations
and educational backgrounds, religious affiliations and ethnic and racial
identities, their neighborhoods and social
networks, and the duration of their marriages. (p. 399)
Although not generally representative of either the regional
population or of the population of the United States, Quinn claims
that her sample of informants represents the regions' population in terms of the high degree of
recent in-migration to the area from regions outside the South. Her
sample is an attempt to capture the range of diversity found in the region. In my
view, the consistency of her findings in
this diverse sample of informants makes her case stronger (see Johnson 1990). That is, finding commonality in the face of diversity
provides stronger evidence of a shared
cultural model (Johnson and Griffith 1996). In principle, this is similar to Cook
and Campbell 's (1979) model of deliberate
sampling for heterogeneity as one of several means for warding off threats to external validity.
Based
on an in-depth analysis of informants' discourse about marriage, Quinn produced
a cultural model incorporating a number of causal links in informants'
reasoning as to a "lasting marriage" (see Figure 5). Although the model
appeared to be widely shared among
informants from Quinn's sample and data collected from other studies on marriage, research still has to be designed to test this model across settings and researchers .
Issues
of validity in this case are not as overriding as they would be in a purely explanatory study
. Quinn was careful and diligent in her selection of
informants, and her diligence certainly contributes to the potential validity
of her model. However, further research
in the explanatory mode is now warranted.
Participant Observation and the Search for Validity
As seen in Figure 2,
exploratory and descriptive research are
often essential components of an overall explanatory
research design. In a
series of papers, Koester (1996) and his colleagues (Koester et al. 1996) offer excellent examples of the role of participant
observation in more clearly defining the set of HIV
risk behaviors
surrounding injection drug use. In most earlier research on injection
drug users (IDUs
) and HIV risk
, the primary risk factor was viewed in terms
of direct needle sharing . Thus, most large
epidemiological studies of IDUs focused mainly on direct sharing behaviors
in attempts to understand seroconversion rates and other risk factors.
Based
on participant observation among IDUs ,
Koester (1996) identified nine other behaviors
that were outside the realm of the direct sharing
of a single syringe by two or more IDUs . Termed "indirect sharing
," these nine behaviors can promote the
transmission of HIV among IDUs who, although not sharing needles
directly, often share water for mixing of drugs
or for rinsing syringes, share drug -mixing
containers (cookers and spoons), share
cottons for filtering, and share the actual drug
solution itself. These findings
are undeniably important for larger epidemiological work that examines elements
of IDUs ' behaviors
and such things as producing valid models of seroconversion.
In
a subsequent study , Koester et al. (1996) used these additional distinctions in sharing to look at the prevalence of injection -related HIV
risk behaviors
among several subpopulations of injection drug users (see Figure
6). A major component of the study
was the comparison of IDUs who engaged in both direct sharing and indirect sharing with IDUs
who engaged in indirect sharing only and
those who several subpopulations of injection
drug users (see Figure
6). A major component of the study
was the comparison of IDUs who engaged in both direct sharing and indirect sharing with IDUs
who engaged in indirect sharing only and
those who neither shared directly nor
indirectly. Statistical tests of group differences provided a greater
understanding of the risk factors associated
with the different types of behavior . This is a good example of the application of exploratory research in the production of better measures of potentially important explanatory variables
.
Case-Control
Study Design: Susto, A Folk Illness
Here we look at an example of a study design used to investigate the extent to
which disease is molded by culture. Rubel et al. (1985) report on a study of
a folk illness known as susto, found in many
cultural groups throughout North and South America. Folk
beliefs surrounding susto attribute loss of a critical substance or force due
to a frightening experience . The authors were interested
in three primary hypotheses relating to role
performance and the presence of the illness, psychiatric impairment and susto,
and relationship between organic disease and susto. The
ultimate aim of the study was to show the
relationship between various social forces
and susto susceptibility.
The
design involved three communities that differed in history, language, and
culture but had similarities in social ,
demographic, and economic factors. Rubel et al. carefully selected communities that were as similar as possible
in terms of forms of government and genderspecific role expectations. The proposed design could have been conducted in a single
community, but the authors felt that the
generalizability of the results would be enhanced with a multiple case-control study design.
One
subsample was of individuals who complained of susto during the fieldwork or who had admitted their condition
to relatives or curers. Selection depended on the condition
by men in the community and may also have affected the reporting of the
condition by women. Another problem
involved the existence of more social
stratification in one community than expected, leading to a lower incidence of.
reported cases of the illness (higher-income people
recognized the condition but felt that belief in it was more superstitious than
real). However, Rubel et al. felt
comfortable with the comparability among the susto subsamples from the
communities. These groups will be referred to as the
asustados groups.
The
researchers were careful to make the control
group as comparable to the asustados groups as possible. Because
the asustados were "sick," control group members must also be sick. Thus, sick people were compared to sick people and control group
members were selected from the pool of patients at the project clinics in each
of the communities. Patient records provided the information
on which to make the final selection. In addition to the
control group being sick, males were matched with males and females with
females and asustados and controls were matched in terms of age. Matched pairs were made within communities only. This design allowed for a variety of comparisons , including comparisons by controls and asustados, by
gender, and by matched pairs both within and between cultural groups (see Figure 7).
Symptomology
and health problems were operationalized
using a panel of physicians. Psychiatric impairment was
operationalized using the 22-item Screening Score for Psychiatric Impairment. Based on earlier ethnographic
research , social
stress, an important component for understanding an individual's inability to
perform social roles, was operationalized
using the Social Stress Gauge developed by
one of the researchers .
Using
standard methods of statistical inference,
Rubel et al. found that
there was, in fact, an association between susto and an individual's perception
of the adequacy of his or her performance of critical social roles. Although there
was no association between susto and psychiatric impairment, there was a
relationship between susto and the suffering of more organic disease signs.
This
study is important because of the authors ' candor about the potential threats to validity they encountered in
conducting the research . The
stigma of susto among males and the greater social
stratification encountered in one of the communities are possible threats to the validity of their conclusions . But the researchers ' awareness of the problems , combined with the strength of their
multiple case-control study design,
increases our confidence in their conclusions
. This is an excellent example of a study design that incorporates within-study replication or multiple tests of a theory
. Multiple tests are
always much more convincing than a single test
(Stinchcombe 1987).
Multimethod
Ethnography and the Comparison of Models
Many peasant societies in Central America are experiencing dramatic economic and cultural
change. One consequence of these changes is increasing
economic differentiation. Parallel to these economic
changes, there has been a shift in religious preference over the last 70 years.
For some researchers , the
shift from Catholicism to Protestantism helps account for economic change, as
Protestantism is more compatible with capitalist ideology and the accumulation
of wealth. Goldin (1996) wanted to understand the
relationships among religious affiliation, economic ideology, occupation, and
economic status in a Guatemalan township (Almolonga). Her study design incorporates quantitative and
qualitative methods in the overall ethnographic
enterprise.
Based
on extensive participant observation, Goldin constructed four plausible models
that might account for what she observed while in the field. Using
her experience as a participant observer,
Goldin developed a survey which she applied to a random sample of 10% of the
heads of households in the township (n = 57). She made an
earnest attempt to control for as many biases as possible and, using the data
collected during the survey, conducted statistical
tests of the four competing models. This provided for an evaluation of the explanatory power of each. Her
selection of variables allowed a comparison of different levels (for example,
Catholic versus Protestant) across the four variables
. Using path analytic modeling, she applied different statistical controls in each of the competing
models. As she describes the process leading to the
selection of the best model:
The results of my study ,
of course, must be interpreted within the constraints of the data collection
methods. First, qualitative approaches were used to suggest
different mechanisms and relationships that might be operating within
Almolonga. Then, a survey approach was used to evaluate the
viability of these mechanisms in terms of characterizing general trends within
Almolonga. My conclusions
must be interpreted in terms of these general trends. I
don't doubt that there are exceptions to them. Indeed, I
interacted with several individuals who had life histories that were
inconsistent with my general characterization and who were the basis for
suggesting the competing models discussed
above. However, when a large representative sample of the
township was aggressively pursued, the different data sets tended to support
model C as the one that characterizes the general tendencies within the
township. (p. 72)
This study is an example
of multimethod ethnography in which there
was a combination of exploratory and explanatory
approaches-that is, qualitative data and tests
of models with data collected using a cross-sectional design. The
combination helped Goldin in the specification of appropriate variables , in the development of a sound
survey instrument, and in the specification and assessment of the four
competing models. The study
shows how the use of multiple methods fosters triangulation that contributes to
the production of valid conclusions (see Figure 8). Her research design illustrates the danger in
relying on a single method without attention to sampling. Had
Goldin relied exclusively on, say, the life histories of a nonprobabilistic
sample of informants without specified selection criteria (Johnson 1990), she might have arrived at a very
different, and possibly erroneous, conclusion
.
Summary
This review of research
design and strategies in cultural anthropology only scratches the surface of the research designs, hybrid designs, and
combinations of designs possible within an ethnographic
context. The newer forays into experimental
and other ethnographic forms of presentation
are more reflexive in character and more concerned with believable and moving
representations rather than the production of valid accounts or conclusions .
With
advances in computer technology, qualitative data analysis can now be a
powerful mode to test theories . Similarly,
advances in computer-intensive methods for testing
hypotheses have the potential to expand the
range of designs possible, particularly in the imperfect world of fieldwork (Johnson
and Murray 1997). The strength of the ethnographic approach is its ability to
incorporate a wide range of methods, strategies
, and designs within a single enterprise, all combining in ways to improve the
chances for credible results.
As
anthropologists , we should take full
advantage of both our current understanding of research
design and these new developments to produce a "powerful mode of
argumentation. " It is mostly through attention to
these concerns that anthropology and anthropologists will have the opportunity to,
as Agar says, "move the world. "
REFERENCES
Agar, M. 1980. The Professional Stranger:
An Informal Introduction to Ethnography . New York: Academic Press.
Agar,
M. 1996. AAA Newsletter. January.
Babble,
E. 1990. Survey Research
Methods, 2d ed. Belmont, CA: Wadsworth.
Behar,
R. 1993. Translated Woman. Boston: Beacon
Press.
Bernard,
H. R. 1994. Research
Methods in Anthropology : Qualitative and
Quantitative Approaches, 2d ed. Walnut Creek, CA: Alta Mira
Press.
Boas , F. 1920. The Methods
of Ethnology. American Anthropologist
22(4):311-321.
Boster,
J. S., and J. C. Johnson . 1989.
Form or Function: A Comparison
of Expert and Novice Judgments of Similarity Among Fish . American Anthropologist 91(4): 866-889.
Brim , J. A., and D. H. Spain . 1974. Research Design in Anthropology
: Paradigms and Pragmatics in the Testing of
Hypotheses . New York:
Holt, Rinehart and Winston.
Campbell , D. T. 1975. "Degrees
of Freedom" in the Case Study . Comparative Political Studies
8(2):178-213.
Campbell , D. T. 1988. Qualitative
Knowing in Action Research . In
Methodology and Epistomology for Social Science
: Selected Papers E. S. Overman, ed. Pp. 360-376. Chicago: University of Chicago Press.
Cook , T. D. 1994. Criteria
of Social Scientific
Knowledge: Interpretation, Prediction, Praxis. Lanham, MD:
Rowman and Littlefield.
Cook , T. D., and D. T. Campbell . 1979. Quasi -Experimentation : Design and Analysis for Field
Settings. Chicago: Rand McNally.
Denzin,
N. K., and Y. S. Lincoln. 1994. Entering
the Field of Qualitative Research . In Handbook of Qualitative Research
. N. K. Denzin and Y. S. Lincoln, eds. Pp.
1-19. Thousand Oaks, CA: Sage Publications.
Ellen,
R. F. 1984. Introduction. In Ethnographic Research
: A Guide to General Conduct. R F. Ellen, ed. Pp. 1-12. London: Academic Press.
Ember,
M. 1985. Evidence and Science
in Ethnography : Reflections on the Mead -Freeman
Controversy. American Anthropologist
87(4):906-910.
Faia,
M. A. 1993. What's Wrong with the Social Sciences? The Perils
of the Postmodern . Lanham,
MD: University Press of America.
Freeman , D. 1983. Margaret Mead
and Samoa : The Making and Unmaking of an Anthropological Myth. Cambridge:
Harvard University Press.
Glaser,
B. G., and A. L. Strauss. 1967. The
Discovery of Grounded Theory : Strategies for Qualitative Research . Chicago: Aldine.
Goldin,
L. R. 1996. Models of Economic Differentiation and Cultural
Change. Journal of Quantitative Anthropology I-2(6):49-74.
Harris,
M., J. G. Consorte, J. Lang, and B. Byrne. 1993. Who Are the Whites: Imposed Census Categories and the Racial
Demography of Brazil. Social
Forces 72(2): 451 X62.
Hubert,
L. J. 1987. Assignment Methods in Combinational Data
Analysis. New York: Marcel Dekker.
Hurlbert,
S. H. 1984. Pseudoreplication and Design of Ecological
Field Experiments . Ecological
Monographs 54(2):187-211.
Johnson , J. C. 1990. Selecting
Ethnographic Informants. Qualitative
Research Methods Series, Vol. 22. Thousand Oaks, CA: Sage Publications.
Johnson J. C. 1994. Anthropological Contributions to the Study of Social
Networks: A Review. In Advances in Social Network Analysis. S.
Wasserman and J. Galaskiewicz, eds. Pp. 113151. Thousand Oaks, CA: Sage Publications.
Johnson , J. C., and D. C. Griffith. 1996. Pollution, Food Safety, and the
Distribution of Knowledge. Human Ecology 24(1):87-110.
Johnson , J. C., M. Ironsmith, A. L. Whitcher,
G. M. Poteat, and C. W. Snow. 1997. The
Development of Social
Networks in Preschool Children. Early Education and
Development 8(4):389-406.
Johnson , J. C., and J. D. Murray. 1997. Evaluating FAD Effectiveness in
Development Projects: Theory and Praxis. In Fish Aggregation Devices
in Developing Fisheries: Potential and Pitfalls. R. Pollnac
and J. Poggie, eds. Pp. 143-158. Kingston:
ICMRD.
Johnson , J. C., and R. Pollnac, eds. 1989. Managing Marine Conflicts. Special issue of Ocean and Shoreline Management 12(3).
Kempton,
W., J. S. Boster, and J. A. Hartley. 1996. Environmental
Values in American Culture. Cambridge: M.I.T. Press.
Kincheloe,
J. L., and P. L. McLaren. 1994. Rethinking
Critical Theory and Qualitative Research . In Handbook of
Qualitative Research . N.
K. Denzin and Y. S. Lincoln, eds. Pp. 138-158. Thousand Oaks, CA: Sage Publications.
Kirk,
J., and M. L. Miller. 1986. Reliability and Validity in Quantitative Research . Qualitative Research Methods Series, Vol. 1.
Thousand Oaks, CA: Sage Publications.
Kleinbaum,
D. G., L. L. Kupper, and H. Morgenstern. 1982. Epidemiologic Research :
Principles and Quantitative Methods. Belmont, CA: Lifetime
Learning Publications.
Koester,
S. 1996. The Process of Drug
Injection : Applying Ethnography to the Study of HIV Risk Among IDU
's. In AIDS, Drugs and
Prevention: Perspectives on Individual and Community Action. T.
Rhodes and R. Hartnoll, eds. Pp. 133-148. London:
Routledge Press.
Koester,
S., R. E. Booth, and Y. Zhang. 1996. The
Prevalence of Additional InjectionRelation HIV
Risk Behaviors
Among Injection Drug Users. Journal of
Acquired Immune Deficiency Syndromes and Human Retrovirology 12:202-207.
Kruskal,
J. B., and M. Wish. 1978. Multidimensional
Scaling. Beverly Hills, CA: Sage Publications.
Kuznar,
L. A. 1997. Reclaiming a Scientific
Anthropology . Walnut
Creek, CA: Alta Mira Press.
LeVine , R. A. 1973. Research Design in Anthropological
Field Work. In A Handbook of Methods in Cultural Anthropology . R. Naroll
and R. Cohen, eds. Pp. 183-195. New
York: Columbia University Press.
Lowie,
R. H. 1937. The History of Ethnological Theory . New York:
Rinehart.
Marcus,
G. 1983. One Man's Head. New York Times
Book Review, March 27:3, 22-23.
Melbin,
M. 1987. Night as Frontier: Colonizing the World After
Dark. New York: The Free Press.
Miles,
M. B., and A. M. Huberman. 1994. Qualitative
Data Analysis, 2d ed. Thousand Oaks, CA: Sage Publications.
Moran,
E. F., ed. 1995. The Comparative
Analysis of Human Societies: Toward Common Standards for Data Collection and
Reporting. Boulder: Lynne Rienner.
Naroll,
R., and R. Cohen, eds. 1973. A Handbook
of Method in Cultural Anthropology . New York: Columbia University Press.
Noreen,
E. W. 1989. Computer-Intensive Methods for Testing Hypotheses
. New York: John Wiley.
Orans , M. 1996. Not Even
Wrong: Margaret Mead , Derek Freeman
, and the Samoans. Novato, CA: Chandler and Sharp.
Panourgia,
N. 1995. Fragments of Death, Fables of Identity. Madison: University of Wisconsin Press.
Pelto,
P. J., and G. H. Pelto. 1978. Anthropological Research
: The Structure of Inquiry, 2d ed. Cambridge: Cambridge
University Press.
Planner,
S. In press. Scientific Anthropology at the National Science Foundation
. In Anthropology
Between Science and the Humanities. C. Furlow, ed. Walnut Creek, CA: Alta Mira
Press.
Porter,
T. M. 1995. Trust in Numbers. Princeton:
Princeton University Press.
Quinn,
N. 1996. Culture Contradictions: The Case of America's
Reasoning about Marriage. Ethos 24(3):391-425.
Rabinow,
P. 1986. Representations are Social
Facts: Modernity and Post-Modernity in Anthropology
. In Writing Culture:
The Poetics and Politics of Ethnogrophy. J. Clifford and G.
E. Marcus, eds. Pp. 234-262. Berkeley:
University of California Press.
Ramos,
A. R. 1995. Sanuma Memories. Madison:
University of Wisconsin Press.
Robson,
C. 1993. Real World Research
: A Resource for Social Scientists and PractitionerResearchers. Oxford: Blackwell Publishers.
Rubel,
A. J., C. W. O'Neil, and R. Collado-Ardon. 1985. Susto, A Folk Illness. Berkeley:
University of California Press.
Schneider,
D. 1996. Alarming Nets. Scientific American (September):40-12.
Schwandt,
T. A. 1994. Constructivist, Interpretivist Approaches to
Human Inquiry. In Handbook of Qualitative Research . N. K. Denzin and
Y. S. Lincoln, eds. Pp. 118-138. Thousand
Oaks, CA: Sage Publications.
Sechrest,
L. 1973. Experiments in
the Field. In A Handbook of Methods in Cultural Anthropology . R. Naroll
and R. Cohen, eds. Pp. 196-209. New
York: Columbia University Press.
Seidman,
S. 1994. The Postmodern
Turn: New Perspectives on Social Theory . Cambridge:
Cambridge University Press.
Shankman,
P. 1996. The History of Samoan Sexual Conduct and the Mead -Freeman
Controversy. American Anthropologist
98(3):555-567.
Spindler,
G., and W. Goldschmidt. 1973. An
Example of Research Design: Experimental Design in the Study of Culture Change. In
A Handbook of Method in Cultural Anthropology
. R. Naroll and R. Cohen, eds. Pp.
210-219. New York: Columbia University Press.
Stinchcombe,
A. L. 1987. Constructing Social
Theories . Chicago:
University of Chicago Press.
Taussig,
M. 1987. Shamanism, Colonialism, and the Wild Man: A Study in Terra and Healing. Chicago:
University of Chicago Press.
Tyler,
S. A. 1991. A Post-modern In-stance. In
Constructing Knowledge: Authority and
Critique in Social Science . L. Nencel and P.
Pels, eds. Pp. 78-95. London: Sage
Publications.
Urry,
J. 1984. A History of Field Methods. In
Ethnographic Research
: A Guide to General Conduct. R. F. Ellen ed. Pp. 35-62. London: Academic Press.
Van
Maanen, J. 1988. Tales of the Field: On Writing Ethnography
. Chicago: University of Chicago Press.
Weller,
S. C., and A. K. Romney. 1988. Systematic Data Collection. Qualitative
Research Methods Series, Vol. 10. Thousand Oaks, CA: Sage Publications.
Werner
O., and G. M. Schoepfle. 1987. Systematic Fieldwork
, Vol. 2. Thousand Oaks, CA: Sage
Publications.
Whyte, W. F. 1984. Learning from the Field: A Guide from Experience . Newbury Park, CA: Sage Publications.
Williams, T. R. 1967. Field Methods in the Study of Culture. In the series Studies in Anthropological Method, George Spindler and Louise Spindler, eds. New York: Holt, Rinehart and Winston.
Young, J. C., and L. Y. Garro. 1982. Variation in the Choice of Treatment in Two Mexican Communities. Social Science and Medicine 16:1453-1465.
Zabusky, S. E. 1995. Launching Europe: An Ethnography of European Cooperation in Space Science . Princeton: Princeton University Press.