Between Texts and Contexts:
Advances in Theories of Citation ?
(A Rejoinder)
Loet Leydesdorff and Paul Wouters
Science and Technology Dynamics
Nieuwe Achtergracht 166
1018 WV Amsterdam
The Netherlands
<l.leydesdorff@mail.uva.nl>
<pwouters@xs4all.nl>
Abstract
Scientific literature is expected to contain a body of knowledge that can be indexed and retrieved using references and citations. References are subtexts which refer to a supertext, that is, the body of scientific literature. The Science Citation Index has provided an electronic representation of science at the supertextual level by aggregating the subtextual citations. As the supertext, however, becomes independently available in virtual reality (as a "hypertext"), subtext and supertext become increasingly different contexts. The dynamics of hyperlinks are expected to feedback on the system of indexing, referencing, and retrieval at the level of research practices. References can be considered as part of the retention mechanism of this evolving system of scientific communication, and citations are a codified form of referencing.
1. Introduction
Hyperlinks relate documents on the Internet in terms of a hypertext, while references are embedded as subtexts in (primarily printed) texts. The Greek word hyper, however, is identical to the Latin word super: the hyperlinks refer to a supertext. Subtext and supertext can be considered as different contexts of the embedded texts. The relationships between these contexts have been transformed by the emergence of electronic communication and the Internet. More specifically, the real-time availability of virtual hypertexts changes the system of referencing, indexing, and retrieval by adding degrees of freedom to the evolving system of scientific communication.
Scientometric indicators can be distinguished by the way in which they refer to different levels of textual organization. Words and co-words, for example, are indicators of the texts themselves. From an evolutionary perspective on scientific communication, the texts ─that is, words and co-occurrences of words in sentences─ provide the variation (Callon et al. 1986; cf. Leydesdorff 1995). By referencing, a subset of these texts is selected. References and citations can therefore be used as an indicator of these selection processes. Citations, indeed, are more than ten times as precise as words in the recall (Leydesdorff 1989 and 1997a; cf. Garfield 1996). Words and co-words are volatile indicators, while citations have been codified.
What guides an author when selecting texts to cite? In the introduction to the theme issue on theories of citation, one of us (Leydesdorff 1998) suggested that the functions of citing authors have changed historically. In the 18th century, the main function of a reference was discursive in relation to other scholars. During the 19th century, an argumentative function in relating to other texts gradually emerged in scientific communication because of the institutionalization of the sciences at the (German) university in terms of Chairs (Stichweh 1984). At the turn of the century, the modern reference originated as a selection of the two networks upon each other, thus allowing for more specificity (Bazerman 1988).
In this rejoinder to the discussion, the claim is substantiated empirically by analyzing the emergence of the modern reference in the case of the Journal of the American Chemical Society (JACS). Furthermore, we shall argue that the differentiation of contexts in scientific practices as hypothesized by Gibbons et al. (1994) in terms of so-called Mode-2 research, enables us to specify theories of citation in terms of the selective functions of references and citations for the evolving systems of scientific communication. This perspective on references as a means of communication allows for the specification of the need for indicators other than the reference-based citation (cf. Van Raan 1998).
Differentiation of contexts
The information-theoretical study of scientific communications as practiced in scientometrics is perturbed by changes in the practices of referencing, indexing, and retrieval of the scientists under study. On the Internet, the hypertext is no longer referenced only in subtexts; often the "cited" text is itself available by following the links. The couplings between texts, subtexts, and supertexts are thus changed structurally by the current revolution in information and communication technologies.
The Science Citation Index can be considered as a hypertext avant la lettre (Wouters 1998). The Index (and its derivatives like the Social Science Citation Index) has primarily been an electronic reflection of the practice of printed journal publications. Thus, it has constituted a meta-level to printed communication by using a specific medium, and then allowed for the assessment of contributions at this lower level.
Citation indexing constitutes a specific representation of the literature, enabling us to search both hyper- and subtextually (that is, both "cited" and "citing"). As a hypertext, however, this reflection has been essentially static: although regularly updated, the Citation Index provides a state of the art for each year. Its own hypertextual dynamics have not been part of its construction (except for the yearly turnover of journals; cf. Leydesdorff & Cozzens 1993). The Citation Index has not had to develop with the Internet since the citation is codified in comparison to the hyperlink, and this codification seems to be robust with reference to the change of storage media (Rousseau 1997 and 1998).
Scientists are increasingly able to use electronic communication for timely dissemination of high-quality results. This means, among other things, that scientific publishing has returned to its original function of communicating scientific results, whereas publication in printed journals is gradually assuming an archival role. This transformation was preceded by the widespread distribution of preprints from the 1970s onwards, when photocopying became a cheap technology. With the use of the Internet for (pre‑)publication and dissemination, the communication system has gained additional degrees of freedom. The hypertext is made available in virtual reality and the window available through citations is no longer unique. Hyperlinks and search engines now provide additional points of entry into the hypertext.
All referential and indexing systems provide only a window on what is being indicated: the scientific contents of the texts remain an expectation and therefore uncertain. The addition of new degrees of freedom to the search machinery implies a reorganization of the system of indexing, reference, and retrieval. This structural reorganization realigns the elements of communication within the system. One would expect, for example, that the rules of publication, of attracting attention with a publication, and of gate-keeping, may well be changing. These empirical questions about subtexts as elements related to hypertexts are pertinent to existing theories of citation.
Citations as recursive selections
As long as the hypertext itself was not yet dynamically available, the subtext and the supertext could not be distinguished empirically in terms of operationalizations. At the level of the text, the subtext was a single window on an assumed hypertext. The major division among scientometric indicators has hitherto been between indicators as formalized elements of texts (words and co-words), indicators as formalized elements of subtexts (citations and authors' names), and indicators as formalized elements of paratext (like subject headings and institutional addresses).
Indexing, referencing, and retrieval imply selections. Selections can be assessed in terms of their quality in relation to functions at the next-order level of the system. The next-order level is the implied super-text of the scientific literature. By strengthening the coupling between text and supertext, references and citations, among other elements, can be used to increase the certainty of the outcome at the next level, in other words to improve the expected information content of these literatures. Because of the recursivity of these selections, referencing can acquire an epistemological role during a historical process of intensive scientific communication (cf. Vinkler 1998).
As the supertext becomes available as a virtual reality, the relationships among its different functions at various levels is expected to change because of the possibility for new resonances in the systemic couplings: a dual-layered system of text and supertext comes into being and lock-ins can be expected (Arthur 1988 and 1989). As the two levels select upon each other in a coevolution, Maturana's model of autopoiesis becomes relevant (Fujigaki 1998a). The self-organization of scientific communications can then be studied in terms of empirical questions (Leydesdorff 1995).
In her contribution to this debate, Scharnhorst (1998) introduced another metaphor for this dual-layeredness: the fitness landscape operates on the occupational landscape as a selector, but the occupation which occurs changes the landscapes. Makino (1998) adds that without this reflection the function of the citation cannot be understood: citation is not grounded in a supposedly inherent quality, but in the functions that are carried by the references. These functions are deeply historical because references belong to the memory of the system. References can be further codified by inverting them into citations (Wouters 1998; Garfield 1998). Obliteration (Garfield 1975), whether or not by incorporation, is among the indications of this evolutionary role.
The evolutionary perspective brings us back to citation and citation analysis as historical artifacts particularly useful for indexing and retrieval. Eugene Garfield invented the Science Citation Index by turning the textually available subtexts into a relatively independent supertext. This was made possible by the availability of electronic technology. More specifically, the Science Citation Index was time-stamped by the era of the mainframe computer. Yet, it can be considered as a systems innovation (Sahal 1985): the codification of the reference in the text, and its extraction into the citation, merged with newly available technology and thereby added a reflexive layer to the scientific communication system.
"Citationology"
Is citation, in its codified form, an invention of the late 19th and early 20th century? In a private communication, Garfield responded to a draft of the introduction of the theme issue in which this claim was made (Leydesdorff 1998), by providing examples of citations from before the turn of this century. For example, Weinberg (1997) noted that in Hebrew texts of the 16th century, the subtext of citation was inserted perpendicularly between the columns and within the margins. This is somewhat comparable with the Japanese tradition of using the Latin alphabet in providing international citations (Figure 1).
 |
Figure 1
Citations and references in a Japanese text
References can also be typeset in a smaller pitch. The typography denotes the subtextual domain of referencing. The subtexts embed the message of the main text historically. Rousseau (1998) in his comments noted that citation can therefore be considered as a kind of friction. He would prefer a theory of citation that focuses on substance. Although one can ideal-typically reduce the complexity of the referencing mechanism by focusing on its possible cognitive meaning, the discussion of the reference as a rhetorical instrument of persuasion (Gilbert 1977) and the distortion of citing practices noted by McRoberts & McRoberts (1987) perturbs this type of citation theorizing.
The cognitive domain proliferates uncertainty at an increasing speed. Codification is one of the means available to reduce the uncertainty. Although the coupling between two levels generates a friction, this process cannot be neglected since it indicates selection. The mutual information between the cognitive and the social dimensions propels a coevolution. The historical trajectory emerges as a socio-cognitive interaction in a process of mutual shaping (Bloor 1976).
In our opinion, the modern reference was invented in the historical context of the second scientific-technical revolution of 1870-1910. Industrial R&D, patent legislation, and the further development of the university system demanded that science itself become an institutionalized translation system at the interface of previously existing social and cognitive practices, thereby feeding into, and changing, these practices (Leydesdorff 1997b). Several authors challenged us to support this claim with empirical evidence. Garfield, for example, suggested that Bazerman's (1988) results were biased by using Physical Review which started only in 1893 (cf. Zuckerman & Merton 1971). Did not botany and chemistry adopt this practice earlier, for example, right from the founding of the Journal of the American Chemical Society (JACS) in 1879?
In order to clarify this question, we checked the February issues of JACS for every five years in the period under study (1890-1910). The February issues were used since the January issues contained Presidential addresses which sometimes deviated from the standard codification in research papers. We followed Bazerman's (1988) distinction among (a) references without date, (b) references with modern format, but to documents older than six years, and (c) references with modern format and within a citation window of six years. Given the slower pace of communication in those days, a citation window of six years seemed a reasonable choice.
The results are shown in Figure 2. References were invented in the first half of the 1890s, but first in the form of references without date. This form is replaced with the modern reference around 1900. After the invention of the modern form the older form of referencing rapidly disappeared and the number of modern references began to grow exponentially. In the February issue of JACS in 1910, the number of modern citations had exploded to sometimes more than ten on a single page.[1]
|
Figure 2 The origin of the modern citation in the Journal of the American Chemical Society (JACS) |
In summary, we have argued that references are a result of changing research practices. The invention of the modern citation has led to changes in the practice of information retrieval in scientific practices. Citation indexing has been a consequence of the new possibilities opened up by the existence of citation data, created by the codification of citing practices in science. Citation indexing has made possible a further transformation of information retrieval. It has, moreover, enabled the creation of a new set of formalized indicators which are being used in the evaluation of scientific results. These formalized representations (which are not always mutually compatible or intuitively easy to interpret) feed into research practice and thereby complicate the problem of scientific evaluation (Rip 1997; Wouters 1997; Small 1998).
Citations as translations
When the reference practices of scientists change because of the availability of new technologies, the level of indexing has either to follow or to risk becoming irrelevant. For example, the citationologist (Garfield 1998) has to address the issue of the difference between a reference contained in a citation and one available in a hyperlink (Rousseau 1997).
References are not unambiguous, yet they are considerably more highly codified than words. They refer to stabilized contexts: the body of knowledge as codified in the literature. The reference, however, provides only a window on an otherwise uncertain and evolving environment. This uncertainty about the environment is reflected in the distributed character of lists of references and citations.
Citation indexing enables us to use these distributions systematically for retrieval. The citation analyst steps aside from the research process and is therefore able to develop a picture with hindsight and from an external position. This project is formal since citations abstract from the content of the reference. For the citation analyst, the expected content is less relevant than the observed position. In information theory, however, the question of the content returns as a problem of recall and precision, in sociology as a question of validation, and in the study of scientific communication as citation context analysis.
The citation analyst thus obtains the perspective of a super-observer, while the embedded participant (who may also be an observer at a next moment in time) looks at references as embodiments of specific actor motivations and reasons. Using Maturana's theory of autopoiesis (cf. Fujigaki 1998a) one is able to distinguish between the participant/observer as a structural part of the network and the super-observer. In Maturana's theory, however, the super-observer is a biologist who can clearly be delineated from the participant/observers in the biological phenomena under study. In the social sciences, we lack such a clear distinction between an observer and a super-observer.
The perspective of the super-observer has to be legitimated, on the contrary, in terms of the recognizability and validity of the observations for the participant/observers' discourses. Thus, the loop is to be closed in terms of what Giddens (1984) has called a "double hermeneutics." In other words, the super-text can be assessed in terms of its performative functions for the discussion at the level of texts (Callon 1998; cf. Fujigaki 1998b). The texts need the sub-texts as enrichments, but the supertext of, for example, citation analysis is codified differently and therefore only partially useful.
The citation analysts are themselves caught within this loop as are their subjects of study. One has to gain knowledge from relating one's text to a supertext, while the supertext remains virtual and inherently uncertain. Cronin (1998) in his contribution identified this as Giddens' problem of the duality of structure: the supertext is composed as the aggregate of the contributing texts, but the interaction among these texts conditions the further operation. In other words, advances can only be made by interacting and on the basis of expectations. Thus, the cognitive process is enhanced by using citations as selective operators.
The citations indicate the emerging structures and relationships in a formalized network of communications if one adopts the perspective of a citation analyst (Arunachalam 1998; Van Raan 1998). For practicing scientists, however, the reference is primarily substantive. Both contexts have their own range of uncertainties and probabilities. In this debate, the citation itself is repeatedly translated from one context to the other. These translations constitute an epistemological reinforcement since recognition recursively refines cognition within the observing systems (Maturana and Varela 1980).
The contexts of citation analysis
Scientometrics is not able to escape from the reflexive question about the performative value of its results, however pragmatically one may wish to proceed. The indicator is a representation of its indicated domain. The domain remains latent while being indicated, and one can only hypothesize its existence and properties. This is the crux of the citation debate: ADoes citation indicate quality, cognitive progress, or impact?@ This question is addressed by several authors in the topical issue (e.g., Kostoff 1998; Rousseau 1998; Small 1998; Cronin 1998).
We have argued that references are the result of a specific historical development in the sciences which occurred at the turn of the century. The invention diffused because the format was functional both to the research process and to codifying the communication about scientific results in the context of justification. Citation indexing itself is a creative turn occasioned by the advent of the mainframe computer in the 1950s and early 1960s. The personal computer made possible other indicators, like co-words, multi-dimensional scaling, and the mapping of science and technology. The Internet now gives rise to yet another turn in citation analysis.
For example, one can raise questions about the differences and similarities among references, citations, and hyperlinks. This topic is also discussed under the heading of Acyberscientometrics@. Rousseau (1997) has pleaded for a Asitation analysis@ based on the study of the patterns of hyperlinks at websites. A hyperlink, however, is of a different nature than a print-based reference. AHits@ can perhaps be compared with browsing, sometimes with reading, and only in a small number of cases with citation. The selection is mostly casual. Although many statistical properties of the distribution of sitations may be similar to the distribution of citations, differences can be expected to emerge when these supertextual representations have to be reintegrated into the textual level. Electronic and printed communication have different functions (cf. Kaufer and Carley 1993).
In summary, the theory of citation is continuously contingent on the development of scientific communication. The increasing complexity of the communication requires analysts to become more focused and more reflexive about the status of their arguments. The quality of the citation is itself an indicator of this reflexivity (Garfield 1996).
Citations and innovation studies
The citation has functioned as the specific historical codifier of scientific communications in a period when the sciences have evolved from a transcendental project into a multifacetted project that has become institutionally embedded in a myriad of contexts. In the meantime, the distributed control of a knowledge-economy has enforced a new communications regime of university-industry-government relations (Leydesdorff 1997b). Gibbons et al. (1994) have dubbed this situation "Mode 2"; Etzkowitz and Leydesdorff (1997) have proposed the metaphor and the model of a Triple Helix (cf. Leydesdorff and Etzkowitz 1998).
Innovation is taking place at interfaces, while references and citations codify historical achievements within domains. Thus, the citation analyst tends to take a historical perspective, while innovation studies challenge us to take an evolutionary one. Some of our colleagues have been sensitive to this turn, for example, in terms of relating citations to patent statistics (e.g., Narin and Olivastro, 1992). From this perspective, citations are considered as specific indicators of the domains of scientific communication as shown by attempts of Kostoff et al. (1997) to use citations to define the setting of innovation, following up on Swanson's (1990) use of citations to indicate relations which have not yet been established.
In general, one is able to distinguish between citations as indicators of codification within scientific knowledge and other indicators of the relationships between scientific knowledge and its applications. Le Pair (1988) has signalled that citations do not provide us with a fair representation of technological achievements because the knowledge is usually built into technological artifacts without leaving the formal trace of a citation (Els et al., 1989). Yet, this knowledge is also transferred at the hypertextual level. For example, one could begin searching Triple Helix configurations on the Internet by using hyperlinks between industrial (www.*.com), academic (www.*.edu), and governmental (www.*.gov) texts.
Texts from different institutional spheres can also be linked in terms of references and citations. However, the sociological expectation is that the codification of relations across institutional domains will be relatively weak (Granovetter 1973; cf. Blauwhof 1995). Social institutions can be considered as the retention mechanism of the social system. While citations are expected to cluster within the system of scientific communications as a subsystem, an emerging overlay can be expected to feedback on the specialist subsystems reinforcing and reorganizing its own and the lower-level epistatic relations.
Implications
The metaphor of geometrical mappings of multidimensional spaces is gradually being superseded by evolutionary metaphors and models. Snapshots are being replaced by movies and animations, variables by fluxes, and pictures by simulations (Egghe 1998). The evolutionary metaphor uses operators which develop algorithmically, for example, including recursion.
Citations and references are expected to remain among the operators under study: the references from the side of the citing texts and the citations from the perspective of the hypertexts. Thus, subtexts are indexed, retrieved, obliterated, and reengineered in the present as indicators of relevant pasts. Citations can thus be considered as contributing to scientific codification in the double hermeneutical loop, thereby providing an index of this codification for the next systemic operation (Price 1970).
As noted, citation theories themselves are also part of this double (hermeneutical) loop. By attributing meaning to indicators, every citation theory makes a specific selection at the level of the text from the level of the hypertext. Given the multitude of formalized representations (each providing a window on the hypertext), it is therefore theoretically plausible that a multitude of apparently incompatible citation theories should arise (Garfield 1998; cf. Leydesdorff 1997c). This is not a problem that must be resolved, but an opportunity to forge more complex relationships within the system of scientific communication.
The process of post-modern fragmentation seems to be accelerated by the rise of the Internet as a medium of scientific communication. The virtual hypertext can be downloaded selectively at the level of the text, giving rise to a new hypertext in a following operational cycle of the system. The fact that the resulting citation networks are all connected by weak links across the board (Small 1998), however, does not mean that science is inherently unified in the traditional sense. Rather, Asmall world@ effects based on recursive selections may be responsible for these empirical findings (Van Alstyne and Bynjolfsson 1996; Watts and Strogatz 1998).
In other words, science is no longer envisioned as a solid body of unified knowledge in a single (cognitive) dimension. On the contrary, science may be better represented as a network in a multi-dimensional space which develops not only within the boundaries of this space, but also by co-evolutionary processes creating dimensions to this space (Scharnhorst 1998). As the specialties develop, near-orthogonalities are expected and sometimes specifiable in terms of evolutionarily relevant dimensions (Leydesdorff and Van den Besselaar 1997). The consequent "incommensurabilities" from the various perspectives can be recombined if they can be appreciated from an evolutionary perspective. Although the transition towards a new basin of attraction remains unlikely (Bruckner et al. 1994), only the super-observer who is able to translate reflexively into the discourse of the participant-observers can close the loop and thereby contribute to radical innovation.
Radical innovation is dependent on both the operational perspectives of research practices and the theoretical perspectives that abstract from practices (Leydesdorff 1996). In this confrontation with qualitative theorizing, the formal perspective of scientometrics is able to celebrate its triumph over more qualitative, that is, embedded understandings of STS-perspectives.
*
The authors acknowledge support from TSER project PL97-1296, entitled "The Self-Organization of the European Information Society."
*
References
Arunachalam, Subbiah (1998). Citation Analysis: Do we need a theory? Scientometrics 43, 141-142.
Arthur, W. Brian (1988). Competing technologies, in: Giovanni Dosi, Chris Freeman, Richard Nelson, Gerald Silverberg, and Luc Soete (Eds.), Technical Change and Economic Theory, Pinter, London, pp. 590-607.
Arthur, W. Brian (1989). Competing Technologies, Increasing Returns, and Lock-In by Historical Events, Economic Journal 99, 116-131.
Bazerman, Charles (1988). Shaping Written Knowledge: The Genre and Activity of the Experimental Article in Science. University of Wisconsin Press, Madison, WI.
Blauwhof, Gertrud (1995). The non-linear dynamics of technological developments: an exploration of telecommunications technology. Ph.D. Thesis, University of Amsterdam.
Bloor, David (1976). Knowledge and Social Imagery. Routledge & Kegan Paul, London.
Bruckner, Eberhard, Werner Ebeling, Miguel A. Jiménez Montaño, and Andres Scharnhorst (1994). Hyperselection and Innovation Described by a Stochastic Model of Technological Evolution. In: Loet Leydesdorff and Peter Van den Besselaar (Eds.), Evolutionary Economies and Chaos Theory: New directions in technology studies. Pinter, London, pp. 79-90.
Callon, Michel, John Law, and Arie Rip (Eds.) (1986). Mapping the Dynamics of Science and Technology. Macmillan, London.
Callon, Michel (1988). Is There Any Future for Scientometrics? And If Yes, Which One?, International Conference on STS: Book of Abstracts. Tokyo, p. 26.
Cronin, Blaise (1998). Metatheorizing Citation, Scientometrics 43, 45-54.
Egghe, Leo (1998). Comments on the paper of Leydesdorff "Theories of Citation", Scientometrics 43, 57-62.
Els, W.P. van, C.N.M. Jansz, and C. le Pair (1989). The citation gap between printed and instrumental output of technological research: the case of the electron microscope, Scientometrics 17, 415‑425.
Etzkowitz, Henry and Loet Leydesdorff (Eds.) (1997). Universities in the Global Economy: A Triple Helix of University-Industry-Government Relations. Cassell Academic, London.
Fujigaki, Yuko (1998). The Citation System: Citation networks as repeatedly focusing on difference, continuous reevaluation, an a persistent knowledge accumulation, Scientometrics 43, 77-85.
Fujigaki, Yuko (1998). A Future Perspective on STS and Scientometrics, EASST Review 17, No. 2, 16-19.
Garfield, Eugene (1975). The Obliteration Phenomenon, Current Contents, Nr. 51/52, 5-7.
Garfield, Eugene (1996). When to Cite, Library Quarterly 66, Nr. 4, 449-58.
Garfield, Eugene (1998). Random Thoughts on Citationology ─ Its Theory and Practice, Scientometrics 43, 69-76.
Gibbons, Michael, Camille Limoges, Helga Nowotny, Simon Schwartzman, Peter Scott, and Martin Trow (1994). The new production of knowledge: the dynamics of science and research in contemporary societies. Sage, London.
Giddens, Anthony (1984). The Constitution of Society. Polity Press, Cambridge.
Granovetter, Marc S. (1973). The Strength of Weak Ties, American Journal of Sociology 78, No. 6, 1360-1380.
Kaufer, David S. and Kathleen M. Carley (1993). Communication at a Distance: The Influence of Print on Sociocultural Organization and Change. Erlbaum, Hillsdale, NJ.
Kostoff, Ronald N., H. J. Eberhart, D. R. Toothman, and R. Pellenberg (1997). Database Tomography for Technical Intelligence: Comparative Roadmaps of the Research Impact Assessment Literature and the Journal of the American Chemical Society, Scientometrics 40, 103-138.
Kostoff, Ronald N. (1998). The Use and Misuse of Citation Analysis in Research Evaluation, Scientometrics 43, 27-43.
Leydesdorff, Loet (1989). Words and Co-Words as Indicators of Intellectual Organization, Research Policy 18, 209-223.
Leydesdorff, Loet (1995). The Challenge of Scientometrics: the development, measurement, and self-organization of scientific communications. DSWO, Leiden University Press, Leiden.
Leydesdorff, Loet (1996). The Possibility of a Mathematical Sociology of Scientific Communication, Journal for General Philosophy of Science 27, 243-65.
Leydesdorff, Loet (1997a). Why Words and Co-Words Cannot Map the Development of the Sciences, Journal of the American Society for Information Science 48, No. 5, 418-27.
Leydesdorff, Loet (1997b). The New Communications Regime of University-Industry-Government Relations. In: Etzkowitz and Leydesdorff (1997), pp. 106-117.
Leydesdorff, Loet (1997c). The Non-linear Dynamics of Sociological Reflections, International Sociology 12, 25-45.
Leydesdorff, Loet (1998). Theories of Citation? Scientometrics 43, 5-25.
Leydesdorff, Loet (forthcoming). Saientometorikus no chôsen: kagaku-gijyutsu-joho no jiko-soshiki-ka [The Challenge of Scientometrics: The development, measurement, and self-organization of scientific communications], translated into Japanese by Yuko Fujigaki, Takayuki Hayashi, Hideyuki Hirakawa, Junichiro Makino, Masahi Shirabe, and Hiroyuki Tomizawa.
Leydesdorff, Loet and Susan Cozzens (1993). The Delineation of Specialties in terms of Journals Using the Dynamic Journal Set of the SCI, Scientometrics 26, 133-54.
Leydesdorff, Loet and Peter van den Besselaar (1997). Scientometrics and Communication Theory: Towards Theoretically Informed Indicators, Scientometrics 38, 155-174.
Leydesdorff, Loet and Henry Etzkowitz (1998). The Triple Helix as a model for innovation studies, Science and Public Policy 25, 195-203.
Makino, Junichiro (1998). Productivity of research groups: relation between citation analysis and reputation within research communities, Scientometrics 43, 87-93.
Maturana, Huberto R. and Francisco J. Varela (1980). Autopoiesis and Cognition: The Realization of the Living. Reidel, Boston.
Narin, Frances and Dominic Olivastro (1992). Status report: Linkages between technology and science, Research Policy 21, 237-49.
Pair, Cees le (1988). The citation gap of applicable science, In: A.F.J. van Raan (Ed.), Handbook of Quantitative Studies of Science and Technology, Elsevier Science/North‑Holland, Amsterdam, pp. 537-553.
Price, Derek de Solla (1970). Citation measures of hard science, soft science, technology, and nonscience. In: C. E. Nelson and D. K. Pollack (Eds.), Communication among Scientists and Engineers. Heath, Lexington, MA, pp. 3-22.
Rip, Arie (1997). Qualitative Conditions of Scientometrics: The New Challenges, Scientometrics 38, 7-26.
Rousseau, Ronald (1997). Sitation: an exploratory study, Cybermetrics 1, Issue I, Paper 1 at http://www.cindoc.csic.es/cybermetrics/articles/v1i1p1.html
Rousseau, Ronald (1998). Citation analysis as a theory of friction or polluted air, Scientometrics 43, 63-67.
Sahal, Devendra (1985). Technological Guideposts and Innovation Avenues, Research Policy 14, 61-82.
Scharnhorst, Andrea (1998). Citation ─ Networks, Science Landscapes, and Evolutionary Strategies, Scientometrics 43, 95-139.
Small, Henry (1998). Citations and Consilience in Science. Scientometrics 43, 143-148.
Stichweh, Rudolf (1984). Zur Entstehung des modernen Systems wissenschaftlicher Disziplinen. Physik in Deutschland, 1740-1890. Suhrkamp, Frankfurt a.M.
Swanson, Don R. (1990). Medical Literature as a Potential Source of New Knowledge,' Bull. Med. Libr. Assoc. 78, 29-37.
Van Alstyne, Marshall and Erik Brynjlofsson (1996). Could the Internet Balkanize Science? Science 247 (29th November), 1479-1480.
Van Raan, Anthony F. J. (1998). In Matters of Quantitative Studies of Science the Fault of Theorists is Offering Too Little and Asking Too Much, Scientometrics 43, 129-148.
Vinkler, Peter (1998). Comparative Investigation of Frequency and Strength of Motives toward Referencing: The Reference Threshold Model. Scientometrics 43, 107-127.
Watts, Duncan J. and Steven H. Strogatz (1998). Collective dynamics of `small-world' networks, Nature 393 (4 June), 440-2.
Wouters, Paul (1997). Citation Cycles and Peer Review Cycles. Scientometrics 38, 39-55.
Wouters, Paul (1998). The signs of science. Scientometrics 41, 225-241.
Zuckerman, Harriet and Robert K. Merton (1971). Patterns of Evaluation in Science: Institutionalization, Structure and Functions of the Referee System, Minerva 9, 66-100.