Diversity of Science Linkages: A Survey of Innovation Performance Effects and Some Evidence from Flemish Firms

This paper discusses the diversity of mechanisms which firms can deploy to link to science and how science links are associated with their innovation performance. Using a sample of Flemish firms, we show that there exists considerable heterogeneity in the type of links to science at the firm level. Overall, firms with a science link enjoy superior innovation performance, in particular with respect to innovations that are new to the market. At the invention level, our findings confirm that patents from firms engaged in science are more frequently cited and have a broader technological and geographical impact. We show that it is crucial to distinguish between direct science links at the invention level and indirect science links at the firm level to encounter distinct positive effects.


Introduction
An important and recurrent concern in economics has been to understand to what extent science influences technological progress. This literature has shown that knowledge flows from universities and public research centres make a substantial contribution to industrial innovation and, consequently, to economic growth and public welfare (e.g. Adams 1990;Mansfield 1991;Cohen et al. 2002).
More recent research suggests that the links to basic research have dramatically increased in the last decade. There is evidence of rising university spin-offs (Jensen and Thursby 2001;Thursby and Thursby 2002), universityindustry collaboration (Liebeskind et al. 1996;Darby and Zucker 2001;Zucker et al. 2001;, mobility of university researchers (Kim et al. 2005), and science-linkage in private patents (Narin et al. 1997;Hicks et al. 2001). Narin et al. (1997) for instance, report a threefold increase in the number of academic citations in industrial patents in the United States through the mid 1990s. These patterns suggest an increased opportunity for innovation offered by linking to science and scientific institutions.
In spite of this evidence about firms' growing connectedness to science, our understanding about how knowledge transfers occur through these links and how they translate into innovation at the firm level remains unclear. For private organizations to create and maintain such links to science, ultimately, this knowledge should improve their innovative performance.
In this article, we shed some light on the debate on the "diversity" of linkages to science used by firms and their relationship to firms' innovative performance. After briefly reviewing the literature, we provide some evidence from Flemish firms. Combining patent, publication and innovation survey information for these firms, we consider a variety of industry science link indicators: (i) cooperative R&D agreements with public research centers and universities, (ii) use of public scientific information sources-universities, public research centers, conferences, meeting and publications-to innovate, (iii) citation to scientific literature in patents of the firm, and, (iv) involvement in scientific publications by the firm.
A companion paper, Cassiman et al (2008), investigates the link between the quality of inventions produced by firms and the existence of an industry-science link (ISL). This paper focuses instead on the diversity of ISLs being used by firm. A first contribution of the paper is to underline the diversity in Industry-Science Link (ISL) mechanisms being used by firms, suggesting the need to look beyond a single "silver-bullet" industry-science link, to include the full portfolio of industry science links. A second contribution of this paper consists in evaluating how the use of these different types of ISL is associated with the innovation performance of the firms using ISL. Two types of analysis are presented. First, we relate linkages to science to the different indicators of innovation and economic performance at the firm level. Second, we delve further into the micro-level connections between science and innovation performance, focusing on the invention (i.e. patent) level. For this we compare the differences in patent quality (forward citation) between patents with and without science linkages. We also return to the firm level, by comparing the quality of patents (forward citations) of firms with science linkages vis-à-vis patents of other firms.
The paper is organized as follows. Section 2 presents a summary of the literature and reviews previous empirical work on the value of science for firm's innovations. 1 While our contribution is intended to be rather descriptive, the review does lead to the formulation of our main hypotheses. Section 3 describes our data and the methodology. Basic descriptive statistics are presented on the frequency of ISL, and the adoption of ISL by firms across various types of industries. Section 4 evaluates the relationship between ISL and firms' innovation performance. The final section concludes and suggests avenues for further research.

The Value of Science
The value of science for innovation and growth has been demonstrated using a diverse set of methodologies. At the macro-level, Griliches (1979) and Adams (1990) have shown the important contribution of basic research (e.g. public research expenditures and scientific publications) to economic growth.
Complementary research based on firm surveys has provided an alternative estimation of the contribution of basic research for industrial innovation and _________________________ 1 We restrict our review of industry science links to the perspective of the impact on firm's innovative performance. For a broader review of the literature on industry science links, see e.g. Agrawal (2001). economic performance. In a survey of 76 U.S. firms in seven industries, Mansfield (1991) found that 11% of new product innovations and 9% of process innovations would not have been developed (without substantial delay) in the absence of recent academic research; these innovations represented respectively 3% and 1% of sales. Both the 1983 Yale Survey and the 1994 Carnegie Mellon Survey of R&D have also shown the relevance of university research for industrial innovation (Cohen et al. 2002). These studies provide some insight as to the importance of different channels to link to science. The results indicate that the key channels include published papers and reports, public conferences and meetings, informal information exchange, and consulting. For a set of surveyed large European industrial firms, Arundel and Geuna (2004) find that public science is amongst the most important sources of technical knowledge for their innovative activities. Evidence from a wider set of firms, surveyed across firm size categories, sectors and European countries, indicates that less than 4% of all innovating firms find universities a highly important source of information for their innovation process (European community innovation surveys (Parvan 2007). However, for firms that develop products or processes that are new to the market, this percentage increases to 31%. Therefore, it seems that science is more important as a source of knowledge when innovations new to the market are developed.
The management literature has tried to open the firm's black box on why science linkages matter for firms. Different mechanisms have been associated with this beneficial effect of science on innovation performance of firms. As science provides a codified form of problem-solving, it increases the efficiency of private research (Arrow, 1962). In addition, science serves as a map for technological landscapes guiding private research in the direction of most promising technological venues avoiding thereby wasteful experimentation (Fleming and Sorenson (2004)). Probably the most discussed argument of how actively engaging in science might increase applied research productivity is the fact that this link to science leads to a better identification, absorption and integration of external (public) knowledge (Cohen and Levinthal 1989;Gambardella 1992;Henderson and Cockburn 1996;Cassiman and Veugelers 2006). Faster identification, absorption and integration of external knowledge in turn leads to increased productivity of the applied research process, resulting in faster translation of research into new technologies (Fabrizio 2009;Cassiman et al. 2008). Furthermore, a better and more fundamental understanding of the technology landscape encourages non-local search for improving technologies as opposed to local search, leading to more diverse research projects' being explored. More basic knowledge can simultaneously fertilize different research projects. At the same time, scientifically active firms can be expected to generate "unexpected" outcomes, which in turn improves the productivity of applied R&D and as a consequence the productivity of the innovation process (Sobrero and Roberts 2001;Aghion et al. 2009). Finally, rather than affecting the output of the innovation process, Stern (2004) argues that science active firms might affect the inputs of the innovation process. By setting up a science friendly environment, the firm attracts researchers willing to accept a lower salary in return for the freedom to publish. These researchers provide value along two dimensions: they not only generate important labor costs reductions and consequently higher productivity of internal research, but they also constitute a "bridge" ('gatekeepers' and "boundary spanners") with the scientific or academic world.
A growing literature has tried to empirically assess the impact of ISLs on firm performance. Most empirical evidence shows that adoption of science is indeed not costless. It is highly conditional on firms having "absorptive capacity" (Cohen and Levinthal 1989). This branch of research builds further on the notion that a firm's ability to apply university research for its own commercial gain is a function of its investment in R&D, which is taken to be the indicator for a firm's absorptive capacity. Using university collaboration as the mechanism for ISL, these papers find support for the hypothesis that these links boost internal R&D investment (e.g. Adams et al. 2000), and affect consequently innovation productivity and sales (Belderbos et al. 2005). Cockburn and Henderson (1998) build further on this notion of absorptive capacity, but add that the degree to which firms are "connected" to universities is also important for effectively linking to science. Firms must be connected to science by being actively involved in sharing research results (publishing) and also engaged in research collaboration. Using data on co-authorship of scientific papers for a sample of pharmaceutical firms, they show that firms connected to science show a higher performance in drug discovery. Differences in the effectiveness with which a firm is accessing the upstream pool of knowledge correspond to differences in the research productivity of firms of as much as 30%.
Zucker and various co-authors further investigate the importance of connectedness by examining location decisions relative to star scientists, who either left their university to found firms or who established tight working relationships with colleagues in industry. Zucker et al. (1998Zucker et al. ( , 2002 and Darby and Zucker (2001) found that the location of top star scientists predicts firm entry into biotechnology (by new and existing firms) both in the United States and Japan. Darby and Zucker (2005) find similar evidence that firms enter nanotechnology where and when scientists are publishing breakthrough academic articles. In addition, collaborations between particular university star scientists and firms had a large positive impact on firm research productivity, increasing the average firm's biotech patents by 34%, products in development by 27%, and products on the market by 8% .
Less empirical work exists on the effectiveness of science links at the invention level. Previous empirical research has shown that patents of universities are broader in scope and cited more frequently than private patents (e.g. Jaffe et al. 1993;Henderson et al. 1998;Narin et al. 1997). Yet, there is little evidence about the effectiveness of science-links to explain the quality of private patents. To analyse the link to science at the invention (i.e. patent) level, mainly the effect of the citation of scientific literature or the involvement of an academic researcher as inventor has been examined. Patents with references to science are found to be more important "applied" patents (Cassiman et al. 2008), and to generate more economic value for pharmaceutical and chemical patents, but not in other technical fields (Harhoff et al. 1999). Fleming and Sorenson (2004) show that having a "scientific" reference matters for technological impact of patents but that the benefits of using science depend upon the difficulty of the inventive problem being addressed: science only appears as beneficial when researchers work with highly interdependent-or coupled-knowledge pieces.
In spite of apparent benefits from science links, the benefits of science links seem hard to trace at the firm level as evidenced by different empirical studies. Only a select set of firms within specific industries tend to show strong interest in the scientific know-how offered by universities or other science institutes. Science is a more natural important source of information for innovation in those so-called "science-based technology fields" where new breakthrough innovations can be achieved and transferred into new products and processes. For instance, Branstetter (2010) has shown that the number of scientific references in corporate patents is highly concentrated within a limited number of patent classes. Fields with frequent references to scientific knowledge, are biotechnology, information technologies and new materials. Especially these science-based technologies are strong contributors to the growth in patents, partly explaining the growing trend in knowledge transfers from science to industry. But on average, there is nevertheless a strong suggestion that the link between science and innovations is not evident, even in "science based technologies". Due to the highly specific nature of the know-how involved, being often still early stage, embryonic, industry science links are characterised by high uncertainty and high transaction costs for knowledge exchange (Jensen and Thursby 2001). In a survey based study on 38 Advanced Technology Projects (ATP), Hall et al. (2003) found that projects with university involvement tend to experience more difficulty and delay. 2 As a result, R&D managers often resent dealing with such joint projects (see ).
To summarize, linking with science can be beneficial for a firm's innovative performance, but it is not costless as it requires "connectedness" by building absorptive capacity, the recruitment of qualified scientists, and adapting new organizational practices. Given these obstacles, firms will carefully assess the expected costs and benefits from developing ISL and only a handful of firms will profit from ISL.
While most studies have focused on a particular type of ISL (most notably cooperative agreements or (co-)publications,) and in specific technologies (most notably pharmaceuticals, biotechnology or nanotechnology), there is no reason why a variety of types of ISL could not be viable conditional on the underlying industry, firm and technological conditions. Firms interested in ISL are expected to access science through different complementary modes as the marginal cost of investing in additional modes of linking with science is lower once the cost of organizing accordingly has been sunk.
In what follows, we will first document the diversity of ISL that firms can develop. As argued, we expect a certain degree of complementarity between these different ISL measures. For example, we expect that firms actively engaged in _________________________ 2 In a sample of 62 U.S. university licensing officers, Jensen and Thursby (2001) find that over 75% of the inventions licensed by these universities were in a very early, or embryonic stage. Further, 71% of the inventions licensed required cooperation between the professor and the licensing firm in order to commercialize a product successfully. Relying on the CIS for Belgium, Veugelers and Cassiman (2005), find that firms for which risk is an important obstacle to innovation are less likely to cooperate. publishing their research are likely to have collaborative agreements with universities and find publicly available knowledge important for their innovation process. Second, we will examine the performance of companies engaged in various combinations of ISL. While the overall performance is expected to be higher for firms with ISL, little is actually known about the relative performance of different types of ISL. Furthermore, we will examine ISL at a more disaggregated level: the invention (i.e. patent) level. We expect that at the invention level ISL would also be positively associated with performance, affecting the quality of inventions, as well as the nature of the inventions.

The Diversity of Linkages to Science
A first objective of our analysis is to assess the heterogeneity in industry science links used by firms. To this end, we use a combination of data-sets: innovation survey results, patent and publication data. 3 Innovation data are obtained from the Third Community Innovation Survey (1998)(1999)(2000) conducted in Belgium in 2000. 4 A mail survey was sent out to a representative sample of firms across all sectors and size classes. The intended respondent was the CEO or the R&D manager of the establishment. A total of 1471 responses were obtained from the 2726 surveys sent, resulting in a response rate of 54%. In this paper, we limit our sample to the responding firms that are engaged in innovation activities. 5 We also exclude firms from services sectors, as for these firms, patent data are less reflective of their innovative processes. This leaves in total 842 innovative _________________________ manufacturing firms. For these firms we can construct the following measures of ISL: • A dummy variable (0/1) indicating whether the firm had at least one cooperative agreement in R&D with universities or public research centers (Source: CIS-3). • A dummy variable (0/1) indicating whether the firm considers public scientific information a very important source for innovation. Firms scored the importance of public scientific information-(i) information from universities, (ii) public research institutions, or (iii) conferences, meetings or publicationson a scale from "0" (not relevant) to "3" (very important). Firms scoring "3" (very important) on one of these information sources are coded "1" in our measure of the importance of public scientific information for the innovation process 6 (Source: CIS-3). • A dummy variable (0/1) indicating whether the firm is simultaneously engaged in a cooperative agreement in R&D with universities or public research centers, and, considers public scientific information a very important source for innovation (effectively this measure is an interaction between the two previous measures) (Source: CIS-3). • A dummy variable (0/1) indicating whether the firm has been engaged in scientific publication activity. The measure takes the value of "1" if the firm published at least one scientific article between 1990 and 1995, predating the survey years and any patents of the firm (see below). Data on publications is collected from the ISI-Web of Knowledge database using the affiliation of the authors. A publication is considered scientific if it is found in the ISI Web of Knowledge and one of the authors is affiliated to the firm. • A dummy variable (0/1) indicating whether the firm has patents that contain references to scientific papers. These references are termed scientific nonpatent references (NPR). 7 An NPR is considered scientific if it is found in the _________________________ 6 We did not include lower scores of 1 and 2 in the dummy construction as this would lead to too many extra firms included in our sample, which were not using any of the other measures for ISL considered. Already with the highest possible cut-off rate of 3, this measure introduces the highest number of firms active on this dimension compared to the other ISL measures used (cf infra). 7 Some researchers consider patent and non patent citations as a "noisy signal" of knowledge flows, with examiners adding much of the noise (e.g. Jaffe et al. 1993). As patent and non patent references ISI-Web of Science. We collected all the patents of our CIS-3 firms in the European patent database (EPO ESPACE-B database) with grant dates between 1995 and 2001. These patents fall within the same time frame as the CIS survey. A total of 1186 patents were granted to 79 firms reported in CIS-3.
Each of these variables is regularly used as individual measure in empirical studies on ISL. This analysis combines these measures for the first time.
Tables 1 and 2 report the distribution of firms across the different measures of ISL and across industries. The first finding that emerges from these tables is the high number of firms without any ISL. About 75% of innovating firms do not have any linkage to science as witnessed by having cooperative R&D agreements with universities, being engaged in scientific publications, having references to scientific work in their patents or having rated public scientific information very important for their innovation process. A second finding is that engagement in ISL is indeed highly technology and sector specific. Not surprisingly, Table 2 shows that the low R&D intensive industries have the highest percentage of firms not having any connection to science (82%) while the opposite is true in the high R&D intensive industries. These results confirm the importance of technology and industry characteristics driving ISL. 8 A third finding is the heterogeneity across sectors with respect to the different types of ISL being used. Within the high R&D intensive industries, 25% of firms are engaged into cooperation with public _________________________ are issued from the examiner revision of the prior art in the European Patent Office, citations may rarely reflect or coincide with the science used by inventors. Care should therefore be exercised when interpreting the citation results as measures of direct knowledge flows. Nevertheless, some evidence exists that scientific NPRs are more likely inventor given compared to regular patent references (see Alcacer and Gittelman, 2006 Note: Only Cooperation with Public Institutes: firms that declare cooperating with universities and/or public research institutes (either national and international) as the only mean of accessing scientific knowledge. Only Use of Public sources: firms that consider public information sources as very important for innovation (score=3). The sources of information are: from universities or other higher education institutions, government or private non profit research institutes and from professional conferences, meeting and journals. institutions, 33% consider the use of public information as very important for innovation; while 16% declare to be engaged in both strategies. These results are related to the underlying industry effects as seen in Table 1. Not surprisingly, Electronics and Medical and Precision Instruments followed by Chemicals (including pharmaceuticals) score high on all types of ISL, while firms in low-tech sectors are less engaged in ISL. Some interesting industry variation in the relative use of ISL mechanisms emerges across industries from Table 1. Firms in Wood, Printing and Publishing industries or in Glass/Ceramics find public information sources relatively more important compared to engaging in cooperative agreements. Contrarily, firms in the Medical and Precision Instruments, Electronics or Vehicles business rely more on cooperation than on public information for their connections to science. Only very few firms report citations to the scientific literature in their patents. They represent less than 3% of the population of manufacturing firms in CIS casting some doubt on the coverage of such indicator for understanding links to science in the population of innovative firms. If we consider only the population of patenting firms, 24% (19 firms out of 79) of these firms have a reference to science in their patents. Figure 1 maps the overlap between different types of ISL. We classify firms according to whether they cooperate with universities, find public information very important, or, have patents that refer to scientific publications, or any combination of these ISL. 9 While we might have expected some complementarity between the different types of ISL, the diversity we actually find at the firm level is striking. While at the sector level there seems to be substantial correlation between different types of ISL as shown by Tables 1 and 2, at the firm level we find considerable diversity. Only five firms combine all three types of ISL (and four of those firms are also involved in publishing). Interestingly, seven firms that do not report any other ISL appear as firms with scientific NPRs in their patents. Out of these seven firms, five belong to the medium R&D intensive industries. Only 45 firms use both cooperation in R&D agreements with universities and public information as ISL. Out of these 45 firms only five report citations to science in their innovation outputs measured by patents. Similarly, the majority of firms with cooperative R&D agreements with a university (60 out of 100), and firms that find public _________________________ 9 Engagement in scientific publications only occurs as exclusive ISL engagement for 1 firm, and is therefore not treated in the graph as a separate set. information very important (74 out of 121) do not have any other type of ISL. Very few firms (8) are directly involved in open science through publication, but more surprisingly, five of these firms are found in the medium to low R&D intensive industries. Figure 1: Overlap between Types of ISL These simple descriptive statistics-although unfortunately based on a rather small sample-do corroborate the heterogeneity that exists in the channels firms use to access scientific knowledge both across firms as well as across industries. At the same time our results reveal that each individual ISL measure could provide specific information about firms linkages to science. Being able to use one channel does not seem to necessarily mean that the firm is able to use other channels, perhaps reflecting that each channel requires its own specific absorptive or connective capacity. This suggests an interesting avenue for further research.

Linkages to Science and Performance
In this section we focus the analysis on whether and which type of ISL enables firms to achieve higher innovation performance. Following the literature previously exposed, we expect that firms connected to science develop a comparative advantage in the production of innovation and notably, in the production of break-Firms with scientific NPR 7 (2 involved in scientific publication) Use of public sources of information 74 Cooperation with public institutions 60 (2 involved in scientific publication)

40
Firms without linkage to science 649 (1 involved in scientific publication) 2 5 (4 involved in science)

5
(1 involved science) through innovation. ISL facilitate the absorbing and understanding of fundamental knowledge, allow firms to follow new discoveries, upgrade internal technological competences, and detect new opportunities for industrial innovation. All of these effects of ISL are expected to improve the productivity of applied research. We present two levels of analysis. First, we relate ISL to the indicators of performance at the firm level reported in the CIS-3 data (Section 4.1). The key performance measure we will use is the percentage of innovations that are new to the market. We will also use the percentage of turnover due to innovations introduced in the past 2 years (i.e. 1998-2000), and, the percentage of turnover due to new market introductions during those past two years. Second, we restrict our analysis to the 79 firms with patents. For these firms, we analyze the effect from ISL on the quality of patents, both at the level of the invention (i.e. the patent) and at the level of the patenting firms (Section 4.1).
Before we present the results, we discuss a few caveats. First, the restricted number of firms retained in the analysis includes almost exclusively large, incumbent firms. As we do not have young and/or small firms in our sample, the results may not be representative for all firms. Second, the analysis simply correlates the use of industry science links to performance, and can thus not be considered as evidence for a causal relationship. This would require controlling multivariately for the influence of other firm-, industry-, technology-or timeconfounding characteristics. Such a full analysis is beyond the scope of this paper. Cassiman et al. (2008) provide such a full multivariate analysis, using quality of patents as performance measure. Their results are in line with the findings reported here, suggesting that they are robust to multivariate controls. Third, the information on sales from new innovations as a measure of innovative performance is taken in the same time horizon as the ISL indicators that are built on the CIS-3 data, namely cooperative agreements and use of public scientific information. 10 For the publication and patent-based ISL measures, we avoided this simultaneity by building in time lags. Nevertheless, and as the use of ISL are highly persistent over time, particularly for the set of large companies included in our analysis, the data set at hand is not able to fully account for the simultaneity problem. We will therefore be careful to avoid any "causal" interpretation of the results.  Table 3 displays the means for the R&D intensity, the number of employees, sales, and the measures of performance broken down by ISL. The percentage of firms that declare innovation new to the market is reported in the last row. Not surprisingly, firms with at least one ISL (column 2) are larger in turnover, have more employees and have high R&D intensity. These firms can more likely cover the sunk cost of becoming science linked. At the same time-corroborating the hypothesis advanced in the literature-firms with ISL have a higher percentage of sales from new or improved products (innovation turnover ratio) and a higher percentage of sales from innovative products that are new to the market (as opposed to new to the firm). Firms that have science linkages also show a higher frequency of innovations new to the market (47% versus 38%). When comparing different science linkages, firms declaring to cooperate with public institutions and also declaring the use of public sources of information as very important (column 5) have a high frequency of introducing innovations new to the market (44%). It is within the group of firms having scientific references in their patents (column 7) that we find the largest percentage of firms having introduced radical innovations (63%). Firms with this ISL also display the highest innovation turnover ratio and turnover due to new market introductions, but these firms are also larger and have a significantly higher R&D intensity. While firms with different ISL do display significant differences in size and R&D intensity, the differences in innovation output is not significant. 11 The correlation matrix in Table 4 offers additional insights on the correlation of ISL with respect to performance. Consistent with our finding about the diversity in ISL of firms, we find that having a link (column 1) has the highest correlation with the innovation turnover ratio, the new to market innovation turnover ratio and the new to market introduction indicator, while no one specific link seems to account for this positive effect.

Science Linkages and Performance
Overall, the evidence confirms that ISL and firm innovation performance are at least weakly positively related. At the same time, no single ISL seems to drive this result. Rather, firms can have higher (innovation) performance while engaging in various types of ISL, reinforcing the fact that there is substantive heterogeneity across firms in ISL. _________________________ 11 Small sample size is clearly an issue to obtain robust results.

Science Linkages and Quality of Patents
We now turn to the analysis of the association between industry science links and performance at the invention (i.e. patent) level. The performance measure we are considering is the quality of the firm's patents. As measure of patent quality we use the number of forward citations received 12 . Past research has shown that the number of citations a patent receives is highly correlated with its technological importance and social value (Trajtenberg 1990). Moreover, forward citations are correlated with the renewal rate of patents, the estimated economic value of inventions and patent opposition (Lanjouw and Schankerman 1999;Harhoff et al. 1999;Hall et al. 2000).
We have also computed two additional indicators describing the nature of the invention related to the generality of the impact of the patent. These indicators are based on the sources of forward citations across different technology classes and geographic areas. A high technology generality score indicates that the patent had a broad technological impact where it influenced subsequent innovations in a broad set of technological fields (Hall et al. 2003). This indicator is build as a Herfindahl index (Jaffe et al. 1993;Hall et al. 2003): , where s ij denotes the percentage of citations received by patent j that belong to patent class i, out of n i patent classes. 13 If the patent receives all of its future citations from a single patent class, the index is equal to zero. A higher generality index implies a more technologically diverse set of patents that cite the focal patent. The index of geographical dispersion is built in a similar way: , where s ij denotes the percentage of citations received by patent j that come from country i, out of n i countries. The index is based on the country location of the inventors. A higher index means that future citations come from a more diverse set of countries, which relates to the notoriety of the technology.

_________________________
12 To allow for more variability in the forward citation measure we count all citations received. In the absence of a fixed citation window, a correction needs to be made for application year, as younger patents have a shorter window to receive citations. The multivariate analysis of the companion paper Cassiman et al. (2008) introduces such application year corrections and shows the robustness of the results for these corrections. 13 Patents are classified according to a system of technological patent classes (IPC-codes).
As a measure of ISL at the invention level, we use scientific NPRs (cf supra). To test the association between industry science links and quality of the invention, we compare patent quality and generality-number, technical and geographical scope of citations to the patent-of patents with scientific NPRs to the quality and generality of patents without scientific NPRs. However, we also want to test any possible firm-specific effect of how industry science connections may be effectively transferred to other inventions inside the firms, beyond the directly linked inventions. To this end, we compare the quality and generality of all patents (including both those with and without NPR) of firms with ISL to the quality and generality of patents of firms without ISL. For assessing firms with ISL we use various measures, like publishing, cooperating or scanning public information.
The analysis is performed on the 79 Flemish firms which hold granted patents from the European Patent Office with grant dates between 1995 and 2001. These 79 firms together account for 1,186 patents. The forward citations to these patents-the number of citations received by the patent from future patents-are computed until 2003.
The breakdown of patent quality measures across the firms distinguished according to the different types of ISL they use, is reported in Table 5a. As expected, firms having at least one ISL to scientific communities report a higher likelihood of their patents being cited (dummy for having at least one forward citation), their patents appear more general in scope (are cited more across different technology classes) and have a higher geographical dispersion. However, the difference in means is significant (at 10%) only for geographical dispersion and the frequency of being cited at least once (dummy for forward citation). Firms that cooperate or use public sources of information report on average 0.69 and 0.71 citations to their patents respectively and firms involved directly in science through own publication activity report an average 0.72 forward citations to their patents. This compares to 0.67 for the overall sample. While these results seem to confirm the superior performance in terms of patent quality and generality from firms engaged in science linkages, the effects are small and only marginally significant and no particular type of firm ISL seems to stand out. Table 5b reports the comparison of patents with scientific references (NPRs) to patents without scientific references. Contrary to our expectations, we find that patents without NPRs are more likely to be cited (33% versus 24%) and have a higher mean of forward citations. Patents with NPRs are more general and more geographically dispersedly cited, reflecting their more general knowledge characteristic. Patents citing a scientific publication appear to cover more fundamental knowledge and they are therefore more likely to be cited across a broad range of technology classes and across different countries. Although the results are on a small sample and not robust, 14 they suggest that while patents with scientific NPRs protect more general technologies, more applied patents-patents without scientific NPRs-actually capture the value for the firm.

_________________________
14 Only for generality and the likelihood of receiving a forward citation, these differences are significant, but only at the 10% level.
Finally, in Table 6 we combine the invention and the firm level of analysis. Controlling for whether the firm has developed an ISL (either by publishing, or by cooperating or by using scientific information), we compare the quality and generality of all their patents, both with and without NPRs, to all patents of firms without ISL. This allows testing whether the impact at the patent level of having a science connectedness may depend on the firm's overall science connectedness. In the first panel we consider only firms with scientific publications and look at the quality of patents with and without scientific NPRs (columns 1 and 2). We confirm the results from Table 5b that patents with scientific NPRs are more general and their citations are more geographically dispersed, but these patents are less likely to be cited. However, and more interestingly, comparing the forward citations of patents without NPRs of these firms to the patents of other firms that have no publications (columns 2 and 4), we find that the non-NPR patents of publishing firms are more likely to be cited and receive more citations (0.36 versus 0.27) and (0.72 versus 0.55) respectively. Firms with scientific publications not only are more likely to have patents with scientific NPRs, but also have higher quality patents of the applied type (i.e. patents without scientific NPRs). As this only holds for publishing firms, only those firms that have developed a science connectedness at the firm level, will be able to capitalize on an invention-specific science link through having more valued applied (i.e. non-NPR) patents. These results strongly suggest that when assessing the benefits from ISL for firms, one should look beyond the direct invention level effect, but take into account indirect firm level effects.
This result is confirmed in the panels below for firms that cooperate in R&D with public research institutions, or, for firms that consider public sources of information very important. We conclude that controlling for the firm level science links when evaluating patent quality is crucial to pick up the innovation performance effect of science links. Patents from firms engaged in ISL will be more valuable. This higher value is not necessarily realized on the patents with the direct science connections, but also and a fortiori on the more applied patents, which do not have the direct link to science. These latter indirect effects are not obvious. It requires that the firm has an overall science connectedness profile to transfer effects from science linked inventions to more applied inventions. This important result is consistent with the multivariate analysis in Cassiman et al. (2008) that does not consider the scope of ISL mechanisms of this paper, but does show that a similar relation between a specific measure of ISL and patent quality is robust to firm, technology and time controls.

Conclusions
This paper examines the diversity of the types of links to science and their association to innovation performance for a sample of Flemish firms. We identify different ways to access scientific knowledge, using information from the Eurostat Community Innovation Survey, and add additional measures on the use of science by firms by analyzing publication data and citations to science in these firms' patents.
We confirm previous findings in the literature that firms with science linkages seem to enjoy at least some superior innovation performance. However, contrary to our expectation we find an important heterogeneity in the use of different forms of ISL and furthermore that different types of ISL are not complementary. A positive association of these links to performance cannot be related to a particular type of linkage.
Furthermore, the association may not necessarily run as expected. Patents that directly cite science are actually less likely to receive forward citations. This could be related to their more basic nature. But if cited, these citations are more likely to come from a broader set of technologies and geographies, consistent with the more basic nature of science linked inventions. Patents from firms that are actively engaged in ISL at the firm level through cooperative R&D agreements, publishing or scanning public information sources are more highly cited. Interestingly, this superior performance holds especially for the patents of these firms that do not refer to science directly. We speculate that firms with active ISL have a better understanding of fundamental technologies. As a result their regular patents (i.e. not directly linked to science) are also more valuable.
Our analysis merely establishes associations between the use of industry science links and innovative firms and this for a very selective and small sample of firms, precluding drawing any robust policy conclusions at this stage. Nevertheless, some insights can be drawn from the results to direct further research. Our results indicate that several indicators need to be tracked to obtain a representative picture of the ISL activity of a firm, an industry and a country. Future analysis should concentrate more on examining which mechanisms are used by which firms in which sectors and in which combinations. In addition, to bring out the true effect of these links, firm and invention level industry science link indicators need to be interacted. More particularly, our results indicate that further research is needed to understand the process of how the link with science affects the productivity of applied research internal to the firm. Understanding this process would open the door to develop more relevant measures related to the effect of science on innovation at the micro-level.