Seventy years of scientific impact using neutron beams at the Chalk River Laboratories

The 31 March 2018 closure of the National Research Universal reactor marked the end of over 70 years of materials research using neutron beams from major neutron sources at the Chalk River Laboratories in Chalk River, Ontario, Canada. This closure will have a major impact on the Canadian materials research community, including researchers in the physics, chemistry, and engineering of materials. After a brief review of the history of neutron beams at the Chalk River Laboratories, we present the results of a bibliometric study of the scientific output of the research with neutron beams. In this study, we compiled a complete bibliographic record of the research papers beginning with the first neutron scattering experiments at the National Research Experimental reactor in 1947, analyzed the citations from 1980 onward, and benchmarked the results against major neutron beam facilities in other countries and against other major research facilities in Canada. We also conducted a broader bibliometric analysis of the use of neutron scattering data among all Canadians, regardless of where the data were taken. The results provide a useful metric of the size of the Canadian neutron scattering community and places into context the importance of access to this research tool.


Introduction
Canada has a proud heritage in the development and application of neutron beams, now recognized as invaluable tools for the study of materials. The 2018-2019 wind-down of neutron beam science at Chalk River was an appropriate time for a retrospective study of the scientific impact such science has had in Canada. This retrospective study was undertaken by the Canadian Neutron Beam Centre and the Canadian Institute for Neutron Scattering, which represents the interests of Canadian researchers who use neutron beams for research. Research publications in scientific journals and proceedings are the main product of scientific research. Although there is an ongoing debate on the relevance of publication numbers, citation counts, and related indicators as a measure for the quality of science, such indicators still provide a useful gauge of productivity of, and importance to, the nation's researchers (Gutberlet et al. 2018b).

A brief history of neutron beams at Chalk River Laboratories
In 1942, a secret Canada-Britain lab was established in Montreal as part of the war effort under the leadership of the National Research Council of Canada (NRC). Preparations were then made to prepare a new site for a much larger secret nuclear research lab at what is now known as the Chalk River Laboratories. The Chalk River site opened in 1944. In September 1945, the NRC began operating the Zero Energy Experimental Pile (ZEEP) reactor, the first nuclear reactor outside the USA. ZEEP was followed by the National Research Experimental (NRX) reactor in 1947, where the first neutron scattering experiments were conducted.
In 1952, Atomic Energy of Canada Ltd. (AECL) was created out of the NRC to operate the Chalk River site and promote peaceful uses of nuclear energy. Lessons from NRX informed the development of the 10-times more powerful National Research Universal (NRU) reactor, which began operating in November 1957 and remained one of the world's most powerful research reactors (operating at over 100 MW) until it closed on 31 March 2018. NRU was the primary facility for in-reactor testing of nuclear components during the development of Canada's fleet of nuclear power stations. The NRU reactor also produced radioisotopes such as molybdenum-99, which have been used to diagnose an estimated 500 million patients around the world over its lifetime. NRU also provided a flux of 3 × 10 14 neutrons/cm 2 /s in its core at the neutron beam tube entrance. This high a flux enabled AECL's neutron nuclear physicists as well as its neutron scatterers to perform world-leading experiments.
In this section, we present a brief summary of the 70-year neutron beam program at Chalk River, focused on neutron scattering. Additional historical details are found in reference (Daniel Banks 2018).

Neutron beams at NRX
The NRX reactor was built as a source of neutrons for in-reactor irradiation, but also included external beam ports for various scientific applications, only one of which was the idea that neutron beams might be useful as probes of materials in the future. These beams were most heavily used for neutron nuclear physics research, as this field was then of great interest to understand the potential of nuclear science and technology.
Don Hurst was the inspiration behind the neutron scattering program at NRX. Hurst brought together key scientists in the late 1940s to develop early neutron spectrometers and demonstrate their potential by taking early measurements on gases and solving the structure of deuterated ammonium chloride. The experimentalists were organized as the Neutron Spectrometers Section of the General Physics Branch of AECL, while the theorists belonged to the Theoretical Physics Branch.

Neutron beams at NRU
The availability of higher-flux neutrons at NRU led to an expansion of research activities with neutrons and the creation of a Neutron Physics Branch at AECL in 1960. The Branch was headed by Brockhouse until he departed for McMaster University in 1962.
By the late 1970s, AECL's Neutron and Solid State Physics Branch (as it was then called) operated several beamlines: triple-axis neutron spectrometers at the C4, C5, L3, and N5 beam ports, a gamma-ray monochromator for Compton scattering at C1, a fast neutron chopper at C2, and a thermal neutron beam for neutron-capture gamma ray experiments at N4. In addition, university researchers also partnered with AECL to establish two additional beamlines at NRU: the McMaster University double-monochromator triple-axis spectrometer at E3, used by Professor Brockhouse's students, and the GWELFNEUD II diffractometer at D3, built by Professor Peter Egelstaff (Guelph University) for neutron scattering of liquids and gases. There was also an ultra-cold neutron experiment at T3 for Professor John Robson of McGill University.
In the 1980s, AECL developed a world-leading neutron imaging beamline at the N1 beam port where it demonstrated usefulness of neutron imaging for quality inspections of turbine blades. Commercial work for the aerospace industry developed into a business line that was spun-off as a private company, Nray Services, in 1994.
Also in the 1980s was the first funding partnership with academic granting agencies to fund new beamlines. The DUALSPEC instrument, which included a triple-axis spectrometer on C5 and a powder diffractometer on C2, was funded jointly by AECL and the Natural Sciences and Engineering Research Council (NSERC) in 1985. Since DUALSPEC's monochromator drum would take up the entire C-face of the reactor, the instruments there had to be moved to other beam ports, leading to a domino effect in which most of the instruments were either moved or decommissioned. The instruments that remained in 1991 when DUALSPEC was completed were three triple-axis spectrometers at E3, L3, and N5, a prototype low-angle scattering instrument at T3, and the N1 imaging facility.
The new suite of instruments reflected the changing mission of the neutron beam lab. In response to reduced funding and a changing mandate, AECL had begun shedding its basic research activities in the mid-1980s. AECL chose to focus remaining resources for basic research efforts on just a few large, unique, world-leading facilities that could be made available to all Canadian scientists, namely, the neutron beam lab, the Tandem Accelerator Super-Conducting Cyclotron, and the Sudbury Neutrino Observatory (an experiment led by Queen's University). Thus, there was an increasing emphasis on neutron scattering (over neutron nuclear physics) and on a user access model for the beam lab. Prior to DUALSPEC, external researchers had obtained access to neutron beams primarily by collaborating with AECL scientists.
Although there were specific proposals in later years to add a cold neutron source at NRU and to develop new instruments, the instrument suite remained constant, with just a few exceptions. Nray Services moved its operations to the McMaster Nuclear Reactor shortly after Nray's inception, and the N1 beamline ceased operation. A new reflectometer was built at the D3 beam port in 2007 through a funding partnership with academic granting agencies. The T3 beamline was decommissioned in 2009.

The Canadian Neutron Beam Centre
The neutron beam laboratory at the NRU reactor was transferred to the NRC in April 1997, following a 40% cut in AECL's government funding effective in the 1996 fiscal year. The neutron beam lab was initially known as the NRC Neutron Program for Materials Research, and was later renamed the Under the NRC, the CNBC was initially smaller but more focused on neutron scattering experiments and on the user-access mission. The program was able to expand again by attracting partnership funding from academic granting agencies. The most notable contribution to operations was a grant from NSERC to maintain the facility in a state of readiness for access by external researchers. In addition, the program generated further revenue by continuing its highly successful commercial service begun under Tom Holden in the 1980s. These services were primarily based on stress measurements for clients in risk-sensitive industries such as nuclear energy, oil and gas, aerospace, rail, defence, and automotive industries.

Banks and Harroun
The CNBC reached a peak around 2008, running at full capacity with six beamlines highly subscribed by a community of over 700 frequent and occasional research participants, over 5 years, of all types: scientists, engineers, and students from universities, industry, and government laboratories from Canada and around the world. The CNBC's operations peaked at $4M per year and had achieved a funding balance of 60% from NRC for baseline operations, 30% from NSERC, and 10% from commercial services and other research income. Numerous beamline upgrades were put in place, distinguishing the CNBC in stress scanning, powder diffraction, and polarized triple-axis spectroscopy.
Austerity measures following the global recession in 2008 led to cutbacks at both NSERC and NRC that culminated in AECL again resuming responsibility for funding and operating the CNBC effective April 2013 (NRC retained ownership of the equipment and continued to act as the employer for the staff). This change reflected the on-going restructuring of NRC and AECL in which NRC needed to refocus on "market-driven research", while AECL needed to maintain the CNBC as a nuclear science capability that might continue to be needed for a new nuclear innovation agenda or for the remaining lifetime of the NRU reactor-both of which were under consideration by the federal government.
The restructuring of AECL led to the creation of a new legal entity in 2014, Canadian Nuclear Laboratories (CNL), which was given responsibility for operations of AECL's assets, notably the site and staff of the Chalk River Laboratories. Ownership of CNL was then transferred to a private sector consortium under a contract managed by AECL. Through these arrangements, CNL also became responsible for funding and operating the CNBC.
In February 2015, the Government of Canada informed CNL that it would not support a license extension of the NRU reactor beyond 31 March 2018. Planning to maximize value from the reactor's remaining years, CNL hired many of the CNBC staff to provide job security in the face of the reactor's pending closure. The CNBC then operated in its final years as a team of individuals drawn from both NRC and CNL. The CNBC officially closed on 29 March 2019 after a year of decommissioning and other wind-down activities.

Chalk River's role in three Nobel Prizes
While numerous scientific contributions were made using neutrons at Chalk River, in this section we summarize three cases where Chalk River neutron scientists played roles in Nobel Prize winning research. In two of these cases, essential contributions were made using neutron scattering. In the third case, neutron nuclear physicists applied their expertise to make significant contributions to a Nobel Prize winning experiment.

Pioneering neutron scattering
Bertram Brockhouse, who had arrived at Chalk River in 1950, took the lead on examining solids with neutron beams, and specifically on the development of inelastic neutron scattering. Brockhouse took over as Head of the Neutron Spectrometers Section in 1955 while continuing this ground-breaking research. The first demonstration of individual phonon frequencies by inelastic neutron scattering was reported in the same year by Bert Brockhouse and Alec Stewart using the first triple-axis spectrometer. In 1958, Brockhouse developed the "constant-Q" method in 1958, which greatly simplified inelastic neutron scattering experiments. These were pioneering accomplishments that led to Brockhouse's Nobel Prize in 1994. The prize was shared with Clifford Shull of Oak Ridge National Lab (USA). Their selection for the prize reflected the versatility and irreplaceability of neutron beams as scientific tools, providing insights about materials that cannot be obtained by other scientific techniques. In the 35 years between Brockhouse's pioneering work and the prize, his methods had been replicated and further advanced at major neutron sources around the world, enabling many areas of research in solid state physics.
A summary of Brockhouse's life and some of his pioneering experiments are available in Svensson and Rowe (2004) and Cowley (2005).

Experimental confirmation of topological materials
The 2016 prize was awarded to three physicists working in the United States, for theoretical discoveries in quantum materials: David Thouless, Duncan Haldane, and Michael Kosterlitz. In the 1970s and 1980s, they challenged conventional understanding of materials using topology to model materials and make theoretical predictions about how such topological materials would behave. In 1982, Duncan Haldane predicted that one-dimensional chains of magnetic atoms would have different properties depending on whether their atomic spins contained an even or odd number of electrons. Specifically, he predicted that an observable energy gap, which came to be known as the Haldane gap, would form for integral values of the spin but not for half-integer spins.
W.J.L. (Bill) Buyers at Chalk River led an investigation to verify Haldane's prediction. He conducted the first experiment to observe the Haldane gap in 1985 with Rose Morra, Robin Armstrong, and Mike Hogan (University of Toronto) and Peter Gerlach (Chalk River) (Buyers et al. 1986). Using a triple-axis spectrometer at the NRU reactor, they studied a CsNiCl 3 crystal grown by Kin Hirakawa (University of Tokyo). They also compared their results with computer calculations by French theorists Botet, Jullien, and Kolb to verify that the phenomenon they observed in the crystal was indeed the Haldane gap (Botet et al. 1983). CsNiCl 3 was thus the first example of a new type of material distinguished by its topology. The experiment overturned the wisdom of the day and led to the acceptance of topological theory, thereby opening the burgeoning field of topological materials.
Haldane discussed the importance of this experiment in his Nobel lecture (Haldane 2016), and a brief, detailed description of this discovery and a simplified explanation of the Haldane ground state can be found in Enderle et al. (2018).

Contributions to the Sudbury Neutrino Observatory
AECL's Neutron and Solid State Physics Branch included prominent neutron nuclear physicists who relied on the beams from the two reactors for portions of their research in addition to neutron and other particle beams produced by accelerators at Chalk River. These physicists included Davis Earle, Aslam Lone, and Warwick Knowles who collaborated with Art McDonald to publish over 30 papers in the 1970s and early 1980s, while McDonald was then a member of the Nuclear Physics Branch at AECL. These were primarily parity violation studies using accelerator-produced gamma rays that formed an important part of the experimental basis for the Sudbury Neutrino Observatory (SNO) experiment, led by Queen's University. Davis Earle and many other AECL scientists and technical staff contributed to the technical development of SNO into the 1990s, and the discoveries of electron neutrino oscillations arising from SNO led to McDonald's 2015 Nobel Prize in Physics as the leader of the SNO experiment.

The publication record of the CNBC: a Science-Metrix study
We compiled a list of over 2300 publications arising from the CNBC and its predecessors under AECL primarily from 160 progress reports of the AECL Physics Division from 1949 to 1996 archived by the Canadian Nuclear Laboratories library, and from annual activity reports from the CNBC for 1997-2017.
The AECL reports contained official publication lists and conference talks for each Branch. In the reports up to 1959, the neutron papers had to be distinguished manually from those of other physics fields. In 1960, neutron scattering, neutron nuclear physics, and neutron detector development were consolidated into a single branch of AECL, making the task of finding the neutron papers easier. The lists from the Theoretical Physics Branch were also examined for papers related to the theory of neutron scattering.
Online searches were then conducted to identify gaps in the official lists. The conference talks were examined to determine if there were published conference proceedings. Author searches were conducted on key journal websites using names of the most prolific users and staff. Individual author profiles were examined on Scopus.com. The full list was also cross-checked with off-line sources, such as curriculum vitae from retirees.
Finally, each paper's bibliographic entry was cross-checked with the record on the journal's website to check for errors and to obtain the digital object identifiers or other links to the publications online.
The full list is available on the CNBC Resources page the website of the Canadian Institute for Neutron Scattering (http://cins.ca/resources/cnbc/).
Citations are frequently used as proxy measurements of scientific impact, or quality, of the papers. Proper benchmarks are key to the interpretation of citation-based indicators, however. For this reason, the CNBC commissioned Science-Metrix Inc. to analyze the citations relative to the world averages in the sub-fields in which the CNBC papers appear. The Science-Metrix study also compared the results with that of five major neutron beam facilities in the United States and France and with three other major Canadian research facilities that provide users with access to probes for materials research (X-rays, muons, and electrons at the Canadian Light Source, TRIUMF, and the Brockhouse Institute for Materials Research, respectively).
The international neutron facilities were chosen for their size, reputation, and location. The High-Flux Isotope Reactor (HFIR, USA) and Laboratoire Léon Brillouin (LLB, France) are reactor sources with a mid-sized complement of instruments and scientific histories comparable to the CNBC. The Los Alamos Neutron Science Center (LANSCE, USA) is a spallation source of comparable infrastructure size for neutron scattering, with some additional scientific missions. The NIST (National Institute for Standards and Technology) Center for Neutron Research (NCNR, USA) is the North American pre-eminent reactor-based facility, running a robust program and whose location in Maryland fostered frequent collaboration with the CNBC personnel. The new Spallation Neutron Source at Oak Ridge National Lab was not chosen, because we desired facilities with publications over many years. Finally the Institut Laue-Langevin (ILL, France) is the global flagship facility as the brightest reactor source with the most beamlines and publications of any neutron beam facility.
This particular collection of comparison facilities we believe should have similar level of biases and gaps in the bibliometric data, including comparable levels of missing publications and citations, misattributed authorship, and nonindexed specialty publications.

Publication data set
The Science-Metrix study was limited to papers published since 1980, because of incompleteness in the citation data in the Web of Science (Clarivate Analytics) database for earlier years.
Publications from the comparator facilities were identified primarily if the name of the facility was found in the author affiliations, the abstract (since 1991), or the acknowledgements (since 2008). For the HFIR, the study relied on the publication database posted on the Oak Ridge National Laboratory website. For most of the comparator facilities, these methods failed to identify papers for the whole period. The Canadian Light Source is a new facility, which officially opened in 2004. Some of others have had names changes or changes in how authors have identified their affiliations over time. For example, the name "Canadian Neutron Beam Centre" dates back only to 2005, and for decades the authors from the CNBC's predecessor branches of AECL often listed their affiliation simply with AECL without reference to their branch. Such practices would have make it difficult to identify the papers from the CNBC prior to 2005 had we not compiled the full publication list manually.
All papers found from each facility were included. For example, research using muon beams to study materials at TRIUMF were not distinguished from TRIUMF's larger body of research focused on nuclear and particle physics. Figures 1 and 2 show the publications identified by Science-Metrix for each facility over time and serve primarily to illustrate the data set on which the citation analysis was performed.
From this publication data and numbers of neutron beamlines at each facility, it can be estimated that the CNBC's publication intensity, measured by number of papers per beamline per year, has been in the low end of the range of publication intensities for this group of neutron facilities in recent years. This finding contrasts with an international peer review in 2004 that found the CNBC to have a publication intensity that was competitive with the most prominent neutron beam facilities in the world. The same review, however, warned that investment in the beamlines and staff at the CNBC had not been keeping pace with other neutron facilities, thereby predicting a relative decline in productivity, which we now observe here; although the CNBC maintained a steady output of research over time, other facilities have increased their productivity significantly.

Scientific impact metrics
The four measures of scientific impact calculated were the average of relative citations (ARC), the share of highly cited papers (HCP 10% ), the Citation Distribution Index (CDI), and the average of relative impact factors (ARIF).
In the calculation of all of these quality metrics, citation rates are normalized according to the world average for each subfield and year, thereby producing a relative citation score for each paper or journal. There is no subfield for neutron scattering in Science-Metrix's journal-based classification system. Rather, since neutron beams are applied to a broad range of problems, research using neutron beams is published in many different journals, which are in turn classified into many different subfields. Some primary subfields include: general physics, biophysics, chemical physics, inorganic and nuclear chemistry, materials, and applied physics. Thus, relative citations are determined in comparison to the citation rates of such topical fields rather than against other papers that used a neutron scatttering technique. Furthermore, the citations are more often made by experts in these broadly defined fields who are interested in the materials being studied than by those mainly interested in the methods used.
ARC is the mean of the relative citation scores of all papers from each facility in a given year. By definition, the world average of relative citations equals 1.
As the mean of the distribution of relative citations can be significantly increased by a few highly cited papers, another common indicator of impact is an institution's ability to produce such "hit" papers. HCP 10% is the fraction of an entities' papers that was among the top decile of all papers in their subfields in that year. By definition, the world average of HCP 10% is 10%.
A limitation of HCP 10% as an indicator is that it only uses the citations from the top decile of papers. The CDI is a measure that represents the full shape of the distribution by using the relative citations from all 10 deciles of papers as inputs. The CDI is higher if an entities' papers are underrepresented in the low deciles or over-represented in the high deciles. By definition, the world average of CDI equals 0. In theory, the range of CDI goes from −50, the case of all papers being in the lowest decile of relative citations, to 50, the case of all papers being in the highest decile of relative citations. In practice, −25 or +25 are very low or high scores that actually occur. See Campbell et al. (2016) for more information onthe CDI (where the CDI is referred to as the "Relative Integration Score").
ARIF is the average impact factor of the journals the papers are published in. As with the other metrics, ARIF is normalized by year and subfield. By definition, the world ARIF equals 1. ARIF is different from the preceding metrics because it is indicative of perceived scientific value at the time of publication, rather than the subsequent impact, which is indicated through citations of the paper itself.
These four impact metrics were calculated for each facility over the range of available data and are summarized in Table 1. To estimate uncertainties, Science-Metrix took many samples on the data to observe variance between the samples and produce confidence intervals. These intervals are not formal estimates of error, because the results in Table 1 are exact calculations on all of the available data. Rather they illustrate the range that is expected to be observed if many new publications are added in the future and if these new publications follow the same distribution of citation scores. For example, if the further publications were to arise from the CNBC that followed the same pattern as seen from the CNBC over 2000-2017, ARC scores ranging from 1.19 to 1.53 would be expected 95% of the time. The intervals narrow with increasing numbers of publications. The same confidence interval for the ILL, which has the largest numbers of papers and relatively stable results over time (shown in section 4.2.1), has a width of only 0.06.
A key observation from this data is that the publications arising from all these facilities are performing at or above world average on all four indicators, and most are performing well above average (e.g., ARC > 1.3). These results suggest that having access to unique national shared resources for materials research, including neutron beams, enhances scientific impact.
The performance of the CNBC's publications were competitive with those of most of the comparator facilities. The NCNR in the United States was the top neutron beam facility in all four indicators, while the two neutron beam facilities based in France (ILL, LLB) were both at the bottom. This results for the ILL seem surprising since it has been the preeminent neutron facility globally for decades and it maintains a very high reputation. Further examination of the data and citation trends in France compared with Canada and the United States did not reveal a clear explanation for the difference. The most likely explanation is that the very large number of papers arising from the ILL, across a wide breadth of subject areas, mask the highly cited publications in the averages. To the best of our knowledge, there is no similar published institutional analysis of the ILL. These results may be also impacted by relative subject-area strengths and sociological factors, such as, language, scientific culture, and Science-Metrix also examined the performance of the CNBC publications within individual subfields, according to its journal classification system. The CNBC papers were found to fit into 30 of the 176 research subfields, whereas 80% of the papers fit into just six subfields. The three subfields making the top contribution to citation scores were applied physics, materials, and biophysics, as shown in Table 2. Other subfields in which CNBC publications were frequently found included general physics, materials, chemical physics, inorganic and nuclear chemistry, fluids and plasmas, health sciences, geochemistry and geophysics, mining and metallurgy, mechanical engineering and transport, and polymers.
Science-Metrix examined trends in the five top subfields that are contributing to the results in our paper to determine general trends between United States, Canada, France, United Kingdom, and Germany. Over the period 1980-2017, France scored the lowest ARC of this group 1.2, while Canada at 1.3 scored the same as the United Kingdom at 1.3. Germany scored 1.2, while the United States scored 1.5. Similarly for HCP 10% , France was again at the bottom with 12%, Canada at 13%, Germany 13%, United Kingdom 14%, and United States 17%. These results suggest that systematic factors that reduce France's citation scores in these subfields overall could be dragging down ILL and LLB's scores, although the effect is not large enough to be conclusive.

Impact metrics over time
To explore the above data on a deeper level, the impact metrics were calculated in 3-year rolling intervals and shown in Figs. 3-8. Scores of ARC, CDI, and HCP 10% were not calculated for intervals in which fewer than 30 papers were identified.       -8 show a significant amount of fluctuation from period to period, which indicates some uncertainty in the data. Some of the fluctuation can be attributed to random causes. Indeed, we observe that those facilities with the most publications tend to exhibit lower fluctuations in these quality indicators because more of the random effects are averaged out than for the facilities with fewer publications. The indicator that is calculated from the fewest data points, the HCP 10% data, exhibits the most fluctuation.
Although it is difficult to determine the nonrandom causes, some correlation with historical events affecting the CNBC can be suggested. AECL faced significant budget cuts beginning in the mid-1980s and was mandated to focus increasingly on the nuclear power industry. Although AECL had to cut back on fundamental research, it chose to keep the CNBC, providing stable support throughout the 1980s and early 1990s. Concerns about the future of the CNBC became more acute in a second wave of federal government budget cuts in 1993-1996, ultimately leading to a decision by AECL to Fig. 7. Citation Distribution Index for major Canadian facilities using a 3-year moving window. Scores were not calculated where fewer than 30 papers with a relative citation score were not identified. Fig. 8. Fraction of highly cited papers for major Canadian facilities using a 3-year moving window. Scores were not calculated where fewer than 30 papers with a relative citation score were not identified.

Banks and Harroun
FACETS | 2019 | 4: 507-530 | DOI: 10.1139/facets-2019-0003 518 facetsjournal.com eliminate funding for the CNBC in 1996. The negative effect that such concerns undoubtedly had on the remaining staff morale, coupled with the departure of some senior scientists in 1997 and 1998, may explain some of the drop in these indicators, especially visible in the HCP 10% data. In contrast, performance dramatically recovered after the CNBC was taken over by the NRC in 1997, which was fully supportive of fundamental research using the CNBC and encouraged pride in operating a national resource for the scientific community. Following this recovery, there was another small performance drop in the mid-2000s. The cohort of senior scientists that had characterized the 1970s, 1980s, and 1990s had mostly retired by this time, and the science staff in the mid-2000s could be described as a mix of mid-career scientists and new hires, many of whom were learning the role of an instrument scientist. After this brief drop, performance returned to near its historic average. Despite these fluctuations, the general stability of the CNBC's performance over a 37-year period is perhaps the most striking feature. The business model of the CNBC as a user facility and its ability to attract high quality users may help explain the facility's resilience, since the quality of its publications over this period have depended on the excellence of the facility's users. Expert users who maintain their engagement despite turnover in facility staff can serve as a bridge that helps maintain continuity and high scientific impact.
The general trends of ARC and CDI over time for the CNBC are similar with a few contrasting features. The ARC shows a local maximum in 1986-1988, which does not appear in the CDI. Such a difference could be explained by a small number of highly cited papers that have a large impact on the ARC, but have less impact on the CDI. Indeed, two very influential papers by W.J.L. (Bill) Buyers and collaborators appear in these years. They concern experimental confirmation of the Haldane gap (Buyers et al. 1986) (described earlier in section 3.2) and the discovery of superconducting pairs in an uniaxial antiferromagnet, URu 2 Si 2 in collaboration with Riso National Laboratory, Denmark (Broholm et al. 1987). In addition, Brian Powell and collaborators from the NRC published a highly influential paper on a clathrate hydrate structure (Ripmeester et al. 1987).
Opposite patterns are seen in the years around 1982 and 1994, in which the ARC drops to the world average level, but the CDI remains strong and well above world average. This outcome implies a strong performance overall, despite under-representation in the highest decile of relative citations (as the HCP 10% data in Fig. 5 shows).
Boh ARCs and CDI are high in the time windows that include the years 1998-2002 (coinciding with the recovery of the CNBC under the NRC as already mentioned). An examination of the papers in these years revealed a large number of highly cited papers. Although CDI is not strongly influenced by a one or two highly cited papers, the CDI will rise if a large portion of the papers are highly cited. Indeed, the HCP 10% data in Fig. 5 shows that over a quarter of the CNBC's papers in 1999-2001 were in the top decile.  (Greedan et al. 1998;Gardner et al. 1999aGardner et al. , 1999bGardner et al. , 2001Wills et al. 1999;Gingras et al. 2000), four publications by John Katsaras and collaboators using small-angle neutron scattering on morphology of phospholipid mixtures and the location of cholesterol in membranes (Léonard et al. 2001;Luchette et al. 2001;Nieh et al. 2001Nieh et al. , 2002, and two publications by Thomas Holden and collaborators on measurement of intergranular strains in structural alloys of aluminum or zirconium (Pang et al. 1998(Pang et al. , 1999. More influential papers from these years include a study of stresses in a magnesium-aluminum alloy by John Root and collaborators (Gharghouri et al. 1999), a study of the high-density amorphous phase of ice by Ian Swainson, Eric Svensson, and collaborators (Tse et al. 1999), and a study of the relaxor Pb(Mg 1/3 Nb 2/3 )O 3 by Bill Buyers and collaborators (Wakimoto et al. 2002).
A second set of years with high ARC, CDI, and HCP 10% is 1990-1992. The most highly cited paper on the CNBC's publication list was Varley Sears' tables of neutron scattering lengths and cross-sections from 1992 (Sears 1992), cited over 2800 times, according to Google Scholar. These years included three more influential studies by Buyers and collaborators on the superconductor URu 2 Si 2 Mason et al. 1990;Broholm et al. 1991) and a study on spin freezing in the geometrically frustrated pyrochlore antiferromagnet Tb 2 Mo 2 O 7 by Bruce Gaulin, Zin Tun (CNBC), and collaborators (Gaulin et al. 1992).
In more recent years (2007)(2008)(2009)(2010)(2011)(2012)(2013), the CNBC's most highly cited papers have been in quantum materials, biophysics, and materials engineering. The quantum materials papers include Lachlan Cranswick and collaborators' study on superconductivity in LaFe 1−x Co x AsO (Sefat et al. 2008); Zahra Yamani, Bill Buyers and, collaborators' study that found the reason for missing entropy in URu 2 Si 2 (Wiebe et al. 2007); and Zahra Yamani and collaborators' study of magnetic correlations in multiferroic LuFe 2 O 4 (Christianson et al. 2008). In biophysics, John Katsaras and collaborators published five influential papers on model membranes (Kuĉerka et al. 2008(Kuĉerka et al. , 2011Marrink et al. 2008;Pabst et al. 2010;Heberle et al. 2013), including two notable studies on structural parameters of lipid bilayers, i.e., area per lipid and bilayer thickness, using simultaneous data analysis from both X-rays and neutron experiments (Kuĉerka et al. 2008(Kuĉerka et al. , 2011) and a demonstration that cholesterol prefers the interior of polyunsaturated lipid membranes (Marrink et al. 2008). In materials engineering, Michael Gharghouri and collaborators showed that the difference in yield strength between tension and compression in magnesium 0150 aluminum alloys can be greatly reduced by increasing precipitates (Jain et al. 2010). Ron Rogge and collaborators studied the role of residual stress in cracking of steel pipelines, providing data that is now used by the oil and gas pipeline industry to better predict pipeline lifetimes (Van Boven et al. 2007).

International collaborations rates and impact metrics
An institution's international co-publication rate (ICR) may serve as an indication of scientific demand for its resources. It may also have some utility as a predictor of impact using the hypothesis that higher impact may be expected when scientists seek out the world experts on their topics as collaborators. ICR, however, is heavily impacted by factors such as size of the scientific community in the host country and proximity to potential foreign collaborators. For example, institutions in the United States often have strong performance despite lower ICRs than similar institutions in Canada or European countries. This is simply due to the greater probability in the United States that the best collaborator can be found domestically, because of its large scientific community; this trend is shown in Fig. 9. The papers from each facility were identified as international collaborations if more than one country was represented in the papers' author affiliations, and ICR was calculated as the fraction of such papers among all the facility's papers. The United States' neutron beam facilities have lower ICRs than the counterparts in France and Canada, despite higher performance in the quality metrics. The ILL consistently had the highest ICR over the period of this study. This high ICR is consistent with its mandate to provide access to the member countries who contribute funds to its operations. There has been a worldwide trend toward increasing ICR over time, and all the facilities examined in this study exhibited an upward trend. However, the greatest increase in the CNBC's ICR occurred in the several years following the transfer of CNBC to the NRC in 1997. This post-1997 rise in ICR was consistent with the NRC's model for the CNBC as a national user facility that acted as Canada's contribution to a global network of neutron beam facilities.

Banks and Harroun
To quantify the impact of international collaborations, Science-Metrix calculated the average relative citations on the subset of papers that were international collaborations, denoted ARC international, for all the available data in 1980-2017, and shown in Table 3.
International collaboration helped HFIR's ARC score, but interestingly, had little or no effect on ARC for the other facilities (ILL, LLB, LANSCE, NCNR). One interpretation of this result is that the users of these other facilities may have already maximized the value that they can get from effective Fig. 9. International co-publication rate (ICR) for neutron beam facilities using a 3-year moving window. collaboration. For the facilities in France, it is relatively easy to collaborate with other countries, both because of geography and the facilities' business models that encourage collaboration. For the facilities in the United States, heavily competitive merit-based processes for distributing beam time may strongly encourage the collaborations that will be most effective.
For all of the Canadian facilities, ARC international was higher than the ARC for all papers, consistent with two ideas about how Canada can best engage with, and utilize Major Research Facilities. First, for those facilities hosted in Canada, combining competitive and easy-to-access infrastructure with welcoming and knowledgeable staff is a major attraction for global scientists. For the CNBC, the continuous application and scheduling process was an important part of its reputation for being responsive to scientist's needs. Second, Canadian scientists, more frequently than American counterparts, need to engage collaborators outside their country to make the biggest impact, a phenomenon that may simply be a function of Canada's smaller scientific community.

Canadian utilization of neutron beams: a self study
The official list of publications arising from the CNBC reveals the depth of scientific impact that the facility has had. However, the official list does not capture the whole story of neutron scattering by the larger Canadian research community.
The CNBC lacked a cold neutron source and its small suite of six beamlines could never provide all the neutron scattering techniques that Canadian researchers needed. Thus, many researchers frequently traveled to neutron facilities abroad, especially for longer wavelength neutrons and techniques such as small-angle neutron scattering and certain inelastic neutron scattering measurements. These trips result in the development of international collaborations and an important exchange of experience and expertise not captured in the Science-Metrix study. We therefore undertook an additional self study of the community of Canadian scientists using neutron scattering as part of their research programs.
A survey of the members of the Canadian Institute for Neutron Scattering (CINS) was taken in May 2017 (Harroun 2017) (n = 82, 23% response rate); 71% of respondents resided in Canada, and colleagues in 10 other countries also responded. Most respondents were active neutron beam users, and 79% of Canadian members had participated in an experiment in the last 5 years, and 74% of Canadian members planned to participate in a neutron beam experiment in 2018.
CINS members preferred the CNBC; 55% of trips planned by Canadian members were to the CNBC even in its final year of operations. Internationally, Canadians most commonly use the facilities in the United States (32% of planned trips in 2018). Data provided by the various facilities in the United States indicated that between 86 and 120 Canadians participated in research at their facilities in 2012. In the same year, the number of American scientists that participated in research using the CNBC was 91. The next popular destinations were in Europe (12% of planned trips) followed by Australia and Asia (1%).
In addition to publishing results from neutron scattering experiments abroad, Canadians also use previously published neutron data as part of their research motivation and in discussion of, or comparison with, other experimental or computational results. We consider this usage of neutron data an important part of the scientific impact of neutron scattering: motivating new research directions and facilitating interpretation of other experimental or theoretical results. Measuring this broader impact is important for the scientific case to secure stable funding for continued access to neutron beam facilities following closure of the CNBC. We searched the Web of Science for all publications on neutron scattering, either with new data or reference to published data, and have at least one author with a Canadian affiliation. The detailed methods of the search are given in the Supplementary Material, and the results were curated manually for papers not meeting our criteria. We limited the search to the years 2008-2017, since affiliation data prior to 2008 are not complete. We assumed that by the mid-point of 2018, nearly all 2017 publications would have appeared in the database. Figure 10 shows an annual count of the number of papers from Web of Science that present or otherwise use neutron scattering data and involve at least one author with a Canadian affiliation. The data show there was a relatively constant average of approximately 120 such papers per year. Not every publication in the official CNBC list is in this total. The Web of Science does not index many book chapters or most conference monographs, which are especially important in engineering research communication. Nevertheless, this represents around 2% of the annual publications in physics and astronomy, using the total of 33 783 publications in 2009-2014 according to Blouw (2016).

Banks and Harroun
This number (120 papers per year) is below that found by the recent study by Gutberlet et al. (2018b) that, in their global analysis of the publication record of neutron scattering, found an average number of Canadian neutron papers 2005-2015 to be ∼165, placing Canada in twelfth place in total papers over that period (Gutberlet et al. 2018a). Although the keyword search criteria in that analysis attempted to be prescriptive, we believe it was no more efficient than ours at excluding irrelevant papers. Also, the refinement of the search results was made by the assumed relevance of the journal, not manually. This may have led to more false positives, that is, irrelevant papers, in cross-disciplinary journals.
Individual authors and their affiliations were identified from the bibliographic record for each year. Figure 10 shows the total number of Canadian authors on neutron beam papers average about 250 per year. This total includes all principal investigators, postdocs, and students, and it provides a rough gauge of the extent neutron scattering helps Canadian researchers. The number of foreign collaborators on these papers seems to be increasing, from about 300 per year to over 400. The cause of this increase is not clear, as care was taken to remove papers from large international collaborations not germane to neutron scattering. The trend is, however, consistent with growth in Canadian utilization of the Spallation Neutron Source (SNS) at Oak Ridge National Lab (Tennessee, USA), which was strengthened by a McMaster University led 2003 Canada Foundation for Innovation grant to build two instruments at the SNS in exchange for Canadian access. Canadians received preferred access to beam time in an amount equivalent to 30% of a beamline distributed as 10% of each of the two instruments and 10% of a beamline spread over the rest of the instruments. This arrangement ran from the opening of the SNS in 2008 until the end of January 2018 and resulted in Canadians being awarded half of the biennial Outstanding Student Research Prizes from the Neutron Scattering Society of America in 2014 (Dr. Kate A. Ross) and 2018 (Dr. Alannah Hallas). Fig. 11. The inclusion of neutron data in a publication was curated for how neutrons were utilized in the publication for certain years. Experimental reports of new data appear in red scale, with the location of the neutron source indicated. Papers that present older, published data are shown in blue; some papers only discuss neutron experiments in their context, others directly compare non-neutron results to previously published neutron data, and others merely cite neutron papers. References to neutron scattering in these papers arise mostly from new data. Figure 11 shows a breakdown of the reasons for inclusion of neutron beam data for selected years. Generally, a little over half the papers presented new data as shown in the red shaded areas. In a given year, the origin of the data was split between the CNBC and foreign facilities. We also see that the McMaster Nuclear Reactor, the best remaining source of neutrons in Canada, provided some data in each year.

Banks and Harroun
As shown in the blue shaded portion of Fig. 11, the remainder of Canadian-authored papers from Web of Science were split between direct comparisons with neutron data and discussion of results from neutron data. Clearly, neutron data help generate and inform a large body of new knowledge even after the experiment's original publication.
To better gauge the size of the actual research community that more routinely uses neutron scattering, the number of Canadian authors was further analyzed for those that publish at least two papers over 5 years (Canadian community), and those who publish at least four papers in 2 years (expert authors). The number of papers attributed to these expert authors are also shown in Fig. 12. The data show that expert authors comprise a little more than ∼1/5 of the total number of Canadians who use or refer to neutron scattering. Indeed, only half of the total Canadian authors in a given year rate as occasional users of neutrons, indicating neutron scattering is a powerful tool that informs a very broad cross-section of Canadian-based science.
Most Canadian researchers are based at Canadian universities and in nearly every province. Figure 13 shows a breakdown of the types of Canadian institutions involved with neutron scattering publications. On average, each year we find ∼5 Canadian-based private companies engaged with neutron research, and ∼5 Canadian-based research institutes such as the various research arms of the NRC, Natural Resources Canada, and the Department of National Defence. Interestingly, each year between 25 and 30 different countries participate in collaborative publications with Canadians where neutron beams are used. To better gauge the size of the size actual research community that more routinely utilize neutron scattering, the total number of Canadian authors was further analyzed for those that published at least two papers over 5 years (Canadian community), and those who published at least four papers in 2 years (expert authors). The number of papers attributed to these expert authors are also shown.

Conclusion
The data in this paper are in excellent agreement with the Council of Canadian Academies 2016 report on Canadian research performance and international reputation (Blouw 2016). The Council report found the subject area of physics and astronomy to have an ARC score among the highest of any discipline (1.54), but a low specialization index because of lower than expected publication output. We find both of these points extend to the use of neutron scattering, which although is cross-disciplinary, is most used by those in physics.
Canadians have been at the forefront of neutron scattering for 70 years, from its pioneering days until now. It is well known that the Chalk River Laboratories maintained a high scientific reputation from the early days of neutron scattering through the 1980s. The bibliographic analysis from 1980 to 2017 shows that the neutron scattering community continued to have a high level of scientific impact using the NRU reactor, up to the end of the reactor's life. Many of the highly cited papers arising from the NRU reactor have been in quantum materials, biophysics, and materials engineering.
The community of Canadian scientists who use neutron scattering as a research tool is large and diverse. The closure of the CNBC will undoubtedly have a profound impact on Canadian research programs. This study shows that over half of the output of neutron scattering from hundreds of researchers arose from the CNBC. The bibliographic analysis shows that all comparator facilities perform above world average, suggesting that accessing unique national shared resources for materials research, including neutron beams, enhances scientific impact.