Open access

Lack of molecular detection of frog virus 3-like ranavirus (FV3) in mosquitoes during natural outbreak and nonoutbreak conditions

Publication: FACETS
15 October 2020

Abstract

Ranaviruses are worldwide pathogens of ectothermic vertebrates that can threaten herptile conservation efforts. Identifying transmission routes is critical for understanding disease ecology and promoting species conservation. Frog virus 3 (FV3) DNA was detected in mosquitoes during a ranavirus outbreak in semicaptive box turtles, but the role that insect vectors play under natural conditions is unknown. To address this knowledge gap, we collected mosquito species known to take blood meals from reptiles and amphibians (Aedes canadensis, Culex erraticus, Culex territans, and Uranotaenia sapphirina) from mid-May to early August, 2014, at four study sites in Vermilion County, Illinois, two of which had historic or ongoing FV3 outbreaks in box turtles and amphibians. Mosquitoes were batched by date and species, DNA was extracted, and quantitative polymerase chain reaction was performed for detection of FV3. FV3 was not detected despite one of the sites having an active FV3 outbreak during the study period. Our findings indicate that FV3 detection is uncommon in mosquitoes during outbreak and nonoutbreak conditions at these sites in Illinois. Thus, we cannot establish that mosquitoes contribute to transmission during natural mortality events without performing further studies.

Introduction and purpose

Frog virus 3 (FV3) is the type species of the genus Ranavirus and belongs to a group of iridoviruses that has been responsible for die-offs in captive and wild amphibian, reptile, and fish populations around the world (Lesbarrères et al. 2012). Due to their wide host range and movement of ranavirus-infected animals in commercial settings, ranaviruses have become an increasing threat to captive and wild herptile populations (Lesbarrères et al. 2012). Ultimately, ranaviruses may pose a threat to the conservation of ectothermic vertebrates and may impede the natural functioning of the ecosystems in which they live.
FV3 is the only iridovirus identified in turtles in North America and is also the most commonly reported iridovirus for anurans in North America (Duffus et al. 2015). Ranavirus die-offs in wild and captive settings are often marked by rapid onset of clinical signs and mortality (Brunner et al. 2015). FV3 mortality events in turtles are infrequently described (De Voe et al. 2004; Johnson et al. 2008; Kimble et al. 2015; Sim et al. 2016; Adamovicz et al. 2018). However, chelonians represent one of the most-at risk group of vertebrates, with more than 60% of species considered threatened or worse (Mittermeier et al. 2015). Elucidating the epidemiology of significant pathogens such as ranaviruses may facilitate the development of more effective conservation strategies for these imperiled animals.
Eastern box turtles (Terrapene carolina carolina) are experiencing range-wide declines due to both anthropogenic factors and disease (Gray et al. 2009; Lesbarrères et al. 2012; Adamovicz et al. 2018). Mortality associated with disease, especially ranavirus, is reported for captive (De Voe et al. 2004; Johnson et al. 2008), translocated (Farnsworth et al. 2013; Kimble et al. 2015), and free-living box turtles (Adamovicz et al. 2018). Turtle population stability can be threatened by high-mortality diseases like ranavirus due to a combination of life history traits such as delayed sexual maturity and low juvenile survivorship (Heppell 1998). Conservation efforts for the box turtle may be aided by strategies that control and (or) mitigate the effects of ranavirus in natural environments.
Ranaviruses can be transmitted through several routes including direct contact with infected individuals, water, or fomites (Brunner et al. 2015); however, natural transmission dynamics are poorly understood. FV3 DNA has been previously detected in mosquitoes sampled during a ranavirus outbreak in semicaptive eastern box turtles (Kimble et al. 2015). The authors concluded that while FV3 DNA was identified in mosquitoes near the outbreak, their role in transmission during that outbreak and in cases of natural disease is unknown (Kimble et al. 2015). To investigate the potential of mosquitoes to serve as vectors for FV3 under natural conditions with and without a concurrent outbreak, we collected mosquitoes at four different sites in Vermillion County, Illinois, that are regularly surveyed for the presence of FV3 in box turtles. Our specific biological hypothesis was that if mosquitoes contribute substantially to FV3 transmission, mosquitoes would not be detected with FV3 DNA under nonoutbreak conditions, but they would be detected during an outbreak.

Methods

Locations and mosquito trap placement

Mosquitoes were collected for 12 weeks spanning mid-May to early-August, 2014, from four sites in Vermilion County, Illinois. These four sites were: Collison (part of Middle Fork State Fish & Wildlife Area: 40.23941, −87.78548), Kickapoo State Park Location 1 (pre-outbreak; 220 m from outbreak site: 40.14084, −87.74854), Kickapoo State Park Location 2 (35 m from outbreak site: 40.14105, −87.74676), Kennekuk Cove County Park (40.19239, −87.72994), and Forest Glen Nature Preserve (40.01367, −87.57977). These sites are regularly surveyed from April to October for the presence of FV3 in free-ranging eastern box turtles (Archer et al. 2017; Adamovicz et al. 2018).
Mosquitoes were collected using CO2-baited CDC light traps (John W. Hock Company Inc., Gainesville, Florida, USA), which utilize both carbon dioxide from sublimating dry ice and a light attractant to capture host-seeking female mosquitoes (Sudia and Chamberlain 1962). The traps were placed strategically where mosquitoes were likely to occur and encounter eastern box turtles (Archer et al. 2017; Adamovicz et al. 2018) (Fig. 1). These areas were categorized as either: (i) on edge, where wooded area meets more open area, or (ii) near water, especially standing water. Once weekly, CDC light traps were deployed with 2 kg of dry ice in the evening and collected the next morning to coincide with periods of peak mosquito activity. Collected insects were kept in a cooler with dry ice during transport. Environmental data (air temperature, substrate temperature, wind speed, and relative humidity) were collected each time the traps were emptied (Kestrel 3000 Weather Meter, Nielsen-Kellerman, Boothwyn, Pennsylvania, USA; Taylor 9878 Digital Pocket Thermometer, Taylor Precision Products, Oak Brook, Illinois, USA).
Fig. 1.
Fig. 1. Spatial relationships between mosquito traps at Kickapoo State Park (grey circles), frog virus 3 (FV3) negative box turtles (white squares), and FV3 positive box turtles (grey squares) in 2014.

Mosquito sorting

All female mosquitoes were sorted by date, collection site, and genus. Mosquitoes were not sorted for engorgement. Ross and Horsfall’s (1965) “A Synopsis of the Mosquitoes of Illinois (Diptera, Culicidae)” from the state of Illinois Department of Registration and Education, Natural History Survey Division was used to speciate mosquitoes. The species of interest were Uranotaenia sapphirina, Aedes canadensis, Culex territans, Culex erraticus, and Culex pecator due to their reported propensity to take blood meals from herptiles and their roles as infectious disease vectors in multiple taxa (Molaei et al. 2008). The authors have previously observed mosquitoes taking blood meals from eastern box turtles at each of these study sites while performing routine field work.

DNA extraction and qPCR for detection of FV3

Female mosquitoes were pooled by date, site, and species. DNA was extracted from mosquito pools using a phenol/chloroform protocol followed by ethanol precipitation. DNA quantity and purity was evaluated using a spectrophotometer (NanoDrop 1000, Thermo Fisher Scientific, Waltham, Massachusetts, USA).
Quantitative polymerase chain reaction (qPCR) was performed using a validated FV3 assay as previously described (Allender et al. 2013a). All samples, standards, and nontemplate controls were run in triplicate, and FV3 quantities were determined using a standard curve (10-fold dilutions from 10 to 10 000 000 copies/reaction).

Results

Mosquito abundance by site

Two hundred mosquitoes collected from 18 May to 13 August 2014 met our inclusion criteria for species and sex (Table 1). No C. pecator were captured during this study.
Table 1.
Table 1. Total number of mosquitoes by species and location in the state of Illinois including number of pools containing the insects.
 CollisonKickapooKennekukForest Glen
Aedes canadensis246511
Culex erraticus11182414
Culex territans16021
Uranotaenia sapphirina14360
Number of pools2717127
Number of mosquitoes65863316
During the sampling period, an FV3 outbreak occurred in box turtles at Kickapoo State Park (Adamovicz et al. 2018). One mosquito trap was positioned 35 m from the epicenter of this outbreak (Fig. 1). The mosquito trap at Collison was positioned at the edge of a stream where a box turtle FV3 mortality event had occurred the previous year (Adamovicz et al. 2018), though FV3 was not detected at this site during 2014. FV3 was not detected in eastern box turtles at Forest Glen or Kennekuk during the study period (Archer et al. 2017).
DNA was extracted from 63 mosquito pools which included one to 26 individuals per pool (mean = 3.33 mosquitoes/pool, SD = 4.4 mosquitoes/pool) (Tables 1 and 2). DNA concentration ranged from 15.6 to 811.4 ng/μL, with an average of 112.8 ng/μL (SD = 140.5 ng/μL). DNA purity was high, with Absorbance260/Absorbance280 ratios falling between 1.7 and 1.9. FV3 was not detected by qPCR in any sample.
Table 2.
Table 2. Date and species of mosquitoes collected in Kickapoo State Park at the time of frog virus 3 outbreak per Adamovicz et al. (2018).
DateMosquitoes speciesNo. of mosquitoes/pool
14 July 2014Aedes canadensis3
14 July 2014Culex erraticus1
22 July 2014Culex erraticus4
22 July 2014Uranotaenia sapphirina1
26 July 2014Aedes canadensis1
26 July 2014Culex erraticus4
26 July 2014Uranotaenia sapphirina4

Discussion

We set out to detect the presence of ranavirus in native mosquito species in areas of known ranavirus occurrence. Ranavirus transmission occurs through direct or indirect contact with an infected host, contact with environmental material (water, sediment) containing live virus (Allender 2019), or possibly through mosquito vectors (Kimble et al. 2015). Several challenge studies have demonstrated that ranavirus can be transmitted via intramuscular inoculation and have confirmed the systemic distribution of this pathogen (A.J. Johnson et al. 2007). While these characteristics may facilitate vector transmission, we failed to detect FV3 DNA in mosquito species known to feed on herptiles during an active box turtle FV3 outbreak. These findings may be attributable to our methodology or could represent a true absence of FV3 DNA in mosquitoes with implications for ranavirus ecology and epidemiology.
This study utilized CDC CO2-baited light traps, which primarily attract female mosquitoes via the release of carbon dioxide, mimicking the exhalation of a potential blood meal host. These traps attract the greatest diversity of mosquito species, including several species targeted in this study that are not easily collected using other trap designs (Service 1993). Furthermore, they are commonly deployed for arboviral surveillance programs, demonstrating their utility for pathogen detection (Springer et al. 2016). However, these types of traps are biased towards host-seeking mosquitoes that have not previously taken a blood meal (Springer et al. 2016). Engorged female mosquitoes are more likely to contain pathogen DNA and are the preferred targets for blood-borne disease surveillance. Gravid traps, which utilize fetid water to attract engorged females seeking an oviposition site, can be used to increase the likelihood of sampling mosquitoes that have already taken a blood meal (Reiter 1983). However, these were not utilized in the present study due to logistical constraints associated with obtaining and transporting suitably attractive water samples. Mosquitoes were also not sorted based on engorgement in this study, and it is difficult to determine what effect this may have had on our findings. Future studies should seek to maximize the number of engorged female mosquitoes in each pool. Incorporation of a molecular marker for box turtle DNA would also be useful to determine whether one or more members of a pool had fed on the vertebrate host of interest.
Mosquito trapping was performed in known areas of box turtle occurrence based on several years of previous surveys (e.g., Archer et al. 2017; Adamovicz et al. 2018). Within these areas of frequent turtle occurrence, traps were positioned in moist microhabitats to maximize the likelihood of capturing mosquitoes that had fed on FV3-infected herptiles (Adamovicz et al. 2018). However, it is possible that our trapping effort was insufficient to detect FV3-positive mosquitoes. A previous study (Kimble et al. 2015) divided six CDC CO2-baited light traps between two eastern box turtle enclosures. One night of trapping in July resulted in 1986 mosquito captures. Of these, one Aedes sp. pool and one individual Ochlerotatus sp. tested qPCR positive for FV3 DNA. The total number of Aedes sp. captured that evening (211 mosquitoes) exceeded the entire sample size of the present study (200 mosquitoes), potentially indicating that a very large sample size is necessary to detect FV3 DNA in mosquitoes during an outbreak.
Determining an appropriate a priori sample size for xenosurveillance studies is challenging, as it requires knowledge of host and vector abundance, infection prevalence within hosts and vectors, and transmission rates between hosts and vectors, among other things (Ferguson et al. 2014). While box turtle abundance estimates and FV3 prevalence was known during this study (Adamovicz et al. 2018), mosquito abundance estimates are inherently imprecise (Springer et al. 2016) and the transmission rate of FV3 between box turtles and mosquitoes is completely unknown. Rough sample size guidelines suggest that testing 2301 mosquitoes is necessary to detect a “rare” pathogen (0.1% mosquito prevalence), testing 459 mosquitoes is necessary to detect a “common” pathogen (0.5% mosquito prevalence), and testing 152 mosquitoes is necessary to detect a pathogen during an outbreak (1.5% mosquito prevalence), assuming a 90% detection probability (Springer et al. 2016). These guidelines indicate that the sample size utilized in the present study may have been inadequate to detect FV3, as a total of 86 mosquitoes were sampled at the site of the observed FV3 outbreak. Future studies should consider placing a higher number of mosquito traps within study areas to increase the total number of mosquitoes for pathogen testing.
The discrepancy between our findings and those of Kimble et al. (2015) could also be attributable to the difference in total number of turtles present and their density. In the Kimble study, over 300 turtles were divided between two 1000 m2 enclosures. At Kickapoo State Park, the site of the FV3 outbreak, the box turtle population is estimated to include approximately 150 individuals which range over a 500 000 m2 area. The artificial concentration of turtles in the Kimble study may have increased the likelihood of mosquitoes feeding on an FV3-infected host. This, combined with the larger sample size in that study, may have contributed to the detection of FV3 in mosquitoes. It is possible that FV3 detection in mosquitoes is less likely under natural conditions when turtles are present in a significantly lower density, though additional studies are needed to investigate this hypothesis further.
In addition to methodological reasons for our lack of FV3 detection in mosquitoes, biological explanations may also apply. For example, ranavirus has a very rapid course in chelonians (median survival time 17 d in red-eared sliders (Trachemys scripta elegans); Allender et al. 2013b), and a short period of accompanying viremia (14–21 d). This creates a relatively narrow window of time during which a mosquito may ingest FV3 DNA in conjunction with a blood meal. In contrast, Texas tortoises (Gopherus berlandier) challenged with western equine encephalitis virus are viremic for up to 105 d (Bowen et al. 1977) and spotted turtles (Clemmys guttata) remain viremic for over six months following challenge with eastern equine encephalitis virus (Smith and Anderson 1980). This relatively short period of viremia may explain the lack of FV3 detection in this study and indicate that xenosurveillance for FV3 is inefficient. Traditionally used in studies focusing on human viral pathogens (Grubaugh et al. 2015), xenosurveillance is a promising approach for wildlife disease surveillance (Egizi et al. 2018), but future studies are needed to determine its usefulness in detection of FV3 in reptiles. Testing additional herptile ectoparasites, such as ticks and leeches, for FV3 may further elucidate the role of hematophagous vectors in ranavirus transmission.
Given the many limitations and time constraints of the xenosurveillance used in this study, exposing naïve mosquitoes to FV3 infected herptiles in a laboratory setting may provide a more efficient means of assessing the transmission potential of this virus through this insect vector. Understanding the transmission mechanisms of diseases, especially in a natural setting, can guide us in determining methods to protect and preserve overall ecosystem health and wellness.

Conclusions

In this study, we sought to determine the role of mosquitoes as ranavirus vectors in natural settings. We targeted species of mosquitoes that are known to feed on reptiles and amphibians, and performed surveillance in locations with and without concurrent FV3 outbreaks. Ultimately, FV3 was not detected in mosquitoes during the survey period, even though a significant outbreak happened in the same area. This indicates that ranavirus transmission by mosquito vectors must continue to be explored and more scientific studies performed under controlled laboratory conditions to determine the vector potential of mosquito species in this viral infection.

Acknowledgements

The authors thank Page Fredericks for assistance in training for mosquito identification.

References

Adamovicz L, Allender MC, Archer G, Rzadkowska M, Boers K, Phillips C, et al. 2018. Investigation of multiple mortality events in eastern box turtles (Terrapene carolina carolina). PLoS ONE, 13(4): e0195617.
Allender MC. 2019. Chapter 52. Ranaviral disease in reptiles and amphibians. In Fowler’s zoo and wild animal medicine current therapy. Edited by RE Miller, N Lamberski, and PP Calle. W.B. Saunders Company, Philadelphia, Pennsylvania. Vol. 9, pp. 364–370.
Allender MC, Bunick D, and Mitchell MA. 2013a. Development and validation of TaqMan quantitative PCR for detection of frog virus 3-like virus in eastern box turtles (Terrapene carolina carolina). Journal of Virological Methods, 188(1–2): 121–125.
Allender MC, Mitchell MA, Torres T, Sekowska J, and Driskell EA. 2013b. Pathogenicity of frog virus 3-like virus in red-eared slider turtles (Trachemys scripta elegans) at two environmental temperatures. Journal of Comparative Pathology, 149: 356–367.
Archer G, Phillips CA, Adamovicz L, Band M, and Allender MC. 2017. Detection of co-pathogens in free-ranging eastern box turtles (Terrapene carolina carolina) in Illinois and Tennessee. Journal of Zoo and Wildlife Medicine, 48(4): 1127–1134.
Bowen GS. 1977. Prolonged western equine encephalitis viremia in the Texas tortoise (Gopherus berlandieri). The American Journal of Tropical Medicine and Hygiene, 26(1): 171–175.
Brunner JL, Storfer A, Gray MJ, and Hoverman JT. 2015. Ranavirus ecology and evolution: from epidemiology to extinction. In Ranaviruses. Edited by M Gray and V Chinchar. Springer, Cham, Switzerland. pp. 71–104.
De Voe R, Geissler K, Elmore S, Rotstein D, Lewbart G, and Guy J. 2004. Ranavirus-associated morbidity and mortality in a group of captive eastern box turtles (Terrapene carolina carolina). Journal of Zoo and Wildlife Medicine, 35(4): 534–543.
Duffus ALJ, Waltzek TB, Stöhr AC, Allender MC, Gotesman M, Whittington RJ, et al. 2015. Distribution and host range of ranaviruses. In Ranaviruses. Edited by MJ Gray and VG Chinchar. Springer, New York, New York. pp. 9–57.
Egizi A, Martinsen ES, Vuong H, Zimmerman KI, Faraji A, and Fonseca DM. 2018. Using bloodmeal analysis to assess disease risk to wildlife at the new northern limit of a mosquito species. EcoHealth, 15: 543–554.
Farnsworth SD, Seigel RA. 2013. Responses, movements, and survival of relocated box turtles during construction of the intercounty connector highway in Maryland. Transportation Research Record, 2362: 1–8.
Ferguson JM, Langebrake JB, Cannataro VL, Garcia AJ, Hamman EA, Martcheva M, et al. 2014. Optimal sampling strategies for detecting zoonotic disease epidemics. PLoS Computational Biology, 10(6): e1003668.
Gray MJ, Miller DL, and Hoverman JT. 2009. Ecology and pathology of amphibian ranaviruses. Diseases of Aquatic Organisms, 87: 243–266.
Grubaugh ND, Sharma S, Krajacich BJ, Fakoli LS III, Bolay FK, Diclaro JW II, et al. 2015. Xenosurveillance: a novel mosquito-based approach for examining the human-pathogen landscape. PLoS Neglected Tropical Diseases, 9(3): e0003628.
Heppell SS. 1998. Application of life-history and population model analysis to turtle conservation. Copeia, 1998(2): 367–375.
Johnson AJ, Pessier AP, and Jacobson ER. 2007. Experimental transmission and induction of ranaviral disease in western ornate box turtles (Terrapene ornata ornata) and red-eared sliders (Trachemys scripta elegans). Veterinary Pathology, 44(3): 285–297.
Johnson AJ, Pessier AP, Wellehan JF, Childress A, Norton TM, Stedman NL, et al. 2008. Ranavirus infection of free-ranging and captive box turtles and tortoises in the United States. Journal of Wildlife Diseases, 44(4): 851–863.
Johnson PD, Azuolas J, Lavender CJ, Wishart E, Stinear TP, Hayman JA, et al. 2007. Mycobacterium ulcerans in mosquitoes captured during outbreak of Buruli ulcer, southeastern Australia. Emerging Infectious Diseases, 13(11): 1653–1660.
Kimble SJA, Karna AK, Johnson AJ, Hoverman JT, and Williams RN. 2015. Mosquitoes as a potential vector of ranavirus transmission in terrestrial turtles. EcoHealth, 12: 334–338.
Lesbarrères D, Balseiro A, Brunner J, Chinchar VG, Duffus A, Kerby J, et al. 2012. Ranavirus: past, present and future. Biology Letters, 8(4): 481–483.
Mittermeier RA, van Dijk PP, Rhodin AGJ, and Nash SD. 2015. Turtle hotspots: an analysis of the occurrence of tortoises and freshwater turtles in biodiversity hotspots, high-biodiversity wilderness areas, and turtle priority areas. Chelonian Conservation and Biology, 14(1): 2–10
Molaei G, Andreadis TG, Armstrong PM, and Diuk-Wasser M. 2008. Host-feeding patterns of potential mosquito vectors in Connecticut, USA: molecular analysis of bloodmeals from 23 species of Aedes, Anopheles, Culex, Coquillettidia, Psorophora, and Uranotaenia. Journal of Medical Entomology, 45(6): 1143–1151.
Reiter P. 1983. A portable battery-powered trap for collecting gravid Culex mosquitoes. Mosquito News, 43: 496–498.
Ross HH, and Horsfall WR. 1965. A synopsis of the mosquitoes of Illinois (Diptera, Culicidae). Biological Notes No. 52. Available from inhs.illinois.edu/files/4213/3892/7641/mosquitosILkey.pdf
Service MW. 1993. Mosquito ecology: field sampling methods. Elsevier Applied Science, London, UK.
Sim RR, Allender MC, Crawford LK, Wack AN, Murphy KJ, Mankowski JL, et al. 2016. Ranavirus epizootic in captive eastern box turtles (Terrapene carolina carolina) with concurrent herpesvirus and mycoplasma infection: management and monitoring. Journal of Zoo and Wildlife Medicine, 47(1): 256–270.
Smith AL, and Anderson CR. 1980. Susceptibility of two turtle species to eastern equine encephalitis virus. Journal of Wildlife Diseases, 16(4): 615–617.
Springer YP, Hoekman D, Johnson PTJ, Duffy PA, Hufft RA, Barnett DT, et al. 2016. Tick-, mosquito-, and rodent-borne parasite sampling designs for the National Ecological Observatory Network. Ecosphere, 7(5): e01271.
Sudia WD, and Chamberlain RW. 1962. Battery-operated light trap, an improved model. Mosquito News, 22: 126–129.

Information & Authors

Information

Published In

cover image FACETS
FACETS
Volume 5Number 1January 2020
Pages: 812 - 820
Editor: Ellen Ariel

History

Received: 3 April 2020
Accepted: 4 August 2020
Version of record online: 15 October 2020

Notes

This paper is part of a Collection titled “Ranavirus research: 10 years of global collaboration.”

Key Words

  1. mosquitoes
  2. ranavirus
  3. conservation
  4. eastern box turtles
  5. Illinois
  6. FV3

Sections

Subjects

Plain Language Summary

Mosquitoes are unlikely to contribute substantially to FV3 disease dynamics

Authors

Affiliations

Nicole A. Lopez Vargas
Wildlife Epidemiology Laboratory, University of Illinois College of Veterinary Medicine, 2001 South Lincoln Avenue, Urbana, IL 61802, USA
Laura Adamovicz
Wildlife Epidemiology Laboratory, University of Illinois College of Veterinary Medicine, 2001 South Lincoln Avenue, Urbana, IL 61802, USA
Brittany Willeford
Wildlife Epidemiology Laboratory, University of Illinois College of Veterinary Medicine, 2001 South Lincoln Avenue, Urbana, IL 61802, USA
Brian F. Allan
Department of Entomology, University of Illinois, 505 South Goodwin Avenue, Urbana, IL 61801, USA
Matthew C. Allender [email protected]
Wildlife Epidemiology Laboratory, University of Illinois College of Veterinary Medicine, 2001 South Lincoln Avenue, Urbana, IL 61802, USA

Author Contributions

BFA and MCA conceived and designed the study.
BW and MCA performed the experiments/collected the data.
NALV, BW, BFA, and MCA analyzed and interpreted the data.
BFA and MCA contributed resources.
All drafted or revised the manuscript.

Competing Interests

The authors have declared that no competing interests exist.

Metrics & Citations

Metrics

Other Metrics

Citations

Cite As

Export Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

There are no citations for this item

View Options

View options

PDF

View PDF

Get Access

Media

Media

Other

Tables

Share Options

Share

Share the article link

Share on social media