Tag: Antibiotic resistance

Published paper: Mumame

I am happy to share the news that the paper describing out software tool Mumame is now out in its final form! (1) The paper got published today in the journal Metabarcoding and Metagenomics after being available as a preprint (2) since last autumn. This version has not changed a whole lot since the preprint, but it is more polished and better argued (thanks to a great review process). The software is virtually the same, but is not also available via Conda.

In the paper, we describe the Mumame software, which can be used to distinguish between wildtype and mutated sequences in shotgun metagenomic sequencing data and quantify their relative abundances. We further demonstrate the utility of the tool by quantifying antibiotic resistance mutations in several publicly available metagenomic data sets (3-6), and find that the tool is useful but that sequencing depth is a key factor to detect rare mutations. Therefore, much larger numbers of sequences may be required for reliable detection of mutations than is needed for most other applications of shotgun metagenomics. Since the preprint was published, Mumame has also found use in our recently published paper on selection for antibiotic resistance in a Croatian macrolide production wastewater treatment plant, unfortunately with inconclusive results (7). Mumame is freely available here.

I again want to stress the fantastic work that Shruthi Magesh did last year as a summer student at WID in the evaluation of this tool. As I have pointed out earlier, I did write the code for the software (with a lot of input from Viktor Jonsson), but Shruthi did the software testing and evaluations. Thanks and congratulations Shruthi, and good luck in pursuing your PhD program!

References

  1. Magesh S, Jonsson V, Bengtsson-Palme JMumame: A software tool for quantifying gene-specific point-mutations in shotgun metagenomic data. Metabarcoding and Metagenomics, 3: 59–67 (2019). doi: 10.3897/mbmg.3.36236
  2. Magesh S, Jonsson V, Bengtsson-Palme JQuantifying point-mutations in metagenomic data. bioRxiv, 438572 (2018). doi: 10.1101/438572
  3. Bengtsson-Palme J, Boulund F, Fick J, Kristiansson E, Larsson DGJ: Shotgun metagenomics reveals a wide array of antibiotic resistance genes and mobile elements in a polluted lake in India. Frontiers in Microbiology, 5, 648 (2014). doi: 10.3389/fmicb.2014.00648
  4. Lundström S, Östman M, Bengtsson-Palme J, Rutgersson C, Thoudal M, Sircar T, Blanck H, Eriksson KM, Tysklind M, Flach C-F, Larsson DGJ: Minimal selective concentrations of tetracycline in complex aquatic bacterial biofilms. Science of the Total Environment, 553, 587–595 (2016). doi: 10.1016/j.scitotenv.2016.02.103
  5. Pal C, Bengtsson-Palme J, Kristiansson E, Larsson DGJ: The structure and diversity of human, animal and environmental resistomes. Microbiome, 4, 54 (2016). doi: 10.1186/s40168-016-0199-5
  6. Kraupner N, Ebmeyer S, Bengtsson-Palme J, Fick J, Kristiansson E, Flach C-F, Larsson DGJ: Selective concentration for ciprofloxacin in Escherichia coli grown in complex aquatic bacterial biofilms. Environment International, 116, 255–268 (2018). doi: 10.1016/j.envint.2018.04.029
  7. Bengtsson-Palme J, Milakovic M, Švecová H, Ganjto M, Jonsson V, Grabic R, Udiković Kolić N: Pharmaceutical wastewater treatment plant enriches resistance genes and alter the structure of microbial communities. Water Research, 162, 437-445 (2019). doi: 10.1016/j.watres.2019.06.073

Thank you Alice!

This week marked the departure of our summer internship student Alice Zublena, who is now heading back to France to finish her masters program. Alice has been working on establishing effect concentrations for beta-lactam antibiotics for different bacteria, and has generated a very exciting and useful data set for our work in the coming years. I am tremendously happy that I have got to work with Alice this summer and very thankful for having the opportunity to supervise such a talented student. Thanks for your great work this summer Alice and good luck with everything you pursue in the future! 

Published paper: Increased antibiotic resistance in Croatian pharmaceutical wastewater treatment plant

I celebrate the fourth of July with the coincidental publishing of my most recent paper, in collaboration with the lab of Nikolina Udikovic-Kolic. The study used shotgun metagenomics to investigate the taxonomic structure and resistance gene composition of sludge communities in a treatment plant in Croatia receiving wastewater from production of the macrolide antibiotic azithromycin (1). We compared the levels of antibiotic resistance genes in sludge from this treatment plant and municipal sludge from a sewage treatment plant in Zagreb, and found that the total abundance of resistance genes was three times higher in sludge from the treatment plant receiving wastewater from pharmaceutical production. To our great surprise, this was not true for macrolide resistance genes, however. Instead, those genes had overall slightly lower abundances in the industrial sludge. At the same time, the genes that are associated with mobile genetic elements (such as integrons) had higher abundances in the industrial sludge.

This leads us to think that at high concentrations of antibiotics (such as in the industrial wastewater treatment plant), selection may favor taxonomic shifts towards intrinsically resistant species or strains harboring chromosomal resistance mutations rather than acquisition of mobile resistance genes. Unfortunately, the results regarding resistance mutation – obtained using our recent software tool Mumame (2) – were uninformative due to low number of reads mapping to the resistance regions of the 23S rRNA target gene for azithromycin.

Often, the problem of environmental pollution with pharmaceuticals is perceived as primarily being a concern in countries with poor pollution control, since price pressure has led to outsourcing of global antibiotics production to locations with lax environmental regulation (3). If this was the case, there would be much less incentive for improving legislation regarding emissions from pharmaceutical manufacturing at the EU level, as this would not move the needle in a significant way. However, the results of the paper (and other work by Nikolina’s group (4,5)) underscore the need for regulatory action also within Europe to avoid release of antibiotics into the environment.

References

  1. Bengtsson-Palme J, Milakovic M, Švecová H, Ganjto M, Jonsson V, Grabic R, Udiković Kolić N: Pharmaceutical wastewater treatment plant enriches resistance genes and alter the structure of microbial communities. Water Research, accepted manuscript (2019). doi: 10.1016/j.watres.2019.06.073
  2. Magesh S, Jonsson V, Bengtsson-Palme JQuantifying point-mutations in metagenomic data. bioRxiv, 438572 (2018). doi: 10.1101/438572
  3. Bengtsson-Palme J, Gunnarsson L, Larsson DGJ: Can branding and price of pharmaceuticals guide informed choices towards improved pollution control during manufacturing? Journal of Cleaner Production, 171, 137–146 (2018). doi: 10.1016/j.jclepro.2017.09.247
  4. Bielen A, Šimatović A, Kosić-Vukšić J, Senta I, Ahel M, Babić S, Jurina T, González-Plaza JJ, Milaković M, Udiković-Kolić N: Negative environmental impacts of antibiotic-contaminated effluents from pharmaceutical industries. Water Research, 126, 79–87 (2017). doi: 10.1016/j.watres.2017.09.019
  5. González-Plaza JJ, Šimatović A, Milaković M, Bielen A, Wichmann F, Udikovic-Kolic N: Functional repertoire of antibiotic resistance genes in antibiotic manufacturing effluents and receiving freshwater sediments. Frontiers in Microbiology, 8, 2675 (2017). doi: 10.3389/fmicb.2017.02675


Published paper: NGS and antibiotic resistance

AMR Control just released (some of) the articles of their 2019-20 issue, and among the papers hot of the press is one that I have co-authored with Etienne Ruppé, Yannick Charretier and Jacques Schrenzel on how next-generation sequencing can be used to address antibiotic resistance problems (1).

The paper contains a brief overview of next-generation sequencing platforms and tools, the resources that can be used to detect and quantify resistance from sequencing data, and descriptions of applications in clinical genomics, clinical/human metagenomics as well as in environmental settings (the latter being the part where I contributed the most). Compared to much of the writing on antibiotic resistance and sequencing applications, I think this paper is pretty easily accessible to a general audience.

I first met Etienne on the JRC workshops for how next-generation sequencing could be implemented in the EU’s Coordinated Action Plan against Antimicrobial Resistance (2,3), and it seems quite fitting that we now ended up writing a paper on such implementations together.

  1. Ruppé E, Bengtsson-Palme J, Charretier Y, Schrenzel J: How next-generation sequencing can address the antimicrobial resistance challenge. AMR Control, 2019-20, 60-65 (2019). [Paper link]
  2. Angers A, Petrillo P, Patak, A, Querci M, Van den Eede G: The Role and Implementation of Next-Generation Sequencing Technologies in the Coordinated Action Plan against Antimicrobial Resistance. JRC Conference and Workshop Report, EUR 28619 (2017). doi: 10.2760/745099 [Link]
  3. Angers-Loustau A, Petrillo M, Bengtsson-Palme J, Berendonk T, Blais B, Chan KG, Coque TM, Hammer P, Heß S, Kagkli DM, Krumbiegel C, Lanza VF, Madec J-Y, Naas T, O’Grady J, Paracchini V, Rossen JWA, Ruppé E, Vamathevan J, Venturi V, Van den Eede G: The challenges of designing a benchmark strategy for bioinformatics pipelines in the identification of antimicrobial resistance determinants using next generation sequencing technologies. F1000Research, 7, 459 (2018). doi: 10.12688/f1000research.14509.2 [Paper link]

The Lennart Sparell Prize

I am happy to announce that Cancer- och Allergifonden [the Cancer and Allergy Foundation] have awarded me with the first Lennart Sparell prize. The prize was instated in memory of the foundations founder – Lennart Sparell, who passed away last year – and is awarded to researchers (or other persons) who have thought outside-of-the-box or challenged the current paradigms. A particular emphasis is given to research on environmental pollutants that affect human health through food or environmental exposure.

Naturally, I am honored to be the recipient of this prize. The award was motivated by the research I have done on the role of ecological and evolutionary processes in the external environment in driving antibiotic resistance development, and how that can have consequences for human health. Particularly, I am happy that the research that I, Joakim Larsson, Erik Kristiansson and a few others on the role of environmental processes in the development of antibiotic resistance and the recruitment of novel resistance genes are given attention. This view, which perhaps do not challenge the paradigm but at the very least points to an alternative risk scenario, has often been neglected when environmental antibiotic resistance has been discussed.

The prize will be awarded on a ceremony on June 10 in Stockholm, but I would already now take the opportunity to thank everyone who has been involved in the research being recognized, particularly Joakim Larsson and Erik Kristiansson – this award is to a very very large extent to your merit.

Published paper: benchmarking resistance gene identification

Since F1000Research uses a somewhat different publication scheme than most journals, I still haven’t understood if this paper is formally published after peer review, but I start to assume it is. There have been very little changes since the last version, so hence I will be lazy and basically repost what I wrote in April when the first version (the “preprint”) was posted online. The paper (1) is the result of a workshop arranged by the JRC in Italy in 2017. It describes various challenges arising from the process of designing a benchmark strategy for bioinformatics pipelines in the identification of antimicrobial resistance genes in next generation sequencing data.

The paper discusses issues about the benchmarking datasets used, testing samples, evaluation criteria for the performance of different tools, and how the benchmarking dataset should be created and distributed. Specially, we address the following questions:

  • How should a benchmark strategy handle the current and expanding universe of NGS platforms?
  • What should be the quality profile (in terms of read length, error rate, etc.) of in silico reference materials?
  • Should different sets of reference materials be produced for each platform? In that case, how to ensure no bias is introduced in the process?
  • Should in silico reference material be composed of the output of real experiments, or simulated read sets? If a combination is used, what is the optimal ratio?
  • How is it possible to ensure that the simulated output has been simulated “correctly”?
  • For real experiment datasets, how to avoid the presence of sensitive information?
  • Regarding the quality metrics in the benchmark datasets (e.g. error rate, read quality), should these values be fixed for all datasets, or fall within specific ranges? How wide can/should these ranges be?
  • How should the benchmark manage the different mechanisms by which bacteria acquire resistance?
  • What is the set of resistance genes/mechanisms that need to be included in the benchmark? How should this set be agreed upon?
  • Should datasets representing different sample types (e.g. isolated clones, environmental samples) be included in the same benchmark?
  • Is a correct representation of different bacterial species (host genomes) important?
  • How can the “true” value of the samples, against which the pipelines will be evaluated, be guaranteed?
  • What is needed to demonstrate that the original sample has been correctly characterised, in case real experiments are used?
  • How should the target performance thresholds (e.g. specificity, sensitivity, accuracy) for the benchmark suite be set?
  • What is the impact of these performance thresholds on the required size of the sample set?
  • How can the benchmark stay relevant when new resistance mechanisms are regularly characterized?
  • How is the continued quality of the benchmark dataset ensured?
  • Who should generate the benchmark resource?
  • How can the benchmark resource be efficiently shared?

Of course, we have not answered all these questions, but I think we have come down to a decent description of the problems, which we see as an important foundation for solving these issues and implementing the benchmarking standard. Some of these issues were tackled in our review paper from last year on using metagenomics to study resistance genes in microbial communities (2). The paper also somewhat connects to the database curation paper we published in 2016 (3), although this time the strategies deal with the testing datasets rather than the actual databases. The paper is the first outcome of the workshop arranged by the JRC on “Next-generation sequencing technologies and antimicrobial resistance” held October 4-5 2017 in Ispra, Italy. You can find the paper here (it’s open access).

On another note, the new paper describing the UNITE database (4) has now got a formal issue assigned to it, as has the paper on tandem repeat barcoding in fungi published in Molecular Ecology Resources last year (5).

References and notes

  1. Angers-Loustau A, Petrillo M, Bengtsson-Palme J, Berendonk T, Blais B, Chan KG, Coque TM, Hammer P, Heß S, Kagkli DM, Krumbiegel C, Lanza VF, Madec J-Y, Naas T, O’Grady J, Paracchini V, Rossen JWA, Ruppé E, Vamathevan J, Venturi V, Van den Eede G: The challenges of designing a benchmark strategy for bioinformatics pipelines in the identification of antimicrobial resistance determinants using next generation sequencing technologies. F1000Research, 7, 459 (2018). doi: 10.12688/f1000research.14509.1
  2. Bengtsson-Palme J, Larsson DGJ, Kristiansson E: Using metagenomics to investigate human and environmental resistomes. Journal of Antimicrobial Chemotherapy, 72, 2690–2703 (2017). doi: 10.1093/jac/dkx199
  3. Bengtsson-Palme J, Boulund F, Edström R, Feizi A, Johnning A, Jonsson VA, Karlsson FH, Pal C, Pereira MB, Rehammar A, Sánchez J, Sanli K, Thorell K: Strategies to improve usability and preserve accuracy in biological sequence databases. Proteomics, 16, 18, 2454–2460 (2016). doi: 10.1002/pmic.201600034
  4. Nilsson RH, Larsson K-H, Taylor AFS, Bengtsson-Palme J, Jeppesen TS, Schigel D, Kennedy P, Picard K, Glöckner FO, Tedersoo L, Saar I, Kõljalg U, Abarenkov K: The UNITE database for molecular identification of fungi: handling dark taxa and parallel taxonomic classifications. Nucleic Acids Research, 47, D1, D259–D264 (2019). doi: 10.1093/nar/gky1022
  5. Wurzbacher C, Larsson E, Bengtsson-Palme J, Van den Wyngaert S, Svantesson S, Kristiansson E, Kagami M, Nilsson RH: Introducing ribosomal tandem repeat barcoding for fungi. Molecular Ecology Resources, 19, 1, 118–127 (2019). doi: 10.1111/1755-0998.12944

Published paper: Diarrhea-causing bacteria in the Choqueyapu River in Bolivia

My first original paper of the year was just published in PLoS ONE. This is a collaboration with Åsa Sjöling’s group at the Karolinska Institute and the Universidad Mayor de San Andrés in Bolivia, and the project has been largely run by Jessica Guzman-Otazo.

Poor drinking water quality is a major cause of diarrhea, especially in the absence of well-working sewage treatment systems. In the study, we investigate the numbers of bacteria causing diarrhea (or actually, marker genes for those bacteria) in water, soil and vegetable samples from the Choqueyapu River area in La Paz – Bolivia’s third largest city (1). The river receives sewage and wastewater from industries and hospitals while flowing through La Paz. We found that levels of ETEC – a bacterium that causes severe diarrhea – were much higher in the city than upstream of it, including at a site where the river water is used for irrigation of crops.

In addition, several multi-resistant bacteria could be isolated from the samples, of which many were emerging, globally spreading, multi-resistant variants. The results of the study indicate that there is a real risk for spreading of diarrheal diseases by using the contaminated water for drinking and irrigation (2,3). Furthermore, the identification of multi-resistant bacteria that can cause human diseases show that water contamination is an important route through which antibiotic resistance can be transferred from the environment back to humans (4).

The study was published in PLoS ONE and can be found here.

References

  1. Guzman-Otazo J, Gonzales-Siles L, Poma V, Bengtsson-Palme J, Thorell K, Flach C-F, Iñiguez V, Sjöling Å: Diarrheal bacterial pathogens and multi-resistant enterobacteria in the Choqueyapu River in La Paz, Bolivia. PLoS ONE, 14, 1, e0210735 (2019). doi: 10.1371/journal.pone.0210735
  2. Graham DW, Collignon P, Davies J, Larsson DGJ, Snape J: Underappreciated Role of Regionally Poor Water Quality on Globally Increasing Antibiotic Resistance. Environ Sci Technol 141001154428000 (2014). doi: 10.1021/es504206x
  3. Bengtsson-Palme J: Antibiotic resistance in the food supply chain: Where can sequencing and metagenomics aid risk assessment? Current Opinion in Food Science, 14, 66–71 (2017). doi: 10.1016/j.cofs.2017.01.010
  4. Bengtsson-Palme J, Kristiansson E, Larsson DGJ: Environmental factors influencing the development and spread of antibiotic resistance. FEMS Microbiology Reviews, 42, 1, 68–80 (2018). doi: 10.1093/femsre/fux053

Published book chapter: Resistance Risks in the Environment

Time flies, and my first 2019 publication (wait what?) is now out! It’s a chapter in the book “Management of Emerging Public Health Issues and Risks: Multidisciplinary Approaches to the Changing Environment(1), edited by Benoit Roig, Karine Weiss and Véronique Thireau. I have to confess to not having read the other chapters in the book yet, but I think the subject is exciting and hope for a lot of good reading over Christmas here!

My chapter deals with assessment and management of risks associated with antibiotic resistance in the environment (2), and particularly I make an attempt at clarifying the different types of risks and how to deal with them. In short, I partition resistance risks into two categories: dissemination risks and risks for acquisition of new types of resistance (see also 3). While the former category largely encompasses quantifiable risks, the latter is to a large extent impossible (or at least extremely hard) to quantify with current means. This means that we need to be a bit more creative in assessing, prioritizing and managing these risks. Some lessons can be learnt from other fields dealing with very uncertain (and rare) risks, such as asteroid impact assessment, nuclear energy accidents and ecosystem destabilization (4,5). Incorporating elements from such risk management schemes will be necessary to understand and delay emergence of novel resistance in the future.

All these aspects are further discussed in the book chapter (2), which I encourage everyone working with environmental antibiotic resistance risks to read!

References

  1. Roig B, Weiss K, Thoreau V (Eds.) Management of Emerging Public Health Issues and Risks: Multidisciplinary Approaches to the Changing Environment. Academic Press/Elsevier, UK (2019). doi: 10.1016/C2016-0-00995-6
  2. Bengtsson-Palme J: Assessment and management of risks associated with antibiotic resistance in the environment. In: Roig B, Weiss K, Thoreau V (Eds.) Management of Emerging Public Health Issues and Risks: Multidisciplinary Approaches to the Changing Environment, 243–263. Elsevier, UK (2019). doi: 10.1016/B978-0-12-813290-6.00010-X
  3. Bengtsson-Palme J, Kristiansson E, Larsson DGJ: Environmental factors influencing the development and spread of antibiotic resistance. FEMS Microbiology Reviews, 42, 1, 68–80 (2018). doi: 10.1093/femsre/fux053
  4. WBGU GACOGC: World in Transition: Strategies for Managing Global Environmental Risks. Springer
    Berlin Heidelberg, Berlin, Heidelberg (2000).
  5. Government Office for Science: Blackett Review of High Impact Low Probability Events. Department for
    Business, Innovation and Skills, London (2011).

Mumame – Quantifying mutations in metagenomes

Let me get straight to something somewhat besides the point here: summer students can achieve amazing things! One such student I had the pleasure to work with this summer is Shruthi Magesh, and a preprint based on work she did with me at the Wisconsin Institute for Discovery this summer just got published on bioRxiv (1). The preprint describes a software tool called Mumame, which uses database information on mutations in DNA or protein sequences to search metagenomic datasets and quantifies the relative proportion of resistance mutations over wild type sequences.

In the preprint (1), we first of all show that Mumame works on amplicon data where we already knew the true outcome (2). Second, we show that we can detect differences in mutation frequencies in controlled experiments (2,3). Lastly, we use the tool to gain some further information about resistance patterns in sediments from polluted environments in India (4,5). Together these analyses show that one of the most central aspects for Mumame to be able to find mutations is having a very high number of sequenced reads in all libraries (preferably more than 50 million per library), because these mutations are generally rare – even in polluted environments and microcosms exposed to antibiotics. We expect Mumame to be a useful addition to metagenomic studies of e.g. antibiotic resistance, and to increase the detail by which metagenomes can be screened for phenotypically important differences.

While I did write the code for the software (with a lot of input from Viktor Jonsson, who also is a coauthor on the preprint, on the statistical analysis), Shruthi did the software testing and evaluations, and the paper would not have been possible hadn’t she wanted a bioinformatic summer project related to metagenomics, aside from her laboratory work. The resulting preprint is available from bioRxiv and the Mumame software is freely available from this site.

References

  1. Magesh S, Jonsson V, Bengtsson-Palme JQuantifying point-mutations in metagenomic data. bioRxiv, 438572 (2018). doi: 10.1101/438572 [Link]
  2. Kraupner N, Ebmeyer S, Bengtsson-Palme J, Fick J, Kristiansson E, Flach C-F, Larsson DGJ: Selective concentration for ciprofloxacin in Escherichia coli grown in complex aquatic bacterial biofilms. Environment International, 116, 255–268 (2018). doi: 10.1016/j.envint.2018.04.029 [Paper link]
  3. Lundström S, Östman M, Bengtsson-Palme J, Rutgersson C, Thoudal M, Sircar T, Blanck H, Eriksson KM, Tysklind M, Flach C-F, Larsson DGJ: Minimal selective concentrations of tetracycline in complex aquatic bacterial biofilms. Science of the Total Environment, 553, 587–595 (2016). doi: 10.1016/j.scitotenv.2016.02.103 [Paper link]
  4. Bengtsson-Palme J, Boulund F, Fick J, Kristiansson E, Larsson DGJ: Shotgun metagenomics reveals a wide array of antibiotic resistance genes and mobile elements in a polluted lake in India. Frontiers in Microbiology, 5, 648 (2014). doi: 10.3389/fmicb.2014.00648 [Paper link]
  5. Kristiansson E, Fick J, Janzon A, Grabic R, Rutgersson C, Weijdegård B, Söderström H, Larsson DGJ: Pyrosequencing of antibiotic-contaminated river sediments reveals high levels of resistance and gene transfer elements. PLoS ONE, Volume 6, e17038 (2011). doi:10.1371/journal.pone.0017038.

Published paper: Breast milk and the infant gut resistome

This week, a paper by my former roommate Katariina Pärnänen was published by Nature Communications. In the paper (1), we use shotgun metagenomics to show that infants carry more resistant bacteria in their gut than adults do, irrespective of whether they themselves have been treated with antibiotics or not. We also found that the antibiotic resistance gene and mobile genetic element profiles of infant feces are more similar to those of their own mothers than to those of unrelated mothers. This is suggestive of a pathway of transmission of resistance genes from the mothers, and importantly we find that the mobile genetic elements in breastmilk are shared with those of the infant feces, despite vast differences in their microbiota composition. Finally, we find that termination of breastfeeding and intrapartum antibiotic prophylaxis of mothers are associated with higher abundances of specific ARGs in the infant gut. Our results suggest that infants inherit the legacy of past antibiotic consumption of their mothers via transmission of genes, but that the taxonomic composition of the microbiota still strongly dictates the overall load of resistance genes.

I am not going to dwell in to details of the study here, but I instead encourage you to read the paper (hey, it’s open access!) or the excellent popular summary that Katariina has already written. Finally, I want to emphasize the great work Katariina has put into this (I would know, since I shared room with her) and congratulate her on her own little infant!

Reference

  1. Pärnänen K, Karkman A, Hultman J, Lyra C, Bengtsson-Palme J, Larsson DGJ, Rautava S, Isolauri E, Salminen S, Kumar H, Satokari R, Virta M: Maternal gut and breast milk microbiota affect infant gut antibiotic resistome and mobile genetic elements. Nature Communications, 9, 3891 (2018). doi: 10.1038/s41467-018-06393-w [Paper link]