[REVIEW]: DetecTree: Tree detection from aerial imagery in Python

[whedon]

Submitting author: @martibosch (Martí Bosch)
Repository: https://github.com/martibosch/detectree
Version: v0.3.1
Editor: @kthyng
Reviewer: @JeffWalton-PSC , @rmsare
Archive: 10.5281/zenodo.3908338

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/10979c12afde3ba59bcdbc7fbbb6ed2d"><img src="https://joss.theoj.org/papers/10979c12afde3ba59bcdbc7fbbb6ed2d/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/10979c12afde3ba59bcdbc7fbbb6ed2d/status.svg)](https://joss.theoj.org/papers/10979c12afde3ba59bcdbc7fbbb6ed2d)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@JeffWalton-PSC & @rmsare, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @kthyng know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @JeffWalton-PSC

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@martibosch) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @rmsare

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@martibosch) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @JeffWalton-PSC , @rmsare it looks like you're currently assigned to review this paper 🎉.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Reference check summary:

OK DOIs

- 10.1007/3-540-59119-2_166 is OK
- 10.1023/A:1011139631724 is OK
- 10.1109/TPAMI.2004.60 is OK
- 10.1145/1653771.1653792 is OK
- 10.1109/MCSE.2011.37 is OK
- 10.7717/peerj.453 is OK
- 10.25080/Majora-7b98e3ed-013 is OK
- 10.25080/Majora-4af1f417-011 is OK
- 10.1038/s41592-019-0686-2 is OK

MISSING DOIs

- https://doi.org/10.25080/majora-92bf1922-00a may be missing for title: Data structures for statistical computing in python

INVALID DOIs

- 10.5555/1953048.2078195 is INVALID

[whedon]

👉 Check article proof 📄 👈

[martibosch]

Regarding the DOIs: I have added the missing DOI for pandas as suggested by @whedon. I have also found that the invalid DOI (of scikit-learn) actually corresponds to a record in the digital ACM library rather than a DOI, nevertheless I could not find the DOI of the actual journal article. Shall I instead add the zenodo DOI for the library, i.e., 10.5281/zenodo.3696718?

[martibosch]

I have fixed the DOIs as suggested above in 0128e18

[martibosch]

@whedon generate pdf

[whedon]

👉 Check article proof 📄 👈

[kthyng]

Hi everyone! We are officially starting JOSS back up today. We know that everyone has different circumstances with work and home life right now, so we want to figure out a way forward that will work for everyone.

@JeffWalton-PSC and @rmsare — are you both still able to review this submission? If so, what timelines would work for you?

[JeffWalton-PSC]

Kristen, I will be able to get to this over the next 2-3 weeks. Does that work? From: Kristen Thyng [mailto:[email protected]] Sent: Wednesday, 20 May, 2020 13:17 To: openjournals/joss-reviews <[email protected]> Cc: Jeff Walton <[email protected]>; Mention <[email protected]> Subject: Re: [openjournals/joss-reviews] [REVIEW]: DetecTree: Tree detection from aerial imagery in Python (#2172) Hi everyone! We are officially starting JOSS back up today. We know that everyone has different circumstances with work and home life right now, so we want to figure out a way forward that will work for everyone. @JeffWalton-PSC<https://github.com/JeffWalton-PSC> and @rmsare<https://github.com/rmsare> — are you both still able to review this submission? If so, what timelines would work for you? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#2172 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AI5VWGZ3VSRQG3J33KH5Q3DRSQGARANCNFSM4MDL4QJQ>.

[kthyng]

Kristen, I will be able to get to this over the next 2-3 weeks. Does that work?

Absolutely! Thanks!

[rmsare]

@kthyng Yes, I will be able to complete this review within 3 weeks.

[kthyng]

@rmsare excellent!

[rmsare]

@martibosch @kthyng Thank you for your patience with my review! I enjoyed playing around with detectree.

This is a nice package that solves a remote sensing problem - tree detection in aerial imagery. It re-implements a published method in Python and this work is appropriately referenced. The API is clear and well documented and it uses the modern Python data science stack. The functionality seems like it will be useful for spatial, social science, and environmental science research, or even commercial applications in real estate or insurance.

My main suggestions are to provide more concrete examples of use in the paper and consider reorganizing the text so readers can get a high-level overview of detectree’s research applications. Adding a simple example to the docs would lower the barrier to entry for potential users.

Recommendation: Accept after minor revisions

Review suggestions:

• Improve statement of need and other small edits (#3)
• Add an example use case notebook (#4)
• Discuss related work (#5)

Full disclosure: The author reviewed a submission I made to JOSS last year, but I don't know him personally and we don't share any collaborators.

[JeffWalton-PSC]

@kthyng @martibosch Please accept my apologies for taking so long on this review. It is the first JOSS review I have been involved with. I have learned a great deal. Thank you for your patience.

DetecTree is a nice piece of software. I ran through all of the example notebooks in the detectree-example repository. I have made some comments in the detectree-example issue tracker. It took me quite a while to get all of the dependencies installed and working. The LAS-tools required some effort. As @rmsare has suggested, a simple end-to-end example on the main DetecTree repository would likely be helpful. The use of LIDAR as a method to collect training data is impressive, but adds a high level of complexity for a user who is just checking out the DetecTree software.

Recommendation: Accept after minor edits to the paper.

Minor edit suggestions:

• in Step 1 "involve manual digitizing"
• in Step 2 "exploited"
• "Another approach by Tianyang et al. [@tianyang2018single] employs"
• "The computation ... makes use"

@kthyng Thank you.

[martibosch]

Hello all,

first of all thank you @rmsare and @JeffWalton-PSC for your reviews.

I have addressed your issues in the detectree and detectree-example repositories. An end-to-end example is now available at https://github.com/martibosch/detectree-example/blob/master/notebooks/aussersihl-canopy.ipynb On the other hand, I have corrected the typos pointed out by @JeffWalton-PSC in 4c34a3.

Let me know if you have any further suggestions. Best,
Martí

[rmsare]

Thanks! These changes address my review.

I have one last minor suggestion about the README. The statement of need is a bit long and hard to read. It could be split into two sentences. So

The target audience is researchers and practitioners in GIS that are interested in two-dimensional aspects of trees, such as their proportional abundance and spatial distribution throughout a region of study and thereafter assess important aspects of urban planning such as the provision of urban ecosystem services.

might become something like

The target audience is researchers and practitioners in GIS that are interested in two-dimensional aspects of trees, such as their proportional abundance and spatial distribution throughout a region of study. These measurements can be used to assess important aspects of urban planning such as the provision of urban ecosystem services.

[martibosch]

Indeed splitting the statement makes it more readable. I have amended it (both in the README and in the paper) in 003f76

[kthyng]

Ok all, it looks to me like @martibosch has addressed concerns from @rmsare.

@JeffWalton-PSC can you confirm if your review is complete and you recommend publication?

[JeffWalton-PSC]

@martibosch Thank you for making the changes. DetecTree is a nice piece of work.

@kthyng my review is complete. I recommend for publication.

[kthyng]

@whedon generate pdf

[whedon]

👉 Check article proof 📄 👈

[kthyng]

@whedon check references

[martibosch]

@whedon generate pdf

[whedon]

👉 Check article proof 📄 👈

[martibosch]

Hello @kthyng, I have merged your PR, thank you for your corrections. Regarding your three comments:

• I believe that I was taught that might and can are synonyms in this context, but I am no native English speaker, so we can use can as set in your PR.

• I suppose that you refer to the part of:

  New York achieve a pixel classification accuracy of 91.7%, whereas the example applications of DetecTree in Zurich and Lausanne achieve accuracies of 85.98% and 91.75% respectively

  In such case, the accuracy scores are taken from GitHub repositories (i.e., detectree-example and lausanne-tree-canopy for which I do not have any proper citation. I hope to have a citation for the Lausanne tree canopy soon, since such a canopy map was produced in the context of a study of the urban heat islands which has been submitted to GMD. Therefore, should I leave it like that, with no citation? Alternatively, I can archive both repositories to Zenodo so that we can cite them in this manuscript.

• All references should be properly formatted as of 1516fa6

[kthyng]

Hi @martibosch!

• "might" has a bit of uncertainty to it usually, so "can" is better here I think. Is this change in there?
• Ah, I see my misunderstanding. I read it as two people's last names instead of two places! Sorry about that. I haven't been to Switzerland before, sadly. You can choose what makes the most sense here.

Please let me know when this is all accounted for, thanks.

[martibosch]

Hello @kthyng ,

• the "might" was changed to "can" in your PR, so since I merged it, the current version reads

  Furthermore, the functionalities of DetecTree can be used through its Python API as well as through its command-line interface (CLI)

• in the absence of a proper academic article to cite, I believe that we can leave the sentence as it is.

I hope that this addresses the two remarks.

[kthyng]

@whedon generate pdf

[whedon]

👉 Check article proof 📄 👈

[kthyng]

I made a change in my PR for one "might" but there was another under Step 4 that I hadn't been sure about at the time. Can you change that one?

Next, can you verify the version, and then submit your code to an archive like Zenodo and report the doi here? Make sure that the title and author list on the Zenodo archive match those of your JOSS paper (you may have to alter the Zenodo metadata to do this).

[martibosch]

I have checked for all the occurences of "might" in the manuscript and replaced the one at Step 4 in 3ab057.

I have created a released named joss-paper for the version v0.3.1 of DetecTree and archived it in Zenodo at 10.5281/zenodo.3908338. Finally, I edited the title and author list from the Zenodo archive so that they now match those of the JOSS paper.

[kthyng]

@whedon set v0.3.1 as version

[whedon]

OK. v0.3.1 is the version.

[kthyng]

@whedon set 10.5281/zenodo.3908338 as archive

[whedon]

OK. 10.5281/zenodo.3908338 is the archive.

[kthyng]

@whedon accept

[whedon]

Attempting dry run of processing paper acceptance...

[whedon]

Reference check summary:

OK DOIs

- 10.1007/3-540-59119-2_166 is OK
- 10.1023/A:1011139631724 is OK
- 10.1109/TPAMI.2004.60 is OK
- 10.1145/1653771.1653792 is OK
- 10.25080/majora-92bf1922-00a is OK
- 10.5281/zenodo.3696718 is OK
- 10.1109/MCSE.2011.37 is OK
- 10.7717/peerj.453 is OK
- 10.25080/Majora-7b98e3ed-013 is OK
- 10.25080/Majora-4af1f417-011 is OK
- 10.1117/12.2529442 is OK
- 10.1038/s41592-019-0686-2 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

👋 @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof 👉 openjournals/joss-papers#1516

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#1516, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

[kthyng]

@whedon accept deposit=true

[whedon]

Doing it live! Attempting automated processing of paper acceptance...

[whedon]

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

[whedon]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.02172 joss-papers#1517
1. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.02172
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

[kthyng]

Congrats to @martibosch on your new paper!! Thanks to reviewers @JeffWalton-PSC and @rmsare — we are so grateful for your time and expertise!

[whedon]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.02172/status.svg)](https://doi.org/10.21105/joss.02172)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.02172">
  <img src="https://joss.theoj.org/papers/10.21105/joss.02172/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.02172/status.svg
   :target: https://doi.org/10.21105/joss.02172

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss