Skip to content

Option --scihub-mirror added #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ferru97 merged 1 commit into from
Mar 23, 2021
Merged

Option --scihub-mirror added #21

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion PyPaperBot/Downloader.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,10 @@ def saveFile(file_name,content, paper,dwn_source):
paper.downloadedFrom = dwn_source


def downloadPapers(papers, dwnl_dir, num_limit):
def downloadPapers(papers, dwnl_dir, num_limit, SciHub_URL=None):
if SciHub_URL:
NetInfo.SciHub_URL = SciHub_URL

if NetInfo.SciHub_URL==None:
setSciHubUrl()

Expand Down
7 changes: 4 additions & 3 deletions PyPaperBot/__main__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from .Crossref import getPapersInfoFromDOIs


def start(query, scholar_pages, dwn_dir, min_date=None, num_limit=None, num_limit_type=None, filter_jurnal_file=None, restrict=None, DOIs=None):
def start(query, scholar_pages, dwn_dir, min_date=None, num_limit=None, num_limit_type=None, filter_jurnal_file=None, restrict=None, DOIs=None, SciHub_URL=None):

to_download = []
if DOIs==None:
Expand Down Expand Up @@ -42,7 +42,7 @@ def start(query, scholar_pages, dwn_dir, min_date=None, num_limit=None, num_limi
if num_limit_type!=None and num_limit_type==1:
to_download.sort(key=lambda x: int(x.sc_cites) if x.sc_cites!=None else 0, reverse=True)

downloadPapers(to_download, dwn_dir, num_limit)
downloadPapers(to_download, dwn_dir, num_limit, SciHub_URL)

Paper.generateReport(to_download,dwn_dir+"result.csv")
Paper.generateBibtex(to_download,dwn_dir+"bibtex.bib")
Expand All @@ -64,6 +64,7 @@ def main():
parser.add_argument('--max-dwn-cites', default=None, type=int, help='Maximum number of papers to download sorted by number of citations')
parser.add_argument('--journal-filter', default=None, type=str ,help='CSV file path of the journal filter (More info on github)')
parser.add_argument('--restrict', default=None, type=int ,choices=[0,1], help='0:Download only Bibtex - 1:Down load only papers PDF')
parser.add_argument('--scihub-mirror', default=None, type=str, help='Mirror for downloading papers from sci-hub. If not set, it is selected automatically')

args = parser.parse_args()

Expand Down Expand Up @@ -115,7 +116,7 @@ def main():
max_dwn_type = 1


start(args.query, args.scholar_pages, dwn_dir, args.min_year , max_dwn, max_dwn_type , args.journal_filter, args.restrict, DOIs)
start(args.query, args.scholar_pages, dwn_dir, args.min_year , max_dwn, max_dwn_type , args.journal_filter, args.restrict, DOIs, args.scihub_mirror)


if __name__ == "__main__":
Expand Down
5 changes: 3 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ PyPaperBot arguments:
| \-\-max-dwn-cites | Maximum number of papers to download sorted by number of citations |int|
| \-\-journal-filter | CSV file path of the journal filter (More info on github) |string|
| \-\-restrict | 0:Download only Bibtex - 1:Down load only papers PDF |int|
| \-\-scihub-mirror | Mirror for downloading papers from sci-hub. If not set, it is selected automatically |string|
| \-h | Shows the help |--|

### Note
Expand All @@ -53,9 +54,9 @@ The argument *\-\-doi-file* require the path of a txt file containing the list
If access to SciHub is blocked in your country, consider using a free VPN service like [ProtonVPN](https://protonvpn.com/)

## Example
Download a maximum of 30 papers given a query and starting from 2018:
Download a maximum of 30 papers given a query and starting from 2018 using the mirror https://sci-hub.do:
```bash
python -m PyPaperBot --query="Machine learning" --scholar-pages=3 --min-year=2018 --dwn-dir="C:\User\example\papers"
python -m PyPaperBot --query="Machine learning" --scholar-pages=3 --min-year=2018 --dwn-dir="C:\User\example\papers" --scihub-mirror="https://sci-hub.do"
```

Download a paper given the DOI:
Expand Down