Skip to content

Commit

Permalink
[modify] notice web request error
Browse files Browse the repository at this point in the history
[add] independent chinese README
  • Loading branch information
findix committed Apr 22, 2019
1 parent f6e8a65 commit c4b91a6
Show file tree
Hide file tree
Showing 7 changed files with 112 additions and 51 deletions.
14 changes: 14 additions & 0 deletions CHANGELOG
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
20180611 0.1.0-alpha1
允许在txt中使用Python风格的注释,即以#开头的内容会被忽略
对txt中的空白符进行处理
保存路径不再强制包含 ArtStation
默认保存路径现在为用户根路径

20190131 0.1.1-alpha
重构代码结构
加入命令行模式
GUI和命令行均加入允许选择下载文件类型(所有、图片、视频)功能

20190422 0.1.1-alpha1
对不同报错情况进行提示
增加独立的中文README
71 changes: 71 additions & 0 deletions README-zh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# ArtStation Downloader

ArtStation Downloader 是一个帮助你批量从[ArtStation](https://www.artstation.com/)网站下载图片和视频的小工具

## 用法

### 从这里开始

[在此下载](https://github.com/findix/ArtStationDownloader/releases)

### 如何下载

输入你希望下载的作者的主页地址,或者其用户名


`https://www.artstation.com/xrnothing` 或者 `xrnothing`

然后点击 Download 按钮

你可以一次下载多位作者的作品

只需要在输入他们的 URL 或用户名时,在中间加入英文的","即可

或者,你也可以新建一个文本文件(.txt),每一行输入一位作者的信息。
(允许使用 Python 风格的注释,即 # 后的内容会被忽略,空白符与空行同样也会被忽略)

然后点击 Download txt 按钮选择文件即可。

Type 下拉选单可以设置下载资源的类型, 你可以选择“只下载图片”、“只下载视频”以及“全部下载”

Path 输入框内是下载文件夹位置,你可以按需求设置。

## 缺陷与反馈

如果在使用过程中发现任何错误、问题或是希望讨论,请使用 [Github Issues](https://github.com/findix/ArtStationDownloader/issues).

非常欢迎提交 Pull requests!

## 打包

需要 Pyinstaller.

运行 `build.bat`

## For macOS/Linux/Shell

首先执行 `pip install -r requirements.txt` 安装依赖

在 shell 中执行 `python ./src/ArtStationDownloader.py` 开启图形界面

或者

直接运行类似下面的命令:

`python ./src/ArtStationDownloader.py -u username_of_artist other_username or_more -d where/you/what`

您可以尝试输入 `python ./src/ArtStationDownloader.py --help` 查看更多用法

## FAQ

> **为什么我在点击下载后报错 `[Error] [403 Forbidden] You are blocked by artstation`**
ArtStation 有一个 [验证码](https://zh.wikipedia.org/zh-hans/%E9%AA%8C%E8%AF%81%E7%A0%81) 系统 (由 Cloudflare 提供)。如果你无法证明你不是爬虫,就会被禁止访问,这个问题目前我还没有找到解决办法,如果您有方式解决,请告诉我。


## LICENSE

MIT License

Copyright (c) 2018-2019 Sean Feng
42 changes: 11 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,22 @@
# ArtStation Downloader

ArtStation Downloader is a lightweight tool to help you download images and videos from the [ArtStation](https://www.artstation.com/)
[中文说明](./README-zh.md)

ArtStation Downloader 是一个帮助你批量从[ArtStation](https://www.artstation.com/)网站下载图片和视频的小工具
ArtStation Downloader is a lightweight tool to help you download images and videos from the [ArtStation](https://www.artstation.com/)

## Usage

### Getting started

[Download here](https://github.com/findix/ArtStationDownloader/releases)
[在此下载](https://github.com/findix/ArtStationDownloader/releases)

### Download from ArtStation

Input the URL of the artist or just the username

just like

https://www.artstation.com/xrnothing or xrnothing
`https://www.artstation.com/xrnothing` or `xrnothing`

and click the Download button.

Expand All @@ -34,42 +33,17 @@ The combobox named Type means you can choose what resources are required. image

The Path means the path of the download folder,just set it.

输入你希望下载的作者的主页地址,或者其用户名


https://www.artstation.com/xrnothing 或者 xrnothing

然后点击 Download 按钮

你可以一次下载多位作者的作品

只需要在输入他们的 URL 或用户名时,在中间加入英文的","即可

或者,你也可以新建一个文本文件(.txt),每一行输入一位作者的信息。
(允许使用 Python 风格的注释,即 # 后的内容会被忽略,空白符与空行同样也会被忽略)

然后点击 Download txt 按钮选择文件即可。

Type 下拉选单可以设置下载资源的类型, 你可以选择“只下载图片”、“只下载视频”以及“全部下载”

Path 输入框内是下载文件夹位置,你可以按需求设置。

## Bugs and Feedback

For bugs, questions and discussions please use the [Github Issues](https://github.com/findix/ArtStationDownloader/issues).

Pull requests are all welcome!

如果在使用过程中发现任何错误、问题或是希望讨论,请使用 [Github Issues](https://github.com/findix/ArtStationDownloader/issues).

非常欢迎提交 Pull requests!

## Build
## Package

Require Pyinstaller.

Just execute build.bat
Just execute `build.bat`

## For macOS/Linux/Shell

Expand All @@ -85,6 +59,12 @@ run like this:

try `python ./src/ArtStationDownloader.py --help` to get more usage

## FAQ

> **Why I get a message says `[Error] [403 Forbidden] You are blocked by artstation`?**
The ArtStation has a [CAPTCHA](https://en.wikipedia.org/wiki/CAPTCHA) system (by Cloudflare) which need you proof you are human otherwise forbid you. I haven't found a way solve this for now. If you have idea, please tell me.

## LICENSE

MIT License
Expand Down
3 changes: 2 additions & 1 deletion build.bat
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
pyinstaller --windowed .\src\ArtStationDownloader.py
pyinstaller -y --windowed .\src\ArtStationDownloader.py
pyinstaller -y --windowed -F .\src\ArtStationDownloader.py
@pause
Binary file modified requirements.txt
Binary file not shown.
16 changes: 2 additions & 14 deletions src/ArtStationDownloader.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,9 @@
"""批量下载ArtStation图片
Copyright 2018-2019 Sean Feng([email protected])
CHANGELOG
20180611 0.1.0-alpha1
允许在txt中使用Python风格的注释,即以#开头的内容会被忽略
对txt中的空白符进行处理
保存路径不再强制包含 ArtStation
默认保存路径现在为用户根路径
20190131 0.1.1-alpha
重构代码结构
加入命令行模式
GUI和命令行均加入允许选择下载文件类型(所有、图片、视频)功能
"""
__version__ = "0.1.1-alpha"

__version__ = "0.1.1-alpha1"
# $Source$

import argparse
Expand Down
17 changes: 12 additions & 5 deletions src/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,14 @@ def __init__(self, log_print=None):
self.executor_video = futures.ThreadPoolExecutor(1)
self.root_path = None
self.futures = []
self.session = requests.session()

def download_file(self, url, file_path, file_name):
file_full_path = os.path.join(file_path, file_name)
if os.path.exists(file_full_path):
self.log('[Exist][image][{}]'.format(file_full_path))
else:
r = requests.get(url)
r = self.session.get(url)
os.makedirs(file_path, exist_ok=True)
with open(file_full_path, "wb") as code:
code.write(r.content)
Expand All @@ -47,15 +48,15 @@ def download_video(self, id, file_path):
else:
video = pafy.new(id)
best = video.getbest(preftype="mp4")
r = requests.get(best.url)
r = self.session.get(best.url)
os.makedirs(file_path, exist_ok=True)
with open(file_full_path, "wb") as code:
code.write(r.content)
self.log('[Finish][video][{}]'.format(file_full_path))

def download_project(self, hash_id):
url = 'https://www.artstation.com/projects/{}.json'.format(hash_id)
r = requests.get(url)
r = self.session.get(url)
j = r.json()
assets = j['assets']
title = j['slug'].strip()
Expand Down Expand Up @@ -92,9 +93,15 @@ def get_projects(self, username):
page += 1
url = 'https://www.artstation.com/users/{}/projects.json?page={}'.format(
username, page)
r = requests.get(url)
r = self.session.get(url)
if not r.ok:
self.log("[Error] Please input right username")
err = "[Error] [{} {}] ".format(r.status_code, r.reason)
if r.status_code == 403:
self.log(err + "You are blocked by artstation")
elif r.status_code == 404:
self.log(err + "Username not found")
else:
self.log(err + "Unknown error")
break
j = r.json()
total_count = int(j['total_count'])
Expand Down

0 comments on commit c4b91a6

Please sign in to comment.