Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

想请问下为什么我运行manager.py里面的check-ip指令时,会有报错 #1

Open
CloudDing0818 opened this issue Apr 7, 2019 · 3 comments

Comments

@CloudDing0818
Copy link

大佬你好~具体情况是,在WINDOWS环境下运行了三个爬取爬虫,然后执行python manager.py check-ip指令会得到以下报错(The system cannot find the path specified.):
(proxy_demo-V1UiAS_e) C:\Users\CLOUDING\proxy_demo\proxy_pool>python manager.py check-ip
2019-04-07 16:00:27,013 - check_proxy.py[line:112] - WARNING: 58.254.220.116:52470 ---- The port is not open
2019-04-07 16:00:31,171 - check_proxy.py[line:129] - WARNING: 182.92.113.183:8118 ------ can"t access
The system cannot find the path specified.
2019-04-07 16:00:33,774 - check_proxy.py[line:112] - WARNING: 121.225.25.134:3128 ---- The port is not open
2019-04-07 16:00:33,779 - check_proxy.py[line:112] - WARNING: 125.40.109.154:44641 ---- The port is not open
2019-04-07 16:00:35,346 - check_proxy.py[line:112] - WARNING: 211.147.239.101:60999 ---- The port is not open
2019-04-07 16:00:35,843 - check_proxy.py[line:112] - WARNING: 182.88.191.79:8123 ---- The port is not open
2019-04-07 16:00:36,177 - check_proxy.py[line:112] - WARNING: 116.209.55.39:9999 ---- The port is not open
2019-04-07 16:00:37,116 - check_proxy.py[line:129] - WARNING: 124.235.135.87:80 ------ can"t access
The system cannot find the path specified.

然后去检查数据库(Mysql),使用pymysql连接,发现数据库中只更新了update_time,不可用的代理score被置为0,其他的可用代理score和weight并没有更新:
image
运行了挺久的时间,发现都没有一个weight>0的IP,speed也都是0,尽管有些明明日志上都打印出代理可用以及速度了:
2019-04-07 16:05:48,644 - check_proxy.py[line:137] - INFO: 119.102.188.101:9999 ------ active proxy, speed:3607, open_ports:9999,22,80
The system cannot find the path specified.
image

所以我考虑是不是这个报错导致更新数据库错误了,还是我有哪里使用出问题了,麻烦大佬帮忙看下呢~感激不尽
以下是我的数据库文件配置:
[mysql]
dsn = mysql+pymysql://root:[email protected]:3306/proxy
init.py文件中的配置:

-- coding: utf-8 --

import logging
import configparser
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

logging.basicConfig(
level=logging.INFO,
format=
'%(asctime)s - %(filename)s[line:%(lineno)d] - %(levelname)s: %(message)s')
#engine = create_engine('mysql+mysqldb://scott:tiger@localhost/foo')
config = configparser.ConfigParser()
config.read('C:\Users\CLOUDING\proxy_demo\proxy_pool\proxy_pool\config.ini')
engine = create_engine(config.get('mysql', 'dsn'), echo=False, pool_size=500, pool_recycle=3600)
#engine = create_engine('mysql+pymysql://root:[email protected]:3306/proxy', echo=False, pool_size=500, pool_recycle=3600)
Session = sessionmaker(engine)

@meng950813
Copy link

同问

@netzeng
Copy link

netzeng commented Jul 14, 2019

此处系统找不到指定的路径。 产生问题的原因是:
在utils.py 当中 check_network函数造成的。

def check_network():
exit_code = os.system('cmd.exe /c ping www.baidu.com -c 4 > /dev/null 2>&1')
if exit_code:
return False
return True

windows不支持类似 /dev/null 2>&1的写法。

import os
os.system('cmd.exe /c ping www.baidu.com -c 4 > /dev/null 2>&1')
系统找不到指定的路径。
1

此处可以使用request.get 请求一下百度,如果状态码是200,就可以认为是在线,没必要调用系统的ping命令,以上是个人理解。

@wanggnag
Copy link

wanggnag commented Aug 5, 2019

楼主请问解决了吗? @CloudDing0818

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants