Skip to content
This repository has been archived by the owner on Dec 20, 2024. It is now read-only.

After deployment, it can be used normally, but after a period of time, it is impossible to retrieve the image. #545

Closed
lijianfeng1993 opened this issue May 24, 2019 · 8 comments · Fixed by #582
Assignees

Comments

@lijianfeng1993
Copy link

lijianfeng1993 commented May 24, 2019

Using k8s deployment, deployment deployment deployment supernode, daemonset deployment dfclient

// 使用k8s部署,deployment部署supernode,daemonset部署dfclient

Configure httpproxy to pull the private image warehouse image.

// 配置httpproxy,拉取私有镜像仓库镜像。

After deployment, the node can pull the mirror smoothly. Walk on the P2P network.

// 部署完成后,节点可以顺利拉取镜像。走的p2p网络。

But after a day or two of use, there will be an inability to pull the mirror. The phenomenon is that when the mirror is pulled, part of the mirror layer is retry, and the final report is 502.

// 但是在使用一两天后,就会出现无法拉取镜像。现象就是拉镜像的时候,部分镜像层一直retry,最后报502.

[root@paasn4 ~]# docker pull registry.paas/library/mysql:5.6
5.6: Pulling from library/mysql
6ae821421a7d: Downloading
a9e976e3aa6d: Download complete
e3735e44a020: Download complete
bfd564e9483f: Download complete
df705f26e488: Download complete
25097d3ea7b8: Download complete
606c1b74fda2: Download complete
eda5767f8d3f: Download complete
5b57c9a598ba: Download complete
57f28f294e2f: Download complete
94b29ab4a8ab: Download complete
received unexpected HTTP status: 502 Bad Gateway
The solution is to restart the pod of supernode at this time, and it will return to normal, but after using it for a day or two, the problem will arise again.

// 解决方法,此时重启supernode的pod,就会恢复正常,但是使用一两天后,又会出现该问题。

Log information, supernode log no error, dfclient log error

// 日志信息,supernode日志无报错,dfclient日志有报错

image

!!!!WE STRONGLY ENCOURAGE YOU TO DESCRIBE YOUR ISSUE IN ENGLISH!!!!

@pouchrobot pouchrobot changed the title 部署完成后可以正常使用,但是使用一段时间后,无法拉取镜像 After deployment, it can be used normally, but after a period of time, it is impossible to retrieve the image. May 24, 2019
@starnop
Copy link
Contributor

starnop commented May 24, 2019

It looks like a similar issue with #311. Please keep an eye on it.

@lijianfeng1993
Copy link
Author

https://anjia0532.github.io/2019/03/25/dragonfly/
我看其他网友也碰到了一样的问题,就是部署完成后可以使用,但是过几天就不行了。

@lijianfeng1993
Copy link
Author

怀疑是否是supernode节点资源不够导致的,是否supernode对节点配置有要求,官方能够给出一个节点配置要求说明呢。

@lowzj
Copy link
Member

lowzj commented May 27, 2019

Could you paste the failure logs in dfclient.log?

Steps:

  • find a failed task: grep 'download FAIL' dfclient.log, such as:
    2019-05-22 05:40:58.120 INFO sign:38923-1558496382.915 : download FAIL cost:75.208s length:4120442 reason:0
    
  • get all the logs of this task through the sign 38923-1558496382.915: grep 38923-1558496382.915 dfclient.log, such as:
    2019-05-22 05:39:42.919 INFO sign:38923-1558496382.915 : get cmd params:["dfget" "-u" "https://xxx" "-o" "./a.test"]
    ...
    ...
    2019-05-22 05:40:58.120 INFO sign:38923-1558496382.915 : download FAIL cost:75.208s length:4120442 reason:0
    

@lijianfeng1993
Copy link
Author

@lowzj
image

@lowzj
Copy link
Member

lowzj commented May 28, 2019

The result's code 607 means url not reachable. You can search the <url> in the supernode's log to find out why it's not reachable.

@lijianfeng1993
Copy link
Author

change supernode image from registry.cn-hangzhou.aliyuncs.com/dragonflyoss/supernode:0.3.0 to hub.c.163.com/hzlilanqing/supernode:0.3.0, it has no problem。

@lijianfeng1993
Copy link
Author

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants