New mirror in India

[sahilister]

Service(s)

mirrors.jenkins.io

Summary

Hello, I'm Sahil Dhiman, maintainer of two mirrors (ie in India and Germany) where I host bunch of Free Software projects. For my India location I want to add Jenkins mirror.

I went through the Google Groups and existing mirror addition issues here around bandwidth and other requirements.
I have synced the repo and setup hourly cron for updates from ftp.halifax.rwth-aachen.de.

HTTP - http://mirrors.in.sahilister.net/jenkins/
HTTPs - https://mirrors.in.sahilister.net/jenkins/
rsync - None. I'll not prefer adding rsync support as mirror speeds right now is 400Mbps/s, so other mirrors (if they want) will get faster speeds for sync from other mirrors.
Location - Mumbai, India
Speeds - 400Mbps/s (have provisions for horizontal scaling upto 1Gbps/s if mirror starts pushing more than 400Mbps/s).

Small pointer though about Geo-location. My provider recently opened India location so GeoIP with some providers still says it's in Oslo, Norway. The mirror is indeed located in India (can be confirmed with mtr/traceroute). So a manual overwrite for location would be required for Mirrorbits for now.

Let me know if anything else is required. I'm all set to serve Jenkins traffic.

[sahilister]

rsync endpoint added.
URL - rsync://jenkins.in.sahilister.net/jenkins

For now, I have opened the endpoint for public access.

[MarkEWaite]

Small pointer though about Geo-location. My provider recently opened India location so GeoIP with some providers still says it's in Oslo, Norway. The mirror is indeed located in India (can be confirmed with mtr/traceroute). So a manual overwrite for location would be required for Mirrorbits for now.

I'm not aware of any way to perform a manual overwrite of the GeoIP location. Are you sure that is possible?

[timja]

Neither a form should be submitted here for an update:
https://www.maxmind.com/en/geoip-location-correction

We update automatically so shouldn't take too long to be corrected

(I can see it says its currently in Oslo)

[dduportal]

Small pointer though about Geo-location. My provider recently opened India location so GeoIP with some providers still says it's in Oslo, Norway. The mirror is indeed located in India (can be confirmed with mtr/traceroute). So a manual overwrite for location would be required for Mirrorbits for now.

I'm not aware of any way to perform a manual overwrite of the GeoIP location. Are you sure that is possible?

We can override the GeoIP informations in mirrorbits itself when the mirror is added though

[dduportal]

rsync endpoint added. URL - rsync://jenkins.in.sahilister.net/jenkins

For now, I have opened the endpoint for public access.

Thanks @sahilister ! We'll most probably try to add your mirror next week.

We just published how to get the list of "outbound" IPs used by our system in #4114 (comment), if you want to add restrictions for the Rsync server you added.

[sahilister]

Neither a form should be submitted here for an update:
https://www.maxmind.com/en/geoip-location-correction

We update automatically so shouldn't take too long to be corrected

(I can see it says its currently in Oslo)

Thanks! Submitted a correction request for both IPv4 and IPv6 addresses for the mirror yesterday and can see them updated to Mumbai, India on https://www.maxmind.com/en/geoip-web-services-demo .

[dduportal]

We're going to add this mirror to get.jenkins.io today (Monday 24 June 2024) between 01:35pm UTC and 02:00pm UTC (along with #4147)

[dduportal]

Hi @sahilister!

Could you share with us your administrator email so we can set it up in the mirror system? You can share it privately to jenkins-infra-team googlegroups.com or publicly in this issue

[dduportal]

We're going to add this mirror to get.jenkins.io today (Monday 24 June 2024) between 01:35pm UTC and 02:00pm UTC (along with #4147)

Mirror added, scanned and used by get.jenkins.io:

• See https://get.jenkins.io/index.html?mirrorstats:

Many thanks @sahilister! I'm closing the issue but don't forget to share your administrator email

[sahilister]

@dduportal great! thanks for the work.

My contact mail is mirrors AT sahilister.in

[sahilister]

Seems like Mirrorbits got sad with the my mirror load balancing.

Seeing higher traffic number, today I horizontally scaled the cluster to add a machine to serve Jenkins only. Added HTTP redirection to serve 302 towards the Jenkins server ie https://2.mirrors.in.sahilister.net from https://mirrors.in.sahilister.net for Jenkins route. Traffic moved flawlessly to other machine but now no requests seems to be redirected towards the mirror.

@dduportal Can check and share Mirrorbits log for my mirror and see what's wrong?

rsync points to new server IPs but the TTLs is/was 6 hours, which'll take time to propagate.
GeoIP for the other machine is correct ie Mumbai, India.

[MarkEWaite]

Seeing higher traffic number, today I horizontally scaled the cluster to add a machine to serve Jenkins only. Added HTTP redirection to serve 302 towards the Jenkins server ie https://2.mirrors.in.sahilister.net/ from https://mirrors.in.sahilister.net/ for Jenkins route.

I don't think that mirrorbits supports that load balancing technique.

https://get.jenkins.io/war/2.464/jenkins.war?mirrorlist shows that https://mirrors.in.sahilister.net/jenkins/ is unreachable.

[dduportal]

reopening the issue so we will update the mirror configuration to the new URL next week

[dduportal]

Seeing higher traffic number, today I horizontally scaled the cluster to add a machine to serve Jenkins only. Added HTTP redirection to serve 302 towards the Jenkins server ie https://2.mirrors.in.sahilister.net/ from https://mirrors.in.sahilister.net/ for Jenkins route.

I don't think that mirrorbits supports that load balancing technique.

https://get.jenkins.io/war/2.464/jenkins.war?mirrorlist shows that https://mirrors.in.sahilister.net/jenkins/ is unreachable.

Gotta check but i concur, mirrorbits most probably expects a certain URL to answer http/200 on the configured HTTP endpoint. We should update it and we will be ok

[sahilister]

Yesterday, I added a script to update Jenkins outbound IP and serve them from primary machine, which seem to make Mirrorbits happy then.

Though, see if you could see any option in Mirrorbits instance which follow the 302 redirect -> 200 and Mirrorbits check status of final node.

If that doesn't work, for now we can update the URLs to
HTTP(s) - https://2.mirrors.in.sahilister.net/jenkins
rsync - rsync://jenkins.in.sahilister.net (same as before).

[dduportal]

Yesterday, I added a script to update Jenkins outbound IP and serve them from primary machine, which seem to make Mirrorbits happy then.

Though, see if you could see any option in Mirrorbits instance which follow the 302 redirect -> 200 and Mirrorbits check status of final node.

If that doesn't work, for now we can update the URLs to HTTP(s) - https://2.mirrors.in.sahilister.net/jenkins rsync - rsync://jenkins.in.sahilister.net (same as before).

Hi @sahilister , thanks! We've updated the mirror configuration with the new HTTP URL (and kept the same config for everything else).

We see the result in https://get.jenkins.io/war/2.464/jenkins.war?mirrorlist

Is that OK for you?

[sahilister]

All good from my end as well.

[dduportal]

Hi @sahilister ! We're seeing issues with your mirror: the Rsync scan are failing from our production system:

$ rsync -v rsync://jenkins.in.sahilister.net/jenkins/
rsync: [Receiver] failed to connect to jenkins.in.sahilister.net (94.136.187.92): Connection refused (111)
rsync: [Receiver] failed to connect to jenkins.in.sahilister.net (2400:d321:2202:3330::1): Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(139) [Receiver=3.2.7]

while the HTTP server is still up and seems up to date: https://2.mirrors.in.sahilister.net/jenkins/

It looks like our outbound IPs might be blocked: if you need,n you can subscribe to https://reports.jenkins.io/infrastructure/v2/index.json where we keep our outbound IPs up to date!

[sahilister]

@dduportal checked and my rsyncd process was dead for some reason. Restarted, rsync is up now.

You can check and verify.

(sorry for the trouble)

[dduportal]

@dduportal checked and my rsyncd process was dead for some reason. Restarted, rsync is up now.

You can check and verify.

(sorry for the trouble)

Thanks for checking!

Scanning was a success and your mirror is listed again: https://get.jenkins.io/windows-stable/2.426.3/jenkins.msi?mirrorlist

[sahilister]

Hello @dduportal , seeing Mirrorbit unhappy with my mirror, marking it down at https://get.jenkins.io/windows-stable/2.426.3/jenkins.msi?mirrorlist . Server and rsync logs seems successful recently from 20.7.192.189

Is something amiss?

[dduportal]

Hello @dduportal , seeing Mirrorbit unhappy with my mirror, marking it down at https://get.jenkins.io/windows-stable/2.426.3/jenkins.msi?mirrorlist . Server and rsync logs seems successful recently from 20.7.192.189

Is something amiss?

Thanks for noticing and telling us!

It's most probably related to #4369 (comment). Let's wait 24h before mirrorbits scans it again.

[sahilister]

Can confirm, traffic levels are back to normal.

thanks @dduportal