PVE 7 support

[lae]

I think I recall seeing an email saying PVE 7 was released (or at least an RC came out), so I'd like to work on supporting it in this role. However, per current circumstances (which I also alluded to in lae/ansible-role-netbox#132) I probably won't be able to get to this for several months. I also haven't been gainfully employed/haven't had access to bare metal environments since 2018, so I can't really experiment/test a few things effectively (Ceph being one) without using something expensive like Packet.

Also with PVE 7, I think we can sunset support for PVE 5/Debian Stretch since they're EOL (but I don't think this means much code cleanup like we had with PVE 4).

Anyway, if there is anyone who wants to get a headstart and work on getting this role working with 7, please feel free to start a branch (if I've already made you a collaborator) or fork this repo+create a new feature branch and open a PR. If discussion is needed, you can use this thread.

[zenntrix]

I have created a feature branch for this work 'feature/pve7'

[trickert76]

Ah, great. I've started to think about the migration process yesterday and now I see - there is already an open ticket. In production I'm waiting for Debian Bullseye to be official released - just to be sure.

Some questions. I see, you already did added the most necessary changes to the repo for PVE7. I think, most of it is the same I did last time with PVE6. Did you tested the installation process?

Main question is: I'm not 100% sure - do you think, this steps are also useful in case of "upgrade". I've always installed new major versions in the past. After reading the upgrade guide I think your changes are enough (adding key, switching apt-source-list, apt dist-upgrade). But I'm not sure. I'm thinking about some kind of "upgrade detection" during a play. Is the existing installation older then run pve6to7 (or later pve7to8), check that and then stop all VMs if configured and upgrade.

I can try to run the play on two single servers, but I don't have a test-cluster at the moment. Only a productive cluster (upgrading it will be fun).

[trickert76]

Ah, sorry - the last commit was wrong - I wanted to merge the pve7 branch to my local branch - but it was commited vice verse.

[zenntrix]

I have been testing the installation process as fresh install using my test servers and i have also tested a partial update so far where i upgrade the OS to bullseye manually and then use Ansible to upgrade pve6to7.

My next stage is to try and upgrade from 6to7 fully using just this ansible but before then i am experiencing some excessive performance issues of VMs on PVE7 so i would advise against upgrading at the moment.

The other thing to consider if doing an ansible upgrade is the highlighted issue of potential mac address changes when going from 6 to 7.

[trickert76]

Upgrading OS, later Proxmox works? Based on the upgrade guide, the PVE repo must be switched too. I'm trying that at the moment manually on a test machine, to have a look of the changes.

I've started to add some tasks for an explicit upgrade from x to bullseye like:

---
- name: Gather distribution specific variables
  include_vars: "debian-{{ pve_upgrade_distribution }}.yml"

- name: "Switching distribution from {{ ansible_distribution_release }} to {{ pve_upgrade_distribution }}"
  copy:
    backup: true
    dest: "/etc/apt/sources.list"
    content: |
      # {{ ansible_managed }}
      deb http://deb.debian.org/debian bullseye main
      deb-src http://deb.debian.org/debian/ bullseye main
      deb http://security.debian.org/debian-security bullseye-security main
      deb-src http://security.debian.org/debian-security bullseye-security main
      deb http://deb.debian.org/debian bullseye-updates main
      deb-src http://deb.debian.org/debian/ bullseye-updates main

- name: "Trust Proxmox' packaging key"
  apt_key:
    data: "{{ lookup('file', pve_release_key) }}"
    id: "{{ pve_release_key_id }}"
    state: "present"

- name: "Add Proxmox repository"
  apt_repository:
    repo: "deb {{ pve_repository }} {{ pve_upgrade_distribution }} {{ pve_subscription }}"
    filename: "proxmox"
    state: "present"
  register: _pve_repo

- name: "Add Proxmox Ceph repository"
  apt_repository:
    repo: "{{ pve_ceph_repository_line }}"
    filename: "ceph"
    state: "present"
  register: _pve_ceph_repo
  when: "pve_ceph_enabled | bool"


- name: "Run apt-get dist-upgrade on repository changes"
  apt:
    update_cache: yes
    cache_valid_time: 3600
    upgrade: "dist"
  when: "_pve_repo is changed or _pve_ceph_repo is changed"
  retries: 2
  register: _dist_upgrade
  until: "_dist_upgrade is succeeded"

- name: "Reboot for kernel update"
  reboot:
    msg: "PVE kernel update detected by Ansible"

- name: "Gather new facts"
  setup:

This would be a new file that is called at start of main.yml with

- name: "Upgrade system"
  include_tasks: "upgrade_system_{{ pve_upgrade_distribution }}.yml"
  when:
    - "pve_upgrade_distribution is defined"
    - "pve_upgrade_distribution != ansible_distribution_release"

And the new default vars:

pve_subscription: "pve-no-subscription"
pve_repository: "http://download.proxmox.com/debian/pve"
pve_upgrade_distribution: "bullseye"

For that pve_repository_line is replaced with deb {{ pve_repository }} {{ ansible_distribution_release }} {{ pve_subscription }}

[trickert76]

Hmm, I'm getting a file not found ... files/00_remove_checked_command_bullseye.patch.

I've added ignore_errors: true to the patch-task. Funny - I don't get a message after upgrading from 6.4. So either the script isn't changed or the warning is not available anymore.

[mweinelt]

That's because the file does not exist and needs to be created.

diff -u /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
--- /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
+++ /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
@@ -493,7 +493,8 @@ utilities: {
     },
 
     checked_command: function(orig_cmd) {
-	Proxmox.Utils.API2Request(
+	orig_cmd()
+	false && Proxmox.Utils.API2Request(
 	    {
 		url: '/nodes/localhost/subscription',
 		method: 'GET',

Basically the same as buster, but different lines.

[zenntrix]

Hmm, I'm getting a file not found ... files/00_remove_checked_command_bullseye.patch.

I've added ignore_errors: true to the patch-task. Funny - I don't get a message after upgrading from 6.4. So either the script isn't changed or the warning is not available anymore.

Thanks for highlighting this, i didn't have that flag set to remove so hadn't checked it as of yet! Now fixed.

[mweinelt]

Did a fresh install with PVE 7.0 yesterday and pull requested some fixes and updates.

[penguin02007]

We just tested this and it works fine. Thank you @zenntrix

Curious when will this merge to master so we can do something like specifying pve=7 as an argument? I am happy to help if needed!

[zenntrix]

Afternoon @penguin02007, glad it works for you. Out of interest are you using CEPH with your installation?

[lae]

I also tested it earlier last week (manually did a dist-upgrade beforehand) and it was working fine, but for a single node install. At the moment I'm waiting for Richard to make the call and open a PR when he feels the branch is ready, since I don't know if there's anything missing (haven't personally read through PVE changelog myself to know what all needs changes).

[penguin02007]

No Ceph. This was tested on (1) Dell R7515 via local disk.

We do have few old R630 and supermicros in the lab that can help test if needed.

[zenntrix]

Good to know, so although these changes have worked for me, it had an immense impact on my CEPH performance which i am still debugging and so have held off PR'ing in case it is related.

If anyone else has / will test this with CEPH and have no issues, i will PR it :-)

[penguin02007]

Is there a ticket open with Proxmox?

Sounds like this is related to the Proxmox/ceph than install problem, i believe we can still PR it and allow users to toggle between 6/7?

[lae]

A toggle isn't necessary, unless further work is done to actually support upgrades. The working branch doesn't upgrade PVE 6 installs to PVE 7. The end user makes that decision when they decide which Debian version to install or upgrade to (PVE 7 is for Debian 11/PVE 6 for Debian 10).

[trickert76]

I've started to implement a "toggle" - which is at least a change to the sources.list. Please have a look at https://github.com/acoby/ansible-role-proxmox/blob/develop/tasks/upgrade_system_bullseye.yml

I didn't add this to this branch yet, because I didn't test it yet and I'm not sure about other sources in apt. I don't know a "general best practice" way to switch all sources to a new debian distribution (except using "stable" at all).

[lae]

So, these are just my thoughts on the matter and probably not everyone agrees with me, but I think trying to support upgrades, at least via something simple like a "switch/toggle", for something with many moving parts that I doubt are all tracked by this role in many environments isn't really a straightforward feat and prone to getting things wrong. I'm not sure if we should add that to the scope of our role.

In any case, if we want to do that I think we should open a new thread to discuss, so we can capture the various upgrade paths/procedures and figure out an optimal course of action. At the very least I think it might be more appropriate to provide separate "administrative" playbooks that a sysadmin can apply? Similar to what Ceph does here: https://github.com/ceph/ceph-ansible/blob/master/infrastructure-playbooks/rolling_update.yml

Anyway, one issue in the upgrade_system_bullseye.yml you've provided is that it tries to upgrade PVE and Ceph simultaneously and the wiki says as a precondition to upgrade Ceph Nautilus to Octopus before upgrading to PVE 7.0.

[trickert76]

From my point of view, I think a separate playbook would be ok for that (as I wrote, this toggle can really be a hugh problem when the sources are not completly managed by "this" role, maybe because you've added other repositories (like Icinga or so)).

Good point regarding Ceph update - I don't use Ceph because of the cost of harddisc - Gluster is a much "easier" solution with nearly the same bandwith (I've tested both solution on my "reduced" hardware.

Maybe add a new issue about that playbook for a "rolling update".

[zenntrix]

Ok, good news, the ceph issue is now resolved and was not related to the changes on this piece of work! I shall now continue...

[Jakkerus]

Am not using ceph here. Just tried to deploy this to Dell R630 for single node install. Was unsuccessful. I am also able/willing to help test to get this finished up.

[lae]

@Jakkerus New install? Or are you upgrading?

Please provide a copy of your role variables+playbook and further details about the issue and how you reproduced it. Is a task failing? Provide the failing task's output if so.

[Jakkerus]

New install on scratch Debian machine. downloaded local copy of PVE 7 branch. Followed setup exactly from your readme.

TASK [ansible-role-proxmox-feature-pve7 : Install Proxmox VE and related packages] **********************************************************************************************************
fatal: [192.168.101.10]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ", "unreachable": true}

PLAY RECAP **********************************************************************************************************************************************************************************
192.168.101.10 : ok=34 changed=14 unreachable=1 failed=0 skipped=30 rescued=0 ignored=0

I make it to this stage and then fail...

...as a side note. I tried to join the discord server, but it keeps dropping on me. Is the invite link in the readme only temporary?

Thanks.

[lae]

for those having the same issue:
The pve-manager package was failing to install because of conflicting entries in the hosts file. The regexp that handled this was fixed in develop but isn't in the pve 7.x working branch. If you're testing this on a standalone node (cluster setups don't have this particular issue) then rebase this branch on develop before you deploy.