0.17.0-rc1 not updating container

[rreilly-edr]

New containers are not being updated by keel,
keel 0.17.0-rc1
helm 3.2.4
k8s v1.15.11-eks-af3caf

below are my values that apply to keel

keel:
  policy: all
  trigger: poll
  pollSchedule: "@every 2m"
  approvals: 1
  approvalDeadline: 24
  images:
   - repository: image.repository
     tag: image.tag
     imagePullSecret: regcred

I Check approvals and I see there is one waiting for the upgrade from 0.1.9 to 0.1.10

{
    "id": "01327ddc-e587-490e-bdbe-3df536e0b548",
    "archived": false,
    "provider": "helm",
    "identifier": "data-api-dev/xx-api:0.1.10",
    "event": {
      "repository": {
        "host": "",
        "name": "index.docker.io/xx/lb-xx-api",
        "tag": "0.1.10",
        "digest": ""
      },
      "createdAt": "0001-01-01T00:00:00Z",
      "triggerName": "poll"
    },
    "message": "New image is available for release xx-api-dev/xx-api (0.1.9 -> 0.1.10).",
    "currentVersion": "0.1.9",
    "newVersion": "0.1.10",
    "digest": "",
    "votesRequired": 1,
    "votesReceived": 0,
    "voters": null,
    "rejected": false,
    "deadline": "2020-07-16T12:53:17.383369874Z",
    "createdAt": "2020-07-15T12:53:17.383507486Z",
    "updatedAt": "2020-07-15T12:53:17.383507756Z"
  },

I approve the release with

curl -u "admin:admin" -k -XPOST -H "Content-Type: application/json" -d @/home/rreilly/tmp/data.json  https://keel.xx.com/v1/approvals
{"id":"01327ddc-e587-490e-bdbe-3df536e0b548","archived":false,"provider":"helm","identifier":"xx-api-dev/xx-api:0.1.10","event":{"repository":{"host":"","name":"index.docker.io/xx/xx-xx-api","tag":"0.1.10","digest":""},"createdAt":"0001-01-01T00:00:00Z","triggerName":"approval"},"message":"New image is available for release xx-api-dev/xx-api (0.1.9 -\u003e 0.1.10).","currentVersion":"0.1.9","newVersion":"0.1.10","digest":"","votesRequired":1,"votesReceived":1,"voters":{"Rob":"2020-07-15T12:59:42.456443341Z"},"rejected":false,"deadline":"2020-07-16T12:53:17.383369874Z","createdAt":"2020-07-15T12:53:17.383507486Z","updatedAt":"2020-07-15T12:59:42.456686965Z"}

The keel log appears as if it is updating the container:

time="2020-07-15T12:53:17Z" level=info msg="policy for release xxx-api-dev/xx-api parsed: all"
time="2020-07-15T12:55:17Z" level=info msg="policy for release xxx-api-dev/xx-api parsed: all"
time="2020-07-15T12:57:17Z" level=info msg="policy for release xx-api-dev/xx-api parsed: all"
time="2020-07-15T12:59:17Z" level=info msg="policy for release xx-api-dev/xx-api parsed: all"
time="2020-07-15T12:59:42Z" level=info msg="approvals.manager: approved" identifier="xx-api-dev/xx-api:0.1.10"
time="2020-07-15T12:59:44Z" level=info msg="policy for release xx-api-dev/xx-api parsed: all"
time="2020-07-15T12:59:45Z" level=info msg="preparing upgrade for xx-api"
time="2020-07-15T12:59:45Z" level=info msg="reusing the old release's values"
time="2020-07-15T12:59:52Z" level=info msg="performing update for xx-api"
time="2020-07-15T12:59:52Z" level=info msg="creating upgraded release for xx-api"
time="2020-07-15T12:59:52Z" level=info msg="checking 5 resources for changes"
time="2020-07-15T12:59:52Z" level=info msg="Looks like there are no changes for ServiceAccount \"xx-api\""
time="2020-07-15T12:59:52Z" level=info msg="Looks like there are no changes for ConfigMap \"env\""
time="2020-07-15T12:59:52Z" level=info msg="Looks like there are no changes for Service \"xx-api\""
time="2020-07-15T12:59:53Z" level=info msg="Looks like there are no changes for Deployment \"xx-api\""
time="2020-07-15T12:59:53Z" level=info msg="Looks like there are no changes for Ingress \"xx-api-dev-xx-com\""
time="2020-07-15T12:59:53Z" level=info msg="updating status for upgraded release for xx-api"
time="2020-07-15T12:59:53Z" level=info msg="provider.helm3: release updated" overrideValues="map[image.tag:0.1.10]" release=data-api version=37

but describing the pod show we are still running version 0.1.9

    Image:          index.docker.io/xx/lb-xx-api:0.1.9

and the last restart was 19 hours ago

Created 19 hours ago / Restarts: 2

the one thing i notice about 0.17-0-rc1 versus the older version i tried to use ( before i realized they did not support helm3) was that it installed in kube-system and this release installed in default.

[rreilly-edr]

I thought my issue was keel was not running in kube-system, but i have since fixed that but i am still seeing the above behavior.

[rusenask]

does helm list show updated revision?

[rreilly-edr]

Yes it has incremented from 36 which was from yesterday

NAME    	NAMESPACE   	REVISION	UPDATED                                	STATUS  	CHART         	APP VERSION
xx-api	xx-api-dev	38      	2020-07-15 13:51:46.482712132 +0000 UTC	deployed	xx-api-0.1.0	0.1.9 

the updated date is from this AM when i deployed it, does it only update the revision and not the app version ? but I would expect the container version in the pod to be updated.

[rusenask]

sorry, couldn't reproduce.. My chart values.yaml:

image:
  repository: keelhq/helmv3-example
  pullPolicy: IfNotPresent
  # Overrides the image tag whose default is the chart appVersion.
  tag: "0.1.0"

keel:
  # keel policy (all/major/minor/patch/force)
  policy: all
  # images to track and update
  trigger: poll
  # polling schedule
  pollSchedule: "@every 10s"
  approvals: 1
  images:
    - repository: image.repository # [1]
      tag: image.tag  # [2]

I have approved via UI:

Then it updated the image, checked pod details:

Events:
  Type    Reason     Age        From               Message
  ----    ------     ----       ----               -------
  Normal  Scheduled  <unknown>  default-scheduler  Successfully assigned helmtest/example-hello-world-649f46c67d-55wgx to minikube
  Normal  Pulled     18s        kubelet, minikube  Container image "keelhq/helmv3-example:0.2.0" already present on machine
  Normal  Created    18s        kubelet, minikube  Created container hello-world
  Normal  Started    18s        kubelet, minikube  Started container hello-world

Keel logs:

time="2020-07-15T19:07:36Z" level=info msg="checking 3 resources for changes"
time="2020-07-15T19:07:36Z" level=info msg="Looks like there are no changes for ServiceAccount \"example-hello-world\""
time="2020-07-15T19:07:36Z" level=info msg="Looks like there are no changes for Service \"example-hello-world\""
time="2020-07-15T19:07:36Z" level=info msg="Replaced \"example-hello-world\" with kind Deployment for kind Deployment\n"
time="2020-07-15T19:07:36Z" level=info msg="updating status for upgraded release for example"
time="2020-07-15T19:07:36Z" level=info msg="provider.helm3: release updated" overrideValues="map[image.tag:0.2.0]" release=example version=2

Maybe your values.yaml image tag doesn't match what keel config specifies? Does it look like:

image:
  repository: "xx/lb-xx-api"
  tag: "0.1.9"

[rreilly-edr]

Yes, just FYI at this point i am trying to use keel to publish 0.1.10, but helm is still on 0.1.9

image:
  repository:  index.docker.io/xx/lb-xx-api
  tag: "0.1.9"
  pullPolicy: IfNotPresent

[rusenask]

but helm is just a templating cli for k8s manifests, Keel will only update resources in k8s cluster

[rreilly-edr]

Yes, Understand that, i changed the logging to debug did not really get anymore pertinent info except this error

helper doesn't support this registry" error="unsupported registry"

time="2020-07-16T16:09:36Z" level=debug msg="extension.credentialshelper: helper doesn't support this registry" error="unsupported registry" helper=aws tracked_image="namespace:data-xx-dev,image:index.docker.io/xx/lb-xx-api:0.1.12,provider:helm3,trigger:poll,sched:@every 2m,secrets:[regcred]"

time="2020-07-16T16:09:36Z" level=debug msg="secrets.defaultGetter: secret looked up successfully" image=index.docker.io/xx/lb-xx-api namespace=data-xx-dev provider=helm3 registry=index.docker.io
time="2020-07-16T16:09:36Z" level=debug msg="registry.tags url=https://index.docker.io/v2/xx/lb-xx-api/tags/list repository=xx/lb-xx-api"
time="2020-07-16T16:09:36Z" level=debug msg="trigger.poll.WatchRepositoryTagsJob: checking tags" current_tag=0.1.12 image_name="index.docker.io/xx/lb-xx-api:0.1.12" repository_tags="[0.1.0 0.1.1 0.1.10 0.1.11 0.1.12 0.1.2 0.1.3 0.1.4 0.1.6 0.1.7 0.1.8 0.1.9]"

[rreilly-edr]

I have updated the eks cluster from v1.15.11-eks-af3caf to v1.17.6-eks-4e7f64 deleted and redeployed keel, same behavior. thinking it may have been an issue with the older version of k8s.

[rreilly-edr]

Ok, I figured it out, I had to update the deployment.yml template to point to Values.image.tag not, Chart.AppVersion, which was not evident to me from the docs. but it is now working.

[mansing2]

@rreilly-edr - I have been trying to do the same, and only the deployment is getting updated and not the helm chart revision, any specific parameters to be passed in values.yaml, Chart.yaml, or Deployment.yaml, other than shown on this thread. Would you able to suggest please? #567
@rusenask