🚨 Argo CD Error: ComparisonError — repository not accessible

Overview

An Argo CD Application stops reconciling and its health/sync status goes to Unknown. The GitOps loop is not applying anything because it can no longer generate the desired manifests — it cannot read the Git repository. The condition surfaces in the Application status and the argocd-repo-server logs:

ComparisonError: rpc error: code = Internal desc = Failed to load target state:
failed to generate manifest for source 1 of 1:
rpc error: code = Unknown desc = failed to list refs:
authentication required: Repository not found.

A related host-key variant appears with SSH remotes:

ComparisonError: rpc error: code = Unknown desc = error creating SSH agent:
ssh: handshake failed: knownhosts: key mismatch

Because Argo CD never obtains the desired state, the Application shows Unknown, auto-sync is paused, and drift accumulates silently — the cluster keeps running the last-applied version while Git changes go unshipped.

Symptoms

• Application status is Unknown (not OutOfSync or Synced), with a ComparisonError condition.
• argocd app get <app> shows Repository not accessible or failed to list refs.
• argocd-repo-server pod logs authentication required, Repository not found, or knownhosts: key mismatch.
• New commits to the tracked branch never trigger a sync; auto-sync appears “stuck.”
• Other Applications on the same repo may fail together (shared credential) or one at a time (per-app repo).
• Manual argocd app sync returns immediately with the same comparison error instead of applying.

Common Root Causes

• Missing or wrong repository credentials. The repo is private and no matching repo secret exists, or a rotated PAT/deploy key expired. Repository not found from a private host almost always means auth, not a missing repo.
• SSH host key not trusted or changed. The argocd-ssh-known-hosts-cm ConfigMap lacks the host, or the Git provider rotated its host key, producing knownhosts: key mismatch.
• Credential template does not match the repo URL. Argo CD matches credentials by URL prefix; an https:// credential will not apply to an ssh:// (or differently-cased) remote.
• Private Helm/OCI or chart dependency unreachable. A Helm chart pulls a dependency from a registry the repo-server cannot authenticate to, so manifest generation — not the Git clone — fails.
• Repo-server cannot reach the host at all. Egress firewall, proxy, or DNS failure from the argocd-repo-server pod.
• Manifest tooling failure masquerading as access. helm template, kustomize build, or a config-management plugin errors, and the wrapped message reads like a generation/access failure.

Diagnostic Workflow

Read the exact condition on the Application — the wrapped message tells you auth vs host-key vs generation:

argocd app get my-app
kubectl -n argocd get application my-app -o jsonpath='{.status.conditions}' | jq

Go straight to the repo-server logs, which do the cloning and templating:

kubectl -n argocd logs deploy/argocd-repo-server --tail=100 | grep -iE 'error|auth|knownhosts|refs'

List the repositories Argo CD knows about and their connection state:

argocd repo list
argocd repo get https://github.com/acme/platform-manifests.git

Confirm a matching credential secret actually exists and its type/URL line up:

kubectl -n argocd get secret -l argocd.argoproj.io/secret-type=repository
kubectl -n argocd get secret repo-acme-platform -o jsonpath='{.data.url}' | base64 -d; echo

For SSH remotes, verify the host key is present and reproduce the handshake from inside the repo-server pod:

kubectl -n argocd get cm argocd-ssh-known-hosts-cm -o jsonpath='{.data.ssh_known_hosts}' | grep github
kubectl -n argocd exec deploy/argocd-repo-server -- \
  sh -c 'ssh-keyscan github.com 2>/dev/null'   # compare fingerprint to the ConfigMap

Test raw egress and DNS from the pod to rule out network:

kubectl -n argocd exec deploy/argocd-repo-server -- \
  sh -c 'getent hosts github.com; nc -zv github.com 443'

Example Root Cause Analysis

A platform team rotated the GitHub deploy key used by Argo CD but only updated it in their password manager, not in the cluster. Within minutes every Application backed by that private repo flipped to Unknown with ComparisonError: ... authentication required: Repository not found. Auto-sync silently stopped; a config change merged that afternoon never shipped.

argocd-repo-server logs showed failed to list refs: authentication required for the SSH remote. kubectl get secret -l argocd.argoproj.io/secret-type=repository showed the repo-acme-platform secret still held the old private key, and argocd repo get reported the connection as Failed. The host key itself was fine — ssh-keyscan from inside the pod matched argocd-ssh-known-hosts-cm.

The fix was to update the repository secret’s sshPrivateKey with the new deploy key and restart the repo-server so it dropped its cached clone. Within one reconcile the Applications recomputed desired state, returned to OutOfSync, and the pending change synced. The team then moved the deploy key into External Secrets so rotation updates the cluster automatically, and added an alert on ComparisonError so a broken credential is caught in minutes rather than at the next incident.

Prevention Best Practices

• Alert on ComparisonError and Unknown status. An Application that cannot compute desired state is worse than one that is OutOfSync — it fails open and stops shipping. Monitor argocd_app_info for health_status="Unknown".
• Manage repo credentials as rotated secrets. Store deploy keys / PATs in External Secrets or Sealed Secrets so rotation propagates to the cluster instead of living only in a human’s vault.
• Match credential templates to the exact repo URL scheme. Keep https and ssh credentials consistent with the spec.source.repoURL you actually use.
• Pin and pre-load SSH host keys. Populate argocd-ssh-known-hosts-cm and update it deliberately when a provider announces a host-key rotation.
• Give the repo-server reachable egress. Ensure firewall/proxy/DNS from the argocd-repo-server pod to every Git host and chart registry it must reach.
• Validate manifest tooling in CI. Run helm template / kustomize build in CI so a generation error is caught before it becomes a comparison failure in the cluster.

Quick Command Reference

# Why is the app not comparing?
argocd app get my-app
kubectl -n argocd get application my-app -o jsonpath='{.status.conditions}' | jq

# The component that clones and templates
kubectl -n argocd logs deploy/argocd-repo-server --tail=100 | grep -iE 'error|auth|refs|knownhosts'

# Known repos and their connection state
argocd repo list
argocd repo get <repoURL>

# Credentials present and correctly typed?
kubectl -n argocd get secret -l argocd.argoproj.io/secret-type=repository

# SSH host key present + matches the provider
kubectl -n argocd get cm argocd-ssh-known-hosts-cm -o yaml | grep -i <host>
kubectl -n argocd exec deploy/argocd-repo-server -- ssh-keyscan <host>

# Network reachability from the repo-server
kubectl -n argocd exec deploy/argocd-repo-server -- sh -c 'getent hosts <host>; nc -zv <host> 443'

Conclusion

ComparisonError: repository not accessible means Argo CD’s repo-server cannot clone or template the source, so it never learns the desired state and the Application fails open to Unknown. The message almost always narrows to one of three causes — an expired or missing credential, an untrusted or rotated SSH host key, or a manifest-generation/network failure. Read the wrapped error in the repo-server logs, confirm the matching repository secret and host key, and test egress from the pod. Then close the loop for good: manage repo credentials as auto-rotated secrets and alert on ComparisonError, so a broken credential surfaces in minutes instead of as silent, accumulating drift.