--- title: Vault, Consul and Nomad Monitoring: Key Metrics and Setup description: Monitoring Vault, Consul and Nomad means turning on each tool's telemetry, scraping its HTTP metrics endpoint with Prometheus, and alerting on Raft leadership,... canonical: https://daily.dev/posts/vault-consul-and-nomad-monitoring-key-metrics-and-setup-rt8pwstjg twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Vault, Consul and Nomad Monitoring: Key Metrics and Setup | daily.dev og:description: Monitoring Vault, Consul and Nomad means turning on each tool's telemetry, scraping its HTTP metrics endpoint with Prometheus, and alerting on Raft leadership,... og:url: https://daily.dev/posts/vault-consul-and-nomad-monitoring-key-metrics-and-setup-rt8pwstjg og:image: https://api.daily.dev/og/posts/rT8pwstjg.png og:image:alt: Vault, Consul and Nomad Monitoring: Key Metrics and Setup og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Vault, Consul and Nomad Monitoring: Key Metrics and Setup **[Last9](https://daily.dev/sources/last9)** · 12 min read · 0 upvotes · 0 comments ## Summary Monitoring Vault, Consul and Nomad means turning on each tool's telemetry, scraping its HTTP metrics endpoint with Prometheus, and alerting on Raft leadership, commit latency, throughput, and Go runtime memory, plus one signal per tool. Vault serves metrics at /v1/sys/metrics (not /metrics) and needs a token with read and list capabilities; standby nodes return no metrics unless you set unauthenticated_metrics_access = true. Consul's Prometheus output is off by default until prometheus_retention_time exceeds 0s, and the format=prometheus flag must be passed as a parameter, not in the path, since '?format=prometheus' gets escaped. Nomad needs prometheus_metrics set to true, and publish_allocation_metrics and publish_node_metrics add per-allocation and per-node resource metrics, and its /v1/metrics endpoint needs no ACL token. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### why is Prometheus only scraping metrics from my active Vault node and not the standbys The /v1/sys/metrics endpoint is only accessible on active nodes and is automatically disabled on standby nodes by default. On Vault Community edition, a standby redirects the metrics request to the leader instead of serving its own data. To fix this, set unauthenticated_metrics_access = true in the listener's telemetry block so each standby responds with its own metrics, though this exposes the endpoint without a token. _Standing up Vault HA monitoring? Track setup gotchas like this one on daily.dev._ ### why doesn't format=prometheus work when I add it to the Consul metrics path Passing ?format=prometheus directly in the metrics path does not work because the question mark gets escaped. Consul serves metrics at /v1/agent/metrics and needs a token with agent:read, but format must be passed as a separate query parameter in the Prometheus scrape config rather than appended to the metrics_path string. Also set telemetry.prometheus_retention_time above its default of 0s, or Prometheus export stays disabled. _Wiring up Consul telemetry? Keep scrape-config fixes like this one handy on daily.dev._ ### what Nomad metrics show that jobs are stuck and cannot be scheduled Watch nomad.nomad.blocked_evals.total_blocked, which counts evaluations blocked by cluster resource exhaustion or quota limits, alongside nomad.nomad.plan.queue_depth for plans waiting to be evaluated and nomad.client.unallocated.memory for free scheduler memory. A rising blocked count combined with falling unallocated memory usually means the cluster needs more capacity. _Debugging stuck Nomad jobs? Save capacity-signal metrics like these on daily.dev._ ## Similar posts on daily.dev - [Kubernetes Monitoring Tools: What Actually Works at Scale](https://daily.dev/posts/kubernetes-monitoring-tools-what-actually-works-at-scale-zdg0bbpod) · Last9 · 2 upvotes · 0 comments - [Who Watches the Watchmen? Tales from Our Monitoring Meltdown](https://daily.dev/posts/who-watches-the-watchmen-tales-from-our-monitoring-meltdown-xkez9h7ep) · Medium · 1 upvotes · 0 comments - [Building a Kubernetes Platform: What You Need to Monitor in K8s & Why](https://daily.dev/posts/building-a-kubernetes-platform-what-you-need-to-monitor-in-k8s-why-wpwseyzn9) · Fairwinds Blog · 0 upvotes · 0 comments - [Kubernetes Metrics: Types, Tools, & Monitoring Guide](https://daily.dev/posts/kubernetes-metrics-types-tools-monitoring-guide-vfxngka7f) · Spacelift · 4 upvotes · 0 comments - [Multi-Cluster Monitoring: Stop Managing Configs, Start Automating Infrastructure](https://daily.dev/posts/multi-cluster-monitoring-stop-managing-configs-start-automating-infrastructure-tromcghtc) · ITNEXT · 0 upvotes · 0 comments --- Tags: [#observability](https://daily.dev/tags/observability), [#prometheus](https://daily.dev/tags/prometheus), [#hashicorp](https://daily.dev/tags/hashicorp) [View this post on daily.dev](https://daily.dev/posts/vault-consul-and-nomad-monitoring-key-metrics-and-setup-rt8pwstjg) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Vault, Consul and Nomad Monitoring: Key Metrics and Setup","url":"https://daily.dev/posts/vault-consul-and-nomad-monitoring-key-metrics-and-setup-rt8pwstjg","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/vault-consul-and-nomad-monitoring-key-metrics-and-setup-rt8pwstjg"},"datePublished":"2026-10-29T18:51:25.632Z","dateModified":"2026-10-29T19:03:12.096Z","description":"Monitoring Vault, Consul and Nomad means turning on each tool's telemetry, scraping its HTTP metrics endpoint with Prometheus, and alerting on Raft leadership,...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/db6cfaf759462f9fec5396a458474861?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/db6cfaf759462f9fec5396a458474861?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Last9","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Last9","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/0542036d401f4cb1885680f1984fb880","url":"https://daily.dev/sources/last9"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/vault-consul-and-nomad-monitoring-key-metrics-and-setup-rt8pwstjg","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"observability,prometheus,hashicorp","timeRequired":"PT12M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Last9","item":"https://daily.dev/sources/last9"},{"@type":"ListItem","position":3,"name":"Vault, Consul and Nomad Monitoring: Key Metrics and Setup"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/vault-consul-and-nomad-monitoring-key-metrics-and-setup-rt8pwstjg#faq","mainEntity":[{"@type":"Question","name":"why is Prometheus only scraping metrics from my active Vault node and not the standbys","acceptedAnswer":{"@type":"Answer","text":"The /v1/sys/metrics endpoint is only accessible on active nodes and is automatically disabled on standby nodes by default. On Vault Community edition, a standby redirects the metrics request to the leader instead of serving its own data. To fix this, set unauthenticated_metrics_access = true in the listener's telemetry block so each standby responds with its own metrics, though this exposes the endpoint without a token. Standing up Vault HA monitoring? Track setup gotchas like this one on daily.dev."}},{"@type":"Question","name":"why doesn't format=prometheus work when I add it to the Consul metrics path","acceptedAnswer":{"@type":"Answer","text":"Passing ?format=prometheus directly in the metrics path does not work because the question mark gets escaped. Consul serves metrics at /v1/agent/metrics and needs a token with agent:read, but format must be passed as a separate query parameter in the Prometheus scrape config rather than appended to the metrics_path string. Also set telemetry.prometheus_retention_time above its default of 0s, or Prometheus export stays disabled. Wiring up Consul telemetry? Keep scrape-config fixes like this one handy on daily.dev."}},{"@type":"Question","name":"what Nomad metrics show that jobs are stuck and cannot be scheduled","acceptedAnswer":{"@type":"Answer","text":"Watch nomad.nomad.blocked_evals.total_blocked, which counts evaluations blocked by cluster resource exhaustion or quota limits, alongside nomad.nomad.plan.queue_depth for plans waiting to be evaluated and nomad.client.unallocated.memory for free scheduler memory. A rising blocked count combined with falling unallocated memory usually means the cluster needs more capacity. Debugging stuck Nomad jobs? Save capacity-signal metrics like these on daily.dev."}}]} ```