---
title: Guide to HTTP Response Header Monitoring
date: 2026-10-09
description: How to monitor HTTP response headers like X-Robots-Tag, Content-Type, and HSTS, with examples based on the header checks real sites run.
author: Rudi Kraeher
categories: [Guides]
---
import Callout from '../../components/post/Callout.astro';
import Check from '../../components/post/Check.astro';
import BlogCTA from '../../components/post/BlogCTA.astro';
import CopyCode from '../../components/post/CopyCode.astro';
import RelatedArticleCta from '../../components/post/RelatedArticleCta.astro';
Of over 1,300 real response header monitoring rules we analyzed, the most prominent rule we found asserts the absence of one word: `noindex`.
That one rule tells Google to drop the page from search results, without any visible change to the page itself.
Among the many header checks that our users are running, the biggest missed opportunity we found is monitoring security headers.
Almost no one checks them.
This article covers the header checks teams already rely on as well as the ones worth adding, starting with the basics of HTTP headers.
## What are HTTP headers?
[HTTP headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers) are name-value pairs that let the client and the server
pass additional information with a request or a response. When a browser or a monitoring tool asks for a page, it sends headers along with the request,
and the server answers with headers of its own before it sends the page content.
Each header sits on its own line. It starts with a case-insensitive name, followed by a colon and a value, for example `Content-Type: text/html; charset=utf-8`.
That specific header tells the client how to read the body. In this case, it says that the body is an HTML document and uses UTF-8 character encoding.
Here is part of the head testomato.com returns for its homepage:
```http
HTTP/2 200
content-type: text/html; charset=utf-8
vary: Accept, Accept-Encoding
x-frame-options: SAMEORIGIN
x-content-type-options: nosniff
strict-transport-security: max-age=31540000
```
The first line is the status line, with the protocol version (HTTP/2) and the three-digit [status code](/blog/http-status-codes/).
After that come all of the response headers.
### What is the purpose of HTTP headers?
Headers carry instructions and context for the body content. With headers, a server can tell clients and proxies
what to cache and for how long, restrict pages to specific users, keep a session across requests with cookies,
and allow a page to load resources from other domains.
Headers also make HTTP easy to extend. New functionality can be introduced when a client and a server agree on what a new header means,
without any change to the protocol.
### Request headers vs response headers
[Request headers](https://developer.mozilla.org/en-US/docs/Glossary/Request_header) describe the client's preferences and context.
They tell the server what is being requested and who is asking, for example which browser sends the request,
which formats it accepts, or which credentials it has.
[Response headers](https://developer.mozilla.org/en-US/docs/Glossary/Response_header) carry the server's decisions and instructions.
They describe the response and the server that sent it, for example the format of the content, how it should be cached,
or where a redirect points.
Here is how the headers look when you request testomato.com with [curl](https://curl.se/docs/manpage.html).
**Request headers** are indicated with `>`, and **response headers** with `<`.
```http
> GET / HTTP/2
> Host: testomato.com
> User-Agent: curl/8.7.1
> Accept: */*
>
< HTTP/2 200
< server: nginx/1.31.6
< content-type: text/html; charset=utf-8
< vary: Accept
```
A monitoring check works on both sides of this exchange. It sends a request, sometimes with custom headers you add yourself, and checks the response headers that come back.
### What are the most commonly used HTTP headers?
MDN's [header reference](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers) groups headers by function, from authentication and caching
to cookies, CORS, and security. For monitoring a website, the headers that matter most fall into a few use cases:
- **Content type and encoding**: `Content-Type` tells the browser what the content is and which character set to decode it with.
- **Performance**: `Cache-Control`, `Content-Encoding`, and `Vary` control caching and compression.
- **Redirects**: `Location` holds the destination of a redirect.
- **SEO**: `X-Robots-Tag` and `Link` control how search engines index and canonicalize a page.
- **Security**: `Content-Security-Policy`, `Strict-Transport-Security`, and `X-Frame-Options` protect visitors from attacks.
We cover each of these headers in detail in [Important response headers](#important-response-headers).
### How to check HTTP headers
The quickest way is our [HTTP headers checker](/tools/http-inspect/). Enter a URL, optionally add your own request headers,
and it returns the status code, the response headers, and a preview of the body. The request is proxied through our servers,
so it's especially useful to check responses to requests made from outside your network and when setting up or debugging
your monitoring checks.
To check headers via the command line, you can use curl. To see both the request headers curl sends and
the response headers it gets back, run:
```bash
curl -v https://example.com
```
To print only the response headers, run:
```bash
curl -s -D - -o /dev/null https://example.com
```
## Why monitor your HTTP response headers?
Headers can change for many reasons, for example after a server migration, a CMS or plugin update, a new CDN rule, or a deploy that ships the wrong settings
to production.
Since they are not visible on the page, you need to be watching the headers themselves to see if they change.
A [monitoring check](/blog/what-is-website-monitoring/) requests the page on a schedule, compares the response headers against what you expect, and alerts you when they stop matching.
Below are the core use cases we've extrapolated from our data — what real professionals look for in their HTTP response headers.
### Content type and encoding
The `Content-Type` header tells the browser what format a page is in and how to decode its text. If the charset is missing or wrong,
the browser guesses, and umlauts can come out garbled. If the format is wrong, the browser can refuse the file.
Monitor `Content-Type` so you know when your pages stop being served in the format and encoding you expect.
### Performance
Response headers control how your pages are compressed, which affects [how fast they load](/blog/response-time-monitoring/).
Whether a server compresses a response depends on its configuration, so a server can turn compression off without any change to the page content.
Monitor `Content-Encoding` so you know when your compression settings change.
### Redirects
The `Location` response header holds the destination of a redirect. Checking the status code alone confirms that a redirect exists, but not where it goes.
Monitoring your redirect location headers can help you identify [redirect loops](/blog/redirect-monitoring/#redirect-loops) or incorrect destinations.
While it is possible to check the `Location` header, Testomato has a dedicated HTTP Redirect Location check for this use case.
### SEO
Several response headers tell search engines whether to index a page, which URL is the canonical, and which language version to show.
#### [**Indexing**](https://developers.google.com/search/docs/fundamentals/how-search-works#indexing)
Indexing is the step after crawling in which a search engine
analyzes a page's content, picks a canonical version among duplicates, and decides whether to store it in its index, the database it serves search results from.
The `X-Robots-Tag` header can carry indexing rules such as `noindex`.
When Googlebot finds `noindex`, Google drops the page entirely from search results, regardless of whether other sites link to it.
The same index-blocking rule can also come from a robots meta tag in the HTML, which we cover in our [meta tags guide](/blog/meta-tags-monitoring/#robots-crawling-and-indexing).
A common way `noindex` reaches production is a configuration meant for a staging or preview site that isn't meant to rank. Monitoring this header
ensures that you'll be alerted immediately if the wrong indexing rule accidentally gets deployed to prod.
#### [**Canonicalization**](https://developers.google.com/search/docs/crawling-indexing/canonicalization)
Canonicalization is how a search engine picks one URL to represent a set of duplicate pages. Besides the canonical tag in the HTML, a canonical can also be declared in the `Link` response header.
If the canonical starts pointing to the wrong URL, for example a staging domain, the search engine
consolidates ranking signals onto the wrong target.
Read more about duplicate content and [canonical links](/blog/meta-tags-monitoring/#canonical-link-managing-duplicate-content) in our meta tags guide.
#### [**Localization**](https://developers.google.com/search/docs/specialty/international/localized-versions)
Localization with [hreflang](/blog/meta-tags-monitoring/#hreflang-link-international-targeting) connects the language and regional versions of a page, so search engines can show
the most fitting URL for the searcher's language. For non-HTML files, `hreflang` goes in the `Link` header.
SEO rules are the largest group of header rules in our data: more than five hundred rules in over a hundred projects,
and 98% of them assert that `noindex` or `nofollow` is **absent**.
You should monitor your SEO headers because a wrong value can block indexing without any change to the page itself.
### Security
Security headers tell the browser how to protect your visitors from attacks such as cross-site scripting and clickjacking.
Surprisingly, many sites haven't added them at all.
According to the [2025 Web Almanac](https://almanac.httparchive.org/en/2025/security),
`Content-Security-Policy` (CSP) adoption is at 21.9%, and only 36% of mobile pages send the HTTP `Strict-Transport-Security` (HSTS) header, though adoption
is steadily increasing.
Consistent with HTTP Archive's findings, our own data shows few active monitoring rules checking site security headers —
if there are no headers sent, then there is nothing to check. But CSP and HSTS can be monitored like any other headers.
CSP has a suite of dedicated Testomato checks, covered in our [CSP guide](/blog/content-security-policy-monitoring/).
If you don't already have a content security policy, use our free [CSP Generator](/tools/csp-generator/) tool,
which crawls your live site and bootstraps a configurable, strict CSP.
If you do have some security headers deployed, scanners like MDN's [HTTP Observatory](https://developer.mozilla.org/en-US/observatory)
assess your headers and report what to fix. Whether you are improving existing headers or starting from scratch,
after you implement security headers make sure you are monitoring them.
## Important response headers
This section covers the specific headers behind each use case above —
what each one does and what warnings or errors to watch out for.
### Content type and encoding headers
#### [Content-Type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Content-Type)
Content-Type gives the [MIME type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/MIME_types) of the body and, for text,
its [charset](https://developer.mozilla.org/en-US/docs/Glossary/Character_encoding). Without a charset, browsers
fall back to a default encoding and non-ASCII characters can render as [mojibake](https://en.wikipedia.org/wiki/Mojibake).
```http
Content-Type: text/html; charset=utf-8
```
**Watch out for:** a missing or different charset. Charset names are case-insensitive, so `UTF-8` and `utf-8` are considered the same value.
To fix garbled text, set the charset explicitly.
### Performance headers
#### [Cache-Control](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Cache-Control)
Cache-Control tells browsers and shared caches whether to store a response and for how long. `max-age` sets the lifetime in seconds,
`no-cache` allows storing but requires revalidation before reuse,
and `no-store` forbids storing. While the cache lifetime lasts, browsers use their local copy instead of downloading the file again,
which speeds up repeat visits.
```http
Cache-Control: max-age=31536000
```
**Watch out for:** Lighthouse's [Use efficient cache lifetimes](https://developer.chrome.com/docs/performance/insights/cache) insight,
which flags fonts, images, media, scripts, and stylesheets with a cache lifetime under 30 days.
Consider giving static assets that don't change a long lifetime, such as `max-age=31536000` (one year).
#### [Content-Encoding](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Content-Encoding)
Content-Encoding lists the compression applied to the body,
such as `gzip`, `br`, or `zstd`. The server picks from the methods the client offers in `Accept-Encoding`.
```http
Content-Encoding: gzip
```
**Watch out for:** `ERR_CONTENT_DECODING_FAILED`, the browser error when the body doesn't match the declared encoding.
To check, request the URL with `curl --compressed`, which reports an error
on a mismatch, then look at each layer that can compress the response, from your application to a proxy or CDN.
#### [Vary](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Vary)
Vary lists the request headers that influenced the response, so caches keep a separate copy for each value and serve the right version to each client.
If your server compresses responses based on the client's `Accept-Encoding`, list that header:
```http
Vary: Accept-Encoding
```
**Watch out for:** a URL that serves different formats through [content negotiation](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Content_negotiation)
without listing the deciding request header in Vary. For example, a page that serves different file formats to different user agents needs [Vary: Accept](#accept),
or a cache can serve the wrong version.
### Redirect headers
#### [Location](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Location)
The Location header gives the URL to redirect to. Browsers only follow it on a [3xx redirect](/blog/http-status-codes/#3xx--redirects), or, after a [201 Created](/blog/http-status-codes/#201-created) response,
read it as the address of the new resource.
The URL value can be relative or absolute.
```http
HTTP/1.1 301 Moved Permanently
Location: https://www.example.com/new-page/
```
**Watch out for:** Search Console's "Redirect error". Our redirect guide covers [how to fix chains and loops](/blog/redirect-monitoring/#how-to-fix-redirect-chains-and-loops),
and how to use the HTTP Redirect Location check.
### SEO headers
#### [Link](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Link)
The Link header lists links from the response to related resources, the way `` elements do in an HTML head.
For SEO, it can declare a canonical URL or language alternates.
```http
Link: ; rel="canonical"
```
**Watch out for:** when Google picks a different canonical than the one you declared, Search Console reports
"Duplicate, Google chose different canonical than user". To fix it, inspect the URL
to compare your declared canonical with Google's choice, and make sure your `Link` header and HTML don't declare different ones.
#### [X-Robots-Tag](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/X-Robots-Tag)
X-Robots-Tag accepts any rule that a [robots meta tag](https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag) does,
providing a way to give instructions to crawlers for non-HTML documents, like PDFs and images.
```http
X-Robots-Tag: noindex, nofollow
```
**Watch out for:** Search Console's URL Inspection tool reporting
"Indexing allowed? No: 'noindex' detected in 'X-Robots-Tag' http header". If the page should be indexed, remove the rule and request a recrawl with the same tool.
### Security headers
#### [Content-Security-Policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Content-Security-Policy)
Content-Security-Policy (CSP) tells the browser which sources a page may load scripts, styles, images, and other resources from.
Our [CSP guide](/blog/content-security-policy-monitoring/) covers it in depth.
```http
Content-Security-Policy: default-src 'self'; frame-ancestors 'none'
```
**Watch out for:** console errors that say a resource "…violates the following Content Security Policy directive…". Either add the source to the directive named in the error
or remove the resource, and try policy changes in [report-only mode](/blog/content-security-policy-monitoring/#content-security-policy-report-only) first.
#### [Cross-Origin-Opener-Policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Cross-Origin-Opener-Policy)
Cross-Origin-Opener-Policy (COOP) keeps your page from sharing a browsing context with cross-origin windows. When a cross-origin site opens
your page, the references between the two windows are cut, so the opener can't reach into your page. This protects against
[cross-site leaks](https://xsleaks.dev/) (XS-Leaks) and Spectre attacks.
```http
Cross-Origin-Opener-Policy: same-origin
```
**Watch out for:** a blank or missing popup when visitors sign in or pay through a cross-origin service, such as an OAuth or payment provider.
`same-origin` cuts the connection to popups your page opens, which breaks communication between the windows. Use `same-origin-allow-popups` on pages that open these services.
#### [Cross-Origin-Resource-Policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Cross-Origin-Resource-Policy)
Cross-Origin-Resource-Policy (CORP) limits which sites can load your resources. This protects against [Spectre-style attacks](https://spectreattack.com/),
because the browser blocks the response before it reaches the other site's page.
```http
Cross-Origin-Resource-Policy: same-origin
```
**Watch out for:** `ERR_BLOCKED_BY_RESPONSE.NotSameOrigin`, which appears when another site loads a resource with a
same-origin policy. If your own subdomains need to load it, use `same-site`; if other websites should be able to, use `cross-origin`.
#### [Permissions-Policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Permissions-Policy)
Permissions-Policy allows or denies browser features such as the camera or geolocation, for the page and the frames it embeds.
An empty list, `()`, turns a feature off everywhere. Chrome and Edge support it, but Firefox and Safari don't, so it only protects visitors on Chromium-based browsers.
```http
Permissions-Policy: geolocation=(), camera=(), microphone=()
```
**Watch out for:** console messages like "[Violation] Permissions policy violation: camera is not allowed in this document."
If something on the page needs a restricted feature, allow it for that origin, for example `camera=(self "https://video.example.com")`.
#### [Referrer-Policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Referrer-Policy)
Referrer-Policy controls how much of the current URL the browser sends in the `Referer` header. Without one, browsers
default to `strict-origin-when-cross-origin`, which sends only the origin to other sites and nothing from HTTPS to HTTP.
If you set a stricter policy, such as `no-referrer`, and the header disappears, browsers fall back to the default and send more of your URL to other sites.
```http
Referrer-Policy: strict-origin-when-cross-origin
```
**Watch out for:** an empty or invalid value, which Chrome rejects with "Failed to set referrer policy." In such cases, Chrome ignores the value and
keeps its default policy. Fix the value rather than removing the header, and set it explicitly rather than relying on browser defaults.
#### [Server](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Server) and [X-Powered-By](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/X-Powered-By)
These two headers should stay absent or generic. They can reveal the software and version behind your site.
According to the [Web Almanac](https://almanac.httparchive.org/en/2025/security), `Server` appears on 91.5% of hosts and `X-Powered-By` on 23.9%,
partly because many web technologies set them automatically. That also means an update or a new server can add them back.
```http
X-Powered-By: PHP/7.4.33
```
**Watch out for:** security scanners often flag version numbers in these headers.
Hide the version or remove the header, if possible. Check your stack's documentation.
#### [Strict-Transport-Security](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Strict-Transport-Security)
HTTP Strict-Transport-Security (HSTS) tells browsers to reach the host only over HTTPS. Once a browser has seen the header,
it upgrades later HTTP requests to HTTPS and doesn't let visitors click through certificate errors. Without the header,
new visitors don't get that protection, and returning visitors lose it once their stored `max-age` runs out.
```http
Strict-Transport-Security: max-age=31536000; includeSubDomains
```
**Watch out for:** `max-age=0`, which disables HSTS. Browsers also ignore the header when it's sent over plain HTTP.
Send it over HTTPS with a positive `max-age`.
#### [X-Content-Type-Options](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/X-Content-Type-Options)
X-Content-Type-Options set to `nosniff` tells the browser to trust the `Content-Type` header instead of guessing the type from the content.
With it, the browser blocks scripts and stylesheets whose MIME type doesn't match. Without it, browsers can guess a file's content type.
If a file served from your site, such as an uploaded image, contains valid HTML markup, some browsers treat it as a document
and run its scripts, which makes your site vulnerable to cross-site scripting.
```http
X-Content-Type-Options: nosniff
```
**Watch out for:** a script URL that returns HTML, which the browser refuses with "Refused to execute script from URL because its MIME type is not executable,
and strict MIME type checking is enabled." Fix the script URL, or serve the file with a JavaScript MIME type such as `text/javascript`.
#### [X-Frame-Options](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/X-Frame-Options)
X-Frame-Options controls whether other pages can embed yours in a frame, which
protects against clickjacking. It only works as an HTTP header; setting it in a `` tag has no effect.
```http
X-Frame-Options: SAMEORIGIN
```
**Watch out for:** "Refused to display URL in a frame because it set 'X-Frame-Options' to 'sameorigin'", which appears when a page
from another origin tries to embed yours. To allow specific other sites, use the CSP `frame-ancestors` directive instead.
## Custom request headers
That covers the headers your server sends back. The headers in this section go the other way: they're request headers you can add
to your own checks.
By default, a check requests a page the way an anonymous visitor would, so some pages need an extra header
before the check sees what you want to monitor, such as an API endpoint that requires a token.
In Testomato, you can add custom headers to your checks or to a whole Group.
### [Accept](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Accept)
Accept lists the content types the client can understand. The server picks one through content negotiation and names its choice
in the `Content-Type` response header.
```http
Accept: text/markdown
```
Send it to check a version of a page that the server only returns for a specific type, such as Markdown for AI agents or JSON from an API.
An Accept header in the extra headers replaces the check's default.
Content negotiation is also how sites serve Markdown to AI agents, an emerging convention. The agent sends `Accept: text/markdown`,
the server answers with `Content-Type: text/markdown`, and `Vary: Accept` keeps the HTML and Markdown versions apart in caches.
Test your own site with our [Markdown for Agents checker](/tools/accept-markdown/).
### API key headers
Many APIs expect a key in a custom request header. The name depends on the API, for example,
[AWS API Gateway](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-api-key-call.html) expects `X-API-Key`.
```http
X-API-Key: your-api-key
```
Send it to monitor an API endpoint the same way you monitor a page. Check the API's documentation for the exact header name.
### [Authorization](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Authorization)
Authorization carries credentials that give the client access to a protected resource. With Basic authentication, the value is `Basic`
followed by an encoded `username:password`.
```http
Authorization: Basic dXNlcjpwYXNzd29yZA==
```
Send it to monitor a staging site or another page behind HTTP authentication, so you can run the same checks there as on production.
If several checks use the same credentials, set the header once on their Group.
### [Cache-Control](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Cache-Control)
As a request header, Cache-Control asks caches between the check and your server how to handle the request.
```http
Cache-Control: no-cache
```
A monitoring check can get a cached copy instead of the live page. "Cache busting" forces a fresh response, for example by sending
`Cache-Control: no-cache` as a request header, which asks caches to check with the origin server before reusing a stored response.
In Testomato, the "Add no-cache headers" button in the check editor adds a set of cache busting headers for you.
### [Cookie](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Cookie)
Cookie sends cookies stored for the site, set earlier by the server with `Set-Cookie` or by JavaScript with `document.cookie`.
```http
Cookie: session=abc123
```
Send it when a page only shows certain content to visitors who have a specific cookie.
### [User-Agent](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/User-Agent)
User-Agent identifies the application making the request. Testomato's checks identify themselves as [Testomatobot](/bot/).
```http
User-Agent: Testomatobot/1.0 (Linux x86_64; +https://testomato.com/bot) minicrawler/5.2.7
```
Some servers and firewalls treat requests differently based on this header. Send a browser's User-Agent to see the page your visitors see.
## Monitoring headers with Testomato
This section shows how to set up header monitoring in Testomato. It starts with the use cases that already have dedicated checks,
then walks through example checks for compression, HSTS, and an API with a bearer token.
If you just want to know which headers to watch, check the final section for a list of [headers to monitor first](#what-to-monitor-first).
### Dedicated checks
Three of the use cases above have their own checks in Testomato, so you don't need a generic HTTP Response Header check for them.
**Content-Security-Policy:** the [CSP check](/blog/content-security-policy-monitoring/#how-testomato-monitors-csp) and its per-directive checks read the policy from the header.
**Redirects:** the [HTTP Redirect Location check](/blog/redirect-monitoring/#how-to-set-up-redirect-checks) asserts where a redirect points.
**SEO:** on the "Add check" page, after loading the page URL, you can use the SEO preset buttons to add a set of SEO checks,
including one for the `X-Robots-Tag` header.
There are two robots-related checks that are included when you use the presets.
The Robots Meta & Link check **does not match** the following regex:
And the HTTP Response Header check **does not match**:
Both checks fail on `noindex` or `none`, which Google treats as `noindex, nofollow`. One looks in the robots meta tag,
the other in the `X-Robots-Tag` header.
### Compression
[testomato.com](https://testomato.com/) compresses its pages with gzip, so a check on the homepage can assert it
by using an HTTP Response Header check that **matches pattern**:
Testomato's checks tell the server they accept `gzip`, the format they can decompress, so expect `gzip` even if your server
sends Brotli (`br`) or something else.
Servers can send `Content-Encoding` or `content-encoding`. Starting the pattern with `(?i)` makes it match either way.
### HSTS
Our site sends `strict-transport-security: max-age=31540000`, so a check on the homepage can assert that HSTS stays on
by using an HTTP Response Header check that **matches pattern**:
Checking only that the header contains `strict-transport-security` isn't sufficient, because `max-age=0` turns HSTS off while the header
is still present. So we include `[1-9]` in our regex pattern to make the check fail on `max-age=0`.
### API with a bearer token
Many APIs only answer requests that carry a token in the Authorization header. The [Testomato API](https://help.testomato.com/api/) functions this way.
It takes a [personal access token](/blog/personal-access-tokens/) as a bearer token.
To monitor an endpoint like this, add the token to the check's Extra HTTP headers field:
```http
Authorization: Bearer
```
Then add an HTTP Status Code check that equals `200`, with the check pointed at an endpoint such as
`https://testomato.com/api/project/{project-id}/simple-status`.
Without a valid token, the endpoint answers [401](/blog/http-status-codes/#401-unauthorized), so the check fails when the token expires or is revoked.
The same setup works for any API that takes a bearer token.
{/*
TODO: Add this back once we have published the Group feature's help documentation:
For a page behind HTTP Basic authentication, you don't need a header. Use the Group's Protected by authentication setting instead.
*/}
### What to monitor first
1. `X-Robots-Tag` without `noindex`, using the SEO basics preset.
2. `Content-Type` with the charset you expect.
3. `Location` on your key redirects, using the HTTP Redirect Location check.
4. `Content-Encoding` and `Vary: Accept-Encoding` on compressed pages.
5. `Cache-Control` on static assets.
6. `Content-Security-Policy`, using the Content-Security-Policy check.
7. The security headers web.dev [recommends for all websites](https://web.dev/articles/security-headers).
8. `Referrer-Policy`, set explicitly.
9. `Server` and `X-Powered-By` without version numbers.
## Resources
- **[HTTP Observatory](https://developer.mozilla.org/en-US/observatory)** — MDN's scanner that grades a site's security headers
- **[HTTP Response Inspector](/tools/http-inspect/)** — our tool that shows the status code and response headers for any URL
- **[http.dev](https://http.dev/)** — a reference covering HTTP headers, status codes, and related concepts
- **[OWASP Security Headers Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/HTTP_Headers_Cheat_Sheet.html)** — recommended values for security headers
Monitor your HTTP response headers
{/*
NOTES
- Add screenshots, compressed
- Since this is part of the website monitoring pillar, we should probably add a subsection to that article about headers and link to this article.
*/}