Find dead-links (broken links)
Documentation • Installation • Github Action • Contributing • Changelog
Dead link (broken link) means a link within a web page that cannot be connected. These links can have a negative impact to SEO and Security. This tool makes it easy to identify and modify.
Looking for v1 (Ruby gem)? It now lives on the
legacy/v1branch and continues to publish thedeadfindergem for bug-fix and security releases.mainhosts the Crystal rewrite (v2+).
Installation
Homebrew
brew install deadfinder
# https://formulae.brew.sh/formula/deadfinderChocolatey (Windows)
Starting with the release after 2.1.0:
choco install deadfinderDocker
docker run ghcr.io/hahwul/deadfinder:latest deadfinder url https://example.comPrebuilt binary
Download the archive for your platform from the latest release, extract, and place deadfinder (deadfinder.exe on Windows) on your PATH.
Nix
nix run github:hahwul/deadfinder
nix profile install github:hahwul/deadfinder
nix develop github:hahwul/deadfinderBuild from source
Requires Crystal >= 1.19.1 and cmake (for the lexbor HTML parser's postinstall — without it shards install fails with 'cmake': No such file or directory).
# macOS
brew install crystal cmake
# Debian / Ubuntu
sudo apt install crystal cmakeshards install
crystal build src/cli_main.cr -o deadfinder --release
# or: just buildUsing In
CLI
deadfinder sitemap https://www.hahwul.com/sitemap.xmlGitHub Action
Pin a specific release tag. @latest is not a valid Actions ref.
steps:
- name: Run DeadFinder
uses: hahwul/deadfinder@v2 # tracks the latest 2.x — pin a specific tag (e.g. @2.1.0) for stricter reproducibility
id: broken-link
with:
command: sitemap # url / file / sitemap / pipe
target: https://www.hahwul.com/sitemap.xml
# timeout: 10
# concurrency: 50
# silent: false
# headers: "X-API-Key: 123444"
# worker_headers: "User-Agent: Deadfinder Bot"
# include30x: false
# user_agent: "Apple"
# proxy: "http://localhost:8070"
# proxy_auth: "id:pw"
# insecure: false
# match: ""
# ignore: ""
# limit: 0
# output_format: json
# coverage: true
# visualize: report.png
# fail_on_dead: true # fail the step when a dead link is found
# method: auto # auto / head / get
# retry: 2
# delay: 0 # ms between requests to the same host
# accept_status: "403,999"
# dead_status: ""
# target_concurrency: 10
# check_anchors: false
- name: Output Handling
run: echo '${{ steps.broken-link.outputs.output }}'If you have found a Dead Link and want to automatically add it as an issue, please refer to the "Automating Dead Link Detection" article.
Usage
Usage: deadfinder <command> [options]
Commands:
pipe Scan the URLs from STDIN
file <FILE> Scan the URLs from File (`-` for STDIN)
url <URL> Scan the Single URL
sitemap <SITEMAP-URL> Scan the URLs from sitemap
completion <SHELL> Generate completion script (bash/zsh/fish)
version Show version
Options:
-r, --include30x Include 30x redirections as dead links
-c, --concurrency=N Number of concurrent workers (default: 50)
--target-concurrency=N Targets scanned in parallel; total in-flight
requests stay capped at -c (default: 10)
-t, --timeout=N Timeout in seconds (default: 10)
--method=METHOD Link check method: auto, head, get (default: auto)
--retry=N Retry transient failures N times (default: 2)
--delay=MS Minimum ms between requests to the same host
--accept-status=LIST Statuses to treat as alive (e.g. 200,403,999)
--dead-status=LIST Statuses to treat as dead (e.g. 500-599)
--exclude-status=LIST Alias of --dead-status
-o, --output=FILE File to write results (`-` for stdout)
-f, --output_format=FORMAT Output format: json, yaml, toml, csv, sarif (default: json)
-H, --headers=HEADER Custom HTTP headers for initial request
--worker_headers=HEADER Custom HTTP headers for worker requests
--user_agent=UA User-Agent string
-p, --proxy=PROXY Proxy server (HTTP and HTTPS CONNECT)
--proxy_auth=USER:PASS Proxy authentication
-k, --insecure Skip TLS certificate verification (not recommended)
-m, --match=PATTERN Match URL pattern (regex)
-i, --ignore=PATTERN Ignore URL pattern (regex)
-s, --silent Silent mode
-v, --verbose Verbose mode
--debug Debug mode
--limit=N Limit number of URLs to scan
--check-anchors Verify #fragment targets exist in the linked document
--coverage Enable coverage tracking and reporting
-F, --fail-on-dead Exit with code 2 when anything dead is found
--visualize=PATH Generate visualization PNG
-h, --help Show help
Modes
# Scan the URLs from STDIN (multiple URLs)
cat urls.txt | deadfinder pipe
# Scan the URLs from a file
deadfinder file urls.txt
# `-` and other non-regular files (process substitution, /dev/stdin, FIFOs) work too
deadfinder file - < urls.txt
deadfinder file <(subfinder -d example.com -silent | httpx -silent)
# Scan a single URL
deadfinder url https://www.hahwul.com
# Scan the URLs from a sitemap
deadfinder sitemap https://www.hahwul.com/sitemap.xmlURL list format (pipe / file)
One URL per line. The list is normalized before scanning:
- surrounding whitespace is trimmed, and a leading UTF-8 BOM is dropped
- blank lines and
#comments are skipped - duplicate URLs are scanned once
- lines that are not absolute
http:///https://URLs are reported and skipped -
--limit Ncounts only real targets, and stops reading the input once it is reached
# production
https://www.hahwul.com
https://www.hahwul.com/cullinan/
# staging
https://staging.hahwul.com
JSON Handling
deadfinder sitemap https://www.hahwul.com/sitemap.xml -o output.json
cat output.json | jq{
"Target URL": [
"DeadLink URL",
"DeadLink URL",
"DeadLink URL"
]
}With --coverage:
deadfinder sitemap https://www.hahwul.com/sitemap.xml --coverage -o output.json{
"dead_links": {
"Target URL": ["DeadLink URL 1", "DeadLink URL 2"]
},
"coverage": {
"targets": {
"Target URL": {
"total_tested": 14,
"dead_links": 7,
"coverage_percentage": 50.0
}
},
"summary": {
"total_tested": 14,
"total_dead": 7,
"overall_coverage_percentage": 50.0
}
}
}CI Usage
By default a scan always exits 0, so a broken link never fails a build. Opt in with
-F / --fail-on-dead, which exits 2 when anything dead was found (1 stays
reserved for usage and I/O errors):
deadfinder sitemap https://example.com/sitemap.xml --fail-on-dead
echo $? # 2 when a dead link or a dead target was found-o - streams the report to stdout — logs move to stderr in that mode, so the
structured output is safe to pipe:
deadfinder url https://example.com -f json -o - | jq '.'Shell Completion
deadfinder completion bash > /etc/bash_completion.d/deadfinder
deadfinder completion zsh > ~/.zsh/completion/_deadfinder
deadfinder completion fish > ~/.config/fish/completions/deadfinder.fishContributing
Contributions are welcome! If you have an idea for an improvement or want to report a bug:
- Fork the repository.
-
Create a new branch for your feature or bug fix (e.g.,
feature/awesome-featureorbugfix/annoying-bug). - Make your changes.
- Commit your changes with a clear message.
- Push to the branch.
-
Submit a Pull Request (PR) to our
mainbranch.
Contributors
