Secret detection rules

Before a file is uploaded, dotsync checks its path, then every line of its content, against the rules below. A file that matches isn’t uploaded, unless its entry sets allow_secrets. The tables are generated from internal/dotsync/secrets.go, so they’re always the rules your version applies.

What happens to a match depends on when it’s found:

dotsync add refuses a file that matches.
A managed file that starts to match is blocked: its new version isn’t sent, and the version already in the repository stays.
A matching file inside a managed directory is held back and shown as BLOCKED, while the rest of the directory syncs.

Only uploads are checked. A file another machine already sent is installed as it is.

Path rules

Paths are matched case-insensitively. A pattern that contains / is matched against the path relative to your home directory, and there * also matches /. Any other pattern is matched against the file name alone.

Patterns	Covers
`.ssh/id_` `.ssh/_key`	SSH private keys
`.pem` `.key` `.p12` `.pfx` `.jks` `.keystore`	TLS and signing keys, keystores
`.kdbx` `.password-store/`	Password databases
`.gnupg/private-keys-v1.d/` `.gnupg/secring.` `.gnupg/*.key`	GnuPG private keys (gpg.conf and gpg-agent.conf are fine)
`.local/share/keyrings/*`	Desktop keyrings
`.config/mise/age.txt` `.config/sops/age/` `Library/Application Support/sops/age/`	age private keys used by mise and sops
`.aws/credentials` `.aws/sso/` `.azure/` `.config/gcloud/*` `.kube/config`	Cloud credentials
`.docker/config.json` `.netrc` `.git-credentials` `.pgpass` `.vault-token` `.config/gh/hosts.yml`	Tool credentials
`.env` `.env.` `.env`	Environment files
`secret` `credential`	Anything named like a secret
`.*history`	Shell and REPL histories

Content rules

Each line is checked on its own, against these regular expressions (Go syntax):

Detects	Regular expression
Private key blocks of any kind: RSA, EC, OpenSSH, encrypted PKCS #8, PGP…	`-----BEGIN (?:[A-Z0-9]+ )*PRIVATE KEY(?: BLOCK)?-----`
age private keys (AGE-SECRET-KEY-1…)	`AGE-SECRET-KEY-1[02-9AC-HJ-NP-Z]{58}`
AWS access key IDs (AKIA…, ASIA…)	`\b(?:AKIA\|ASIA)[0-9A-Z]{16}\b`
GitHub tokens (ghp_, gho_, ghu_, ghs_, ghr_, github_pat_)	`\b(?:gh[pousr]_[A-Za-z0-9]{36,}\|github_pat_[A-Za-z0-9_]{22,})`
GitLab personal access tokens (glpat-)	`\bglpat-[A-Za-z0-9_-]{20,}`
Slack tokens (xoxa-, xoxb-, xoxo-, xoxp-, xoxr-, xoxs-)	`\bxox[abposr]-[A-Za-z0-9-]{10,}`
sk- style API keys, including sk-ant- and sk-proj-	`\bsk-(?:ant-\|proj-)?[A-Za-z0-9_-]{20,}`
Google API keys (AIza…)	`\bAIza[0-9A-Za-z_-]{35}`
npm auth tokens in .npmrc (_authToken=…)	`_authToken\s=\s[^\s$]{8,}`
A password in a URL: scheme://user:password@host	`\b[A-Za-z][A-Za-z0-9+.-]://[^\s/:@]:[^\s/@$({<%][^\s/@]*@`
password, passwd, secret, token, api_key, access_key or client_secret, then = or :, then a literal value of 8 or more characters	`(?i)(?:password\|passwd\|secret\|token\|api[_-]?key\|access[_-]?key\|client[_-]?secret)["']?\s[:=]\s["']?[^\s"'$(){}<>]{8,}`

Values read at runtime aren’t secrets

The last two rules skip a value that is read when the file is used rather than stored in it. For an assignment, the literal part of the value has to be at least 8 characters with no $, quote, parenthesis, brace or angle bracket; for a password in a URL, the password must not start with $, (, {, < or %. So these lines pass:

export GITHUB_TOKEN=$(op read op://dev/github/token)
password = "${MAIL_PASSWORD}"
DATABASE_URL=postgres://app:${PGPASSWORD}@db/app

and these are blocked:

password = hunter2hunter2
DATABASE_URL=postgres://app:hunter2@db/app

Encrypted files

dotsync recognizes encrypted file formats, whichever tool wrote them:

age files, ASCII-armored (-----BEGIN AGE ENCRYPTED FILE-----) or binary (age-encryption.org/v1), are entirely ciphertext. No rule applies to them, whatever they’re called.
sops files are recognized by the encrypted mac that sops writes into every format it supports: JSON, YAML, dotenv and INI. The path rules don’t apply to them, so a .env.json or secrets.yaml can sync. Their encrypted values (ENC[AES256_GCM,…]) are ignored, but the content rules still check every other part of every line, because sops leaves keys, comments and _unencrypted values readable.

The age private key that decrypts these files is always refused, by its path and by its content (AGE-SECRET-KEY-1…). Copy it to each machine yourself.

Exceptions

A line containing {ref.secrets.allow_marker} anywhere, usually in a comment, is skipped.
An entry with "allow_secrets": true, set with dotsync add --allow-secrets or dotsync set --allow-secrets, skips every rule.

Found a credential format that should be caught, or a common false positive? Open an issue.