Gitleaks – How to Audit git repository for secrets

Do you store unencrypted passwords, secrets and any other unwanted data types in your git source code repositories?. Gitleaks gives you a way to scan your git repositories for these unwanted data which should be private. The scans can be automated to fit perfectly into CI/CD workflow for secrets identification before they make it deeper into the codebase.

Gitleaks Features

Some of the cool features of Gitleaks include:

• Support for private repository scans as well as repositories that require key-based authentication
• Support for Gitlab bulk organization and repository owner (user) repository scans, and pull request scanning for use in common CI workflows.
• You can output the scan results in JSON and CSV and formats for consumption in other reporting tools and frameworks.
• Externalised configuration for environment specific customisation including regex rules
• Customisable repository name, file type, commit ID, branchname and regex whitelisting to reduce false positives
• High performance through the use of src-d’s go-git framework

How to Install Gitleaks

Gitleaks is written in Go and the binary file is available for many popular platforms and OS types from the releases page.

Step 1: Download Gitleaks

Check the latest release and save it to a variable like below.

export VER="1.24.0"

Then download the binary file to your OS.

For Linux Users

wget https://github.com/zricethezav/gitleaks/releases/download/v$VER/gitleaks-linux-amd64

For macOS user:

wget https://github.com/zricethezav/gitleaks/releases/download/v$VER/gitleaks-darwin-amd64

If you’re a Windows user, download and install the gitleaks-windows-amd64.exe package.

Step 2: Install Gitleaks

Once the file is downloaded, give it executable bot and put it inside the /usr/local/bin directory.

mv gitleaks-linux-amd64 gitleaks
chmod +x gitleaks
sudo mv gitleaks /usr/local/bin/

For macOS:

mv gitleaks-darwin-amd64 /usr/local/bin/gitleaks

Confirm that you can call the gitleaks command.

$ gitleaks --version

1.24.0

How to Use Gitleaks to Audit Git repositories

Gitleaks has lots of tunables that you don’t actually need for basic usage. The default mode should work against a single repo without any tweaks.

$ gitleaks --repo=https://github.com/jmutai/dotfiles
 INFO[2019-02-13T15:55:43+03:00] cloning https://github.com/jmutai/dotfiles
 Enumerating objects: 42, done.
 Counting objects: 100% (42/42), done.
 Compressing objects: 100% (34/34), done.
 Total 2255 (delta 10), reused 26 (delta 8), pack-reused 2213
 INFO[2019-02-13T15:55:57+03:00] 0 leaks detected. 159 commits inspected in 13 seconds 389 milliseconds

To view the output of the audit as gitleaks processes the repository, use the -v or --verbose flags which turns on verbose mode.

$ gitleaks --repo=https://github.com/gitleakstest/gronit -v
 INFO[2019-02-13T16:06:08+03:00] cloning https://github.com/gitleakstest/gronit 
 Enumerating objects: 135, done.
 Total 135 (delta 0), reused 0 (delta 0), pack-reused 135
 {
    "line": "const AWS_KEY = \"AKIALALEMEL33243OLIAE\"",
    "commit": "eaeffdc65b4c73ccb67e75d96bd8743be2c85973",
    "offender": "AKIALALEMEL33243OLIA",
    "reason": "AWS",
    "commitMsg": "remove fake key",
    "author": "Zachary Rice \[email protected]\u003e",
    "file": "main.go",
    "repo": "gronit",
    "date": "2018-02-04T19:43:28-06:00"
 }
 {
    "line": "const AWS_KEY = \"AKIALALEMEL33243OLIAE\"",
    "commit": "cb5599aeed261b2c038aa4729e2d53ca050a4988",
    "offender": "AKIALALEMEL33243OLIA",
    "reason": "AWS",
    "commitMsg": "fake key",
    "author": "Zachary Rice \[email protected]\u003e",
    "file": "main.go",
    "repo": "gronit",
    "date": "2018-02-04T19:10:58-06:00"
 }
 WARN[2019-02-13T16:06:11+03:00] 2 leaks detected. 33 commits inspected in 2 seconds 598 milliseconds 

To enable Threading, use the --threads option.

CPU=$(cat /proc/cpuinfo | grep -ic ^processor)
gitleaks --repo=https://github.com/jmutai/dotfiles --threads=$CPU

This option specifies the max number of threads spawned.

Running Gitleaks in Redact mode

The --redact will help show lines containing the secrets without logging the content.

$ gitleaks --repo=https://github.com/gitleakstest/gronit --redact
 INFO[2019-02-13T16:10:16+03:00] cloning https://github.com/gitleakstest/gronit 
 Enumerating objects: 135, done.
 Total 135 (delta 0), reused 0 (delta 0), pack-reused 135
 WARN[2019-02-13T16:10:20+03:00] 2 leaks detected. 33 commits inspected in 3 seconds 786 milliseconds

Saving Gitleaks audit results to file

You can also run an audit on a bunch of repositories and save reports for each repo in a file. For this, use the --report option.

$ gitleaks --repo=https://github.com/jmutai/dotfiles --report=gitleaks_results.csv
 INFO[2019-02-13T16:13:57+03:00] cloning https://github.com/jmutai/dotfiles   
 Enumerating objects: 42, done.
 Counting objects: 100% (42/42), done.
 Compressing objects: 100% (34/34), done.
 Total 2255 (delta 10), reused 26 (delta 8), pack-reused 2213
 INFO[2019-02-13T16:14:15+03:00] 0 leaks detected. 159 commits inspected in 17 seconds 685 milliseconds 

The report must end in .csv or .json.

Check the project Git documentation for more advanced configurations and examples.

Similar articles:

How to remove git files, directories in .gitignore from a remote repository

GitLab and Slack Integration for notifications

How to Configure GitLab FreeIPA Authentication