Kevin van Zonneveld

On Development and Internet System Engineering

Git migration - Remove Passwords from History

| Comments

When migrating projects over to GitHub, I found there were still some passwords inside my SVN repositories. Obviously it’s not good practice to store your passwords in a code repository - let alone at a remote location, so I wanted to replace all passwords. Not only in the current version, but in all commits that have been made over the past 3 years. Luckily with Git - you can.

Now, there is a guide to Remove sensitive data on GitHub; but that removes files completely.

I wanted to preserve the files and just replace the passwords in Git history.

So my plan was to:

  • Create GitHub accounts for every SVN comitter
  • Store the SVN<>GitHub account mapping in ~/.authors
  • Checkout SVN tree as a local Git repo (using git-svn)
  • Go over all the commits and replace all passwords with xXxXxXxXxXx

  • Go over all code in the HEAD - the current version of the project

  • find xXxXxXxXxXx

  • replace with App::config('Database.main.password')
  • Have App::config take the password from a config file that’s outside the repository

Now that I have a working HEAD without real passwords or xXxXxXxXxXx, and a lot of previous versions with just xXxXxXxXxXx in them:

  • Send it to GitHub
  • Continue leading a happy life without worries.

Here are the commands I ended up using:

(git_clear_pwd.sh) download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

#// LANG::Bash
#// Sample starts here
# Import from SVN
cd ${HOME}/workspace
git svn clone --authors-file=${HOME}/.authors svn://svn.example.com/projectX/trunk projectX

cd projectX

# Rewrite history
git filter-branch --tree-filter 'git ls-files -z "*.php" |xargs -0 perl -p -i -e "s#(PASSWORD1|PASSWORD2|PASSWORD3)#xXxXxXxXxXx#g"' -- --all

# Make workspace look like HEAD
git reset --hard

# Try to recompress and clean up, then check the new size
git gc --aggressive --prune

# To GitHub
git remote add origin git@github.com:kvz/projectX.git
git push origin master

Lookout for these keywords as you’ll have to substitute them with your own:

  • projectX
  • example.com
  • kvz
  • .authors
  • PASSWORD1
  • PASSWORD2
  • PASSWORD3

Warning! Rewriting history Can be Dangerous! : )

Seriously though.. Be absolutely sure you know what you’re doing and make backups before doing anything.

Comments