urllib3 fork with SSL certificate pinning using sha1 sha256 Pubkey digests

I maintain a fork of urllib3. I added assertion of PubKey digests, so every request to https will have its SSL certificate pinned.

Because git isn’t my tool of choice, I keep forgetting how I keep the fork in sync with the original repo.

I have not done a sync for quite some time, so I decided to write it down for posterity.

How I keep the fork in sync with the official urllib3 repo by shazow:

$ git clone https://github.com/encodeltd/urllib3.git
$ cd urllib3
➜  urllib3 git:(master) git remote add upstream https://github.com/urllib3/urllib3  # add urllib's repo as remote upstream
➜  urllib3 git:(master) git pull upstream master                                    # and pull from it
remote: Counting objects: 3334, done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 3334 (delta 837), reused 841 (delta 836), pack-reused 2485
Receiving objects: 100% (3334/3334), 1.01 MiB | 280.00 KiB/s, done.
Resolving deltas: 100% (2379/2379), completed with 125 local objects.
From https://github.com/urllib3/urllib3
 * branch            master     -> FETCH_HEAD
 * [new branch]      master     -> upstream/master
Removing urllib3/packages/backports/__init__.py
Removing urllib3/contrib/__init__.py
Auto-merging test/appengine/test_gae_manager.py
Removing test/appengine/requirements.txt
Removing test/appengine/nose.cfg
Removing test/appengine/app.yaml
Auto-merging src/urllib3/util/url.py
Auto-merging src/urllib3/util/timeout.py
Auto-merging src/urllib3/util/ssl_.py
Auto-merging src/urllib3/util/retry.py
Auto-merging src/urllib3/util/request.py
Auto-merging src/urllib3/util/connection.py
Auto-merging src/urllib3/util/__init__.py
Auto-merging src/urllib3/response.py
Auto-merging src/urllib3/request.py
Auto-merging src/urllib3/poolmanager.py
Auto-merging src/urllib3/packages/ssl_match_hostname/_implementation.py
Auto-merging src/urllib3/packages/ssl_match_hostname/__init__.py
Auto-merging src/urllib3/filepost.py
Auto-merging src/urllib3/fields.py
Auto-merging src/urllib3/exceptions.py
Auto-merging src/urllib3/contrib/socks.py
Auto-merging src/urllib3/contrib/pyopenssl.py
Auto-merging src/urllib3/contrib/appengine.py
Auto-merging src/urllib3/connectionpool.py
Auto-merging src/urllib3/connection.py
CONFLICT (content): Merge conflict in src/urllib3/connection.py
Auto-merging src/urllib3/_collections.py
Auto-merging src/urllib3/__init__.py
Removing _travis/fetch_gae_sdk.py
Automatic merge failed; fix conflicts and then commit the result.
$ vim urllib3/connection.py                                              # resolve conflicts manually
$ git add urllib3/connection.py
$ git commit -m "Merged changes from https://github.com/urllib3/urllib3 master"
$ git push

Installation

To install the fork, run:

pip install https://github.com/encodeltd/urllib3/archive/master.zip

Usage

Here is a testing script (pin_ssl_cert.py) for sha1, sha256 PubKey digests SSL certificate pinning using my fork of urllib3:

from __future__ import print_function

import sys
import urllib3
import logging

from urllib3.contrib import pyopenssl

pyopenssl.inject_into_urllib3()

log = logging.getLogger('urllib3.connection')
log.setLevel(logging.INFO)
log.addHandler(logging.StreamHandler(sys.stdout))


def test_pin_ssl_cert(test_request):
    try:
        test_response = test_request.request('GET', 'https://google.com/')
        print(test_response.status)
    except ValueError, e:
        print(e)


# test ssl cert pinning with correct set of digests
test_request = urllib3.PoolManager(assert_pubkey_digests=(
    'ncSxX0vhBA8b60xgmhytIUPfYkNVRUPNLnzIApgO16I=',
    'nDg/4zpPacE7OGtJSc2/jRnSiW4=',
    '7HIpactkIAq2Y49orFOOQKurWxmmSFZhBCoQYcRhJ3Y=',
    'Q9rWMO5T+KmAym79hfRqo3mQ4Oo=',
    'h6801m+z8v3zbgkRHpq6L29Esgfzhj89C1SyUCOQmqU=',
    'wHqYaI2J+6sFZAwRfap9ZbjKzE4='))
test_pin_ssl_cert(test_request)

# test ssl cert pinning with incorrect set of digests
test_request = urllib3.PoolManager(assert_pubkey_digests=('-'))
test_pin_ssl_cert(test_request)

The following is its output:

$ python pin_ssl_cert.py
Verifying connection using provided PubKey digests: ('ncSxX0vhBA8b60xgmhytIUPfYkNVRUPNLnzIApgO16I=', 'nDg/4zpPacE7OGtJSc2/jRnSiW4=', '7HIpactkIAq2Y49orFOOQKurWxmmSFZhBCoQYcRhJ3Y=', 'Q9rWMO5T+KmAym79hfRqo3mQ4Oo=', 'h6801m+z8v3zbgkRHpq6L29Esgfzhj89C1SyUCOQmqU=', 'wHqYaI2J+6sFZAwRfap9ZbjKzE4=')
Verifying connection using provided PubKey digests: ('ncSxX0vhBA8b60xgmhytIUPfYkNVRUPNLnzIApgO16I=', 'nDg/4zpPacE7OGtJSc2/jRnSiW4=', '7HIpactkIAq2Y49orFOOQKurWxmmSFZhBCoQYcRhJ3Y=', 'Q9rWMO5T+KmAym79hfRqo3mQ4Oo=', 'h6801m+z8v3zbgkRHpq6L29Esgfzhj89C1SyUCOQmqU=', 'wHqYaI2J+6sFZAwRfap9ZbjKzE4=')
200
Verifying connection using provided PubKey digests: -
Public key digests (sha1, or sha256) do not match any of the expected pins!

Received:
(ncSxX0vhBA8b60xgmhytIUPfYkNVRUPNLnzIApgO16I=,
nDg/4zpPacE7OGtJSc2/jRnSiW4=,
7HIpactkIAq2Y49orFOOQKurWxmmSFZhBCoQYcRhJ3Y=,
Q9rWMO5T+KmAym79hfRqo3mQ4Oo=,
h6801m+z8v3zbgkRHpq6L29Esgfzhj89C1SyUCOQmqU=,
wHqYaI2J+6sFZAwRfap9ZbjKzE4=)

Expected:
(LQ==)

Installation

Usage

Comments