Getting https pages through a proxy using PHP and Requests

September 24, 2012 5:50 pm 0 comments

Today I was searching the internet looking for a HTTP client that supports authentication and https connections because I couldn’t use the function

with htpps addresses.

I found some interesting and modern projects like Httpful and Guzzle, but they created with REST in mind, and I was looking for something much simpler, but powerful enough to allows to get https pages and authentication.

And searching a bit deeper I could finally find Requests. Yeah, with that name it’s really difficult to find it, everyting is called request on the internet. What I found is a Http client for PHP created by Ryan McCue, inspired by the python’s Requests library.

Requests uses cURL and fsockopen, depending on what your system has available, but abstracts all the nasty stuff out of your way, providing a consistent API. Long story short, make easy to get http/https pages from your php scripts. Using it, getting a secure webpage as https://www.google.com is as easy as:

It does not matter if your php installation doesn’t have the openssl wrapper installed, that one that let’s you get the pages using the file_get_contents function. Requests will find it way to get the page for you.

But I had another problem, and I could not find info in the internet about: How to get those pages through my office’s proxy?? I found a solution by myself, and I am glad to share it :)

As I said before, Requests can use cURL to get the pages, so we can config cURL to handle the proxy for us. Unfortunately, Requests don’t give any simple way to pass config parameter to cURL, so we will have to extend the cURL transport class to set it up, and force Requests to use it. The result is this piece of code

This way I could, after hours of research, get my https pages through a proxy. I hope this make your resarch time much shorter. Go try Requests library!

javi

Let’s push your website a bit further

My name is Javier Márquez and I have more than 10 years of web programming and web designing experience. If you have a difficult development to complete, maybe you can stop by and see what I can do for you. You can find me on and Twitter.

Leave a reply


contact meAnything related to web development like javascript and CSS to create responsive designs, or PHP and node.js to make your website work properly, is my pleasure. If you have an interesting project in mind, I can help to make it real.