Perl, Scripting, Tutorials

How To fetch header information of a website using Perl

Fetching response header using Perl

This is a part of banner grabbing methods and it is highly useful for penetration testers to gather server information. You can use either http or ftp or https or any supported protocol to know what information it reveals for each cases.

For a web developer and penetration testers it is like daily routine to check what header response is being sent from the server to debug any server level issues. Server response can give us some important information like which server it is running, which programming language it is running, cookie value(s), encoding type etc.

Note: You may not get all these information for some sites if they use customized code to send header response)

You should try for any https url to see what response google sends for https connection.

Here is the https response from gmail.com

https response header from gmail
https response header from gmail

And here is the ftp response of a site

 

ftp response header using perl
ftp response header using perl

It can be also useful to determine if your site is built on WordPress and xmlrpc is enabled in your website. Check for X-Pingback: http://www.sanjeevjaiswal.com/xmlrpc.php

 

header information to determine if xmlrpc is enabled in WordPress
header information to determine if xmlrpc is enabled in WordPress

 

You can also check if site is built in Drupal and which version. Check for X-Generator: Drupal 7 (http://drupal.org)

 

header info to determine which drupal version
header info to determine which drupal version

 

How can we get the header response?

I think you already know some of the known methods. Like using dev tool in browser or using cURL with -I option i.e. curl -I aliencoders.org I will share Perlish method. You can get the header response using Perl script as well.

 

Any prerequisites to run such Perl script?

Yes, you must have WWW::Mechanize or LWP::UserAgent installed in your system and Perl interpreter must be installed. O.S. hardly matters in this case 😉

 

Code snippet, Please

Method 1 using LWP::UserAgent

# Using LWP::UserAgent
my $ua = LWP::UserAgent->new();

# connect and get
my $response = $ua->get($url);
print $response->headers()->as_string;

 

 

Method 2 using WWW::Mechanize (My Favorite one)

# Using WWW::Mechanize
my $mech = WWW::Mechanize->new();
my $resp = $mech->get($url);
my $string = $resp->headers->as_string;
print $string;

 

Want fully working code to run in command line or through browser? Please check the code here.

Share your Thoughts