AOL’s venerable instant messaging network, AOL Instant Messenger (AIM), silently intercepts URLs in private chats and retrieves the content at the sent address without user consent. AOL appears to be retrieving any and all URLs sent during chats indiscriminately, at the network level. Presumably AOL is indexing the users included in the chat, the URL, and the contents of the URL, but it is not clear (at all) what they are doing with this data.
USERS OF AOL INSTANT MESSENGER ARE WARNED THAT URLS SENT IN PRIVATE INSTANT MESSAGES ARE BEING INTERCEPTED AND RETRIEVED BY AOL AT THIS TIME.
Unfortunately virtually no information on what AOL is doing is available at this time. However, the evidence is clear to anyone with access to their own web server. This was discovered by a security analyst affiliated with StealYour.Info:
While chatting with a buddy, I sent them a link to a private file. I had accessed the file to ensure it was there, and then sent it to my buddy. Before he was able to even click the link, the server was hit twice by two different user agents (below). Seconds later, my buddy clicked the link and retrieved the file. Puzzled, I searched for the reason – suspecting some kind of spyware on my buddy’s computer. But in fact, it was the AIM client itself according to our testing. Using completely fake URLs, it was clear that AOL Instant Messenger was in fact passing on the URL I sent for some kind of retrieval and analysis by AOL.
64.12.71.15 – – [08/Oct/2011:20:02:55 -0400] “GET /pwned.fake HTTP/1.1” 404 292 “-” “Java/1.6.0_22”
64.12.71.15 – – [08/Oct/2011:20:02:55 -0400] “GET /pwned.fake HTTP/1.1” 404 292 “-” “Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13”
149.174.17.177 – – [08/Oct/2011:19:52:56 -0400] “GET /pwned.fake HTTP/1.1” 200 357 “-” “Java/1.6.0_22”
149.174.17.177 – – [08/Oct/2011:19:52:57 -0400] “GET /pwned.fake HTTP/1.1” 200 357 “-” “Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13”
We are seeking any available information on this practice and correlating data from other webmasters. If you are able to observe this behavior with the latest AIM client, please confirm our findings in the comments section below and share additional IP addresses AOL may be using for this purpose.
Note that third party instant messaging software (e.g. Pidgin) is affected by this despite the initial description of the discovery, as the monitoring appears to be taking place at the network level.
I grepped logs dating back to the 1st of September and the first recorded instance of this user agent from an AOL owned IP address I found was 27 Sep 2011:
149.174.12.109 - - [27/Sep/2011:09:23:19 -0400] "GET /2011/09/26/nvidia-launches-battlefield-3-gpu-driver-offers-38-performance-increase/ HTTP/1.1" 200 69502 "-" "Java/1.6.0_22"149.174.17.177 - - [28/Sep/2011:17:49:22 -0400] "GET /forum/images/smilies/Kappa.png HTTP/1.1" 200 1342 "-" "Java/1.6.0_22"
64.12.71.15 - - [30/Sep/2011:00:39:44 -0400] "GET /wp-content/uploads/SI_boomerfemale_collage.jpg HTTP/1.1" 200 54761 "-" "Java/1.6.0_22"
149.174.17.176 - - [30/Sep/2011:02:39:11 -0400] "GET /2011/09/29/8-ways-to-not-suck-at-the-battlefield-3-open-beta/ HTTP/1.1" 200 65094 "-" "Java/1.6.0_22"
64.12.71.15 - - [30/Sep/2011:03:29:39 -0400] "GET /2011/09/29/8-ways-to-not-suck-at-the-battlefield-3-open-beta/ HTTP/1.1" 200 65094 "-" "Java/1.6.0_22"
64.12.71.15 - - [30/Sep/2011:17:47:33 -0400] "GET /2011/09/30/this-video-explains-the-featured-of-battlefield-3s-battlelog/ HTTP/1.1" 200 45383 "-" "Java/1.6.0_22"
149.174.17.178 - - [01/Oct/2011:20:06:41 -0400] "GET /2011/02/15/16-year-old-kills-mother-over-playstation-argument/ HTTP/1.1" 200 106438 "-" "Java/1.6.0_22"
149.174.17.176 - - [03/Oct/2011:12:05:54 -0400] "GET /2011/09/29/8-ways-to-not-suck-at-the-battlefield-3-open-beta/ HTTP/1.1" 200 73494 "-" "Java/1.6.0_22"
149.174.17.178 - - [04/Oct/2011:05:26:12 -0400] "GET /wp-content/uploads/INF2-header1.jpg HTTP/1.1" 200 61830 "-" "Java/1.6.0_22"
64.12.71.15 - - [05/Oct/2011:14:39:32 -0400] "GET /category/pc/ HTTP/1.1" 200 72949 "-" "Java/1.6.0_22"
64.12.71.15 - - [06/Oct/2011:00:45:25 -0400] "GET /2011/08/25/thanks-to-ea-there-will-be-no-porsches-in-forza-4/ HTTP/1.1" 200 74631 "-" "Java/1.6.0_22"
149.174.17.176 - - [06/Oct/2011:02:42:48 -0400] "GET /wp-content/uploads/e861_pacman_fleece_blanket_whole.jpg HTTP/1.1" 200 69063 "-" "Java/1.6.0_22"
64.12.71.6 - - [06/Oct/2011:17:59:52 -0400] "GET /2011/10/06/26-year-old-rapes-12-year-old-with-a-wiimote/ HTTP/1.1" 200 61308 "-" "Java/1.6.0_22"
149.174.17.176 - - [06/Oct/2011:19:59:47 -0400] "GET /forum/showthread.php?20039-Guy-gets-robbed-goes-to-jail-for-child-porn HTTP/1.1" 200 63882 "-" "Java/1.6.0_22"
149.174.17.178 - - [07/Oct/2011:11:14:35 -0400] "GET /2011/09/26/nvidia-launches-battlefield-3-gpu-driver-offers-38-performance-increase/ HTTP/1.1" 200 75790 "-" "Java/1.6.0_22"
64.12.71.6 - - [07/Oct/2011:14:29:10 -0400] "GET /2011/10/07/dice-opens-caspian-border-for-all-pc-players/ HTTP/1.1" 200 55574 "-" "Java/1.6.0_22"
149.174.17.178 - - [07/Oct/2011:17:44:37 -0400] "GET /2011/10/07/dice-opens-caspian-border-for-all-pc-players/ HTTP/1.1" 200 63654 "-" "Java/1.6.0_22"
149.174.17.178 - - [07/Oct/2011:17:52:07 -0400] "GET /2011/10/07/dice-opens-caspian-border-for-all-pc-players/ HTTP/1.1" 200 63653 "-" "Java/1.6.0_22"
Confirmed, I first noticed this behavior in Early october. The Java user agent is a giveaway but the AOL IP ranges are another factor. the reverse on the IPs is picnicapi-d01.blue.aol.com (for 149.174.17.178) but any of the IPs will all have picnicapi at the beginning. I tried googling ofr Picnic API and a few variants but i can’t find any info.
Makes me wonder why they want to cache the content of a URL at the time it was sent. I wonder if they are trying to get new URLs for their search engine this way, or if it is something even more evil.
Workaround: Send your friend the link before you upload the file, tell them to wait a sec, then upload the file. the bot will grab a 404, and your friend will grab the real URL
Since both of the IPs above resolve to blahblah.aol.com. We can use that to serve false information to AIM web crawlers.
http://www.blackhatacademy.org/security101/index.php?title=Facebook
will explain this further.
Use the PHP below to filter it out…
$host_aol = ‘/aol.com/i’;
$u_host = gethostbyaddr($_SERVER[‘REMOTE_ADDR’]);
if (preg_match($host_aol,$u_host))
{
echo “Content for AOL to see.”
} else {
echo “Content for everyone else to see.”
}