Requests for robots.txt with ip and user agent

Log Parser Plus example query

Requests for robots.txt with ip and user agent

This query grabs all requests for the robots.txt file, outputting ip address and user agents, with counts for each.

Keywords:

iisw3c

Statement:

logparser -i:w3c -o:xml "SELECT c-ip AS [ClientIp], cs(user-agent) AS [ClientUserAgent], COUNT(*) AS [Requests] INTO robots.xml FROM u_ex1009*.log WHERE cs-uri-stem = '/robots.txt' GROUP BY ClientIp, ClientUserAgent ORDER BY Requests DESC"

Notes: This particular query outputs the data to XML, and renames columns because of that. See The benefits of using a file for Log Parser SQL queries for more information. Leave a comment on this query.

View more examples.