Webbot activity on Jul 12, 2025

name		activity	Country
IP 185.125.219.119	Spammer	1 page	RUSSIAN FEDERATION
IP 45.89.66.53	Spammer	1 page	RUSSIAN FEDERATION
IP 45.89.66.19	Spammer	1 page	RUSSIAN FEDERATION
IP 45.89.66.57	Spammer	1 page	RUSSIAN FEDERATION
IP 45.89.66.206	Spammer	1 page	RUSSIAN FEDERATION
IP 141.94.194.132		1 page	FRANCE
AhrefsBot	crawler	1 page	FRANCE
DotBot	crawler	1 page	UNITED STATES
DotBot	crawler	1 page	UNITED STATES
Applebot	search engine	1 page	UNITED STATES
AhrefsBot	crawler	1 page	FRANCE
IP 141.94.194.132		1 page	FRANCE
IP 162.221.197.235	Zombie	2 pages in 36s	UNITED STATES
IP 141.94.194.132		1 page	FRANCE
DotBot	crawler	1 page	UNITED STATES
bingbot	crawler	1 page	UNITED STATES
Googlebot	crawler	1 page	UNITED STATES
AhrefsBot	crawler	1 page	FRANCE
IP 141.94.194.132		1 page	FRANCE
IP 216.73.216.64		1 page	UNITED STATES
517,565 visits of identified bots about 94 a day in 2025 16 today at 5:06 (+1 visitor) zombies : 1 visit / 2 requests - spammer : 0 visit

Where does the data come from?

It has been extracted from $_SERVER["HTTP_USER_AGENT"] -$HTTP_USER_AGENT with PHP 3-, $_SERVER ["QUERY_STRING"], $_SERVER ["SCRIPT_NAME"] and gethostbyaddr().
As this website host (free.fr) sometimes filters access, it is biased.
The site statistics do not consider robots as visitors. The browser and country they show are ignored.
Even if they read a few pages they are stored once except if they come back after more than 10 minutes (for Google Desktop or semrush after more than 30 minutes).
This list gives approximate information as it assumes a perfect connection to MySQL which is not the case for this site. But web hosting here is free, so...

Robots represent at least 77 % of the visits in 2025:

30,62 % of these visits come from search engine bots, site indexing, analysis, link or code checking, SEO optimisation...
18,35 % come from spammers (57,75 % try to send emails, the 42,25 % left spam the log files).
51,03 % are looking for the site vulnerabilities.

Robot Detection

This routine is commented in the page about webbot traps.
Logging other visitors' user agent is necessary to update the lists of webbots visiting the site and of their User Agents.
$UA=$_SERVER["HTTP_USER_AGENT"]; $no_ip=$_SERVER["REMOTE_ADDR"]; $brow = strtolower($UA);//in lower case to spot any version $bots = array ("googlebot","webcrawler","grub.org","slurp","openfind","antibot", "netresearchserver","nutch","ia_archiver","scooter","fluffy"); /* The list of user agents allows me to add the most significant part of robots names in lower case in the array $bot As the connection to MySQL is not perfect 24 hours a day, I use an array so that robots always get a light page. */ $n=0; do { $robot = strchr($brow,$bots[$n]); $n=$n+1; }while ((!$robot) && ($n<count($bots))); //for robots who do not identify with $HTTP_USER_AGENT if (!$robot) { $robot=strchr(gethostbyaddr($no_ip),".dir.com"); //etc } //

The Data Table

Here is the structure of the table robots I use:

#
# Structure of the table `robots`
#
CREATE TABLE `robots` (
  `timeoflastaccess` int(10) unsigned NOT NULL default '0',
  `timeofarrival` int(10) unsigned NOT NULL default '0',
  `nameofbot` varchar(64) NOT NULL default '',
  `lastpage` varchar(30) NOT NULL default '',
  `numberofpages` mediumint(8) unsigned NOT NULL default '0',
  KEY `timeoflastaccess` (`timeoflastaccess`),
  KEY `timeofarrival` (`timeofarrival`),
  KEY `nameofbot` (`nameofbot`),
  KEY `numberofpages` (`numberofpages`)
) TYPE=MyISAM;

You can use double or datetime for times, double or int for numberofpages, if necessary, increase the number of caracters for lastpage.

Table Update

//connection to the server $serveur=mysql_connect("your host","user name","password"); $base=mysql_select_db("your base",$serveur); //we set time $timeoflastaccess=time(); $timeoflastaccesslessten=$timeoflastaccess-600; if ($robot!="") { $result=@mysql_query ("SELECT COUNT(`timeoflastaccess`) FROM `robots`"); $test=@mysql_fetch_row ($result); //Some cleaning from time to time if ($test[0]>100) { $result=@mysql_query ("SELECT `timeoflastaccess` FROM `robots` ORDER BY `timeoflastaccess` LIMIT 80,1"); $test=@mysql_fetch_row ($result); $result=@mysql_query ("DELETE FROM `robots` WHERE `timeoflastaccess`<'$test[0]'"); $result=@mysql_query ("OPTIMIZE TABLE `robots`"); } /* Many robots have a dynamically allocated IP address, thus the name of the robot is a better field to store even if many versions arrive simultaneously! Did the robot load a page in the last 600 seconds? */ $result=@mysql_query ("SELECT * FROM `robots` WHERE `nameofbot`='$robot' AND `timeoflastaccess`>$timeoflastaccesslessten"); $test=@mysql_fetch_row ($result); $page_self=$_SERVER["REQUEST_URI"];//$PHP_SELF for PHP 3 if (empty($test[0])) { //no, it is the first page $result=@mysql_query ("INSERT INTO `robots` (`timeoflastaccess`,`timeofarrival`,`nameofbot`,`lastpage`,`numberofpages`) VALUES ('$timeoflastaccess','$timeoflastaccess','$robot','$page_self','1');"); } else { //time of last access is updated $result=@mysql_query ("UPDATE `robots` SET `timeoflastaccess`=$timeoflastaccess WHERE `nameofbot`='$robot' AND `timeoflastaccess`>$timeoflastaccesslessten"); //we get lastpage and numberofpages $result=@mysql_query ("SELECT * FROM `robots` WHERE `nameofbot`='$robot' AND `timeoflastaccess`='$timeoflastaccess'"); $res=@mysql_fetch_row ($result); if ($page_self!=$res[3]) { //This is not the same page //we update $result=@mysql_query ("UPDATE `robots` SET `lastpage`='$page_self',`numberofpages`=numberofpages+1 WHERE `nameofbot`='$robot' AND `timeoflastaccess`='$timeoflastaccess'"); } //Et voilà ! You surely can do better. } } else { /* you can do the same for visitors with their IP address or store the $HTTP_USER_AGENT to update the array $bots. */ }

Data Display

We have the name of the robot, the time of its arrival, the time of the last page it loaded and the total number of pages indexed.
If there is one page and different values for timeoflastaccess and timeofarrival, then the page was reloaded.
I chose to display the number of pages loaded and the length of reading time.
//connection to the server $server=mysql_connect("your host","user name","password"); $base=mysql_select_db("your base",$server); //selection of the last 20 insertions $result=@mysql_query ("SELECT COUNT('timeoflastaccess') FROM `robots`"); $numberofrows=@mysql_fetch_row ($result); $start=$numberofrows[0]-20;if ($start<0){$start=0;} $result=@mysql_query ("SELECT * FROM `robots` ORDER BY `timeofarrival` LIMIT $start,20"); // while ($tableau=@mysql_fetch_row ($result)) { //time spent $tpslect=$tableau[0]-$tableau[1]; if ($tpslect>60) { $tpslect=Floor($tpslect/60)."'".$tpslect % 60; } //display of name number and time spent echo "$tableau[2] $tableau[4] pages in $tpslect''<br>"; }

A similar script is now online here

Top of the page

With javascript