Webbot activity on May 23, 2025

name		activity	Country
IP 216.73.216.30		1 page	UNITED STATES
SemrushBot	crawler	1 page	UNITED STATES
IP 216.73.216.182		2 pages in 10s	UNITED STATES
IP 216.73.216.158		1 page	UNITED STATES
IP 216.73.216.194		2 pages in 3s	UNITED STATES
IP 194.67.210.77	Spammer	1 page	RUSSIAN FEDERATION
IP 185.5.249.185	Spammer	1 page	RUSSIAN FEDERATION
IP 194.67.207.94	Spammer	1 page	RUSSIAN FEDERATION
IP 185.125.219.13	Spammer	1 page	RUSSIAN FEDERATION
IP 194.67.207.9	Spammer	1 page	RUSSIAN FEDERATION
IP 193.124.191.92	Spammer	1 page	RUSSIAN FEDERATION
IP 94.142.141.230	Spammer	1 page	RUSSIAN FEDERATION
IP 185.105.116.12	Spammer	1 page	RUSSIAN FEDERATION
IP 194.67.207.55	Spammer	1 page	RUSSIAN FEDERATION
IP 185.125.219.59	Spammer	1 page	RUSSIAN FEDERATION
IP 105.214.79.73	Spammer	6 pages in 22s	SOUTH AFRICA
IP 216.73.216.9		1 page	UNITED STATES
IP 146.185.135.37	Spammer	6 pages in 10s	NETHERLANDS
IP 192.111.134.10	Spammer	6 pages in 11s	UNITED STATES
IP 216.73.216.44		1 page	UNITED STATES
511,587 visits of identified bots about 85 a day in 2025 185 today at 22:27 (+1 visitors) zombies : 13 visits / 14 requests - spammers : 45 visits

Where does the data come from?

It has been extracted from $_SERVER["HTTP_USER_AGENT"] -$HTTP_USER_AGENT with PHP 3-, $_SERVER ["QUERY_STRING"], $_SERVER ["SCRIPT_NAME"] and gethostbyaddr().
As this website host (free.fr) sometimes filters access, it is biased.
The site statistics do not consider robots as visitors. The browser and country they show are ignored.
Even if they read a few pages they are stored once except if they come back after more than 10 minutes (for Google Desktop or semrush after more than 30 minutes).
This list gives approximate information as it assumes a perfect connection to MySQL which is not the case for this site. But web hosting here is free, so...

Robots represent at least 78 % of the visits in 2025:

30,97 % of these visits come from search engine bots, site indexing, analysis, link or code checking, SEO optimisation...
18,28 % come from spammers (60,13 % try to send emails, the 39,87 % left spam the log files).
50,75 % are looking for the site vulnerabilities.

Robot Detection

This routine is commented in the page about webbot traps.
Logging other visitors' user agent is necessary to update the lists of webbots visiting the site and of their User Agents.
$UA=$_SERVER["HTTP_USER_AGENT"]; $no_ip=$_SERVER["REMOTE_ADDR"]; $brow = strtolower($UA);//in lower case to spot any version $bots = array ("googlebot","webcrawler","grub.org","slurp","openfind","antibot", "netresearchserver","nutch","ia_archiver","scooter","fluffy"); /* The list of user agents allows me to add the most significant part of robots names in lower case in the array $bot As the connection to MySQL is not perfect 24 hours a day, I use an array so that robots always get a light page. */ $n=0; do { $robot = strchr($brow,$bots[$n]); $n=$n+1; }while ((!$robot) && ($n<count($bots))); //for robots who do not identify with $HTTP_USER_AGENT if (!$robot) { $robot=strchr(gethostbyaddr($no_ip),".dir.com"); //etc } //

The Data Table

Here is the structure of the table robots I use:

#
# Structure of the table `robots`
#
CREATE TABLE `robots` (
  `timeoflastaccess` int(10) unsigned NOT NULL default '0',
  `timeofarrival` int(10) unsigned NOT NULL default '0',
  `nameofbot` varchar(64) NOT NULL default '',
  `lastpage` varchar(30) NOT NULL default '',
  `numberofpages` mediumint(8) unsigned NOT NULL default '0',
  KEY `timeoflastaccess` (`timeoflastaccess`),
  KEY `timeofarrival` (`timeofarrival`),
  KEY `nameofbot` (`nameofbot`),
  KEY `numberofpages` (`numberofpages`)
) TYPE=MyISAM;

You can use double or datetime for times, double or int for numberofpages, if necessary, increase the number of caracters for lastpage.

Table Update

//connection to the server $serveur=mysql_connect("your host","user name","password"); $base=mysql_select_db("your base",$serveur); //we set time $timeoflastaccess=time(); $timeoflastaccesslessten=$timeoflastaccess-600; if ($robot!="") { $result=@mysql_query ("SELECT COUNT(`timeoflastaccess`) FROM `robots`"); $test=@mysql_fetch_row ($result); //Some cleaning from time to time if ($test[0]>100) { $result=@mysql_query ("SELECT `timeoflastaccess` FROM `robots` ORDER BY `timeoflastaccess` LIMIT 80,1"); $test=@mysql_fetch_row ($result); $result=@mysql_query ("DELETE FROM `robots` WHERE `timeoflastaccess`<'$test[0]'"); $result=@mysql_query ("OPTIMIZE TABLE `robots`"); } /* Many robots have a dynamically allocated IP address, thus the name of the robot is a better field to store even if many versions arrive simultaneously! Did the robot load a page in the last 600 seconds? */ $result=@mysql_query ("SELECT * FROM `robots` WHERE `nameofbot`='$robot' AND `timeoflastaccess`>$timeoflastaccesslessten"); $test=@mysql_fetch_row ($result); $page_self=$_SERVER["REQUEST_URI"];//$PHP_SELF for PHP 3 if (empty($test[0])) { //no, it is the first page $result=@mysql_query ("INSERT INTO `robots` (`timeoflastaccess`,`timeofarrival`,`nameofbot`,`lastpage`,`numberofpages`) VALUES ('$timeoflastaccess','$timeoflastaccess','$robot','$page_self','1');"); } else { //time of last access is updated $result=@mysql_query ("UPDATE `robots` SET `timeoflastaccess`=$timeoflastaccess WHERE `nameofbot`='$robot' AND `timeoflastaccess`>$timeoflastaccesslessten"); //we get lastpage and numberofpages $result=@mysql_query ("SELECT * FROM `robots` WHERE `nameofbot`='$robot' AND `timeoflastaccess`='$timeoflastaccess'"); $res=@mysql_fetch_row ($result); if ($page_self!=$res[3]) { //This is not the same page //we update $result=@mysql_query ("UPDATE `robots` SET `lastpage`='$page_self',`numberofpages`=numberofpages+1 WHERE `nameofbot`='$robot' AND `timeoflastaccess`='$timeoflastaccess'"); } //Et voilà ! You surely can do better. } } else { /* you can do the same for visitors with their IP address or store the $HTTP_USER_AGENT to update the array $bots. */ }

Data Display

We have the name of the robot, the time of its arrival, the time of the last page it loaded and the total number of pages indexed.
If there is one page and different values for timeoflastaccess and timeofarrival, then the page was reloaded.
I chose to display the number of pages loaded and the length of reading time.
//connection to the server $server=mysql_connect("your host","user name","password"); $base=mysql_select_db("your base",$server); //selection of the last 20 insertions $result=@mysql_query ("SELECT COUNT('timeoflastaccess') FROM `robots`"); $numberofrows=@mysql_fetch_row ($result); $start=$numberofrows[0]-20;if ($start<0){$start=0;} $result=@mysql_query ("SELECT * FROM `robots` ORDER BY `timeofarrival` LIMIT $start,20"); // while ($tableau=@mysql_fetch_row ($result)) { //time spent $tpslect=$tableau[0]-$tableau[1]; if ($tpslect>60) { $tpslect=Floor($tpslect/60)."'".$tpslect % 60; } //display of name number and time spent echo "$tableau[2] $tableau[4] pages in $tpslect''<br>"; }

A similar script is now online here

Top of the page

With javascript