Webbot activity on Oct 25, 2025

name		activity	Country
Baiduspider	search engine	1 page	CHINA
IP 45.67.230.75	Spammer	4 pages in 3s	RUSSIAN FEDERATION
IP 52.169.13.133	Zombie	98 pages in 32s	IRELAND
SemrushBot	crawler	1 page	UNITED STATES
IP 141.94.194.132		1 page	FRANCE
Yandex	search engine	1 page	RUSSIAN FEDERATION
Yandex	search engine	1 page	RUSSIAN FEDERATION
googleusercontent robot		1 page	UNITED STATES
IP 41.216.188.2	Zombie	1 page	GERMANY
IP 182.43.76.198	Zombie	21 pages in 1min 2s	CHINA
Qwantify	search engine	1 page	FRANCE
IP 38.170.211.183	Zombie	2 pages in 7s	UNITED STATES
Python-urllib		2 pages in 1s	NETHERLANDS
Baiduspider	search engine	1 page	CHINA
IP 141.94.194.132		1 page	FRANCE
IP 38.152.154.178	Zombie	2 pages in 6s	UNITED STATES
Yandex	search engine	1 page	RUSSIAN FEDERATION
Amazonbot	crawler	1 page	UNITED STATES
Baiduspider	search engine	1 page	CHINA
IP 216.73.216.137		1 page	UNITED STATES
528,105 visits of identified bots about 96 a day in 2025 45 today at 17:03 (+3 visitors) zombies : 8 visits / 179 requests - spammers : 4 visits

Where does the data come from?

It has been extracted from $_SERVER["HTTP_USER_AGENT"] -$HTTP_USER_AGENT with PHP 3-, $_SERVER ["QUERY_STRING"], $_SERVER ["SCRIPT_NAME"] and gethostbyaddr().
As this website host (free.fr) sometimes filters access, it is biased.
The site statistics do not consider robots as visitors. The browser and country they show are ignored.
Even if they read a few pages they are stored once except if they come back after more than 10 minutes (for Google Desktop or semrush after more than 30 minutes).
This list gives approximate information as it assumes a perfect connection to MySQL which is not the case for this site. But web hosting here is free, so...

Robots represent at least 80 % of the visits in 2025:

29,32 % of these visits come from search engine bots, site indexing, analysis, link or code checking, SEO optimisation...
16,76 % come from spammers (56,50 % try to send emails, the 43,50 % left spam the log files).
53,92 % are looking for the site vulnerabilities.

Robot Detection

This routine is commented in the page about webbot traps.
Logging other visitors' user agent is necessary to update the lists of webbots visiting the site and of their User Agents.
$UA=$_SERVER["HTTP_USER_AGENT"]; $no_ip=$_SERVER["REMOTE_ADDR"]; $brow = strtolower($UA);//in lower case to spot any version $bots = array ("googlebot","webcrawler","grub.org","slurp","openfind","antibot", "netresearchserver","nutch","ia_archiver","scooter","fluffy"); /* The list of user agents allows me to add the most significant part of robots names in lower case in the array $bot As the connection to MySQL is not perfect 24 hours a day, I use an array so that robots always get a light page. */ $n=0; do { $robot = strchr($brow,$bots[$n]); $n=$n+1; }while ((!$robot) && ($n<count($bots))); //for robots who do not identify with $HTTP_USER_AGENT if (!$robot) { $robot=strchr(gethostbyaddr($no_ip),".dir.com"); //etc } //

The Data Table

Here is the structure of the table robots I use:

#
# Structure of the table `robots`
#
CREATE TABLE `robots` (
  `timeoflastaccess` int(10) unsigned NOT NULL default '0',
  `timeofarrival` int(10) unsigned NOT NULL default '0',
  `nameofbot` varchar(64) NOT NULL default '',
  `lastpage` varchar(30) NOT NULL default '',
  `numberofpages` mediumint(8) unsigned NOT NULL default '0',
  KEY `timeoflastaccess` (`timeoflastaccess`),
  KEY `timeofarrival` (`timeofarrival`),
  KEY `nameofbot` (`nameofbot`),
  KEY `numberofpages` (`numberofpages`)
) TYPE=MyISAM;

You can use double or datetime for times, double or int for numberofpages, if necessary, increase the number of caracters for lastpage.

Table Update

//connection to the server $serveur=mysql_connect("your host","user name","password"); $base=mysql_select_db("your base",$serveur); //we set time $timeoflastaccess=time(); $timeoflastaccesslessten=$timeoflastaccess-600; if ($robot!="") { $result=@mysql_query ("SELECT COUNT(`timeoflastaccess`) FROM `robots`"); $test=@mysql_fetch_row ($result); //Some cleaning from time to time if ($test[0]>100) { $result=@mysql_query ("SELECT `timeoflastaccess` FROM `robots` ORDER BY `timeoflastaccess` LIMIT 80,1"); $test=@mysql_fetch_row ($result); $result=@mysql_query ("DELETE FROM `robots` WHERE `timeoflastaccess`<'$test[0]'"); $result=@mysql_query ("OPTIMIZE TABLE `robots`"); } /* Many robots have a dynamically allocated IP address, thus the name of the robot is a better field to store even if many versions arrive simultaneously! Did the robot load a page in the last 600 seconds? */ $result=@mysql_query ("SELECT * FROM `robots` WHERE `nameofbot`='$robot' AND `timeoflastaccess`>$timeoflastaccesslessten"); $test=@mysql_fetch_row ($result); $page_self=$_SERVER["REQUEST_URI"];//$PHP_SELF for PHP 3 if (empty($test[0])) { //no, it is the first page $result=@mysql_query ("INSERT INTO `robots` (`timeoflastaccess`,`timeofarrival`,`nameofbot`,`lastpage`,`numberofpages`) VALUES ('$timeoflastaccess','$timeoflastaccess','$robot','$page_self','1');"); } else { //time of last access is updated $result=@mysql_query ("UPDATE `robots` SET `timeoflastaccess`=$timeoflastaccess WHERE `nameofbot`='$robot' AND `timeoflastaccess`>$timeoflastaccesslessten"); //we get lastpage and numberofpages $result=@mysql_query ("SELECT * FROM `robots` WHERE `nameofbot`='$robot' AND `timeoflastaccess`='$timeoflastaccess'"); $res=@mysql_fetch_row ($result); if ($page_self!=$res[3]) { //This is not the same page //we update $result=@mysql_query ("UPDATE `robots` SET `lastpage`='$page_self',`numberofpages`=numberofpages+1 WHERE `nameofbot`='$robot' AND `timeoflastaccess`='$timeoflastaccess'"); } //Et voilà ! You surely can do better. } } else { /* you can do the same for visitors with their IP address or store the $HTTP_USER_AGENT to update the array $bots. */ }

Data Display

We have the name of the robot, the time of its arrival, the time of the last page it loaded and the total number of pages indexed.
If there is one page and different values for timeoflastaccess and timeofarrival, then the page was reloaded.
I chose to display the number of pages loaded and the length of reading time.
//connection to the server $server=mysql_connect("your host","user name","password"); $base=mysql_select_db("your base",$server); //selection of the last 20 insertions $result=@mysql_query ("SELECT COUNT('timeoflastaccess') FROM `robots`"); $numberofrows=@mysql_fetch_row ($result); $start=$numberofrows[0]-20;if ($start<0){$start=0;} $result=@mysql_query ("SELECT * FROM `robots` ORDER BY `timeofarrival` LIMIT $start,20"); // while ($tableau=@mysql_fetch_row ($result)) { //time spent $tpslect=$tableau[0]-$tableau[1]; if ($tpslect>60) { $tpslect=Floor($tpslect/60)."'".$tpslect % 60; } //display of name number and time spent echo "$tableau[2] $tableau[4] pages in $tpslect''<br>"; }

A similar script is now online here

Top of the page

With javascript