PetalBot
Huawei •
Since 2020
Quick Actions
Official Docs
What is PetalBot?
PetalBot is the web crawler for Huawei's Petal Search, a search engine developed as part of Huawei's push for technological independence. Launched as Huawei faced restrictions on using Google services, Petal Search has become the default search engine on Huawei devices. PetalBot indexes content globally with particular strength in Asian markets. The crawler supports modern web standards and respects robots.txt directives. For developers and content creators targeting Huawei's ecosystem of over 700 million devices, PetalBot represents an important channel for content discovery.
User Agent String
Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; PetalBot;+https://webmaster.petalsearch.com/site/petalbot)
How to Control PetalBot
Block Completely
To prevent PetalBot from accessing your entire website, add this to your robots.txt file:
# Block PetalBot
User-agent: PetalBot
Disallow: /
Block Specific Directories
To restrict access to certain parts of your site while allowing others:
User-agent: PetalBot
Disallow: /admin/
Disallow: /private/
Disallow: /wp-admin/
Allow: /public/
Set Crawl Delay
To slow down the crawl rate (note: not all bots respect this directive):
User-agent: PetalBot
Crawl-delay: 10
How to Verify PetalBot
Verification Method:
Check published IP ranges and user agent
Check published IP ranges and user agent
Learn more in the official documentation.
Detection Patterns
Multiple ways to detect PetalBot in your application:
Basic Pattern
/PetalBot/i
Strict Pattern
/^Mozilla/5\.0 \(Linux; Android 7\.0;\) AppleWebKit/537\.36 \(KHTML, like Gecko\) Mobile Safari/537\.36 \(compatible; PetalBot;\+https\://webmaster\.petalsearch\.com/site/petalbot\)$/
Flexible Pattern
/PetalBot[\s\/]?[\d\.]*?/i
Vendor Match
/.*Huawei.*PetalBot/i
Implementation Examples
// PHP Detection for PetalBot
function detect_petalbot() {
$user_agent = $_SERVER['HTTP_USER_AGENT'] ?? '';
$pattern = '/PetalBot/i';
if (preg_match($pattern, $user_agent)) {
// Log the detection
error_log('PetalBot detected from IP: ' . $_SERVER['REMOTE_ADDR']);
// Set cache headers
header('Cache-Control: public, max-age=3600');
header('X-Robots-Tag: noarchive');
// Optional: Serve cached version
if (file_exists('cache/' . md5($_SERVER['REQUEST_URI']) . '.html')) {
readfile('cache/' . md5($_SERVER['REQUEST_URI']) . '.html');
exit;
}
return true;
}
return false;
}
# Python/Flask Detection for PetalBot
import re
from flask import request, make_response
def detect_petalbot():
user_agent = request.headers.get('User-Agent', '')
pattern = r'PetalBot'
if re.search(pattern, user_agent, re.IGNORECASE):
# Create response with caching
response = make_response()
response.headers['Cache-Control'] = 'public, max-age=3600'
response.headers['X-Robots-Tag'] = 'noarchive'
return True
return False
# Django Middleware
class PetalBotMiddleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
if self.detect_bot(request):
# Handle bot traffic
pass
return self.get_response(request)
// JavaScript/Node.js Detection for PetalBot
const express = require('express');
const app = express();
// Middleware to detect PetalBot
function detectPetalBot(req, res, next) {
const userAgent = req.headers['user-agent'] || '';
const pattern = /PetalBot/i;
if (pattern.test(userAgent)) {
// Log bot detection
console.log('PetalBot detected from IP:', req.ip);
// Set cache headers
res.set({
'Cache-Control': 'public, max-age=3600',
'X-Robots-Tag': 'noarchive'
});
// Mark request as bot
req.isBot = true;
req.botName = 'PetalBot';
}
next();
}
app.use(detectPetalBot);
# Apache .htaccess rules for PetalBot
# Block completely
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} PetalBot [NC]
RewriteRule .* - [F,L]
# Or redirect to a static version
RewriteCond %{HTTP_USER_AGENT} PetalBot [NC]
RewriteCond %{REQUEST_URI} !^/static/
RewriteRule ^(.*)$ /static/$1 [L]
# Or set environment variable for PHP
SetEnvIfNoCase User-Agent "PetalBot" is_bot=1
# Add cache headers for this bot
<If "%{HTTP_USER_AGENT} =~ /PetalBot/i">
Header set Cache-Control "public, max-age=3600"
Header set X-Robots-Tag "noarchive"
</If>
# Nginx configuration for PetalBot
# Map user agent to variable
map $http_user_agent $is_petalbot {
default 0;
~*PetalBot 1;
}
server {
# Block the bot completely
if ($is_petalbot) {
return 403;
}
# Or serve cached content
location / {
if ($is_petalbot) {
root /var/www/cached;
try_files $uri $uri.html $uri/index.html @backend;
}
try_files $uri @backend;
}
# Add headers for bot requests
location @backend {
if ($is_petalbot) {
add_header Cache-Control "public, max-age=3600";
add_header X-Robots-Tag "noarchive";
}
proxy_pass http://backend;
}
}
Should You Block This Bot?
Recommendations based on your website type:
| Site Type | Recommendation | Reasoning |
|---|---|---|
| E-commerce | Allow | Essential for product visibility in search results |
| Blog/News | Allow | Increases content reach and discoverability |
| SaaS Application | Block | No benefit for application interfaces; preserve resources |
| Documentation | Allow | Improves documentation discoverability for developers |
| Corporate Site | Allow | Allow for public pages, block sensitive areas like intranets |
Advanced robots.txt Configurations
E-commerce Site Configuration
User-agent: PetalBot
Crawl-delay: 5
Disallow: /cart/
Disallow: /checkout/
Disallow: /my-account/
Disallow: /api/
Disallow: /*?sort=
Disallow: /*?filter=
Disallow: /*&page=
Allow: /products/
Allow: /categories/
Sitemap: https://example.com/sitemap.xml
Publishing/Blog Configuration
User-agent: PetalBot
Crawl-delay: 10
Disallow: /wp-admin/
Disallow: /drafts/
Disallow: /preview/
Disallow: /*?replytocom=
Allow: /
SaaS/Application Configuration
User-agent: PetalBot
Disallow: /app/
Disallow: /api/
Disallow: /dashboard/
Disallow: /settings/
Allow: /
Allow: /pricing/
Allow: /features/
Allow: /docs/
Quick Reference
User Agent Match
PetalBot
Robots.txt Name
PetalBot
Category
search
Respects robots.txt
Yes
Copied to clipboard!