Sistrix
SISTRIX •
Since 2008
Quick Actions
Official Docs
What is Sistrix?
SISTRIX is a leading SEO tool provider from Germany, famous for their Visibility Index - a metric showing a domain's visibility in search results. Their crawler collects data to calculate this index and provide competitive intelligence for SEO professionals. SISTRIX is particularly strong in European markets and provides detailed competitor analysis, keyword research, and ranking tracking. The platform's historical data goes back over a decade, making it valuable for long-term SEO trend analysis and competitive research.
User Agent String
Mozilla/5.0 (compatible; SISTRIX Crawler; +https://crawler.sistrix.net/)
How to Control Sistrix
Block Completely
To prevent Sistrix from accessing your entire website, add this to your robots.txt file:
# Block Sistrix
User-agent: SISTRIX Crawler
Disallow: /
Block Specific Directories
To restrict access to certain parts of your site while allowing others:
User-agent: SISTRIX Crawler
Disallow: /admin/
Disallow: /private/
Disallow: /wp-admin/
Allow: /public/
Set Crawl Delay
To slow down the crawl rate (note: not all bots respect this directive):
User-agent: SISTRIX Crawler
Crawl-delay: 10
How to Verify Sistrix
Verification Method:
SISTRIX Crawler identification
SISTRIX Crawler identification
Learn more in the official documentation.
Detection Patterns
Multiple ways to detect Sistrix in your application:
Basic Pattern
/Sistrix/iStrict Pattern
/^Mozilla/5\.0 \(compatible; SISTRIX Crawler; \+https\://crawler\.sistrix\.net/\)$/Flexible Pattern
/Sistrix[\s\/]?[\d\.]*?/iVendor Match
/.*SISTRIX.*Sistrix/iImplementation Examples
// PHP Detection for Sistrix
function detect_sistrix() {
$user_agent = $_SERVER['HTTP_USER_AGENT'] ?? '';
$pattern = '/Sistrix/i';
if (preg_match($pattern, $user_agent)) {
// Log the detection
error_log('Sistrix detected from IP: ' . $_SERVER['REMOTE_ADDR']);
// Set cache headers
header('Cache-Control: public, max-age=3600');
header('X-Robots-Tag: noarchive');
// Optional: Serve cached version
if (file_exists('cache/' . md5($_SERVER['REQUEST_URI']) . '.html')) {
readfile('cache/' . md5($_SERVER['REQUEST_URI']) . '.html');
exit;
}
return true;
}
return false;
}
# Python/Flask Detection for Sistrix
import re
from flask import request, make_responsedef detect_sistrix():
user_agent = request.headers.get('User-Agent', '')
pattern = r'Sistrix'
if re.search(pattern, user_agent, re.IGNORECASE):
# Create response with caching
response = make_response()
response.headers['Cache-Control'] = 'public, max-age=3600'
response.headers['X-Robots-Tag'] = 'noarchive'
return True
return False# Django Middleware
class SistrixMiddleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
if self.detect_bot(request):
# Handle bot traffic
pass
return self.get_response(request)
// JavaScript/Node.js Detection for Sistrix
const express = require('express');
const app = express();// Middleware to detect Sistrix
function detectSistrix(req, res, next) {
const userAgent = req.headers['user-agent'] || '';
const pattern = /Sistrix/i;
if (pattern.test(userAgent)) {
// Log bot detection
console.log('Sistrix detected from IP:', req.ip);
// Set cache headers
res.set({
'Cache-Control': 'public, max-age=3600',
'X-Robots-Tag': 'noarchive'
});
// Mark request as bot
req.isBot = true;
req.botName = 'Sistrix';
}
next();
}app.use(detectSistrix);
# Apache .htaccess rules for Sistrix# Block completely
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Sistrix [NC]
RewriteRule .* - [F,L]# Or redirect to a static version
RewriteCond %{HTTP_USER_AGENT} Sistrix [NC]
RewriteCond %{REQUEST_URI} !^/static/
RewriteRule ^(.*)$ /static/$1 [L]# Or set environment variable for PHP
SetEnvIfNoCase User-Agent "Sistrix" is_bot=1# Add cache headers for this bot
<If "%{HTTP_USER_AGENT} =~ /Sistrix/i">
Header set Cache-Control "public, max-age=3600"
Header set X-Robots-Tag "noarchive"
</If>
# Nginx configuration for Sistrix# Map user agent to variable
map $http_user_agent $is_sistrix {
default 0;
~*Sistrix 1;
}server {
# Block the bot completely
if ($is_sistrix) {
return 403;
}
# Or serve cached content
location / {
if ($is_sistrix) {
root /var/www/cached;
try_files $uri $uri.html $uri/index.html @backend;
}
try_files $uri @backend;
}
# Add headers for bot requests
location @backend {
if ($is_sistrix) {
add_header Cache-Control "public, max-age=3600";
add_header X-Robots-Tag "noarchive";
}
proxy_pass http://backend;
}
}
Should You Block This Bot?
Recommendations based on your website type:
| Site Type | Recommendation | Reasoning |
|---|---|---|
| E-commerce | Optional | Evaluate based on bandwidth usage vs. benefits |
| Blog/News | Allow | Increases content reach and discoverability |
| SaaS Application | Block | No benefit for application interfaces; preserve resources |
| Documentation | Selective | Allow for public docs, block for internal docs |
| Corporate Site | Limit | Allow for public pages, block sensitive areas like intranets |
Advanced robots.txt Configurations
E-commerce Site Configuration
User-agent: SISTRIX Crawler
Crawl-delay: 5
Disallow: /cart/
Disallow: /checkout/
Disallow: /my-account/
Disallow: /api/
Disallow: /*?sort=
Disallow: /*?filter=
Disallow: /*&page=
Allow: /products/
Allow: /categories/
Sitemap: https://example.com/sitemap.xml
Publishing/Blog Configuration
User-agent: SISTRIX Crawler
Crawl-delay: 10
Disallow: /wp-admin/
Disallow: /drafts/
Disallow: /preview/
Disallow: /*?replytocom=
Allow: /
SaaS/Application Configuration
User-agent: SISTRIX Crawler
Disallow: /app/
Disallow: /api/
Disallow: /dashboard/
Disallow: /settings/
Allow: /
Allow: /pricing/
Allow: /features/
Allow: /docs/
Quick Reference
User Agent Match
SistrixRobots.txt Name
SISTRIX CrawlerCategory
seoRespects robots.txt
Yes
Copied to clipboard!