Bulk Remove Meta Robots Tag using Google Tag Manager

This tutorial offers guidance on how to remove the meta robots tag. Using Google Tag Manager and a simple JavaScript variable this guide will show you how to remove the meta robots tags on multiple URLs and/or URLs in bulk without the need for a Lookup Table.

An example use case would be conflicting indexation signals such as canonicalized canonical tag and an index, follow meta robots tag.

[h2]Table of Contents[/h2]
1, Create the Tag
2, Build the Variable
3, Set the Trigger
4, Publish the Container and Testing

[h3] 1, Create the Tag[/h3]
Create a Custom HTML tag. Use either of the code snippets below. The 1st snippet is JavaScript and the second snippet is jQuery. In most cases you will most likely want to use the 1st snippet unless your website is heavily reliant on jQuery, then it may make sense to use the 2nd snippet.

The snippet below is instructing the browser to remove the meta robots tag (meta tag).

1st snippet

<script>
document.querySelector("[name='robots']").remove();

</script>

2nd snippet

<script>
$('meta[name=robots]').remove();
 
</script>

[h3] 2, Build the Variable[/h3]
Now it is time to build out the array of URLs that we are dropping the meta robots tag. The script below is a JavaScript function that lists all the URLs that we are dropping the meta tag. Wrap each URL with apostrophes and commas, the final doesn't require a comma.

If you are building a list of many URLs, I suggest using Excel or Google Sheets to wrap apostrophes and commas and then use concatenate to wrap apostrophes and commas (remember the final URL doesn't need a comma).

In Tag Manager create a new variable. Set the variable type to Custom JavaScript.

The script works by finding a match (return true). If the script finds a match in the array of URLs, the custom HTML in the Tag is deployed. If the script does not find a match then nothing happens, the tag is not fired to the page.

function() {
  var dropMetaRobots = [
    '/template1/',
    '/template2.html',
    '/template3.php',
  ];
 
  for (var i = 0; i < dropMetaRobots.length; i += 1) {
    if (document.location.href.indexOf(dropMetaRobots[i]) > -1) {
      return true;
    }
  };

  return false;
}

[h3] 3, Set the Trigger[/h3]
The final step is to set trigger conditions. Make sure you get this part right otherwise the script will not fire correctly or you may end up having the script firing on all URLs.

The trigger conditions:

• Page View
• Fire on Some Pages
• Condition settings are your Custom Variable >Equals > True
• References the Custom HTML Tag

[h3] 4, Publish the Container and Testing[/h3]
That's it, go ahead with publishing the container. Use the Tag Assistant tool to check if the Tag is firing on the correct pages.

To check and see if the meta robots tag has been removed from a URL. You need to check the rendered HTML (the raw HTML will still show the meta robots tag). I use a Chrome Extension from Jon Hogg called View Rendered Source as it colour codes the differences between the rendered and raw HTML.

[h3]That's a Wrap[/h3]
That's everything. Thank you for visiting and reading the tutorial. Feel free to comment below if you need support.

*Tip: the custom HTML tag can be modified to fire, modify or remove any types of meta tags. Have a read at some of my other guides for inspiration.

 

[Chris]

Testing the Xenforo API x2