What I learned from analysing thousands of robots.txt files

A talk by Sam Gipson
SEO Consultant, SamGipson.com

Login to watch this content

About this talk

The humble robots.txt file (Robots Exclusion Protocol) has been in existence for 25 years and can be hugely influential for SEO. This talk covers an analysis of its current usage by some of the top-ranking sites across key verticals, uncovering some interesting statistics and some of the most common implementation mistakes. It is increasingly important that marketing professionals ensure they are getting theirs right, as Google strives to make robots.txt an internet standard.

Proudly supported by

Want to sponsor? Contact us to find out more.