Adding a dynamic robots.txt to Kohana 3

September 16, 2011

Adding a function that dynamically generates a robots.txt  to Kohana 3 is actually very easy.

First we need to add a route to bootstrap.php that will route the yourdomain.com/robots.txt url to the function.

Route::set('robots', 'robots.txt')
 ->defaults(array(
  'controller' => 'home',
  'action'     => 'robots',
 ));

Now we create the action “action_robots” in the home controller and output the exclusions we are looking for.

public function action_robots()
{
	
		header('Content-type: text/plain');
		echo "User-agent: *\r\n";
		echo "disallow: /";
		exit();
}

The content-type here is very important, as search engines will not read it as a php file.

NOTE: This will not work if your application if the index_file configuration in bootstrap.php is set to TRUE. This is because if it is set to TRUE then the url would turn out to be /index.php/robots.txt which is obviously not where the search engines will be looking for the restrictions.

Author

Stay in Touch!

Stay in Touch!
Solutions Architecture

browse through our blog articles

Blog Archive