LGSCrawler

AuditAgent

This bot audits web pages for customers of LGS: usually the owner of the site, sometimes a business comparing its own site with others in its trade. It downloads the page, robots.txt, sitemap.xml and the internal links to check SEO, accessibility, security headers and performance.

What else it asks your server for

To warn the owner about files and admin pages that should not be open to the public, it asks once for each of a short list of well-known addresses, such as /.env, /.git/config and /wp-admin/. These are plain GET requests, at most 20 per audit, with nothing sent to log in or to change anything. If you see them in your logs, that is what they are.

It does not index your pages and it never submits forms. It keeps what the report needs: the findings and short excerpts of the page as evidence.

An in-depth audit also takes screenshots of the page, tries harmless test values in its address and makes a short burst of requests to see how the site copes. It only runs on a site whose owner has proved they control it.

How to block it

It obeys robots.txt. These two lines keep it off your site entirely:

User-agent: AuditAgent
Disallow: /

It also respects Crawl-delay. By default it makes no more than four requests per second to any one host, except during the short burst of an in-depth audit described above.

Contact

This service has no contact address configured yet. The robots.txt block above takes effect immediately.