Frequently Asked Questions
How does CMS Crawler work?
The system has two main functions: a crawler, which searches the web for
homepages and puts them into our database and secondly an analysis tool that
analyzes the tools used at different sites.
How do you identify which tool that a web site uses?
We use different methods, first of all we look at the META GENERATOR-tag but we
are also looking through the site's home page and server information for codes
that are significant for different systems.
Earlier versions of the CMS Crawler was more aggressive and attempted to load
multiple addresses that are unique to different systems but we received some
criticism when we did so we decided to only read the website's main page.
How certain are you that you have the right data?
We are not sure at all. Our method requires first of all, that the crawler can
find sites and then we need to identify the publishing tool. There are many
publising tools which can not be identified at all and there are many site
owners who choose not to publish META GENERATOR-tag. We also can't reach
intranet or password protected sites.
But what we know is that we have very few false-positive values, that is if we
can find a tool on a site then we feel confident that the site uses that tool.
I miss a tool that I think should be included, what can I do?
Email us at firstname.lastname@example.org. In
order to be able to evaluate if we can identify the tool, we need some general
information about the tool and some examples of sites that uses the tool.