Web Watch: Update of a Survey of the Numbers of UK University Web Servers
How many Web servers are there in use within the UK higher education community? What is the profile of server usage within the community - do most institutions take a distributed approach, running many servers, or is a centralised approach more popular? A WebWatch survey was published in June 2000  which aimed to provide answers to these questions. The survey has been repeated recently in order to see if there has been any significant changes.
Using Netcraft's Web-Based Service
Netcraft  is a company based in Bath which carries out surveys of Web server software. The Netcraft survey is very comprehensive, with over 36 million sites having been polled. Netcraft publish monthly reports on their surveys .
Netcraft also provide an interface which enables details of the Web server used at a site to be obtained. The service also provides a search facility which can be used to list the Web servers used within a domain (or part of a domain). This facility can be used to, for example, list the Web servers which have .microsoft in the domain name.
The Netcraft interface is shown in Figure 1.
Figure 1: Netcraft interface
This service has been used to analyse Web servers used in UK Universities. The results are provided in Appendix 1 of this article.
Why Carry This Survey?
The aim of the initial survey was to profile the server usage within the UK HE community. Knowledge of the server usage within the community has a number of possible benefits: for example, knowledge of server profiles will help developers of robot software.
The survey has been repeated in order to detect interesting trends: are we, for example, seeing a growth in the numbers of servers due to use of personal Web servers and desktop computers? Or are we seeing a reduction in the numbers, due to institutions taking a more centralised approach to use of Web servers?
A number of caveats to this survey should be noted:
- Web servers which are not publicly available, such as Intranets, will not be reported.
- Duplicate names for Web servers may be included in the reports.
- The report does not differentiate between real and virtual servers.
- The survey is reliant on the quality of the data obtained from Netcraft.
Summary of Findings
The total number of Web servers found on 1 May 2000 was 3,986. Excluding institutions for which Netcraft reported 0 Web servers, an average of 19.4 servers per institution was found. When the survey was repeated on 5 June 2000 the total number of Web servers had reduced to 3,773, with an average of 24.2 servers per institution.
For the most recent survey the total number of Web servers had increased to 4,571, with an average of 28.6 servers per institution.
A histogram of the results is shown in Figure 2.
Figure 2: Histogram of Results
As can be seen, most institutions have a small number of publicly-accessible Web servers, with 73 institutions running between 1 and 9 servers (79 in May 2000) and only 11 running more than 100 (9 in May 2000).
There has not been a major change in the profile in the numbers of Web servers used across the community. We have seen a increase, small in many cases, although there have been a number of substantial jumps (e.g. Stirling with a growth from 26 to 87 and Strathclyde from 46 to 110).
After the previous survey was published there was speculation that we would shortly be seeing a substantial jump, as broadband networks became available, which would allow home computers to be "always on", allowing users to run Web servers from home. There was an exception that this change, which would be accompanied by enhanced personal Web server tools, would influence the profile of server usage within institutions (with the accompanying challenges of management, security, etc.) However this change has not happened, and we are still awaiting the widespread deployment of broadband networks.
We also do not appear to be seeing significant changes in institutions moving from a small number of centrally managed servers to a distributed environment or, alternatively, from a distributed environment to a centrally managed one.
- A Survey Of Numbers of UK University Web Servers, Ariadne issue 24, June 2000
- Web Server Survey, Netcraft
Appendix 1: Survey Results
The Netcraft service was used to count the number of web servers used by UK Universities. The survey was initially carried out on 1 May 2000 and repeated on 5 June 2000. The data given below was obtained on 11 March 2002.
The following table contains details of the survey. Column 1 gives the name of the institution and a link to the institutional Web server. Column 2 gives the number of Web servers found within the university's domain.
Note the following limitations in the survey:
- Domain Name Variants
- The survey will not detect unusual domain names variants used within an institution: e.g. a link from www.lse-maths.ac.uk will be included in the numbers of servers in the domain lse.ac.uk.
- Short and Link Domain Names
- Institutions which make use of a long and short form of their domain name (e.g. www.bham.ac.uk and www.birmingham.ac.uk) will have misleading results.
- IP Numbers
- Servers which use an IP number rather than a domain name will not be included.
- Netcraft Limitations
- The database used by the Netcraft service may be incomplete.
- One institution was reported as having no Web servers (Royal Scottish Academy of Music and Drama). This is believed to be due to an error in the data collection.
- The data for the initial survey of Coventry University was lost.
- The initial surveys of Lancaster Universities used the domain name .lancaster.ac.uk. This survey also provides findings for .lancs.ac.uk.
Table 2 gives a summary of the top 5 web sites with the largest number of web servers as reported by Netcraft.
|Institution||No. of Web Servers (1 May 2000)||No. of Web Servers (5 June 2000)||No. of Web Servers (12 March 2002)||Current Value|
UK Web Focus
University of Bath
Brian Kelly is UK Web Focus. He works for UKOLN, which is based at the University of Bath