Information Systems Services
Storage Quotas
We currently apply a number of different quotas for email and file storage. These are as follows.
§ Home directories: 200MB[1] for all account holders.
§ Email: 50MB students, 100MB staff
§ Private web space: 10MB
§ Departmental shared space: 30GB per department
As each quota is typically applied to space provided by different means, we have to look at the affects of modifying each individually.
Home Directories
We provide the majority of the home directories (also known as M: drives) for University staff and students. The main exception is provision for members of ISER and The Data Archive. The disk space is currently provided from Debian Linux servers using either samba or nfs for access.
The total amount of disk space used for home directories has risen from around 11GB in August 1994 to 550GB in April 2006. The set quotas have risen slowly – the last change being to allow an increase for students to 200MB in 2002.
In general, there isn’t an issue with providing a higher quota for an individual staff member for their M: drive, so long as we have space to provide it. There is also an emerging requirement from Computing Science to provide a higher basic quota for some of their students who need to more space for their course work. Again, we have no basic objection to this so long as we can plan for the increased space requirements.
Summer 2006, we plan to upgrade 6 of our 8 home directory servers which will result in the ability to provide higher quotas. However, as space provided grows, so does the amount used and therefore backups (and perhaps more importantly, recovery) can take longer. Also, the total cost of backup tapes rises proportionally with the amount of tapes required.
Whilst over half of our users are using less than 10% of their M: drive quota, we also have some individuals who claim to require very large quotas. Currently, over 300 people have a quota larger than the default and of these 27 have a quota of larger than 1G. More often than not, the reason for these larger quotas is to store Outlook Personal Folders (PST files) due to stricter quota restrictions placed on email quotas.
We propose that with planned increased disk space that will shortly be available, that the basic quota is increased to 500MB for start of the next academic year. There will be some other knock-on increases in costs but most of these will be in tapes which we can accommodate within budget.
However, we currently have no clear guidance on how big a quota should be allowed for any individual and have to take special steps to allow a quota over 3GB. It is often the case that these very large quotas are requested by senior administrative staff of the University and are primarily used for email storage. We have also had requests for quotas of 10GB or higher and expect calls for higher quotas to arrive in time as storage requirements increase.
Recommendations
1. Increase quotas for staff and students to 500MB from September 2006.
2. Allow quotas of 1GB for staff on request. Quotas greater than this will be granted at the discretion of the Director of Information Systems.
3. Allow higher quotas for postgraduates where their supervisor provides a clear justification for it.
By today’s standards, we are sometimes seen as restrictive with our email quotas. The basic quota of 100MB for staff seems fairly paltry compared to the 2.5GB (e.g. 25 times greater quota) offered by Google’s email service. The main reason for the restrictions is due to the email solution that we run (Microsoft Exchange) and how it is backed-up and restored. This places some limitations on how big we can comfortably allow our email stores to grow.
When we first started using Microsoft Exchange approximately 10 years ago, we ran it on a single server with 2GB of storage. It was at that time that we introduced the current quotas. We are now running on 6 servers and have 350GB of active storage. This generally increases slowly but steadily (approximately 20% over the last year). In general, for those receiving a lot of email, their mailbox will continue to fill until such time as they hit their quota. At that time, many are fairly likely to ask for a greater quota before actually spending the time deleting old emails that are no longer required. By being relatively unwilling to increase these quotas, w believe that we have been able to ensure that our users do take some care and delete unwanted email. However, email is seen as one of the primary ways of communicating and our users expect to be able to store large amounts of data in their mailbox. Add to that the simplicity with which large attachments can be sent and a problem with the current quotas is easily generated.
Our current Exchange backup solution will scale to meet the expected growth with quotas left as they are. If we increase quotas (even just for some), the knock-on effect will be that the backup solution will be inadequate before expect to replace it in a couple of years time. We also need to consider whether the current backup arrangements are still appropriate or whether we need to adopt something faster to cope with the increasing requirements.
We believe that the current quotas are adequate for most students (so long as staff don’t expect to be able to mail large attachments to them – there is a training issue here for greater use of the CMR). For a large majority of the staff, the current quotas are adequate. However, it is with around 10% of staff that we have a problem. We can increase quotas for some and try to cope with the increased load but fairly soon, the knock on effects will be that we need to invest in more servers to handle the additional storage requirements.
An alternative is to use one of the emerging email archiving technologies. These move emails based on certain criteria (e.g. over a year old) onto a second-level storage and out of the mail email inbox. These solutions can also re-import mail stored in PST files on users M: drives (and elsewhere), thereby allowing universal access to your entire mailbox from Outlook. These solutions provide some benefits for us (less to backup on a daily basis) and for our users, there may be slightly slower access to archived emails which they are likely to need to access less. These solutions can typically be purchased on a per-mailbox basis (so would be relatively cost effective and affordable) and could be applied to senior staff needing to store more email.
Recommendations
1. Allow staff to request 200MB quota where their work requires that they deal with a large mailbox.
2. Allow staff a quota of greater than 200MB at the discretion of the Director of Information Systems Services.
3. Investigate other options (e.g. archiving as described above) as a way of enabling substantially higher quotas for some.
4. Radically re-think how this service that is so vital to many staff can be expanded over the longer-term. As we are constrained by the limits of our current resource base, the solution may involve a substantial investment in the service to meet the demands placed upon it.
5. Try to find short-term solutions that help the quotas grow within the current limitations (e.g. connect all exchange servers to the SAN and use some SAN based disk for faster initial backups)
6. Examine the implications of advising students to use the forwarding service to forward all their emails to an external email address.
Workgroup or Departmental shares
Another area where the Computing Service applies quotas is on workgroup or departmental shares. These are provided to groups of individuals (e.g. members of a department) as a shared file repository where all the group members can place files.
We currently allocate 30GB per share and let the users of each share arrange to control the space used. We could in theory increase the amount of space used by one department without adversely affecting others but would need to know how much is appropriate.
Recommendation
1. Allow an increased quota where suitable justification can be made.
Non-central and Third party backups
Finally, we backup our ISS systems for disaster recovery purposes but also offer the service to a select few others, namely departments, centres, and some associated organisations. When this was originally set up, there was a proposed one-off charge but this was rarely implemented. We therefore treat some areas of the University differently and absorb the cost centrally.
For the last month, the amount backed up on non-ISS servers was as follows.
Full backups for May 2006 by department:
Biology 525 GB
Data_Archive 178 GB
Estates 6 GB
Maths 85 GB
Incremental backups for May 2006 by department:
Biology 206 GB
Data_Archive 264 GB
Estates 4 GB
Maths 14 GB
The basic cost to provide this is around £0.30p per gigabyte so Biology (or more particularly, one research group in Biology) received over £200 of backup support free (at the point of delivery) last month.
We no longer offer the facility to backup additional servers but these legacy servers that we still backup have grown considerably in size over recent years.
Recommendations
1. Implement a policy of charging a flat-rate of £100 pa. plus £0.40p per gigabyte backed up from October 1st, 2006 (such a charge would be levied prospectively).
2. Set limits on ‘reasonable’ amounts that can be backed up for third-parties in order to keep the backup service performing satisfactorily within overall constraints.
3. Decide whether the University wishes ISS to continue offering this service – at present, we can support it due to the relatively low requirements, but this could change.
Bret Giddings, Systems Manager, ISS
Richard Murphy, Director of Information Systems
June 2006
4 / 4
[1] MB = Megabyte – 106 bytes, GB = Gigabyte – 109 bytes, TB = Terabyte– 1012 bytes