We were experiencing trouble with a module in our gateway router leading to a loss of connectivity between the department and the outside. We have restarted the module and we believe things are back up. We will be closely monitoring the systems tonight.

On Wednesday, January 17, 2007, from 8:00am to 8:05am EST, the CS Department IMAP/POP server will be taken down briefly for an upgrade. This should take no more than a minute, and most clients should barely notice the outage.

Users of Microsoft Outlook may need to close the program and restart in order to re-connect to the server.

This will be the first step in a transition to a new Solaris OS for the IMAP server.

On Tuesday, December 19, 2006, we will have a scheduled downtime from 6:00am to 8:00am EST. Please note that this partially overlaps an OIT-scheduled interruption of all campus networking services from 5:00am to 7:00am EST. For details on the OIT maintenance, see their posting.

Planned maintenance:

• Room 002 network switch firmware updates.
• Minor network configuration modifications.
• Routine OS updates on some of our servers.

We are experiencing difficulties and our systems are down due to a problem with our main file system server, depot. We are working the problem but do not have an estimate for when we will be back online.

UPDATE (9:45am): Systems are back online. As a result of the trouble, there was a short window where incoming mail was bounced back to the sender. Also, expect mail to operate slower than normal for a few hours while the backlog is processed.

On Tuesday, November 14, 2006, we will have a scheduled downtime from 4:00am to 8:00am EST.

Planned maintenance:

• We will reconfigure our network infrastructure at our gateway router to streamline our connection to OIT and the Internet.
• We will perform a minor version upgrade to MySQL on our database servers.

On Tuesday, October 31, 2006, there will be an interruption of all campus networking services from 5:00am to 7:00am EST. For details, see the OIT posting. During this time, CS systems will remain up but there will be an interruption in network connectivity outside the department.

On Thursday, October 26, 2006, we will have a scheduled downtime from 4:00am to 8:00am EDT.

Planned maintenance:

• We will copy user home directories from one disk array to another. This is the second step in a multi-step process to more effectively use our disk space. Note that we will not be copying the contents of the snapshot backups. If, in the next week or so, you wish to restore a file that would ordinarily be in a snapshot, please use the file restore form on the CS Guide and we will restore it from the nightly backup.
• We will re-install CentOS on the Linux cycle servers. Due to an unexpected (and frankly, strange) change in the installer, several software packages went missing in the transition from RHEL to CentOS during our October 10, downtime. We have revised our installation procedure and will re-install the operating system on these machines to get them in a known and repeatable state. NOTE: user crontabs on the penguin machines will be lost. Be sure to back them up before the downtime and restore them after the downtime.
• We will apply other miscellaneous patches to our systems.

On Tuesday, October 10, 2006, we will have a scheduled downtime from 4:00am to 8:00am EDT.

Planned maintenance:

• We will move a disk volume from one disk array to another. This is the first step in a multi-step process to more effectively use our disk space.
• We will transition the Linux cycle servers from RHEL to CentOS. Most users will not notice the change. However, user crontabs on the penguin machines will be lost. Be sure to back them up before the downtime and restore them after the downtime.
• We will apply other miscellaneous patches.

The trouble we were having yesterday was due to an overload on the main department firewall and was, in fact, due to people fetching the local copies of the voting machine demonstration video on the itpolicy site. Originally, all the content was distributed through CoBlitz which kept the load on the firewall down. Because CoBlitz uses non-standard ports (a current, but temporary PlanetLab limitation), many people were not able to view the video as originally posted. At around noon, we added links to local copies of the video to accommodate visitors who were in this situation. Demand for the local copies was higher than we anticipated — it may be that people were simply mirroring the site without trying to get the CoBlitz\’ed versions. We have since made some changes to the voting page and the situation has stabilized. It is not clear that the improvement was due to the changes we made or due to a drop in visitors. Looking at the CoBlitz load suggests that there was a drop in visitors to the site. We will continue to monitor the situation closely.

In the not-too-distant future, we will be deploying a new firewall which will be able to handle much higher loads.

We are having some issues with one of our departmental firewalls. It appears that it is due to an overload of traffic from the voting video. We are working with Ed and Alex to make some changes to the itpolicy page to reduce the traffic.