All

Date: Thursday, May 5, 2016 (07:30-08:00)

Who is affected:
Users of CS Department Email Services (IMAP, POP, webmail, etc.)

What is happening:
One of the two CS mailbox servers supporting the IMAP, POP, and Webmail
services will be shutdown for replacement of a defective DIMM. As a result,
users whose accounts are hosted by this server will experience
disconnection and an inability to read e-mail for a brief time while the
server is offline.

Why is it happening:
Hardware monitoring has indicated that a DIMM in this system is in a
pre-failure state and likely to fail soon. So as to avoid an unplanned
failure that may threaten the stability of the host, we are scheduling this
outage to replace the failing DIMM preemptively.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.
Sincerely,
CS Staff
_______________________________________________
downtime mailing list
downtime@lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/downtime

Date: Tuesday, March 15, 2016 (08:00-14:00)

Who is affected:
All users of the CS Department HPC Cluster (ionic.cs.princeton.edu).

What is happening:
On this day, the \”ionic\” cluster will be reinstalled and upgraded to the
latest OS distribution version, Springdale 7.2. This is a complex upgrade,
which is the reason for the unusually long outage window.

SPECIAL NOTE: As we are reloading the operating system, any crontabs on the
cluster will be lost. If you have crontabs that you wish to persist, you
will need to back up your crontabs before the downtime, and restore them
after.

Why is it happening:
This is part of normal maintenance of the publicly-accessible systems, and,
along with the patching of the cycles and penguins machines (announced
separately), will bring our primary public computational systems into
harmony running the same versions of Springdale Linux.

Please note that with this upgrade, some older versions of software, or
some packages which are no longer part of the distribution, may no longer
be available. We encourage you to verify your workflows after this upgrade
to ensure you are able to continue your work. CS Staff stands ready to
assist with any unforeseen trouble.

For most workflows, it should be possible to test your code on the cycles
machines in advance of the upgrade, as they are already running a version
of Springdale Linux that is close to the version to be installed during
this window. Some things will likely still differ after the upgrade, since
the cycles machines will also be updated, but the cycles machines are
receiving only minor updates, so breaking changes should be few or none.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.

Sincerely,
CS Staff
_______________________________________________
downtime mailing list
downtime@lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/downtime

Date: Tuesday, March 15, 2016 (08:00-09:00)

Who is affected:
All users of the CS Department public login servers (tux and opus, aka
\”penguins\”) and cycle servers (soak, wash, rinse, and spin, aka cycles\”)

What is happening:
During this window, these systems will be patched and updated from their
current Springdale 7.1 OS to the latest version, Springdale 7.2.

This is considered a minor update. Any resulting problems should be small
and relatively easily addressed. Since these machines are being patched
rather than reinstalled, cron jobs will remain intact.

Why is it happening:
This is part of normal maintenance of the publicly-accessible systems, and,
along with the upgrade of our HPC cluster (announced separately), will
bring our primary public computational systems into harmony running the
same versions of Springdale Linux.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.

Sincerely,
CS Staff
_______________________________________________
downtime mailing list
downtime@lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/downtime

Date: Thursday, February 18, 2016 (09:00-15:00)

Who is affected:
CS email users who attempt to interact with the Proofpoint spam filtering
system

What is happening:
We will be updating the Proofpoint spam filtering software on our mail
servers.

While no email outage is expected, it is possible that you may receive some
extra spam during the upgrade as the filters go offline for updates. You
may also find that you are unable to interact with the Proofpoint systems
during this time to perform actions such as releasing quarantined messages.
All mail sending, delivery, and reading mechanisms are expected to remain
operational.

Note that the maintenance window does not start until 09:00, so as to
provide some time after the 08:00 digests are sent out during which you may
release messages from the morning digest.

Why is it happening:
This is a necessary part of the regular maintenance of our systems.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.
Sincerely,
CS Staff
_______________________________________________
downtime mailing list
downtime@lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/downtime

Date: Tuesday, December 22, 2015 (06:00-09:00)

Who is affected:
All users of CS Department email services

What is happening:
All incoming email services will be shutdown while the Zimbra server
software is upgraded.

During this time, IMAP, POP, and Webmail will all be unavailable, as will
any email forwarding or filtering. All incoming messages should be safely
queued on our MTAs for delivery after the downtime. IMAP and POP users
should not notice any changes after the downtime. Webmail users will see an
updated interface.

Why is it happening:
This is a reschedule of an upgrade originally scheduled for November 5. At
that time, problems with the upgrade procedure resulted in an inability to
complete the upgrade during the scheduled window. In the intervening time,
several behind-the-scenes updates have taken place and paved the way for a
successful upgrade on this pass.

This upgrade will bring our mail server software up-to-date with the
current release version from Zimbra. Among other things, it will address
several recent vulnerabilities announced in various portions of the
software.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.
Sincerely,
CS Staff
_______________________________________________
downtime mailing list
downtime@lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/downtime

Date: Wednesday, November 25, 2015 (07:30-08:30)

Who is affected:
All users of CS Department public login (cycles/penguins/portal) and
compute (ionic) systems.

What is happening:
During this window, these machines will be rebooted in order to clear some
unkillable user processes which are interfering with some research work.

We will also take this opportunity to make configuration changes to reduce
the likelihood of recurrence for this and related issues.

Why is it happening:
As some user processes have entered an unkillable state, and as those
processes are preventing research work, they require a system reboot to
clear.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.
Sincerely,
CS Staff
_______________________________________________
downtime mailing list
downtime@lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/downtime

As you may have noticed, this upgrade was not completed this morning due to issues with the upgrade process. Some critical parts of the system were successfully upgraded, which should allow us to upgrade more components in a less-disruptive way, but the process is not completed at this time.

More updates will follow at a future date. Thanks for your patience.

-CS Staff

—– Original Message —–
From: csstaff@CS.Princeton.EDU
To: downtime@lists.cs.princeton.edu
Sent: Tuesday, October 27, 2015 11:38:30 AM
Subject: [downtime] CS Email Downtime, Thursday, November 5, 2015, 06:00-09:00

Date: Thursday, November 5, 2015 (06:00-09:00)

Who is affected:
All users of CS Department email services

What is happening:
All incoming email services will be shutdown while the Zimbra server
software is upgraded.

During this time, IMAP, POP, and Webmail will all be unavailable, as will
any email forwarding or filtering. All incoming messages should be safely
queued on our MTAs for delivery after the downtime. IMAP and POP users
should not notice any changes after the downtime. Webmail users will see an
updated interface.

Why is it happening:
This upgrade will bring our mail server software up-to-date with the
current release version from Zimbra. Among other things, it will address
several recent vulnerabilities announced in various portions of the
software.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.

Sincerely,
CS Staff
_______________________________________________
downtime mailing list
downtime@lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/downtime

Date: Thursday, November 5, 2015 (06:00-09:00)

Who is affected:
All users of CS Department email services

What is happening:
All incoming email services will be shutdown while the Zimbra server
software is upgraded.

During this time, IMAP, POP, and Webmail will all be unavailable, as will
any email forwarding or filtering. All incoming messages should be safely
queued on our MTAs for delivery after the downtime. IMAP and POP users
should not notice any changes after the downtime. Webmail users will see an
updated interface.

Why is it happening:
This upgrade will bring our mail server software up-to-date with the
current release version from Zimbra. Among other things, it will address
several recent vulnerabilities announced in various portions of the
software.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.

Sincerely,
CS Staff

UPDATE 08:09 : This downtime is being extended due to problems with the upgrade. At present, we hope to have service restored by 10:00. Further updates will be posted here as needed.

UPDATE 08:30 : After further work, rather than extending to 10:00, service is returned presently, and the upgrade process has been postponed to a later date. Thank you for your patience.

Date: Thursday, October 22, 2015 (06:00-08:00)

Who is affected:
All users of CS Department Email Services

What is happening:
All incoming email services will be shutdown for a brief period.

During this time, IMAP, POP, and Webmail will all be unavailable, as will
any email forwarding or filtering. All incoming messages should be safely
queued on our MTAs for delivery after the downtime.

Why is it happening:
This shutdown will allow us to move around a couple of the underlying
services supporting our email servers. This shuffle is necessary to prepare
for a major Zimbra version upgrade planned for the Fall Break time period
(more details to follow later), by retiring some older servers in favor of
newer versions that can better handle the coming upgrade.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.

Sincerely,
CS Staff
_______________________________________________
downtime mailing list
downtime@lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/downtime

Date: Tuesday, September 15, 2015 (06:00-08:00)

Who is affected:
All users of the CS Department public cycle servers (soak, wash, rinse, and
spin, aka \”cycles\”)

What is happening:
During this window, these systems will be replaced with newer hardware.
This also means their OSes will be reinstalled, but they will receive the
same Springdale 7.1 OS currently installed.

SPECIAL NOTE: As we are installing fresh OSes, all crontabs on the cycles
machines will be deleted. If you have crontabs that you wish to persist,
you will need to back up your crontabs before the downtime, and restore
them after. Any crontabs on tux or opus (\”penguins\”) will persist, as those
machines are not being reinstalled at this time.

Why is it happening:
This is part of the normal refresh cycle of the public login systems, and
will bring newer, faster processors and larger memory to the cycles
systems.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.
Sincerely,
CS Staff