All

Date: Wednesday, March 22, 2017 (07:00-08:00)

Who is affected:
All users of CS Department public login (cycles/portal) and
courselab systems.

What is happening:
During this window, these machines will be rebooted in order to clear some
defunct user processes which are interfering with some research work.

Why is it happening:
As some user processes have entered a defunct state, and as those processes
are preventing research work, they require a system reboot to clear.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.

Sincerely,
CS Staff
_______________________________________________
downtime mailing list
downtime@lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/downtime

Date: Wednesday, February 1, 2017 (07:00-08:00)

Who is affected:
All users of departmental webservers, including user webspace
(www.cs.princeton.edu/~${user}), project webspace
(${projectname}.cs.princeton.edu), departmental web services
(www.cs.princeton.edu/courses/), and other internal applications
(iw.cs.princeton.edu et al).

What is happening:
During this window, OS patches will be applied to update our managed
systems to Springdale 7.3. After patching, all systems will be rebooted.

Note that though our user and project webservers will be rebooted, this
maintenance will not affect any user-owned files in either user or project
webspace directories.

Why is it happening:
This is part of regular maintenance to keep systems up-to-date. In
addition, this will bring the programming environment for userspace CGI in
line with the environment available on the cycles machines.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.

Sincerely,
CS Staff
_______________________________________________
downtime mailing list
downtime@lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/downtime

Date: Wednesday, December 28, 2016 (05:00-09:00)

Who is affected:
All users of the CS Department public cycle servers (soak, wash, rinse, and
spin, aka \”cycles\”) and course lab servers (courselab01 and courselab02,
aka \”courselab\”); all users of the CS Department HPC Cluster
(ionic.cs.princeton.edu).

What is happening:
During this window, OS patches will be applied to update our managed
systems to Springdale 7.3. After patching, all systems will be rebooted.

Why is it happening:
This is part of regular maintenance to keep systems up-to-date.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.

Sincerely,
CS Staff
_______________________________________________
downtime mailing list
downtime@lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/downtime

Date: Thursday, November 17, 2016 (07:30-08:00)

Who is affected:
All users of the CS Department network

What is happening:
The CS Department uplink to OIT\’s network will be temporarily unavailable.
The actual outage time should be only a few seconds, but we are allocating
a half hour for this task in case of trouble.

Why is it happening:
This work will upgrade the uplink between the CS Department and OIT,
providing more total bandwidth to the rest of campus and onward to the
internet.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.

Sincerely,
CS Staff
_______________________________________________
downtime mailing list
downtime@lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/downtime

Date: Tuesday, September 6, 2016 (07:00-09:00)

Who is affected:
All users of the CS Department public cycle servers (soak, wash, rinse, and
spin, aka \”cycles\”) and course lab servers (courselab01 and courselab02,
aka \”courselab\”)

What is happening:
During this window, the cycles systems will be replaced with newer
hardware, and the courselab systems will be reinstalled and updated with
the latest distribution version of Springdale Linux 7.2.

SPECIAL NOTE: As we are installing fresh OSes, all crontabs on the cycles
and courselab machines will be deleted. If you have crontabs that you wish
to persist, you will need to back up your crontabs before the downtime, and
restore them after.

Why is it happening:
This is part of normal maintenance of the publicly-accessible systems, and
will bring newer versions of installed tools and software.

Please note that with this upgrade, some older versions of software, or
some packages which are no longer part of the distribution, may no longer
be available. We encourage you to verify your workflows after this upgrade
to ensure you are able to continue your work. CS Staff stands ready to
assist with any unforeseen trouble.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.
Sincerely,
CS Staff
_______________________________________________
downtime mailing list
downtime@lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/downtime

Date: Wednesday, July 27, 2016 (09:00-11:00)

Who is affected:
All users of the CS Department Courselab servers (courselab01 and
courselab02).

What is happening:
During this window, these systems will have their OSes reinstalled and
upgraded to the latest distribution version, Springdale 7.2.

Why is it happening:
This is part of normal maintenance of the publicly-accessible systems, and
will bring newer versions of installed tools and software.

Please note that with this upgrade, some older versions of software, or
some packages which are no longer part of the distribution, may no longer
be available. We encourage you to verify your workflows after this upgrade
to ensure you are able to continue your work. CS Staff stands ready to
assist with any unforeseen trouble.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.

Sincerely,
CS Staff
_______________________________________________
downtime mailing list
downtime@lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/downtime

Date: August 1, 2016

Who is affected:
All users of the CS Department public login hosts, tux.cs.princeton.edu and
opus.cs.princeton.edu (penguins.cs.princeton.edu).

What is happening:
On August 1, 2016, tux and opus will be turned off and retired. Future
logins should use cycles.cs.princeton.edu (soak, wash, rinse, or spin).

NOTE: As these servers are retiring, all crontabs on tux and opus will be
retired with them. If you need to maintain a crontab currently active on
one of these systems, you will need to relocate it to one of the cycles
machines before August 1.

The DNS names \”tux\”, \”opus\”, and \”penguins\” will all remain for a period of
at least six months, and will become aliases for \”cycles.cs.princeton.edu\”.
We encourage you to use the time to update your scripts, configurations, or
other references to the retiring names.

Why is it happening:
The purpose of tux and opus, for the last several years, has been to
provide a space for lightweight interactive work such as reading email or
organizing files. This was specifically intended to separate these
interactive activities from the more computationally intensive work done on
the cycles servers, so as to reduce the incidence of conflict.

In recent years, advances in kernel technology and our configuration
management systems have enabled us to provide a more stable and fair
environment in the cycles servers such that most users can maintain a
reasonable share of system resources, even while other users are doing
computationally intensive work. For this reason, the separation of the
penguins servers is no longer as useful as it once was, and the costs of
maintaining the distinct system configurations (as well as user confusion
resulting in computationally intensive work running on penguins) has risen
enough to outweigh the benefits.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this system retirement will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.

Date: Tuesday, June 21, 2016 (05:00-08:00)

Who is affected:
All users of the CS department computing.

What is happening:
We are upgrading our storage operating system, which requires for CS
storage to be rebooted. All services that depend upon access to storage
will be unavailable, including – cycle servers, ionic cluster, web content,
home directories, CIFS, etc.

Why is it happening:
This is necessary in order to fix numerous bugs on our file system.

We will post updates to the status page: http://www.csstaff.org
as necessary.

If this downtime will cause you undue hardship, please contact
csstaff@cs.princeton.edu immediately, so we can discuss options to reduce
any negative impact. Your patience is appreciated.

Sincerely,
CS Staff
_______________________________________________

Update – 08:05 – An unexpected problem with the wrap-up up this morning\’s maintenance has resulted in a widespread outage of CS Department Services. We are working to restore service ASAP.

Update – 09:02 – We are still working to restore service ASAP.

Update – 09:20 – We are in the process of bringing services back online. We will have another update at 9:45.

Update – 09:48 – Most services are now back online. We should have everything restored by 10:00.

Update – 10:14 – A few services are still coming up. SMTP server is not up yet so sending email is not working. You can use webmail.cs.princeton.edu to send and receive email.

Update – 10:50 – SMTP service is now working.

Date: Tuesday, May 3, 2016 9:30PM

Who is affected:
Users of CS Department Services

Problem:
We are currently having issues with the CS file server. We are working to restore service and will post updates here as we learn more. CS Staff is currently on-site at the data center. We are working with the vendor to track down the issue. We should have another update by 10:00 PM.

Update 10:00 PM:

Services are starting to get restored. We are now working to bring things back online. We will have another update at 10:30 PM

Update 10:30 PM:

We are still in the process of restoring services. We will post another update at 11:00 PM

Update 11:00 PM:

We had to reboot all the file server nodes and we have one node left to reboot. Once the nodes come back online we will need to check and maybe reboot some CS servers to restore all services back to normal. We will post another update at 11:30 PM.

Update 11:30 PM:

All file server nodes are back online and working as expected. We are in the process of checking on each CS server and rebooting if needed. We will post another update at 12:00 AM.

Update 12:00 AM:

We are still checking on the status of all the CS servers. Some services have already been restored. We will post another update at 12:30 AM.

Update 12:15 AM:

Most CS services have been restored. We have a few servers that are still coming up.

Update 12:30 AM:

All CS services have been restored. Please let us know if you experience any continued trouble.

At this time, CS Department services are experiencing a widespread outage. We are aware of the problem, and working to bring things back. More updates will be posted as we learn more.