OIT Unplanned Outage

Thursday, March 13, 2025

The Office of Information Technology is investigating a service disruption affecting several applications and services. At this time, faculty, staff, researchers and students may have difficulty accessing WiFi and signing-in to various applications and services.

Updates are planned every 30 minutes or as information becomes available.

Due to the disruption to campus WiFi, please use mobile phones to dial 911 or to reach emergency services.

Updates will be provided as information becomes available.

Impacted services include, but not limited to: Eduroam WiFi, PeopleSoft (TigerHub, Self-Service), Canvas, ServiceNow, Listserv, GlobalProtect VPN, Prime, Virtual Machines.

Outage Status

Investigating

Updates

9:09 p.m.

The IT disruption affecting several Princeton University services has been resolved. The eduroam WiFi network and other services are now available.

If you continue to experience any issues, please contact the Service Desk via live chat, email or by opening a ticket.

We apologize for the inconvenience caused during this time and appreciate your patience.

8:10 p.m.

OIT has identified an issue related to network equipment and has implemented a fix. We are monitoring progress to confirm service has been restored.

Campus Update: March 13, 5:20 p.m.

The response to the ongoing OIT disruption continues. OIT is engaging with the respective vendor(s) and continues to investigate. 

Many Princeton University applications and services are unavailable. At this time, it may not be possible to use the single-sign-on service to sign-in to applications. Campus WiFi via the eduroam network is also affected.

Critical life safety systems such as the fire alarm system and 911 remain operational, but the OIT team has identified that some desk phones are no longer connected to the network. For that reason, the University recommends mobile phones to reach 911 or emergency services.

Additional personnel from Public Safety and Facilities will be on campus until the outages are resolved.

Due to the unavailability of some systems related to remotely monitoring building access, the following actions will be taken this evening:

• Administrative buildings were manually locked at 5 pm. Card access is disabled.
• Academic and research buildings will be manually locked at 8:30 pm.  In the meantime, local alarm systems such as fume hoods and refrigeration alarms may not work in these buildings. After 8:30 pm, these buildings should be vacated. Only emergency access will be permitted after this time.
• Residence halls will remain on their normal locking schedule. Students should make sure to bring their TigerCards when leaving their buildings as systems enabling temporary cards are not functioning.  
• Firestone Library circulation will remain open until 9 pm and the study areas will be open until 11 pm. Firestone Special Collections, the Milberg Gallery, and the Cotsen Children’s Library are closed for the remainder of the day.
• Campus recreational facilities will remain open as scheduled.

5:10 p.m.

Our team has implemented a temporary workaround, enabling off-campus sign-in access to cloud-based solutions, such as Zoom, Canvas, GlobalProtect VPN, and ServiceNow.

3:30 p.m.

Investigation into the disruption continues. The issue may be related to hardware that controls internet traffic. OIT is engaging with the respective vendor(s) and continues to investigate.

2:10 p.m.

Our team has also identified that some desk phones are no longer connected to the network. For that reason, we recommend using mobile phones when dialing 911 or reaching emergency services. 

1:43 p.m.

The data center cooling has been restored. It may take 6-12 hours for Research Computing storage to come back online.

1:22 p.m.

OIT is also investigating overheating equipment in the data center. At this time, Research Computing equipment is being shut down. Additional updates to follow. 

1 p.m.

Investigation into the disruption continues. The issue may be related to hardware that controls internet traffic. OIT is engaging with the respective vendor(s) and continue to investigate.

12:30 p.m.

Technicians are currently restarting various systems and equipment as we work to identify the root cause. We are working to restore service as soon as possible. 

12 p.m.

OIT continues to investigate the root cause of the service disruption and is working to restore service as soon as possible. 

11:20 a.m.

OIT continues to investigate the root cause of the service disruption and is working to restore service as soon as possible. 

10:30 a.m.

The Office of Information Technology is investigating a service disruption affecting several applications and services. Additional communications are expected as impact and restoration information becomes available.

Many Princeton University applications and services are unavailable. At this time, it may not be possible to use the single-sign-on service to sign-in to applications.

Updates from OIT will be posted at https://oit.princeton.edu/oit-unplanned-outage 

After the database upgrade this morning we are having an issue. We are working on the issue and should be back soon. We will post updates here.

TigerFile is now working. If you have any issues please email csstaff.

OIT had an outage last night at the HPCRC. We are working this morning to bring servers back online.

Email is back online. 5/28/2023 8:45AM

Web servers are back online 5/28/2023 9:00AM

UPDATE: As of 11:55 AM Zimbra is back up and running.

We are currently experiencing issues with Zimbra Webmail. Zimbra is reporting a network error when accessing Webmail. We are working to address the issue and will post updates here.

Business Service:
Campus Data Network (CDN Wired)
Start Time:
08/07/19 12:51 PM
End Time:
Last Update:
08/07/19 02:09 PM
Details:
We received reports of slow network performance on campus. OIT Network Administrators are aware of the problem and currently investigating.

More Information can be found at https://princeton.service-now.com/snap/?id=outage&sys_id=cc578ef2db8f7b000dc79a82ca961910&table=cmdb_ci_outage&spa=1

Date: Tuesday, May 3, 2016 9:30PM

Who is affected:
Users of CS Department Services

Problem:
We are currently having issues with the CS file server. We are working to restore service and will post updates here as we learn more. CS Staff is currently on-site at the data center. We are working with the vendor to track down the issue. We should have another update by 10:00 PM.

Update 10:00 PM:

Services are starting to get restored. We are now working to bring things back online. We will have another update at 10:30 PM

Update 10:30 PM:

We are still in the process of restoring services. We will post another update at 11:00 PM

Update 11:00 PM:

We had to reboot all the file server nodes and we have one node left to reboot. Once the nodes come back online we will need to check and maybe reboot some CS servers to restore all services back to normal. We will post another update at 11:30 PM.

Update 11:30 PM:

All file server nodes are back online and working as expected. We are in the process of checking on each CS server and rebooting if needed. We will post another update at 12:00 AM.

Update 12:00 AM:

We are still checking on the status of all the CS servers. Some services have already been restored. We will post another update at 12:30 AM.

Update 12:15 AM:

Most CS services have been restored. We have a few servers that are still coming up.

Update 12:30 AM:

All CS services have been restored. Please let us know if you experience any continued trouble.

6:10 PM We are currently experiencing network issues. We are looking to resolve the problem and should have another update soon.

6:20 PM There is presently a Campus wide Network outage ongoing. OIT Administrators are aware of this and are working to restore service ASAP. Updates can be found at http://helpdesk.princeton.edu/outages/view.plx?ID=5226

At 8:40pm OIT Technical Operations and Network Monitoring Services staff swapped in new hardware and all buildings listed below have been stable since that time.

New South building network (does not impact any systems in the New South Data Center)
Computer Science
Eno Hall
1879/Marx Hall
Firestone Library
Nassau Hall
Guyot/Moffett/Schultz
Maclean House
Jadwin Hall
Lewis Thomas Lab

Jul 10, 2015 @ 3:40PM We are currently experiencing some service outages and are working to resolve the problems. Email is currently one of those services not working. More updates to come soon…

Jul 10, 2015 @ 4:05PM We have located the cause of the problem and services are starting to get restored now.

As part of the network relocation work, we will need to bring down the
wireless network in the CS Building. This work will take place
6:00am-9:00am tomorrow morning (1/31/2014).

Users may have received a phishing attempt to their CS e-mail account. The subject may be \”Account Login Alert!\” or \”Princeton Security\”.

Please be aware that there seems to be a coordinated spear-phishing campaign targeting the CS Department. Various e-mails are being delivered to accounts making assorted claims from improper password use to free money or local news and events.

Remember to always exercise caution with e-mail and web sites. These messages prey on your inattention in order to steal your credentials and use them for various nefarious purposes. DO NOT CLICK SUSPICIOUS LINKS, and DO NOT TYPE YOUR USERNAME OR PASSWORD on any site that you are not 100% sure belongs to the CS Department.

If you are not sure, please feel free to forward suspicious messages to CS Staff (including full headers if you can) and we can verify whether or not the message is valid.

If you may have compromised your account, contact CS Staff

If you need any additional assistance, please contact the CS Staff at any time.