Website

The website is the current public website for CES. It was built on CraftCMS by Mildly Geeky and released on July 13, 2016. It is hosted on the Web Production Server. It will be replaced by Rodeo two.

Last minute change of plans

Peter Stevens built the original web server on a CentOS-based Digital Ocean droplet. Patrick Harrington was tasked with deploying the code and migrating the data. On the night of July 12, 2016, Harrington called Stevens, who was on a swing set in Somerville, and explained that he had lost SSH access to the server. He expected that Fail2ban was locking him out for his repeated access. Stevens did not find any IPs banned in the blacklist. In the morning of the 13th, upon closer investigation, Stevens found that Harrington had mistakenly chmodded and chowned the entire /var directory, causing the server to no longer be usable. Harrington ran a script to reset all rights and ownership to their default values, but the script did not fix the issues with the server.

Due to financial deadlines, the launch of the site could not be delayed without the project becoming an FY 2017 project. The DNS name change had already been scheduled with NOC for noon. The ticket's assignee contacted Stevens by phone in the morning, but Stevens later forgot his name and contact information. He attempted to contact HUIT's help desk to reach the assignee of the ticket. Because there was not enough time to provision an identical server and deploy the site, Harrington offered to spin up an Ubuntu droplet and use ServerPilot to deploy by the afternoon. Stevens hesitated, due to his preference for CentOS, but relented because of the time restriction. Harrington began provisioning the server.

At 11:40am, Stevens had not yet been able to contact the assignee, but he knew that he was located at 60 Oxford Street. He sprinted up Oxford street in his dress shoes. He arrived within a few minutes but was unsure of NOC's office location. He explained the situation to a passerby, who used his ID badge to give him access and helped locate the assignee of the ticket. At around 11:55, they found another HUIT employee who pulled up Stevens's ticket history and went through one by one to find the request for a DNS change. The employee found the ticket and saw that it was being updated with old DNS information. The assignee was beginning the change, which would point CES's public domain to a broken server. The assignee was James Perlman, who was in a cubicle twenty feet away. Stevens and the two HUIT employees walked over to Perlman's cubicle and said, "Stop the change. The server is borked."