UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved, and we have implemented what we believe to be a permanent fix for the recent issues with NYRYZEN102. We are continuing to monitor this server closely.
If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: It appears to be a PDU related issue within the rack, on-site datacenter engineers are looking into it.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Affected host-nodes - DAL155KVM,DALRYZEN104,DAL124KVM, DAL123KVM,DALRYZEN102
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Update : The switch that this server is connected to is currently undergoing maintenance. Short downtime is expected until 13:00 UTC+1. We would notify once the maintenance is over. There was an Email that should have Informed you about this. Please refer to the email - "02/21/2024 Scheduled Maintenance - Miami Datacenter - Feb/29/2024" .
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Update : On the switch the server is currently connected to is under the maintenance. The maintenance should be done today at 12:00 UTC. We would notify once the maintenance is over. There was an Email that should have Informed you about this. Please refer to the email - "02/21/2024 Scheduled Maintenance - Miami Datacenter"
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
.
]]>We will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>This maintenance is now in progress, and being actively worked on. To summarize the maintenance event, an additional carrier will be added to the BGP Blend to significantly enhance connectivity in our Singapore datacenter location. There may be very brief periods of interruption as traffic shifts over.
UPDATE:The scheduled maintenance is now completed
]]>SJ100KVM - This is now completed SJ101KVM - This is now completed SJ103KVM - This is now completed SJ155KVM - This is now completed SJ160KVM - This is now completed SJ102KVM - This is now completed SJ113KVM - This is now completed
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now fully resolved. Please feel free to open a support ticket if you are still experiencing any issues and we will be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Our team is actively working on resolving this matter, and we appreciate your understanding and patience as we address and rectify the situation with our domain registrar.
UPDATE: This is now permanently resolved and we’ve since implemented additional redundant measures to prevent this from happening again in the future. All websites on our shared hosting nameservers and main server hostnames should now be loading without any issues.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We have this resolved for most of the sites already and are still working on it. We'll update this status incident once we have a full resolution on this.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This node is still experiencing symptoms which will required extended offline troubleshooting, to help avoid prolonged downtime we are going to perform migrations of virtual machines over to another physical host node at this time (VM's will be migrated to another stable host node, NYRN222KVM). All data will be migrated intact and you will not notice any changes on your end (your VPS IP will also remain the same).
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now fully resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is still being actively troubleshooted, it appears to be a hardware issue.
UPDATE: We are replacing the affected hardware of the server. It will be back online shortly.
UPDATE: Remote hands in San Jose is still working on this. We will update once they provide us with additional information.
UPDATE: We've implemented a temporary solution to get VM's back online at this time. A permanent fix is in the works.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: 7950XNY206 will be going offline for at least the next 1-2 hours, it appears this particular host node is continuing to give us trouble and we are going to work on additional/further troubleshooting to attempt to find a permanent resolution for the issues with this host node. Thank you for your patience and we will share additional updates once available.
UPDATE: We are nearing a resolution here, please standby.
UPDATE: We've replaced several hardware components within this server that we believe to have been problematic in our findings. This host node is now back online and VM's should now be accessible. We will continue monitoring closely to ensure the stability of this host node.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We are working on troubleshooting the hardware issues on this server with the goal to implement a permanent resolution. We sincerely apologize for the recent issues of this host node.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>It appears to be an isolated switch issue which is causing sporadic connectivity issues. We will update once this is resolved.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>7950XNY203
We will update this status incident once the maintenance window has been completed.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>7950XNY201
We will update this status incident once the maintenance window has been completed.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>7950XNY200
We will update this status incident once the maintenance window has been completed.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>If your VPS is hosted on this node, you may experience intermittent periods of degraded performance during this time. We appreciate your patience and will update this incident once the RAID rebuild has been completed.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>7950XNY206
We will update this status incident once the maintenance window has been completed.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: A resolution strategy has been identified and put into motion (affected clients were sent an email with details). We will be closing out this status incident for the time being, we remain available 24x7 to assist. Please feel free to reach out to us via support ticket if you have any questions.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: The issue has been resolved with the exception of 4 nodes which are still offline: SJ206KVM, SJ214KVM, SJ211KVM, and SJ207KVM
We are investigating the issue of these 4 nodes (SJ206KVM, SJ214KVM, SJ211KVM, and SJ207KVM) and will share another update within this status incident once more information is available.
UPDATE: We resolved the issue with connectivity on those 4 nodes. All nodes mentioned in this status incident are now back online. If your VM is hosted within any of these mentioned nodes - you should be back online by now, but if you are still experiencing issues, please open a support ticket and we'll be sure to assist.
]]>7950XNY202
We will update this status incident once the maintenance window has been completed.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We are still looking into the issue on this server -- it does appear to be hardware-related. On-site datacenter engineers are actively working on it.
UPDATE: A resolution path is underway and actively being implemented by the technicians. We are continuing to monitor this, and we will update this status incident again once a full conclusion has been reached.
UPDATE: A resolution strategy has been identified and put into motion (affected clients were sent an email with details). We will be closing out this status incident at this time, we remain available 24x7 to assist. Please feel free to reach out to us via support ticket if you have any questions.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Update : This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Affected host-nodes - ATL135KVM ATL110KVM ATL133KVM ATL101KVM ATL106KVM ATL134KVM
Our network engineering team is currently looking into this further. We will have more information on this shortly.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: As this looks to be a network-related issue impacting these servers - the issue has been escalated. Our upstream and network engineers are actively looking into the issue. We will update this status page with more information once we have an update from them. We appreciate your patience and please know that this is our top priority to bring back online.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This was resolved within 7 minutes of the incident, we apologize for the brief interruption. Services should be back online and accessible at this time.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
Update : This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: The temporary routing issue has now been fully resolved and services are now stable. We apologize for the inconvenience this slight blip may have caused. If you are still experiencing any issues, feel free to reach out to us and we'd be happy to help.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>This maintenance is now in progress, and being actively worked on. We will provide additional updates within this status incident once available. Thank you.
UPDATE: This is still actively being worked on. We will continue to share additional updates within this status incident once we know more information.
UPDATE: Everything is moving along at a good pace and we are on track for this to be completed before the original maintenance end time. We will continue to share additional updates within this status incident once more info is available. Thank you for your patience!
UPDATE: Servers have been successfully transported safely to the new Singapore facility. We will share additional updates once available.
UPDATE: sg3000 - We have completed the maintenance for this node. Please standby for further updates for the remaining servers.
UPDATE: At this stage, all servers have been successfully racked at the new facility, and we're currently in the process of configuring the networking aspect of things. We are nearing the finish line of the maintenance window here and we appreciate your patience. Once we have additional details or information to share, we will be sure to do so within this status page incident.
UPDATE: The maintenance was completed as of approximately two hours ago and things have been stable since, we will be closing out this status incident as resolved now.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: The repeated issues on this node appear to be linked to a network interface firmware issue. This has since been corrected, and we will continue to monitor the status of this server closely. Thank you for your understanding and we apologize for any inconvenience caused. All VM's hosted on this server are now back online.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This server appears to be under a DDoS attack. We have enabled advanced mitigation to mitigate the attack, and websites are loading at this time. Please note that you may experience false positives from third party monitoring services during this time due to enhanced mitigation measures in place.
UPDATE: This is now fully resolved and attack has been fully mitigated.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Maintenance Start Date: November 28, 2023
Maintenance Start Time: 10:30 AM PST
Maintenance End Time: 11:00 AM PST
Affected Services: RackNerd Website & RackNerd Client Portal
Scope of Work: We will be implementing backend related improvements/updates to the above services. The frontend of our website and the client area portal may be unavailable for up to 30 minutes during this time.
Client servers will not be affected, and will remain online.
If you require assistance during this maintenance timeframe, please e-mail us at support@racknerd.com, or submit a support ticket in our client area after the maintenance window is completed.
We will be moving this status incident to active once the maintenance window commences.
UPDATE: This maintenance is about to commence. We will share another update once this is completed.
UPDATE #2: This maintenance is now completed, and our website and client portals are now available again. We will be closing out this status incident as resolved now. Feel free to open a support ticket if you have any questions.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Maintenance Start Date: November 21, 2023
Maintenance Start Time: 11:00 AM PST
Maintenance End Time: 11:30 AM PST
During this maintenance, we will be adding additional resources and performing other related updates to our underlying infrastructure that empowers our website (racknerd.com), client portal (my.racknerd.com) and SolusVM Control Panel. This upgrade is in response to increased traffic and demand. Our website and client portal features may be unavailable for up to 30 minutes during this time.
Client services will not be affected, and will remain online as usual.
If you require assistance during this maintenance timeframe, we will remain available via email during the maintenance at support@racknerd.com, or alternatively you may submit a support ticket in our client area after the maintenance window is completed.
We will be moving this status incident to active once the maintenance window commences.
UPDATE: This maintenance is about to commence. We will share another update once this is completed.
UPDATE #2: This maintenance is now completed, and our website and client portals are now available again. We will be closing out this status incident as resolved now. Feel free to open a support ticket if you have any questions.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: DAL109KVM - This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist. Please standby for the updates on DAL126KVM
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
UPDATE : It's down again
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>The timeframe provided to us by our datacenter provider in this location is as follows:
START TIME: 10/24/2023 12:00 AM Eastern
END TIME: 10/24/2023 12:00 PM Eastern
We will provide additional updates within this status incident once available. Thank you.
UPDATE: The migration process is actively being worked on by datacenter engineers. We anticipate a smooth transition and all IPs will remain the same. We will continue to share additional updates within this status incident once we know more information.
UPDATE: Things are still moving along at a good pace, and the servers are actively in the process of being transported by vehicle. We sincerely appreciate your patience and we will update once additional information is available.
UPDATE: The datacenter technicians are nearing arrival to the destination facility and we will provide additional updates once available.
UPDATE: The servers have safely arrived to the destination facility a little while ago, and we are preparing to rack the servers. We’ll continue to share updates within this status incident as they become available.
UPDATE: Most servers are re-racked and back online, we are working on the final batch at the moment and should be 100% completed shortly.
UPDATE: The datacenter migration has been completed successfully and all services are now back online. Feel free to open a ticket with us if you require any assistance.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this status incident once additional information is available.
UPDATE: This is now completed.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Our network engineering team is currently looking into this further. We will have more information on this shortly.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>NYRN138KVM, NYRN146KVM
We will update this status incident once the maintenance window has been completed.
UPDATE :The maintenance has been completed successfully for NYRN146KVM. If you are still experiencing any issues, please open a support ticket, and we'd be happy to assist. Please stand by for further updates on NYRN138KVM.
UPDATE: This is now completed for NYRN138KVM as well. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Network engineers at the upstream are currently looking into this and we will update this status incident once we have more information available.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This looks to be impacting only one of our cabinets in this location (Rack 5.3.15) -- in this cabinet, the following host nodes are impacted: ATL100KVM, ATLRYZEN103, ATL132KVM, ATL116KVM, ATL104KVM, ATLRYZEN101, ATL135KVM, ATL129KVM, and ATL108KVM
We are currently awaiting feedback from the remote hands team for further feedback on this. We appreciate your patience and will post another update once available.
UPDATE: We received a response from the datacenter team just now that they are working on this rack at this time. We believe the outage at this time to be PDU/power related. We hope to have this resolved very soon and we will update once we have additional information on this matter.
UPDATE: Good news that the following host-nodes are back online – ATLRYZEN103 ATL132KVM ATL104KVM ATL135KVM ATL108KVM – Please standby for further updates on the status of the rest of the nodes in outage.
UPDATE: This particular host-node is back online at the moment - ATLRYZEN101 - We will update this incident with additional information once available.
UPDATE: Except this particular host-node - ATL116KVM , all other nodes are back online. We would resolve the pending node in outage shortly. Please standby for further updates on the status of the node in outage.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>SJ132KVM, SJ127KVM, SJ128KVM, SJ203KVM. SJ157KVM. SJRYZEN101, SJRYZEN102
We will update this status incident once the maintenance window has been completed.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: DDoS filtering is active across the network, including ICMP filtering. That being said, you may notice false positives in third party monitoring services during this time due to the nature of the DDoS filtering, though services should be online and accessible. If you are still experiencing any issues, please attempt to reboot your VPS from within the SolusVM control panel, or open a support ticket for assistance from our Technical Support team.
We will continue to monitor this situation closely to ensure network uptime. We will move this incident to "Watching" status for the meantime.
UPDATE: This is fully resolved.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Our network engineering team is currently looking into this further. We will have more information on this shortly.
UPDATE : NYRN120KVM,NYRN121KVM,NYRN156KVM - All these three nodes are back online. We will update about this particular node - NYRN124KVM shortly.
UPDATE : This is NYRN124KVM - now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues, please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>SJ170KVM, SJ149KVM, SJ151KVM, SJ104KVM, SJ106KVM, and SJ105KVM
We will update this status incident once the maintenance window has been completed.
UPDATE: The maintenance window is now fully completed. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This issue has now been resolved by our upstream provider. Please feel free to open a ticket if you are still experiencing any issues in this location.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We are still looking into this node. We will update once we have more information.
UPDATE: We have narrowed down the issue and we are currently awaiting additional feedback from our remote hands team in ATL. We appreciate your continued patience.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
UPDATE : It's down again. We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
UPDATE : It's down again. We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this status incident once the maintenance window has been completed.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>SJ117KVM, SJ118KVM, SJ125KVM
We will update this status incident once the maintenance window has been completed.
UPDATE: Maintenance is still ongoing and we apologize for the delay. We will update once we have received an update from the datacenter engineers on the ground.
UPDATE: SJ125KVM is back up and running. We'll update on the remaining two devices shortly.
UPDATE: SJ117KVM and SJ118KVM are still being worked on by the datacenter engineers on the ground in San Jose. We appreciate your patience and will share additional updates once available.
UPDATE: We are still awaiting a response from the remote hands team in San Jose. We are applying maximum pressure on our end and hope to have further updates on this soon. We do apologize for the unexpected delay in getting SJ117KVM and SJ118KVM back online.
UPDATE: We are still working with the remote hands team in San Jose regarding SJ117KVM and SJ118KVM. We will share additional updates once available.
UPDATE: We are making progress and anticipate service for SJ118KVM being restored soon. ETA for service restoration on SJ118KVM is Aug/22/2023 6:00 AM Pacific, potentially much sooner than that.
UPDATE: SJ118KVM is now back online, and VM's hosted upon this node are now accessible. We are actively working on SJ117KVM and will update once we have additional information.
UPDATE: Current ETA for full service restoration on SJ117KVM is August 23, 2023 at 1:00 PM Pacific. There is a possibility it will be fully restored sooner. We will update once additional information is available.
UPDATE: SJ117KVM will be coming back online shortly.
UPDATE: SJ117KVM is still on track to be coming back up shortly. The server was repaired and working on the test bench, at the moment we are waiting on our remote hands request with our San Jose datacenter to re-rack the server into our production environment.
UPDATE: Work on SJ117KVM has been completed and VM's hosted upon this host node are now back online. Please feel free to reach out if you are still experiencing any issues, or if you have any questions.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is still being worked on.
UPDATE: A resolution path is underway and actively being implemented by the technicians. We are continuing to monitor this, and we will update this status incident again once a full conclusion has been reached.
UPDATE: A resolution strategy has been identified and put into motion. We will be closing out this status incident at this time, we remain available 24x7 to assist. Please feel free to reach out to us via support ticket if you have any questions.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Our network engineering team is currently looking into this further. We will have more information on this shortly.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: A resolution strategy has been identified and is being put into motion. We have sent an email to customers hosted on ATL106KVM with additional information.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: The maintenance is now completed.
]]>We are currently looking into these nodes further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>SEA124KVM, SEA125KVM, SEA102KVM, SEA130KVM, SEA106KVM, SEA109KVM, SEA108KVM, SEA107KVM, SEA132KVM, SEA101KVM, SEARYZEN100, SEARYZEN101, SEA112KVM
We will update this status incident once the maintenance window has been completed.
UPDATE: Great news -- ahead of schedule, the maintenance window is now fully completed. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We are having our on-site team replace the motherboard on this server. We ask for your patience during this time.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
UPDATE: It has high packet loss . We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update once everything looks to be fully resolved from our end.
UPDATE: Things look to be stabilizing. Before we close this out as fully resolved, we will continue to keep a close eye on the network to ensure everything remains optimal. In the meantime, we will move this incident to "Watching" status.
UPDATE: We have been monitoring the status of the Los Angeles DC-02 network, and conclude it to be stable, without any issues or hiccups. This is now considered fully resolved and we will now proceed to close out this status incident.
]]>Network Engineers are actively looking into this issue, and our plan at the moment is to re-route against the problematic carrier. In the meantime, you should not notice a complete outage or downtime during this time, as only a select few network routes seem to be impacted.
ETA for a full and complete resolution on this is 1 hour or less.
UPDATE: This is now resolved, all routes should be back to normal. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: As remotely accessing & rebooting this server did not resolve connectivity, we've requested assistance of our datacenter's remote hands team to physically troubleshoot this machine further.
While they diligently work on this, we ask for your patience and understanding. As soon as we have more information, we will share it here.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now completed. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now completed. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We are still looking into this. Thank you for your patience.
UPDATE: The RAID Controller on this server appears to be having issues. Our systems administrators are currently troubleshooting further with the on-site datacenter technicians accordingly. Further updates to follow.
UPDATE: We are still working on this. We will be sure to post additional updates once available.
UPDATE: A RAID rebuild is currently underway for this node. To maintain integrity and ensure best practices given the situation, we will conduct this operation offline. Based on our current observations, the rebuild is advancing at a rate of approximately 1% every 5 to 10 minutes. We ask for your patience as this procedure moves through the process.
UPDATE: This matter is now addressed. We have communicated additional details to customers hosted on this node via email. Please don't hesitate to contact us if you're still encountering any problems or if you need further assistance.
]]>UPDATE: This is now completed. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now completed. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Existing customers on ATL115KVM were sent an email notification with details. We will update this status incident as resolved once the RAM replacement is completed.
UPDATE: We have placed the remote hands ticket request with our Atlanta datacenter partner. We are currently awaiting for the remote hands team to work on this request.
UPDATE: This is now completed.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>During this time, if you have a VM hosted upon this physical host node, you may notice temporary, slower than usual disk performance.
Given the nature of RAID array rebuilds, this issue may take up to 72 hours until a full resolution is met.
UPDATE: Please note that due to the nature of the degraded RAID array, you may notice/experience intermittent connectivity to your VM at times.
We are diligently working to resolve this issue as soon as possible and appreciate your patience.
UPDATE: This is now resolved and the matter is considered closed. Please feel free to reach out to us by creating a support ticket if you are still experiencing any issues.
]]>UPDATE: This is now completed. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
UPDATE: It's experiencing connectivity issues again. Please standby.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
UPDATE: Due to the continued issues with this server, we are proceeding to investigate this server further with the help of on-site datacenter hands. You may notice intermittent connectivity during this time.
UPDATE: The server looks to be stable today, we will continue to keep a close eye on this server, both virtually and physically.
]]>UPDATE: This is now completed. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We have implemented some enhanced firewall and web server level rules to help prevent unwanted traffic and attacks towards this server. This is a temporary measure to ensure this server fully stabilizes. During this time, you may experience false positives in your uptime monitors (if you are using any), though rest assured, your website is actually up and loading fine.
UPDATE: We are continuing to monitor the stability of this server.
UPDATE: Services look to be stabilized and we have reduced the sensitivity of the firewall. We will continue to keep a close eye on this server. In the meantime, while sensitivity is heavily reduced, enhanced rules are still in place, so you may still experience some false positives in your uptime monitors (if you are using any).
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We are actively troubleshooting the continued unexpected issues with this host node. More updates to follow.
UPDATE: This is online for now, we are still closely monitoring/watching this node and attempting to identify a permanent resolution path. Thank you for your patience and understanding in the meantime.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We have implemented some enhanced firewall and web server level rules to help prevent unwanted traffic and attacks towards this server. This is a temporary measure to ensure this server fully stabilizes. During this time, you may experience false positives in your uptime monitors (if you are using any), though rest assured, your website is actually up and loading fine.
UPDATE: Services continue to be stable and websites are loading without any issues. Firewall sensitivity has been significantly reduced as of now. We will continue to keep a close eye on this server, though at this point we believe it is safe to conclude and consider this incident as resolved.
]]>We will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: We are still monitoring this server closely in order to ensure stability. Services should be operating normally at this time.
UPDATE: We have implemented some enhanced firewall and web server level rules to help prevent unwanted traffic and attacks towards this server. This is a temporary measure to ensure this server fully stabilizes. During this time, you may experience false positives in your uptime monitors (if you are using any), though rest assured, your website is actually up and loading fine.
UPDATE: Services continue to be stable and websites are loading without any issues. Firewall sensitivity has been significantly reduced as of now to ensure most real world use cases are not negatively impacted. For now, only some external uptime monitoring services (for example Pingdom, UptimeRobot, etc) may be blocked due to excessive connections from ping probe IPs, which may create false positives in monitoring. That will be fixed too in the coming day or two once we further reduce the firewall sensitivity some more.
]]>Despite the server remaining operational throughout the incident, there was indeed intermittent connectivity issues experienced earlier today for database-interactive websites hosted on DE2000. The primary effect of the excessive temporary files was on websites that heavily rely on database interactions. The built-up temporary files saturated the /tmp directory, thereby causing erratic website performance, marked by periods of intermittent availability.
We have successfully implemented a range of proactive measures and safeguards designed to prevent such situations from arising in the future. These strategies are not solely specific to the DE2000 server, but have been implemented across our entire shared hosting platform to avoid such a situation from occurring again in the future. Our Engineering team is also exploring additional improvements to our alert systems to detect and address such issues before they become a problem.
In conclusion, the recent incident earlier today involving the DE2000 server was caused by an accumulation of temporary files in the /tmp directory. While the server itself did not experience any downtime, it caused intermittent connectivity issues for websites that frequently interact with databases. However, we have taken corrective and preventive actions to ensure such scenarios do not recur across our shared hosting platform.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Our datacenter facility team is currently looking into this further. We will have more information on this shortly.
UPDATE: This looks to be a power related event. More details to come.
UPDATE: As of now, power for approximately half of the devices facility wide is restored. The datacenter facility is currently investigating to discover what exactly went wrong. Utility power went down and did not fail over to the generators as was expected for a seamless transition of power. We will update when we have more information as to the cause for the outage and once power has been fully restored.
UPDATE: Progress is being made. Power is restored to approximately 80% of affected devices. We are diligently working on the remaining, and will update once more information is available.
UPDATE: Power is restored to approximately 95% of affected devices. We are diligently working on the remaining 5%, and will update once more information is available.
UPDATE: Excellent progress is being made. We're almost there... just a few more physical devices that need to be worked on. We hope to have this incident fully closed out & resolved within the next hour.
UPDATE: The power incident is now fully resolved facility-wide. A RFO from our facility partner will be available in the next few business days. If you are still experiencing any issues with your services hosted in New York, please feel free to open a support ticket and we will assist you further.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: We will need to proceed with having the on-site datacenter technicians further investigate this server for potential hardware issues. You may notice intermittent connectivity during this time.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>While we don't foresee any actual service interruption at this time, the risk of temporary power fluctuations cannot be completely ruled out. We will provide any further updates to this status incident as they become available.
UPDATE: Great news! As of 8:30PM EST, the New York datacenter facility was successfully transferred off of generator power and back to the primary utility power. After close monitoring and several secondary checks, we can safely say that the power incident has concluded. We saw no customer impacts and everything was functional as expected.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: The RAID Controller on this server appears to be having issues. We are currently troubleshooting further with the on-site datacenter technicians.
UPDATE: We have replaced the RAID controller on this server, however, it is still showing signs of a degraded array. Please standby while we continue to investigate this issue with the on-site technicians.
UPDATE: The RAID rebuild was ultimately not successful and did not move past a certain percentage even after an extended period of time. This could be due to incomplete or unavailable RAID metadata. We are going to go with plan B, which is to migrate the VM's to another hypervisor/host node. We will begin working on VM migrations today.
You will know the migration is done by logging into the SolusVM VPS Control Panel, which will tell you what node your VPS is on. Once it says a node name that is not LAXSSD5004nerd3DC02, it will mean that your VPS is deployed on the new host node already.
If any action is required from your end, you will receive an e-mail notification from us. Please keep an eye out for your e-mail for any communication from us, if any.
UPDATE: We will be closing this incident out because we are working on Plan B, as described in the above update. If anyone has any questions - please feel free to reach out to us.
We will reach out to you via e-mail if any action is required from your end.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is fully resolved.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE 1: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
UPDATE 2: It's down again. The upstream provider is reseating the raid controller to fix the issue. It will be resolved soon. Please follow the status page - https://status.racknerd.com/
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: Thank you for your patience. This is resolved and VM's are responsive at the moment, we are continuing to monitor this server closely in the background to ensure of its stability.
UPDATE: We are continuing to monitor this server. We are moving the status incident to "Watching".
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues, please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
UPDATE: It went offline for a few minutes, not more than 5 minutes.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We are still working on this.
UPDATE: We are still troubleshooting the root cause of the instability issues with this host node. We are working closely with the datacenter engineers on the ground and will provide additional updates once available.
UPDATE: We are still running through some troubleshooting procedures. During this time, you may notice intermittent connectivity to your VM hosted on ASH100KVM.
UPDATE: We are continuing to monitor this server closely. Services are online and accessible at this time, but you may notice intermittent connectivity.
UPDATE: Services have been stable for the past 12 hours and counting, we are continuing to monitor this server closely and will move this status incident to "Watching" status for now.
UPDATE: We have sent an email to all ASH100KVM customers with regards to a scheduled maintenance window for replacing various hardware components within this physical server in order to ensure the long-term stability of this server. That will occur at 2 PM Eastern on April 13th, 2023. In the meantime, we are continuing to monitor this server closely. Thank you.
UPDATE: We are still working on this maintenance window.
UPDATE: While VPS's hosted on ASH100KVM are currently online, a permanent resolution path is being put in place to ensure long-term stability, and customers on ASH100KVM will be sent an email with more detailed information within the next 24 hours. If you have a VPS service hosted on this host node, please keep an eye out from an e-mail from us, and you will be receiving it in your inbox within the next 24 hrs or less. The email will originate from support@racknerd.com (be sure to keep an eye on your junk folder too). Because of the permanent resolution path we are putting in place, we will be marking this status incident as resolved. Once again, just keep an eye out for an email from us, within the next 24 hours or less.
By the way, the email we will send to you will also include compensation details (we will be compensating all affected services).
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: Server connectivity has been restored, though we are currently troubleshooting a possible hardware issue with this server. Please standby for further updates.
UPDATE: A motherboard replacement has been scheduled to occur today. Existing customers were sent an e-mail with more information. We will update this status incident once this is completed.
UPDATE: Maintenance is ongoing now.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
]]>UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved by rebooting the server quickly. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to those nodes only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: There appears to be an underlying hardware issue with this host node. Our Engineers are actively working with the datacenter remote hands to troubleshoot this server. Thank you for your patience. We will share additional updates once available.
UPDATE: This is still being worked on.
UPDATE: SJ129KVM customers were sent a follow-up email with additional information. We will now be closing out this status incident.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We are still looking into this matter as it looks to be a RAID controller related problem, thank you for your patience. We will share additional updates once available.
UPDATE: On-site datacenter engineers are still working on this server. We are closely coordinating with them to ensure a prompt resolution and conclusion of this matter. Thank you again for your continued patience.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Update: We are still investigating the issues related to the hardware part; please standby for further updates.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We've replaced a faulty network cable and network connectivity to this server is now healthy and restored. Please feel free to open a ticket if you have any questions.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: ASH100KVM services are back online (please allow up to 15 minutes for all VMs to be started again). We are continuing to closely monitor the health and stability of this server.
]]>UPDATE: It's not fully resolved, please standby for further updates.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Our upstream provider is actively investigating this issue and we will update this status incident with more information once available.
UPDATE: This was a power related event, which was resolved. If you are still experiencing any issues accessing any of your services hosted in this datacenter, please open a support ticket and we would be happy to further troubleshoot and assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This has now been completed.
]]>UPDATE: Scheduled maintenance on NJ111KVM is now completed.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: Scheduled maintenance on ASH103KVM is completed.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this status incident once the maintenance is complete.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This scheduled maintenance has now been completed.
]]>UPDATE: Scheduled maintenance on ATLRYZEN100 is completed.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch which has been resolved by itself. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: The server is back online and should be operational at this time, we will be closely monitoring this server to ensure that it is completely resolved. We will update this status incident again if we need to take any additional action with this server.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We will soon be scheduling a motherboard and PSU replacement of this server as a permanent resolution path for this node. Customers on this node will be sent an email with more information on the scheduled maintenance timeframe which will be occurring next week. In the meantime, our engineers have deployed a fix that should keep this server online between now and then. We will go ahead and close out this status incident at this time as the server is back online and available for use.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: On-site datacenter technicians are still investigating this issue. We will update this status incident once more information is available. Thank you for your patience.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>The upstream is currently looking into this, and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Our systems administrators are actively looking into the issue, and we will update this status incident once this matter has been resolved. Thank you for your patience.
Note: You may experience extremely brief/intermittent outages during this time.
UPDATE: We have a grasp of the situation for the most part, and most websites hosted on MIA1000 with MySQL connectivity is restored. A small number of websites are still experiencing intermittent MySQL related issues, we are working on investigating those one by one. We appreciate your patience while we work through this, and will update this status incident with more information once available. As mentioned earlier, websites as a whole may experience brief MySQL database connection interruptions while we work on this matter.
UPDATE: For the accounts that are still having MySQL issues, we have determined that the best course of action is to restore databases from the latest available backup. We are proceeding with that now.
UPDATE: Progress is being made, and more accounts are being restored to normal operation. If your website database is still not functioning properly you can safely assume at this time that we are still working on it, and we thank you for your patience.
UPDATE / RESOLUTION: We are pleased to bring this incident to a close. All databases have been restored, and data is fully intact. A majority of websites hosted on this server should already be back to normal operation without any manual intervention required. If your website is still not functioning properly, then it is possible that there is a database-dbuser permission issue specific to your environment, and in this case to resolve it, you will simply need to manually re-create the database user and associate it to the database with all permissions granted. Below are the steps to do so:
Step #1: Log into your cPanel account
Step #2: Go to cPanel > MySQL Databases
Step #3: Delete the database username (the username, not the database itself). Once you are on that page, scroll all the way down and under "Current Users", click Delete on the respective database username. Do not delete the database — only the database user. As you are only removing the database user, you will not incur any data loss as a result of this process.
Step #4: Create a new user under the "Add New User" section, and note down the MySQL username and password that you used (you will need this for the last step)
Step #5: Under "Add User to Database" section in the cPanel MySQL databases page, associate your database with the new database user that you created and grant it all permissions. If you have multiple databases in your cPanel account, pay attention to ensure you are associating the correct database to the correct user.
Step #6: Lastly, update your website's configuration file (for example - if you are using Wordpress, it'd be within wp-config.php) to reflect the new DB_USER and DB_PASS. Do not modify the DB_NAME section, only the DB_USER and DB_PASS to reflect the details you created in Step #4. Once modified, save changes to the configuration file. Verify your website is loading fine by visiting it on your web browser.
If you require any assistance, or would like for us to perform this for you, please feel free to reach out. Our support team is available 24x7 to help.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Estimated downtime is less than one hour. We will update this status incident once resolved.
UPDATE: The maintenance window is now completed and we are continuing to closely monitor the health of this server. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: On-site datacenter technicians are now physically troubleshooting this server. We will share additional updates in this status incident page once available. Thank you for your patience.
UPDATE: VM's on DAL124KVM are back online now, however we are still working on restoring connectivity for the SolusVM controls for the host node. We're looking into that now. In the meantime you should be able to access your VPS via SSH.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We apologize for the repeat issues with this node and are committed to getting to the bottom of this issue.
We are currently investigating into this node further with datacenter remote hands, and will provide additional updates on this matter once we have more information.
UPDATE: Due to the continued unexpected shutdowns we are experiencing with this server, we are having datacenter remote hands proceed with replacing the motherboard on this server. We will update once additional information is available.
UPDATE: We received an update from remote hands that they will be beginning work on this motherboard replacement shortly. Thank you for your patience.
UPDATE: This motherboard replacement is now completed and services are back to normal operation. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: The server is powered on but it is not outputting any video using a crash cart. On-site datacenter technicians are currently troubleshooting further.
UPDATE: We are still in the process of troubleshooting this and understand the severity of the situation. Thank you for your patience and understanding.
UPDATE: We are making progress on this matter and should have a resolution shortly.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>NYRN131KVM - This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
NYRN13K6KVM - This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This server is unresponsive to out of band access, we submitted a task for datacenter remote hands to look further into this issue. We will update this status incident with more information once available. Thank you for your patience.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We are still investigating the hardware issues on DAL124KVM. Thank you for your patience and we will share updates once available.
UPDATE: Hardware tests are ongoing. We hope to have this resolved by later tonight. Thank you for your continued patience and we will share additional updates in this incident once available.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this incident further with our upstream provider, and will provide additional updates on this matter once we have more information.
UPDATE: Our upstream provider’s networking team is actively aware and working on this issue. It appears to be an issue related to border routers. We will update this status incident again once additional progress has been made.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
UPDATE 1 : It's down again, We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE 2 : This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Estimated downtime is less than one hour. We will update this status incident once resolved.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Existing customers on CHI135KVM were sent an email notification with details. We will update this status incident as resolved once the RAM replacement is completed.
UPDATE: RAM replacement on CHI135KVM has been completed. Please feel free to open a ticket if you have any questions.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Estimated downtime is less than one hour. We will update this status incident once resolved.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are working to mitigate this, and will provide additional updates on this matter once we have more information. UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are working to mitigate this, and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this status incident once the maintenance is completed.
UPDATE: This maintenance is now completed. We will be closing out this incident now.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This node requires further physical investigation by an on-site datacenter technician. This could take a while before a full resolution is concluded, due to the weather-related State of Emergency issued in the Buffalo, NY area. We sincerely appreciate your patience during this time.
UPDATE: As a faster resolution path, we are currently working towards migrating VM's to another healthy host node. Your data and VPS IP address will remain intact, though you may notice up to 30 minutes of downtime while your VM migrates. You will know the migration is done by logging into the SolusVM VPS Control Panel, which will tell you what node your VPS is on. Once it says a node name that is not NYRN135KVM, it will mean that your VPS has been successfully migrated and booted up to the new host node already.
If any action is required from your end, you will receive an e-mail notification from us.
We will be closing this incident out but we are working on migrations, if anyone has any questions regarding the migration please feel free to reach out to us.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network related problem on the server end. The server is back online now. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this and will share additional updates here ASAP.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will share additional updates on this status incident once available.
UPDATE: While services have been restored, we are continuing to closely monitor this server. We will keep this status incident open until we have full confidence the matter is resolved. Thank you for your patience.
UPDATE: This matter has been fully resolved, and websites hosted on this server should be back to a fully operational state now. Thank you for your patience while we worked to mitigate this attack.
]]>We are currently looking into this node further and will share additional updates as soon as possible.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to those nodes. all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this status incident once resolved.
UPDATE: It appears that this requires further troubleshooting, thank you for your patience while we continue to look into this.
UPDATE: Our efforts to allow the RAID controller to rebuild in a live environment were unsuccessful. It is most safe to rebuild this RAID array offline. We are taking the server down to our work bench now, and will allow time for the RAID array to rebuild. This process could take several hours. We apologize for the unexpected delay and will share updates once available.
UPDATE: The RAID rebuild is currently at 19%.
UPDATE: The server is back online on a degraded RAID array, and our attempts to rebuild the RAID array in a timely fashion was not successful due to corrupt RAID metadata. However, since we utilize RAID-10 we were still able to get the server online. To minimize further impact and downtime, we believe the best course of action is to bring this server back online and then migrate VM's on this node one by one over to another host node. This will be a seamless migration and you will not experience any IP changes, and all data will be migrated in place. We will be working on these migrations starting shortly and we anticipate all migrations being completed within the next 24-48 hours.
UPDATE: Albeit a degraded RAID status, we are currently working towards migrating VM's to another healthy host node. Your data and VPS IP address will remain intact, though you may notice up to 30 minutes of downtime while your VM migrates. You will know the migration is done by logging into the SolusVM VPS Control Panel, which will tell you what node your VPS is on. Once it says a node name that is not NYRN135KVM, it will mean that your VPS has been successfully migrated and booted up to the new host node already.
If any action is required from your end, you will receive an e-mail notification from us.
We will be closing this incident out but we are working on migrations, if anyone has any questions regarding the migration please feel free to reach out to us.
]]>UPDATE: We are still looking into this. Thank you for your patience.
UPDATE #2: Our efforts to resolve this remotely have been unsuccessful. We have requested on-site datacenter technicians (remote hands) to replace the network cable connections to this server. We will update shortly.
UPDATE #3: This was resolved after replacing network cable connections and re-configuring network configuration files. Please feel free to contact us if you have any questions.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will post more information once available.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We identified the issue being an OSPF communication issue between our routers and core switches, and applied a fix immediately. This has now been resolved, and services are fully operational again. Feel free to contact us if you have any questions.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Maintenance Start Date: November 26, 2022
Maintenance Start Time: 11:00 AM PST
Maintenance End Time: 11:30 AM PST
Affected Services: RackNerd Website, RackNerd Client Portal, RackNerd VPS Control Panel (NerdVM/SolusVM)
Scope of Work: We will be implementing backend related improvements/updates to the above services. The frontend of our website and SolusVM panel may be unavailable for up to 30 minutes during this time.
Client services will not be affected, and will remain online.
If you require assistance during this maintenance timeframe, please e-mail us at support@racknerd.com, call us at 888-881-NERD, or submit a support ticket in our client area after the maintenance window is completed.
We will be moving this status incident to active once the maintenance window commences.
UPDATE: This maintenance is about to commence.
UPDATE #2: The SolusVM maintenance is now completed and accessible.
We are now working on the maintenance of the RackNerd website and client portal.
UPDATE #3: The backend improvements for our website and client portal are in progress of being rolled out - during this time, the frontend will be inaccessible. The maintenance will run on a bit longer than originally planned, due to the fact we started on this a bit later in efforts to get SolusVM done first. ETA is now 12:00 PM PST.
UPDATE #4: This maintenance is now completed. We will be closing out this incident now.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Maintenance Start Date: November 23, 2022
Maintenance Start Time: 11:00 AM PST
Maintenance End Time: 11:30 AM PST
Scope of Work: We will be upgrading our underlying infrastructure that empowers our website (racknerd.com) and client portal (my.racknerd.com) due to the surge of traffic and demand. Our website and client portal features may be unavailable for up to 30 minutes during this time.
Client services will not be affected, and will remain online.
If you require assistance during this maintenance timeframe, please e-mail us at support@racknerd.com, call us at 888-881-NERD, or submit a support ticket in our client area after the maintenance window is completed.
We will be moving this status incident to active once the maintenance window commences.
UPDATE: This maintenance is about to commence. We will share another update once this is completed.
UPDATE #2: This maintenance is now completed, and our website and client portal is now available again. We will be closing out this status incident as resolved now. Feel free to open a support ticket if you have any questions.
]]>UPDATE: This is now resolved. It was a switch-related problem specific to the rack hosted the nodes.all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further with our on-site engineers, and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we are happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
UPDATE #2: It appears that this node is still experiencing symptoms - this time related to failed RAM. We are currently investigating this matter with on-site datacenter engineers and will replace the failed DIMM accordingly. In the meantime, you may find your service hosted on this node to be intermittently available.
UPDATE #3 : This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this status page as resolved once this maintenance is completed. As mentioned in the maintenance notification email, we expect for actual downtime to be under 45-60 minutes.
UPDATE: This maintenance window is now completed. We will now be closing out this status incident as resolved. If you have any questions or if your VM is still inaccessible, please open a support ticket and we'd be happy to assist you.
]]>UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This has now been resolved. If you are still experiencing any issues or if you have any questions, feel free to contact us and we'd be happy to assist you.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
UPDATE : It's experiencing connectivity issues again, we are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: The issue was resolved, we will continue to monitor this closely and if you have any questions or concerns, feel free to reach out to us. At this time the network is fully operational.
]]>Network engineers are currently investigating this issue and we will provide more information as soon as possible.
UPDATE: The routing issue was resolved, we will continue to monitor this closely and if you have any questions or concerns, feel free to reach out to us. At this time the network is fully operational.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Client services are not affected and will remain online.
UPDATE: This has now been completed. Access to the NerdVM panel is now restored and available.
]]>UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this status incident once the maintenance is completed.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This has now been resolved.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VMs are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: On-site technicians are still running diagnostics on LAXSSD4009nerd6DC02. Thank you for your patience.
UPDATE: After diagnostics we believe this issue may have been as a result of a failed Hardware RAID controller itself. We have replaced the Hardware RAID controller on this server and we are currently performing an offline RAID array rebuild.
The RAID rebuild is currently at 6%.
UPDATE: The RAID rebuild is currently at 11%.
UPDATE: Great news, the RAID rebuild completed successfully without any errors, and VM's on LAXSSD4009nerd6DC02 are now responsive again. If you are still experiencing any issues, feel free to contact us.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This maintenance event is almost completed, and went smoothly without any issues for the most part. We need to power cycle the server one more time in order to double check a BIOS setting, after that, this maintenance will be completed. Thank you for your patience and we will have this wrapped up shortly!
UPDATE: The maintenance event is now completed successfully. Please feel free to contact us if you need any assistance.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this status incident once the maintenance is completed.
UPDATE: This maintenance is now concluded/finished, we will be closing out this status incident at this time. If you have any questions or if your VM hosted on this host node is still inaccessible, please feel free to open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into these nodes further and will provide additional updates on this matter once we have more information.
UPDATE: The RAID array is currently degraded on this server. We are currently working to rebuild the RAID array to an optimal state. This could take several hours, but we will update this status incident once additional information is available. Thank you for your patience.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further.
UPDATE: This is now resolved, and all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This scheduled maintenance is about to commence. We will update this status incident as resolved once the maintenance is completed.
UPDATE: This maintenance is now complete and VMs are now back online. If you have any questions please feel free to open a support ticket and we'd be happy to help.
COPY OF SCHEDULED MAINTENANCE E-MAIL BELOW:
START/END DATE: Sept/17/22-Sept/17/22 START TIME: 12:00 PM Eastern Time ESTIMATED END TIME: 1:00 PM Eastern Time
Hello,
Engineers will be performing an emergency maintenance on a power circuit at the New Jersey datacenter.
SERVICE: The circuit needs preventative work performed to avoid a potential failure scenario in which some devices could lose power unexpectedly. This resulted to a couple unexpected shutdowns we have experienced with this node (NJ101KVM) in the past week, and as such we are scheduling an emergency maintenance to address this ASAP.
IMPACT OF WORK: During this time frame, the node listed below will be powered off and then powered back on as the circuit maintenance is completed. We expect the total downtime of each node to be approximately 45-60 minutes.
IMPACTED NODE: NJ101KVM
If you have any questions or concerns, please create a support ticket from within your client area and we'd be happy to assist.
Regards,
RackNerd Support
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network related problem specific to the shared server. The server is back online and running fine now. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: The server needed to run a FSCK check, this has been completed and the server is now back online.
]]>UPDATE: This scheduled maintenance is about to commence. We will update this status incident as resolved once the maintenance is completed.
UPDATE: This maintenance is now complete and VM's are now back online. If you have any questions please feel free to open a support ticket and we'd be happy to help.
COPY OF SCHEDULED MAINTENANCE E-MAIL BELOW:
START/END DATE: Sept/13/22-Sept/13/22 START TIME: 4:00 PM Central Time ESTIMATED END TIME: 6:00 PM Central Time
Hello,
As a result of the recent unexpected shutdowns experienced on the host node CHIRYZEN103, on-site datacenter engineers will be performing maintenance event on this host node.
SERVICE: During the maintenance window, CHIRYZEN103 will be un-racked from the datacenter floor, moved to the datacenter work bench, and diagnostic testing will be run to ensure the integrity of hardware components on the server. If any failed pieces of hardware components are identified during the diagnostics testing, we will replace the failed piece of hardware component immediately and re-rack the server back on the datacenter floor. We will also be re-applying CPU thermal paste during this maintenance event.
IMPACT OF WORK: We expect the total downtime to be approximately 1 hour, though it could run on slighlty longer if the diagnostics take a bit longer to run. We will do everything we can to keep downtime to a minimum.
IMPACTED NODE:
CHIRYZEN103 If you have any questions or concerns, please create a support ticket from within your client area and we'd be happy to assist.
Regards,
RackNerd Support
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: Maintenance will be beginning at 11 AM Pacific. We will update this status incident once the maintenance is completed.
UPDATE: This maintenance window is now completed. We will now be closing out this status incident as resolved. If you have any questions or if your VM is still inaccessible, please open a support ticket and we'd be happy to assist you.
COPY OF SCHEDULED MAINTENANCE E-MAIL BELOW:
START/END DATE: Sept/13/22-Sept/13/22 START TIME: 11:00 AM Pacific Time ESTIMATED END TIME: 12:00 PM Pacific Time
Hello,
Engineers will be performing maintenance on a power circuit at the San Jose datacenter.
SERVICE: The circuit needs preventative work performed to avoid a potential failure scenario in which some devices could lose power unexpectedly.
IMPACT OF WORK: During this time frame, the devices listed below will be powered off and then powered back on as the circuit maintenance is completed. We expect the total downtime of each node to be approximately 45-60 minutes.
IMPACTED NODES:
SJ118KVM SJ120KVM SJ122KVM SJ123KVM If you have any questions or concerns, please create a support ticket from within your client area and we'd be happy to assist.
Regards,
RackNerd Support
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further.
UPDATE: This is now resolved and appropriate measures will be taken to minimize this from occurring again in the future. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This particular host node appears to be experiencing some physical hardware related issues, at this time we are submitting a request with the datacenter technical team to further investigate.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into these nodes further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved as rebooted the node. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are looking into this now.
UPDATE: This was a slight network blip at the upstream level - this was resolved in under a couple minutes and services are now online.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This host node appears to be exhibiting symptoms of unstable uptime, which we believe could be related to a physical hardware fault such as a bad PSU. Please standby while we work with remote hands in Atlanta to physically troubleshoot this machine. Your patience in the meantime is greatly appreciated.
UPDATE: On-site datacenter technicians are still actively working on this issue.
UPDATE: This was resolved after transplanting the drives over to another physical server. You should have noticed your VM come back online and remain in a stable state. If you are still experiencing any issues please feel free to open a support ticket and we'd be happy to help.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This maintenance event is now completed. Please feel free to contact us if you have any questions.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
UPDATE : it's down again as our team is working with the upstream provider.
UPDATE : This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE 1. The network team has been informed, and they are checking it.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Update 1 - The network team has been informed, and they are checking it.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
UPDATE 2: There's a packet loss again, we are investigating this.
UPDATE 3: Our upstream's network engineering team is monitoring DDoS and doing their best to mitigate it. If you are seeing any issues with your instance or service, feel free to reach out via ticket and we'll be happy to help.
UPDATE 4: Our upstream's network engineering team is still monitoring DDoS and doing their best to mitigate it. If you are seeing any issues with your instance or service, feel free to reach out via ticket and we'll be happy to help.
UPDATE 5: This is now resolved and there is no more packet loss. Please feel free to reach out if you are still experiencing any issues, or if you have any questions.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: It appears that the RAID controller is having issues, at this time we are having remote hands swap out the RAID card with another on that we have in stock. We will provide additional updates once available.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch from the data center side which has been resolved by their team just now. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further.
UPDATE: This is now resolved, and all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently working on having an on-site technician replace the failed SSD with a new SSD. We will update this status incident once the degraded drive has been replaced, at which point, the RAID-10 array can rebuild accordingly.
UPDATE: We had replaced the failed SSD however are still showing symptoms of a degraded and unstable RAID array. Please standby while we continue to investigate this issue with the on-site technicians.
UPDATE: We are currently having the RAID array on CHI119KVM rebuild offline, in the interest of data integrity. We will continue to closely monitor the RAID rebuild status.
UPDATE: The RAID array rebuild has failed. We are currently having on-site technicians troubleshoot further and have provided them with clear instructions on how to move forward. Thank you for your patience and please be assured we are doing everything in our power to get this node back to normal operation as soon as possible.
UPDATE: On-site datacenter technicians are working on a RAID card replacement at the moment. We will update once we know more.
UPDATE: The RAID rebuild process is currently ongoing with the new RAID card. It is currently at 21%. We will continue to monitor the status of the RAID rebuild and as soon as it finishes, we will boot the server back online to production.
UPDATE: The RAID array rebuild is currently at 24%. We will continue to monitor the status of this rebuild and provide additional updates once available.
UPDATE: The RAID array rebuild is currently at 27%. We will continue to monitor the status of this rebuild and provide additional updates once available.
UPDATE: The RAID array rebuild is currently at 32%. We will continue to monitor the status of this rebuild and provide additional updates once available.
UPDATE: The RAID array rebuild is currently at 37%. We will continue to monitor the status of this rebuild and provide additional updates once available.
UPDATE: The RAID array rebuild is currently at 40%. We will continue to monitor the status of this rebuild and provide additional updates once available.
UPDATE: The RAID array rebuild is currently at 45%. We will continue to monitor the status of this rebuild and provide additional updates once available.
UPDATE: The RAID rebuild was ultimately not successful and did not move past 45% even after an extended period of time. This could be due to incomplete or unavailable RAID metadata. We are going to go with plan B, which is to migrate the VM's to another hypervisor/host node. We will begin working on VM migrations today.
You will know the migration is done by logging into the SolusVM VPS Control Panel, which will tell you what node your VPS is on. Once it says a node name that is not CHI119KVM, it will mean that your VPS is deployed on the new host node already.
If any action is required from your end, you will receive an e-mail notification from us. Please keep an eye out for your e-mail for any communication from us, if any.
UPDATE: We will be closing this incident out because we are working on Plan B, as described in the above update. If anyone has any questions - please feel free to reach out to us. We will reach out to you via e-mail if any action is required from your end.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: There was a switch related issue which has now been corrected. Services in this location are now fully restored. Please feel free to open a support ticket if you are still experiencing issues, or if you have any questions.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now fully resolved by the upstream. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
UPDATE: It appears this node is experiencing symptoms again despite our recent scheduled maintenance, which should have corrected this. At this time we are having on-site technicians further troubleshoot this server. We will update you with our findings as soon as possible.
UPDATE: This is now resolved - we have transplanted the drives on this host over to another physical machine. All VMs are now back online. We will continue to monitor this host node to ensure of its stability. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this status incident once the scheduled maintenance window has been concluded.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We are still working with datacenter remote hands to identify the issue. Thank you for your patience during this time.
UPDATE: We are currently in the process of replacing the PSU and motherboard on this machine. We will update once we hear more from the hands on the ground at the ATL datacenter.
UPDATE: Remote hands in Atlanta is still working on this. We will update once they provide us with additional information.
UPDATE: We have been actively working on this node with remote hands in ATL, but have been unsuccessful so far. We had originally expected the PSU replacement to rectify the issue at hand earlier this morning (which it did not, the server's BMC is up but server stays shut down). Because that was unsuccessful, we then decided the next course of action would be to replace the motherboard as that was displaying symptoms of a bad motherboard.
We had remote hands perform an emergency motherboard swap from inventory, but came to find out the heatsinks we utilize for this CPU were not compatible with the replacement motherboard that the datacenter had in inventory. At this time, we are preparing a shipment to overnight a new heatsink to ATL. In the meantime, we are currently working with on-site technicians to explore other options for bringing this server back online.
We sincerely apologize for this unexpected outage and greatly appreciate your patience regarding this node.
UPDATE: While we are still working on exploring other possibilities with our datacenter partner in ATL, we have a kit of preassembled RAM/CPUs/heatsink/motherboard that has been verified working and will be overnighted to arrive tomorrow morning. If we are unable to identify a solution before this, we will have remote hands then swap the entire kit into the server to eliminate any other potential faults.
UPDATE: Despite a replacement of major components (including full motherboard replacement), this machine is still experiencing issues (this time the machine is at least powering on, but returning post code errors). The symptoms include several post issues, so our team is continuing their efforts to diagnose and restore service.
We fully understand the impact and downtime is something that we all wish to avoid. Rest assured, our team is working closely with facility hands to resolve and troubleshoot this issue. We will share additional updates once available on this status incident.
UPDATE: This incident has been resolved, the on-site technicians have corrected the post code errors and the machine is now online with the replacement kit. All VMs on this node are now responding again, and this host node is now considered to be fully operational. If you are still experiencing issues in connecting to your VM on this node, please feel free to open a support ticket and we'd be happy to assist. We will be working closely with our datacenter partner in Atlanta on a preventative follow-up plan to avoid such incidents from occurring again in the future.
]]>During this time, you may notice moments of unavailable connectivity. Thank you for your patience during this time as we work towards implementing a permanent resolution towards this node.
UPDATE: This server is now back online, on-site technicians have reseated several components. If any further issues persist, we will proceed with scheduling a motherboard replacement.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We are still working on this, you may notice intermittent connectivity in the meantime.
UPDATE: We've narrowed down the issue to a RAID Controller issue, we are currently working to replace the RAID controller on this system.
UPDATE: We have replaced the RAID controller on this system and will continue to monitor the stability of this machine. All VM's should be online and responsive again. Please feel free to open a support ticket if you are still experiencing any issues.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: Because of the recent numerous outages on this particular host node, we believe there may be a PSU related issue (perhaps, PSU that is going bad) on this physical host node which is causing an unstable power connection to this node. On-site datacenter technicians are checking further and we will replace the PSU if necessary.
UPDATE: On-site datacenter technicians are aware and actively working on this node.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this two nodes further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a PDP port problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this outage further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this outage further and will provide additional updates on this matter once we have more information.
UPDATE: RackNerd customers should begin seeing network connectivity now. We sincerely apologize for the hiccup and are actively working with our upstream provider to prevent this from occurring again in the future. If you have any questions or if you are still experiencing issues, feel free to open a support ticket (or email support@racknerd.com) and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VMs are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are working towards a resolution and further updates will follow.
During this time you may notice our corporate website to be unavailable.
UPDATE: Connectivity to our corporate website has been restored successfully.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Our network engineering team is currently looking into this further. We will have more information on this shortly.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: The slight blip was corrected and resolved. Please let us know if you are experiencing any issues in accessing your sites.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE : This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>During this time, while we analyze and mitigate the latest attack, you may experience intermittent connectivity to your sites hosted on this server. Feel free to contact us with any questions.
UPDATE: Progress is being made on our side, however, the attack patterns are changing. Network engineers are on standby and working to adapt our mitigation solutions accordingly whenever new patterns are targeted towards this server.
UPDATE: Please note that if you are using a third party monitoring service, you may receive false positive alerts. We recommend pausing any monitoring services for the meantime (at least until the attack is fully mitigated). This is because we have enabled elevated mitigation efforts including disabling ICMP ping and connection throttling which could trigger false positives to automated monitoring probes.
We're still working on this and will share additional updates once available. In the meantime, you'll experience intermittent connectivity.
UPDATE: Websites should be loading now. We will continue to monitor this before closing out this incident as resolved.
UPDATE: We have finally managed to resolve all the outages that happened on the FR1000 server due to sophisticated DDOS attacks. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This issue is still being worked on. This appears to be a sophisticated attack targeted towards this server only. The issue has been escalated to our upstream provider for further analysis and mitigation.
Rest assured, we're working to get this fully resolved in the quickest manner possible.
UPDATE: We are still working towards a full mitigation of this attack as it appears to have resurfaced. We will update this status incident with more information once available.
UPDATE: This is now resolved, however, we are still actively monitoring the status of this server in order to ensure of its stability. We will move this status incident to "Watching" status for now, however, your websites should be online and loading quickly now. After the server has been confirmed to be stable for some time, we will graduate this status incident to "fixed" and mark it as fully resolved. Thank you for your understanding and patience!
UPDATE: Network engineers are actively following up on this. You may notice some intermittent connectivity while they work towards applying a permanent fix.
UPDATE: Network engineers have fixed the network mitigations. We are currently monitoring the network stats on the server.
UPDATE : We have completely resolved the attack which hit towards our server network with the help of Network Engineers and the server is running fine without any interruptions or outages. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
UPDATE: It looks like another type of attack has now surfaced. Network engineers are now working on mitigating this pattern.
UPDATE: Good news! We were able to resolve the latest attack, and the server has been stable for the past 2 hours and counting. Since the server has been in a stable state for some time, we are graduating this status incident to a "Watching" status as we continue to closely monitor the status of this server. However, for the meantime, your websites should be online and loading. If you have any questions in the meantime, feel free to open a support ticket to let us know. After additional time has passed, we will be closing out this status incident and marking it as fully resolved.
UPDATE: We will be marking this incident as fully resolved. We are always here to help, feel free to open a support ticket if you have any questions, or if you are still experiencing any issues.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: On-site technicians will be taking this server offline to replace some molex cables. We will update this status incident once it is resolved.
UPDATE: This is now resolved and VM's are now accessible and stable. We will continue monitoring this node to ensure its stability. If you have any questions in the meantime, feel free to contact us at any time.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was due to network glitches specific to these host nodes only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: Because of the continued issues this server is having despite our previous PSU replacement, we are going to replace this server entirely with a new one and move the drives over to the new server.
Please wait while we complete this.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
UPDATE - It's down once again. We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This appears to be a hardware related fault, potentially with the motherboard or with the PSU. At this time we are going to have on-site technicians un-rack this server and perform a physical inspection and test. We will then replace the faulty hardware component accordingly so that this server will be stable again. Thank you for your patience while our on-site technicians complete this task! We will update this status incident with further updates once available.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently investigating this issue further with on-site technicians, and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: Because of the continued issues this particular host node is facing, our on-site datacenter technicians are currently working on building out a replacement server for it. Once the replacement server is ready, we will be moving the drives over. Thank you for your patience and rest assured we will have a full resolution on this soon!
UPDATE: This appears to be a RAID related issue. We replaced the RAID controller and replaced it with a new server/motherboard, however, the RAID array is currently not optimal. We are working towards rebuilding the RAID array. Given the sensitive nature, we are currently performing an offline RAID array rebuild. The RAID rebuild is currently at 1%, and we will continue to provide updates in this status page incident.
UPDATE: The server is back online on a degraded RAID array, our attempts to rebuild the RAID array in a timely fashion was not successful due to corrupt RAID metadata. However, since we utilize RAID-10 we were still able to get the server online.
Albeit a degraded RAID status, we are currently working towards migrating VM's to another healthy host node. Your data and VPS IP address will remain intact, though you may notice up to 30 minutes of downtime while your VM migrates. You will know the migration is done by logging into the SolusVM VPS Control Panel, which will tell you what node your VPS is on. Once it says a node name that is not LAXSSD830nerd2DC02, it will mean that your VPS has been successfully migrated and booted up to the new host node already.
If any action is required from your end, you will receive an e-mail notification from us.
We will be closing this incident out but we are working on migrations, if anyone has any questions regarding the migration please feel free to reach out to us.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>You may experience intermittent connectivity during this time. We will update this status incident once we have this fully resolved.
UPDATE: This is now fully resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>On-site datacenter technicians are currently investigating this further.
UPDATE: We are still looking into this.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Services hosted within these nodes are online and accessible, but may experience degraded performance. Network engineers upstream are actively working on these nodes.
Updates:' We are still working on it.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this status incident once the scheduled maintenance window has been concluded.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this status incident once the scheduled maintenance window has been concluded.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further.
UPDATE: This was a network routing issue with one of the network carriers in the network blend (Telia) - this has now been resolved. If you’re still experiencing any issues accessing your VM on SJ123KVM feel free to open a support ticket and we’d be happy to assist.
]]>UPDATE: We are still troubleshooting this machine physically, with the help of on-site datacenter technicians. We will share another update shortly. Thank you for your patience.
UPDATE: The server is currently on the workbench, and on-site datacenter technicians are still investigating into the issue with SJ151KVM.
UPDATE: This has been resolved.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: Because there is a possible hardware related issue with this physical host node, we are going to be taking this server offline physically to the work bench to further troubleshoot and identify which hardware component needs further replacement.
We will begin working on this immediately. Thank you for your patience while we work on this.
UPDATE: We have determined a potential problematic Hardware RAID controller, and at this time, it has been replaced. This server is now back online, and VM's hosted on it are in the process of booting up. If you have any questions or if you're still experiencing any issues, please feel free to open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: The kernel update has been completed. DE1000 is back online and running stable. If you have any questions, please feel free to contact us.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into these nodes further.
UPDATE: We have identified a switch fabric failure, which only affected these 7 nodes as these servers. Replacement fabrics have been deployed in the affected router, and we are now monitoring for the router's stability.
All affected VM's should be back online now, if you are still experiencing any issues or if you have any questions, please let us know.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>During this maintenance window you may experience brief moments of intermittent network connectivity.
UPDATE: This maintenance window is now concluded.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This issue has been escalated to our upstream provider, they are actively looking into this.
UPDATE: We have found a network drop on the server network interface while checking with the help of upstream provider. It has been resolved now with the help of them.
]]>UPDATE: This issue has been escalated to our upstream provider, they are actively looking into this.
UPDATE: We have resolved the network issue on the server and the server is working fully now. We have already rectified the network issue with the help of upstream provider and it won’t happen again. Sorry for the inconvenience caused from our end.
]]>UPDATE: This issue has been escalated to our upstream provider, they are actively looking into this.
UPDATE: We have resolved the network issue on the server and the server is working fully now. We have already rectified the network issue with the help of upstream provider and it won’t happen again. Sorry for the inconvenience cause from our end.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will provide additional updates on this matter once we have more information.
UPDATE: On-site technicians are taking the server offline now to run some diagnostics.
UPDATE: Although generally a hotswap drive replacement would suffice, given the unique circumstances that have occurred specific to this host node, we are currently performing an offline RAID array rebuild after replacing the failed drives in the RAID-10 array.
The RAID rebuild is currently at 29%.
UPDATE: The RAID rebuild process is currently at 50%.
UPDATE: The rebuild process was not successful, at this time - we are attempting to replace the RAID controller with an identical model and will re-start the RAID array rebuild process.
UPDATE: The rebuild process is currently at 19%.
UPDATE: The rebuild process is currently at 42%, we are continuing to monitor the status of this rebuild.
UPDATE: The rebuild process is currently at 47%, we are continuing to monitor the status of this rebuild.
UPDATE: The rebuild process is currently at 52%, we are continuing to monitor the status of this rebuild.
UPDATE: The rebuild process is currently at 57%, we are continuing to monitor the status of this rebuild
UPDATE: The rebuild process is currently at 62%, we are continuing to monitor the status of this rebuild
UPDATE: The rebuild process is currently at 67%, we are continuing to monitor the status of this rebuild
UPDATE: The rebuild process is currently at 73%, we are continuing to monitor the status of this rebuild
UPDATE: The RAID rebuild was ultimately not successful and did not move past 73% even after an extended period of time. This is due to incomplete or unavailable RAID metadata. The good news is, data is still intact as of right now because of the fact we run a RAID-10 array for redundancy. As such, VM's are now back online and we are going to go with plan B, which is to migrate the VM's to another hypervisor/host node. We will begin working on VM migrations between today and tomorrow.
You will know the migration is done by logging into the SolusVM VPS Control Panel, which will tell you what node your VPS is on. Once it says a node name that is not LAXSSD5005nerd3DC02, it will mean that your VPS has been successfully migrated and booted up to the new host node already.
If any action is required from your end, you will receive an e-mail notification from us.
UPDATE: We will be closing this incident out, if anyone has any questions regarding the migration please feel free to reach out to us.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VMs are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VMs are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: The emergency maintenance has completed, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We're actively looking into this issue and will report back with additional details as soon as possible.
UPDATE: This is now resolved and was narrowed down to a Top of Rack switch related issue which has since been fully corrected. Please open a ticket if you require any further assistance.
]]>We're actively looking into this issue and will report back with additional details as soon as possible.
UPDATE: This is now resolved. Please open a ticket if you require any further assistance.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: The recent connectivity issues with SEA100KVM have been narrowed down to a bad top of rack switch which will need to be replaced.
On-site technicians are currently working on replacing the Top of Rack Switch. We will have this back online and resolved shortly.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Our network engineering team is currently looking into this matter further, and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved, if you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now completed. If you have any questions, please feel free to contact us.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to the Fiber10 shared server. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved, if you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Our technicians suspect this may be a bad NIC connection, on-site technicians are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved, if you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently investigating further into this matter.
UPDATE: This is now resolved, if you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: Although our network engineering team managed to resolve this for most of the nodes, some nodes are still experiencing intermittent connectivity issues. We will provide additional updates on this matter once we have more information.
UPDATE: This is now fully resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>================================
Hello,
You are receiving this email because you have one or more VPS hosted on the node: NYRYZEN103.
One of the NVMe's have gone undetected on this server and to prevent any issues in production, we are scheduling an emergency maintenance window for our on-site technicians to physically inspect this server and remedy accordingly.
Affected Node:
NYRYZEN103
Given the nature of the issue, it is in everyone's best interest that we do this as soon as possible, rather than scheduling this for a future date (to avoid the possibility of any major issues/symptoms from developing). We will be proceeding with this task at the earliest availability of the datacenter technicians. At any point within the next 1-2 hours, you may observe up to 30 minutes of downtime to your VM. We will strive to keep downtime to a minimum.
If you have any questions or concerns please create a ticket with our support team.
Thank you, RackNerd Operations
================================
UPDATE: This maintenance is now completed successfully, VM's are back online and the server is now in good health.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: There is an emergency upstream maintenance progressing on the mentioned nodes, the nodes will back online after the maintenance window is finished within 1 hour.
UPDATE: All nodes are back online with the exception of node: SJ130KVM. We are currently looking into SJ130KVM further and will have this resolved momentarily.
UPDATE: This is now fully resolved, if you are still experiencing any issues accessing your services in San Jose, please contact us.
]]>We are currently looking into this node further.
UPDATE: We are still working on resolving this and will have an additional update shortly.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This server has a faulty DIMM, we have a request in with facility hands to replace the faulty stick of RAM. We will update this incident once this matter is resolved.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: We're aware of some VM's on this host node not booting up, and are currently looking into this.
UPDATE: We have VM's back online for the time being. However, we have detected that one of the NVMe's on this server are no longer being detected by the OS, this can cause temporary I/O degradation. We are going to have facility hands take this server down shortly to rectify this. Once facility hands takes the server offline physically to work on correcting this, you may notice your VM's go offline again for a short period of time. Rest assured, we'll do everything possible to keep downtime to a minimum.
UPDATE: Facility hands (on-site team) is working on this now.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this status incident once this maintenance has been completed.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved, it was a network routing issue with the upstream provider, which has now been resolved by their network engineers. If you are still experiencing any issues or if you have any questions, please open a support ticket.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Given the recent issues that we experienced with the current hardware on ATLRYZEN101 and the fact that we are not confident/certain about the current hardware, we would like to perform an emergency maintenance to perform this system swap on Friday, January 7, 2022 at 2:00 PM EST, that way we can be proactive about the situation and put a full resolution to this matter once and for all. We will be proceeding with this emergency maintenance at the above timeframe and during this maintenance, access to your VM may be unavailable for up to 60 minutes, though, we will do everything in our power to expedite and to restore services as soon as humanly possible.
We will be moving this incident status from closed to open once the maintenance window begins. Please feel free to open a support ticket in your client area if you have any questions or concerns regarding this emergency maintenance event.
UPDATE: We are taking this server down now for the system swap. We will update with more information shortly.
UPDATE #2: This maintenance is now completed and the system swap has been completed successfully. We will continue to monitor the status of this node and at this time we do not anticipate any additional issues/outages.
Thank you for your understanding and patience while we worked on this server. If you have any questions, please feel free to open a support ticket and we'd be happy to assist.
]]>Datacenter engineers are now proceeding to troubleshoot the issue. Thank you for your patience and we will update this status incident with more information as soon as possible.
UPDATE: We have resolved the issue for the time being with the help of on-site engineers. We'll be following up in a separate status incident in the coming days once we have a better understanding of the situation.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Because this node is continuing to experience symptoms we are proceeding to take this server offline to the workbench in order to replace the problematic drive retention clip with a new one. Once this is done we will update this status incident and continue to monitor the status of this node to ensure the health of it.
Thank you for your patience and we will have this node back online ASAP.
UPDATE: Facility hands are working on this server. We will share an additional update once available.
UPDATE #2: This node is continuing to experience symptoms, on-site technicians are actively working on implementing a resolution. At this point we have so far attempted full RAM replacements and re-seating NVMe drives on this system (as well as ensuring they are healthy), however, the symptoms are persisting. Before we put this machine back in production, we would like to identify the problematic hardware component in order to ensure long-term stability. We are currently running hardware diagnostics and closely monitoring the server on a physical level to identify the problematic hardware component causing these issues. Thank you for your continued patience and we will share another update on this status incident once available.
UPDATE #3: Current ETA for service restoration is around ~9:45 AM EST, could be sooner - but this is based on the rate the current hardware diagnostics are running.
UPDATE: We have restored services hosted upon this node and VM's should now be accessible (if you notice your VM is offline or if you are having any issues, please feel free to open a ticket). We will mark this status incident as resolved for the time being, however in the background we are continuing to actively monitor the status and health of this node in order to ensure its stability. Thank you for your understanding!
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved, this was as a result of a faulty/degraded power strip within the rack, which has now been replaced. These nodes are now back online, and VM's are accessible. If you have any questions, please open a support ticket.
]]>UPDATE #1: We are going to dispatch on-site technicians to physically troubleshoot this server due to the repeat symptoms this server is experiencing. We apologize for any inconvenience caused, and rest assured - we will get to the bottom of this as soon as possible. Thank you for your patience.
UPDATE #2: We are actively working on this. While your VM is online at this time, you may notice brief moments of intermittent unavailability/reboots while our techs work on this server physically.
UPDATE #3: On-site technicians are moving the server to the workbench for further troubleshooting work, you may notice your VM to be inaccessible during this time.
UPDATE #4: On-site technicians have reseated and resecured several hardware components which could have been the culprit for the unexpected reboots. We will continue to actively monitor the status of this node. In the meantime, services have been restored and are accessible. Please open a support ticket if you have any questions.
]]>UPDATE: As of now, all systems network connectivity have been restored. Our upstream's network engineering team is monitoring DDoS and doing their best to mitigate, if you are seeing any issues with your instance or service, feel free to reach out via ticket and we'll be happy to help.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>On-site datacenter technicians have been dispatched/requested to physically inspect this server, you may notice some downtime while we work on this server physically. We will update this status incident with additional details shortly.
UPDATE #1: On-site datacenter technicians are actively working on troubleshooting this matter. We will share additional updates once available.
UPDATE #2: We are currently running a FSCK check on this server and will update with the status shortly.
UPDATE #3: FSCK is now completed and this server is now back online. VM's are also responsive - if you're experiencing any issues or if you have any questions, feel free to open a support ticket.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to this server only, the server is now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was due to a network glitch on the Shared Hosting server. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. Please open a support ticket if you have any questions.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>This will take place at Saturday, Dec 4th, at 10pm Pacific Time. The scheduled event will take place in a one hour window from this time forward, and while no downtime is anticipated or planned, there is a small chance that you may notice slight network blips as we shift traffic to the new links.
If you have any questions please feel free to open a support ticket.
UPDATE: This is now completed and went through smoothly without any interruptions. Please let us know if you have any questions.
]]>UPDATE: This is fully resolved - feel free to open a support ticket if you have any questions.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: The issue on Fiber 5 shared server has been resolved completely. It was due to network glitch that happened on the server end and has been fully resolved. Please raise a support ticket to us if you experience any issues further.
]]>Please be aware that a RAID array rebuild is currently ongoing on the Fiber9 cPanel shared hosting server. No customer downtime is expected during this time as our servers run on hot-swap chassis. While the RAID array is rebuilding you may notice temporary degraded performance. Although not too impactful or noticeable, as we are a company that believes in providing full transparent updates to our customers, we are still creating this status incident for full transparency and to keep our customers in the loop.
UPDATE: The RAID array rebuild has been completed successfully, and server operations should be back to normal conditions. If you are experiencing any issues or if you have any questions, please feel free to open a support ticket (we'll be available to help).
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: On-site technicians have been dispatched to inspect this server physically.
UPDATE: On-site technicians have reseated several hardware components on this server and re-racked it. VM's hosted within this server are now back online and accessible. If you are still experiencing any issues or if you have any questions, please feel free to open a support ticket.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved, please open a support ticket if you have any questions.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: We are still investigating the issue with this node.
UPDATE: This node is back online, we will continue monitoring it. If you have any questions or if you are experiencing any issues, please open a support ticket.
]]>UPDATE: This is now resolved. It was a network glitch specific to this server only, the server is now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved, after re-seating hardware components physically on this server. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved, after re-seating hardware components physically on this server. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved, if you are still experiencing any issues please open a support ticket.
]]>UPDATE: Customers on this node have been sent an e-mail with more information regarding this incident. We will be marking this incident as resolved at this time.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved, if you are still experiencing any issues please open a support ticket.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>During this maintenance, access to your VM may be unavailable for up to one hour, though, we will do everything in our power to expedite and restore services as soon as humanly possible.
Upon the completion of this maintenance, we will be posting an update on this status event. Please feel free to open a support ticket in your client area if you have any questions or concerns regarding this emergency maintenance event.
UPDATE: This is now completed, and VM's that reside on this host node is now booting back up.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: This is now resolved, if you are still experiencing any issues please open a support ticket.
]]>UPDATE: This is now resolved. It was a kernel problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>To address this, we are once again issuing a reboot to the host node: SJ116KVM and will have this node back online very shortly.
UPDATE: This is resolved and back online.
]]>UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>This will be back online shortly, thank you for your patience.
UPDATE: This is resolved and back online.
]]>This will be back online within a few moments. If you have any questions, please let us know.
UPDATE: This is resolved and back online.
]]>UPDATE: Most services have been restored and are back online, however a small handful of services in this location are still impacted. We will share additional news regarding this shortly.
UPDATE : This is now resolved, if you are still experiencing any issues, please open a support ticket and we'd be happy to assist. This was a facility power related issue, we are currently pending an official RFO from the datacenter/building. We will update this status incident with more information once we receive that (within the next 72 business hours). In the meantime, all services should be back online and powered on.
]]>UPDATE : This is now resolved, if you are still experiencing any issues, please open a support ticket and we'd be happy to assist.
]]>UPDATE #2: This is now resolved, if you are still experiencing any issues, please open a support ticket and we'd be happy to assist.
]]>We are currently working into this node further.
UPDATE #1: On-site datacenter technicians are currently inspecting this server, we will update with additional information shortly.
UPDATE #2: On-site datacenter technicians are still testing various hardware components within this server to help determine the root cause of the unexpected reboots/outages within this server. We apologize for the inconvenience caused and appreciate your patience. We will update this incident with more information once available.
UPDATE #3: Diagnostics/in-depth tests are currently being run on this server.
UPDATE #4: We are still working on narrowing down this issue. Our on-site technicians are actively working on this and we hope to have a full resolution on this shortly. Thank you for your continued patience.
UPDATE #5: This is now resolved and VM's are back online and accessible. If you are still experiencing any issues at this time, please open a support ticket and we'd be happy to help. If in the future we need to schedule a maintenance event for this node, clients will be sent advance notice via email. If you have any questions please let us know.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE AS OF 8:48 AM PDT -- The DDoS attack is being actively mitigated - websites are now loading without any issue. Please be advised that incoming PING (ICMP PING) has been temporarily disabled. This may create false positives with external monitoring services, if you have any uptime monitors set up for your site.
We will continue to keep this incident open as we continue to monitor the situation, though, currently everything appears to be online and accessible.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE #1: This is now resolved after reseating hardware components on this server. If you are still experiencing any issues please open a support ticket and we'd be happy to assists
UPDATE #2: It's is experiencing issues again, we are checking on it
UPDATE #3: This is now resolved after reseating hardware components on this server. If you are still experiencing any issues please open a support ticket and we'd be happy to assists
]]>UPDATE: Facility hands is currently inspecting the server physically. We will update this incident with additional information as it is made available to us.
UPDATE #2: This is now resolved after reseating hardware components on this server. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this incident with additional information shortly.
UPDATE: This is now resolved, a switch firmware update has resolved the intermittent network connectivity issue. Please open a support ticket if you have any questions or if you are still experiencing any issues.
]]>Facility hands is currently investigating the issue. We will share additional updates in this incident once available.
UPDATE: This was determined to be a switch related issue. The switch within this cabinet has been replaced. Connectivity has mostly been restored but will be intermittent until the switch has been fully reconfigured. We will update this incident once it is fully resolved.
UPDATE #2: This is now resolved, if you are still experiencing any issues, please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are actively looking into this and expect to have a full resolution on this matter shortly.
UPDATE: This is now resolved, if you are still experiencing any issues or if you have any questions, please feel free to let us know.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will report additional information once available.
UPDATE: This has been resolved after re-seating some hardware components on this server. VM's are in the process of booting back online. If you are still experiencing any issues after a few minutes, please feel free to open a support ticket and we would be happy to assist.
]]>Our network engineering team is currently looking into this further. We will have more information on this shortly.
==UPDATE== This is now resolved. Please open a ticket if you have any questions or if you require any assistance.
]]>We will update this incident with additional information once we have a full resolution on this matter. Thank you for your patience.
UPDATE: Network engineers are on-site and actively investigating the issue. We will update this incident once additional information is made available to us.
UPDATE #2: This is now resolved after replacing a failed top of rack switch. If you are still experiencing any issues, please open a support ticket and we'd be happy to assist.
]]>UPDATE: This has been resolved. If you are still experiencing any issues after a few minutes, please feel free to open a support ticket and we would be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved and all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved and all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will update this incident with additional information shortly.
UPDATE: This is related to a faulty RAM DIMM on the physical host node, we are currently troubleshooting this and in the meantime, have submitted a request with facility hands for RAM replacement. We will update this incident with more information as we progress further with AMS104KVM. Thank you for your patience.
UPDATE: This is now resolved and all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Our upstream providers are looking into this matter and we will update this incident once additional information is available.
UPDATE: The network routing issue has been resolved - all routes are now optimal (total impact was less than 5 minutes). If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Our upstream providers are looking into this matter and we will update this incident once additional information is available.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this incident once this has been resolved.
Thank you for your patience and understanding. Please feel free to contact us if you have any questions.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>As a precautionary measure, we will be performing the same maintenance task on ATLRYZEN101 which has also experienced some signs of symptoms recently, as the RAM for both of these servers came from the same batch. ATLRYZEN101 customers will get an e-mail shortly with the maintenance timeframe details.
UPDATE #1: The maintenance is about to begin on ATLRYZEN100. We are moving this status incident to "Open/Investigating" for the time being. We'll share additional updates as we progress further.
UPDATE #2: ATLRYZEN100 has been safely powered off. Facility hands have been instructed to begin working on this task. We will share additional updates as progress is made.
UPDATE #3: Work on ATLRYZEN100 is ongoing. We have also scheduled an emergency maintenance window for ATLRYZEN101 for 4PM to 5PM EDT today, affected ATLRYZEN101 customers have been sent an e-mail.
UPDATE #4: The work on ATLRYZEN100 is now completed. The server is now re-racked and VM's are now back online. We will continue monitoring ATLRYZEN100 to ensure stability.
We will be keeping this status incident open, and will share an additional update once we begin working on ATLRYZEN101.
UPDATE #5: ATLRYZEN101 has been safely powered off. Facility hands have been instructed to begin working on this task. We will share additional updates as progress is made.
UPDATE #6: The work on ATLRYZEN101 is now completed. The server is now re-racked and VM's are now back online. We will continue monitoring ATLRYZEN101 to ensure stability. We will be closing out this status incident now, if you have any questions please feel free to open a support ticket.
COPY OF SCHEDULED MAINTENANCE E-MAIL BELOW:
Hello,
You are receiving this e-mail because you have one or more VPS's located on the host node: ATLRYZEN100.
Maintenance Date: Monday, October 11, 2021
Maintenance Window: 3PM to 4PM EDT
During this maintenance, you will notice your VM go offline for about ~30-40 minutes at any point within the above specified maintenance window. Facility hands will be taking the server offline, moving it to the workbench, and properly replacing the RAM modules on this server, which our system administrators believe will assist with the unexpected shutdowns this node has experienced recently. After the RAM modules are replaced, facility hands will be re-racking the server and ensure proper power and network cabling, at which point VM's will be automatically booted up again.
We apologize for any inconvenience caused. Please feel free to contact us by opening a support ticket in your client area if you have any questions or concerns.
Thank you,
RackNerd Support Department
support@racknerd.com
While the node is currently online and VM's are accessible - our senior system administrators are currently working together internally to properly identify and implement a permanent resolution towards this matter. We have already rolled out some live patches, on the host-node level that we believe may help with the symptoms this host node is experiencing.
At the moment, it is not confirmed if the recent issues are as a result of a failed hardware component, such as a faulty RAM DIMM. If our system administrators identify an issue that is related to the physical hardware components on this server, we will notify affected customers via e-mail with a planned downtime window for us to correct/replace the faulty hardware component, if any.
Thank you for your patience and understanding regarding this matter.
UPDATE: We will be performing a maintenance for RAM replacement on ATLRYZEN100 which we believe to be the permanent solution for this issue going forward. Affected customers have been sent an e-mail with additional information.
]]>The network team is currently investigating this further. We will update this incident once we know more.
UPDATE: This has been resolved (it was a network routing issue specific to this host node only). All VM's are now responding without any packet loss on a global ping test. If you are still experiencing any issues, please feel free to open a support ticket and we would be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this incident with additional information shortly.
UPDATE: This has now been resolved, all routes are now optimal.
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a kernel-related issue specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a kernel-related issue specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: There is a physical hardware component related failure on this server. The on-site datacenter technicians are currently running diagnostics on this server in order to be able to narrow down and identify which component is faulty so that we can replace the correct failed part accordingly. We appreciate your patience while these diagnostics are being ran.
We will share additional updates as soon as we know more.
UPDATE #2: We are still working on this matter. We will continue to make progress on this and will keep affected customers updated.
UPDATE #3: Additional information regarding this matter was emailed to all SJ126KVM customers.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently working into this node further and will update with more information ASAP.
UPDATE: This has been resolved and VM's are currently in the process of booting back up. If you are still experiencing any issues after a few minutes, please feel free to open a support ticket and we would be happy to assist.
]]>UPDATE: This is now resolved. It was due to a slight network glitch on this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist
]]>UPDATE: This is now resolved. It was a network glitch specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>During the time of the firmware update, you may notice very brief intermittent network connectivity - though this should be a very quick update with minimal impact to uptime.
We will update this incident again once the switch update has been completed.
UPDATE: This has been completed, we will keep this incident on a monitor status while we continue to monitor the stability of the new switch firmware.
UPDATE: Connectivity looks to be stable since the update, if you have any questions please feel free to open a support ticket, we'll be closing out this incident for now.
]]>UPDATE: The PSU's on this server were showing signs of failure. We have replaced the PSU's on this server at this time. This node is now accessible and VM's are now back online. If you have any questions, please feel free to open a support ticket.
]]>UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved.
]]>We apologize for the issues with this node and please know that we are treating this as a top priority to permanently resolve.
We will be providing customers affected with one month free credit upon request once this incident has been resolved. Thank you for your patience.
This incident will be updated shortly with additional information as we make progress with this matter.
UPDATE: Upon taking the server down for physical inspection, we were able to identify that a stick of RAM on this server is faulty. We are currently in the process of getting this replaced.
UPDATE: The problematic stick of RAM has been replaced (and we've reseated all the other DIMM's on the server too) and the node is now back to production. VM's are currently in the process of booting up. Please feel free to open a ticket if you have any questions.
]]>UPDATE: This is now resolved. All VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Network engineers upstream are currently looking into this, and we will update this incident with additional information once available.
Update - This has now been resolved and services in this location are back to normal operation. If you have any questions please feel free to reach out to us.
]]>All services are back to normal operational now. If you are still experiencing issues or if you have any questions, please feel free to let us know.
]]>UPDATE: This is now resolved. All VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will have this node back online and accessible shortly!
UPDATE: This is now resolved and kernel on this node has been updated to the newest version. All VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further.
UPDATE #1: On-site datacenter engineers are still investigating this issue further.
UPDATE #2: We are still actively working on this issue. This has been narrowed down to a RAID array corruption issue with this specific host node, and we have replaced the necessary failed hardware components. At this time, we are currently working to import the RAID configuration to resolve the issue and update this incident with additional information once available.
UPDATE #3: The attempted import is currently in progress and we will update this incident again once we make some additional progress regarding this matter. Please be assured that resolving this incident is currently a top priority of ours and we have multiple senior systems administrators actively working on this task.
UPDATE #4: This has been now resolved and VM's are back online and accessible. If you are still experiencing any issues please feel free to open a support ticket.
]]>Our team is currently working on it, we will update with more information shortly.
Update- This has been now resolved. If you are still experiencing any issues please feel free to open a support ticket.
]]>UPDATE: This is now resolved. All VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We will update this incident once the node is back online.
UPDATE: The maintenance work is now completed, and the RAM DIMM's on this server have been replaced to avoid this issue from occurring again in the future. All VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved.All VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Network engineers upstream are currently looking into this, and we will update this incident with additional information once available.
UPDATE: Network engineers upstream are continuing to work on this situation. We are in communication with them and will post additional updates on this incident page once available.
UPDATE: We have identified the issue to be related to a DDoS attack upstream. Network engineers are actively working to mitigate this attack against the network. As you may know, these types of DDoS attacks continue to innovate and evolve, so there are various levels of attacks that need to be learned, identified, and mitigated (not something as simple as the click of a button). The network team has applied some communities to help with this and is still working towards applying additional rules and mitigation to adapt to this evolving DDoS attack.
During this time, you may continue to notice intermittent network connectivity but as these rules become effective, you will notice that the network connectivity will begin to stabilize. We will continue to communicate here as best as we can as we continue to learn additional updates from the networking team.
Update: This has now been resolved and services in this location are back to normal operation. If you have any questions please feel free to reach out to us.
]]>UPDATE: This is now resolved.All VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved (it was caused by a faulty network cable which has been replaced). All VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Our team is currently working on implementing a full resolution and mitigation towards this, we will update with more information shortly.
UPDATE: We replaced the network cable connections on this server and will continue to monitor the situation. For the meantime we will mark this incident as resolved as it has been stable for some time since the cable replacement. If you are still experiencing any issues please feel free to open a support ticket.
]]>UPDATE: This is now resolved. All VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We believe this to be a network routing related issue and have escalated the problem to our upstream provider. We will update this status page with more information once available.
UPDATE: Our upstream provider is aware of the problem and is actively working on a solution. We will update once additional information is available. Thank you for your patience!
UPDATE: This appears to be a network issue localized with the EGIHosting network blend, which is the bandwidth provider that our upstream provider utilizes in this location. We have been informed that EGIHosting's network team is aware of the issue.
Once we have more info, we'll be sure to share it here.
UPDATE: Network engineers are still working on this issue. Some routes are restored, but not all. We will update this status event once we have a full resolution on this.
UPDATE/RESOLUTION: This matter is now fully resolved, and services hosted in San Jose are now responding globally without any packet loss. If you're still experiencing any problems, or if you have any questions regarding this, please feel free to open a support ticket and we would be happy to help.
]]>To minimize negative impact to production, we will be performing an emergency maintenance to replace the RAID Controller on this server as soon as possible. During this time, your VM may be inaccessible for up to one hour. We will update this event again once this maintenance has been completed.
UPDATE: The RAID Controller has been replaced and all VM's are online and accessible. If you are still experiencing any issues or if you have any questions, please feel free to open a support ticket.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: Within minutes our networking team was able to resolve this, there was an attack targeted to one IP range that needed to be filtered. All services are back to normal operational now. If you are still experiencing issues or if you have any questions, please feel free to let us know.
]]>UPDATE: This is confirmed to be a power related event - the facility has determined a failed PDU - and we're working close with facility hands for replacement. We'll continue to post updates here.
UPDATE #2: Our facility team has completed the PDU replacement. Affected servers are now in the process of booting back up, and individual VM's will begin to boot up automatically afterwards - that will take maximum 10 minutes from the time of this post. If you still experience any issues after this time, please feel free to open a support ticket.
]]>UPDATE: There appears to be a hardware component related failure on this server physically. The on-site datacenter technicians are currently running diagnostics on this server in order to be able to narrow down and identify which component is faulty so that we can replace the correct failed part accordingly. We appreciate your patience while these diagnostics are being ran.
We will share additional updates as soon as we know more.
UPDATE: Our technicians are still actively working on this issue.
UPDATE: We're making great progress and we're hoping to share some positive news within the next hour or so. Thank you for your continued patience!
UPDATE: This issue was narrowed down to a complex RAID-10 array degradation issue - our team worked tirelessly to get this node back to operation. VM's have been booted up at this time, if you are experiencing any issues specific to your container, please open up a ticket and we'll be here to help. Additionally if you have any questions regarding this specific event that you would like to discuss with our management team, please e-mail bizdev@racknerd.com. In regards to technical support, please be sure to open a support ticket from within the client portal at https://my.racknerd.com/clientarea.php
]]>As our VPS nodes are RAID-10 protected, you will not experience any data loss during this event.
UPDATE: The on-site datacenter technicians will be replacing the failed drive on this server shortly.
UPDATE: The failed drive has been replaced, and this server is back online. VM's are now responsive again, if you are experiencing any issues or if you have any questions, please open a support ticket.
]]>UPDATE AS OF 6:01 AM PDT -- As this looks to be a network related problem impacting this server only - the issue has been escalated upstream and network engineers are actively looking into the issue. We will update this status page with more information once we have more information to share. We appreciate your patience and please know that this is our top priority to bring back online.
UPDATE AS OF 7:35 AM PDT -- This server is experiencing a sophisticated DDoS attack. Network engineers are actively working on mitigating the attack.
UPDATE AS OF 7:50 AM PDT -- The DDoS attack is being actively mitigated. Please be advised that incoming PING (ICMP PING) has been temporarily disabled. This may create false positives with external monitoring services, if you have any uptime monitors set up for your site.
UPDATE AS OF 8:02 AM PDT -- We are continuing to monitor the situation.
UPDATE AS OF 8:44 AM PDT -- We are still monitoring this situation and the server is still routed through our DDoS mitigation. We will share more information soon.
UPDATE AS OF 9:27 AM PDT -- Since our network engineers worked to implement some rules to combat and mitigate this sophisticated DDoS attack, the server looks to be stable for quite some time now and we are still continuing to monitor this situation actively. Websites hosted upon this server are loading fine and the DDoS attack is actively being mitigated. As a reminder, please note that your external monitoring services may report false positives during this time as ICMP PING has temporarily been disabled. However, your websites are actually up and loading fine.
UPDATE AS OF 10:56 AM PDT -- Since routing this server to our active mitigation technology, websites have been loading quickly and without any issues for the past several hours. As websites are loading fine, we will go ahead and mark this incident as resolved but will be continuing to monitor this in the backend. Please note that ICMP PING will continue to be disabled on this server for the meantime, though this does not mean the server is down. If you have any questions or if you are still experiencing any issues, please feel free to open a support ticket and we would be happy to assist.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved, and all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Our team is currently working on implementing a full resolution and mitigation towards this, we will update with more information shortly.
UPDATE: This has been successfully mitigated and everything is functioning as intended now. Please feel free to contact us if you notice any further problems.
]]>UPDATE: This is now resolved, and all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will provide additional updates on this matter once we have more information.
UPDATE: On-site datacenter engineers in Amsterdam are engaged and working on the issue.
UPDATE: We are still actively working on this issue.
UPDATE: This is now resolved, and all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved, and all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>Our network engineering team is currently looking into this further. We will have more information on this shortly.
UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network glitch specific to both two nodes. all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This was a network routing issue which was resolved upstream. All VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. All VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further and will update with our findings.
UPDATE: This appears to be an issue related to one of the RAM DIMM's on this server, our on-site technicians are troubleshooting further.
UPDATE: The faulty DIMM has been replaced. This is now back online.
]]>UPDATE: We are still troubleshooting this server physically with the on-site datacenter technicians. Additional updates will be posted again shortly.
UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further.
UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>SJ127KVM is down with experiencing network connectivity issues. We are aware of it and are currently looking into this node further.
UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further.
UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further.
UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further.
UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further.
UPDATE: This is now resolved. It was a network-related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>This network related issue has been escalated upstream and network engineers are actively looking into the problem.
UPDATE: This was an RPKI related issue impacting a single /24 IP range in Los Angeles DC-05 (no other IP ranges were affected). The RPKI issue has now been corrected upstream on this IP range at this time. The range is now responding perfectly fine to approximately 95% of the world, but ~5% are still awaiting propagation before routes will begin to respond from those locations. We will leave this incident open for the time being, until propagation has completed globally - but chances are, you should be back online and all set now.
UPDATE #2: The range appears to now be propagating globally without any issues. If you are still experiencing any issues, or if you have any questions, please submit a support ticket in your client area. We will now be marking this incident as resolved.
]]>Update: SJ117KVM is back online. This incident is now fully resolved
]]>This network related issue has been escalated upstream Provider and their network engineers are actively looking into the problem.
Update --
We found the root cause and are still working with our upstream to fix it.
UPDATE: The network routing issue has been resolved by the upstream. If you are still experiencing any issues please feel free to open a support ticket.
UPDATE #2: It appears that one KVM VPS node in San Jose (SJ116KVM) is still experiencing connectivity issues, the on-site team is currently looking into this node further. All other San Jose KVM VPS nodes appear to be accessible and working as intended.
UPDATE #3: SJ116KVM is back online. This incident is now fully resolved.
]]>UPDATE: The network routing issue has been resolved by the upstream. If you are still experiencing any issues please feel free to open a support ticket.
]]>UPDATE: This is now resolved. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>We are currently looking into this node further.
UPDATE: This is now resolved. It was a network related problem specific to this host node only, all VM's are now responsive again. If you are still experiencing any issues please open a support ticket and we'd be happy to assist.
]]>This network related issue has been escalated upstream (to Sharktech) and their network engineers are actively looking into the problem.
UPDATE: The routing issue with Los Angeles DC-05 was caused due to one of our upstream's aggregate switches being problematic - this has now been resolved and services in this location are back to normal operation. If you have any questions please feel free to reach out to us.
]]>We are currently looking into this node further.
-- Update --
We found the root cause and are still working with our upstreams to fix it. It appears to be a network routing related problem impacting a few select routes. This issue has been escalated upstream and we will report back with additional details once available.
UPDATE: The network routing issue has been resolved by the upstream. If you are still experiencing any issues please feel free to open a support ticket.
]]>We are currently looking into this node further.
Update: This is now resolved and accessible again.
]]>Update 7:47 AM PDT: Thank you for your continued patience. As some customers were inquiring for more information, we are able to confirm this is a network related incident that is partially affecting some servers hosted out of Los Angeles DC-02, this incident is not power or facility related. The issue is being worked on by our upstream provider's network engineers and we were told it should be resolved soon.
Update 8:16 AM PDT: As an update our upstream has made progress and services appear to be restored. We will be updating you with more information as made available to us. In the meantime, if you are still experiencing any connectivity issues, please be sure to open a ticket and we’ll help.
The event was caused due to a software related problem in one of the core routers. Both IPv4 and IPv6 connectivity should now be fully stabilized.
]]>We are aware of intermittent network connectivity affecting a portion of our San Jose KVM VPS nodes. Network engineers at our upstream provider have been notified and are looking into optimizing and resolving this.
Update: This appears to have normalized and network engineers upstream have optimized some routes. Please feel free to open a support ticket if you are still experiencing any issues.
]]>UPDATE: This is now resolved and Fiber2 is now accessible again. There was a routing issue impacting the main shared IP address only. Customers with a dedicated IP were not affected by this incident.
]]>During this time, network access to VM's may not be available (or degraded).
UPDATE: We are aware of degraded network performance for VM's hosted on DC02LAXKVM69 which might be related to a bad network cable connection. We are working to replace the network cable connections to this server in order to correct the problem.
UPDATE #2: This incident is resolved. The network cable connections to node DC02LAXKVM69 have been replaced, and network performance is now back to normal.
]]>During this emergency maintenance event we will be re-applying thermal paste and re-seating both of the processors on the node, as well as using this as an opportunity to perform a BIOS update to the physical host node.
Due to the nature of the issue causing symptoms that could be negatively impactful to production, we will be proceeding with this emergency maintenance as soon as possible (within approximately one hour from the time of this message). During this maintenance, access to your VM may be unavailable for up to 2 hours, though, we will do everything in our power to expedite and to restore services as soon as humanly possible.
We will be updating our status page at https://status.racknerd.com/ - please follow this page for updates. Upon the completion of this maintenance we will be posting an update on the status page. Please feel free to open a support ticket in your client area if you have any questions or concerns regarding this emergency maintenance event.
UPDATE: We have successfully re-applied thermal paste and re-seated the processors at this time, and VM's are now back online on ATL102KVM. We will need to schedule one more maintenance next week to wrap this up. We will email all customers on ATL102KVM again to schedule another maintenance window at that time.
]]>Update: We are currently still looking into the situation and will update this incident with more details shortly.
Update #2: There was a utility power outage this morning, at which point load was successfully transferred over to generators. The on-site datacenter engineers are currently looking into why this was not sustained. We will have services restored as soon as possible and thank you for your patience.
Update #3: While attempting to receive communication from our upstream (the facility/building) we unfortunately have been provided limited information. It appears all hands are on deck thus limited information from our upstream. The folks providing updates are operating with limited information as the facility staff works hard to restore power. The information we have received so far is that switching over to generators caused the extended downtime, with difficulties to restore to ground power as a result. We will continue to dig for more answers / timeframes and update this incident accordingly with more information. We are just as eager and realize the limited information causes frustration.
Update #4: We received an update upstream. The issue was identified as one of the building’s generators which has historically worked for years, was recently load tested and had a mechanical failure. This resulted to power being cut to the core routers and fire suppression system being activated as a precaution. Unfortunately, the city’s fire department opted to cut power to the rest of the building as a precaution even though the power systems were independent. At this time we are awaiting for an emergency inspector to arrive onsite to give the approval so that power can be restored to cabinets and so that servers can be powered on again. We appreciate your patience and we will continue to update this incident with additional information as we are able.
Update #5: The facility is still awaiting for the inspector to come out and provide approval. Once we have additional information, we will be sharing it on this page.
Update #6: The facility hands are making some great progress, and the inspector is on-site. Some servers in Utah are powering up, and we will continue to update this status page with more updates as available.
Update #7: Here is a brief overview of the events as they transpired. Sunday afternoon the city power was disrupted and, as designed, the facility's backup generators automatically switched on. However, during that transition, one of its backup generators that had been recently tested and benchmarked specifically for this situation experienced a catastrophic failure, caught fire, and as a result initiated the fire suppression protocol. No fire damage was inflicted on customer's servers. However, there are a handful of machines that may need to be inspected for water damage, before they can be powered on (and/or rebuilt).
The current course of action that the facilities team is working on, is to restore power throughout the datacenter in accordance to the inspector's guidelines. We will continue to update this status page with additional updates once available.
Update #8: A majority of our servers in the Utah datacenter that did not experience water damage have been powered on. A couple more cabinets remain and the electricians are working on it.
Additionally, we are working on preparing a plan of action to rebuild servers that did experience water damage. As of right now, the preliminary plan is to build new machines of identical/similar specs, and then move the drive(s) over, followed by routing the same IPv4 space over to the new machine so that your server is back up seamlessly as it was before. As the drives were not submerged throughout this incident, the chance of data loss is very unlikely. Please trust that we are doing everything we can in cooperation with the on-site facility team to restore service for everyone in Utah.
We will continue to share additional updates here as they are made available to us.
Update #9: The facility staff in Utah are working to rebuild servers that experienced water damage. This is being done in a triage process system. As mentioned earlier, data loss is unlikely as drives were not submerged.
Update #10: We also wanted to mention that we've successfully restored some customers, upon request, to other datacenter sites of ours, such as Los Angeles and New York. We understand uptime is important, and for customers who are time-sensitive we do have alternatives available. If you are an affected customer in Utah and have offsite backups of your own, or have obtained an offsite backup service with us and would like to discuss getting deployments set up in other datacenter locations of ours, please email bizdev@racknerd.com and we will review our available inventory and work with you to get you set up as soon as possible in another datacenter location. Of course, at no additional charge to you.
Update #11: The servers that did not experience water damage are back online and accessible again. With regards to the servers that did experience damage, our upstream has indicated that while they are still assessing the situation, it could ultimately be additional weeks of downtime in efforts to rebuild the servers and restore service. While the risk is extremely minimal in terms of data loss as the drives were not submerged, it is important for you as well as us to understand that your uptime is critical, and service is ultimately what you seek for at the end of the day. We are open to help restore affected customers with servers available in other datacenter locations of ours. We have been reaching out and communicating to our customers that were affected by this Utah incident to offer alternative solutions. If you would like to explore the possibility of deploying service in another location, please feel free to email bizdev@racknerd.com and we will work with you to get you set up as fast as possible in another location. There will be no additional charge/cost to you for this.
Update #12: The on-site team in Utah is making great progress with getting more of our affected servers rebuilt & back online. The on-site team is still working to rebuild some of our affected servers that are still offline, but great progress was definitely made today and a good amount of our customers have noticed their servers start responding again today.
On our side we are also making great strides in communicating with our affected customers and working closely with them on solutions. For time sensitive situations, we have relocated a handful of customers who were open to it - to alternative datacenter locations such as Los Angeles and New York. As mentioned in the last update if this is something you were open to exploring please definitely email bizdev@racknerd.com so we can review our available inventory and get you set up in another datacenter location ASAP. For customers that are needing to stay in Utah (i.e. due to data, or due to specific location requirements or some other reason), please know that everything in our power is being done to help expedite and we hope to have everybody back online soon.
Update #13: A lot of servers in Utah have been re-racked and brought online each and every day in the past week, and by now, all infrastructure in this location should now be back online and accessible. Please feel free to contact us if you are experiencing any issues or if you have any questions.
]]>Update: The issue which impacted some servers network-wise within our Los Angeles DC-02 location appears to be resolved and we are able to now ping services hosted within our Los Angeles DC-02 location globally without any issues or packet loss. Total downtime for this incident appears to have been less than 4 minutes. We are continuing to communicate with our upstream provider to figure out what exactly happened, and create resiliency plans to resolve such network route incidents without any visible or noticeable downtime/impact.
We will be marking this incident as resolved, but if you have any questions, feel free to reach out to us via support ticket.
]]>We will update this incident once additional information is available regarding this matter.
Update: Datacenter engineers are currently investigating this issue. We will have this resolved shortly.
UPDATE: This is now fully resolved. All VM's hosted within DC05LAXKVM501 are responding again. If you have any questions or if you are still experiencing any issues, please feel free to reach out to us.
]]>Our upstream is currently looking into this.
Update: Our upstream's network engineers are actively looking into the situation. We will have this resolved very shortly!
Update #2: While this appears to be resolved for majority of servers that were initially impacted, we are aware of intermittent network connectivity that is still impacting some servers. The network engineers are actively working to resolve the remaining section of servers impacted.
Update #3: This is now resolved. This was identified as an issue with one of the networking infrastructure supervisor modules which affected a small portion of our servers hosted within Los Angeles DC-02 - and our upstream is actively working on implementing safeguards to prevent an incident like this from repeating again in the future. If you are still experiencing any issues, or if you have any questions, please open a support ticket and we would be glad to assist you.
]]>Update: We have identified this as a partial network issue that is impacting certain routes to our San Jose datacenter. This issue has been escalated to the network engineers who are currently working to resolve this.
Update: The issue affecting certain routes to/from SJC has been resolved by the network engineers. Everything appears to be operational and running optimally now. If you have any questions or if you are still experiencing any issues, please open a support ticket.
]]>Update: We have identified the issue as a degraded RAID array on this machine. Our on-site engineers are working to replace any problematic component(s). We will update this with more information as it becomes available.
Update #2: We have resolved the degraded RAID array by replacing the problematic drive. All VM's should now be accessible, if you are experiencing any issues or if you have any questions, please open a support ticket in your client area.
]]>Update: This was an issue affecting the main IP of Fiber4 only. Our upstream provider has corrected the routing issue with that range. This is now fully resolved.
]]>Update: This has been resolved after a reboot. Our systems administrators are currently working on identifying the root cause. In the meantime, all services should be back online and accessible.
]]>Update: This was resolved. The network cable connections to this server have been replaced.
]]>VM's are currently starting back up.
UPDATE: All VM's residing on this host node are back online.
]]>UPDATE: This has been resolved.
]]>UPDATE: Connectivity to Fiber2 has been restored.
]]>Services in Los Angeles DC-02 are operational and running as we have a multi-homed network, but if you are accustomed to your network routes typically going thru Any2 and/or CenturyLink, you may notice that these routes have been temporarily shifted to another carrier in our blend, such as NTT.
We are keeping a close eye and monitoring this situation. Once Any2 and CenturyLink are stable again, we will proceed with adding these back to our BGP blend.
UPDATE: Any2 peering connectivity has been restored in Los Angeles DC-02.
]]>Replacing the faulty RAM module on this server will resolve this issue.
Due to the nature of the issue causing symptoms that could be negatively impactful to production, we will be proceeding with this emergency maintenance as soon as possible. During this maintenance, access to your VM may be unavailable for up to one hour, though, we will do everything in our power to expedite and to restore services as soon as humanly possible.
Upon the completion of this maintenance we will be posting an update on this status event. Please feel free to open a support ticket in your client area if you have any questions or concerns regarding this emergency maintenance event.
UPDATE: This is now completed, and VM's that reside on this host node are now booting back up.
]]>UPDATE: This has been resolved and client connectivity to our SolusVM has been restored.
]]>Update: This is now resolved. This was a routing issue due to one of the peers within the BGP blend. The problematic peer has been temporarily put on standby and a global test now shows connectivity without any packet loss. Please feel free to open a ticket with our Support Department if you are still experiencing any issues or if you have any questions.
]]>Update #1: This appears to be a partial network outage affecting connectivity to the server when connecting from certain regions - this has been escalated to our upstream at this time.
Update #2: Network engineers upstream are actively engaged and working on the reported issue. Additional updates to follow shortly.
Update #3: Connectivity is now restored. Please contact our Support Department if you are still experiencing any issues after this time. More details and a RFO (reason for outage) will be updated within this incident within the next 72 hours.
]]>We are proactively working with on-site datacenter engineers to replace the faulty RAM module on this host node, which should correct the issue.
UPDATE: This is now fully resolved and VM's that reside on this host node are now booting back up.
]]>Update: Packet loss is resolved at around 3:09 AM PST.
]]>Update: The culprit appears to be related to the rack PDU, we are working with the facility remote hands to help resolve it. If needed, we will replace the rack PDU.
Update: 3:22 AM PST -- Power is restored to the cabinet, though we are attempting to restore connectivity from the switch level.
Update: 3:21 AM PST -- Power to actual servers was not an issue, power to the switch was the actual issue. Facility remote hands assisted in bringing the switch back online.
This event is now concluded, resolved and all VM's within DAL102KVM are now accessible.
]]>