Client Portal

There are no Open Network Issues Currently
Bend Oregon - Lumen Fiber Cut (Resolved) High

Affecting Other - Bend, Oregon Datacenter

  • 09/17/2021 12:27 - 09/17/2021 14:39
  • Last Updated 09/17/2021 14:37

At 12:27PM PST the NOC observed anomalies in our Bend, Oregon datacenter. Upon investigation it appears that one of our primary transit links to Lumen/CenturyLink/Level3 is flapping, as such we are engaging the Lumen NOC team.

Update: At 12:55PM PST the Lumen NOC team confirmed a partial fiber cut in the area affected circuits, causing the bounces. They have advised that additional bounces and/or full outages are possible. As such we have removed Lumen from our network mix temporarily until the issue is resolved. Please note that you may see packet loss during this time. Once Lumen has confirmed that the fiber cut is fully repaired we will update this network advisory.

Update: At 2:39PM PST the Lumen NOC team confirmed that the partial fiber cut has been repaired. As such, we are closing this network event.

North Carolina: Network Issue (Resolved) Critical

Affecting Other - North Carolina

  • 08/30/2021 10:30
  • Last Updated 08/30/2021 10:33

North Carolina: We are currently investigating an internal network anomaly that is causing packetloss.  There is no ETA at this time but we have all hands on deck investigating and working towards a resolution, hopefully any minute.

 

As of 1:05pm EST, we resolved what turned out to be a spoofed outbound attack originating from inside our own network.   The magnitude of the bandwidth should not have affected our network but the packet rate and type of packets sent (payload) managed to overload some of our devices and cause packetloss and misbehavior.    We've identified the source of the attack vector and we are working with Cisco and our device manufacturers to harden our settings to permanently protect against this sort of abuse.    We sincerely apologize for the problems this caused.

8/23/2021 Aptum LA Network Issues (Resolved) Critical

Affecting Other - Aptum Los Angeles

  • 08/23/2021 21:06 - 08/23/2021 22:29
  • Last Updated 08/23/2021 22:29

At approximately 9:04PM pacific time our NOC team noticed anomalies in the network in our Aptum, Los Angeles location. These anomalies are creating packet loss, or outages intermittently for random locations.

We immediately began investigating. The issue does not appear to be local to our network, as such we have opened a ticket with the Aptum NOC team.

Update: 9:46PM - The Aptum NOC team has acknowledged an issue in their network and have escalated it to their engineering team. This is being handled as a priority 1 service impacting incident. Further updates will be posted when available.

Update: 10:21PM - The Aptum NOC has resolved the issue on their end, and the incident has been resolved.

iCF Cloud + Uniform Issues (Resolved) Critical

Affecting Server - NC-Uniform

  • 07/20/2021 07:00 - 07/20/2021 10:30
  • Last Updated 07/20/2021 07:14

We are currently experiencing issues with our iCF cloud infrastructure in Charlotte, NC due to an unexpected result of some development work.   Affected servers include Uniform as well as other selected Charlotte, NC cloud VPS servers.   We are working on restoring it now and should have a resolution shortly.   ETA:   11am - noon EST.

Scheduled Maintenance - Charlotte, NC (Resolved) Critical

Affecting Other - Distribution Switching

  • 05/14/2021 19:00 - 05/14/2021 16:28
  • Last Updated 05/13/2021 04:38

Dear Valued Client,

 

This notice is to inform you of upcoming scheduled maintenance in our Charlotte, NC facility on Friday, May 14th at 7pm EST.   We intend to fail over from temporary backup distribution switches to our primary (redundant) cluster.   The goal is to restore full redundancy to our switching infrastcuture.    

 

Background:  On Saturday, May 8th, we experienced cascading PSU failures in our primary distribution switch cluster.   It caused a complete outage and we ultimately failed over to backup/standby switches.    These switches have been in use since but are not suitable for continued production utilization due to lack of redundancy.   

 

Scope of work:   We will be replacing 2 failed PSUs (power supplies) with 4 new, tested, and fully redundant PSUs.   Our distribution switch cluster will effectively have twice the power redundancy as it had when it experienced multiple failures.   

 

Timing / window:  Starting at 7pm EST, we are scheduling a 30 minute window for the work but actual downtime is not expected to exceed 5-7 min as cabling is switched from the failover switches to the primary cluster.   Both switches will be operating normally before we proceed so there should be little chance of unexpected issues or extended downtime.   

 

As always, your patience and understanding is appreciated.   We look forward to serving you for many more years with incredible uptime in Charlotte, NC.   

North Carolina Network Issues (Resolved) Critical

Affecting Other - Networking / distribution swiches

  • 05/08/2021 06:10
  • Last Updated 05/08/2021 09:43

Update 12:40PM EST:   Confirmed all network connectivity is restored.

Update 12:21PM EST:   Connectivity is mostly restored.   The remaining disconnected cabinets will be back online momentarily.

Update 12:00PM EST: We are currently transitioning all cabling to our backup distribution switches. Our initial ETA holds true and we should have all services restored by 2PM EST.

Update:  9:10am EST.   We are preparing to replace the distribution switches.   There are a few last ditch efforts being worked to determine if the existing cluster will come back to life.    The emergency maintenance window will extend through 2pm EST though we hope to have services restored before then.  

Update:  7:45am EST.   The distribution switch cluster appears to show multiple failed PSUs with red lights and otherwise totally dark.   We are working on a decision to either replace the entire redundant cluster with a backup distribution switch or if we can fix it in place.   ETA is currently unknown.

Update:  7:09am EST.   Confirmed issues with distribution switches.   We are working on a resolution.   Your continued patience is appreciated.


As of 6:20am EST on Saturday, May 8th, we are currently experiencing a datacenter-wide outage in North Carolina.   We suspect an issue with our distribution switches and are currently investigating but have no ETA.   Hopefully services will be restored shortly.   We appreciate your patience.

Charlotte, NC Service Impacting Maintenance 11/6/2020 1AM EDT (Resolved) Medium

Affecting Other - Segra - Charlotte, NC

  • 11/06/2020 01:00 - 11/06/2020 01:38
  • Last Updated 10/28/2020 16:55

Please note we will be performing service impacting maintenance that will affect services in the Charlotte, NC datacenter. We will be performing router firmware upgrades to address a security vulnerability that requires a reload to apply.

Services affected: Charlotte, NC datacenter
Date: November 6th, 2020
Start time: 1AM EDT (GMT -5)
End time: 2AM EDT (GMT-5)
Outage Duration: Approximately 10 minutes, though 20 minutes is being allocated

If you have any questions, please do not hesitate to reach out to us at my.iwebfusion.net

Network Issue (Resolved) Critical

Affecting Other - CenturyLink

  • 08/30/2020 05:07 - 08/30/2020 09:58
  • Last Updated 08/30/2020 05:09

At approximately 6:10AM EST, we started getting reports of inaccessible servers from clients in some specific regions around the world.   After a thorough investigation, we discovered that the issue is not related to our own networking.   Rather, it seems CenturyLink, a major North American ISP and telco provider has some sort of major issue.   (Link: https://downdetector.com/status/centurylink/) This is causing sporadic inaccessibility issues, packetloss, and latency for many routes to many locations in the USA.     As a direct transit provider, we have contacted them but they are seemingly overwhelmed by the widespread issues and have not been able to promptly reply with specific information.   Through our own connections, we have heard that they are expecting to have all issues sorted by 11:30am EST.    However, we have done our best to route around their networks as much as possible.   Since they are a major ISP in the USA, this proves impossible from some locations.  If your route to us is through CenturyLink, you will likely experience intermittent connectivity until this is fully resolved on their end.   Meanwhile, our network and all systems are 100% operational at this time.    We will continue to monitor this situation and hope that CenturyLink resolves their issues ASAP.    Please do not hesitate to contact us with any questions.

 

Bend, OR (Cascade Divide) Network Maintenance (Resolved) Medium

Affecting Other - Cascade Divide Datacenter

  • 08/12/2020 20:00 - 08/13/2020 11:09
  • Last Updated 08/10/2020 15:40

Beginning 8/12/2020 at 8PM PST we will be upgrading our core network infrastructure to better provide redundancy, speed, and to allow for larger future growth.

Services affected: Cascade Divide, Bend OR datacenter
Date: August 12th, 2020
Start time: 8PM PDT (GMT -8)
End time: 9:30PM PDT (GMT-8)
Outage Duration: Approximately 15 minutes, though 30 minutes is being allocated

Network Maintenance - Los Angeles Psychz (Resolved) Critical

Affecting Other - Distribution Switch

  • 07/09/2020 08:00 - 07/09/2020 10:08
  • Last Updated 07/07/2020 08:54

We will be replacing a distribution switch stack in order to accomodate a higher capacity and to prevent any possible saturation issues due to increased bandwidth consumption.

Details -
Date: Thursday, July 9th, 2020
Start Time: 8:00AM PDT
Work Window: 07/09/20 @ 8:00AM- 10:00AM ( Los Angeles time - (PDT) )
Outage Estimated Duration: 10-30 Minutes
Facility: Los Angeles, California

Charlotte, NC Network Issues (Resolved) Critical
  • 01/03/2020 22:25 - 01/04/2020 00:07
  • Last Updated 01/04/2020 00:07

Dear Customer,

We are currently seeing issues with some IP ranges in our Charlotte, NC location.   Network engineers are working on this issue right now.  There is no ETA but we are hoping the issue will be resolved any minute.    This issue appears to affect only certain IPs/ranges. 

UPDATE (1:37am):   Gateways are pinging and we are expecting routing issues to be solved any minute.

UPDATE (2:28am):  Partial service restoration - we expect full resolution shortly.

UPDATE (2:49am):   Most services restored.   We are learning that the cause of this was poorly informed network maintenance upstream.   We are working with the provider to resolve this issue.  

 

UPDATE (2:50am):   Incident fully resolved.   The issue was indeed a result of PLANNED network maintenance by our provider, who did not properly communicate ahead of time.   We are taking action to prevent this from occurring again in the future.   Our apologies for the problems this caused.   We will hold our upstream provider accountable.      Please contact us for any more details or for SLA requests.

 

 

 

 

Network Maintenance: Psychz LA, October 9 (Resolved) Medium

Affecting Other - Psychz Los Angeles

  • 10/09/2019 14:00 - 12/31/2019 12:34
  • Last Updated 10/07/2019 12:32

Thank you for your continued business! This is a maintenance notification for the Psychz LA location, during which routing firmware will be upgraded and a potentially faulty linecard will be replaced

Details:
Facility: Psychz, Los Angeles, CA
Date: Wednesday, October 9, 2019
Start Time: 2:00PM PDT
Work Window: 2PM PDT - 4PM PDT (-7 Hours GMT)
Outage Estimated Duration: 5-10 minutes
Effect: Brief Outage

If you have any questions, please do not hesitate to contact us.

Emergency Maintenance 5PM PDT Bend, OR Datace (Resolved) Medium

Affecting Other - Cascade Divide Datacenter

  • 09/11/2019 17:00 - 09/11/2019 20:40
  • Last Updated 09/11/2019 12:35

Dear valued clients,

Thank you for your continued business! This is an emergency service impacting maintenance notification:

Services affected: Cascade Divide, Bend OR datacenter
Date: September 11th, 2019
Start time: 5PM PDT (GMT -8)
Duration: Approximately 15 minutes, though 30 minutes is being allocated
Scope: Replacing a faulty line card that appears to have unexpectedly rebooted this morning, out of an abundance of caution

We appreciate your cooperation and continued business.

June 25th 2019 Segra Datacenters Maintenance (Resolved) Medium

Affecting Other - Segra Datacenters (DC74, Charlotte NC)

  • 06/26/2019 00:01 - 07/17/2019 08:26
  • Last Updated 06/18/2019 17:03

Dear Valued Customer,

The SEGRA Data Center network team has been planning network upgrades at our Charlotte, NC data centers -  CLT1, CLT2 and CLT4 for the past few months.  During each night of the maintenance window, we will be performing router reboots to complete these upgrades.  Each reboot is only expected to last 5-10 minutes. 

These reboots will allow us to provide additional services at the data center and we do not anticipate any extended period of outages beyond the standard reboot time of each router themselves. 

If you have any questions or concerns, please contact H4Y Customer Care by ticket at my.iwfhosting.net and we will be happy to assist you further.

Los Angeles (Psychz) Power Issues (Resolved) Critical

Affecting Other - Los Angeles Psychz

  • 06/12/2019 15:47 - 06/18/2019 17:00
  • Last Updated 06/12/2019 16:13

Dear Valued Customers,

Over the course of the past week, we have faced 3 power outages (including today) as a result of a faulty PDU unit at the datacenter.   Attempted repairs to said PDU at the facility failed and replacement  unit is on order.  The replacement ETA is 1-2 weeks.   Meanwhile, the existing unit has been bypassed so it cannot malfunction again.   Servers that are fed by A/B redundant circuits have not been affected.   Today, at approximately 1:21PM PST, the Los Angeles (Psychz) Datacenter lost utility power entirely.   This is a common Los Angeles phenomenon during the hot summer months and typically occurs,  without any sign of disruption or service impact, on average once or twice over the course of a year.   Upon the loss of utility power today,  one of two diesel backup generators was activated at the facility in Los Angeles and failover occurred within 30 sec, but it still caused downtime for all branch circuits up the line.   Additional downtime may have been observed as individual machines, switches, and routers boot back up.   Those with A/B redundancy were unaffected as the healthy battery backups took over.   We are working with the datacenter to ensure that full resolution to this issue occurs immediately and does not cause any more power interruptions.   As of 3:40pm PST, the utility power has still not been restored and generator power is in use.   The generator is capable of running indefinitely as fuel is available.    We will provide updates at https://my.iwfhosting.net/serverstatus.php  as they become available.

Thank you for your patience and understanding!

DDOS - Cascade Oregon (Resolved) Critical

Affecting Other - Network issues - Oregon

  • 05/24/2019 08:31 - 05/26/2019 00:00
  • Last Updated 05/26/2019 08:35

Bend, OR datacenter: 

We have detected and mitigated or nulled several DDOS attacks to IPs in our Oregon datacenter.   These attacks began Friday, May 24th 2019 and continued until early Sunday morning, May 26th.   iWF has taken action to make sure that future attacks will not affect our network and we working with upstream providers on the same.   We are hoping that there will be no further negative effects.    Clients may have noticed 3-4 short periods of packetloss that ranged from minor to severe.   Each incident has been promptly mitigated within 10 minutes or less and we are in the process of confirming that the risk of further impact has been 100% eliminated.

Peer1 LA Network Outage 3/12/2019 (Resolved) Critical

Affecting System - Peer1 Los Angeles

  • 03/12/2019 14:49 - 03/12/2019 21:06
  • Last Updated 03/12/2019 16:22

Beginning 3/12/2019 at approximately 07:45AM PST (GMT -8) our Peer1 Los Angeles datacenter began to experience various network interruptions intermittently, with services flapping.

At approximately 14:49 PST (GMT -8) the Peer1 network started to suffer a prolonged outage. The outage is still under investigation at this time, however it is believed to be sourced from an aggregation switch that may have become faulty.  A previous distribution switch was already replaced, as part of the diagnosis. The Peer1 Los Angeles datacenter team has all resources working on the incident, however there is no expected ETA at this time for complete resolution. We will be posting updates as we receive them at this status page, and you may also monitor https://status.cogecopeer1.com/ directly if desired.

At approximate 16:14PM PST (GMT -8) services were restored. An aggregation switch was determined to be faulty, as well as the previously replaced distribution switch. JTAC has been involved to determine the root cause of both switches failing. We will continue to monitor the situation and provide a complete RFO as soon as a root cause analysis is available.

Once the incident has been resolved, clients can open a ticket in the billing department for SLA credit as applicable.

Thank you for your continued patience and understanding.

NC-Gemini Network Outage (Resolved) Critical

Affecting Server - NC-Gemini

  • 12/03/2018 13:24 - 12/03/2018 13:49
  • Last Updated 12/03/2018 13:49

Dear valued customers,

Beginning 12/3/2018 at 13:24 PDT (GMT-8) the server NC-Gemini became under an extremely large DDOS attack. Please note that this attack is only against the IP 192.158.238.133, any customers that have a dedicated IP or are not on said IP should experience no service interruption.

We are actively working on mitigating the attack, and once safe will enable the IP 192.158.238.133 again.

If you have any questions, please do note hesitate to get in contact with us.

Thank you again for your continued business

 

UPDATE: 12/3/2018 at 13:48 PDT -- at this time the issue has been resolved. If any further assistance is required, please do not hesitate to contact us.

Emergency Maintenance: PDU Replacement (Resolved) Critical

Affecting System - North Carolina Rack 105 PDU 2

  • 01/19/2018 18:00 - 01/19/2018 16:16
  • Last Updated 01/19/2018 13:16

Dear Valued Clients,

During a routine walkthrough we noticed that PDU 2 of Rack 105 in our Lumos Datacenters (DC74), Charlotte NC facility appeared to be malfunctioning. We have scheduled an emergency maintenance beginning at 1/19/2017 at 6PM EST. At this time we will shut down servers (gracefully) that are sourced from a single power supply on PDU 2 beginning at 6PM EST. Dual power supply servers and servers on PDU 1 will not be effected. We will attempt to have all servers up as soon as possible, we estimate the downtime to be approximately 30 minutes or less.

We sincerely apologize for the inconvenience, if you have any questions please do not hesitate to contact us.

Psychz LA - Cooling system issue (Resolved) Critical

Affecting Other - Los Angeles Psychz ONLY

  • 01/09/2018 06:43 - 01/09/2018 08:03
  • Last Updated 01/09/2018 08:53

URGENT (Los Angeles Psychz ONLY) - Cooling system issue 1/9/2018

**RESOLVED**

A problem with the A/C / cooling system at the Los Angeles (Psychz) datacenter has been discovered and some equipment has been taken offline while the issue is resolved.   We do not have an exact ETA for resolution but we will post updates as we have them.   We have a great history of uptime and strive for 100%.   Billing credits will be available per our SLA and we sincerely apologize for any problems this is causing you.

UPDATE:

-----------------------------

At approximately 5am PST, an overheat situation developed on a faulty A/C unit powering core routers.   It did not failover properly and caused automated shut downs of several systems and damage to fiber optic cabling.  Cabinet Z7 in Los Angeles Psychz was first affected by loss of connectivity and then other cabinets were affected as well.   As of 7:15am PST, We've replaced the faulty equipment and cabling as needed and ensured proper cooling.   BGP routes may continue to converge and cause some packetloss as operations return to normal.

Update:

----------------------------

Service is restored and servers are back online now. Sorry for any inconvenience this has caused.

Emergency Maintenance 7/19/17 1AM EST (Resolved) Medium

Affecting Server - NC-Uniform

  • 07/18/2017 22:00 - 07/18/2017 22:43
  • Last Updated 07/18/2017 11:29

Dear valued clients,

Thank you for your continued business!

Please be advised we will be performing service impacting emergency maintenance tonight at 10PM PST / 1AM EST which will affect the following server you are on: NC-Uniform

We expect this maintenance to last no longer then one hour. During this time, we will be replacing a failing RAID card.

If you have any questions, please do not hesitate to contact us.

USSHC Network Outage (Resolved) Critical

Affecting System - USSHC

  • 07/10/2017 22:03 - 07/11/2017 00:44
  • Last Updated 07/11/2017 00:55

On 7/10/2017 at 22:41 PDT the NOC was alerted to a network-wide issue effecting the USSHC datacenter. Immediately on-scene staff were alerted.

At this time all services have been restored.

All of our redundant transit providers had unexpected outages concurrently, a phenomenon we have never seen before. This affected the Iowa datacenter and beyond. It was a regional outage affecting most of Iowa and parts of Illinois. We will post more details later today but for now HE.net has recovered and we are routing all traffic through it. We will add CenturyLink/others back to the BGP mix once it is available. We sincerely apologize for any problems this caused you.

Service Affecting Maintenance Sat, November 5 (Resolved) Medium

Affecting System - Cascade Divide

  • 11/05/2016 18:00
  • Last Updated 11/06/2016 08:18

UPDATE 7:34AM PST - All servers up.   If you are still experiencing any issues.   Please file a ticket.   Welcome to Bend, OR!

UPDATE 4:32AM PST - 50% of servers are networked, cabled, and powered up.   We continue to work on the remainder.   All networking and routing has been tested and given the OK.  

UPDATE 1:33AM PST - All equipment has arrived at the Bend facility.   Staff is unpacking and racking the equipment into our cabinets as planned.  Switches will be powered up first followed by servers. 

UPDATE 10:03PM PST - IP routing has been tested and confirmed except for a few stragglers.   Equipment is still enroute and more updates will follow shortly.

UPDATE 7:38PM PST - All servers have been cleanly shut down.   We are now safely transporting them to the Bend facility.   So far so good!   Please check back for updates.


Dear Valued Clients,

This notice is to inform you that we will be relocating equipment at our Cascade Divide Roseburg, OR location to the Cascade Divide Bend, OR facility overnight Saturday, November 5th at 6pm PST. This relocation is necessary due to our requirements for a larger and more redundant facility. The new facility features more transit providers, additional redundancy, more space, and larger capacity in general. This relocation will indeed cause a service interruption on the night of November 5th, but it should be as minimal as humanly possible. There will be NO IP space changes, rack-level networking changes, and the new facility is in the same geographic region and state. We chose the most off-peak time while allowing for unforeseen conditions to complete the maintenance with plenty of time to spare before the following morning. Your data, IPs, and configurations are NOT at risk. There will be no functional difference to your service, though you can look forward to enhanced redundancy and reliability, plus pricing benefits for bandwidth and much more in the future. Updates as we go will be posted at https://my.iwfhosting.net/serverstatus.php 

Details:

If you have a dedicated or colo server, we suggest shutting it down (halting) it before November 5th at 6PM PST. If that is not possible, our staff will also begin powering down equipment at that time. We will attempt to shut down all servers cleanly in all cases (using CTRL+ALT+DEL where possible, or by logging in and halting). We will also cleanly shut down all VPS accounts and shared servers. Servers will all be booted up and checked once they arrive at the Bend, OR facility.

The physical relocation will begin by 7pm PST. We have at least one staff member assisting PER CABINET so the equipment will be loaded and relocated as quickly as possible. We expect that equipment will be at the Bend facility and powering up within 4 hours. The maintenance period will extend to 3AM PST on November 6th to account for any unforeseen issues. Please check our network status page for updates and be aware that we will be busy with phonecalls and support requests the entire night. If possible, refrain from contacting us for status updates until we announce that servers are racked/cabled and should be powered back up.

Your business is appreciated! Please contact us if you have any questions, concerns, or special instructions for us during this relocation. Our goal is to ensure clients waking up Sunday morning will simply return with ease to business as usual. We will post status updates as they become available on the night of November 5th. We will also post the Bend, OR datasheet and info to our site within the coming days. Thank you for choosing us!

**EMERGENCY** Scheduled Network Maintenance (Resolved) Medium

Affecting System - Peer1 LA

  • 07/22/2016 00:00 - 08/09/2016 11:11
  • Last Updated 07/21/2016 18:56

Dear Customer,

Please be advised that we will be performing scheduled network maintenance in our Los Angeles (West 7th Street) facility during the following date and time:

From: July 22, 2016 - 00:00 PDT (July 22, 07:00 UTC)

To:   July 22, 2016 - 02:00 PDT (July 22, 09:00 UTC)

The window will occur on Friday July 22nd from 00:00 - 02:00. During this timeframe, Network Engineers will reboot a virtual switch chassis, due to the nature of the maintenance downtime of around 10 minutes is expected during this window and services will be affected.

This work will be SERVICE IMPACTING. The appropriate staff will be present for the entire duration of this maintenance window.

Cascade Divide Datacenter Outage 7/18 (Resolved) Critical

Affecting Other - Cascade Divide Datacenter

  • 07/18/2016 15:01 - 07/18/2016 20:38
  • Last Updated 07/18/2016 15:52

07/18/2016 15:01 PST: We are currently investigating an outage in our Cascade Divide, Roseburg OR datacenter. Datacenter staff is aware of the outage. Updates to follow.

7/18/2016 15:49 PST: We have been advised by the datacenter that their primary router has failed, and failover to the backup router failed. A manual failover is currently taking place.

Cascade Divide Datacenter Outage (Resolved) Critical

Affecting System - Cascade Divide Datacenter

  • 04/13/2016 10:57 - 04/18/2016 10:16
  • Last Updated 04/13/2016 17:49

At 10:57 AM PST (GMT-8) our NOC team noticed an outage at our Cascade Divide, Roseburg OR datacenter. The NOC made contact with the datacenter and they are aware of the issue, and investigating.

UPDATE: 11:12 AM PST -- There is a confirmed fiber cut which is affecting all circuits (including redundant and protected circuits) in and out of the datacenter.

UPDATE: 11:23 AM PST -- Response teams are en route from all major fiber providers to the facility (LSN, HE, Level3, CenturyLink, etc). We do not yet have any ETA.

UPDATE: 12:04 PM PST -- Teams are on site from all carriers and repairs are underway. Substantial damage has been done to the multiple fiber paths and utility poles in the area, including full fiber sheath cuts. Crews are giving a rough estimate of 4-5 hours.

UPDATE: 1:55PM PST -- Crews have updated their original estimate. The new expected resolution time is 7:00PM PST. Our on-scene technicians have shared pictures, available at: http://imgur.com/a/023V2

UPDATE: 4:06PM PST -- Aerial lines have been pulled across roadways, and work is underway on terminating and splicing the fiber bundles. The ETA remains the same.

UPDATE: 5:47PM PST -- Service has been restored to the first fiber bundle. At this time if you have any remaining issues please open a ticket so we can address them.

SERVICE IMPACTING Maintenance: 9/24/15 9PM-11 (Resolved) Medium

Affecting System - Peer1 LA

  • 09/24/2015 21:00 - 11/17/2015 14:06
  • Last Updated 09/16/2015 13:04

Dear valued clients,

Thank you for your continued business! This is a maintenance notification regarding our Los Angeles (Peer1) location.

On Thursday, September 24th starting at 9PM PST (Pacific Standard Time), we will be relocating some of our equipment in the Los Angeles datacenter into new cabinet space that we already setup only a few yards away.

The goal of this maintenance period is to provide you a more reliable infrastructure. Brand new switches and PDUs are already installed, and ready to go!

SHARED CUSTOMERS:

This is a notification for the following servers:
CA-Alpha
CA-Bravo
CA-Foxtrot
CA-Kilo
CA-Lima
CA-Zeta

DEDICATED CUSTOMERS:

The new switches will increase our capacity and port limits for all clients. Most dedicated clients will receive free port upgrades.

We will take incredible care in cleanly shutting down all machines where possible (including self-managed servers where possible by using CTRL+ALT+DEL). If you have a Windows self-managed server or otherwise would like to ensure the shutdown is clean, please take it down before 9AM PST. We will ensure that all dedicated servers are powered back up and are online once the migration is complete.

We anticipate this maintenance window to last 1.5-2 hours.

Please let us know how we can make this as painless as possible for you and your clients. We will be available for any questions you have before, during, and after the move.

DC74 Datacenter Maintenance April 11th (Resolved) Medium

Affecting Other - DC74 Datacenter

  • 04/11/2015 23:00 - 09/16/2015 13:04
  • Last Updated 04/09/2015 18:28

Thank you for your continued business!

Please be advised of the following service impacting maintenance notification.

Reason for Notification: Router Maintenance
Location: DC74 Data Centers – CLT4 facility, 1612 Cross Beam Drive, Charlotte NC
Start Date & Time: Saturday, April 11th, 2015 starting at 2300 EST


Expected End Date & Time: Saturday, April 11th, 2015 ending at 2330 EST

PLEASE NOTE: This maintenance does not affect shared, reseller, and VPS clients as you are in a seperate datacenter. This affects DC74 datacenter ONLY.

Description: We will be performing router maintenance potentially requiring a reboot of the routers. This will impact BGP routing and could potentially cause up to a 15-30 minute loss of network connectivity. The date and time chosen is the lowest traffic routing period for the entirety of the datacenter. Please be aware of this maintenance window and potential service disruption.

Please let me us know if you have any questions.

DDOS attack: IA-Echo (Resolved) Critical
  • 01/07/2015 12:04 - 01/07/2015 14:04
  • Last Updated 01/07/2015 14:04

At approximately 12:04PM PST (GMT-8) 1/7/2015 network engineers discoverd a DDOS attack against the shared hosting server: IA-Echo. The attack is currently being actively mitigated however services on IA-Echo are partially degraded.

We apologize for the incovenience, and thank you for your patience.

UPDATE:01/07/2015 14:04 the DDOS has been sucessfully mitigated.

12/3/14 Cascade Divide Final Network Upgrade (Resolved) Medium

Affecting Other - Cascade Divide datacenter

  • 12/03/2014 23:30 - 01/07/2015 13:19
  • Last Updated 11/30/2014 20:37

Please note that we will be performing the final upgrade to our networking infrastructure at our Cascade Divide, Roseburg OR location starting December 3rd, 2014 at 11:30PM PST (GMT-8) and lasting until 12:00AM.

You may experience a service disruption of approximately five minutes or less during this window.

Thank you for your continued business.

[SERVICE IMPACTING MAINTENANCE] Sat. 11/29/20 (Resolved) Critical

Affecting System - DC74 Datacenter

  • 11/29/2014 22:00 - 11/30/2014 20:33
  • Last Updated 11/24/2014 15:53

Thank you for your continued business! This is a service impacting maintenance notification scheduled for Saturday, 10/29/2014 beginning at 10PM EST (GMT-5) for our DC74, Charlotte NC facility.

At this time we will be upgrading our core switching infrastructure. Clients will be going down one by one as you are moved to the new switch stack. We expect the maintenance window to be complete by 11PM EST (GMT-5). There is an expected 5-10 minutes of downtime per machine.

If you have any questions, please do not hesitate to contact us.

SERVICE IMPACTING Maintenance notification fo (Resolved) Medium

Affecting System - Cascade Divide, Roseburg OR

  • 09/06/2014 23:45 - 10/05/2014 16:06
  • Last Updated 08/22/2014 17:49

Thank you for your continued business! This is a SERVICE IMPACTING maintenance notification. This notification is for our Cascade Divide, Roseburg OR location.

We will be performing switch upgrades in order to provide a more robust and redundant switching infrastructure.

This upgrade is schedule to take place on: 9/6/2014 from 23:45 (11:45 PM) until 9/7/2014 00:15 (12:15 AM) GMT-8 (PST)

Should you have any questions please do not hesitate to contact us regarding this issue.

Maintenance Notification 5/7/14 18:00 PST: IA (Resolved) Low

Affecting Server - CA-Kilo

  • 05/07/2014 18:00 - 05/17/2014 16:46
  • Last Updated 05/05/2014 20:25

Dear Valued Clients,

This is a maintenance notification for the following server: IA-Charlie. Beginning at 05/07/2014 18:00 we will be performing service impacting maintenance which may last up to thirty minutes.

This maintenance is necessary to replace a BBU for the RAID array which powers this server.

Updates will be posted at https://www.iwfhosting.net/forums/viewtopic.php?f=11&t=652 as well as via our twitter, at https://twitter.com/iwfhosting 

If you have any questions, please do not hesitate to contact us via our helpdesk at http://billing.iwebfusion.net 

Thank you for your continued business

Maintenance Notification 5/7/14 18:00 PST: IA (Resolved) Low
  • 05/07/2014 18:00 - 05/17/2014 16:46
  • Last Updated 05/05/2014 20:25

Dear valued clients,

This is a maintenance notification for the following server: IA-Echo. Beginning at 05/07/2014 18:00 we will be performing service impacting maintenance which may last up to thirty minutes.

This maintenance is necessary to replace a BBU for the RAID array which powers this server.

Updates will be posted at: https://www.iwfhosting.net/forums/viewtopic.php?f=11&t=651 

If you have any questions, please do not hesitate to contact us via our helpdesk at http://billing.iwebfusion.net

Thank you for your continued business

Server Maintenance: Pacific (Resolved) Low

Affecting Server - CA-Pacific

  • 04/12/2014 20:00 - 05/05/2014 14:06
  • Last Updated 04/11/2014 13:05

Dear Valued Clients,

Thank you for your continued business! This is a service notification for scheduled maintenance which will be performed on the server "Pacific". This maintenance is schedule to begin on 04/12/2014 20:00 PST and has the potential to last until 04/12/2014 22:00 though expected downtime is far less.

This server is due to be upgraded and clients will enjoy new blazing fast disk performance, as we will be upgrading this server to SSD drives. This upgrade will be free of cost to you.


Once again, thank you for choosing iWF Hosting / H4Y Technologies LLC. If you have any questions please do not hesitate to contact us via our helpdesk at http://billing.iwebfusion.net or by sending us an email at [email protected]

Scranton/Dunmore Scheduled Maintenance Notifi (Resolved) Low

Affecting Other - Scranton/Dunmore

  • 03/30/2014 18:00 - 04/08/2014 12:40
  • Last Updated 03/21/2014 12:31

Dear Client,
This notice is for our Scranton/Dunmore clients only.   THIS DOES NOT AFFECT ANYONE ELSE.   Due to a number of factors, we have determined that it is necessary to upgrade clients to a new datacenter.   There are a number of significant advantages:

*Tier 3 Data Center in North Carolina (dc74)
*More Redundant Network with 3X the carriers currently provided
*Larger bandwidth capacity
*Type II audited facility

This decision was made as an alternative to keeping servers elsewhere in Pennsylvania mainly because the IP space will remain the same.   YOUR EXISTING IPS WILL KEEP WORKING.   Our equipment will be shut down around 9pm EST Sunday, March 30th and should come back online without any complications,  around 9am EST Monday morning.  Equipment is being moved cabinet by cabinet so the job is not as extensive as it would be if individaul servers were being moved.  

We apologize for the problems this disruption will cause but we hope most of our clients will be sleeping soundly while we make the move.  We will have tech support available during the entire operation to keep everyone updated.   The advantages will be plentiful!   Your business is appreciated.   Please open a ticket if you have any questions.

[RESCHEDULE] Scheduled Network Maintenance No (Resolved) Low

Affecting System - Peer1 Los Angeles

  • 03/22/2014 00:01 - 04/08/2014 12:40
  • Last Updated 03/10/2014 18:52

Dear Valued Customers,

Thank you for your continued business.

Please be advised that we will be performing scheduled network maintenance during the following window:

Saturday, March 22, 2014 from 00:01 PT to 05:00 PT (Mar. 22, 07:01-12:00  GMT)

This maintenance is required to migrate customers from an aging switching segment to our new Juniper switch stack and provide higher capacity. This is a reschedule of the network maintenance sent last 22-Feb. Due to an unexpected emergency in the Los Angeles data center, adequate resources were not available to complete the previous maintenance window.

During this maintenance, customers will notice up to 15 minutes of downtime while cabling and configurations are being performed. Customers will be moved one at a time and connectivity will be verified both before and after migration. 

If you have any questions please do not hesitate to contact us.

It is always a pleasure serving you.
iWFHosting.net
H4Y Technologies LLC
The worldwide leader in quality hosting.
866-435-5642 
FACEBOOK: www.facebook.com/h4ytec 
TWITTER: www.twitter.com/iwfhosting 

Scheduled maintenance on server: Alpha 3/2/20 (Resolved) Low

Affecting Server - CA-Alpha

  • 03/02/2014 18:00 - 03/03/2014 10:16
  • Last Updated 03/02/2014 18:13

Dear Valued Clients,

Thank you for your continued business. In an effort to better serve you and to ensure the highest service standards we will be performing regular maintenance on the server Alpha, which you have an account on.

This maintenance process will upgrade the operating system from CentOS 5 to CentOS 6 bringing with it a wealth of updates and stability patches.

Please note this maintenance window will be service impacting. Scheduled maintenance time:

3/2/2014 6:00PM PST (GMT-8) until 3/2/2014 9:00PM PST (GMT-8)

While we have scheduled a three hour window, we estimate downtime will be far less.

If you have any questions please do not hesitate to get in contact with us.

Any updates will be posted at http://www.iwfhosting.net/forums/viewtopic.php?f=11&t=646&p=1014, as well as via our twitter account, https://twitter.com/iwfhosting

Thank you for your continued business.

Scheduled maintenance on servers: Zeta, Alpha (Resolved) Critical

Affecting System - Peer1 LA: Zeta, Alpha, iWF/H4Y sites

  • 02/10/2014 19:00 - 02/10/2014 19:21
  • Last Updated 02/10/2014 19:21

Dear Valued Clients,

Thank you for your continued business!

We wanted to alert you to the fact there there will be scheduled server maintenance on the Peer1 LA network which will affect the following servers: Alpha, Zeta, and the main iWF / H4Y websites.

This maintenance window will begin at 7:00 PST (GMT-8) on 2/10/2014 and last until 7:30M PST (GMT-8) 2/10/2014. This maintenance will be service impacting, though expected downtime will be less than ten(10) minutes.

If anything is to go wrong, updates will be posted at: http://www.twitter.com/iwfhosting

If you have any questions, please do not hesitate to get in contact with us.

Once again, thank you for choosing iWF Hosting.

Scheduled maintenance on servers: Zeta, Alpha (Resolved) Critical

Affecting Server - CA-Alpha

  • 02/10/2014 19:00 - 02/10/2014 19:20
  • Last Updated 02/10/2014 19:20

Dear Valued Clients,

Thank you for your continued business!

We wanted to alert you to the fact there there will be scheduled server maintenance on the Peer1 LA network which will affect the following servers: Alpha, Zeta, and the main iWF / H4Y websites.

This maintenance window will begin at 7:00 PST (GMT-8) on 2/10/2014 and last until 7:30M PST (GMT-8) 2/10/2014. This maintenance will be service impacting, though expected downtime will be less than ten(10) minutes.

If anything is to go wrong, updates will be posted at: http://www.twitter.com/iwfhosting

If you have any questions, please do not hesitate to get in contact with us.

Once again, thank you for choosing iWF Hosting.

Delta Outage (Resolved) Critical
  • 01/28/2014 07:54 - 01/28/2014 10:59
  • Last Updated 01/28/2014 09:21

At this time there is a large DDOS attack which is affecting the following nodes:

Delta
Pacific

We are investigating and actively mitigating the threat.

We thank you for your patience.

Updates will be provided at: http://www.iwfhosting.net/forums/viewtopic.php?f=11&t=643 

Pacific Outage (Resolved) Critical

Affecting Server - CA-Pacific

  • 01/28/2014 07:54 - 01/28/2014 10:59
  • Last Updated 01/28/2014 09:20

At this time there is a large DDOS attack which is affecting the following nodes:

Delta
Pacific

We are investigating and actively mitigating the threat.

We thank you for your patience.

Updates will be provided at: http://www.iwfhosting.net/forums/viewtopic.php?f=11&t=643 

Scheduled maintenance on server: Hotel 1/21/2 (Resolved) Low

Affecting Server - IA-Hotel

  • 01/21/2014 21:30 - 01/23/2014 14:12
  • Last Updated 01/21/2014 20:04

Dear Valued Clients,

Thank you for your continued business!

We wanted to alert you to the fact there there will be scheduled server maintenance on the server: Hotel. This maintenance will upgrade MySQL to resolve some stability issues you may have been experiencing recently.

This maintenance window will begin at 9:30PM PST (GMT-8) on 1/21/2014 and last until 10:00PM PST (GMT-8) 1/21/2014. This maintenance will be service impacting, though expected downtime will be less than fifteen (15) minutes.

If you have any questions, please do not hesitate to get in contact with us.

Once again, thank you for choosing iWF Hosting.

Cloud SAN Failure (Resolved) Critical

Affecting System - Cloud SAN

  • 01/16/2014 17:28 - 01/17/2014 04:12
  • Last Updated 01/17/2014 19:13

At 01/16/2014 17:28 PST (GMT-8) our engineers noticed a failure in the SAN which hosts our cloud infrastructure. We are investigating now. Updates will be provided at: http://www.iwfhosting.net/forums/viewtopic.php?f=11&t=640

SRV6 SAN Failure (Resolved) Critical
  • 01/16/2014 17:28 - 01/17/2014 04:12
  • Last Updated 01/17/2014 19:13

At 01/16/2014 17:28 PST (GMT-8) our engineers noticed a failure in the SAN. We are investigating now.

Updates will be posted at: http://www.iwfhosting.net/forums/viewtopic.php?f=11&t=639

Scheduled maintenance on server: Alpha (Resolved) Low

Affecting Server - CA-Alpha

  • 11/13/2013 21:00 - 12/23/2013 00:56
  • Last Updated 11/13/2013 23:13

Beginning 11/13/2013 21:00 PST (GMT-8) lasting until 11/13/2013 21:30 PST (GMT-8) there will be an upgrade taking place on the following server(s):
Alpha

While this upgrade should not be service impacting, there is always a potential for brief moments of downtime. The upgrade will last at most for half an hour.

If you have any questions at all, please do not hesitate to let us know.

Thank you for your continued business.

Scheduled maintenance on server: Hotel (Resolved) Low

Affecting Server - IA-Hotel

  • 11/13/2013 20:30 - 12/23/2013 00:56
  • Last Updated 11/13/2013 16:24

Beginning 11/13/2013 20:30 PST (GMT-8) lasting until 11/13/2013 21:00 PST (GMT-8) there will be an upgrade taking place on the following server(s):
Hotel

While this upgrade should not be service impacting, there is always a potential for brief moments of downtime. The upgrade will last at most for half an hour.

If you have any questions at all, please do not hesitate to let us know.

Thank you for your continued business.

HostNOC Datacenter Move (Resolved) Low

Affecting Other - Datacenter

  • 07/18/2013 08:00 - 10/24/2013 18:30
  • Last Updated 07/15/2013 12:20

Moving on up! Thursday, July 18th 2013 we will be transporting client dedicated servers and colo servers currently housed in Scranton, PA to the new HostNOC datacenter a few miles away in Dunmore, PA. Downtime should be minimal and the advantages are huge! Many of you will see free gigabit port upgrades. The new location is impressive, extremely redundant, and with great connectivity across the US and overseas. New bandwidth providers, state of the art infastructure, upgraded security and network, new network gear wholly owned by H4Y and it all comes standard with the great service and staff you have grown to love over the years 

New pictures and DC specs to come!

USSHC Planned outage for network upgrades and (Resolved) Low

Affecting Other - USSHC network

  • 08/09/2009 00:00 - 09/05/2009 15:04
  • Last Updated 08/07/2009 13:47

On Monday August 10th we will begin preparing for network improvements, systems testing, and network failover testing.  The testing will take place Monday night/Tuesday morning with planned outages starting Tuesday at 12:00 AM - 2:00 AM CST.  We do not anticipate the outages to take anywhere near the full 2 hours.  We expect to have perhaps 2-3 smaller outages from 5-15 minutes each barring any unforeseen circumstances.

 

We are going forward with this outage as a way to avoid unplanned outages in the future after reviewing our systems and infrastructure.

URGENT - Scheduled Maintenance - Platinum Net (Resolved) Medium

Affecting Other - Peer1 Platinum network

  • 03/15/2009 23:30 - 09/05/2009 15:04
  • Last Updated 03/10/2009 19:36

Dear Customer,

Please be advised that we will be performing a scheduled network
maintenance during the following date and time:

Sunday, March 15th, 2009 at 23:30PM PT to Monday, March 16th, 2009 at
01:00AM PT.

This maintenance is necessary to upgrade the IOS on a core router
located in our Los Angeles 600 West 7th Street Facility. This will affect ONLY Platinum Network customers.

This maintenance will be SERVICE IMPACTING and during this maintenance
window, you will see brief periods of latency and packet loss as traffic
is routed around the affected links. The duration of this maintenance
window will be 90 minutes.

If you have any questions, please reply to this email.

URGENT: Peer 1 LA (Platinum Network) Maintena (Resolved) Medium

Affecting Other - Peer1 Platinum network

  • 02/21/2009 00:00 - 09/05/2009 15:04
  • Last Updated 02/16/2009 21:18

Dear Customer,

Please be advised that we will be performing scheduled network
maintenance during the following date and time:

Saturday, February 21st, 2009 between 00:01AM PT and 01:00AM PT.

This maintenance is necessary to upgrade the IOS on a core router
located in our Los Angeles 600 West Street Facility.

This maintenance will be SERVICE IMPACTING and during this maintenance
window, you will see brief periods of latency and packet loss as traffic
is routed around the affected links. The duration of this maintenance
window will be 1 hour.

If you have any questions or concerns please contact iWebFusion support (see contact information below).

-iWebFusion Technologies LLC

ph: 866-216-2926
e: [email protected]

DDOS attack (Resolved) Critical

Affecting Other - All VPS/Reseller/Shared customers

  • 01/07/2009 23:37 - 01/07/2009 23:29
  • Last Updated 01/07/2009 23:37

Dear iWebFusion VPS, Shared, and Reseller customers.

You were recently affected by a DDOS attack. This attack originated from a bug in the DirectAdmin control panel. This caused an estimated 29 minutes downtime. We apologize for any invonvenience this may have caused you. If you are a VPS customer we suggest you disable roundcube webmail until the patch can be fixed with this, which is what the exploit originated from.

Thanks for choosing iWebFusion Technologies LLC