<?xml version="1.0" encoding="UTF-8"?>
<feed xml:lang="en-US" xmlns="http://www.w3.org/2005/Atom">
  <id>tag:status.oit.co,2005:/history</id>
  <link rel="alternate" type="text/html" href="https://status.oit.co"/>
  <link rel="self" type="application/atom+xml" href="https://status.oit.co/history.atom"/>
  <title>UCaaS Status Page Status - Incident history</title>
  <updated>2026-03-27T06:00:00.000+00:00</updated>
  <author>
    <name>UCaaS Status Page</name>
  </author>
  
<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmn7x913r03h2rj42g5uawuud</id>
  <published>2026-03-27T06:00:00.000+00:00</published>
  <updated>2026-03-27T06:00:00.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmn7x913r03h2rj42g5uawuud"/>
  <title>Urgent Core1-ATL Security Updates 3/27/26</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 2 hours</p>
    <p><strong>Affected Components:</strong> Manager Portal, SNAPmobile Web, API</p>
    <p><small>Mar <var data-var='date'> 27</var>, <var data-var='time'>06:00:00</var> GMT+0</small><br /><strong>Identified</strong> -
  We will be performing urgent security updates on Core1-ATL on **March 27th, 2026 at 2:00 am ET**. There is no expected interruption of service during this time, but users may be logged out of the Manager Portal and SNAPmobile Web during the maintenance window.

We appreciate your patience throughout this process. If you need additional support, please contact [support@oit.co](mailto:support@oit.co)..</p>
<p><small>Mar <var data-var='date'> 27</var>, <var data-var='time'>06:00:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Mar <var data-var='date'> 27</var>, <var data-var='time'>08:00:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmmxli2u900w55diqp6hfj32l</id>
  <published>2026-03-27T03:00:00.000+00:00</published>
  <updated>2026-03-27T03:00:01.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmmxli2u900w55diqp6hfj32l"/>
  <title>Fax Server Maintenance 3/26/26</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 4 hours</p>
    <p><strong>Affected Components:</strong> Native Fax</p>
    <p><small>Mar <var data-var='date'> 27</var>, <var data-var='time'>03:00:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Mar <var data-var='date'> 27</var>, <var data-var='time'>07:00:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>
<p><small>Mar <var data-var='date'> 27</var>, <var data-var='time'>03:00:00</var> GMT+0</small><br /><strong>Identified</strong> -
  We will be performing security maintenance on our fax server on **March 26th, 2026, at 11:00 pm ET**. There will be an interruption in the delivery and sending of faxes while maintenance is performed.

We appreciate your patience throughout this process. If you need additional support, please contact [support@oit.co](mailto:support@oit.co)..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cmmarhl7j0046q52lk2fq9081</id>
  <published>2026-03-03T15:27:33.052+00:00</published>
  <updated>2026-03-03T15:27:33.052+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cmmarhl7j0046q52lk2fq9081"/>
  <title>Call Quality Degradation for Inbound and Outbound calls through Bandwidth</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 1 hour and 15 minutes</p>
    <p><strong>Affected Components:</strong> United States Outbound Calling Services, United States Inbound Calling Services</p>
    <p><small>Mar <var data-var='date'> 3</var>, <var data-var='time'>15:27:33</var> GMT+0</small><br /><strong>Monitoring</strong> -
  At approximately 9:25am ET, we became aware of inbound and outbound call quality concerns through our carrier Bandwidth, causing choppiness on call audio.

**When it began: 9:25am ET**

**Current Status:** While gathering details and examples from partners, Bandwidth updated their status page at 9:39am ET acknowledging call failures. At 9:58am ET, Bandwidth updated that they are observing expected inbound and outbound calls and are monitoring.

**Next Steps:** We have temporarily rerouted outbound calls through another carrier to preserve outbound call quality. Outbound calls should be functioning as expected. Inbound calls through Bandwidth may continue to have degraded quality until the incident is resolved by them.

**Next Update:** 11:20am ET.</p>
<p><small>Mar <var data-var='date'> 3</var>, <var data-var='time'>16:23:27</var> GMT+0</small><br /><strong>Monitoring</strong> -
  **What is affected:** At approximately 9:25am ET, we became aware of inbound and outbound call quality concerns through our carrier Bandwidth causing choppiness on call audio.

**When it began: 9:25am ET**

**Current Status:** While gathering details and examples from partners, Bandwidth updated their status page at 9:39am ET acknowledging call failures. At 9:58am ET, Bandwidth updated that they are observing expected inbound and outbound calls and monitoring. 

We have temporarily rerouted outbound calls through another carrier to preserve outbound call quality. Outbound calls should be functioning as expected. Inbound calls through Bandwidth may continue to have degraded quality until the incident is resolved by them.

**Next Steps:** At this time, we are continuing to monitor inbound call quality, however, no further reports of choppiness or degraded quality have been made.

**Next Update:** 12:20pm ET.</p>
<p><small>Mar <var data-var='date'> 3</var>, <var data-var='time'>16:42:56</var> GMT+0</small><br /><strong>Resolved</strong> -
  At approximately 9:25am ET, we became aware of inbound and outbound call quality concerns through our carrier Bandwidth causing choppiness on call audio.

**When it began: 9:25am ET**

**Current Status:** At this time, Bandwidth has declared the incident resolved. We have removed the reroute impacting outbound calls and tested to confirm inbound and outbound calls are successful.

**Next Steps:** A major incident report will be released within 48 business hours..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmltmoc6606vu2qkuhncksdth</id>
  <published>2026-02-26T04:00:00.000+00:00</published>
  <updated>2026-02-26T04:00:00.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmltmoc6606vu2qkuhncksdth"/>
  <title>Fax Server Maintenance 2/25/26</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 4 hours</p>
    <p><strong>Affected Components:</strong> Native Fax</p>
    <p><small>Feb <var data-var='date'> 26</var>, <var data-var='time'>04:00:00</var> GMT+0</small><br /><strong>Identified</strong> -
  We will be performing security maintenance on our fax server on **February 25th, 2026, at 11:00 pm ET**. There will be an interruption in the delivery and sending of faxes while maintenance is performed.

We appreciate your patience throughout this process. If you need additional support, please contact [support@oit.co](mailto:support@oit.co)..</p>
<p><small>Feb <var data-var='date'> 26</var>, <var data-var='time'>04:00:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Feb <var data-var='date'> 26</var>, <var data-var='time'>08:00:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cmko7e42j00rm10plkwnitmyc</id>
  <published>2026-01-21T15:54:20.641+00:00</published>
  <updated>2026-01-22T15:14:31.232+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cmko7e42j00rm10plkwnitmyc"/>
  <title>Significant Connection Delays on Outbound Calls</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 23 hours and 20 minutes</p>
    <p><strong>Affected Components:</strong> United States Outbound Calling Services</p>
    <p><small>Jan <var data-var='date'> 22</var>, <var data-var='time'>15:14:31</var> GMT+0</small><br /><strong>Resolved</strong> -
  **What Occurred**: On 1/21/26, at 10:20 AM ET, we were made aware of outbound calls failing for several clients. After further investigation, we determined that one of our underlying carriers was experiencing degraded services, resulting in outbound call failures.

**What was Affected:** Outbound Calling for DFW Region

**When it Began**: 01/21/2026, 10:20 AM ET

**Current Status**: Bandwidth declared the incident resolved yesterday at 7:22 PM ET. After 24 hours of monitoring and testing, we successfully routed all traffic back through Bandwidth and have confirmed all outbound calls are continuing to process as expected.

**Next Steps:** 

* This incident is now considered resolved.
* The major incident report will be available within 48 business hours.

**Next Update:** N/A.</p>
<p><small>Jan <var data-var='date'> 26</var>, <var data-var='time'>16:16:12</var> GMT+0</small><br /><strong>Resolved</strong> -
  The major incident report is now available: &lt;https://voipdocs.io/en%5FUS/announcements/2026-01-21-connection-failures-on-outbound-calls&gt;.</p>
<p><small>Jan <var data-var='date'> 21</var>, <var data-var='time'>15:54:20</var> GMT+0</small><br /><strong>Monitoring</strong> -
  **What Occurred**: At 10:20 AM ET, we were made aware of outbound calls failing for several clients. After further investigation, we determined that one of our underlying carriers was experiencing degraded services leading to outbound call failures.

**What is Affected:** All Outbound Calling

**When it Began:** 01/21/2026, 10:20 AM ET

**Current Status:** As of 10:34 AM ET, outbound calls have been rerouted through a different carrier and are now going through successfully.

**Next Steps:** The major incident report will be available within 48 business hours.

**Next Update:** 01/22/2026, 10:30 AM ET

We appreciate your patience throughout this process. If you need support, please contact [support@oit.co](mailto:support@oit.co). For more real time status updates as well as discussion, please join our Discord (Not official support)..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmk4c5trn03ymt3zfh1zpzv9i</id>
  <published>2026-01-15T04:00:00.000+00:00</published>
  <updated>2026-01-15T04:00:01.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmk4c5trn03ymt3zfh1zpzv9i"/>
  <title>Fax Server Maintenance 1/14/26</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 4 hours</p>
    <p><strong>Affected Components:</strong> Native Fax</p>
    <p><small>Jan <var data-var='date'> 15</var>, <var data-var='time'>04:00:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Jan <var data-var='date'> 15</var>, <var data-var='time'>08:00:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>
<p><small>Jan <var data-var='date'> 15</var>, <var data-var='time'>04:00:00</var> GMT+0</small><br /><strong>Identified</strong> -
  We will be performing security maintenance on our fax server and firewall on **January 14th, 2026, at 11:00 pm ET**. There will be an interruption in the delivery and sending of faxes while maintenance is performed.

We appreciate your patience throughout this process. If you need additional support, please contact [support@oit.co](mailto:support@oit.co)..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmjitl12u0f07arsmodhegji1</id>
  <published>2026-01-10T15:00:00.000+00:00</published>
  <updated>2026-01-10T15:00:01.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmjitl12u0f07arsmodhegji1"/>
  <title>Manager Portal Pro V4.0.0 Upgrade 01/10/26</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 4 hours</p>
    <p><strong>Affected Components:</strong> Manager Portal Pro</p>
    <p><small>Jan <var data-var='date'> 10</var>, <var data-var='time'>15:00:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Jan <var data-var='date'> 10</var>, <var data-var='time'>19:00:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>
<p><small>Jan <var data-var='date'> 10</var>, <var data-var='time'>15:00:00</var> GMT+0</small><br /><strong>Identified</strong> -
  To improve platform performance and reliability, we will be upgrading Manager Portal Pro (MPP) to version 4.0.0 on **January 10th, 2026, at 10:00 am ET**.

No service downtime is expected during this maintenance window. However, you may need to log out of the Manager Portal and log back in once the upgrade is complete.

**Please Note:** Following this upgrade, webhook integrations (URL Call Pop) and Manager Portal Pro (MPP) features will use a new static IP address: **52.20.230.43**

If you filter inbound traffic by source IP (such as via firewall or WAF rules), please ensure this new IP address is added to your allowlist prior to the maintenance window.

We appreciate your patience throughout this process. If you need additional support, please contact [support@oit.co](mailto:support@oit.co)..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cmjhmkxri03o2n706apzhvldx</id>
  <published>2025-12-22T20:45:26.417+00:00</published>
  <updated>2025-12-22T20:45:26.417+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cmjhmkxri03o2n706apzhvldx"/>
  <title>Service Controlling Voice Calls and Device Registration Crash on Core1-ATL</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 23 hours and 15 minutes</p>
    <p><strong>Affected Components:</strong> Atlanta (ATL) Voice</p>
    <p><small>Dec <var data-var='date'> 22</var>, <var data-var='time'>20:45:26</var> GMT+0</small><br /><strong>Monitoring</strong> -
  **What Occurred**: At 2:50 pm ET, the SiPbx service, which controls voice and registration on the core1-atl server, crashed, causing inbound and outbound calls to redirect to alternate servers, as well as device registration.

**What is Affected:** Atlanta (ATL) Voice

**When it Began:** 12/22/2025, 2:50 PM ET

**Current Status:** At 2:53p ET, the SiPbx service automatically recovered, and device registrations and inbound/outbound calls began returning to the core1-atl server successfully.

**Next Steps:** We are monitoring and investigating the root cause of the service failure.

**Next Update:** 12/23/2025, 3:00 PM ET.</p>
<p><small>Dec <var data-var='date'> 23</var>, <var data-var='time'>20:00:46</var> GMT+0</small><br /><strong>Resolved</strong> -
  **What Occurred:** At 2:50 PM ET, the SiPbx service that controls voice and registration on the Core1-ATL server crashed, causing inbound and outbound calls to redirect to alternate servers, as well as device registration.

**What is Affected:** Atlanta (ATL) Voice

**When it Began:** December 22nd, 2025, 2:50 PM ET

**Current Status:**

As of 9:30 AM ET, Core1-ATL remains stable. Working with the vendor we were able to identify the root cause of the service crash. It was ultimately due to a subprocess not closing cleanly and a fix is implemented in the next system update which is scheduled for Q1 of 2026.

**Next Steps:** This incident is now considered resolved. The major incident report will be available in 48 hours. .</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmj7hqmkz03ip6xclaw3kdhsl</id>
  <published>2025-12-16T06:00:00.000+00:00</published>
  <updated>2025-12-16T06:00:00.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmj7hqmkz03ip6xclaw3kdhsl"/>
  <title>CloudieConnect Unscheduled Maintenance 12/16/25</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 1 hour and 30 minutes</p>
    <p><strong>Affected Components:</strong> CloudieConnect</p>
    <p><small>Dec <var data-var='date'> 16</var>, <var data-var='time'>06:00:00</var> GMT+0</small><br /><strong>Identified</strong> -
  During the December 12th implementation of the CloudieConnect High Availability (HA) infrastructure, our engineering team identified additional configuration requirements that will strengthen the overall HA deployment. To ensure that the infrastructure is implemented to the highest standard, we have scheduled a follow-up maintenance window.

On **Tuesday, December 16th, 2025 at 1:00 AM ET**, our team will finalize the High Availability server activation and integration for your CloudieConnect mobile and desktop apps. This maintenance will complete the infrastructure upgrades, delivering the enhanced reliability and seamless failover protection your operations depend on.

During this 90-minute upgrade window, you may experience temporary service degradation as we bring the HA system fully online. Device registration, inbound/outbound calling, and messaging functions may be briefly affected. All other voice and portal services will continue to operate normally.

Thank you for your patience as we complete this critical infrastructure upgrade. If you need additional support, please contact [support@oit.co](mailto:support@oit.co)..</p>
<p><small>Dec <var data-var='date'> 16</var>, <var data-var='time'>06:00:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Dec <var data-var='date'> 16</var>, <var data-var='time'>07:30:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmit1wujc01nj9jmrlglsko7r</id>
  <published>2025-12-12T05:30:00.000+00:00</published>
  <updated>2025-12-10T16:50:44.420+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmit1wujc01nj9jmrlglsko7r"/>
  <title>CloudieConnect HA Server Implementation</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 1 hour and 30 minutes</p>
    <p><strong>Affected Components:</strong> CloudieConnect</p>
    <p><small>Dec <var data-var='date'> 10</var>, <var data-var='time'>16:50:44</var> GMT+0</small><br /><strong>Identified</strong> -
  On **December 12th, 2025, starting at 12:30 AM ET**, OIT will be upgrading the resiliency of the servers that power our CloudieConnect mobile and desktop apps. This upgrade finalizes our High Availability (HA) infrastructure, providing a solid foundation for enhanced reliability and seamless failover protection.

During the upgrade, CloudieConnect users may experience a brief service degradation lasting approximately 30 minutes as the HA server is activated and integrated. Device registration, inbound/outbound calling, and messaging may be temporarily impacted during this time.

All other voice and portal services will continue to operate normally..</p>
<p><small>Dec <var data-var='date'> 12</var>, <var data-var='time'>05:30:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Dec <var data-var='date'> 12</var>, <var data-var='time'>07:00:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>
<p><small>Dec <var data-var='date'> 12</var>, <var data-var='time'>05:30:00</var> GMT+0</small><br /><strong>Identified</strong> -
  On **December 12th, 2025, starting at 12:30 AM ET**, we will be performing system updates on the CloudieConnect servers to finalize High Availability (HA) services, ensuring failover redundancy is fully operational.

A brief service degradation lasting approximately 30 minutes may occur as the HA server is activated and integrated with the existing infrastructure. During this time, device registration and inbound/outbound calling may be impacted.

All other voice and portal services will continue to operate normally.

We appreciate your patience throughout this process. If you need additional support, please contact [support@oit.co](mailto:support@oit.co)..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmirmu7g501bgzoaeslpvi7bz</id>
  <published>2025-12-12T01:00:00.000+00:00</published>
  <updated>2025-12-12T01:00:01.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmirmu7g501bgzoaeslpvi7bz"/>
  <title>DNS Updates for api.ucaasnetwork.com</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 1 hour</p>
    <p><strong>Affected Components:</strong> API</p>
    <p><small>Dec <var data-var='date'> 12</var>, <var data-var='time'>01:00:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Dec <var data-var='date'> 12</var>, <var data-var='time'>02:00:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>
<p><small>Dec <var data-var='date'> 12</var>, <var data-var='time'>01:00:00</var> GMT+0</small><br /><strong>Identified</strong> -
  DNS updates for **api.ucaasnetwork.com** will be applied on **December 11th, 2025 at 8:00 PM ET**. The changes are to improve redundancy and speed of call responses for the API services. There is no expected downtime from this DNS change. There will also be no downtime for voice or portal services.

As a reminder, please review your scripts for any IP filtering and ensure you are allowing traffic to and from all of our IPs. The full list can be found here: [https://voipdocs.io/faqs/ip-addresses-ports](https://voipdocs.io/faqs/ip-addresses-ports#ip-addresses-for-additional-services-2)

We appreciate your patience throughout this process. If you need additional support, please contact [support@oit.co](mailto:support@oit.co)..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmi4tpbs7002y147im3mkq6ss</id>
  <published>2025-11-26T04:00:00.000+00:00</published>
  <updated>2025-11-26T04:00:01.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmi4tpbs7002y147im3mkq6ss"/>
  <title>Fax Server Maintenance 11/25/25</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 4 hours</p>
    <p><strong>Affected Components:</strong> Native Fax</p>
    <p><small>Nov <var data-var='date'> 26</var>, <var data-var='time'>04:00:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Nov <var data-var='date'> 26</var>, <var data-var='time'>08:00:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>
<p><small>Nov <var data-var='date'> 26</var>, <var data-var='time'>04:00:00</var> GMT+0</small><br /><strong>Identified</strong> -
  We will be performing security maintenance on our fax server and firewall on **November 25th, 2025, at 11:00 pm ET**. There will be an interruption in the delivery and sending of faxes while maintenance is performed..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmhtj1dbp000iumlv2ffkutvp</id>
  <published>2025-11-13T15:30:00.000+00:00</published>
  <updated>2025-11-13T15:30:01.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmhtj1dbp000iumlv2ffkutvp"/>
  <title>Manager Portal Pro Unscheduled Maintenance 11/13/25</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 2 hours</p>
    <p><strong>Affected Components:</strong> Manager Portal Pro</p>
    <p><small>Nov <var data-var='date'> 13</var>, <var data-var='time'>15:30:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Nov <var data-var='date'> 13</var>, <var data-var='time'>17:30:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>
<p><small>Nov <var data-var='date'> 13</var>, <var data-var='time'>15:30:00</var> GMT+0</small><br /><strong>Identified</strong> -
  OITVOIP will be performing unscheduled platform maintenance to deploy system improvements and enhancements for Manager Portal Pro (MPP) on **November 13th, 2025 at 10:30 AM ET**. This is **not for core services**, and includes things like TCR, call pop, and other integrations.

This release delivers multiple feature updates, security hardening, and performance improvements. _All changes are backward compatible and require no action from existing integrations or API consumers_.

**HIGHLIGHTS** 

* CloudieAI billing integration for sentiment and transcription usage
* Major security hardening across API and database access
* Faster API responses for integrations, reporting, and analytics
* Improved stability of nightly reporting and billing jobs

**FEATURES &amp; ENHANCEMENTS**

**Advanced Call Recording Controls**

* Added UI-configurable controls to enable or disable advanced call recording features.
* Centralized recording configuration management for easier administration.

**CloudieAI Billing Integration**

* New billing endpoints for sentiment analysis and call transcription usage.
* Usage metrics are now included in nightly count operations.
* Automatic usage data collection for CloudieAI-related services.
* Improved database connection management when communicating with the external CloudieAI billing database.

**React Upgrades**

* Completed migration of key frontend components

**SECURITY IMPROVEMENTS**

**Hardened Security**

* Completed a security review across the backend API.
* Hardened high-risk areas including authentication, reporting, integrations, CSP registry, user management, and webhooks.
* No changes to expected API behavior; existing integrations continue to work as before.

**API Authentication &amp; Access Control**

* Implemented a dual-token authentication model supporting both JWT and OAuth-based tokens.
* Added token caching strategies to reduce authentication overhead.

**PERFORMANCE &amp; STABILITY**

**API Response Time Optimization**

* Parallelized independent asynchronous operations across multiple controllers (integrations, reporting, analytics, call handlers, phone numbers).
* Refactored legacy promise chains to modern async/await patterns.
* Improved integrations response time by roughly 25–30%.
* Similar performance improvements are expected on call history, phone number management, and analytics endpoints.
* Tuned database connection pooling and query execution for more consistent performance.

**Nightly Processing &amp; Reporting**

* Resolved an issue that could cause nightly processing jobs to fail under certain configuration conditions.
* Added additional safeguards to prevent failures caused by incomplete or inconsistent configuration data.

**Configuration &amp; Runtime Stability**

* Updated service endpoint configuration handling to ensure more consistent authentication behavior.
* Refined connection management and database usage patterns to reduce runtime errors and improve overall system stability.

**We do not anticipate any service interruptions during this maintenance window.** All systems are expected to remain fully operational throughout the deployment. If you need additional support, please contact [**support@oit.co**](mailto:support@oit.co)..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cmgzgbmom06crgnpzv3yuzmge</id>
  <published>2025-10-20T18:10:59.916+00:00</published>
  <updated>2025-10-20T18:10:59.916+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cmgzgbmom06crgnpzv3yuzmge"/>
  <title>Avanan/AWS Outage</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 7 hours and 10 minutes</p>
    
    <p><small>Oct <var data-var='date'> 20</var>, <var data-var='time'>18:10:59</var> GMT+0</small><br /><strong>Identified</strong> -
  **What Occurred:** At 8:30 AM ET, our email filtering service began experiencing significant delays and delivery failures to our client-facing support mailboxes as a result of the widespread AWS US-East-1 outage this morning. This includes [support@oit.co](mailto:support@oit.co), [success@oit.co](mailto:success@oit.co), [porting@oit.co](mailto:porting@oit.co), [billing@oit.co](mailto:billing@oit.co), [sales@oit.co](mailto:sales@oit.co), and [compliance@oit.co](mailto:compliance@oit.co).

**Who is Affected:** All partners and clients attempting to contact OIT support.

**When It Began:** 8:30 AM ET, 10/20/2025

**Current Status:** We have disabled the email filtering service and restored deliverability to the affected mailboxes as of 12:30 PM ET. We are currently working with our email filtering service&#039;s support to complete the delivery of all emails sent to the affected mailboxes during this window.

**Workarounds:** For urgent requests that were sent during this window that cannot wait until the emails are restored, please re-send the email or call our support at 305-967-6756.

**Next Steps:** Continuing to work with vendor support to ensure all emails are delivered.

**Next Update:** We will provide an update when all emails during the affected period are restored.

We appreciate your patience throughout this process. If you need additional support, please contact [support@oit.co](mailto:support@oit.co)..</p>
<p><small>Oct <var data-var='date'> 21</var>, <var data-var='time'>01:21:09</var> GMT+0</small><br /><strong>Resolved</strong> -
  Resolved

What Occurred: At 8:30 AM ET on October 10th, 2025, our email filtering service began experiencing significant delays and delivery failures to our client-facing support mailboxes as a result of the widespread AWS US-East-1 outage that morning. This includes [support@oit.co](mailto:support@oit.co), [success@oit.co](mailto:success@oit.co), [porting@oit.co](mailto:porting@oit.co), [billing@oit.co](mailto:billing@oit.co), [sales@oit.co](mailto:sales@oit.co), and [compliance@oit.co](mailto:compliance@oit.co).

Who is Affected: All partners and clients attempting to contact OIT support

When It Began: 8:30 AM ET, 10/20/2025

Current Status:

This is considered Resolved.

AWS and our email filtering service have confirmed full functionality has been restored. We&#039;ve re-enabled our email filtering service and confirmed inbound emails are being received. 

Next Steps:

If you submitted a support request via email between 8:30 AM ET and 12:30 PM ET and never received a case confirmation email response, please re-send your email to confirm it was received on our end. 

Next Update: N/A.</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cmgfnlpnp00ay5ygeae1m9zhz</id>
  <published>2025-10-06T21:39:22.890+00:00</published>
  <updated>2025-10-06T21:39:22.890+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cmgfnlpnp00ay5ygeae1m9zhz"/>
  <title>CloudieAI – Call Transcription and Sentiment not populating after enablement</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 1 day, 16 hours and 54 minutes</p>
    <p><strong>Affected Components:</strong> CloudieAI</p>
    <p><small>Oct <var data-var='date'> 6</var>, <var data-var='time'>21:39:22</var> GMT+0</small><br /><strong>Investigating</strong> -
  We’re receiving reports that once CloudieAI is enabled, transcription and sentiment are not displaying in Call History across affected tenants. 

**Identified:** Oct 6, 2025, at 8:46 AM ET 

**Impact:** All partners/end users with CloudieAI enabled (previously or newly enabled) may not see transcription or sentiment appear in Call History. 

**Current status:** OIT is actively engaged with the vendor and investigating the root cause. 

**Workaround:** None confirmed at this time. 

**Next update:** We will provide an update when new information becomes available..</p>
<p><small>Oct <var data-var='date'> 8</var>, <var data-var='time'>14:33:08</var> GMT+0</small><br /><strong>Resolved</strong> -
  We identified an issue where Call Transcription and Sentiment data were not populating in Call History for users with CloudieAI enabled.

**Issue Identified:** October 6, 2025, at 8:46 AM ET

**Impact:** All partners and end users with CloudieAI enabled (previously or newly enabled)

**Status:**

* Resolved
* Our team immediately investigated and implemented a fix, and we’re now seeing data populate successfully across affected tenants. We will continue to monitor performance to ensure stability.

We believe **CloudieAI will have a major impact for all users**, empowering them with enhanced visibility into customer interactions and actionable insights. We want to ensure everyone has the maximum opportunity to experience its full potential—**without any risk**.

With that goal in mind, we are proud to announce that we are **extending the free beta period through November 30, 2025\.** 

CloudieAI billing will now begin on **December 1, 2025.**

Thank you for your understanding and for being a valued OIT Partner. If you continue to experience issues or have questions, please contact our support team at **support@oit.co**, and one of our technicians will be happy to assist.</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cmfmmx53s001z96bbrccg6n22</id>
  <published>2025-09-16T14:14:58.101+00:00</published>
  <updated>2025-09-29T15:56:11.462+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cmfmmx53s001z96bbrccg6n22"/>
  <title>Inbound CNAM Lookup Intermittently Not Returning Correct CID</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 21 days and 45 minutes</p>
    <p><strong>Affected Components:</strong> United States Inbound Calling Services</p>
    <p><small>Sep <var data-var='date'> 29</var>, <var data-var='time'>15:56:11</var> GMT+0</small><br /><strong>Identified</strong> -
  What Occurred: We have identified that the upstream CNAM database provider is intermittently providing incorrect Caller ID Name information after performing a CNAM dip.

Who is Affected: All partners and clients.

When It Began: 9:30 AM ET, 9/15/2025

Current Status: Bandwidth has internally escalated this with their engineers as well as with their new upstream CNAM DB provider.

Workarounds: There are no known workarounds at this time.

Next Steps: Bandwidth has requested that any new incorrect inbound CNAM lookups be provided to them. If you have any examples, please email [support@oit.co](mailto:support@oit.co) and provide the caller number, called number, date and time, along with the caller ID that was displayed and what it should have been.

Next Update: We will provide an update when new information becomes available.

We appreciate your patience throughout this process. If you need additional support, please contact [support@oit.co](mailto:support@oit.co)..</p>
<p><small>Oct <var data-var='date'> 7</var>, <var data-var='time'>14:59:56</var> GMT+0</small><br /><strong>Resolved</strong> -
  **What Occurred**: We have identified that the upstream CNAM database provider is intermittently providing incorrect Caller ID Name information after performing a CNAM dip.  
Who is Affected: All partners and clients.  
**When It Began:** 9:30 AM ET, 9-15-25  
**Current Status:** 

* This is considered **Resolved.**
* Bandwidth has informed us that after working with their upstream CNAM database provider they were able to correct the inbound CNAM discrepancies.
* We have confirmed we are seeing proper inbound CNAM at this time.

**Next Steps:** 

* If you continue to experience incorrect inbound CNAM, Bandwidth has requested that any new incorrect inbound CNAM lookups be provided to them.
* If you have any examples please email [support@oit.co](mailto:support@oit.co) and provide the caller number, called number, date and time along with the caller ID that was displayed and what it should have been.

**Next Update:** N/A.</p>
<p><small>Sep <var data-var='date'> 16</var>, <var data-var='time'>14:14:58</var> GMT+0</small><br /><strong>Investigating</strong> -
  What Occurred: We have identified that the upstream CNAM database provider is intermittently providing incorrect Caller ID Name information after performing a CNAM dip.

Who is Affected: All partners and clients.

When It Began: 9:30 AM ET, 9/15/2025

Current Status: Bandwidth has confirmed that the upstream CNAM/LIDB database provider is having a broad issue, and they are investigating internally.

Workarounds: There are no known workarounds at this time.

Next Update: We will provide an update when new information becomes available.

We appreciate your patience throughout this process. If you need additional support, please contact [support@oit.co](mailto:support@oit.co)..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmebtdyzl000610s7ysbxp43a</id>
  <published>2025-08-22T03:00:00.000+00:00</published>
  <updated>2025-08-22T03:00:01.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmebtdyzl000610s7ysbxp43a"/>
  <title>Fax Server Maintenance 08/21/25</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 2 hours and 14 minutes</p>
    <p><strong>Affected Components:</strong> Native Fax</p>
    <p><small>Aug <var data-var='date'> 22</var>, <var data-var='time'>03:00:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Aug <var data-var='date'> 22</var>, <var data-var='time'>03:00:00</var> GMT+0</small><br /><strong>Identified</strong> -
  We will be performing security maintenance on our fax server and firewall on August 21st, 2025, at 11:00 pm ET. There will be an interruption in the delivery and sending of faxes while maintenance is performed..</p>
<p><small>Aug <var data-var='date'> 22</var>, <var data-var='time'>05:13:37</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cmeljjszq00011lwn8oxl844u</id>
  <published>2025-08-21T15:13:08.022+00:00</published>
  <updated>2025-08-21T18:30:32.394+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cmeljjszq00011lwn8oxl844u"/>
  <title>TCR Campaign Management Services Currently Unavailable</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 3 hours and 17 minutes</p>
    <p><strong>Affected Components:</strong> TCR - The Campaign Registry</p>
    <p><small>Aug <var data-var='date'> 21</var>, <var data-var='time'>18:30:32</var> GMT+0</small><br /><strong>Resolved</strong> -
  This incident is now considered **resolved**. We will continue to monitor for the next 24 hours.

We appreciate your patience throughout this process. If you need additional support, please contact [**support@oit.co**](mailto:support@oit.co). .</p>
<p><small>Aug <var data-var='date'> 21</var>, <var data-var='time'>16:19:18</var> GMT+0</small><br /><strong>Identified</strong> -
  \*\*What Occurred:\*\* We are currently experiencing a service disruption affecting The Campaign Registry (TCR) platform. This impacts the ability to create new 10DLC campaigns, register new brands, or edit existing campaigns. \*\*Who is Affected:\*\* All partners and clients who need to create new TCR/10DLC campaigns, register new brands, or modify existing campaign configurations. \*\*When It Began:\*\* 9:31 AM ET, 8-21-25 \*\*Current Status:\*\* \* TCR has confirmed a platform-wide issue affecting campaign management functions \* OIT is actively working with TCR support to monitor the situation \* No updates have been provided by TCR at this time \* You can follow along for live updates from TCR at \[Campaign Registry Status\](https://status.campaignregistry.com/#) \*\*Workarounds:\*\* There are no workarounds available for campaign creation or editing at this time. Please plan accordingly for any urgent campaign needs. \*\*Next Steps:\*\* \* We recommend postponing any planned campaign creation or modifications until service is restored \* OIT will continue monitoring the situation and provide updates as information becomes available \*\*Next Update:\*\* 3:15 PM ET, 8-21-25.</p>
<p><small>Aug <var data-var='date'> 21</var>, <var data-var='time'>15:13:08</var> GMT+0</small><br /><strong>Identified</strong> -
  **What Occurred:** We are currently experiencing a service disruption affecting The Campaign Registry (TCR) platform. This impacts the ability to create new 10DLC campaigns, register new brands, or edit existing campaigns.

**Who is Affected:** All partners and clients who need to create new TCR/10DLC campaigns, register new brands, or modify existing campaign configurations.

**When It Began:** 9:31 AM ET, 8-21-25

**Current Status:**

* TCR has confirmed a platform-wide issue affecting campaign management functions
* OIT is actively working with TCR support to monitor the situation
* No estimated time of resolution has been provided at this time

**Workarounds:** There are no workarounds available for campaign creation or editing at this time. Please plan accordingly for any urgent campaign needs.

**Next Steps:**

* We recommend postponing any planned campaign creation or modifications until service is restored
* OIT will continue monitoring the situation and provide updates as information becomes available

**Next Update:** We will provide an update within 1 hour or when new information becomes available

**Offline Services:**

* TCR Campaign Creation
* TCR Brand Registration
* TCR Campaign Editing/Modifications

**Degraded Services:**

* None

**Operational Services:**

* SMS/MMS message sending and receiving (all existing campaigns continue to function normally)
* All other messaging services
* All voice services
* All portal services

**Important Note:** This outage does NOT affect your ability to send or receive messages. All existing registered campaigns continue to operate normally.

We appreciate your patience throughout this process. If you need support, please contact [support@oit.co](mailto:support@oit.co). For more real-time status updates as well as discussion please join our Discord..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmd6f5kcj01oaynys54jd2am0</id>
  <published>2025-07-19T03:30:00.000+00:00</published>
  <updated>2025-07-19T04:30:00.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmd6f5kcj01oaynys54jd2am0"/>
  <title>CloudieConnect Server Maintenance</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 1 hour</p>
    <p><strong>Affected Components:</strong> CloudieConnect</p>
    <p><small>Jul <var data-var='date'> 19</var>, <var data-var='time'>04:30:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>
<p><small>Jul <var data-var='date'> 19</var>, <var data-var='time'>03:30:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Jul <var data-var='date'> 19</var>, <var data-var='time'>03:30:00</var> GMT+0</small><br /><strong>Identified</strong> -
  On **Friday, July 18, 2025, at 11:30 PM ET**, we’ll be performing routine maintenance on the CloudieConnect system to help ensure continued performance and reliability.

During this time, CloudieConnect users may experience a brief service interruption (up to 5 minutes) during which login access and the ability to make or receive calls may be temporarily unavailable.

All other services will remain fully operational and unaffected.

We appreciate your patience throughout this process. If you need additional support, please contact [support@oit.co](mailto:support@oit.co)..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmd4thqx3003c5j6305lk0i81</id>
  <published>2025-07-16T01:30:00.000+00:00</published>
  <updated>2025-07-16T01:30:01.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmd4thqx3003c5j6305lk0i81"/>
  <title>PHX and IAD Core Server Maintenance</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 1 hour</p>
    <p><strong>Affected Components:</strong> Phoenix (PHX) Voice, Ashburn (IAD) Voice</p>
    <p><small>Jul <var data-var='date'> 16</var>, <var data-var='time'>01:30:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Jul <var data-var='date'> 16</var>, <var data-var='time'>02:30:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>
<p><small>Jul <var data-var='date'> 16</var>, <var data-var='time'>01:30:00</var> GMT+0</small><br /><strong>Identified</strong> -
  As part of our ongoing preparations for the migration to the new IAD and PHX core servers, we will be performing some maintenance on the IAD and PHX core servers tonight, July 15, at 9:30 PM ET.

There will be no interruption of service for the ATL, GRR, or LAS servers.

We appreciate your patience throughout this process. If you need additional support, please contact support@oit.co..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cmd3bvdj701bk10vt7uz7xw1v</id>
  <published>2025-07-14T16:38:38.473+00:00</published>
  <updated>2025-07-14T16:38:38.473+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cmd3bvdj701bk10vt7uz7xw1v"/>
  <title>mFax Intermittent Inbound Fax Failures</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 1 day, 1 hour and 5 minutes</p>
    <p><strong>Affected Components:</strong> United States Inbound Calling Services, mFax, Messaging (SMS/MMS), United States Outbound Calling Services, Manager Portal Pro, Device Provisioning (NDP), Native Fax, Commio, Nuso, Voice, SNAPmobile, TCR - The Campaign Registry, Phoenix (PHX) Voice, Canada Outbound Calling Services, Teammate, Call Recording (PHX), MobileX, Ashburn (IAD) Voice, Call Recording (IAD), United States Inbound Calling Services, Canada Inbound Calling Services, Call Recording (ATL), Atlanta (ATL) Voice, United States Outbound Calling Services, Grand Rapids (GRR) Voice, API, Las Vegas (LAS) Voice, Manager Portal, SNAPmobile Web, VoIPMonitor (QoS)</p>
    <p><small>Jul <var data-var='date'> 14</var>, <var data-var='time'>16:38:38</var> GMT+0</small><br /><strong>Investigating</strong> -
  At 11:45 AM ET, the mFax vendor reported a subset of inbound numbers failing with a &quot;no fax tone detected&quot; error. This is due to one of the vendor&#039;s upstream telecom providers experiencing intermittent failures.

What Is Affected: mFax

When It Began: 11:45 AM ET June 14, 2025

Current Status: The vendor is currently investigating.

Next Steps: No action required.

Next Update: 1:35 PM ET June 14, 2025.</p>
<p><small>Jul <var data-var='date'> 14</var>, <var data-var='time'>17:30:30</var> GMT+0</small><br /><strong>Investigating</strong> -
  What Is Affected: mFax

When It Began: 11:45 AM ET June 14, 2025

Current Status: The vendor is currently investigating.

Next Steps: No action required.

Next Update: 2:35 PM ET June 14, 2025.</p>
<p><small>Jul <var data-var='date'> 14</var>, <var data-var='time'>18:40:10</var> GMT+0</small><br /><strong>Monitoring</strong> -
  **Current Status:** A fix has been implemented with successful faxes received.

**Next Steps:** We will continue to monitor for the next 24 hours.

**Next Update:** 2:35 PM ET June 15, 2025.</p>
<p><small>Jul <var data-var='date'> 15</var>, <var data-var='time'>17:43:11</var> GMT+0</small><br /><strong>Resolved</strong> -
  As of 2:58am, July 15, 2025 mfax is showing the implemented fix performed by their telecom provider is successful and all inbound faxes are working as intended. 

 After 24 hours of monitoring with no additional inbound fax failures, This incident is now considered resolved..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cmcuym6q20049ncwc9l6ji1i4</id>
  <published>2025-07-08T20:05:24.617+00:00</published>
  <updated>2025-07-08T20:05:24.617+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cmcuym6q20049ncwc9l6ji1i4"/>
  <title>GRR Inbound Call Failures</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 23 hours and 34 minutes</p>
    <p><strong>Affected Components:</strong> Grand Rapids (GRR) Voice</p>
    <p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>20:05:24</var> GMT+0</small><br /><strong>Monitoring</strong> -
  \*\*What Occurred:\*\* At 2:37 PM ET, inbound and outbound calls to GRR began receiving a 503 error, causing them to fail. \*\*What Is Affected:\*\* Inbound and outbound calls to the GRR server \*\*When It Began:\*\* 2:37 PM ET on July 8, 2025 \*\*Next Update:\*\* July 9, 2025 at 4:00 PM ET \*\*Current Status:\*\* We have restarted the NMS service on the GRR server and are showing calls processing successfully again. \*\*Next Steps:\*\* \* We will continue to monitor to ensure that calls continue to process successfully. We will also continue investigating the root cause of the failure. \* Devices that were registered on GRR may need to be restarted after the NMS service restart took place. \*\*Next Update:\*\* July 9, 2025 at 4:00 PM ET.</p>
<p><small>Jul <var data-var='date'> 9</var>, <var data-var='time'>19:38:59</var> GMT+0</small><br /><strong>Resolved</strong> -
  \*\*What Occurred:\*\* At 2:37 PM ET, all calls on the GRR server began to fail. \*\*What was Affected:\*\* \* Inbound and Outbound Calls on GRR \*\*When It Began\*\*: 7/8/2025 2:37 PM ET \*\*Current Status\*\*: This incident is now Resolved. \*\*Next Steps:\*\* Major Incident Report will be available within 48 business hours..</p>
<p><small>Jul <var data-var='date'> 8</var>, <var data-var='time'>20:10:22</var> GMT+0</small><br /><strong>Monitoring</strong> -
  We implemented a fix and are currently monitoring the result..</p>
<p><small>Jul <var data-var='date'> 11</var>, <var data-var='time'>20:13:58</var> GMT+0</small><br /><strong>Postmortem</strong> -
  Major Incident Report: [2025-07-08 Inbound and Outbound call failures on GRR](https://voipdocs.io/announcements/-2025-07-08-registration-failure-on-grr).</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmce9r9at000ws5big4k6cgvf</id>
  <published>2025-06-27T04:00:00.000+00:00</published>
  <updated>2025-06-27T04:00:01.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmce9r9at000ws5big4k6cgvf"/>
  <title>V44.3 Core System Upgrade</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 1 hour and 58 minutes</p>
    <p><strong>Affected Components:</strong> Messaging (SMS/MMS), Device Provisioning (NDP), Native Fax, SNAPmobile, Phoenix (PHX) Voice, Call Recording (PHX), Ashburn (IAD) Voice, Call Recording (IAD), Call Recording (ATL), Atlanta (ATL) Voice, Grand Rapids (GRR) Voice, API, Las Vegas (LAS) Voice, Manager Portal, SNAPmobile Web, VoIPMonitor (QoS)</p>
    <p><small>Jun <var data-var='date'> 27</var>, <var data-var='time'>04:00:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Jun <var data-var='date'> 27</var>, <var data-var='time'>05:58:10</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully..</p>
<p><small>Jun <var data-var='date'> 27</var>, <var data-var='time'>04:00:00</var> GMT+0</small><br /><strong>Identified</strong> -
  We are upgrading all core servers to the V44.3 software, which includes bug fixes as well as upgrades to the manager portal. .</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmbibh34l000bn54nw8tginsv</id>
  <published>2025-06-13T06:00:00.000+00:00</published>
  <updated>2025-06-13T10:00:00.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmbibh34l000bn54nw8tginsv"/>
  <title>Core1-ATL Maintenance 06/13/25</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 4 hours</p>
    <p><strong>Affected Components:</strong> SNAPmobile Web, Atlanta (ATL) Voice</p>
    <p><small>Jun <var data-var='date'> 13</var>, <var data-var='time'>10:00:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>
<p><small>Jun <var data-var='date'> 13</var>, <var data-var='time'>06:00:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Jun <var data-var='date'> 13</var>, <var data-var='time'>06:00:00</var> GMT+0</small><br /><strong>Identified</strong> -
  We will be performing general system updates on our ATL core server in preparation for the upcoming migration to the IAD and PHX servers on June 13th, 2025, at 2:00am ET.

During this time we will redirect the portal, device registrations, and calls to alternate servers. There is no expected interruption of service during this time..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cmbts2i5t008njmdrdwsgfcul</id>
  <published>2025-06-12T19:35:14.754+00:00</published>
  <updated>2025-06-12T19:35:14.754+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cmbts2i5t008njmdrdwsgfcul"/>
  <title>Cloudflare Outage</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 3 hours and 44 minutes</p>
    <p><strong>Affected Components:</strong> mFax</p>
    <p><small>Jun <var data-var='date'> 12</var>, <var data-var='time'>19:35:14</var> GMT+0</small><br /><strong>Monitoring</strong> -
  Cloudflare experienced a broad service outage that affected the mFax service. \*\*What Is Affected:\*\* MFax \*\*When It Began:\*\* 1:30 PM ET June 12, 2025 \*\*Current Status:\*\* As of 3:12 pm ET, Cloudflare is showing services are recovering, while mFax services may still experience intermittent &quot;internal server&quot; error messages until fully restored. \*\*Next Steps:\*\* N/A \*\*Next Update:\*\* 4:30PM ET June 13, 2025.</p>
<p><small>Jun <var data-var='date'> 12</var>, <var data-var='time'>23:18:59</var> GMT+0</small><br /><strong>Resolved</strong> -
  Notice: Cloudflare Outage What Is Affected: MFax When It Began: 1:30 PM ET June 12, 2025 Current Status: Resolved - Cloudflare identified the cause to be with one of their upstream vendors, who as of 4:00 PM ET has fully restored services on their end. mFax is showing all fax queues are clearing and services have been restored. This incident is now considered Resolved. Next Steps: \* Major Incident Report will be available within 48 hours. Next Update: 4:30PM ET June 13, 2025 Operational Services: All.</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cmbjrbkuv000c122wqkv52a33</id>
  <published>2025-06-05T19:16:02.114+00:00</published>
  <updated>2025-06-05T19:16:19.411+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cmbjrbkuv000c122wqkv52a33"/>
  <title>Registration Failure on GRR</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 22 hours and 36 minutes</p>
    <p><strong>Affected Components:</strong> Grand Rapids (GRR) Voice</p>
    <p><small>Jun <var data-var='date'> 5</var>, <var data-var='time'>19:16:19</var> GMT+0</small><br /><strong>Monitoring</strong> -
  .</p>
<p><small>Jun <var data-var='date'> 5</var>, <var data-var='time'>19:16:02</var> GMT+0</small><br /><strong>Monitoring</strong> -
  At 2:26 PM ET, we received a notification from our NOC monitoring that the NMS service which handles registration and call processing on GRR crashed. Active calls on this server did drop, and devices on GRR failed over to ATL successfully.

**What Is Affected:**

* Device Registration on GRR
* Active Calls on GRR

**When It Began:** 2:26 PM ET

**Current Status:** The NMS service on GRR is back online as of 2:29 PM ET, and registration for all devices returned to GRR by 2:39 PM ET. At this time, all services on GRR are functioning as expected.

**Next Steps:** We will be monitoring GRR for 24 hours per our major incident policy while we continue to investigate the root cause. If additional outages occur, we will manually redirect devices &amp; calls on GRR to ATL until the problem is resolved.

**Next Update:** 6/6/25 3:00 PM ET.</p>
<p><small>Jun <var data-var='date'> 6</var>, <var data-var='time'>17:51:33</var> GMT+0</small><br /><strong>Resolved</strong> -
  At 2:26 PM ET, we received a notification from our NOC monitoring that the NMS service, which handles registration and call processing on GRR crashed. Active calls on this server did drop, and devices on GRR failed over to ATL successfully.

**What was Affected:**

* Device Registration on GRR
* Active Calls on GRR

**When It Began:** 06/05/2025 2:26 PM ET

**Current Status**: This incident is now **resolved.**

**Next Steps:** 

* Major Incident Report is now available
* MIR report: &lt;https://voipdocs.io/announcements/-2025-06-05-registration-failure-on-grr-&gt;.</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cmbi3en6l000y1hn8m9lm2sr8</id>
  <published>2025-06-04T15:18:44.684+00:00</published>
  <updated>2025-06-04T15:18:44.684+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cmbi3en6l000y1hn8m9lm2sr8"/>
  <title>Loss of Communication to GRR</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 1 day, 1 hour and 36 minutes</p>
    <p><strong>Affected Components:</strong> Grand Rapids (GRR) Voice</p>
    <p><small>Jun <var data-var='date'> 4</var>, <var data-var='time'>15:18:44</var> GMT+0</small><br /><strong>Investigating</strong> -
  At 10:30am ET, our monitoring system alerted us to the GRR server being unreachable for two minutes, resulting in device registrations and inbound calls to failover to other servers. Shortly after, the same datacenter lost communication again. We are investigating quickly and will determine if we need to close GRR temporarily. Updates to follow

What Is Affected: GRR Voice

When It Began: 10:30 AM ET

Current Status: We are investigating quickly and will determine if we need to close GRR temporarily.

Next Steps: Updates to Follow

Next Update: 12:15 PM ET.</p>
<p><small>Jun <var data-var='date'> 4</var>, <var data-var='time'>16:22:22</var> GMT+0</small><br /><strong>Monitoring</strong> -
  What Is Affected: GRR Server

When It Began: 10:30 AM ET

Current Status: We have determined that the loss of traffic was due to a degraded circuit. The offending circuit was removed from the routing profile. Traffic has remained stable since the change. Moving to the monitoring stage.

Next Steps: Continue to monitor

Next Update: 6/5/2025 1:15 PM ET

Degraded Services: None

Operational Services: All.</p>
<p><small>Jun <var data-var='date'> 4</var>, <var data-var='time'>21:18:33</var> GMT+0</small><br /><strong>Monitoring</strong> -
  **Current Status:** At this time we continue to see stability on GRR&#039;s connections. The datacenter will be performing maintenance and repair to the MPLS configuration tonight at 2AM and is expected to be completed before 8AM. No downtime or service interruptions are expected. Our NOC will be monitoring throughout the monitoring phase.

**Next Steps:** 

* Continue to monitor
* Maintenance: Tonight, 2:00 - 8:00 AM ET

**Next Update**: 1:15 PM ET 6/25/2025

**Degraded Services**: None

**Operational Services:** All.</p>
<p><small>Jun <var data-var='date'> 5</var>, <var data-var='time'>16:55:00</var> GMT+0</small><br /><strong>Resolved</strong> -
  At 10:30am ET, our monitoring system alerted us to the GRR server being unreachable for two minutes, resulting in device registrations and inbound calls to failover to other servers. Shortly after, the same datacenter lost communication again. 

**What was Affected:** GRR Server

**When It Began:** 10:30 AM ET

**Current Status:** 

* We continue to see stability on GRR&#039;s connections after 24 hours of monitoring.
* This incident is now considered **Resolved**.

**Next Steps:**

* Major Incident Report will be available within 48 hours..</p>
<p><small>Jun <var data-var='date'> 6</var>, <var data-var='time'>16:17:09</var> GMT+0</small><br /><strong>Resolved</strong> -
  **MIR now available for Loss of Communication to GRR Server**

At 10:30am ET, our monitoring system alerted us to the GRR server being unreachable for two minutes, resulting in device registrations and inbound calls to failover to other servers. Shortly after, the same datacenter lost communication again.

**What Was Affected:** GRR Server

**When It Began:** 6/04/2025 10:30 AM ET

**Resolution:** 

* The degraded circuit was immediately removed from the routing profile to prevent further instability
* Enhanced monitoring was implemented during the recovery period to ensure sustained stability
* **The Major Incident Report is now available:** &lt;https://voipdocs.io/announcements/-2025-06-04-loss-of-communication-to-grr-server&gt;.</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cmbazugqg0001q5lu1231fub1</id>
  <published>2025-05-30T16:04:44.518+00:00</published>
  <updated>2025-05-30T16:04:44.518+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cmbazugqg0001q5lu1231fub1"/>
  <title>Websocket Connection Failure on GRR and LAS</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    
    
    <p><small>May <var data-var='date'> 30</var>, <var data-var='time'>16:04:44</var> GMT+0</small><br /><strong>Identified</strong> -
  At 11:06 AM ET, we were alerted by our monitoring tools that WSS connections on GRR &amp; LAS were failing. SNAPmobile Web uses WSS for the SIP connection. \*\*What Is Affected\*\*: All SNAPmobile Web Connections on GRR &amp; LAS \*\*When It Began:\*\* 11:06AM ET \*\*Current Status:\*\* We have identified the cause and are awaiting vendor confirmation to implement a fix. \*\*Next Steps\*\*: No next steps for partners \*\*Next Update\*\*: 12:06PM ET.</p>
<p><small>Jun <var data-var='date'> 3</var>, <var data-var='time'>15:25:40</var> GMT+0</small><br /><strong>Resolved</strong> -
  \*\*MIR now available for Websocket Connection Failure on GRR and LAS\*\* At 11:06AM ET we were alerted by our monitoring tools that WSS connections on GRR, LAS, IAD, and PHX were failing. SNAPmobile Web uses WSS for the SIP connection. \*\*What was Affected:\*\* All SNAPmobile Web Connections on GRR, LAS, IAD, &amp; PHX \*\*When It Began\*\*: 11:06AM ET 05/06/2025 \*\*Resolution:\*\* \* \*\*Our vendor resolved an SSL issue with the affected FQDNs on 05/31/2025\*\* \* \*\*Major Incident Report\*\* is now available: &lt;https://voipdocs.io/announcements/20250530-websocket-connections-for-grr-las-phx-and-iad-servers-failing-to-connect&gt;.</p>
<p><small>May <var data-var='date'> 30</var>, <var data-var='time'>16:44:04</var> GMT+0</small><br /><strong>Identified</strong> -
  What Is Affected: All SNAPmobile Web Connections on GRR, LAS, IAD, &amp; PHX

When It Began: 11:06 AM ET

Current Status:

* We have temporarily redirected all WSS traffic to ATL, which is confirmed to be working. In the meantime, our engineers will continue to work on a permanent resolution.

Next Steps:

* Users will need to completely close SNAPmobile Web and reopen it to ensure a new connection. If the problem persists, please clear your browser cache and test again. If, after clearing your cache, you still cannot connect, please submit a ticket to [support@oit.co](mailto:support@oit.co)

Next Update: 2:06 PM ET.</p>
<p><small>May <var data-var='date'> 31</var>, <var data-var='time'>18:53:04</var> GMT+0</small><br /><strong>Resolved</strong> -
  This incident has been resolved..</p>
<p><small>May <var data-var='date'> 30</var>, <var data-var='time'>17:50:43</var> GMT+0</small><br /><strong>Monitoring</strong> -
  **What Is Affected:** All SNAPmobile Web Connections on GRR, LAS, IAD, &amp; PHX

**When It Began:** 11:06AM ET

**Current Status**:

* After working with vendor support, we have successfully applied a fix to resolve WSS connections failing on LAS, GRR, IAD, &amp; PHX.
* Per our internal policy, we will keep all WSS traffic redirected to ATL while we continue to monitor for 24 hours. This will prevent further downtime if the problem reoccurs.

**Next Steps:**

* Users will need to completely close SNAPmobile Web and reopen it to ensure a new connection. If the problem persists, please clear your browser cache and test again. If after clearing your cache, you still cannot connect, please submit a ticket to support@oit.co
* We understand that some users may experience poor call quality or dropped calls on ATL. If you experience any of these, please email support@oit.co with call examples or call traces so that we can investigate and potentially move affected clients back to their respective cores in advance.

**Next Update:** 5/31/25 2:00 PM ET.</p>
<p><small>May <var data-var='date'> 31</var>, <var data-var='time'>18:56:52</var> GMT+0</small><br /><strong>Resolved</strong> -
  At 11:06AM ET we were alerted by our monitoring tools that WSS connections on GRR, LAS, IAD, and PHX were failing. SNAPmobile Web uses WSS for the SIP connection.

**What Is Affected:** All SNAPmobile Web Connections on GRR, LAS, IAD, &amp; PHX

**When It Began:** 11:06AM ET

**Current Status**:

* **This incident is considered Resolved.**
* **Our vendor resolved an SSL issue with the affected FQDNs**

**Next Steps**:

* Major Incident Report will be available within 48 hours.

**Next Update**: N/A

**Offline Services: N/A**

**Degraded Services:** N/A.</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmaydg2u10029zq6525coq932</id>
  <published>2025-05-30T06:00:00.000+00:00</published>
  <updated>2025-05-30T06:00:00.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmaydg2u10029zq6525coq932"/>
  <title>ATL Server General Maintenance 05/30/25</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 2 hours and 27 minutes</p>
    <p><strong>Affected Components:</strong> Atlanta (ATL) Voice, VoIPMonitor (QoS)</p>
    <p><small>May <var data-var='date'> 30</var>, <var data-var='time'>06:00:00</var> GMT+0</small><br /><strong>Identified</strong> -
  We will be performing general system maintenance on our ATL Core and QoS servers on May 30th, 2025, at 2:00am ET.

During this time, we will redirect device registrations, calls, and the portal to alternate servers.

There is no expected interruption of service during this time..</p>
<p><small>May <var data-var='date'> 30</var>, <var data-var='time'>06:00:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>May <var data-var='date'> 30</var>, <var data-var='time'>08:26:53</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cmaod58p8001e7gntn4gg3rl8</id>
  <published>2025-05-23T03:00:00.000+00:00</published>
  <updated>2025-05-23T03:00:00.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cmaod58p8001e7gntn4gg3rl8"/>
  <title>Fax Server &amp; Firewall Security Maintenance 05/22/25</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 4 hours</p>
    <p><strong>Affected Components:</strong> Native Fax</p>
    <p><small>May <var data-var='date'> 23</var>, <var data-var='time'>03:00:00</var> GMT+0</small><br /><strong>Identified</strong> -
  We will be performing security maintenance on our fax server and firewall on May 22nd, 2025, at 11:00pm ET. There will be an interruption in the delivery and sending of faxes while maintenance is performed..</p>
<p><small>May <var data-var='date'> 23</var>, <var data-var='time'>03:00:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>May <var data-var='date'> 23</var>, <var data-var='time'>07:00:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cm9xe1f4v001pyqvn7lqqcjel</id>
  <published>2025-04-25T22:53:35.331+00:00</published>
  <updated>2025-04-25T22:53:35.331+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cm9xe1f4v001pyqvn7lqqcjel"/>
  <title>Websocket Connection Failure on ATL, GRR and LAS</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 2 hours and 30 minutes</p>
    <p><strong>Affected Components:</strong> SNAPmobile Web</p>
    <p><small>Apr <var data-var='date'> 25</var>, <var data-var='time'>22:53:35</var> GMT+0</small><br /><strong>Identified</strong> -
  At 6:19PM ET we were alerted by our monitoring tools that WSS connections on ATL, GRR &amp; LAS were failing. SNAPmobile Web uses WSS for the SIP connection.

**What Is Affected**: All SNAPmobile Web Connections on ATL, GRR &amp; LAS

**When It Began**: 6:19PM ET

**Current Status**:

* We have identified a potential cause for and are awaiting vendor confirmation to implement a fix.

**Next Steps**:

* No next steps for partners

**Next Update**: 7:45PM ET

**Offline Services**:

* SNAPmobile Web Connections on ATL, LAS &amp; GRR

**Degraded Services**:

* None.</p>
<p><small>Apr <var data-var='date'> 25</var>, <var data-var='time'>23:35:00</var> GMT+0</small><br /><strong>Identified</strong> -
  At 6:19PM ET we were alerted by our monitoring tools that WSS connections on ATL, GRR &amp; LAS were failing. SNAPmobile Web uses WSS for the SIP connection.

What Is Affected: All SNAPmobile Web Connections on ATL, GRR &amp; LAS

When It Began: 6:19PM ET

Current Status:

We actively engaged with our vendor and awaiting to confirm the issue and implement a fix.

Next Steps:

No next steps for partners.

Next Update: 8:45PM ET

Offline Services:

SNAPmobile Web Connections on ATL, LAS &amp; GRR

Degraded Services:

None.</p>
<p><small>Apr <var data-var='date'> 26</var>, <var data-var='time'>00:43:37</var> GMT+0</small><br /><strong>Identified</strong> -
  At 6:19PM ET we were alerted by our monitoring tools that WSS connections on ATL, GRR &amp; LAS were failing. SNAPmobile Web uses WSS for the SIP connection.

**What Is Affected**: All SNAPmobile Web Connections on ATL, GRR &amp; LAS

**When It Began**: 6:19PM ET

**Current Status**:

* We have confirmed that SNAPmobile Web on GRR &amp; LAS cores are registering successfully.
* We are still working with our vendor to resolve an SSL issue relating to our ATL core.

**Next Steps**:

* If your SNAPmobile Web phones were previously offline on GRR or LAS, please refresh your browser, or relaunch the SNAPmobile Web app to regain registration.

**Next Update**: 9:45PM ET

**Offline Services**:

* SNAPmobile Web Connections on ATL

**Degraded Services**:

* None.</p>
<p><small>Apr <var data-var='date'> 26</var>, <var data-var='time'>01:23:17</var> GMT+0</small><br /><strong>Resolved</strong> -
  At 6:19PM ET we were alerted by our monitoring tools that WSS connections on ATL, GRR &amp; LAS were failing. SNAPmobile Web uses WSS for the SIP connection.**What Is Affected**: All SNAPmobile Web Connections on ATL, GRR &amp; LAS**When It Began**: 6:19PM ET**Current Status**:

* We have confirmed that SNAPmobile Web Registrations have been resolved on all cores.

**Next Steps**:

* Major Incident Report will be available within 48 business hours.

**Next Update**: N/A

**Offline Services**: N/A

**Degraded Services**: N/A.</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cm9txj3dk002ygx0avhso28ph</id>
  <published>2025-04-23T12:48:07.982+00:00</published>
  <updated>2025-04-23T12:48:07.982+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cm9txj3dk002ygx0avhso28ph"/>
  <title>Websocket Connection Failure on GRR and LAS</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 2 hours</p>
    <p><strong>Affected Components:</strong> SNAPmobile Web</p>
    <p><small>Apr <var data-var='date'> 23</var>, <var data-var='time'>12:48:07</var> GMT+0</small><br /><strong>Investigating</strong> -
  **What Occurred**: At 7:42AM ET we were alerted by our monitoring tools that WSS connections on GRR &amp; LAS were failing. SNAPmobile Web uses WSS for the SIP connection. We were able to confirm this within 15 minutes of the alert and identify a possible cause thereafter.

**What Is Affected**: All SNAPmobile Web Connections on GRR &amp; LAS

**When It Began**: 7:42AM ET

**Current Status**:

* We have identified a potential cause for this, and are awaiting vendor confirmation to implement a fix.

**Next Steps**:

* No next steps for partners

**Next Update**: 9:45AM ET

**Offline Services**:

* SNAPmobile Web Connections on LAS &amp; GRR

**Degraded Services**:

* None.</p>
<p><small>Apr <var data-var='date'> 23</var>, <var data-var='time'>13:50:19</var> GMT+0</small><br /><strong>Investigating</strong> -
  **What Is Affected**: All SNAPmobile Web Connections on GRR &amp; LAS

**When It Began**: 7:42AM ET

**Current Status**:

* The attempted resolution did not resolve the connections. We are working with our vendor and investigating further.

**Next Steps**:

* If web phone connectivity is vital and it is not failing over, please input a support case by emailing [support@oit.co,](mailto:support@oit.co) and a technician can reroute your connections to ATL temporarily

**Next Update**: 10:45AM ET

**Offline Services**:

* SNAPmobile Web Connections on LAS &amp; GRR

**Degraded Services**:

* None.</p>
<p><small>Apr <var data-var='date'> 23</var>, <var data-var='time'>14:48:35</var> GMT+0</small><br /><strong>Resolved</strong> -
  **What was Affected**: All SNAPmobile Web Connections on GRR &amp; LAS

**When It Began**: 7:42AM ET

**Current Status**:

* **This incident is considered Resolved**
* Our vendor resolved an SSL issue with the affected FQDNs

**Next Steps**:

* Major Incident Report will be available within 48 hours.

**Next Update**: N/A

**Offline Services:** N/A

**Degraded Services:** N/A.</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cm9iw0mqr008an6vnr1okp2hb</id>
  <published>2025-04-23T03:00:00.000+00:00</published>
  <updated>2025-04-23T03:00:01.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cm9iw0mqr008an6vnr1okp2hb"/>
  <title>Fax Server Maintenance: 04/22/25 11:00 pm ET</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 4 hours</p>
    <p><strong>Affected Components:</strong> mFax</p>
    <p><small>Apr <var data-var='date'> 23</var>, <var data-var='time'>03:00:01</var> GMT+0</small><br /><strong>Identified</strong> -
  Maintenance is now in progress.</p>
<p><small>Apr <var data-var='date'> 23</var>, <var data-var='time'>07:00:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>
<p><small>Apr <var data-var='date'> 23</var>, <var data-var='time'>03:00:00</var> GMT+0</small><br /><strong>Identified</strong> -
  We will be performing security maintenance on our fax server on April 22nd, 2025, at 11:00pm ET. There will be an interruption in the delivery and sending of faxes while maintenance is performed.

Maintenance Start: 11:00pm, April 22nd, 2025, ET

Estimated Time to Complete: 4 hours, 3:00am April 23rd, 2025, ET

Offline Services  
• Fax.</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Maintenance/cm9k6jebb003gmguuhzofuo1s</id>
  <published>2025-04-16T22:00:00.000+00:00</published>
  <updated>2025-04-18T05:00:00.000+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/maintenance/cm9k6jebb003gmguuhzofuo1s"/>
  <title>Voipmonitor Maintenance 04/16/25</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 1 day and 6 hours</p>
    <p><strong>Affected Components:</strong> VoIPMonitor (QoS)</p>
    <p><small>Apr <var data-var='date'> 18</var>, <var data-var='time'>05:00:00</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance has completed successfully.</p>
<p><small>Apr <var data-var='date'> 16</var>, <var data-var='time'>23:00:00</var> GMT+0</small><br /><strong>Identified</strong> -
  We will be performing emergency security maintenance for Voipmonitor on April 16th at 7 PM EST. During this time, there may be a brief interruption to QoS scores showing in the portal, and any calls being conducted at that time may have reduced diagnostic data attached. There will not be any interruption to calls or device registration.

**Maintenance Start:** 7:00 PM, April 16th, ET

**Estimated Time to Complete:** 30 minutes, 7:30 PM, April 16th, ET

**Offline Services:** Voipmonitor.</p>
<p><small>Apr <var data-var='date'> 16</var>, <var data-var='time'>23:00:01</var> GMT+0</small><br /><strong>Completed</strong> -
  Maintenance is now in progress.</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cm9h8ejw6006wmuhbrf5f8cft</id>
  <published>2025-04-14T15:31:29.682+00:00</published>
  <updated>2025-04-15T13:15:51.823+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cm9h8ejw6006wmuhbrf5f8cft"/>
  <title>4/14/2025 Call Recording Failure</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 21 hours and 44 minutes</p>
    <p><strong>Affected Components:</strong> Call Recording (PHX), Call Recording (IAD)</p>
    <p><small>Apr <var data-var='date'> 15</var>, <var data-var='time'>13:15:51</var> GMT+0</small><br /><strong>Resolved</strong> -
  \*\*Current Status:\*\* \* \*\*This incident is considered Resolved\*\* \* After testing and confirmation, all call recordings on GRR &amp; LAS are now available, and call recordings on ATL are available \*\*except between 4/14 2 AM and 4/15 3:30 AM ET.\*\* \* If a specific call is needed from this time period, you can submit a ticket to \[support@oit.co\](mailto:support@oit.co) within the \*\*next 6 calendar days\*\* and we can pull that call recording from voipmonitor for you. \*\*Next Steps:\*\* \* Major Incident Report will be available within 48 hours. \*\*Next Update:\*\* N/A.</p>
<p><small>Apr <var data-var='date'> 15</var>, <var data-var='time'>07:25:29</var> GMT+0</small><br /><strong>Monitoring</strong> -
  \*\*Update: Monitoring: 4/14/25 Call Recording Failure\*\* \*\*What Is Affected\*\*: Call Recording availability across affected tenants \*\*When It Began\*\*: 2:00 AM, April 14, 2025, ET \*\*Current Status\*\*: \* As of 2:45 AM ET on April 15th, call recordings for calls on ATL placed after tonight&#039;s maintenance window are showing in the Manager Portal. \*\*Next Steps\*\*: \* Call recordings for calls on ATL placed throughout the duration of the incident (2 AM ET on April 14th through 1 AM ET on April 15th) can be retrieved by submitting a support ticket to \[support@oit.co\](mailto:support@oit.co). \* We will continue to monitor for the next 48 hours to ensure call recording functionality remains stable. \*\*Next Update\*\*: 9 AM ET, April 16th, 2025 \*\*Offline Services\*\*: \* None \*\*Degraded Services\*\*: \* None \*\*Operational Services\*\*: \* Inbound Calling – ATL \* Outbound Calling - ATL \* Inbound Calling – GRR \* Outbound Calling - GRR \* Inbound Calling – LAS \* Outbound Calling - LAS \* API \* Fax \* Messaging \* Teammate Connector \* Mobile X \* Mobile Apps \* Integrations We appreciate your patience throughout this process. If you need support, please contact \[support@oit.co\](mailto:support@oit.co). For more real time status updates as well as discussion please join our Discord (Not official support).</p>
<p><small>Apr <var data-var='date'> 14</var>, <var data-var='time'>15:31:29</var> GMT+0</small><br /><strong>Investigating</strong> -
  At approximately 2:00 AM ET on April 14th, 2025, we completed maintenance on our call recording servers to migrate them to the new data centers in IAD and PHX. At 10:20 AM ET, we identified an issue where call history is not displaying call recordings. Remote call recording storage may also be unavailable, but no other functionality appears to be impacted at this time. --- \*\*What Is Affected:\*\* \* \*\*Call Recording\*\* availability across affected tenants --- \*\*When It Began:\*\* 2:00 AM, April 14, 2025, ET --- \*\*Current Status:\*\* \* \*\*\\\[10:45 AM ET, April 14\\\]\*\* Issue was confirmed by support team; it is currently under investigation by engineering \* No known workarounds at this time \* Engineering has been engaged to review recording service logs and assess potential causes --- \*\*Next Steps:\*\* \* \*\*\\\[10:45 AM ET, April 14\\\]\*\* The Engineering team is reviewing logs and service dependencies \* Partners do not need to take action at this time \* We will provide an update once the root cause is identified, or a workaround is available --- \*\*Next Update:\*\* 12:30 PM, April 14, 2025, ET --- \*\*Offline Services:\*\* \* Call Recording --- \*\*Degraded Services:\*\* \* None.</p>
<p><small>Apr <var data-var='date'> 14</var>, <var data-var='time'>16:28:31</var> GMT+0</small><br /><strong>Investigating</strong> -
  **Current Status:** We&#039;ve engaged vendor support to develop a short-term solution while we determine the root cause.

---

**Next Steps:**

* Partners do not need to take action at this time
* We will provide an update for the next steps once the root cause is identified or a workaround is available

---

**Next Update:** 1:30 PM, April 14, 2025, ET.</p>
<p><small>Apr <var data-var='date'> 14</var>, <var data-var='time'>17:46:40</var> GMT+0</small><br /><strong>Identified</strong> -
  **Current Status:**

* Call recordings are now available for LAS and GRR.
* This includes past, current and future call recordings, and recordings that took place during the incident timeline.

---

**Next Steps:**

* We are still investigating, as ATL call recordings are still unavailable at this time. Call recordings for ATL are not accessible, this includes active/future call recordings.
* Partners do not need to take action at this time

---

**Next Update:** 2:30 PM, April 14, 2025, ET.</p>
<p><small>Apr <var data-var='date'> 14</var>, <var data-var='time'>19:28:43</var> GMT+0</small><br /><strong>Identified</strong> -
  **Current Status:**

\-Call recordings are available for LAS and GRR.

\-This includes past, current, and future call recordings, including recordings that took place during the incident timeline.

**Next Steps:**

\-We continue to troubleshoot ATL with the help of our vendors. Call recordings are still unavailable for past, active, and future recordings on ATL at this time.

\-Partners do not need to take action

**Next Update:** 4:30 PM, April 14, 2025, ET.</p>
<p><small>Apr <var data-var='date'> 14</var>, <var data-var='time'>20:31:32</var> GMT+0</small><br /><strong>Identified</strong> -
  **Current Status:**

\-Call recordings are available for LAS and GRR.

 \-This includes past, current, and future call recordings, including recordings that took place during the incident timeline. 

**Next Steps:**

\-We continue to troubleshoot ATL with vendor support. Call recordings are still unavailable for past, active, and future recordings on ATL at this time. 

\-Partners do not need to take action 

**Next Update:** 5:30 PM, April 14, 2025, ET.</p>
<p><small>Apr <var data-var='date'> 14</var>, <var data-var='time'>18:29:38</var> GMT+0</small><br /><strong>Identified</strong> -
  **Current Status:** 

* Call recordings are available for LAS and GRR.
* This includes past, current and future call recordings, and recordings that took place during the incident timeline.

---

**Next Steps:**

* We are still investigating ATL call recordings. ATL recordings are still unavailable at this time, including active/future call recordings.
* Partners do not need to take action at this time

---

**Next Update:** 3:30 PM, April 14, 2025, ET

---.</p>
<p><small>Apr <var data-var='date'> 14</var>, <var data-var='time'>21:30:29</var> GMT+0</small><br /><strong>Identified</strong> -
  We will be performing emergency maintenance on our ATL server on April 15th, at 1:00 AM ET, to resolve the call recording failures on ATL. 

During this time, there will be a brief window where calls and device registration will be rerouted to alternate servers.

**Maintenance Start:** 1:00 AM ET, April 15th, 2025

**Estimated Time to Complete:** 4:00 AM ET, April 15th, 2025 (3 Hours)

**Offline Services**

Inbound Calling – ATL 

Outbound Calling -ATL 

**Degraded Services**

None

**Operational Services**

Inbound Calling – GRR 

Outbound Calling -GRR 

Inbound Calling – LAS 

Outbound Calling - LAS 

API 

Fax 

Messaging 

Teammate Connector

Mobile X

Mobile Apps 

Integrations 

We appreciate your patience throughout this process. For additional support, please contact [support@oit.co](mailto:support@oit.co)..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cm7jkija8013r10iislye7vd7</id>
  <published>2025-02-24T21:26:40.928+00:00</published>
  <updated>2025-02-24T21:26:40.944+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cm7jkija8013r10iislye7vd7"/>
  <title>API outage</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 3 minutes</p>
    
    <p><small>Feb <var data-var='date'> 24</var>, <var data-var='time'>21:26:40</var> GMT+0</small><br /><strong>Investigating</strong> -
  API cannot be accessed at the moment. This incident was created by an automated monitoring service..</p>
<p><small>Feb <var data-var='date'> 24</var>, <var data-var='time'>21:29:40</var> GMT+0</small><br /><strong>Resolved</strong> -
  API is now operational! This update was created by an automated monitoring service..</p>

        ]]>
  </content>
</entry>

<entry>
  <id>tag:status.oit.co,2005:Incident/cm7jgdvee00avmf56aids1xrr</id>
  <published>2025-02-24T19:31:04.808+00:00</published>
  <updated>2025-02-24T19:31:04.910+00:00</updated>
  <link rel="alternate" type="text/html" href="https://status.oit.co/incident/cm7jgdvee00avmf56aids1xrr"/>
  <title>API outage</title>

  <content type="html">
  <![CDATA[
    <p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 2 minutes</p>
    
    <p><small>Feb <var data-var='date'> 24</var>, <var data-var='time'>19:31:04</var> GMT+0</small><br /><strong>Investigating</strong> -
  API cannot be accessed at the moment. This incident was created by an automated monitoring service..</p>
<p><small>Feb <var data-var='date'> 24</var>, <var data-var='time'>19:32:39</var> GMT+0</small><br /><strong>Resolved</strong> -
  API is now operational! This update was created by an automated monitoring service..</p>

        ]]>
  </content>
</entry>

</feed>