Reopened: Intermittent 504 Errors in Nylas Due to Upstream Google 502 Responses
Incident
Report for Nylas v3
Subscribe to Updates
Update
Over the past 24 hours, we have continued to observe normal levels of 502 errors from Google Workspace. In fact, the error rate has decreased by more than 5x compared to the period prior to the initial incident.
We are still awaiting confirmation from Google regarding the root cause and any remediations that were applied. As such, we will keep this incident in Monitoring status until we receive more information.
We will continue to monitor closely and provide updates as new information becomes available.
Thank you for your continued patience and understanding.
We are still awaiting confirmation from Google regarding the root cause and any remediations that were applied. As such, we will keep this incident in Monitoring status until we receive more information.
We will continue to monitor closely and provide updates as new information becomes available.
Thank you for your continued patience and understanding.
Monitoring
As of 11:00 AM PT on July 29, Nylas observed that the rate of Google 502 errors had returned to normal levels. Given prior instances where error rates temporarily subsided before spiking again, we continued active monitoring and reached out to Google Workspace Support for confirmation.
At 11:25 PM PT on July 29, Google Support contacted us to ask whether we were still seeing improvement. We confirmed that error rates remained stable and again requested confirmation of any changes made on their end, along with a formal root cause.
We are now moving the incident status to Monitoring and will continue to track the situation closely while pressing Google for additional details.
Thank you for your continued patience and understanding.
At 11:25 PM PT on July 29, Google Support contacted us to ask whether we were still seeing improvement. We confirmed that error rates remained stable and again requested confirmation of any changes made on their end, along with a formal root cause.
We are now moving the incident status to Monitoring and will continue to track the situation closely while pressing Google for additional details.
Thank you for your continued patience and understanding.
Update
We are observing a new spike in error rates related to Google’s services, which may result in 504 responses from Nylas APIs.
Our team is actively monitoring the situation and remains engaged with Google Workspace Support to obtain clarification and work toward a permanent resolution.
We will continue to provide updates as we learn more or receive additional information from Google. Thank you for your patience and understanding.
Our team is actively monitoring the situation and remains engaged with Google Workspace Support to obtain clarification and work toward a permanent resolution.
We will continue to provide updates as we learn more or receive additional information from Google. Thank you for your patience and understanding.
Update
Error rates have returned to normal levels and have remained stable, even during peak periods. However, we are keeping this incident open until we receive formal confirmation and additional details from Google.
Our team continues to monitor the situation closely and is in active communication with Google Workspace Support to ensure a permanent resolution.
Thank you for your continued patience and understanding. We will share further updates as soon as we have new information.
Our team continues to monitor the situation closely and is in active communication with Google Workspace Support to ensure a permanent resolution.
Thank you for your continued patience and understanding. We will share further updates as soon as we have new information.
Identified
We have reopened this incident after observing a recurrence of intermittent 502 errors from Google, which surface as 504 Gateway Timeout responses in the Nylas API. While the error volume is lower than during the initial incident, it has again reached unacceptable levels during peak traffic hours.
These upstream errors are impacting message send operations for a subset of users. Most requests succeed on retry; however, we are now also receiving reports of some messages being sent despite a 502 response from Google, which may result in unintended duplicates when retries occur.
Our engineering team is actively monitoring the situation and has re-engaged Google Workspace Support. We are escalating through all available channels and providing additional diagnostic data at their request to help identify the root cause and prevent further disruptions.
We will continue to share updates as we learn more or receive further information from Google.
Thank you for your continued patience and understanding.
These upstream errors are impacting message send operations for a subset of users. Most requests succeed on retry; however, we are now also receiving reports of some messages being sent despite a 502 response from Google, which may result in unintended duplicates when retries occur.
Our engineering team is actively monitoring the situation and has re-engaged Google Workspace Support. We are escalating through all available channels and providing additional diagnostic data at their request to help identify the root cause and prevent further disruptions.
We will continue to share updates as we learn more or receive further information from Google.
Thank you for your continued patience and understanding.
This incident affects: Nylas Application Services (API v3) and 3rd Party Providers (Google).