How do I know if a failure is token expiry vs. a permissions change?

Read the error body, not just the status code — both cases can surface as a 401. An expired or revoked token comes back as an invalid-session or invalid-grant style error at the token or API endpoint, and it fails for every call the integration makes, including trivial ones like fetching the current user. A permissions change is narrower: the token still authenticates, identity endpoints still work, and only specific objects, fields, or operations fail with an insufficient-access style error. The fastest discriminator is a cheap identity call — if that succeeds and your business call fails, it is authorization, not authentication. If both fail, it is the token. Then check whether the failure hit every connected org at once (something changed on your side — a rotated client secret, a bad deploy, a clock problem) or one org only (something changed in that customer's org — an admin revoked the connected app, deactivated the integration user, or tightened the IP or session policy).

Should I refresh proactively or reactively?

Proactively, with a reactive path kept as a backstop. Proactive refresh means you store the token's expiry timestamp when you receive it and refresh on a safety margin before that time, so the token in hand is always valid when a request goes out. Reactive-only refresh — send the request, catch the 401, refresh, retry — looks simpler but fails badly under concurrency: ten in-flight workers all get a 401 at the same instant, all attempt a refresh, and with rotation enabled nine of them invalidate each other's tokens. Do both: refresh ahead of expiry as the normal path, and keep a single-flight, lock-guarded reactive refresh for the cases proactive refresh cannot cover, such as an admin resetting the token early. The critical detail in either mode is that only one refresh may be in flight per credential at a time.

Does this differ for HubSpot or NetSuite OAuth?

The mechanics are the same because they are properties of OAuth 2.0, not of Salesforce — you still store a refresh token, exchange it for short-lived access tokens, handle rotation, and treat revocation as a re-auth event rather than a retryable error. The details that differ are the ones you must never hardcode from memory: access token lifetime, whether refresh tokens rotate on every exchange, whether they expire on their own after a period of inactivity, and what the error payload looks like when a grant is dead. Check each provider's current documentation for those four values, keep them in configuration rather than in code, and your refresh layer transfers between providers with only the token endpoint and the error-string mapping changing.

Why your Salesforce integration suddenly stops authenticating (OAuth token refresh, done properly)

An integration that ran fine for months starts throwing 401s overnight. The cause is almost always refresh token handling — expiry, revocation, rotation, or clock skew. Here's the diagnosis path and the refresh pattern that survives production.

The integration worked. That is what makes this failure mode so annoying. It was built, tested, demoed, signed off, and then it ran quietly in production for six weeks or six months, syncing records nobody thought about. Then one morning every job in the queue is failing with a 401, and the last code change was a month ago.

Nothing broke. A token reached the end of its life and the integration had no working plan for that moment.

This is the single most common cause of "it just stopped working" in Salesforce integrations I get called into, and it is almost never the interesting part of the system. It is the plumbing underneath it — a refresh path written in an afternoon, tested against a fresh token that had hours of life left, and never exercised under the exact conditions that eventually kill it.

The symptom

The shape is consistent enough to recognise on sight:

Everything worked, unchanged, for weeks or months.
Failures started all at once, not gradually.
Every call fails, including the trivial ones — not just writes, not just one object.
The last deploy was nowhere near the failure.
Re-running the OAuth flow by hand fixes it instantly, which everyone takes as proof that the credentials are fine.

That last point is the trap. Manually reconnecting fixes it because it mints a brand-new grant. It tells you nothing about why the old grant died, and it guarantees you will be back here in another six weeks. Teams often re-auth two or three times before anyone treats it as a bug rather than an errand.

The other tell is what the failures look like in aggregate. If every customer org fails simultaneously, something changed on your side: a rotated client secret, a redeployed environment that lost its token store, a container whose clock drifted. If exactly one org fails and the rest are healthy, something changed in that org: an admin revoked the connected app, the integration user was deactivated or had their password reset, or a session or IP policy tightened.

A ten-minute diagnosis path

Before changing any code, work through these in order. Each one eliminates a whole class of cause, and together they usually name the culprit inside ten minutes.

Read the full error body from the failing call. Not the status code, not your wrapper's message — the raw payload the provider returned. Expired sessions, dead grants, and permission problems all arrive as 401s and are distinguished only by their bodies.
Make one cheap identity call with the current token. If it succeeds, authentication is fine and you are chasing an authorization or data problem. If it fails, the token is the issue.
Attempt exactly one manual refresh using the stored refresh token, by hand, and read that response. A refresh that returns an invalid-grant error tells you the grant is dead; a refresh that succeeds tells you your code's refresh path is broken, not your credentials.
Check the blast radius. One org failing means something changed in that org. Every org failing means something changed on your side.
Check the host clock against a known-good time source, on the machine that actually runs the job rather than your laptop.
Read the connected app's refresh token policy and the org's session settings as they are configured today, not as they were configured when the integration was built. Both are editable by an admin, and both change behaviour retroactively for tokens already issued.

Whatever those six steps point at will be one of the four causes below.

Why it happens

There are four root causes, and they need different fixes.

1. The refresh token expired or was never long-lived

Access tokens are deliberately short-lived. The refresh token is the durable credential, and the mistake is assuming durable means permanent. A connected app's refresh token policy decides this, and one of the available policies expires the refresh token after a period of inactivity. That policy is a landmine for integrations that run in bursts — a sync that fires nightly is fine, a sync that only runs when a customer uploads a file may go quiet for longer than the inactivity window and find its grant dead when it wakes up.

Verify the actual policy on the connected app rather than assuming. An integration whose refresh token expires on inactivity needs either a policy change or a deliberate keep-alive.

2. The refresh token was revoked

Revocation is not decay; it is somebody making a decision. An admin removed the connected app's access, the integration user was deactivated as part of an offboarding sweep, the user's password was reset, or an org-wide security policy changed.

Revocation is unrecoverable by retry. No amount of backoff brings a revoked grant back. It requires a human to re-authorize, and the only useful thing your code can do is recognise it, stop hammering the endpoint, and tell somebody.

3. Refresh token rotation was not handled

If the provider issues a new refresh token alongside each access token, then the old refresh token is spent the moment you use it. An integration that stores the refresh token once at connection time and never updates it will work exactly once after each rotation and then hold a dead credential.

Worse, the failure is often delayed and confusing: the process that did the refresh has a valid access token in memory and keeps working until it restarts, at which point it falls back to the stale stored refresh token and dies. So the outage appears to be caused by a deploy that merely restarted the process.

Rotation also interacts badly with concurrency. Two workers refreshing at the same moment produce two rotations; one worker's freshly issued token is invalidated by the other's, and whichever write to your token store lands second wins. Both may lose.

4. Clock skew

Token expiry is arithmetic on timestamps, and it goes wrong in both directions. A container whose clock runs fast treats a valid token as expired and refreshes far more often than necessary — which, with rotation on, multiplies the chance of a rotation race. A clock running slow does the opposite: the code believes the token is good, sends the request, and gets a 401 that the reactive path may or may not handle.

Skew also affects flows that sign an assertion with a timestamp, where the provider rejects anything outside an allowed window. Those failures look like credential problems and are really an NTP problem.

The fix pattern

The goal is an integration where token expiry is an ordinary, boring event that never surfaces as a failed job, and where revocation surfaces as a clear, actionable alert instead of a retry storm.

Refresh proactively, on a margin

Store the expiry timestamp you were given at issue time, not a lifetime you assumed. Before any outbound call, check whether the token expires within a safety margin — minutes, not seconds, generously sized so that a slow request and a modest clock drift can both be absorbed — and refresh first if it does.

Compute the deadline once, from the provider's response, and store it as an absolute timestamp in UTC. Do not recompute it from a hardcoded constant, because the lifetime is configurable per org and will not match your constant everywhere.

Make refresh single-flight

Only one refresh may be in progress per credential at a time. Take a lock keyed on the credential, and have every other caller wait for the winner's result rather than starting a second refresh. If your integration runs across multiple processes or machines, that lock has to be distributed — a row lock, an advisory lock, or a short-lived lease in Redis — because a per-process mutex protects nothing in a horizontally scaled worker pool.

Inside the lock, re-check expiry before refreshing. By the time a waiter acquires the lock, the token it was worried about has usually already been replaced.

Persist the new refresh token in the same transaction

If the provider returns a new refresh token, write it immediately, atomically, and before you act on the access token. Treat the token store as the source of truth and memory as a cache, not the reverse. A crash between "refreshed" and "saved" is exactly how integrations end up holding a credential that no longer exists on the provider's side.

Keep a reactive path, but a narrow one

Proactive refresh cannot cover everything — an admin can reset a session while your token still looks valid. So keep the 401 path, but make it strict: refresh at most once per request, through the same single-flight lock, and retry the original call exactly one time. If the retry fails, stop. An unbounded refresh-and-retry loop against a dead grant is how you get rate-limited on top of being broken.

Distinguish retryable from terminal

This is the part most implementations skip, and it is the part that decides whether an outage lasts ten minutes or a week.

Transient (network error, timeout, rate limit, provider 5xx): retry with backoff and jitter. The credential is fine.
Expired access token: refresh once, retry once. Routine.
Dead grant (revoked, expired refresh token, rotation lost): terminal. Stop retrying, mark the connection as needing re-authorization, and alert.

Map these off the error body, not the status code alone, and keep the mapping in one place so it can be corrected when a provider changes its wording.

Treat re-auth as a product feature, not an incident

A dead grant needs a human, so build the path that human uses before you need it: a connection status the customer or your ops team can see, an alert when a connection flips to needing re-authorization, and a one-click reconnect that runs the authorization flow again and replaces the stored grant in place.

The alternative — silent failure — is the actual damage. Records stop syncing, nobody notices for days, and the recovery is not a reconnect but a backfill, with all the duplicate-record risk that a replayed sync brings.

Store the credential like the secret it is

The refresh token is a long-lived key to a customer's CRM. Encrypt it at rest, keep it out of logs and out of error payloads (log a fingerprint, never the value), scope access to the service that needs it, and make revoking and reissuing it a supported operation rather than a database surgery.

Fix the clock

Run NTP. Verify it is actually running on every host and container image, including the ones nobody thinks of as servers. Skew is cheap to eliminate and produces failures that look like everything except what they are.

How to verify it before it fails in production

Three tests catch nearly all of this, and none require waiting six weeks:

Force expiry. Set the stored expiry to a time in the past and run a job. It should refresh once, silently, and succeed.
Force revocation. Revoke the connected app's access in the provider, then run a job. It should fail once, mark the connection as needing re-authorization, alert, and not retry in a loop.
Force concurrency. Expire the token and fire a dozen jobs simultaneously. Exactly one refresh should be issued and every job should succeed. If you see several refreshes, your locking is not doing what you think it is — and under rotation, that is the bug that will bite you.

Run all three in staging against a real connected app with the same refresh token policy as production. A test against a policy you do not use in production proves very little.

The takeaway

Nothing here is exotic. Refresh before expiry rather than after, allow exactly one refresh at a time per credential, persist rotated tokens the instant you receive them, tell transient failures apart from dead grants, and give a dead grant a human-facing path back to health.

An integration built that way does not stop working after six weeks. It refreshes, keeps going, and asks for help clearly on the rare day it genuinely needs a person.

If you have a Salesforce, HubSpot, or NetSuite integration that keeps needing to be manually reconnected, that is not maintenance — it is an unfinished auth layer, and it is usually a day or two of work to finish it properly. That work is part of what I do in enterprise integrations, and it pairs closely with the sync-correctness problems covered in why your Salesforce/NetSuite sync creates duplicate records — because an integration that fails silently for a week and then replays its backlog is exactly how duplicates get created in the first place.