API should return updated data in real-time & simplify updates

The way the Intercom API currently works makes it complicated to reliably create and update contacts.

Example 1: Newsletter sign-up

1. List all contacts with that email address.
2. Update existing contact or create new one as needed:
   1. If there is a user then update that user.
   2. Otherwise if there is a lead then update that lead.
   3. Otherwise create a new lead.
3. If an existing contact was updated then also remove an eventual subscription opt-out and "unsubscribe all" flag for that contact.

Step 2 can create duplicate leads if, for example:

• A user was created very recently before the newsletter sign-up (e.g. user created by product sign-up and then they sign up for the newsletter shortly after). Thus the user is not yet returned by the API (delayed) and a new lead created.
• A lead was created very recently for the newsletter sign-up but the person signs up again shortly after (e.g. because of a network issue). Thus the first lead is not yet returned by the API (delayed) and a new one created.
• The network connection between our server and Intercom failed. Thus we don't know that the lead was created successfully and a retry will create a duplicate lead.

Example 2: Product sign-up (first lead, then user)

1. A person gets a price quote and provides their email address.
2. Create a lead in Intercom.
3. The person signs up for the product (typically with same email address).
4. Find an existing contact by email address and convert it into a user.
5. If there is no lead with that email address then create a new user instead.

Step 5 can create duplicate users if, for example:

• The time between getting the quote and signing up for the product is short enough. In that case our server queries for leads by email address but incorrectly receives 0 contacts. Thus a new user will be created.
• The network connection between our server and Intercom failed. Thus we don't see that the user was created successfully and a retry will create a duplicate user.

Another example is when two systems run independently, for example the newsletter logic in its own service and the product service in another. They might interfere with each other's read-and-write processes leading to race conditions and duplicate contacts. Usually those cases are rare but the significant delay can make this very common.

Another example is that we have "sync all" instead of "sync only changes" logic. Various actions performed by our customers trigger a "re-sync Intercom user object with our latest user data" in order to improve data consistency. Occasionally we also re-sync everything because data tends to get our of sync (race conditions, sync errors). Since that can cause plenty of writes we basically skip a write if the contact received from Intercom already has the same properties as the data we want to write. However that optimization causes a loss of data because the data read from Intercom is sometimes stale.

Another example is reading data after receiving a webhook ping. This can also cause stale data to be read because webhook the data might not be sufficient.

Another example is that after sign-up the user is logged in to the front-end JS SDK which creates a user. The back-end fails to see the existing user and thus can neither create a new one nor update the existing one (because of stale data - user doesn't exist yet).

Many user cases require a read before a write to create or update information in Intercom conditionally.

A potential solution for all cases could be to basically mirror the contact database of Intercom on our side, use our database for reading, and only send writes to Intercom. That would cause a lot of work on our side for something that could be solved on the Intercom API level.

Another (partial) solution could be to wait with reading data before writing (or processing webhooks). That might be up to a few minutes to catch all delays. That however would significantly delay the data freshness in all systems, e.g. in the Intercom UI of the support team which is very important for effective customer support (e.g. customer asks "Did my order go through?"/"Did XYZ work?"/"Is XYZ data correct?").

Another (partial) solution could be to enable data updates that conditionally either create or update an object. MongoDB is a good example here which has "upserts": update if exists, create otherwise using additional data of $setOnInsert.

Long story short, we we've experienced plenty of headaches working with Intercom API so far. Many of them are caused by the unnecessarily high delay between writes and subsequent reads of the written data. It would be great if the API could be improved to reduce the effort and learning curve on the API consumer side.