Concurrency Limits

Concurrency Limits Overview

ICE Mortgage Technology enforces concurrent rate limits, which is a limit on the number of simultaneous API calls that are made at a given time.

Each lender has a hard stop limit of concurrent API calls that is set for each Encompass clientId.  The default limit of 30 is set for each lender clientId.

This limit is applied across all Encompass instances rolled up to the clientId. If the lender has multiple Encompass instances (e.g., test and production) tied to the same clientId, the API concurrency limit is distributed across all associated Encompass instances. If the lender's hard stop limit of concurrent API calls is utilized, then all processes (partner and lender initiated calls) are not allowed.

Working with Concurrency Limits

The header of each API response has two concurrency key/value pairs including X-Concurrency-Limit-Limit and X-Concurrency-Limit-Remaining. This information tells the API consumer their allocated concurrency limit, along with the remaining concurrency at the given time.

For lenders and partners that, by the nature of their service and integration, are expected to make a large volume of calls to a lender’s instance (event based services, downloading and uploading large volumes of attachments, use of threading and parallel processing, etc.) it is important to design the integration in a way that ensures that the API caller is able to limit and configure the number of concurrent calls made to a lender’s environment. If the lender's hard stop limit of concurrent API calls is utilized, then all processes (partner and lender initiated calls) are not allowed, a 429 - Too Many Requests status code is returned.  At this point, the lender or any Integration Applications cease to execute. The 429 - Too Many Requests status code will continue to be returned until the concurrency remaining rate is greater then 0.

Best practice is to use an exponential back-off in concurrency violations (the concurrency limit has been reached) and proactive throttling (the available concurrency is outside an acceptable range) and not to exceed 20 percent of the allow Concurrency Limit.

The header of each API response has two concurrency key/value pairs: X-Concurrency-Limit-Limit and X-Concurrency-Limit-Remaining. The Integration Application can use these to calculate the ratio. (X-Concurrency-Limit-Remaining/X-Concurrency-Limit-Limit). If this ratio is below 20 percent, the Integration Application will need to incorporate a proactive throttling process to increase the X-Concurrency-Limit-Remaining. The ratio can also be a configuration in your Integration Application.

Best Practices for Efficient Use of API Concurrency

The best way to avoid being throttled is to reduce the number of simultaneous requests, this can normally be achieved through code optimization.

Consider the following practices when designing your app/integration:

• Cache frequently used data rather than fetching it repeatedly
• Batch requests when processing webhook notifications
• Eliminate unnecessary and redundant API requests
• Use a message queue to regulate call frequency
• Implement an exponential back-off to gracefully recover from throttle errors
• Be cautious with timeouts – concurrency is a factor of how long requests take to execute, not how long your app waits for responses
• Use metadata, included with API responses, to meter and prioritize requests
• Reduce latency by initiating requests as a super administrator – fewer business rules are applied with this type of account

Are you using the right automation for your use case?

• Webhooks, List APIs, Batch Calls - choose the right tool for your use case
• Use webhook events to listen for changes instead of polling via GET calls
• Get multiple attachments in a single call
• Leverage the Pipeline API to avoid multiple requests to retrieve loans
• Take advantage of Enhanced Field Change Events (GA in 24.2) which ALREADY includes the changed data

Options Available to Increase Limit

For clients requiring an increase in API concurrency limits (e.g. those receiving 429 error responses or anticipate heavy API use), options to increase API concurrency limits are available and may be adjusted as needed.
Please contact your ICE Mortgage Technology relationship manager or support team to discuss concurrency limits and pricing.