The purpose of this feature is to allow administrators to selectively control which API calls are throttled, without having to rebuild and deploy a new version every time a throttle is changed.

The current frequency throttle is only able to throttle each user's total call frequency using a semaphore. The InMemoryTimeBlockCountingSemaphore is a map from a key(user id in this case) to a SimpleSemaphore which keeps a count of calls made and the the after which the count will reset. The UserThrottleFilter then compares the user's call count to the maximum allowed.

The new throttle will use a similar mechanism, except that the keys will be userId+normalizedAPI so that each user can be throttled on each api call.

The throttle limits for each api call will be defined by a SQL table, allowing administrators to set the throttling of any api calls dynamically.

Table Schema

This table would contain the api calls that administrators would want to throttle, the maximum number of calls per period, and the period of time after which the max calls would reset.

(optional: expiration of throttle rule?? not sure if necessary)

CREATE TABLE 'THROTTLED_CALLS'(
	'THROTTLE_ID' int(20) PRIMARY KEY,              --id of the throttle rule
	'NORMALIZED_URI' varchar(256) NOT NULL,        --normalized api URL, numbers such as {id} replaced with #
	'MAX_LOCKS' int(20) NOT NULL,                   --maximum number of locks per time block
	'LOCK_TIMEOUT_SECONDS' int(20) NOT NULL,        --duration of each time block in seconds
	'THROTTLE_EXPIRATION' bigint(20) DEFAULT NULL,  --optional expiration of the rule in unix timestamp. maybe not necessary?
	UNIQUE ('NORMALIZED_CALL')
)

UserThrottleFilter will have a in-memory cached version of this table. It would periodically check the THROTTLED_CALLS table and update its cached version to reflect the information in the database.

The cached in memory version will be a Map from the normalized uri to a pair of values for maxlocks and locktimeoutSeconds.

Proposed ways to update the cached throtttles:

By having UserThrottleFilter periodically read the table (via a DAO)
Make the Map a synchronized singleton. Allowing a worker in the background update it.

To enforce the Throttle, UserThrottleFilter will use a singleton InMemoryTimeBlockCountingSemaphore, which has already been implemented.

When an request comes in, the request uri will be normalized using a preexisting utility. once normalized, the url is compared to the cached throttled calls to see if it is being throttled. if it is, attempt to get a lock from the semaphore. The key used for the semaphore will be the userID + normalizedThrottledCall.

If we can not get a lock, block the request and return a HTTP 429 error code. otherwise proceed with the other filters.

Dynamic API Call Frequency Throttling

Table Schema