...

The current frequency throttle is only able to throttle each user's total call frequency using a semaphore. The InMemoryTimeBlockCountingSemaphore is a map UserThrottleFilter uses an InMemoryTimeBlockCountingSemaphore that maps from a key(user id in this case) to a SimpleSemaphore, which keeps a count of calls made and the the after which the count will reset. The UserThrottleFilter then compares the user's call count to the maximum allowed. 

The new throttle will use employ a similar mechanism, except that the keys will be userId+normalizedAPI so that each user can be throttled on each api API call. 

The throttle limits for each api API call will be defined by a SQL table, allowing administrators to set the throttling of any api API calls dynamically.

Table Schema

...

CREATE TABLE 'THROTTLED_CALLS'(
	'THROTTLE_ID' int(20) PRIMARY KEY,              --id of the throttle rule
	'NORMALIZED_URI' varchar(256) NOT NULL,        --normalized api URL, numbers such as {id} replaced with #
	'MAX_LOCKS' int(20) NOT NULL,                   --maximum number of locks per time block
	'LOCK_TIMEOUT_SECONDS' int(20) NOT NULL,        --duration of each time block in seconds
	'THROTTLE_EXPIRATION' bigint(20) DEFAULT NULL,  --optional expiration of the rule in unix timestamp. maybe not necessary?
	UNIQUE ('NORMALIZED_CALL')
)

Reducing Table Accesses

UserThrottleFilter will have a in-memory cached version of this table. It would periodically check the THROTTLED_CALLS table and update its cached version to reflect the information in the database.

The cached in memory version will be a Map from the normalized uri URI to a pair of values for maxlocks maxLocks and locktimeoutSecondslockTimeoutSeconds. 

Proposed ways to update the cached throtttlesthrottle limits:

• By having UserThrottleFilter periodically read the table (via a DAO) and updating the Map.
• Make Wrap the Map with a synchronized singleton. Allowing a worker in the background to update it.

...

To enforce the Throttle, UserThrottleFilter will use a singleton InMemoryTimeBlockCountingSemaphore, which has already been implemented.

...

Throttle Logic

When an request comes in, the request uri URI will be normalized using a preexisting utility. once normalized, the url is compared to the cached throttled calls to see if it is being throttled. if it is, attempt to get a lock from the semaphore. The key used for the semaphore will be the userID + normalizedThrottledCall.

If we can not get a lock, block the request and return a HTTP 429 error code. otherwise proceed with the other filters.

Potential problems

If there are many calls being throttled, we could have memory issues. With N throttled calls and M users. The throttle's map for call counts would have to throttle M x N entires. Additionally, we are 

Updates to the SQL table will not be immediately reflected in the UserThrottleFilter.