The purpose of this feature is to allow administrators to selectively control which API calls are throttled, without having to rebuild and deploy a new version every time a throttle is changed.

Current throttle implementation

The current frequency throttle is only able to throttle each user's total call frequency using a semaphore. The UserThrottleFilter uses an InMemoryTimeBlockCountingSemaphore that maps from a key(user id in this case) to a SimpleSemaphore, which keeps a count of calls made and the the after which the count will reset. The UserThrottleFilter then compares the user's call count to the maximum allowed.

New throttle implementation

The new throttle will employ a similar mechanism, except that the keys will be userId+normalizedAPI so that each user can be throttled on each API call. 

...

(optional: expiration of throttle rule?? not sure if necessary)

CREATE TABLE 'THROTTLED`THROTTLE_CALLS'RULES`(
	'THROTTLE_ID' int    `THROTTLE_ID` bigint(20) PRIMARYNOT KEY,NULL,                                            -- id of the throttle rule
	'NORMALIZED_URI'
    `NORMALIZED_URI` varchar(256255) NOT NULL,                                        -- normalized api URL, numbers such as {id} replaced with #
	'MAX_LOCKS'
    `MAX_CALLS` int(20) unsigned NOT NULL,                                         -- maximum number of locks calls per user per URI per bin
time   block
	'LOCK_TIMEOUT_SECONDS' `CALL_PERIOD_IN_SECONDS` int(20) unsigned NOT NULL,                            --duration of each time block in seconds
	'THROTTLE_EXPIRATION' bigint(20) DEFAULT NULL,  --optional expiration of the rule in unix timestamp. maybe not necessary?
	UNIQUE ('NORMALIZED_CALL')
)

...

 bin of time in which a user is allowed to make MAX_CALLS API calls.
    `MODIFIED_ON` TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, -- Timestamp of when the row was changed. used as an etag for migration
    PRIMARY KEY(`THROTTLE_ID`),
    UNIQUE (`NORMALIZED_URI`)
);

The NORMALIZED_URI column must be unique since having 2 different throttling rules for the same API call would not make sense(maybe consider making this primary key instead of using numeric id?).

Reducing Table Accesses

UserThrottleFilter will have a in-memory cached version of this table. It would periodically check the THROTTLED_CALLS table and update its cached version to reflect the information in the database.

The cached in memory version(refreshed every 5 minutes) will be a Map from the normalized URI to a pair of values for maxLocks and lockTimeoutSeconds. 

...

Throttle Logic

When an request comes in, it must first pass the concurrent throttle, then the frequncy throttle, then the service specific throttle.

For the service specific throttle, the request URI will be normalized using normalizeMethodSignature() in AccessRecordUtils of the Synapse-Warehouse-Records project (copy it overmove/refactor?).

Once normalized, the URI is compared to the cached throttle rules to see if it is being throttled. 

• If it is not throttled, proceed to next filter.
• If it is throttled , attempt to get a lock from the semaphore. The key used for the semaphore will be the userID + normalizedThrottledCall.
  • If a lock can be successfully acquired, proceed to next filter.
  • If we can not get a lock, block the request and return a HTTP 429 error code. otherwise proceed with the other filters.

Services

These services make updating rules more convenient.

Administrators could also just directly update the SQL table. 

...

• and the user has exceeded the limit. Return 503 error code (will be changed to 429 when clients are able to handle this code).

Services

No services. Administrators will update the table in MySQL.

Potential problems

If there are many calls being throttled, the throttle could potentially use up a lot of memory. With N throttled calls and M users, the throttle's map for call counts could have up to M x N entires. Additionally, the map will not remove entries for users that are are no longer making calls so memory will not be freed until an administrator calls clearAllLocks().

Updates to the throttle rules will not immediately take effect because they are only written into the SQL table. The actual enforcement of the throttle will not happen until UserThrottleFilter updates its cached version of the rules.