Skip to content

[ENSCORESW-2749] Add a hashing algorithm and user-derived salt to rate limiter

Marek Szuba requested to merge feature/anonymiseratelimit into master

Created by: nerdstrike


Anonymise the IP addresses used to rate-limit REST API users. A user-agent derived salt is added to the IP address and MD5 is used to encode it.

Use case

Production use of ensembl-rest requires a transient database record of user activity above what is collected at the institution load balancer.


This is a good faith effort to mitigate our obligations w.r.t. GDPR compliance. User IP storage is transient and never in plain text form. We would need to expend great effort to extract the IP addresses from the rate limiter.

Possible Drawbacks

A per-request cost to encode the user's IP. Very unlikely hash collision, where two users end up sharing the same rate limit.


A simple cross-check is added that tests the encoded IP address is a hash and not an IP.


This change should be invisible to users.

Merge request reports