Locally Differentially Private (LDP) LinUCB is a variant of LinUCB bandit algorithm with local differential privacy guarantee, which can preserve users’ personal data with theoretical guarantee.
Paper: Kai Zheng, Tianle Cai, Weiran Huang, Zhenguo Li, Liwei Wang. “Locally Differentially Private (Contextual) Bandits Learning.” Advances in Neural Information Processing Systems. 2020.
The server interacts with users in rounds. For a coming user, the server first transfers the current model parameters to the user. In the user side, the model chooses an action based on the user feature to play (e.g., choose a movie to recommend), and observes a reward (or loss) value from the user (e.g., rating of the movie). Then we perturb the data to be transferred by adding Gaussian noise. Finally, the server receives the perturbed data and updates the model. Details can be found in the original paper.
Note that you can run the scripts based on the dataset mentioned in original paper. In the following sections, we will introduce how to run the scripts using the related dataset below.
Dataset used: MovieLens 100K
- Dataset size：5MB, 100,000 ratings (1-5) from 943 users on 1682 movies.
- Data format：csv/txt files
- Hardware (Ascend/GPU)
- Prepare hardware environment with Ascend or GPU processor.
- For more information, please check the resources below：
├── model_zoo ├── README.md // descriptions about all the models ├── research ├── rl ├── ldp_linucb ├── README.md // descriptions about LDP LinUCB ├── scripts │ ├── run_train_eval.sh // shell script for running on Ascend ├── src │ ├── dataset.py // dataset for movielens │ ├── linucb.py // model ├── train_eval.py // training script ├── result1.png // experimental result ├── result2.png // experimental result
Parameters for preparing MovieLens 100K dataset
'num_actions': 20 # number of candidate movies to be recommended 'rank_k': 20 # rank of rating matrix completion
Parameters for LDP LinUCB, MovieLens 100K dataset
'epsilon': 8e5 # privacy parameter 'delta': 0.1 # privacy parameter 'alpha': 0.1 # failure probability 'iter_num': 1e6 # number of iterations
running on Ascend
python train_eval.py > result.log 2>&1 &
The python command above will run in the background, you can view the results through the file
The regret value will be achieved as follows:
--> Step: 0, diff: 348.662, current_regret: 0.000, cumulative regret: 0.000 --> Step: 1, diff: 338.457, current_regret: 0.000, cumulative regret: 0.000 --> Step: 2, diff: 336.465, current_regret: 2.000, cumulative regret: 2.000 --> Step: 3, diff: 327.337, current_regret: 0.000, cumulative regret: 2.000 --> Step: 4, diff: 325.039, current_regret: 2.000, cumulative regret: 4.000 ...
The original paper assumes that the norm of user features is bounded by 1 and the norm of rating scores is bounded by 2. For the MovieLens dataset, we normalize rating scores to [-1,1]. Thus, we set
sigma in Algorithm 5 to be $$4/epsilon * sqrt(2 * ln(1.25/delta))$$.
The performance for different privacy parameters:
- x: number of iterations
- y: cumulative regret
The performance compared with optimal non-private regret O(sqrt(T)):
- x: number of iterations
- y: cumulative regret divided by sqrt(T)
train_eval.py, we randomly sample a user at each round. We also add Gaussian noise to the date being transferred.
Please check the official