p768lwy3/torecsys

ToR[e]cSys is a PyTorch Framework to implement recommendation system algorithms, including but not limited to click-through-rate (CTR) prediction, learning-to-ranking (LTR), and Matrix/Tensor Embedding. The project objective is to develop an ecosystem to experiment, share, reproduce, and deploy in real-world in a smooth and easy way.

Python

106

206 commits

updated Apr 8, 2022

See the code

README

ToR[e]cSys


News

It is happy to know the new package of Tensorflow Recommenders.


ToR[e]cSys is a PyTorch Framework to implement recommendation system algorithms, including but not limited to click-through-rate (CTR) prediction, learning-to-ranking (LTR), and Matrix/Tensor Embedding. The project objective is to develop an ecosystem to experiment, share, reproduce, and deploy in real world in a smooth and easy way (Hope it can be done).

Installation

TBU

Documentation

The complete documentation for ToR[e]cSys is available via ReadTheDocs website.
Thank you for ReadTheDocs! You are the best!

Implemented Models

1. Subsampling

2. Negative Sampling

Model NameResearch PaperYear
TBU

3. Click-Through-Rate (CTR) Model

Model NameResearch PaperYear
Logistic Regression//
Factorization MachineSteffen Rendle, 2010. Factorization Machine2010
Factorization Machine Support Neural NetworkWeinan Zhang et al, 2016. Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction2016
Field-Aware Factorization MachineYuchin Juan et al, 2016. Field-aware Factorization Machines for CTR Prediction2016
Product Neural NetworkYanru QU et al, 2016. Product-based Neural Networks for User Response Prediction2016
Attentional Factorization MachineJun Xiao et al, 2017. Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks2017
Deep and Cross NetworkRuoxi Wang et al, 2017. Deep & Cross Network for Ad Click Predictions2017
Deep Factorization MachineHuifeng Guo et al, 2017. DeepFM: A Factorization-Machine based Neural Network for CTR Prediction2017
Neural Collaborative FilteringXiangnan He et al, 2017. Neural Collaborative Filtering2017
Neural Factorization MachineXiangnan He et al, 2017. Neural Factorization Machines for Sparse Predictive Analytics2017
eXtreme Deep Factorization MachineJianxun Lian et al, 2018. xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems2018
Deep Field-Aware Factorization MachineJunlin Zhang et al, 2019. FAT-DeepFFM: Field Attentive Deep Field-aware Factorization Machine2019
Deep Matching Correlation PredictionWentao Ouyang et al, 2019. Representation Learning-Assisted Click-Through Rate Prediction2019
Deep Session Interest NetworkYufei Feng et al, 2019. Deep Session Interest Network for Click-Through Rate Prediction2019
Elaborated Entire Space Supervised Multi Task ModelHong Wen et al, 2019. Conversion Rate Prediction via Post-Click Behaviour Modeling2019
Entire Space Multi Task ModelXiao Ma et al, 2019. Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate2019
Field Attentive Deep Field Aware Factorization MachineJunlin Zhang et al, 2019. FAT-DeepFFM: Field Attentive Deep Field-aware Factorization Machine2019
Position-bias aware learning frameworkHuifeng Guo et al, 2019. PAL: a position-bias aware learning framework for CTR prediction in live recommender systems2019

4. Embedding Model

5. Learning-to-Rank (LTR) Model

Getting Started

There are several ways using ToR[e]cSys to develop a Recommendation System. Before talking about them, we first need to discuss components of ToR[e]cSys.

A model in ToR[e]cSys is constructed by two parts mainly: inputs and model, and they will be wrapped into a sequential module (torecsys.models.sequential) to be trained by Trainer (torecsys.trainer.Trainer). \

For inputs module (torecsys.inputs), it will handle most kinds of inputs in recommendation system, like categorical features, images, etc., with several kinds of methods, including token embedding, pre-trained image models, etc.

For models module (torecsys.models), it will implement some famous models in recommendation system, like Factorization Machine family. I hope I can make the library rich. To construct a model in the module, in addition to the modules implemented in PyTorch, I will also implement some layers in layers which are called by models usually.

After the explanation of ToR[e]cSys, let's move on to the Getting Started. We can use ToR[e]cSys in the following ways:

  1. Run by command-line (In development)
> torecsys build --inputs_config='{}' \
--model_config='{"method":"FM", "embed_size": 8, "num_fields": 2}' \
--regularizer_config='{"weight_decay": 0.1}' \
--criterion_config='{"method": "MSELoss"}' \
--optimizer_config='{"method": "SGD", "lr": "0.01"}' \
...
  1. Run by class method
import torecsys as trs

# build trainer by class method
pipeline = trs.trainer.TorecsysPipeline()

# start to fit the model
trainer = trs.trainer.TorecsysTrainer()
trainer.fit(pipeline)
  1. Run like PyTorch Module

import torch.nn as nn
import torch.optim
import torecsys as trs

schema = {}
batches = {}
labels = torch.Tensor([])
inputs = trs.inputs.Inputs(schema=schema)
model = trs.models.FactorizationMachineModel()

epochs = 1
optimizer = torch.optim.SGD(params=model.parameters(), lr=1e-4)
criterion = nn.MSELoss()

for i in range(epochs):
    optimizer.zero_grad()
    outputs = model(**inputs(batches))
    loss = criterion(outputs, labels)
    loss.backward()
    optimizer.step()

For further details, please refer to the example in repository or read the documentation. Hope you enjoy~

Examples

TBU

Sample Codes

TBU

Sample of Experiments

TBU

Authors

License

ToR[e]cSys is MIT-style licensed, as found in the LICENSE file.

click-through-rate
embedding-models
python
pytorch
recommendation-system
recommender-system

Contributors

p768lwy3

131 commits

jasper430

74 commits

p768lwy3/torecsys

ToR[e]cSys is a PyTorch Framework to implement recommendation system algorithms, including but not limited to click-through-rate (CTR) prediction, learning-to-ranking (LTR), and Matrix/Tensor Embedding. The project objective is to develop an ecosystem to experiment, share, reproduce, and deploy in real-world in a smooth and easy way.

Python

106

206 commits

updated Apr 8, 2022

See the code

README

ToR[e]cSys


News

It is happy to know the new package of Tensorflow Recommenders.


ToR[e]cSys is a PyTorch Framework to implement recommendation system algorithms, including but not limited to click-through-rate (CTR) prediction, learning-to-ranking (LTR), and Matrix/Tensor Embedding. The project objective is to develop an ecosystem to experiment, share, reproduce, and deploy in real world in a smooth and easy way (Hope it can be done).

Installation

TBU

Documentation

The complete documentation for ToR[e]cSys is available via ReadTheDocs website.
Thank you for ReadTheDocs! You are the best!

Implemented Models

1. Subsampling

2. Negative Sampling

Model NameResearch PaperYear
TBU

3. Click-Through-Rate (CTR) Model

Model NameResearch PaperYear
Logistic Regression//
Factorization MachineSteffen Rendle, 2010. Factorization Machine2010
Factorization Machine Support Neural NetworkWeinan Zhang et al, 2016. Deep Learning over Multi-field Categorical Data: A Case Study on User Response Prediction2016
Field-Aware Factorization MachineYuchin Juan et al, 2016. Field-aware Factorization Machines for CTR Prediction2016
Product Neural NetworkYanru QU et al, 2016. Product-based Neural Networks for User Response Prediction2016
Attentional Factorization MachineJun Xiao et al, 2017. Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks2017
Deep and Cross NetworkRuoxi Wang et al, 2017. Deep & Cross Network for Ad Click Predictions2017
Deep Factorization MachineHuifeng Guo et al, 2017. DeepFM: A Factorization-Machine based Neural Network for CTR Prediction2017
Neural Collaborative FilteringXiangnan He et al, 2017. Neural Collaborative Filtering2017
Neural Factorization MachineXiangnan He et al, 2017. Neural Factorization Machines for Sparse Predictive Analytics2017
eXtreme Deep Factorization MachineJianxun Lian et al, 2018. xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems2018
Deep Field-Aware Factorization MachineJunlin Zhang et al, 2019. FAT-DeepFFM: Field Attentive Deep Field-aware Factorization Machine2019
Deep Matching Correlation PredictionWentao Ouyang et al, 2019. Representation Learning-Assisted Click-Through Rate Prediction2019
Deep Session Interest NetworkYufei Feng et al, 2019. Deep Session Interest Network for Click-Through Rate Prediction2019
Elaborated Entire Space Supervised Multi Task ModelHong Wen et al, 2019. Conversion Rate Prediction via Post-Click Behaviour Modeling2019
Entire Space Multi Task ModelXiao Ma et al, 2019. Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate2019
Field Attentive Deep Field Aware Factorization MachineJunlin Zhang et al, 2019. FAT-DeepFFM: Field Attentive Deep Field-aware Factorization Machine2019
Position-bias aware learning frameworkHuifeng Guo et al, 2019. PAL: a position-bias aware learning framework for CTR prediction in live recommender systems2019

4. Embedding Model

5. Learning-to-Rank (LTR) Model

Getting Started

There are several ways using ToR[e]cSys to develop a Recommendation System. Before talking about them, we first need to discuss components of ToR[e]cSys.

A model in ToR[e]cSys is constructed by two parts mainly: inputs and model, and they will be wrapped into a sequential module (torecsys.models.sequential) to be trained by Trainer (torecsys.trainer.Trainer). \

For inputs module (torecsys.inputs), it will handle most kinds of inputs in recommendation system, like categorical features, images, etc., with several kinds of methods, including token embedding, pre-trained image models, etc.

For models module (torecsys.models), it will implement some famous models in recommendation system, like Factorization Machine family. I hope I can make the library rich. To construct a model in the module, in addition to the modules implemented in PyTorch, I will also implement some layers in layers which are called by models usually.

After the explanation of ToR[e]cSys, let's move on to the Getting Started. We can use ToR[e]cSys in the following ways:

  1. Run by command-line (In development)
> torecsys build --inputs_config='{}' \
--model_config='{"method":"FM", "embed_size": 8, "num_fields": 2}' \
--regularizer_config='{"weight_decay": 0.1}' \
--criterion_config='{"method": "MSELoss"}' \
--optimizer_config='{"method": "SGD", "lr": "0.01"}' \
...
  1. Run by class method
import torecsys as trs

# build trainer by class method
pipeline = trs.trainer.TorecsysPipeline()

# start to fit the model
trainer = trs.trainer.TorecsysTrainer()
trainer.fit(pipeline)
  1. Run like PyTorch Module

import torch.nn as nn
import torch.optim
import torecsys as trs

schema = {}
batches = {}
labels = torch.Tensor([])
inputs = trs.inputs.Inputs(schema=schema)
model = trs.models.FactorizationMachineModel()

epochs = 1
optimizer = torch.optim.SGD(params=model.parameters(), lr=1e-4)
criterion = nn.MSELoss()

for i in range(epochs):
    optimizer.zero_grad()
    outputs = model(**inputs(batches))
    loss = criterion(outputs, labels)
    loss.backward()
    optimizer.step()

For further details, please refer to the example in repository or read the documentation. Hope you enjoy~

Examples

TBU

Sample Codes

TBU

Sample of Experiments

TBU

Authors

License

ToR[e]cSys is MIT-style licensed, as found in the LICENSE file.

click-through-rate
embedding-models
python
pytorch
recommendation-system
recommender-system

Contributors

p768lwy3

131 commits

jasper430

74 commits

Languages

Python

92.7%

Jupyter Notebook

7.3%