常见Loss

对应nn/tf/loss.py

推荐场景无监督loss

对于推荐无监督学习loss，我们实现了常见的两种loss函数，Sigmoid Cross Entropy loss(BCE)和Softmax Cross Entropy loss。 ​

sigmoid_cross_entropy_loss

Sigmoid CE loss就是一个简单的二分类。

def sigmoid_cross_entropy_loss(pos_logit,
                               neg_logit):
  """Sigmoid cross entropy loss.
  Args:
    pos_logit: positive logits, tensor with shape [batch_size]
    neg_logit: negative logits, tensor with shape [batch_size]
  Returns:
    loss, a scalar tensor
  """
  true_xent = tf.nn.sigmoid_cross_entropy_with_logits(
      labels=tf.ones_like(pos_logit), logits=pos_logit)
  negative_xent = tf.nn.sigmoid_cross_entropy_with_logits(
      labels=tf.zeros_like(neg_logit), logits=neg_logit)
  loss = tf.reduce_mean(true_xent) + 1.0 * tf.reduce_mean(negative_xent)
  return loss

​

unsupervised_softmax_cross_entropy_loss

Softmax CE loss，将负样本看作一个batch里共享的形式，从而可以将原来的二分类看成一个正样本和batch_size * neg_num 的负样本的一个“多分类”任务。我们将标签看成one-hot的编码格式，也即只有正样本标记为1，那么总的loss为： ​

其中，sp表示正样本的score。

def unsupervised_softmax_cross_entropy_loss(src_emb,
                                            pos_emb,
                                            neg_emb,
                                            temperature=1.0):
  """Softmax cross entropy loss.
  This loss is mostly used for inner product based two-tower model 
  for recommdener systems.
  Args:
    src_emb: src embedding, tensor with shape [batch_size, dim]
    pos_emb: positive dst embedding, tensor with shape [batch_size, dim]
    neg_emb: negative dst embedding, tensor with shape [batch_size * neg_num, dim]
  Returns:
    loss, a scalar tensor
  """
  # src_emb和pos_emb是有边的正样本
  pos_sim = tf.reduce_sum(tf.multiply(src_emb, pos_emb), axis=-1, keepdims=True)
  # src_emb和neg_emb是无边的负样本
  neg_sim = tf.matmul(src_emb, tf.transpose(neg_emb))
  # 计算softmax
  logit = tf.nn.softmax(tf.concat([pos_sim, neg_sim] , axis=-1) / temperature)
  # 由于是one-hot编码，只用取正样本的softmax的log结果即可得到CE loss.
  loss = -tf.reduce_mean(tf.log(logit[:, :1] + 1e-12))

  return loss