本文介绍了最有效的R余弦计算的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我有两个值向量和一个权重向量,我需要计算余弦相似度.由于复杂的原因,我一次只能计算一对余弦.但是我必须做数百万次.

I have two vectors of values and one vector of weights, and I need to calculate the cosine similarity. For complicated reasons, I can only calculate the cosine for one pair at a time. But I have to do it many millions of times.

cosine_calc <- function(a,b,wts) {
  #scale both vectors by the weights, then compute the cosine of the scaled vectors
  a = a*wts
  b = b*wts
  (a %*% b)/(sqrt(a%*%a)*sqrt(b%*%b))
}

可行,但是我想尝试从中获得更好的性能.

works, but I want to try to eke better performance out of it.

示例数据:

a = c(-1.2092420, -0.7053822, 1.4364633, 1.3612304, -0.3029147, 1.0319704, 0.6707610, -2.2128987, -0.9839970, -0.4302205)
b = c(-0.69042619, 0.05811749, -0.17836802, 0.15699691, 0.78575477, 0.27925779, -0.08552864, -1.31031219, -1.92756861, -1.36350112)
w = c(0.26333839, 0.12803180, 0.62396023, 0.37393705, 0.13539926, 0.09199102, 0.37347546, 1.36790007, 0.64978409, 0.46256891)
> cosine_calc(a,b,w)[,1]
[1,] 0.8390671

问题指出R中还有其他可用的预定义余弦函数,但是他们的相对效率什么也没说.

This question points out that there are other predefined cosine functions available in R, but says nothing about their relative efficiency.

推荐答案

您正在使用的所有函数都是.Primitive(因此已经直接调用了编译代码),因此很难在re之外找到一致的速度提升.用优化的BLAS构建R.话虽如此,对于较大的向量,这是一个可能更快的选择:

All the functions you're using are .Primitive (therefore already call compiled code directly), so it will be hard to find consistent speed gains outside of re-building R with an optimized BLAS. With that said, here is one option that might be faster for larger vectors:

cosine_calc2 <- function(a,b,wts) {
  a = a*wts
  b = b*wts
  crossprod(a,b)/sqrt(crossprod(a)*crossprod(b))
}

all.equal(cosine_calc1(a,b,w),cosine_calc2(a,b,w))
# [1] TRUE

# Check some timings
library(rbenchmark)
# cosine_calc2 is slower on my machine in this case
benchmark(
  cosine_calc1(a,b,w),
  cosine_calc2(a,b,w), replications=1e5, columns=1:4 )
#                    test replications user.self sys.self
# 1 cosine_calc1(a, b, w)       100000      1.06     0.02
# 2 cosine_calc2(a, b, w)       100000      1.21     0.00

# but cosine_calc2 is faster for larger vectors
set.seed(21)
a <- rnorm(1000)
b <- rnorm(1000)
w <- runif(1000)
benchmark(
  cosine_calc1(a,b,w),
  cosine_calc2(a,b,w), replications=1e5, columns=1:4 )
#                    test replications user.self sys.self
# 1 cosine_calc1(a, b, w)       100000      3.83        0
# 2 cosine_calc2(a, b, w)       100000      2.12        0


更新:


UPDATE:

分析表明,将每个向量乘以权重向量需要花费大量时间.

Profiling reveals that quite a bit of time is spent multiplying each vector by the weight vector.

> Rprof(); for(i in 1:100000) cosine_calc2(a,b,w); Rprof(NULL); summaryRprof()
$by.self
             self.time self.pct total.time total.pct
*                 0.80    45.98       0.80     45.98
crossprod         0.56    32.18       0.56     32.18
cosine_calc2      0.32    18.39       1.74    100.00
sqrt              0.06     3.45       0.06      3.45

$by.total
             total.time total.pct self.time self.pct
cosine_calc2       1.74    100.00      0.32    18.39
*                  0.80     45.98      0.80    45.98
crossprod          0.56     32.18      0.56    32.18
sqrt               0.06      3.45      0.06     3.45

$sample.interval
[1] 0.02

$sampling.time
[1] 1.74

如果可以在必须调用该函数数百万次之前进行加权,则可以节省大量时间. cosine_calc3比带有小向量的原始函数要快一些.字节编译功能应该为您提供另一种边际加速.

If you can do the weighting before you have to call the function millions of times, it could save you quite a bit of time. cosine_calc3 is marginally faster than your original function with small vectors. Byte-compiling the function should give you another marginal speedup.

cosine_calc3 <- function(a,b) {
  crossprod(a,b)/sqrt(crossprod(a)*crossprod(b))
}
A = a*w
B = b*w
# Run again on the 1000-element vectors
benchmark(
  cosine_calc1(a,b,w),
  cosine_calc2(a,b,w),
  cosine_calc3(A,B), replications=1e5, columns=1:4 )
#                    test replications user.self sys.self
# 1 cosine_calc1(a, b, w)       100000      3.85     0.00
# 2 cosine_calc2(a, b, w)       100000      2.13     0.02
# 3    cosine_calc3(A, B)       100000      1.31     0.00

这篇关于最有效的R余弦计算的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

09-02 02:58