许多特定的任务需要定制目标函数,来达到更优的效果。这里以xgboost的回归预测为例,介绍一下objective函数的定制过程。一个简单的例子如下:
def customObj1(real, predict):
grad = predict - real
hess = np.power(np.abs(grad), 0.5)
return grad, hess
网上有许多教程定义的objective函数中的第一个参数是preds,第二个是dtrain,而本文由于使用xgboost的sklearn API,因此定制的objective函数需要与sklearn的格式相符。调用目标函数的过程如下:
model = xgb.XGBRegressor(objective=customObj1,
booster="gblinear")
下面是不同迭代次数的动画演示:
我们发现,不同的目标函数对模型的收敛速度影响较大,但最终收敛目标大致相同,如下图:
完整代码如下:
# coding=utf-8
import pandas as pd
import numpy as np
import xgboost as xgb
import matplotlib.pyplot as plt
plt.rcParams.update({'figure.autolayout': True})
df = pd.DataFrame({'x': [-2.1, -0.9, 0, 1, 2, 2.5, 3, 4],
'y': [ -10, 0, -5, 10, 20, 10, 30, 40]})
X_train = df.drop('y', axis=1)
Y_train = df['y']
X_pred = [-4, -3, -2, -1, 0, 0.4, 0.6, 1, 1.4, 1.6, 2, 3, 4, 5, 6, 7, 8]
def process_list(list_in):
result = map(lambda x: "%8.2f" % round(float(x), 2), list_in)
return list(result)
def customObj3(real, predict):
grad = predict - real
hess = np.power(np.abs(grad), 0.1)
# print 'predict', process_list(predict.tolist()), type(predict)
# print ' real ', process_list(real.tolist()), type(real)
# print ' grad ', process_list(grad.tolist()), type(grad)
# print ' hess ', process_list(hess.tolist()), type(hess), '\n'
return grad, hess
def customObj1(real, predict):
grad = predict - real
hess = np.power(np.abs(grad), 0.5)
return grad, hess
for n_estimators in range(5, 600, 5):
booster_str = "gblinear"
model = xgb.XGBRegressor(objective=customObj1,
booster=booster_str,
n_estimators=n_estimators)
model2 = xgb.XGBRegressor(objective="reg:linear",
booster=booster_str,
n_estimators=n_estimators)
model3 = xgb.XGBRegressor(objective=customObj3,
booster=booster_str,
n_estimators=n_estimators)
model.fit(X=X_train, y=Y_train)
model2.fit(X=X_train, y=Y_train)
model3.fit(X=X_train, y=Y_train)
y_pred = model.predict(data=pd.DataFrame({'x': X_pred}))
y_pred2 = model2.predict(data=pd.DataFrame({'x': X_pred}))
y_pred3 = model3.predict(data=pd.DataFrame({'x': X_pred}))
plt.figure(figsize=(6, 5))
plt.axes().set(title='n_estimators='+str(n_estimators))
plt.plot(df['x'], df['y'], marker='o', linestyle=":", label="Real Y")
plt.plot(X_pred, y_pred, label="predict - real; |grad|**0.5")
plt.plot(X_pred, y_pred3, label="predict - real; |grad|**0.1")
plt.plot(X_pred, y_pred2, label="reg:linear")
plt.xlim(-4.5, 8.5)
plt.ylim(-25, 55)
plt.legend()
# plt.show()
plt.savefig("output/n_estimators_"+str(n_estimators)+".jpg")
plt.close()
print(n_estimators)