今年一进公司就开始搞相机开发，一直是在搞相机应用层，不过自己的学习心一直没停，闲暇之余就研究一下相机的framework、HAL层的东西，平时工作中碰到的和自己接触到的，相机中最复杂的就是预览了，有了一些体会也不想放下，所以决定写一系列关于Android相机的博客，把自己学习到的东西记录下来。
说起Android相机的东西，从应用层的角度来看，基本就是四个重要的节点了：openCamera、createCaptureSession、preview、capture，最复杂的就是preview了，要理解preview，那么就要求大家对Android的View显示系统有一定的理解，才能更好的理解相机的预览。相机的预览其实就是使用预览区的SurfaceView对应的surface创建一条预览流，然后framework从预览surface当中获取到显示buffer，这里用于显示的buffer会根据数量来获取，华为手机的相机framework+HAL两部分一般总共需要7个buffer，每个buffer都对应预览区的一屏的大小，它就是HAL、算法各层填充完毕后，要交给SurfaceFlinger用于显示的预览区大小的所有像素点的byte数组，这7个buffer每次在CameraServer进程获取一个，然后通过HIDL下发给CameraDaemon进程，交给算法、HAL层进行着色渲染，完成后再通过CameraServer进程交给SurfaceFlinger，最后显示在屏幕上，这样不断的轮转，我们就看到了预览区会不断的变动，这里的buffer轮转也就是相机最核心的部分了。我们后期的博客具体在讲关于buffer轮转的知识。
这节我们先来说说从应用层调用openCamera之后的执行逻辑。openCamera的方法实现是在frameworks\base\core\java\android\hardware\camera2\CameraManager.java类中完成的，CameraManager和我们应用中经常使用的WindowManager、PackageManager一样，都可以通过context上下文获取到，它也是在frameworks\base\core\java\android\app\SystemServiceRegistry.java类中通过Context.CAMERA_SERVICE键值注册的，源码如下：
我们拿到CameraManager类的对象之后，就可以调用它的openCamera方法来打开相机了，CameraManager类的openCamera方法的源码如下：
@RequiresPermission(android.Manifest.permission.CAMERA)
public void openCamera(@NonNull String cameraId,
@NonNull final CameraDevice.StateCallback callback, @Nullable Handler handler)
throws CameraAccessException {
openCameraForUid(cameraId, callback, handler, USE_CALLING_UID);
}
这个方法的实现很简单，就是调用openCameraForUid来进一步处理，我们先来看看调用该方法需要传递的参数，第一个表示要打开的目标Camera的id，华为手机上该值一般有两个：0和1，0表示后摄，当然也是主摄，1表示前摄，我们怎么知道该值的取值呢？可以通过调用CameraManager类的getCameraIdList()方法来获取，该方法会将当前已经注册成功的camera硬件对应的id列表返回给我们应用层，硬件注册都是驱动层的东西了，那一步离我们现在的阶段还很远。我们再来看一下第二个参数CameraDevice.StateCallback，它是定义在frameworks\base\core\java\android\hardware\camera2\CameraDevice.java类中的一个内部类，StateCallback类的定义源码如下：
public static abstract class StateCallback {
/**
* An error code that can be reported by {@link #onError}
* indicating that the camera device is in use already.
*
* <p>
* This error can be produced when opening the camera fails due to the camera
* being used by a higher-priority camera API client.
* </p>
*
* @see #onError
*/
public static final int ERROR_CAMERA_IN_USE = 1;
/**
* An error code that can be reported by {@link #onError}
* indicating that the camera device could not be opened
* because there are too many other open camera devices.
*
* <p>
* The system-wide limit for number of open cameras has been reached,
* and more camera devices cannot be opened until previous instances are
* closed.
* </p>
*
* <p>
* This error can be produced when opening the camera fails.
* </p>
*
* @see #onError
*/
public static final int ERROR_MAX_CAMERAS_IN_USE = 2;
/**
* An error code that can be reported by {@link #onError}
* indicating that the camera device could not be opened due to a device
* policy.
*
* @see android.app.admin.DevicePolicyManager#setCameraDisabled(android.content.ComponentName, boolean)
* @see #onError
*/
public static final int ERROR_CAMERA_DISABLED = 3;
/**
* An error code that can be reported by {@link #onError}
* indicating that the camera device has encountered a fatal error.
*
* <p>The camera device needs to be re-opened to be used again.</p>
*
* @see #onError
*/
public static final int ERROR_CAMERA_DEVICE = 4;
/**
* An error code that can be reported by {@link #onError}
* indicating that the camera service has encountered a fatal error.
*
* <p>The Android device may need to be shut down and restarted to restore
* camera function, or there may be a persistent hardware problem.</p>
*
* <p>An attempt at recovery <i>may</i> be possible by closing the
* CameraDevice and the CameraManager, and trying to acquire all resources
* again from scratch.</p>
*
* @see #onError
*/
public static final int ERROR_CAMERA_SERVICE = 5;
/** @hide */
@Retention(RetentionPolicy.SOURCE)
@IntDef(prefix = {"ERROR_"}, value =
{ERROR_CAMERA_IN_USE,
ERROR_MAX_CAMERAS_IN_USE,
ERROR_CAMERA_DISABLED,
ERROR_CAMERA_DEVICE,
ERROR_CAMERA_SERVICE })
public @interface ErrorCode {};
/**
* The method called when a camera device has finished opening.
*
* <p>At this point, the camera device is ready to use, and
* {@link CameraDevice#createCaptureSession} can be called to set up the first capture
* session.</p>
*
* @param camera the camera device that has become opened
*/
public abstract void onOpened(@NonNull CameraDevice camera); // Must implement
/**
* The method called when a camera device has been closed with
* {@link CameraDevice#close}.
*
* <p>Any attempt to call methods on this CameraDevice in the
* future will throw a {@link IllegalStateException}.</p>
*
* <p>The default implementation of this method does nothing.</p>
*
* @param camera the camera device that has become closed
*/
public void onClosed(@NonNull CameraDevice camera) {
// Default empty implementation
}
/**
* The method called when a camera device is no longer available for
* use.
*
* <p>This callback may be called instead of {@link #onOpened}
* if opening the camera fails.</p>
*
* <p>Any attempt to call methods on this CameraDevice will throw a
* {@link CameraAccessException}. The disconnection could be due to a
* change in security policy or permissions; the physical disconnection
* of a removable camera device; or the camera being needed for a
* higher-priority camera API client.</p>
*
* <p>There may still be capture callbacks that are invoked
* after this method is called, or new image buffers that are delivered
* to active outputs.</p>
*
* <p>The default implementation logs a notice to the system log
* about the disconnection.</p>
*
* <p>You should clean up the camera with {@link CameraDevice#close} after
* this happens, as it is not recoverable until the camera can be opened
* again. For most use cases, this will be when the camera again becomes
* {@link CameraManager.AvailabilityCallback#onCameraAvailable available}.
* </p>
*
* @param camera the device that has been disconnected
*/
public abstract void onDisconnected(@NonNull CameraDevice camera); // Must implement
/**
* The method called when a camera device has encountered a serious error.
*
* <p>This callback may be called instead of {@link #onOpened}
* if opening the camera fails.</p>
*
* <p>This indicates a failure of the camera device or camera service in
* some way. Any attempt to call methods on this CameraDevice in the
* future will throw a {@link CameraAccessException} with the
* {@link CameraAccessException#CAMERA_ERROR CAMERA_ERROR} reason.
* </p>
*
* <p>There may still be capture completion or camera stream callbacks
* that will be called after this error is received.</p>
*
* <p>You should clean up the camera with {@link CameraDevice#close} after
* this happens. Further attempts at recovery are error-code specific.</p>
*
* @param camera The device reporting the error
* @param error The error code.
*
* @see #ERROR_CAMERA_IN_USE
* @see #ERROR_MAX_CAMERAS_IN_USE
* @see #ERROR_CAMERA_DISABLED
* @see #ERROR_CAMERA_DEVICE
* @see #ERROR_CAMERA_SERVICE
*/
public abstract void onError(@NonNull CameraDevice camera,
@ErrorCode int error); // Must implement
}
我们从这个类所定义的方法就能够非常清楚的看到，它这几个回调的意图了：onOpened就是成功打开camera之后的回调，而且它会返回一个CameraDevice camera对象给我们应用层，基本上操作相机所有重要的工作都是由它来中转实现的，所以应用层拿到这个对象之后，就可以使用它作很多其他的工作了，在接下来的分析过程中，我们也会看到，这个对象在framework中是怎么构建好，然后又是怎么回传给我们应用层的；onClosed方法就是当相机关闭时的回调了；onDisconnected方法就是相机断开连接时的回调；onError方法就是相机出错时的回调了。
我们再来看看最后一个参数Handler handler，为什么要传一个Handler进来呢？它的目的就是为了保证线程不切换，假如我们在应用层在工作线程B中执行openCamera的方法，同时将线程B对应的Handler对象传进来，那么打开成功之后，framework为了保证线程同步，也会使用该handler对象，将消息发送到线程B的Looper循环上，这就是传入一个Handler对象的原因了，我们在后面的分析过程中会看到它的使用。
好，参数分析完了，我们继续来看一下openCameraForUid方法的实现，源码如下：
public void openCameraForUid(@NonNull String cameraId,
@NonNull final CameraDevice.StateCallback callback, @Nullable Handler handler,
int clientUid)
throws CameraAccessException {
if (cameraId == null) {
throw new IllegalArgumentException("cameraId was null");
} else if (callback == null) {
throw new IllegalArgumentException("callback was null");
} else if (handler == null) {
if (Looper.myLooper() != null) {
handler = new Handler();
} else {
throw new IllegalArgumentException(
"Handler argument is null, but no looper exists in the calling thread");
}
}
openCameraDeviceUserAsync(cameraId, callback, handler, clientUid);
}
首先还是参数判断，我们调用方法时传入的参数必须合法，否则直接抛出异常，参数合法之后，再调用openCameraDeviceUserAsync来进一步执行camera的打开工作，源码如下：
private CameraDevice openCameraDeviceUserAsync(String cameraId,
CameraDevice.StateCallback callback, Handler handler, final int uid)
throws CameraAccessException {
CameraCharacteristics characteristics = getCameraCharacteristics(cameraId);
CameraDevice device = null;
synchronized (mLock) {
ICameraDeviceUser cameraUser = null;
android.hardware.camera2.impl.CameraDeviceImpl deviceImpl =
new android.hardware.camera2.impl.CameraDeviceImpl(
cameraId,
callback,
handler,
characteristics,
mContext.getApplicationInfo().targetSdkVersion);
ICameraDeviceCallbacks callbacks = deviceImpl.getCallbacks();
try {
if (supportsCamera2ApiLocked(cameraId)) {
// Use cameraservice's cameradeviceclient implementation for HAL3.2+ devices
ICameraService cameraService = CameraManagerGlobal.get().getCameraService();
if (cameraService == null) {
throw new ServiceSpecificException(
ICameraService.ERROR_DISCONNECTED,
"Camera service is currently unavailable");
}
cameraUser = cameraService.connectDevice(callbacks, cameraId,
mContext.getOpPackageName(), uid);
} else {
// Use legacy camera implementation for HAL1 devices
int id;
try {
id = Integer.parseInt(cameraId);
} catch (NumberFormatException e) {
throw new IllegalArgumentException("Expected cameraId to be numeric, but it was: "
+ cameraId);
}
Log.i(TAG, "Using legacy camera HAL.");
cameraUser = CameraDeviceUserShim.connectBinderShim(callbacks, id);
}
} catch (ServiceSpecificException e) {
if (e.errorCode == ICameraService.ERROR_DEPRECATED_HAL) {
throw new AssertionError("Should've gone down the shim path");
} else if (e.errorCode == ICameraService.ERROR_CAMERA_IN_USE ||
e.errorCode == ICameraService.ERROR_MAX_CAMERAS_IN_USE ||
e.errorCode == ICameraService.ERROR_DISABLED ||
e.errorCode == ICameraService.ERROR_DISCONNECTED ||
e.errorCode == ICameraService.ERROR_INVALID_OPERATION) {
// Received one of the known connection errors
// The remote camera device cannot be connected to, so
// set the local camera to the startup error state
deviceImpl.setRemoteFailure(e);
if (e.errorCode == ICameraService.ERROR_DISABLED ||
e.errorCode == ICameraService.ERROR_DISCONNECTED ||
e.errorCode == ICameraService.ERROR_CAMERA_IN_USE) {
// Per API docs, these failures call onError and throw
throwAsPublicException(e);
}
} else {
// Unexpected failure - rethrow
throwAsPublicException(e);
}
} catch (RemoteException e) {
// Camera service died - act as if it's a CAMERA_DISCONNECTED case
ServiceSpecificException sse = new ServiceSpecificException(
ICameraService.ERROR_DISCONNECTED,
"Camera service is currently unavailable");
deviceImpl.setRemoteFailure(sse);
throwAsPublicException(sse);
}
// TODO: factor out callback to be non-nested, then move setter to constructor
// For now, calling setRemoteDevice will fire initial
// onOpened/onUnconfigured callbacks.
// This function call may post onDisconnected and throw CAMERA_DISCONNECTED if
// cameraUser dies during setup.
deviceImpl.setRemoteDevice(cameraUser);
device = deviceImpl;
}
return device;
}
该方法中首先调用getCameraCharacteristics(cameraId)获取到目标camera的特性参数，这个方法的实现就是在我们前面提到的CameraServer进程当中，因为手机开机时，camera驱动的注册就开始了，注册完成之后，所有camera硬件设备的特性参数都已经保存在CameraServer进程当中了，我们这里只需要获取就可以了，大家如果想了解，可以继续往下追查。接下来使用我们传入的参数构造一个CameraDeviceImpl对象，它其实也就是最终要通过StateCallback回调接口的onOpened方法返回给我们应用层的那个参数对象了，只不过这里刚构建好，它里边最核心的一个成员变量还没有赋值，就和Surface一样，我们虽然可以new Surface创造一个Surface对象，但是它在native层还要执行很多的初始化逻辑，还有它所对应的buffer空间和native层的指针没有初始化，那么这个Surface只是个壳子，没有什么实际的用处。我们来看一下frameworks\base\core\java\android\hardware\camera2\impl\CameraDeviceImpl.java类的构造方法，源码如下：
public CameraDeviceImpl(String cameraId, StateCallback callback, Handler handler,
CameraCharacteristics characteristics, int appTargetSdkVersion) {
if (cameraId == null || callback == null || handler == null || characteristics == null) {
throw new IllegalArgumentException("Null argument given");
}
mCameraId = cameraId;
mDeviceCallback = callback;
mDeviceHandler = handler;
mCharacteristics = characteristics;
mAppTargetSdkVersion = appTargetSdkVersion;
final int MAX_TAG_LEN = 23;
String tag = String.format("CameraDevice-JV-%s", mCameraId);
if (tag.length() > MAX_TAG_LEN) {
tag = tag.substring(0, MAX_TAG_LEN);
}
TAG = tag;
Integer partialCount =
mCharacteristics.get(CameraCharacteristics.REQUEST_PARTIAL_RESULT_COUNT);
if (partialCount == null) {
// 1 means partial result is not supported.
mTotalPartialCount = 1;
} else {
mTotalPartialCount = partialCount;
}
}
这里比较重要的信息就是关于tag的赋值了，它是执行String.format("CameraDevice-JV-%s", mCameraId)逻辑来赋值的，我们也可以在打开相机时搜索到这个关键字，它对我们实际的处理问题可以说是一个点，我们可以从这里判断肯定有应用层想要打开相机设备，而且可以根据id看到想要打开哪个目标设备。回到openCameraDeviceUserAsync方法中，接下来调用刚才创建好的CameraDeviceImpl对象的getCallbacks()方法获取一个ICameraDeviceCallbacks对象，它是通过ICameraDeviceCallbacks.aidl定义的，所以它就是一个binder对象，而它的作用就是把这个对象传递到CameraServer进程当中，来配合回调各个节点方法的，我们马上就会看到。接下来判断if (supportsCamera2ApiLocked(cameraId))条件，它的意思就是当前是否支持camera2.0的协议，这是Android对于相机的优化，我们假设这里支持，那么继续调用cameraUser = cameraService.connectDevice(callbacks, cameraId, mContext.getOpPackageName(), uid)逻辑来给局部变量cameraUser赋值，这句逻辑往下就是我们最核心的地方了，而这个返回值也就是CameraServer进程派给我们的操作代表，我们应用的各项工作都是由它进行中转来实现了。
我们继续往下看完这个方法的实现，然后再回头来分析cameraService.connectDevice的逻辑实现。接下来最后的两句就是调用deviceImpl.setRemoteDevice(cameraUser)方法将CameraServer返回给我们的对象保存下来，这句逻辑执行完成后，这个deviceImpl才算真正具备功能了，最后就是给局部变量device赋值。我们继续看一下frameworks\base\core\java\android\hardware\camera2\impl\CameraDeviceImpl.java类的setRemoteDevice方法，源码如下：
public void setRemoteDevice(ICameraDeviceUser remoteDevice) throws CameraAccessException {
synchronized(mInterfaceLock) {
// TODO: Move from decorator to direct binder-mediated exceptions
// If setRemoteFailure already called, do nothing
if (mInError) return;
mRemoteDevice = new ICameraDeviceUserWrapper(remoteDevice);
IBinder remoteDeviceBinder = remoteDevice.asBinder();
// For legacy camera device, remoteDevice is in the same process, and
// asBinder returns NULL.
if (remoteDeviceBinder != null) {
try {
remoteDeviceBinder.linkToDeath(this, /*flag*/ 0);
} catch (RemoteException e) {
CameraDeviceImpl.this.mDeviceHandler.post(mCallOnDisconnected);
throw new CameraAccessException(CameraAccessException.CAMERA_DISCONNECTED,
"The camera device has encountered a serious error");
}
}
mDeviceHandler.post(mCallOnOpened);
mDeviceHandler.post(mCallOnUnconfigured);
}
}
这里会使用CameraServer返回给我们的remoteDevice构建一个ICameraDeviceUserWrapper对象，就是将它再包装一层，然后赋值给成员变量mRemoteDevice，我们可以看到，最终我们拿到的那个CameraDevice对象其实离真正给我们干活的对象已经很远了，中间都经过了好几层包装；然后再调用mDeviceHandler.post(mCallOnOpened)通过应用层，打开相机的工作已经完成了，mDeviceHandler就是我们一开始调用openCamera方法时传入的第三个参数了，所以使用它来post发送一个消息，那么回调也会在当时执行openCamera方法的线程上，这样就可以保证线程不会切换了。mCallOnOpened是CameraDeviceImpl类的一个成员变量，定义源码如下：
private final Runnable mCallOnOpened = new Runnable() {
@Override
public void run() {
StateCallbackKK sessionCallback = null;
synchronized(mInterfaceLock) {
if (mRemoteDevice == null) return; // Camera already closed
sessionCallback = mSessionStateCallback;
}
if (sessionCallback != null) {
sessionCallback.onOpened(CameraDeviceImpl.this);
}
mDeviceCallback.onOpened(CameraDeviceImpl.this);
}
};
这里的mDeviceCallback成员变量也就是前面构造CameraDeviceImpl对象时传入的，也就是我们调用openCamera方法的第二个参数了，这时的CameraDeviceImpl对象已经准备好了，于是通过回调接口将CameraDeviceImpl对象返回给应用层，应用层就可以通过它来执行各种逻辑真正的控制camera了。
好，回调应用层的逻辑分析完，我们回过头来继续看一下cameraService.connectDevice方法是如何打开camera的。
cameraService是通过binder进程间通信，执行CameraManagerGlobal.get().getCameraService()获取到的Camera服务端的一个binder代理对象，它实际的实现是native层的CameraService。CameraServer进程的启动函数定义在frameworks\av\camera\cameraserver\main_cameraserver.cpp文件中，它的main函数源码如下：
using namespace android;
int main(int argc __unused, char** argv __unused)
{
signal(SIGPIPE, SIG_IGN);
// Set 3 threads for HIDL calls
hardware::configureRpcThreadpool(3, /*willjoin*/ false);
sp<ProcessState> proc(ProcessState::self());
sp<IServiceManager> sm = defaultServiceManager();
ALOGI("ServiceManager: %p", sm.get());
CameraService::instantiate();
ProcessState::self()->startThreadPool();
IPCThreadState::self()->joinThreadPool();
}
这里会调用CameraService::instantiate()方法，而CameraService是继承了BinderService，所以instantiate()方法实际也是执行的BinderService对象的instantiate()方法，该方法定义在frameworks\native\libs\binder\include\binder\BinderService.h文件中，源码如下：
template<typename SERVICE>
class BinderService
{
public:
static status_t publish(bool allowIsolated = false) {
sp<IServiceManager> sm(defaultServiceManager());
return sm->addService(
String16(SERVICE::getServiceName()),
new SERVICE(), allowIsolated);
}
static void publishAndJoinThreadPool(bool allowIsolated = false) {
publish(allowIsolated);
joinThreadPool();
}
static void instantiate() { publish(); }
static status_t shutdown() { return NO_ERROR; }
private:
static void joinThreadPool() {
sp<ProcessState> ps(ProcessState::self());
ps->startThreadPool();
ps->giveThreadPoolName();
IPCThreadState::self()->joinThreadPool();
}
};
在这里执行publish方法时，以CameraService类的getServiceName()方法返回的char*为键值（media.camera）将CameraService对象添加到service_manager当中的。所以接下来我们就来看一下CameraService类的connectDevice方法的实现，源码如下：
Status CameraService::connectDevice(
const sp<hardware::camera2::ICameraDeviceCallbacks>& cameraCb,
const String16& cameraId,
const String16& clientPackageName,
int clientUid,
/*out*/
sp<hardware::camera2::ICameraDeviceUser>* device) {
ATRACE_CALL();
Status ret = Status::ok();
String8 id = String8(cameraId);
sp<CameraDeviceClient> client = nullptr;
ret = connectHelper<hardware::camera2::ICameraDeviceCallbacks,CameraDeviceClient>(cameraCb, id,
CAMERA_HAL_API_VERSION_UNSPECIFIED, clientPackageName,
clientUid, USE_CALLING_PID, API_2,
/*legacyMode*/ false, /*shimUpdateOnly*/ false,
/*out*/client);
if(!ret.isOk()) {
logRejected(id, getCallingPid(), String8(clientPackageName),
ret.toString8());
return ret;
}
*device = client;
return ret;
}
这里的几个参数我们要看一下，第一个cameraCb就是我们在framework中得到的那个binder对象，以后的部分逻辑都会通过它来进行中转；cameraId表示我们要打开的目标设备；clientPackageName表示请求执行打开camera设备的应用进程包名；clientUid表示应用进程的uid；最后的device就是返回给framework的对象了，注释已经写的非常清楚了，它是一个输出参数。接下来执行sp<CameraDeviceClient> client = nullptr声明一个CameraDeviceClient对象，在给它赋值成功之后，将它再赋值给输出参数device，所以说我们在framework中看到的那个cameraUser实际上就是这里的CameraDeviceClient对象了。
接下来我们继续分析connectHelper方法的实现，源码如下：
template<class CALLBACK, class CLIENT>
Status CameraService::connectHelper(const sp<CALLBACK>& cameraCb, const String8& cameraId,
int halVersion, const String16& clientPackageName, int clientUid, int clientPid,
apiLevel effectiveApiLevel, bool legacyMode, bool shimUpdateOnly,
/*out*/sp<CLIENT>& device) {
binder::Status ret = binder::Status::ok();
String8 clientName8(clientPackageName);
int originalClientPid = 0;
ALOGI("CameraService::connect call (PID %d \"%s\", camera ID %s) for HAL version %s and "
"Camera API version %d", clientPid, clientName8.string(), cameraId.string(),
(halVersion == -1) ? "default" : std::to_string(halVersion).c_str(),
static_cast<int>(effectiveApiLevel));
sp<CLIENT> client = nullptr;
{
// Acquire mServiceLock and prevent other clients from connecting
std::unique_ptr<AutoConditionLock> lock =
AutoConditionLock::waitAndAcquire(mServiceLockWrapper, DEFAULT_CONNECT_TIMEOUT_NS);
if (lock == nullptr) {
ALOGE("CameraService::connect (PID %d) rejected (too many other clients connecting)."
, clientPid);
return STATUS_ERROR_FMT(ERROR_MAX_CAMERAS_IN_USE,
"Cannot open camera %s for \"%s\" (PID %d): Too many other clients connecting",
cameraId.string(), clientName8.string(), clientPid);
}
// Enforce client permissions and do basic sanity checks
if(!(ret = validateConnectLocked(cameraId, clientName8,
/*inout*/clientUid, /*inout*/clientPid, /*out*/originalClientPid)).isOk()) {
return ret;
}
// Check the shim parameters after acquiring lock, if they have already been updated and
// we were doing a shim update, return immediately
if (shimUpdateOnly) {
auto cameraState = getCameraState(cameraId);
if (cameraState != nullptr) {
if (!cameraState->getShimParams().isEmpty()) return ret;
}
}
status_t err;
sp<BasicClient> clientTmp = nullptr;
std::shared_ptr<resource_policy::ClientDescriptor<String8, sp<BasicClient>>> partial;
if ((err = handleEvictionsLocked(cameraId, originalClientPid, effectiveApiLevel,
IInterface::asBinder(cameraCb), clientName8, /*out*/&clientTmp,
/*out*/&partial)) != NO_ERROR) {
switch (err) {
case -ENODEV:
return STATUS_ERROR_FMT(ERROR_DISCONNECTED,
"No camera device with ID \"%s\" currently available",
cameraId.string());
case -EBUSY:
return STATUS_ERROR_FMT(ERROR_CAMERA_IN_USE,
"Higher-priority client using camera, ID \"%s\" currently unavailable",
cameraId.string());
default:
return STATUS_ERROR_FMT(ERROR_INVALID_OPERATION,
"Unexpected error %s (%d) opening camera \"%s\"",
strerror(-err), err, cameraId.string());
}
}
if (clientTmp.get() != nullptr) {
// Handle special case for API1 MediaRecorder where the existing client is returned
device = static_cast<CLIENT*>(clientTmp.get());
return ret;
}
// give flashlight a chance to close devices if necessary.
mFlashlight->prepareDeviceOpen(cameraId);
int facing = -1;
int deviceVersion = getDeviceVersion(cameraId, /*out*/&facing);
if (facing == -1) {
ALOGE("%s: Unable to get camera device \"%s\" facing", __FUNCTION__, cameraId.string());
return STATUS_ERROR_FMT(ERROR_INVALID_OPERATION,
"Unable to get camera device \"%s\" facing", cameraId.string());
}
sp<BasicClient> tmp = nullptr;
if(!(ret = makeClient(this, cameraCb, clientPackageName, cameraId, facing, clientPid,
clientUid, getpid(), legacyMode, halVersion, deviceVersion, effectiveApiLevel,
/*out*/&tmp)).isOk()) {
return ret;
}
client = static_cast<CLIENT*>(tmp.get());
LOG_ALWAYS_FATAL_IF(client.get() == nullptr, "%s: CameraService in invalid state",
__FUNCTION__);
err = client->initialize(mCameraProviderManager);
if (err != OK) {
ALOGE("%s: Could not initialize client from HAL.", __FUNCTION__);
// Errors could be from the HAL module open call or from AppOpsManager
switch(err) {
case BAD_VALUE:
return STATUS_ERROR_FMT(ERROR_ILLEGAL_ARGUMENT,
"Illegal argument to HAL module for camera \"%s\"", cameraId.string());
case -EBUSY:
return STATUS_ERROR_FMT(ERROR_CAMERA_IN_USE,
"Camera \"%s\" is already open", cameraId.string());
case -EUSERS:
return STATUS_ERROR_FMT(ERROR_MAX_CAMERAS_IN_USE,
"Too many cameras already open, cannot open camera \"%s\"",
cameraId.string());
case PERMISSION_DENIED:
return STATUS_ERROR_FMT(ERROR_PERMISSION_DENIED,
"No permission to open camera \"%s\"", cameraId.string());
case -EACCES:
return STATUS_ERROR_FMT(ERROR_DISABLED,
"Camera \"%s\" disabled by policy", cameraId.string());
case -ENODEV:
default:
return STATUS_ERROR_FMT(ERROR_INVALID_OPERATION,
"Failed to initialize camera \"%s\": %s (%d)", cameraId.string(),
strerror(-err), err);
}
}
// Update shim paremeters for legacy clients
if (effectiveApiLevel == API_1) {
// Assume we have always received a Client subclass for API1
sp<Client> shimClient = reinterpret_cast<Client*>(client.get());
String8 rawParams = shimClient->getParameters();
CameraParameters params(rawParams);
auto cameraState = getCameraState(cameraId);
if (cameraState != nullptr) {
cameraState->setShimParams(params);
} else {
ALOGE("%s: Cannot update shim parameters for camera %s, no such device exists.",
__FUNCTION__, cameraId.string());
}
}
if (shimUpdateOnly) {
// If only updating legacy shim parameters, immediately disconnect client
mServiceLock.unlock();
client->disconnect();
mServiceLock.lock();
} else {
// Otherwise, add client to active clients list
finishConnectLocked(client, partial);
}
} // lock is destroyed, allow further connect calls
// Important: release the mutex here so the client can call back into the service from its
// destructor (can be at the end of the call)
device = client;
return ret;
}
该方法是一个模板方法，这里的handleEvictionsLocked、client->initialize方法执行出错之后的日志打印对于我们分析问题也会有非常大的帮助，我们平时工作中经常碰到相机打开失败的问题，如果有这些日志，那就说明问题肯定是出在CameraServer进程往下哪里的逻辑中了。
往下我们就主要来看一下makeClient、client->initialize这两句方法的实现。makeClient方法的源码如下：
Status CameraService::makeClient(const sp<CameraService>& cameraService,
const sp<IInterface>& cameraCb, const String16& packageName, const String8& cameraId,
int facing, int clientPid, uid_t clientUid, int servicePid, bool legacyMode,
int halVersion, int deviceVersion, apiLevel effectiveApiLevel,
/*out*/sp<BasicClient>* client) {
if (halVersion < 0 || halVersion == deviceVersion) {
// Default path: HAL version is unspecified by caller, create CameraClient
// based on device version reported by the HAL.
switch(deviceVersion) {
case CAMERA_DEVICE_API_VERSION_1_0:
if (effectiveApiLevel == API_1) { // Camera1 API route
sp<ICameraClient> tmp = static_cast<ICameraClient*>(cameraCb.get());
*client = new CameraClient(cameraService, tmp, packageName, cameraIdToInt(cameraId),
facing, clientPid, clientUid, getpid(), legacyMode);
} else { // Camera2 API route
ALOGW("Camera using old HAL version: %d", deviceVersion);
return STATUS_ERROR_FMT(ERROR_DEPRECATED_HAL,
"Camera device \"%s\" HAL version %d does not support camera2 API",
cameraId.string(), deviceVersion);
}
break;
case CAMERA_DEVICE_API_VERSION_3_0:
case CAMERA_DEVICE_API_VERSION_3_1:
case CAMERA_DEVICE_API_VERSION_3_2:
case CAMERA_DEVICE_API_VERSION_3_3:
case CAMERA_DEVICE_API_VERSION_3_4:
if (effectiveApiLevel == API_1) { // Camera1 API route
sp<ICameraClient> tmp = static_cast<ICameraClient*>(cameraCb.get());
*client = new Camera2Client(cameraService, tmp, packageName, cameraIdToInt(cameraId),
facing, clientPid, clientUid, servicePid, legacyMode);
} else { // Camera2 API route
sp<hardware::camera2::ICameraDeviceCallbacks> tmp =
static_cast<hardware::camera2::ICameraDeviceCallbacks*>(cameraCb.get());
*client = new CameraDeviceClient(cameraService, tmp, packageName, cameraId,
facing, clientPid, clientUid, servicePid);
}
break;
default:
// Should not be reachable
ALOGE("Unknown camera device HAL version: %d", deviceVersion);
return STATUS_ERROR_FMT(ERROR_INVALID_OPERATION,
"Camera device \"%s\" has unknown HAL version %d",
cameraId.string(), deviceVersion);
}
} else {
// A particular HAL version is requested by caller. Create CameraClient
// based on the requested HAL version.
if (deviceVersion > CAMERA_DEVICE_API_VERSION_1_0 &&
halVersion == CAMERA_DEVICE_API_VERSION_1_0) {
// Only support higher HAL version device opened as HAL1.0 device.
sp<ICameraClient> tmp = static_cast<ICameraClient*>(cameraCb.get());
*client = new CameraClient(cameraService, tmp, packageName, cameraIdToInt(cameraId),
facing, clientPid, clientUid, servicePid, legacyMode);
} else {
// Other combinations (e.g. HAL3.x open as HAL2.x) are not supported yet.
ALOGE("Invalid camera HAL version %x: HAL %x device can only be"
" opened as HAL %x device", halVersion, deviceVersion,
CAMERA_DEVICE_API_VERSION_1_0);
return STATUS_ERROR_FMT(ERROR_ILLEGAL_ARGUMENT,
"Camera device \"%s\" (HAL version %d) cannot be opened as HAL version %d",
cameraId.string(), deviceVersion, halVersion);
}
}
return Status::ok();
}
一般驱动版本halVersion和设备版本deviceVersion是相同的，所以进入第一个if分支，effectiveApiLevel参数是在调用connectHelper方法时传入的，值为API_2，所以进入else分支，直接使用我们上边传进来的参数构造一个CameraDeviceClient对象，而该对象也就是我们应用进程和CameraServer进程通信的使者了，所有的工作都是由它来进行中转的。我们继续看一下它的构造方法的实现。frameworks\av\services\camera\libcameraservice\api2\CameraDeviceClient.cpp类的构造方法源码如下：
CameraDeviceClient::CameraDeviceClient(const sp<CameraService>& cameraService,
const sp<hardware::camera2::ICameraDeviceCallbacks>& remoteCallback,
const String16& clientPackageName,
const String8& cameraId,
int cameraFacing,
int clientPid,
uid_t clientUid,
int servicePid) :
Camera2ClientBase(cameraService, remoteCallback, clientPackageName,
cameraId, cameraFacing, clientPid, clientUid, servicePid),
mInputStream(),
mStreamingRequestId(REQUEST_ID_NONE),
mRequestIdCounter(0) {
ATRACE_CALL();
ALOGI("CameraDeviceClient %s: Opened", cameraId.string());
}
因为它是继承Camera2ClientBase的，所以也会执行Camera2ClientBase类的构造方法。frameworks\av\services\camera\libcameraservice\common\Camera2ClientBase.cpp类的构造方法的源码如下：
template <typename TClientBase>
Camera2ClientBase<TClientBase>::Camera2ClientBase(
const sp<CameraService>& cameraService,
const sp<TCamCallbacks>& remoteCallback,
const String16& clientPackageName,
const String8& cameraId,
int cameraFacing,
int clientPid,
uid_t clientUid,
int servicePid):
TClientBase(cameraService, remoteCallback, clientPackageName,
cameraId, cameraFacing, clientPid, clientUid, servicePid),
mSharedCameraCallbacks(remoteCallback),
mDeviceVersion(cameraService->getDeviceVersion(TClientBase::mCameraIdStr)),
mDeviceActive(false)
{
ALOGI("Camera %s: Opened. Client: %s (PID %d, UID %d)", cameraId.string(),
String8(clientPackageName).string(), clientPid, clientUid);
mInitialClientPid = clientPid;
mDevice = new Camera3Device(cameraId);
LOG_ALWAYS_FATAL_IF(mDevice == 0, "Device should never be NULL here.");
}
在这里又出现了一个非常重要的对象Camera3Device，后边我们就会看到，我们操作相机的所有工作在CameraServer进程都是由它中转来和CameraDaemon进程进行通信的。frameworks\av\services\camera\libcameraservice\device3\Camera3Device.cpp的构造方法的源码如下：
Camera3Device::Camera3Device(const String8 &id):
mId(id),
mOperatingMode(NO_MODE),
mIsConstrainedHighSpeedConfiguration(false),
mStatus(STATUS_UNINITIALIZED),
mStatusWaiters(0),
mUsePartialResult(false),
mNumPartialResults(1),
mTimestampOffset(0),
mNextResultFrameNumber(0),
mNextReprocessResultFrameNumber(0),
mNextShutterFrameNumber(0),
mNextReprocessShutterFrameNumber(0),
mListener(NULL),
mVendorTagId(CAMERA_METADATA_INVALID_VENDOR_ID)
{
ATRACE_CALL();
camera3_callback_ops::notify = &sNotify;
camera3_callback_ops::process_capture_result = &sProcessCaptureResult;
ALOGV("%s: Created device for camera %s", __FUNCTION__, mId.string());
}
这里的sProcessCaptureResult是一个函数指针，从它的命名也很容易判断出来，它就是处理结果回调用，那是哪个结果回调呢？当然是CameraDaemon进程对一帧图片处理完成之后的结果回调了，注意这里的capture不单指拍照，预览的回调也是通过该接口传回来进行处理的。
该构造的对象都创建好了，再回到CameraService类的connectHelper方法中，继续来看一下client->initialize(mCameraProviderManager)逻辑的实现，这里的client就是CameraDeviceClient了，frameworks\av\services\camera\libcameraservice\api2\CameraDeviceClient.cpp类的initialize方法的源码如下：
status_t CameraDeviceClient::initialize(sp<CameraProviderManager> manager) {
return initializeImpl(manager);
}
template<typename TProviderPtr>
status_t CameraDeviceClient::initializeImpl(TProviderPtr providerPtr) {
ATRACE_CALL();
status_t res;
res = Camera2ClientBase::initialize(providerPtr);
if (res != OK) {
return res;
}
String8 threadName;
mFrameProcessor = new FrameProcessorBase(mDevice);
threadName = String8::format("CDU-%s-FrameProc", mCameraIdStr.string());
mFrameProcessor->run(threadName.string());
mFrameProcessor->registerListener(FRAME_PROCESSOR_LISTENER_MIN_ID,
FRAME_PROCESSOR_LISTENER_MAX_ID,
/*listener*/this,
/*sendPartials*/true);
return OK;
}
它也是转调initializeImpl方法来实现的，首先调用Camera2ClientBase::initialize(providerPtr)来执行父类的初始化，然后调用mFrameProcessor = new FrameProcessorBase(mDevice)给成员变量mFrameProcessor赋值，从该成员的命名上就可以看出来，它就是用来处理预览帧的，它是继承了Thread类的，我们看到相机的预览实际就是它在一个无限循环当中不断的处理request来完成的。frameworks\av\services\camera\libcameraservice\common\FrameProcessorBase.h类的定义源码如下：
class FrameProcessorBase: public Thread {
public:
explicit FrameProcessorBase(wp<CameraDeviceBase> device);
virtual ~FrameProcessorBase();
struct FilteredListener: virtual public RefBase {
virtual void onResultAvailable(const CaptureResult &result) = 0;
};
// Register a listener for a range of IDs [minId, maxId). Multiple listeners
// can be listening to the same range. Registering the same listener with
// the same range of IDs has no effect.
// sendPartials controls whether partial results will be sent.
status_t registerListener(int32_t minId, int32_t maxId,
const wp<FilteredListener>& listener,
bool sendPartials = true);
status_t removeListener(int32_t minId, int32_t maxId,
const wp<FilteredListener>& listener);
void dump(int fd, const Vector<String16>& args);
protected:
static const nsecs_t kWaitDuration = 10000000; // 10 ms
wp<CameraDeviceBase> mDevice;
virtual bool threadLoop();
Mutex mInputMutex;
Mutex mLastFrameMutex;
struct RangeListener {
int32_t minId;
int32_t maxId;
wp<FilteredListener> listener;
bool sendPartials;
};
List<RangeListener> mRangeListeners;
// Number of partial result the HAL will potentially send.
int32_t mNumPartialResults;
void processNewFrames(const sp<CameraDeviceBase> &device);
virtual bool processSingleFrame(CaptureResult &result,
const sp<CameraDeviceBase> &device);
status_t processListeners(const CaptureResult &result,
const sp<CameraDeviceBase> &device);
CameraMetadata mLastFrame;
};
mFrameProcessor构造好之后，调用mFrameProcessor->run(threadName.string())来启动它，run方法中的参数就是对当前线程的命名，如果我们碰到相机ANR的问题，也可以通过命名来分析这条线程的逻辑。
最后调用mFrameProcessor->registerListener来注册回调。第三个参数的定义类型为FilteredListener，实际参数就是当前的CameraDeviceClient对象，FilteredListener也是定义在FrameProcessorBase.h头文件中，它只有一个方法，onResultAvailable，很明显就是一帧处理好了，需要回调给应用层时，就会通过该方法来完成。
我们接下来在看一下刚才那句Camera2ClientBase::initialize(providerPtr)逻辑，因为它还有很多父类初始化的工作。该方法的源码如下：
template <typename TClientBase>
status_t Camera2ClientBase<TClientBase>::initialize(sp<CameraProviderManager> manager) {
return initializeImpl(manager);
}
template <typename TClientBase>
template <typename TProviderPtr>
status_t Camera2ClientBase<TClientBase>::initializeImpl(TProviderPtr providerPtr) {
ATRACE_CALL();
ALOGV("%s: Initializing client for camera %s", __FUNCTION__,
TClientBase::mCameraIdStr.string());
status_t res;
// Verify ops permissions
res = TClientBase::startCameraOps();
if (res != OK) {
return res;
}
if (mDevice == NULL) {
ALOGE("%s: Camera %s: No device connected",
__FUNCTION__, TClientBase::mCameraIdStr.string());
return NO_INIT;
}
res = mDevice->initialize(providerPtr);
if (res != OK) {
ALOGE("%s: Camera %s: unable to initialize device: %s (%d)",
__FUNCTION__, TClientBase::mCameraIdStr.string(), strerror(-res), res);
return res;
}
wp<CameraDeviceBase::NotificationListener> weakThis(this);
res = mDevice->setNotifyCallback(weakThis);
return OK;
}
它也是一个模板方法，范型TProviderPtr就是我们前面从CameraService类中调用过来时传入的的参数mCameraProviderManager，它的类型为CameraProviderManager。我们一开始说我们可以通过调用CameraManager类的getCameraIdList()方法来获取到当前已成功注册的相机设备的id，那究竟是怎么获取到的呢？就是通过CameraProviderManager类中提供的数据来获取的，因为每一个相机设备的注册信息都会保存在该类中，所以它知道所有的相机设备信息，可以直接返回给我们。接下来我们继续分析mDevice->initialize(providerPtr)这句逻辑的实现。这里的mDevice就是我们前面初始化好的Camera3Device了，frameworks\av\services\camera\libcameraservice\device3\Camera3Device.cpp类的initialize方法的源码如下：
status_t Camera3Device::initialize(sp<CameraProviderManager> manager) {
ATRACE_CALL();
Mutex::Autolock il(mInterfaceLock);
Mutex::Autolock l(mLock);
ALOGV("%s: Initializing HIDL device for camera %s", __FUNCTION__, mId.string());
if (mStatus != STATUS_UNINITIALIZED) {
CLOGE("Already initialized!");
return INVALID_OPERATION;
}
if (manager == nullptr) return INVALID_OPERATION;
sp<ICameraDeviceSession> session;
ATRACE_BEGIN("CameraHal::openSession");
status_t res = manager->openSession(mId.string(), this,
/*out*/ &session);
ATRACE_END();
if (res != OK) {
SET_ERR_L("Could not open camera session: %s (%d)", strerror(-res), res);
return res;
}
res = manager->getCameraCharacteristics(mId.string(), &mDeviceInfo);
if (res != OK) {
SET_ERR_L("Could not retrive camera characteristics: %s (%d)", strerror(-res), res);
session->close();
return res;
}
std::shared_ptr<RequestMetadataQueue> queue;
auto requestQueueRet = session->getCaptureRequestMetadataQueue(
[&queue](const auto& descriptor) {
queue = std::make_shared<RequestMetadataQueue>(descriptor);
if (!queue->isValid() || queue->availableToWrite() <= 0) {
ALOGE("HAL returns empty request metadata fmq, not use it");
queue = nullptr;
// don't use the queue onwards.
}
});
if (!requestQueueRet.isOk()) {
ALOGE("Transaction error when getting request metadata fmq: %s, not use it",
requestQueueRet.description().c_str());
return DEAD_OBJECT;
}
auto resultQueueRet = session->getCaptureResultMetadataQueue(
[&queue = mResultMetadataQueue](const auto& descriptor) {
queue = std::make_unique<ResultMetadataQueue>(descriptor);
if (!queue->isValid() || queue->availableToWrite() <= 0) {
ALOGE("HAL returns empty result metadata fmq, not use it");
queue = nullptr;
// Don't use the queue onwards.
}
});
if (!resultQueueRet.isOk()) {
ALOGE("Transaction error when getting result metadata queue from camera session: %s",
resultQueueRet.description().c_str());
return DEAD_OBJECT;
}
mInterface = std::make_unique<HalInterface>(session, queue);
std::string providerType;
mVendorTagId = manager->getProviderTagIdLocked(mId.string());
return initializeCommonLocked();
}
这里有一句非常重要的逻辑status_t res = manager->openSession(mId.string(), this, /*out*/ &session)，它会在CameraDaemon进程中执行真正的camera的open过程，同时会返回一个session对象给我们，前面我们说过，相机核心的事务其中就包括createCaptureSession，当然两个session不是一个意思，但是这里openSession的逻辑拿到的东西却非常重要，这个我们放到以后再说。最后继续调用initializeCommonLocked完成初始化。initializeCommonLocked方法的源码如下：
status_t Camera3Device::initializeCommonLocked() {
/** Start up status tracker thread */
mStatusTracker = new StatusTracker(this);
status_t res = mStatusTracker->run(String8::format("C3Dev-%s-Status", mId.string()).string());
if (res != OK) {
SET_ERR_L("Unable to start status tracking thread: %s (%d)",
strerror(-res), res);
mInterface->close();
mStatusTracker.clear();
return res;
}
/** Register in-flight map to the status tracker */
mInFlightStatusId = mStatusTracker->addComponent();
/** Create buffer manager */
mBufferManager = new Camera3BufferManager();
mTagMonitor.initialize(mVendorTagId);
/** Start up request queue thread */
mRequestThread = new RequestThread(this, mStatusTracker, mInterface.get());
res = mRequestThread->run(String8::format("C3Dev-%s-ReqQueue", mId.string()).string());
if (res != OK) {
SET_ERR_L("Unable to start request queue thread: %s (%d)",
strerror(-res), res);
mInterface->close();
mRequestThread.clear();
return res;
}
mPreparerThread = new PreparerThread();
internalUpdateStatusLocked(STATUS_UNCONFIGURED);
mNextStreamId = 0;
mDummyStreamId = NO_STREAM;
mNeedConfig = true;
mPauseStateNotify = false;
// Measure the clock domain offset between camera and video/hw_composer
camera_metadata_entry timestampSource =
mDeviceInfo.find(ANDROID_SENSOR_INFO_TIMESTAMP_SOURCE);
if (timestampSource.count > 0 && timestampSource.data.u8[0] ==
ANDROID_SENSOR_INFO_TIMESTAMP_SOURCE_REALTIME) {
mTimestampOffset = getMonoToBoottimeOffset();
}
// Will the HAL be sending in early partial result metadata?
camera_metadata_entry partialResultsCount =
mDeviceInfo.find(ANDROID_REQUEST_PARTIAL_RESULT_COUNT);
if (partialResultsCount.count > 0) {
mNumPartialResults = partialResultsCount.data.i32[0];
mUsePartialResult = (mNumPartialResults > 1);
}
camera_metadata_entry configs =
mDeviceInfo.find(ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS);
for (uint32_t i = 0; i < configs.count; i += 4) {
if (configs.data.i32[i] == HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED &&
configs.data.i32[i + 3] ==
ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS_INPUT) {
mSupportedOpaqueInputSizes.add(Size(configs.data.i32[i + 1],
configs.data.i32[i + 2]));
}
}
return OK;
}
这里又出现了几个对象：Camera3BufferManager、RequestThread、PreparerThread。Camera3BufferManager是用来管理buffer的，RequestThread是用来处理预览的request的，PreparerThread是用来作一些准备工作的，大家知道，因为相机预览是一帧接一帧，不断的在创建、处理请求，频率非常高，一般是30fps，当然，比Android的Vsync信号同步的16ms还是要慢，所以这里开了多条线程用来同时处理不同的工作，以保证相机的效率。
好，到这里，我们这篇博客也就介绍完了，现在回头一想，是不是很奇怪，我们要讲的是openCamera的逻辑，但是从头到尾就只看到了一些对象的创建，没有真正的打开相机设备啊？这也就是我们上面说的status_t res = manager->openSession(mId.string(), this, /*out*/ &session)逻辑的重要性了，真正打开相机的逻辑就是从这里进去处理的，这个我们放在下一次继续讲，openCamera后续还有很大的一段逻辑要我们分析，完成之后，我们才能大体明白相机到底是怎么打开的。