Android音視頻 MediaCodec框架-啟動編碼

簡述

上一節(jié)我們介紹了MediaCodec框架創(chuàng)建編碼器流程，編解碼的流程其實(shí)基本是一樣的，只是底層的最終的實(shí)現(xiàn)組件不同，所以我們只看啟動編碼流程。

MediaCodec啟動編碼

從MediaCodec的start方法開始。

1.1 MediaCodec.start
調(diào)用jni方法native_start

public final void start() {native_start();
}

1.2 native_start
調(diào)用JMediaCodec的start方法。

static void android_media_MediaCodec_start(JNIEnv *env, jobject thiz) {ALOGV("android_media_MediaCodec_start");sp<JMediaCodec> codec = getMediaCodec(env, thiz);// ... JMediaCodec狀態(tài)檢測// 詳見1.3status_t err = codec->start();// ...
}

1.3 JMediaCodec::start
調(diào)用MediaCodec的start方法。

status_t JMediaCodec::start() {// 詳見1.4return mCodec->start();
}

1.4 MediaCodec::start
發(fā)送kWhatStart的AMessage通知CCodec2 start，這個(gè)流程和init很類似，處理消息start的流程詳見1.5

status_t MediaCodec::start() {sp<AMessage> msg = new AMessage(kWhatStart, this);sp<AMessage> callback;status_t err;std::vector<MediaResourceParcel> resources;resources.push_back(MediaResource::CodecResource(mFlags & kFlagIsSecure,toMediaResourceSubType(mDomain)));resources.push_back(MediaResource::GraphicMemoryResource(1));for (int i = 0; i <= kMaxRetry; ++i) {if (i > 0) {// ...sp<AMessage> response;err = PostAndAwaitResponse(mConfigureMsg, &response);// ...}// ...}return err;
}

1.5 MediaCodec::onMessageReceived
消息處理方法，調(diào)用CCodec的initiateStart

void MediaCodec::onMessageReceived(const sp<AMessage> &msg) {// ...case kWhatStart:{// ...// 更新狀態(tài)到STARTINGsetState(STARTING);// 調(diào)用CCodec的initiateStart，詳見1.6mCodec->initiateStart();break;}// ...
}

1.6 CCodec::initiateStart
這里只是修改了狀態(tài)到STARTING，然后做了消息轉(zhuǎn)發(fā)。

void CCodec::initiateStart() {auto setStarting = [this] {Mutexed<State>::Locked state(mState);if (state->get() != ALLOCATED) {return UNKNOWN_ERROR;}state->set(STARTING);return OK;};if (tryAndReportOnError(setStarting) != OK) {return;}(new AMessage(kWhatStart, this))->post();
}

1.7 CCodec::onMessageReceived
調(diào)用CCodec::start方法。

void CCodec::onMessageReceived(const sp<AMessage> &msg) {// ...case kWhatStart: {// 調(diào)用start方法setDeadline(now, 1500ms, "start");start();break;}// ...
}

1.8 CCodec::start
該方法主要做了幾件事：
調(diào)用Component的start方法，Codec2的Component都是基于SimpleC2Component實(shí)現(xiàn)的，SimpleC2Component中處理了一些狀態(tài)管理的邏輯，不同的編解碼組件繼承了SimpleC2Component，實(shí)現(xiàn)生命周期回調(diào)方法，例如onInit等。
從CCodecConfig獲取mOutputFormat或者inputFormat信息，前者是解碼時(shí)會有，后者是編碼才會有的。
調(diào)用CCodecBufferChannel::start。

void CCodec::start() {// ...// 調(diào)用Component start,Codec2的Component都是基于SimpleC2Component實(shí)現(xiàn)的，SimpleC2Component中處理了一些狀態(tài)管理的邏輯// 不同的編解碼最終實(shí)現(xiàn)都繼承于SimpleC2Component，然后實(shí)現(xiàn)對應(yīng)的生命周期回調(diào)方法做自己的事，由于我們H264的實(shí)現(xiàn)里沒有做什么事，就不看了。c2_status_t err = comp->start();if (err != C2_OK) {mCallback->onError(toStatusT(err, C2_OPERATION_Component_start),ACTION_CODE_FATAL);return;}sp<AMessage> inputFormat;sp<AMessage> outputFormat;status_t err2 = OK;bool buffersBoundToCodec = false;{// CCodecConfig是在之前初始化的，里面有配置信息，這里mOutputFormat是解碼時(shí)候才有的，而inputFormat則是編碼的時(shí)候才有// mInputSurface是編碼時(shí)候配置了InputSurface時(shí)傳入的參數(shù)，表示編碼數(shù)據(jù)來源于一個(gè)Surface// 我們之前在SurfaceFlinger章節(jié)說過，Surface是表示一個(gè)窗口，可以作為圖像BufferQueue的一個(gè)生產(chǎn)者。Mutexed<std::unique_ptr<Config>>::Locked configLocked(mConfig);const std::unique_ptr<Config> &config = *configLocked;inputFormat = config->mInputFormat;outputFormat = config->mOutputFormat = config->mOutputFormat->dup();if (config->mInputSurface) {err2 = config->mInputSurface->start();config->mInputSurfaceDataspace = config->mInputSurface->getDataspace();}buffersBoundToCodec = config->mBuffersBoundToCodec;}if (err2 != OK) {mCallback->onError(err2, ACTION_CODE_FATAL);return;}// 調(diào)用CCodecBufferChannel::start,詳見1.9err2 = mChannel->start(inputFormat, outputFormat, buffersBoundToCodec);if (err2 != OK) {mCallback->onError(err2, ACTION_CODE_FATAL);return;}// ... 更新狀態(tài)std::map<size_t, sp<MediaCodecBuffer>> clientInputBuffers;// 根據(jù)input里numSlots數(shù)量，調(diào)用input->buffers->requestNewBuffer創(chuàng)建buffer填充clientInputBuffers。err2 = mChannel->prepareInitialInputBuffers(&clientInputBuffers);if (err2 != OK) {ALOGE("Initial preparation for Input Buffers failed");mCallback->onError(err2, ACTION_CODE_FATAL);return;}// 回調(diào)MediaCodec onStartCompletedmCallback->onStartCompleted();mChannel->requestInitialInputBuffers(std::move(clientInputBuffers));
}

1.9 CCodecBufferChannel::start
這個(gè)方法很長，主要是一些參數(shù)的初始化，分為編碼和解碼的情況，其中最重要的參數(shù)就是C2BlockPool和input/output。
以編碼為例，input里面有一個(gè)buffers，這個(gè)buffers有多種類型，buffers會持有一個(gè)C2BlockPool來分配內(nèi)存，而C2BlockPool又會通過Allocator分配C2Buffer，這里的Allocator也是在hal層，Allocator是通過AllocatorStore到hal獲取到。
所以相當(dāng)于在上層使用一個(gè)buffers來控制buffer到分配釋放等，這個(gè)buffers類型比如LinearInputBuffers，GraphicInputBuffers，GraphicMetadataInputBuffers等，而buffers會通過C2BlockPool分配buffer，而C2BlockPool會通過Allocator到hal層獲取buffer。
解碼的情況類似，這里就不細(xì)說了。

status_t CCodecBufferChannel::start(const sp<AMessage> &inputFormat,const sp<AMessage> &outputFormat,bool buffersBoundToCodec) {C2StreamBufferTypeSetting::input iStreamFormat(0u);C2StreamBufferTypeSetting::output oStreamFormat(0u);C2ComponentKindSetting kind;C2PortReorderBufferDepthTuning::output reorderDepth;C2PortReorderKeySetting::output reorderKey;C2PortActualDelayTuning::input inputDelay(0);C2PortActualDelayTuning::output outputDelay(0);C2ActualPipelineDelayTuning pipelineDelay(0);C2SecureModeTuning secureMode(C2Config::SM_UNPROTECTED);// ... 參數(shù)初始化以及檢測// 由C2AllocateStore來獲取Allocation，而Allocation是用于分配buffer的，有不同類型的buffer，對應(yīng)底層分配的內(nèi)存可能也不同，例如dmastd::shared_ptr<C2AllocatorStore> allocatorStore = GetCodec2PlatformAllocatorStore();int poolMask = GetCodec2PoolMask();C2PlatformAllocatorStore::id_t preferredLinearId = GetPreferredLinearAllocatorId(poolMask);// 編碼if (inputFormat != nullptr) {// ... 參數(shù)配置// 構(gòu)造C2BlockPoolstd::shared_ptr<C2BlockPool> pool;{Mutexed<BlockPools>::Locked pools(mBlockPools);// ... 參數(shù)檢查配置if ((poolMask >> pools->inputAllocatorId) & 1) {// 根據(jù)inputAllocatorId構(gòu)造C2BlockPool，C2BlockPool里面持有Allocator，通過擦歐總Allocator管理Buffer的構(gòu)建err = CreateCodec2BlockPool(pools->inputAllocatorId, nullptr, &pool);// ...} else {err = C2_NOT_FOUND;}// ...異常處理pools->inputPool = pool;}bool forceArrayMode = false;Mutexed<Input>::Locked input(mInput);// ...構(gòu)造填充input對象// 其中input->buffers會根據(jù)buffer的類型不同而不同// 這里的關(guān)系是input->buffers會最終提供給CCodec分配Buffer的能力，而input->buffers是通過前面構(gòu)造的C2BlockPool來獲取或者釋放buffer// 而C2BlockPool通過持有的Allocator來分配不同Buffer，不同的Buffer的區(qū)別在于底層可能使用不同的系統(tǒng)調(diào)用來分配內(nèi)存，可能是dma之類的。input->buffers->setFormat(inputFormat);if (err == C2_OK) {input->buffers->setPool(pool);} else {// TODO: error}if (forceArrayMode) {input->buffers = input->buffers->toArrayMode(numInputSlots);}}// 解碼，這里和上面編碼類似，主要是初始化一些必要對象。if (outputFormat != nullptr) {sp<IGraphicBufferProducer> outputSurface;uint32_t outputGeneration;int maxDequeueCount = 0;{Mutexed<OutputSurface>::Locked output(mOutputSurface);maxDequeueCount = output->maxDequeueBuffers = numOutputSlots +reorderDepth.value + mRenderingDepth;outputSurface = output->surface ?output->surface->getIGraphicBufferProducer() : nullptr;if (outputSurface) {output->surface->setMaxDequeuedBufferCount(output->maxDequeueBuffers);}outputGeneration = output->generation;}bool graphic = (oStreamFormat.value == C2BufferData::GRAPHIC);C2BlockPool::local_id_t outputPoolId_;C2BlockPool::local_id_t prevOutputPoolId;{Mutexed<BlockPools>::Locked pools(mBlockPools);// ... 初始化BlockPools，和解碼流程構(gòu)造C2BlockPools類似。}Mutexed<Output>::Locked output(mOutput);// ... 構(gòu)造output，和上面input類似// 如果接受輸出是一個(gè)Surface，通知給Componentif (outputSurface) {mComponent->setOutputSurface(outputPoolId_,outputSurface,outputGeneration,maxDequeueCount);} else {// ...}// ...}// 編解碼監(jiān)測器初始化if (inputFormat || outputFormat) {Mutexed<PipelineWatcher>::Locked watcher(mPipelineWatcher);watcher->inputDelay(inputDelayValue).pipelineDelay(pipelineDelayValue).outputDelay(outputDelayValue).smoothnessFactor(kSmoothnessFactor);watcher->flush();}mInputMetEos = false;// 初始化buffer的鎖mSync.start();return OK;
}

MediaCodec 獲取buffer index

在創(chuàng)建并啟動解碼器后，我們會通過dequeueInputBuffer獲取一個(gè)Buffer index，然后再通過getInputBuffers獲取所有Buffer數(shù)組，然后根據(jù)索引在Buffer數(shù)組中獲取buffer，往里面寫入需要編碼的數(shù)據(jù)，接下來我們來看看這個(gè)流程。

2.1 MediaCodec.dequeueInputBuffer
jni調(diào)用native_dequeueInputBuffer。

public final int dequeueInputBuffer(long timeoutUs) {// ...// 詳見2.2int res = native_dequeueInputBuffer(timeoutUs);// ...return res;
}

2.2 android_media_MediaCodec_dequeueInputBuffer
調(diào)用JMediaCodec::dequeueInputBuffer

static jint android_media_MediaCodec_dequeueInputBuffer(JNIEnv *env, jobject thiz, jlong timeoutUs) {// ...size_t index;// JMediaCodec::dequeueInputBuffer,詳見2.3status_t err = codec->dequeueInputBuffer(&index, timeoutUs);if (err == OK) {return (jint) index;}return throwExceptionAsNecessary(env, err, codec);
}

2.3 JMediaCodec::dequeueInputBuffer
C++層的MediaCodec，MediaCodec::dequeueInputBuffer

status_t JMediaCodec::dequeueInputBuffer(size_t *index, int64_t timeoutUs) {// 詳見2.4return mCodec->dequeueInputBuffer(index, timeoutUs);
}

2.4 MediaCodec::dequeueInputBuffer
發(fā)送kWhatDequeueInputBuffer消息。

status_t MediaCodec::dequeueInputBuffer(size_t *index, int64_t timeoutUs) {sp<AMessage> msg = new AMessage(kWhatDequeueInputBuffer, this);msg->setInt64("timeoutUs", timeoutUs);sp<AMessage> response;status_t err;// 發(fā)送kWhatDequeueInputBuffer消息，處理邏輯詳見2.5if ((err = PostAndAwaitResponse(msg, &response)) != OK) {return err;}CHECK(response->findSize("index", index));return OK;
}

2.5 MediaCodec::onMessageReceived
調(diào)用handleDequeueInputBuffer處理，并且發(fā)送kWhatDequeueInputTimedOut消息來配置超時(shí)監(jiān)測。

void MediaCodec::onMessageReceived(const sp<AMessage> &msg) {switch (msg->what()) {//...case kWhatDequeueInputBuffer:{sp<AReplyToken> replyID;CHECK(msg->senderAwaitsResponse(&replyID));// ...// 詳見2.6if (handleDequeueInputBuffer(replyID, true /* new request */)) {break;}// ... 通過發(fā)送kWhatDequeueInputTimedOut消息配置超時(shí)監(jiān)測break;}// ...}
}

2.6 MediaCodec::handleDequeueInputBuffer
通過dequeuePortBuffer獲取buffer index，然后返回index。

bool MediaCodec::handleDequeueInputBuffer(const sp<AReplyToken> &replyID, bool newRequest) {// ...異常處理// 詳見2.7ssize_t index = dequeuePortBuffer(kPortIndexInput);if (index < 0) {CHECK_EQ(index, -EAGAIN);return false;}// 返回indexsp<AMessage> response = new AMessage;response->setSize("index", index);response->postReply(replyID);return true;
}

2.7 MediaCodec::dequeuePortBuffer
mAvailPortBuffers里是兩個(gè)int數(shù)組，分別給編碼和解碼使用，這里的int數(shù)組用于存儲可以使用的buffer index。這個(gè)方法做的事就是從這個(gè)int數(shù)組里獲取一個(gè)index，并且將它從數(shù)組里移除。

ssize_t MediaCodec::dequeuePortBuffer(int32_t portIndex) {CHECK(portIndex == kPortIndexInput || portIndex == kPortIndexOutput);BufferInfo *info = peekNextPortBuffer(portIndex);if (!info) {return -EAGAIN;}// 從availBuffers獲取可用buffer index，并且把他從數(shù)組中移除。  std::list<size_t> *availBuffers = &mAvailPortBuffers[portIndex];size_t index = *availBuffers->begin();CHECK_EQ(info, &mPortBuffers[portIndex][index]);availBuffers->erase(availBuffers->begin());// ...return index;
}

MediaCodec 獲取input buffer數(shù)組

3.1 MediaCodec.getInputBuffers

public ByteBuffer[] getInputBuffers() {synchronized (mBufferLock) {// ...// java層有一個(gè)buffer數(shù)組緩存，第一次需要通過jni去獲取，詳見3.2if (mCachedInputBuffers == null) {cacheBuffersLocked(true /* input */);}// ...return mCachedInputBuffers;}
} 

3.2 MediaCodec.cacheBuffersLocked
通過jni獲取buffers，并緩存記錄。

private void cacheBuffersLocked(boolean input) {ByteBuffer[] buffers = null;try {// 通過jni獲取buffer，詳見3.3buffers = getBuffers(input);invalidateByteBuffersLocked(buffers);} catch (IllegalStateException e) {// we don't get buffers in async mode}// ...// 緩存記錄結(jié)果if (input) {mCachedInputBuffers = buffers;} else {mCachedOutputBuffers = buffers;}
}

3.3 JMediaCodec::getBuffers
jni方法通過調(diào)用JMediaCodec的getBuffers，而JMediaCodec又通過調(diào)用c++層的MediaCodec的getInputBuffers來獲取buffers。

status_t JMediaCodec::getBuffers(JNIEnv *env, bool input, jobjectArray *bufArray) const {Vector<sp<MediaCodecBuffer> > buffers;// 詳見3.4status_t err =input? mCodec->getInputBuffers(&buffers): mCodec->getOutputBuffers(&buffers);// ...將C++層的buffers通過jni關(guān)聯(lián)給java層return OK;
}

3.4 MediaCodec::getInputBuffers
發(fā)送kWhatGetBuffers。

status_t MediaCodec::getInputBuffers(Vector<sp<MediaCodecBuffer> > *buffers) const {sp<AMessage> msg = new AMessage(kWhatGetBuffers, this);msg->setInt32("portIndex", kPortIndexInput);msg->setPointer("buffers", buffers);sp<AMessage> response;return PostAndAwaitResponse(msg, &response);
}

3.5 MediaCodec::onMessageReceived
我們目前流程是在獲取inputBuffer，通過inputBuffer給編碼器傳入編碼前的數(shù)據(jù)，而如果我們配置了InputSurface，以InputSurface為輸入，則不需要獲取Buffer。
通過CCodecBufferChannel::getInputBufferArray來獲取Buffer數(shù)組。

void MediaCodec::onMessageReceived(const sp<AMessage> &msg) {// ...case kWhatGetBuffers:{// ...異常處理dstBuffers->clear();// 通過getInputBufferArray獲取buffer，如果配置來InputSurface則不會走這個(gè)模式。詳見3.6if (portIndex != kPortIndexInput || !mHaveInputSurface) {if (portIndex == kPortIndexInput) {mBufferChannel->getInputBufferArray(dstBuffers);} else {mBufferChannel->getOutputBufferArray(dstBuffers);}}mApiUsageMetrics.isArrayMode = true;(new AMessage)->postReply(replyID);break;}// ...
}

3.6 CCodecBufferChannel::getInputBufferArray
這里會調(diào)用input->buffers->getArray來獲取Buffer，這里input->buffers是在start的過程中就已經(jīng)初始化的，具體是在1.8節(jié)，調(diào)用mChannel->prepareInitialInputBuffers時(shí)候分配的，其中會循環(huán)調(diào)用buffers的requestNewBuffer方法，我們之前說過buffers里面有pool，而pool會通過allocator來分配buffer。

void CCodecBufferChannel::getInputBufferArray(Vector<sp<MediaCodecBuffer>> *array) {array->clear();Mutexed<Input>::Locked input(mInput);if (!input->buffers) {ALOGE("getInputBufferArray: No Input Buffers allocated");return;}if (!input->buffers->isArrayMode()) {input->buffers = input->buffers->toArrayMode(input->numSlots);}input->buffers->getArray(array);
}

MediacCodec queueInputBuffer

4.1 MediaCodec.queueInputBuffer
調(diào)用jni層native_queueInputBuffer

public final void queueInputBuffer(int index,int offset, int size, long presentationTimeUs, int flags)throws CryptoException {// ...try {// 詳見4.2native_queueInputBuffer(index, offset, size, presentationTimeUs, flags);} catch (CryptoException | IllegalStateException e) {revalidateByteBuffer(mCachedInputBuffers, index, true /* input */);throw e;}
}

4.2 android_media_MediaCodec_queueInputBuffer
和之前一樣，jni會調(diào)用JMediaCodec的queueInputBuffer方法。

static void android_media_MediaCodec_queueInputBuffer(JNIEnv *env,jobject thiz,jint index,jint offset,jint size,jlong timestampUs,jint flags) {// ...// 詳見4.3status_t err = codec->queueInputBuffer(index, offset, size, timestampUs, flags, &errorDetailMsg);throwExceptionAsNecessary(env, err, ACTION_CODE_FATAL,codec->getExceptionMessage(errorDetailMsg.c_str()).c_str());
}

4.3 JMediaCodec::queueInputBuffer
調(diào)用MediaCodec的queueInputBuffer

status_t JMediaCodec::queueInputBuffer(size_t index,size_t offset, size_t size, int64_t timeUs, uint32_t flags,AString *errorDetailMsg) {//詳見4.4return mCodec->queueInputBuffer(index, offset, size, timeUs, flags, errorDetailMsg);
}

4.4 MediaCodec::queueInputBuffer
發(fā)送kWhatQueueInputBuffer消息。

status_t MediaCodec::queueInputBuffer(size_t index,size_t offset,size_t size,int64_t presentationTimeUs,uint32_t flags,AString *errorDetailMsg) {if (errorDetailMsg != NULL) {errorDetailMsg->clear();}sp<AMessage> msg = new AMessage(kWhatQueueInputBuffer, this);msg->setSize("index", index);msg->setSize("offset", offset);msg->setSize("size", size);msg->setInt64("timeUs", presentationTimeUs);msg->setInt32("flags", flags);msg->setPointer("errorDetailMsg", errorDetailMsg);sp<AMessage> response;return PostAndAwaitResponse(msg, &response);
}

4.5 MediaCodec::onMessageReceived
調(diào)用onQueueInputBuffer處理任務(wù)。

void MediaCodec::onMessageReceived(const sp<AMessage> &msg) {//...case kWhatQueueInputBuffer:{// ...status_t err = UNKNOWN_ERROR;if (!mLeftover.empty()) {mLeftover.push_back(msg);size_t index;msg->findSize("index", &index);err = handleLeftover(index);} else {// 詳見4.6err = onQueueInputBuffer(msg);}PostReplyWithError(replyID, err);break;}// ...
}

4.6 MediaCodec::onQueueInputBuffer
處理一些參數(shù)的檢查和傳遞，這里分需要加密和不需要加密的兩種場景,我們不看加密的邏輯，這里主要構(gòu)建來一個(gè)新的buffer，把數(shù)據(jù)填充，并且調(diào)用attachBuffer把queue進(jìn)來的buffer里面的數(shù)據(jù)拷貝到新建到buffer里。
最后調(diào)用queueInputBuffer。

status_t MediaCodec::onQueueInputBuffer(const sp<AMessage> &msg) {// ... 參數(shù)預(yù)處理和檢查，將一些參數(shù)配置到buffer// ... 加密邏輯if (c2Buffer || memory) {sp<AMessage> tunings = NULL;if (msg->findMessage("tunings", &tunings) && tunings != NULL) {onSetParameters(tunings);}status_t err = OK;if (c2Buffer) {// 拷貝數(shù)據(jù)到新的buffererr = mBufferChannel->attachBuffer(c2Buffer, buffer);} else if (memory) {AString errorDetailMsg;err = mBufferChannel->attachEncryptedBuffer(memory, (mFlags & kFlagIsSecure), key, iv, mode, pattern,offset, subSamples, numSubSamples, buffer, &errorDetailMsg);if (err != OK && hasCryptoOrDescrambler()&& (mFlags & kFlagUseCryptoAsync)) {// ... 加密邏輯}} else {// error log}// ...}// ...if (hasCryptoOrDescrambler() && !c2Buffer && !memory) {// 需要加密場景} else {// 調(diào)用queueInputBuffer，詳見4.7err = mBufferChannel->queueInputBuffer(buffer);// ...}// ...return err;
}

4.7 CCodecBufferChannel::queueInputBuffer
調(diào)用queueInputBufferInternal。

status_t CCodecBufferChannel::queueInputBuffer(const sp<MediaCodecBuffer> &buffer) {QueueGuard guard(mSync);if (!guard.isRunning()) {ALOGD("[%s] No more buffers should be queued at current state.", mName);return -ENOSYS;}return queueInputBufferInternal(buffer);
}

4.8 CCodecBufferChannel::queueInputBufferInternal
這里主要會解析buffer，然后將buffer里的信息封裝到一個(gè)C2Work中，然后把C2Work存到一個(gè)C2Work數(shù)組，調(diào)用Component的queue來處理這個(gè)這個(gè)任務(wù)。
這里Component的queue會在hal層調(diào)用到對方的queue_nb方法，這里就是SimpleC2Component的::SimpleC2Component。

status_t CCodecBufferChannel::queueInputBufferInternal(sp<MediaCodecBuffer> buffer,std::shared_ptr<C2LinearBlock> encryptedBlock,size_t blockSize) {// ...獲取buffer里的參數(shù)// ...構(gòu)造一個(gè)C2Work數(shù)組和一個(gè)C2Work，后續(xù)最終會提交這個(gè)數(shù)組，將信息封裝在C2Work中std::list<std::unique_ptr<C2Work>> items;std::unique_ptr<C2Work> work(new C2Work);work->input.ordinal.timestamp = timeUs;work->input.ordinal.frameIndex = mFrameIndex++;work->input.ordinal.customOrdinal = timeUs;work->input.buffers.clear();sp<Codec2Buffer> copy;bool usesFrameReassembler = false;if (buffer->size() > 0u) {Mutexed<Input>::Locked input(mInput);std::shared_ptr<C2Buffer> c2buffer;// 將buffer數(shù)據(jù)存到c2bufferif (!input->buffers->releaseBuffer(buffer, &c2buffer, false)) {return -ENOENT;}// ...if (input->frameReassembler) {usesFrameReassembler = true;input->frameReassembler.process(buffer, &items);} else {// ...// 將c2buffer存到C2Work里。work->input.buffers.push_back(c2buffer);if (encryptedBlock) {work->input.infoBuffers.emplace_back(C2InfoBuffer::CreateLinearBuffer(kParamIndexEncryptedBuffer,encryptedBlock->share(0, blockSize, C2Fence())));}}} else if (eos) {// ...}if (usesFrameReassembler) {// ...} else {work->input.flags = (C2FrameData::flags_t)flags;// TODO: fill info'swork->input.configUpdate = std::move(mParamsToBeSet);if (tunnelFirstFrame) {C2StreamTunnelHoldRender::input tunnelHoldRender{0u /* stream */,C2_TRUE /* value */};work->input.configUpdate.push_back(C2Param::Copy(tunnelHoldRender));}work->worklets.clear();work->worklets.emplace_back(new C2Worklet);// 將C2Work存到items里items.push_back(std::move(work));eos = eos && buffer->size() > 0u;}if (eos) {// ...}c2_status_t err = C2_OK;if (!items.empty()) {// ...// 將C2Work數(shù)組提交給Component(通過hal層到真正實(shí)現(xiàn)編解碼邏輯實(shí)現(xiàn))，這里是SimpleC2Component::queue_nb,詳見4.9err = mComponent->queue(&items);}if (err != C2_OK) {// ...} else {// ...釋放buffer}feedInputBufferIfAvailableInternal();return err;
}

4.9 SimpleC2Component::queue_nb
這里把前面?zhèn)鬟^來的C2Work都放到mWorkQueue中，然后發(fā)送了kWhatProcess消息觸發(fā)處理邏輯。

c2_status_t
SimpleC2Component::queue_nb(std::list<std::unique_ptr<C2Work>> *const items) {{Mutexed<ExecState>::Locked state(mExecState);if (state->mState != RUNNING) {return C2_BAD_STATE;}}bool queueWasEmpty = false;{Mutexed<WorkQueue>::Locked queue(mWorkQueue);queueWasEmpty = queue->empty();// 將items里面的數(shù)據(jù)放到mWorkQueue中while (!items->empty()) {queue->push_back(std::move(items->front()));items->pop_front();}}if (queueWasEmpty) {// 發(fā)送kWhatProcess，詳見4.10  (new AMessage(WorkHandler::kWhatProcess, mHandler))->post();}return C2_OK;
}

4.10 SimpleC2Component::processQueue
這里主要就是會調(diào)用process方法，這個(gè)方法是由子類實(shí)現(xiàn)的，SimpleC2Component是用于管理流程框架的模版，它的子類來實(shí)現(xiàn)具體編解碼邏輯。
例如H264的編解碼就是由子類C2SoftAacDec和C2SoftAacEnc實(shí)現(xiàn)的，我們本節(jié)不會介紹編解碼的細(xì)節(jié)，下一節(jié)會介紹H264編解碼的一些核心原理。

bool SimpleC2Component::processQueue() {// ...{Mutexed<WorkQueue>::Locked queue(mWorkQueue);if (queue->empty()) {return false;}generation = queue->generation();drainMode = queue->drainMode();isFlushPending = queue->popPendingFlush();// 從隊(duì)列中取出最前面的任務(wù)work = queue->pop_front();hasQueuedWork = !queue->empty();}if (isFlushPending) {// flush的回調(diào)，這里SimpleC2Component是一個(gè)處理流程的架構(gòu)，最終需要由它的子類來實(shí)現(xiàn)對應(yīng)生命周期所做的事。  c2_status_t err = onFlush_sm();// ...}if (!mOutputBlockPool) {// ...創(chuàng)建用于管理輸出Buffer的Pool}// ...// ... input內(nèi)buffer的檢測// 調(diào)用 process，也是由子類實(shí)現(xiàn)的。  // 傳入的參數(shù)是work和用于管理輸出buffer的Poolprocess(work, mOutputBlockPool);// ...return hasQueuedWork;
}

小結(jié)

本節(jié)介紹了MediaCodec編解碼的流程，從java層MediaCodec調(diào)用到JMediaCodec，然后調(diào)用C++層到MediaCodec，通過發(fā)送對應(yīng)流程的消息，觸發(fā)CCodec，而里面通過一個(gè)CCodecBufferChannel來管理這些邏輯，CCodecBufferChannel里面分input和output分別用于管理編解碼，他們中都有一個(gè)buffers用于管理所有buffer，通過一個(gè)C2BlockPool來管理buffer的分配，C2BlockPool內(nèi)有一個(gè)Allocator，會通過hal層進(jìn)行最終的buffer分配釋放。
CCodecBufferChannel工作會通過Component調(diào)用hal層，傳參數(shù)到SimpleC2Component，SimpleC2Component的子類進(jìn)行最終的編解碼操作。
下一節(jié)我們會介紹H264編解碼的核心原理。