Atlas 500 Docker鏡像安裝ffmpeg+OpenCV環(huán)境（Atlascopco空壓機(jī)）
1 ffmpeg概述
根據(jù)百度百科的定義，ffmpeg是開(kāi)源的音視頻處理庫(kù)，它可以用來(lái)記錄、轉(zhuǎn)換數(shù)字音頻、視頻，并能將其轉(zhuǎn)化為流的計(jì)算機(jī)程序。它包含了非常先進(jìn)的音頻/視頻編解碼庫(kù)libavcodec。ffmpeg視頻采集功能非常強(qiáng)大，不僅可以采集視頻采集卡或USB攝像頭的圖像，還可以進(jìn)行屏幕錄制，同時(shí)還支持以RTP方式將視頻流傳送給支持RTSP的流媒體服務(wù)器，支持直播應(yīng)用。ffmpeg庫(kù)主要包含如下的核心組件：
ffmpeg：用于格式轉(zhuǎn)換、解碼或電視卡即時(shí)編碼等
ffsever：一個(gè) HTTP 多媒體即時(shí)廣播串流服務(wù)器
ffplay：簡(jiǎn)單的播放器，使用ffmpeg 庫(kù)解析和解碼
libavformat：用于各種音視頻封裝格式的生成和解析
libavcodec：用于各種類型聲音/圖像編解碼
libavutil：公共的工具函數(shù)
libswscale：用于視頻場(chǎng)景比例縮放、色彩映射轉(zhuǎn)換
libpostproc：用于后期效果處理
下面給出常見(jiàn)的ffmpeg命令：
#圖片序列合成視頻 ffmpeg -f images -i img%d.jpg myvideo.mpg #將視頻分解成圖片序列 ffmpeg -i myvideo.mpg image%d.jpg #從視頻提取聲音，存為demo.mp3 ffmpeg -i source_video.avi -vn -ar 44100 -ac 2 -ab 192 -f mp3 demo.mp3 #視頻格式轉(zhuǎn)換 ffmpeg -i video_src.mpg video_dest.avi #將.avi轉(zhuǎn)成gif動(dòng)畫(huà) ffmpeg -i video_dest.avi anime.gif #合成視頻和音頻 ffmpeg -i demo.wav -i video_src.avi video_dest.mpg
2 OpenCV概述
根據(jù)百度百科的定義，OpenCV是一個(gè)基于Apache2.0協(xié)議的開(kāi)源跨平臺(tái)計(jì)算機(jī)視覺(jué)和機(jī)器學(xué)習(xí)軟件庫(kù)，它可以運(yùn)行在Linux、Windows、Android和Mac OS操作系統(tǒng)上。它提供了Python、Ruby、MATLAB等語(yǔ)言的接口，實(shí)現(xiàn)了圖像處理和計(jì)算機(jī)視覺(jué)方面的很多通用算法。OpenCV用C++語(yǔ)言編寫(xiě)，輕量且高效，主要傾向于實(shí)時(shí)視覺(jué)應(yīng)用。 當(dāng)前在計(jì)算機(jī)視覺(jué)項(xiàng)目中，經(jīng)常需要用到OpenCV，特別是Python版的接口。
3 docker鏡像安裝ffmpeg+OpenCV環(huán)境
之前的一篇博客《華為Atlas 500小站docker鏡像制作》介紹了如何制作Atlas 500 Docker鏡像，下面的安裝是基于之前的Docker鏡像。首先以admin登錄Atlas 500 智能小站，進(jìn)入開(kāi)發(fā)者模式(develop)。這里我們需要啟動(dòng)Docker鏡像，在啟動(dòng)之前，需要查看一下當(dāng)前的Docker鏡像列表，命令如下：
Euler:~ # docker images
輸出結(jié)果如下：
從上圖輸出結(jié)果可以看出，制作的Docker 鏡像 workload-image:v1.0 在列表中。下面將基于此鏡像來(lái)創(chuàng)建容器，命令如下：
docker run \ --device=/dev/davinci0 \ --device=/dev/davinci_manager \ --device=/dev/hisi_hdc \ --device /dev/devmm_svm \ -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \ -v /home/data/miniD/driver/lib64:/home/data/miniD/driver/lib64 \ -v /run/board_cfg.ini:/run/board_cfg.ini \ -it workload-image:v1.0 bash
執(zhí)行結(jié)果如下所示：
如果執(zhí)行成功，會(huì)進(jìn)入Docker容器的交互環(huán)境，這里可以執(zhí)行Linux相關(guān)命令，其中的用戶為root，而容器ID為e9c222179267 。這個(gè)ID后續(xù)的啟動(dòng)和停止容器等操作都會(huì)用到。此時(shí)容器中的Python環(huán)境還未安裝，下面首先安裝Python環(huán)境和相關(guān)依賴項(xiàng)，命令如下：
apt-get update apt-get install python3.7 apt-get install python3-pip apt-get install libtiff5-dev libjpeg8-dev zlib1g-dev libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python-tk
執(zhí)行成功后，系統(tǒng)中會(huì)有python3.6的命令，沒(méi)錯(cuò)就是python3.6 。繼續(xù)執(zhí)行如下命令安裝：
python3.6 -m pip install --upgrade pip --user -i https://mirrors.huaweicloud.com/repository/pypi/simple python3.6 -m pip install Cython numpy pillow tornado==5.1.0 protobuf \ --user -i https://mirrors.huaweicloud.com/repository/pypi/simple
安裝編輯器vim，命令如下：
apt-get install vim
安裝Python OpenCV 庫(kù)，命令如下：
apt-get install python3-opencv
下面介紹如何安裝ffmpeg，這里的安裝可以按照如下網(wǎng)址從源碼進(jìn)行編譯安裝，這個(gè)過(guò)程比較耗時(shí)，估計(jì)要1~2小時(shí)左右。參考網(wǎng)址如下：? https://gitee.com/ascend/samples/blob/master/cplusplus/environment/opencv_install/README_300_CN.md
下面是按照上述文檔將編譯后的庫(kù)進(jìn)行打包成 ascend_ddk.tar.gz ，它將開(kāi)發(fā)環(huán)境安裝的ffmpeg、opencv庫(kù)導(dǎo)入運(yùn)行環(huán)境中，以提供運(yùn)行使用。執(zhí)行的操作如下：
mkdir $HOME/ascend_ddk scp -r HwHiAiUser@X.X.X.X:/home/HwHiAiUser/ascend_ddk/x86 $HOME/ascend_ddk scp -r HwHiAiUser@X.X.X.X:/usr/lib/x86_64-linux-gnu/lib* $HOME/ascend_ddk/x86/lib
下面給出依賴性文件，如下所示：
PyAV.tar.gz ascend_ddk.tar.gz （編譯好的庫(kù)打包）
其中的PyAV可以從下面的網(wǎng)址進(jìn)行克隆源碼，命令如下：
git clone https://gitee.com/mirrors/PyAV.git
將文件從Atlas 500小站拷貝到當(dāng)前容器中，命令如下：
docker cp /opt/mount/docker05/ascend_ddk.tar.gz e9c222179267:/root/ docker cp /opt/mount/docker05/PyAV.tar.gz e9c222179267:/root/
上傳成功后，可以在容器/root目錄下查看：
解壓安裝，命令如下：
tar zxvf ascend_ddk.tar.gz ################################################# pip3 install Cython apt-get install pkg-config libxcb-shm0-dev libxcb-xfixes0-dev #使opencv能找到ffmpeg cp /root/ascend_ddk/x86/lib/pkgconfig/* /usr/share/pkgconfig ################################################# tar zxvf PyAV.tar.gz cd PyAV python3.6 setup.py build --ffmpeg-dir=/root/ascend_ddk/x86 python3.6 setup.py install
如果編譯安裝PyAV過(guò)程中報(bào)如下錯(cuò)誤，則可以嘗試的解決方案如下：
編譯PyAv報(bào)錯(cuò)： Could not find libavdevice with pkg-config. Could not find libavfilter with pkg-config. 解決方法： 步驟1. cp /root/ascend_ddk/x86/lib/pkgconfig/* /usr/share/pkgconfig/ 步驟2. export PKG_CONFIG_PATH=/usr/share/pkgconfig/
編譯PyAV稍微有點(diǎn)耗時(shí)，請(qǐng)耐心等待。等成功安裝完成后，輸出信息如下所示：
#xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Installing pyav script to /usr/local/bin Installed /usr/local/lib/python3.6/dist-packages/av-8.0.4.dev0-py3.6-linux-aarch64.egg Processing dependencies for av==8.0.4.dev0 Finished processing dependencies for av==8.0.4.dev0
成功安裝后，還需要配置環(huán)境變量，具體操作如下所示：
vim /etc/ld.so.conf.d/ffmpeg.conf
添加如下內(nèi)容：
/root/ascend_ddk/x86/lib
命令行執(zhí)行如下命令，配置生效：
ldconfig
繼續(xù)配置環(huán)境變量：
vim /etc/profile
在末尾追加如下內(nèi)容：
export PATH=$PATH:/root/ascend_ddk/x86/bin export PYTHONPATH=/home/data/miniD/driver/lib64:$PYTHONPATH export LD_LIBRARY_PATH=/home/data/miniD/driver/lib64:$LD_LIBRARY_PATH
命令行執(zhí)行如下命令，配置生效：
source /etc/profile
下面需要驗(yàn)證一下是否可以正常運(yùn)行，命令如下：
cd ~ python3.6 ################Python環(huán)境下############ import av import cv2
正確執(zhí)行，界面如下所示：
如果執(zhí)行import av報(bào)如下錯(cuò)誤，則檢查環(huán)境變量是否配置正確，執(zhí)行如下操作：
ImportError: libavcodec.so.58: cannot open shared object file: No such file or directory
如果import命令都可以正確執(zhí)行，沒(méi)有報(bào)錯(cuò)信息，則說(shuō)明ffmpeg 、OpenCV和PyAV都已經(jīng)安裝成功。首先退出當(dāng)前Docker容器，重新啟動(dòng)容器：
root@e9c222179267:~/dist# exit exit Euler:~ # docker ps -a CONTAINER ID IMAGE COMMAND e9c222179267 workload-image:v1.0 "bash" Euler:~ # docker start e9c222179267 e9c222179267 Euler:~ # docker attach e9c222179267 root@e9c222179267:~#
在容器中輸入如下命令，查看當(dāng)前npu信息：
root@e9c222179267:~# npu-smi info
界面如下所示：
下面給出官網(wǎng)示例的修改版，從攝像頭用rstp協(xié)議拉取視頻流，并通過(guò)resnet50.om?模型的分類，來(lái)確定當(dāng)前的視頻中是否有dog，且識(shí)別dog的種類。 可以代碼如下所示：
# Copyright 2020 Huawei Technologies Co., Ltd # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import cv2 import datetime import argparse import numpy as np import acl import os from PIL import Image from constant import ACL_MEM_MALLOC_HUGE_FIRST, \ ACL_MEMCPY_HOST_TO_DEVICE, ACL_MEMCPY_DEVICE_TO_HOST, \ ACL_ERROR_NONE, IMG_EXT, NPY_FLOAT32 buffer_method = { "in": acl.mdl.get_input_size_by_index, "out": acl.mdl.get_output_size_by_index } def check_ret(message, ret): if ret != ACL_ERROR_NONE: raise Exception("{} failed ret={}" .format(message, ret)) class Net(object): def __init__(self, device_id, model_path): self.device_id = device_id # int self.model_path = model_path # string self.model_id = None # pointer self.context = None # pointer self.input_data = [] self.output_data = [] self.model_desc = None # pointer when using self.load_input_dataset = None self.load_output_dataset = None self.init_resource() def __del__(self): print("Releasing resources stage:") ret = acl.mdl.unload(self.model_id) check_ret("acl.mdl.unload", ret) if self.model_desc: acl.mdl.destroy_desc(self.model_desc) self.model_desc = None while self.input_data: item = self.input_data.pop() ret = acl.rt.free(item["buffer"]) check_ret("acl.rt.free", ret) while self.output_data: item = self.output_data.pop() ret = acl.rt.free(item["buffer"]) check_ret("acl.rt.free", ret) if self.context: ret = acl.rt.destroy_context(self.context) check_ret("acl.rt.destroy_context", ret) self.context = None ret = acl.rt.reset_device(self.device_id) check_ret("acl.rt.reset_device", ret) ret = acl.finalize() check_ret("acl.finalize", ret) print('Resources released successfully.') def init_resource(self): print("init resource stage:") ret = acl.init() check_ret("acl.init", ret) ret = acl.rt.set_device(self.device_id) check_ret("acl.rt.set_device", ret) self.context, ret = acl.rt.create_context(self.device_id) check_ret("acl.rt.create_context", ret) # load_model self.model_id, ret = acl.mdl.load_from_file(self.model_path) check_ret("acl.mdl.load_from_file", ret) print("model_id:{}".format(self.model_id)) self.model_desc = acl.mdl.create_desc() self._get_model_info() print("init resource success") def _get_model_info(self,): ret = acl.mdl.get_desc(self.model_desc, self.model_id) check_ret("acl.mdl.get_desc", ret) input_size = acl.mdl.get_num_inputs(self.model_desc) output_size = acl.mdl.get_num_outputs(self.model_desc) self._gen_data_buffer(input_size, des="in") self._gen_data_buffer(output_size, des="out") def _gen_data_buffer(self, size, des): func = buffer_method[des] for i in range(size): # check temp_buffer dtype temp_buffer_size = func(self.model_desc, i) temp_buffer, ret = acl.rt.malloc(temp_buffer_size, ACL_MEM_MALLOC_HUGE_FIRST) check_ret("acl.rt.malloc", ret) if des == "in": self.input_data.append({"buffer": temp_buffer, "size": temp_buffer_size}) elif des == "out": self.output_data.append({"buffer": temp_buffer, "size": temp_buffer_size}) def _data_interaction(self, dataset, policy=ACL_MEMCPY_HOST_TO_DEVICE): temp_data_buffer = self.input_data \ if policy == ACL_MEMCPY_HOST_TO_DEVICE \ else self.output_data if len(dataset) == 0 and policy == ACL_MEMCPY_DEVICE_TO_HOST: for item in self.output_data: temp, ret = acl.rt.malloc_host(item["size"]) if ret != 0: raise Exception("can't malloc_host ret={}".format(ret)) dataset.append({"size": item["size"], "buffer": temp}) for i, item in enumerate(temp_data_buffer): if policy == ACL_MEMCPY_HOST_TO_DEVICE: ptr = acl.util.numpy_to_ptr(dataset[i]) ret = acl.rt.memcpy(item["buffer"], item["size"], ptr, item["size"], policy) check_ret("acl.rt.memcpy", ret) else: ptr = dataset[i]["buffer"] ret = acl.rt.memcpy(ptr, item["size"], item["buffer"], item["size"], policy) check_ret("acl.rt.memcpy", ret) def _gen_dataset(self, type_str="input"): dataset = acl.mdl.create_dataset() temp_dataset = None if type_str == "in": self.load_input_dataset = dataset temp_dataset = self.input_data else: self.load_output_dataset = dataset temp_dataset = self.output_data for item in temp_dataset: data = acl.create_data_buffer(item["buffer"], item["size"]) _, ret = acl.mdl.add_dataset_buffer(dataset, data) if ret != ACL_ERROR_NONE: ret = acl.destroy_data_buffer(data) check_ret("acl.destroy_data_buffer", ret) def _data_from_host_to_device(self, images): print("data interaction from host to device") # copy images to device self._data_interaction(images, ACL_MEMCPY_HOST_TO_DEVICE) # load input data into model self._gen_dataset("in") # load output data into model self._gen_dataset("out") print("data interaction from host to device success") def _data_from_device_to_host(self): print("data interaction from device to host") res = [] # copy device to host self._data_interaction(res, ACL_MEMCPY_DEVICE_TO_HOST) print("data interaction from device to host success") result = self.get_result(res) self._print_result(result) def run(self, images): self._data_from_host_to_device(images) self.forward() self._data_from_device_to_host() def forward(self): print('execute stage:') ret = acl.mdl.execute(self.model_id, self.load_input_dataset, self.load_output_dataset) check_ret("acl.mdl.execute", ret) self._destroy_databuffer() print('execute stage success') def _print_result(self, result): vals = np.array(result).flatten() top_k = vals.argsort()[-1:-6:-1] print("======== top5 inference results: =============") for j in top_k: if vals[j] >= 0.5 : print(">>>>>>>>>>>>>>>>> find dog >>>>>>>>>>>>>") print("[%d]: %f" % (j, vals[j])) def _destroy_databuffer(self): for dataset in [self.load_input_dataset, self.load_output_dataset]: if not dataset: continue number = acl.mdl.get_dataset_num_buffers(dataset) for i in range(number): data_buf = acl.mdl.get_dataset_buffer(dataset, i) if data_buf: ret = acl.destroy_data_buffer(data_buf) check_ret("acl.destroy_data_buffer", ret) ret = acl.mdl.destroy_dataset(dataset) check_ret("acl.mdl.destroy_dataset", ret) def get_result(self, output_data): result = [] dims, ret = acl.mdl.get_cur_output_dims(self.model_desc, 0) check_ret("acl.mdl.get_cur_output_dims", ret) out_dim = dims['dims'] for temp in output_data: ptr = temp["buffer"] # 轉(zhuǎn)化為float32類型的數(shù)據(jù) data = acl.util.ptr_to_numpy(ptr, tuple(out_dim), NPY_FLOAT32) result.append(data) return result def transfer_pic(input_path): input_path = os.path.abspath(input_path) with Image.open(input_path) as image_file: image_file = image_file.resize((256, 256)) img = np.array(image_file) height = img.shape[0] width = img.shape[1] # 對(duì)圖片進(jìn)行切分，取中間區(qū)域 h_off = (height - 224) // 2 w_off = (width - 224) // 2 crop_img = img[h_off:height - h_off, w_off:width - w_off, :] # rgb to bgr，改變通道順序 img = crop_img[:, :, ::-1] shape = img.shape img = img.astype("float16") img[:, :, 0] -= 104 img[:, :, 1] -= 117 img[:, :, 2] -= 123 img = img.reshape([1] + list(shape)) img = img.transpose([0, 3, 1, 2]) result = np.frombuffer(img.tobytes(), np.float16) return result if __name__ == '__main__': parser = argparse.ArgumentParser() parser.add_argument('--device', type=int, default=0) parser.add_argument('--model_path', type=str, default="./model/resnet50.om") parser.add_argument('--images_path', type=str, default="./data") args = parser.parse_args() print("Using device id:{}\nmodel path:{}\nimages path:{}" .format(args.device, args.model_path, args.images_path)) net = Net(args.device, args.model_path) cap = cv2.VideoCapture('rtsp://xxxxxx:554/Streaming/Channels/101') print(cap) ret,frame = cap.read() while ret: ret,frame = cap.read() cv2.imwrite(args.images_path+'/frame.jpg', frame) images_list = [os.path.join(args.images_path, img) for img in os.listdir(args.images_path) if os.path.splitext(img)[1] in IMG_EXT] for image in images_list: print("images:{}".format(image)) img = transfer_pic(image) net.run([img]) if cv2.waitKey(1) & 0xFF == ord('q'): print("===break=========") break print("cv2.destroyAllWindows()") cv2.destroyAllWindows() cap.release() print("*****run finish******")
執(zhí)行如下命令，啟動(dòng)示例：
root@e9c222179267:~/dist# python3 ./src/main.py Using device id:0 model path:./model/resnet50.om images path:./data init resource stage: model_id:1 init resource success
關(guān)于示例項(xiàng)目其他文件和模型，可以參考官網(wǎng)samples示例【基于 Caffe ResNet-50 網(wǎng)絡(luò)實(shí)現(xiàn)圖片分類（同步推理）】。參考網(wǎng)址為：
https://gitee.com/ascend/samples/tree/master/python/level2_simple_inference/1_classification/resnet50_imagenet_classification
Atlas 200 DK開(kāi)發(fā)者套件 Docker OpenCV 鏡像服務(wù)
版權(quán)聲明：本文內(nèi)容由網(wǎng)絡(luò)用戶投稿，版權(quán)歸原作者所有，本站不擁有其著作權(quán)，亦不承擔(dān)相應(yīng)法律責(zé)任。如果您發(fā)現(xiàn)本站中有涉嫌抄襲或描述失實(shí)的內(nèi)容，請(qǐng)聯(lián)系我們jiasou666@gmail.com 處理，核實(shí)后本網(wǎng)站將在24小時(shí)內(nèi)刪除侵權(quán)內(nèi)容。
版權(quán)聲明：本文內(nèi)容由網(wǎng)絡(luò)用戶投稿，版權(quán)歸原作者所有，本站不擁有其著作權(quán)，亦不承擔(dān)相應(yīng)法律責(zé)任。如果您發(fā)現(xiàn)本站中有涉嫌抄襲或描述失實(shí)的內(nèi)容，請(qǐng)聯(lián)系我們jiasou666@gmail.com 處理，核實(shí)后本網(wǎng)站將在24小時(shí)內(nèi)刪除侵權(quán)內(nèi)容。
相關(guān)文章
