昇騰CANN開發&運行環境搭建
大家好,我是Copy攻城獅,有幸參加了【昇騰CANN訓練營】。昇騰CANN訓練營包含一營(應用開發營)、二營(模型開發營)、三營(算子開發營),每營分別開設3節課程,共計9節課程。今天是和一營長學習【昇騰CANN開發&運行環境部署】,謹以此文記錄一下學習過程。
目標
開通 AIECS 并升級環境
安裝驅動
安裝 nnrt
安裝 Toolkit
安裝 MindStudio
跑通 MindStudio 模板APP之ResNet50
大家好,我是Copy攻城獅,有幸參加了【昇騰CANN訓練營】。昇騰CANN訓練營包含一營(應用開發營)、二營(模型開發營)、三營(算子開發營),每營分別開設3節課程,共計9節課程。今天是和一營長學習【昇騰CANN開發&運行環境部署】,謹以此文記錄一下學習過程。
目標
開通 AIECS 并升級環境
安裝驅動
安裝 nnrt
安裝 Toolkit
安裝 MindStudio
跑通 MindStudio 模板APP之ResNet50
坑點
MindStudio 運行依賴特定的環境,因此網頁版的 CloudShell 和 本地的 Xshell 都不能成功調起 MindStudio 界面,按照老師的教程,可以使用MobaXterm。
推薦使用默認的 HwHiAIUser 用戶,但需留意有些過程需要使用 root 用戶,比如驅動的安裝。
各種環境變量配到吐血,稍不留意就會掉入深坑中;此外在安裝 Toolkit 時如果遇到終端窗口意外退出之后,可嘗試先卸載再重新安裝。
一定要留意 HwHiAiUser 的權限,在安裝 MindStudio 時也需文件讀寫的權限。
流水
時間如流水,學習完再操作完到簡單記錄流水賬,就差不多可以遇見凌晨4點的羊城了……
此處記錄方便自己后續查閱。
AIECS 開通
值得留意的是規格選?AI加速型,AI加速型實例提供AI芯片,主要用于機器視覺、圖像分類、語音識別等AI推理場景,此次用單片的 Ascend 310;系統選用的Ubuntu 18.04 server 64bit for Ai1s(40GB),當然還需綁定一個公網IP,讓我們在本地通終端工具能夠訪問到ECS。
賬號準備
登錄修改HwHiAiUser 密碼
cat /etc/passwd # (推薦)如果創建的運行用戶是HwHiAiUser,其為CANN軟件包的默認運行用戶,在安裝時無需指定該運行用戶 passwd HwHiAiUser # (按需)修改sh --> bash vim /etc/passwd
資源下載
基本上昇騰相關的資料都可以通過訪問?ascend.huawei.com?獲取,因此我們要下載的驅動、工具也可以在昇騰官網上找到。
驅動
上文我們已經選用了昇騰310的環境,因此我們需要按照場景二來安裝環境。 驅動選用的是Atlas300 3010 X86 Ubuntu18.04驅動包。
工具
安裝x86平臺開發套件軟件包及x86平臺推理引擎軟件包。
軟件
mkdir downloads wget https://repo.huaweicloud.com/ascend/autoarchive/CANN/3.3.0.alpha001/Ascend-cann-toolkit_3.3.0.alpha001_linux-x86_64.run --no-check-certificate wget https://repo.huaweicloud.com/ascend/autoarchive/CANN/3.3.0.alpha001/Ascend-cann-nnrt_3.3.0.alpha001_linux-x86_64.run --no-check-certificate wget https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/turing/resourcecenter/Software/20.2.0/A300-3010%201.0.9/A300-3010-npu-driver_20.2.0_ubuntu18.04-x86_64.run --no-check-certificate wget https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/turing/resource/mindstudio/2.0.0-beta3/MindStudio_2.0.0-beta3_linux.tar.gz --no-check-certificate
修改 apt 源
一切為了加速,一切使用鏡像!
cd /etc/apt ll cp sources.list sources.list.bak vim sources.list # :%d
## 中科大源 ## 先備份源文件 deb https://mirrors.ustc.edu.cn/ubuntu/ bionic main restricted universe multiverse deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic main restricted universe multiverse deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-security main restricted universe multiverse deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-security main restricted universe multiverse deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse
apt-get update
安裝依賴
依賴的安裝也并非一帆風順,遇到一些錯誤中斷了安裝,需要重新安裝一下 python:?apt-get install python3.7。當前環境只支持特定版本的Python。
參考開發者文檔:?https://support.huaweicloud.com/instg-cli-cann330/atlasrun_03_0033.html
sudo apt-get install -y gcc g++ make cmake zlib1g zlib1g-dev libbz2-dev libsqlite3-dev libssl-dev libxslt1-dev libffi-dev unzip pciutils net-tools libncursesw5-dev
Errors were encountered while processing: python3-lib2to3 python3-distutils libglib2.0-dev-bin libglib2.0-dev:amd64 libharfbuzz-dev:amd64 libicu-le-hb-dev:amd64 libicu-dev libxml2-dev:amd64 libxslt1-dev:amd64 E: Sub-process /usr/bin/dpkg returned an error code (1)
# 解決 apt-get install python3.7 # 再重新執行安裝
配置 pip 源
# 安裝python 依賴 exit # 退回到HwHiAIUser cd ~/.pip vi pip.conf
[global] #以華為源為例,請根據實際情況進行替換。 index-url = https://mirrors.huaweicloud.com/repository/pypi/simple trusted-host = mirrors.huaweicloud.com timeout = 120
# 軟連接 su ln -s /usr/local/python3.7.5/bin/python3 /usr/local/python3.7.5/bin/python3.7.5 ln -s /usr/local/python3.7.5/bin/pip3 /usr/local/python3.7.5/bin/pip3.7.5 exit
# python 環境變量 vim .bashrc # 修改之后使之生效 source .bashrc
#用于設置python3.7.5庫文件路徑 export LD_LIBRARY_PATH=/usr/local/python3.7.5/lib:$LD_LIBRARY_PATH #如果用戶環境存在多個python3版本,則指定使用python3.7.5版本 export PATH=/usr/local/python3.7.5/bin:$PATH
# 安裝依賴 pip3.7.5 install attrs numpy decorator sympy==1.4 cffi==1.12.3 pyyaml pathlib2 psutil protobuf scipy requests xlrd==1.2.0 --user
安裝軟件
# 給 downloads 目錄添加權限 cd downloads # chmod +x A* su chown -R HwHiAiUser:HwHiAiUser ./ # 驅動用 root 裝 su ./A300-3010-npu-driver_20.2.0_ubuntu18.04-x86_64.run --full ## 替換原來的 再重啟 reboot # 其他用 HwHiAIUser 安裝 ./Ascend-cann-nnrt_3.3.0.alpha001_linux-x86_64.run --install # 環境變量配置 41‘50 # 安裝 toolkit ./Ascend-cann-toolkit_3.3.0.alpha001_linux-x86_64.run --install ## 注意保持環境變量配置 45‘47 ## 有三組: 離線推理、在線推理、模型轉換
[NNRT] [20210330-00:56:17] [INFO] Please make sure that: LD_LIBRARY_PATH includes : /home/HwHiAiUser/Ascend/nnrt/latest/acllib/lib64: PYTHONPATH includes : /home/HwHiAiUser/Ascend/nnrt/latest/pyACL/python/site-packages/acl: ASCEND_AICPU_PATH includes : /home/HwHiAiUser/Ascend/nnrt/latest: [NNRT] [20210330-00:56:17] [INFO] Ascend-cann-nnrt_3.3.0.alpha001_linux-x86_64.run install success,The install path is /home/HwHiAiUser/Ascend !
#用于設置python3.7.5庫文件路徑 export LD_LIBRARY_PATH=/home/HwHiAiUser/Ascend/nnrt/latest/acllib/lib64:/usr/local/python3.7.5/lib:$LD_LIBRARY_PATH #如果用戶環境存在多個python3版本,則指定使用python3.7.5版本 export PATH=/usr/local/python3.7.5/bin:$PATH export PYTHONPATH=/home/HwHiAiUser/Ascend/nnrt/latest/pyACL/python/site-packages/acl:${PYTHONPATHG} export ASCEND_AICPU_PATH=/home/HwHiAiUser/Ascend/nnrt/latest:${ASCEND_AICPU_PATH}
記得?source ~/.bashrc
離線、在線、模型轉換三種場景對應的環境變量:
Offline infer develop (cannot coexist with other scenes): PATH includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest/toolkit/bin: LD_LIBRARY_PATH includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest/acllib/lib64: PYTHONPATH includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest/pyACL/python/site-packages/acl: /home/HwHiAiUser/Ascend/ascend-toolkit/latest/toolkit/python/site-packages: ASCEND_AICPU_PATH includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest: ASCEND_OPP_PATH includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest/opp: TOOLCHAIN_HOME includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest/toolkit: Third party AI framework develop (cannot coexist with other scenes): PATH includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest/fwkacllib/ccec_compiler/bin: /home/HwHiAiUser/Ascend/ascend-toolkit/latest/fwkacllib/bin: /home/HwHiAiUser/Ascend/ascend-toolkit/latest/toolkit/bin: LD_LIBRARY_PATH includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest/fwkacllib/lib64: PYTHONPATH includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest/fwkacllib/python/site-packages: /home/HwHiAiUser/Ascend/ascend-toolkit/latest/toolkit/python/site-packages: ASCEND_OPP_PATH includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest/opp: ASCEND_AICPU_PATH includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest: TOOLCHAIN_HOME includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest/toolkit: Model transformation and operator develop (cannot coexist with other scenes): PATH includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest/atc/ccec_compiler/bin: /home/HwHiAiUser/Ascend/ascend-toolkit/latest/atc/bin: /home/HwHiAiUser/Ascend/ascend-toolkit/latest/toolkit/bin: LD_LIBRARY_PATH includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest/atc/lib64: PYTHONPATH includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest/atc/python/site-packages: /home/HwHiAiUser/Ascend/ascend-toolkit/latest/toolkit/python/site-packages: ASCEND_OPP_PATH includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest/opp: TOOLCHAIN_HOME includes : /home/HwHiAiUser/Ascend/ascend-toolkit/latest/toolkit:
# 解壓mindstudio tar zxvf MindStudio_2.0.0-beta3_linux.tar.gz # 運行 mindstudio/bin/mindstudio.sh
遇到錯誤按照提示操作,先用root權限安裝寫apt依賴;再用HwHiAiUser安裝python依賴。
sudo apt-get -y install xterm firefox xdg-utils openjdk-8-jdk fonts-droid-fallback fonts-wqy-zenhei fonts-wqy-microhei fonts-arphic-ukai fonts-arphic-uming gnome-keyring && /usr/local/python3.7.5/bin/pip3 install --user coverage gnureadline pylint matplotlib pandas absl-py
MindStudio端操作
最后
記得截圖保存留念并刪除資源!
晚安!
昇騰
版權聲明:本文內容由網絡用戶投稿,版權歸原作者所有,本站不擁有其著作權,亦不承擔相應法律責任。如果您發現本站中有涉嫌抄襲或描述失實的內容,請聯系我們jiasou666@gmail.com 處理,核實后本網站將在24小時內刪除侵權內容。