Caffe2 - Source 安装及问题解决

Author： AIHGF
发布时间：June 2, 2018
8277views
No comments
6859 words
Categories：深度平台

Caffe2 已经被合并到 Pytorch 中.
坐等 Facebook 出新版的 Caffe2+Pytorch.
原来在 Ubuntu14.04 上已经应用，现在系统变更为 Ubuntu16.04. 再次编译安装.
Caffe2 - 源码编译安装

1. 源码安装

系统环境：

Ubuntu14.04
Ubuntu16.04
CUDA 9.0
cuDNN 7.1.4

1.1 依赖项

sudo apt-get update
sudo apt-get install -y --no-install-recommends \
      build-essential \
      cmake \
      git \
      libgoogle-glog-dev \
      libprotobuf-dev \
      protobuf-compiler \
      python-dev \
      python-pip
sudo pip install numpy protobuf

1.2 GPU 支持

需要安装 NVIDIA CUDA 8 和 cuDNN v5.1 或其它版本，这里是采用了 CUDA9.0 和cuDNN7.1.4，官方推荐安装方式如下：

1.2.1 Step1 - 安装显卡驱动

Ubuntu14.04

sudo apt-get update
sudo apt-get install wget -y --no-install-recommends
wget "http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1404/x86_64/cuda-repo-ubuntu1404_8.0.61-1_amd64.deb"
sudo dpkg -i cuda-repo-ubuntu1404_8.0.61-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda

Ubuntu16.04

sudo apt-get update
sudo apt-get install wget -y --no-install-recommends
wget "http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_8.0.61-1_amd64.deb"
sudo dpkg -i cuda-repo-ubuntu1604_8.0.61-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda

貌似Ubuntu16.04 安装 NVIDIA 驱动更加方便，在 Software&Updates - Additional Drivers 里.
安装驱动以后，再安装 cuda 时，选择不安装 NVIDIA 驱动即可.

1.2.2 Step2 - 安装cuDNN

cuDNN v5.1

CUDNN_URL="http://developer.download.nvidia.com/compute/redist/cudnn/v5.1/cudnn-8.0-linux-x64-v5.1.tgz"
wget ${CUDNN_URL}
sudo tar -xzf cudnn-8.0-linux-x64-v5.1.tgz -C /usr/local
rm cudnn-8.0-linux-x64-v5.1.tgz
sudo ldconfig

cuDNN v7.1
NVIDIA’s cuDNN download官网注册并下载，采用类似的方式安装.

1.2.3 Step3 - 其它依赖项

# libgflags-dev
# for Ubuntu 14.04
sudo apt-get install -y --no-install-recommends libgflags2
# for Ubuntu 16.04
sudo apt-get install -y --no-install-recommends libgflags-dev

# for both Ubuntu 14.04 and 16.04
sudo apt-get install -y --no-install-recommends \
      libgtest-dev \
      libiomp-dev \
      libleveldb-dev \
      liblmdb-dev \
      libopencv-dev \
      libopenmpi-dev \
      libsnappy-dev \
      openmpi-bin \
      openmpi-doc \
      python-pydot
sudo pip install \
      flask \
      graphviz \
      hypothesis \
      jupyter \
      matplotlib \
      pydot python-nvd3 \
      pyyaml \
      requests \
      scikit-image \
      scipy \
      setuptools \
      tornado

1.3 编译

git clone --recursive https://github.com/caffe2/caffe2.git
cd caffe2
make
cd build
sudo make install
python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"

运行命令测试GPU-Caffe2是否编译成功：

python -m caffe2.python.operator_test.relu_op_test

1.4 环境变量设置

sudo vim ~/.bashrc
# 添加如下对应内容
source

echo $PYTHONPATH
# export PYTHONPATH=/usr/local:$PYTHONPATH
# export PYTHONPATH=$PYTHONPATH:/home/ubuntu/caffe2/build
echo $LD_LIBRARY_PATH
# export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

2. 错误解决

2.1 - ImportError: No module named past.builtins

错误问题：
安装完成后，运行：

python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"

出现 Failure.

Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/caffe2/python/operator_test/relu_op_test.py", line 21, in <module>
    from caffe2.python import core
  File "/usr/local/caffe2/python/core.py", line 24, in <module>
    from past.builtins import basestring
ImportError: No module named past.builtins

解决方案：
安装 future package：

sudo pip install future

2.2. gmock/gmock.h

ubuntu下 google gmock 安装.
下载 googletest，并解压

./travis.sh
cmake ./CMakeLists.txt
make
sudo make install

2.3. /usr/bin/ld: cannot find -lxxx

错误问题：

/usr/bin/ld: cannot find -lcudart
/usr/bin/ld: cannot find -lnvrtc
/usr/bin/ld: cannot find -lcudart_static
/usr/bin/ld: cannot find -lcurand
/usr/bin/ld: cannot find -lcublas
/usr/bin/ld: cannot find -lcublas_device
/usr/bin/ld: cannot find -lcudnn
/usr/bin/ld: cannot find -lcudart_static

解决方法：

sudo ln -s /usr/local/cuda/lib64/libcudart.so /usr/lib/libcudart.so
sudo ln -s /usr/local/cuda/lib64/libnvrtc.so /usr/lib/libnvrtc.so
sudo ln -s /usr/local/cuda/lib64/libcudart_static.a /usr/lib/libcudart_static.a
sudo ln -s /usr/local/cuda/lib64/libcurand.so /usr/lib/libcurand.so
sudo ln -s /usr/local/cuda/lib64/libcublas.so /usr/lib/libcublas.so
sudo ln -s /usr/local/cuda/lib64/libcublas.so.9.0 /usr/lib/libcublas.so.9.0
sudo ln -s /usr/local/cuda/lib64/libcublas.so.9.0.176 /usr/lib/libcublas.so.9.0.176
sudo ln -s /usr/local/cuda/lib64/libcublas_device.a /usr/lib/libcublas_device.a
sudo ln -s /usr/local/cuda/lib64/libcudnn.so /usr/lib/libcudnn.so
sudo ln -s /usr/local/cuda/lib64/libcudnn.so.7 /usr/lib/libcudnn.so.7
sudo ln -s /usr/local/cuda/lib64/libcudnn.so.7.1.4 /usr/lib/libcudnn.so.7.1.4

sudo ldconfig

2.4 CPU feature avx2 is present on your machine

警告问题：

E1205 15:13:43.105324 10699 init_intrinsics_check.cc:54] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E1205 15:13:43.105361 10699 init_intrinsics_check.cc:54] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E1205 15:13:43.105366 10699 init_intrinsics_check.cc:54] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
WARNING cnn.py:  40: [====DEPRECATE WARNING====]: you are creating an object from CNNModelHelper class which will be deprecated soon. Please use ModelHelper object with brew module. For more information, please refer to caffe2.ai and python/brew.py, python/brew_test.py for more information.

解决方案：

# 编辑 CAFFE2_ROOT/CMakeLists.txt
# 第 36 行：option(USE_NATIVE_ARCH "Use -march=native" OFF)  # default: OFF
# 修改为 ON：
option(USE_NATIVE_ARCH "Use -march=native" ON)  # default: OFF
# 重新编译安装

2.5 No handlers could be found for logger "caffe2.python.net_drawer"

解决方案：

sudo pip install pydot

Last modification：December 6th, 2018 at 03:51 pm

Caffe2 - Source 安装及问题解决

AIHGF • 2018 年 06 月 02 日

Caffe2 已经被合并到 Pytorch 中.
坐等 Facebook 出新版的 Caffe2+Pytorch.
原来在 Ubuntu14.04 上已经应用，现在系统变更为 Ubuntu16.04. 再次编译安装.
Caffe2 - 源码编译安装

1. 源码安装

系统环境：

Ubuntu14.04
Ubuntu16.04
CUDA 9.0
cuDNN 7.1.4

1.1 依赖项

sudo apt-get update
sudo apt-get install -y --no-install-recommends \
      build-essential \
      cmake \
      git \
      libgoogle-glog-dev \
      libprotobuf-dev \
      protobuf-compiler \
      python-dev \
      python-pip
sudo pip install numpy protobuf

1.2 GPU 支持

需要安装 NVIDIA CUDA 8 和 cuDNN v5.1 或其它版本，这里是采用了 CUDA9.0 和cuDNN7.1.4，官方推荐安装方式如下：

1.2.1 Step1 - 安装显卡驱动

Ubuntu14.04

sudo apt-get update
sudo apt-get install wget -y --no-install-recommends
wget "http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1404/x86_64/cuda-repo-ubuntu1404_8.0.61-1_amd64.deb"
sudo dpkg -i cuda-repo-ubuntu1404_8.0.61-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda

Ubuntu16.04

sudo apt-get update
sudo apt-get install wget -y --no-install-recommends
wget "http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_8.0.61-1_amd64.deb"
sudo dpkg -i cuda-repo-ubuntu1604_8.0.61-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda

貌似Ubuntu16.04 安装 NVIDIA 驱动更加方便，在 Software&Updates - Additional Drivers 里.
安装驱动以后，再安装 cuda 时，选择不安装 NVIDIA 驱动即可.

1.2.2 Step2 - 安装cuDNN

cuDNN v5.1

CUDNN_URL="http://developer.download.nvidia.com/compute/redist/cudnn/v5.1/cudnn-8.0-linux-x64-v5.1.tgz"
wget ${CUDNN_URL}
sudo tar -xzf cudnn-8.0-linux-x64-v5.1.tgz -C /usr/local
rm cudnn-8.0-linux-x64-v5.1.tgz
sudo ldconfig

cuDNN v7.1
NVIDIA’s cuDNN download官网注册并下载，采用类似的方式安装.

1.2.3 Step3 - 其它依赖项

# libgflags-dev
# for Ubuntu 14.04
sudo apt-get install -y --no-install-recommends libgflags2
# for Ubuntu 16.04
sudo apt-get install -y --no-install-recommends libgflags-dev

# for both Ubuntu 14.04 and 16.04
sudo apt-get install -y --no-install-recommends \
      libgtest-dev \
      libiomp-dev \
      libleveldb-dev \
      liblmdb-dev \
      libopencv-dev \
      libopenmpi-dev \
      libsnappy-dev \
      openmpi-bin \
      openmpi-doc \
      python-pydot
sudo pip install \
      flask \
      graphviz \
      hypothesis \
      jupyter \
      matplotlib \
      pydot python-nvd3 \
      pyyaml \
      requests \
      scikit-image \
      scipy \
      setuptools \
      tornado

1.3 编译

git clone --recursive https://github.com/caffe2/caffe2.git
cd caffe2
make
cd build
sudo make install
python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"

运行命令测试GPU-Caffe2是否编译成功：

python -m caffe2.python.operator_test.relu_op_test

1.4 环境变量设置

sudo vim ~/.bashrc
# 添加如下对应内容
source

echo $PYTHONPATH
# export PYTHONPATH=/usr/local:$PYTHONPATH
# export PYTHONPATH=$PYTHONPATH:/home/ubuntu/caffe2/build
echo $LD_LIBRARY_PATH
# export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

2. 错误解决

2.1 - ImportError: No module named past.builtins

错误问题：
安装完成后，运行：

python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"

出现 Failure.

Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/caffe2/python/operator_test/relu_op_test.py", line 21, in <module>
    from caffe2.python import core
  File "/usr/local/caffe2/python/core.py", line 24, in <module>
    from past.builtins import basestring
ImportError: No module named past.builtins

解决方案：
安装 future package：

sudo pip install future

2.2. gmock/gmock.h

ubuntu下 google gmock 安装.
下载 googletest，并解压

./travis.sh
cmake ./CMakeLists.txt
make
sudo make install

2.3. /usr/bin/ld: cannot find -lxxx

错误问题：

/usr/bin/ld: cannot find -lcudart
/usr/bin/ld: cannot find -lnvrtc
/usr/bin/ld: cannot find -lcudart_static
/usr/bin/ld: cannot find -lcurand
/usr/bin/ld: cannot find -lcublas
/usr/bin/ld: cannot find -lcublas_device
/usr/bin/ld: cannot find -lcudnn
/usr/bin/ld: cannot find -lcudart_static

解决方法：

sudo ln -s /usr/local/cuda/lib64/libcudart.so /usr/lib/libcudart.so
sudo ln -s /usr/local/cuda/lib64/libnvrtc.so /usr/lib/libnvrtc.so
sudo ln -s /usr/local/cuda/lib64/libcudart_static.a /usr/lib/libcudart_static.a
sudo ln -s /usr/local/cuda/lib64/libcurand.so /usr/lib/libcurand.so
sudo ln -s /usr/local/cuda/lib64/libcublas.so /usr/lib/libcublas.so
sudo ln -s /usr/local/cuda/lib64/libcublas.so.9.0 /usr/lib/libcublas.so.9.0
sudo ln -s /usr/local/cuda/lib64/libcublas.so.9.0.176 /usr/lib/libcublas.so.9.0.176
sudo ln -s /usr/local/cuda/lib64/libcublas_device.a /usr/lib/libcublas_device.a
sudo ln -s /usr/local/cuda/lib64/libcudnn.so /usr/lib/libcudnn.so
sudo ln -s /usr/local/cuda/lib64/libcudnn.so.7 /usr/lib/libcudnn.so.7
sudo ln -s /usr/local/cuda/lib64/libcudnn.so.7.1.4 /usr/lib/libcudnn.so.7.1.4

sudo ldconfig

2.4 CPU feature avx2 is present on your machine

警告问题：

E1205 15:13:43.105324 10699 init_intrinsics_check.cc:54] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E1205 15:13:43.105361 10699 init_intrinsics_check.cc:54] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E1205 15:13:43.105366 10699 init_intrinsics_check.cc:54] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
WARNING cnn.py:  40: [====DEPRECATE WARNING====]: you are creating an object from CNNModelHelper class which will be deprecated soon. Please use ModelHelper object with brew module. For more information, please refer to caffe2.ai and python/brew.py, python/brew_test.py for more information.

解决方案：

# 编辑 CAFFE2_ROOT/CMakeLists.txt
# 第 36 行：option(USE_NATIVE_ARCH "Use -march=native" OFF)  # default: OFF
# 修改为 ON：
option(USE_NATIVE_ARCH "Use -march=native" ON)  # default: OFF
# 重新编译安装

2.5 No handlers could be found for logger "caffe2.python.net_drawer"

解决方案：

sudo pip install pydot

Caffe2 - Source 安装及问题解决

1. 源码安装

1.1 依赖项

1.2 GPU 支持

1.2.1 Step1 - 安装显卡驱动

1.2.2 Step2 - 安装cuDNN

1.2.3 Step3 - 其它依赖项

1.3 编译

1.4 环境变量设置

2. 错误解决

2.1 - ImportError: No module named past.builtins

2.2. gmock/gmock.h

2.3. /usr/bin/ld: cannot find -lxxx

2.4 CPU feature avx2 is present on your machine

2.5 No handlers could be found for logger "caffe2.python.net_drawer"

※相关文章推荐※

※最新文章推荐※

Leave a Comment Cancel reply

Caffe2 - Source 安装及问题解决

1. 源码安装

1.1 依赖项

1.2 GPU 支持

1.2.1 Step1 - 安装显卡驱动

1.2.2 Step2 - 安装cuDNN

1.2.3 Step3 - 其它依赖项

1.3 编译

1.4 环境变量设置

2. 错误解决

2.1 - ImportError: No module named past.builtins

2.2. gmock/gmock.h

2.3. /usr/bin/ld: cannot find -lxxx

2.4 CPU feature avx2 is present on your machine

2.5 No handlers could be found for logger "caffe2.python.net_drawer"