headless chrome with puppeteer

headless chrome with puppeteer

- 10월 10, 2017

headless software

그래픽 유저 인터페이스 없이 동작하는 소프트웨어

참고 : https://en.wikipedia.org/wiki/Headless_software

headless chrome

그래픽 유저 인터페이스 없이 동장하는 크롬 브라우저

환경 구성

mac
페러럴즈
centos 7 minimal 버전

설치

(wget 설치) sudo yum install wget
(net-tools 설치) sudo yum install net-tools
nodejs 설치

epel(extra package for enterprise linux) 저장소 추가

(체크) yum repolist
(추가) sudo yum install epel-release
(체크) yum repolist

nodejs (current 6.11.3-1)설치

(설치) sudo yum install nodejs

node upgrade

puppeteer는 node 6.4 버전 이후 지원이며, async/await를 사용하려면 7.6 이상 버전이 설치 되어야 한다.

(버전 확인) node -v
(npm 캐쉬 제거) sudo npm cache clean -f
(n 모듈 설치) sudo npm install -g n
(node upgrade) sudo n latest

n 모듈로 설치 시 /usr/bin/node 버전은 6, /usr/local/bin/node 8버전이 설치됨 (별로임)

puppeteer 설치

yarn 설치

(yarn repo 추가) sudo wget https://dl.yarnpkg.com/rpm/yarn.repo -O /etc/yum.repos.d/yarn.repo
(yarn 설치) sudo yum install yarn

puppeteer 설치

mkdir ~/test
cd test
yarn add puppeteer

테스트 코드 실행


const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://example.com');
  await page.screenshot({path: 'example.png'});

  await browser.close();
})();

관련 라이브러리 설치

sudo yum install pango.x86_64 libXcomposite.x86_64 libXcursor.x86_64 libXdamage.x86_64 libXext.x86_64 libXi.x86_64 libXtst.x86_64 cups-libs.x86_64 libXScrnSaver.x86_64 libXrandr.x86_64 GConf2.x86_64 alsa-lib.x86_64 atk.x86_64 gtk3.x86_64 -y
sudo yum install ipa-gothic-fonts xorg-x11-fonts-100dpi xorg-x11-fonts-75dpi xorg-x11-utils xorg-x11-fonts-cyrillic xorg-x11-fonts-Type1 xorg-x11-fonts-misc -y
font 설치

(참고 : https://www.google.com/get/noto/help/install/)

cd ~/Downloads # or wherever you downloaded a font pack.
unzip <font package>.zip
mkdir -p ~/.fonts
cp *otf *otc ~/.fonts
fc-cache -f -v # optional

에러

error while loading shared libraries: libpangocairo-1.0.so.0: cannot open shared object file: No such file or director

(pango 설치) sudo yum install pango

error while loading shared libraries: libXcomposite.so.1: cannot open shared object file: No such file or directory

(libXcomposite 설치) sudo yum install libXcomposite

error while loading shared libraries: libXcursor.so.1: cannot open shared object file: No such file or directory

(libXcursor 설치) sudo yum install libXcursor

error while loading shared libraries: libXi.so.6: cannot open shared object file: No such file or directory

(libXi 설치) sudo yum install libXi

error while loading shared libraries: libXtst.so.6: cannot open shared object file: No such file or directory

(libXtst 설치) sudo yum install libXtst

error while loading shared libraries: libcups.so.2: cannot open shared object file: No such file or directory

(libcups 설치) sudo yum install cups-libs

error while loading shared libraries: libXss.so.1: cannot open shared object file: No such file or directory

(libXss 설치) sudo yum install libXScrnSaver

error while loading shared libraries: libXrandr.so.2: cannot open shared object file: No such file or directory

(libXrandr 설치) sudo yum install libXrandr

error while loading shared libraries: libgconf-2.so.4: cannot open shared object file: No such file or directory

(libgconf 설치) sudo yum install GConf2

error while loading shared libraries: libatk-1.0.so.0: cannot open shared object file: No such file or directory

(libatk 설치) sudo yum install atk

error while loading shared libraries: libgtk-3.so.0: cannot open shared object file: No such file or directory

(libgtk 설치) sudo yum install gtk3

No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/master/docs/linux_suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox.

cd /test/node_modules/puppeteer/.local-chromium/linux-499413/chrome-linux
sudo chown root:root chrome_sandbox && sudo chmod 4755 chrome_sandbox && export CHROME_DEVEL_SANDBOX="$PWD/chrome_sandbox"
sudo chmod 644 locales/

댓글