Dev

Appearance

0

node-gyp ​

在安装某些包，或者执行项目的 pnpm install 的时候，可能会碰到 node-gyp 相关的报错

所以 node-gyp 到底是个什么？

Error stack example
│ node-pre-gyp info it worked if it ends with ok
│ node-pre-gyp info using node-pre-gyp@1.0.11
│ node-pre-gyp info using node@16.20.2 | darwin | arm64
│ node-pre-gyp http GET https://github.com/Automattic/node-canvas/releases/download/v2.11.2/canvas-v2.11.2-node-v93-darwin-unknown-arm64.tar.gz
│ node-pre-gyp ERR! install response status 404 Not Found on https://github.com/Automattic/node-canvas/releases/download/v2.11.2/canvas-v2.11.2-node-v93-darwin-unknown-arm
│ node-pre-gyp WARN Pre-built binaries not installable for canvas@2.11.2 and node@16.20.2 (node-v93 ABI, unknown) (falling back to source compile with node-gyp) 
│ node-pre-gyp WARN Hit error response status 404 Not Found on https://github.com/Automattic/node-canvas/releases/download/v2.11.2/canvas-v2.11.2-node-v93-darwin-unknown-a
│ gyp info it worked if it ends with ok
│ gyp info using node-gyp@9.3.1
│ gyp info using node@16.20.2 | darwin | arm64
│ gyp info ok 
│ gyp info it worked if it ends with ok
│ gyp info using node-gyp@9.3.1
│ gyp info using node@16.20.2 | darwin | arm64
│ gyp info find Python using Python version 3.9.6 found at "/Library/Developer/CommandLineTools/usr/bin/python3"
│ gyp info spawn /Library/Developer/CommandLineTools/usr/bin/python3
│ gyp info spawn args [
│ gyp info spawn args   '/Users/peterroe/.cache/node/corepack/pnpm/7.33.6/dist/node_modules/node-gyp/gyp/gyp_main.py',
│ gyp info spawn args   'binding.gyp',
│ gyp info spawn args   '-f',
│ gyp info spawn args   'make',
│ gyp info spawn args   '-I',
│ gyp info spawn args   '/path/to/your/project/node_modules/.pnpm/canvas@2.11.2/node_modules/canvas/build/config.gypi',
│ gyp info spawn args   '-I',
│ gyp info spawn args   '/Users/peterroe/.cache/node/corepack/pnpm/7.33.6/dist/node_modules/node-gyp/addon.gypi',
│ gyp info spawn args   '-I',
│ gyp info spawn args   '/Users/peterroe/Library/Caches/node-gyp/16.20.2/include/node/common.gypi',
│ gyp info spawn args   '-Dlibrary=shared_library',
│ gyp info spawn args   '-Dvisibility=default',
│ gyp info spawn args   '-Dnode_root_dir=/Users/peterroe/Library/Caches/node-gyp/16.20.2',
│ gyp info spawn args   '-Dnode_gyp_dir=/Users/peterroe/.cache/node/corepack/pnpm/7.33.6/dist/node_modules/node-gyp',
│ gyp info spawn args   '-Dnode_lib_file=/Users/peterroe/Library/Caches/node-gyp/16.20.2/<(target_arch)/node.lib',
│ gyp info spawn args   '-Dmodule_root_dir=/path/to/your/project/node_modules/.pnpm/canvas@2.11.2/node_modules/canvas',
│ gyp info spawn args   '-Dnode_engine=v8',
│ gyp info spawn args   '--depth=.',
│ gyp info spawn args   '--no-parallel',
│ gyp info spawn args   '--generator-output',
│ gyp info spawn args   'build',
│ gyp info spawn args   '-Goutput_dir=.'
│ gyp info spawn args ]
│ /bin/sh: pkg-config: command not found
│ gyp: Call to 'pkg-config pixman-1 --libs' returned exit status 127 while in binding.gyp. while trying to load binding.gyp
│ gyp ERR! configure error 
│ gyp ERR! stack Error: `gyp` failed with exit code: 1
│ gyp ERR! stack     at ChildProcess.onCpExit (/Users/peterroe/.cache/node/corepack/pnpm/7.33.6/dist/node_modules/node-gyp/lib/configure.js:325:16)
│ gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
│ gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:293:12)
│ gyp ERR! System Darwin 22.6.0
│ gyp ERR! command "/Users/peterroe/.nvm/versions/node/v16.20.2/bin/node" "/Users/peterroe/.cache/node/corepack/pnpm/7.33.6/dist/node_modules/node-gyp/bin/node-gyp.js" "co
│ gyp ERR! cwd /path/to/your/project/node_modules/.pnpm/canvas@2.11.2/node_modules/canvas
│ gyp ERR! node -v v16.20.2
│ gyp ERR! node-gyp -v v9.3.1
│ gyp ERR! not ok 
│ node-pre-gyp ERR! build error 
│ node-pre-gyp ERR! stack Error: Failed to execute '/Users/peterroe/.nvm/versions/node/v16.20.2/bin/node /Users/peterroe/.cache/node/corepack/pnpm/7.33.6/dist/node_modules
│ node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/path/to/your/project/node_modules/.pnpm/@mapbox+node-pre-gyp@1.0.11/node_modules/@mapbo
│ node-pre-gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
│ node-pre-gyp ERR! stack     at maybeClose (node:internal/child_process:1100:16)
│ node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)
│ node-pre-gyp ERR! System Darwin 22.6.0
│ node-pre-gyp ERR! command "/Users/peterroe/.nvm/versions/node/v16.20.2/bin/node" "/path/to/your/project/node_modules/.pnpm/@mapbox+node-pre-gyp@1
│ node-pre-gyp ERR! cwd /path/to/your/project/node_modules/.pnpm/canvas@2.11.2/node_modules/canvas
│ node-pre-gyp ERR! node -v v16.20.2
│ node-pre-gyp ERR! node-pre-gyp -v v1.0.11
│ node-pre-gyp ERR! not ok 
│ Failed to execute '/Users/peterroe/.nvm/versions/node/v16.20.2/bin/node /Users/peterroe/.cache/node/corepack/pnpm/7.33.6/dist/node_modules/node-gyp/bin/node-gyp.js confi
└─ Failed in 1.7s at /path/to/your/project/node_modules/.pnpm/canvas@2.11.2/node_modules/canvas
 ELIFECYCLE  Command failed with exit code 1.

Node addons ​

Node 插件是用 C/C++ 语言编写的动态链接库，使用 require() 加载到 Node 环境

在早期的 Node 插件开发中，严重依赖 V8 引擎的 API，对 Node 进行升级之后，插件很可能为因为 API 的变动导致无法使用

所以为了解决这个文件，在 Node 8 之后，发布了 N-API 接口，它是对 V8 引擎的 API 封装，以 C 风格 API 提供对外接口。使用 N-API 编写的 Node 插件能够一次编写、一次编译，跨多个Node 版本运行

而编译的工具，就是 node-gyp。接下来我们展示如何将 C++ 编译到 Node addon

C++ to node ​

node-gyp 是一个可以构建 NodeJs 原生插件的工具。它的设计初衷是为了帮助构建包含 C++ 代码的 NodeJs 模块

安装

$ npm i -g node-gyp

生成配置

新建 binding.gpy 文件（固定名称），指定产物名称 target_name 和，包含的文件，并且写入适当的内容

{
  "targets": [
    {
      "target_name": "binding",
      "sources": [ "addon.cc" ]
    }
  ]
}
#include <node.h>
namespace demo {

using v8::FunctionCallbackInfo;
using v8::Isolate;
using v8::Local;
using v8::Object;
using v8::String;
using v8::Value;

void Method(const FunctionCallbackInfo<Value>& args) {
  Isolate* isolate = args.GetIsolate();
  args.GetReturnValue().Set(String::NewFromUtf8(
      isolate, "world").ToLocalChecked());
}

void Initialize(Local<Object> exports) {
  NODE_SET_METHOD(exports, "hello", Method);
}

NODE_MODULE(NODE_GYP_MODULE_NAME, Initialize)

}

TIP

关于 C++ 文件扩展名，在 Unix 世界之外，主要使用 .cpp，Unix 更多地使用 .cc

执行 node-gyp configure，会在 build 文件夹下生成 Makefile 等文件

$ node-gyp configure
gyp info it worked if it ends with ok
gyp info using node-gyp@10.0.1
gyp info using node@16.20.2 | darwin | arm64
gyp info find Python using Python version 3.9.6 found at "/Library/Developer/CommandLineTools/usr/bin/python3"
...
gyp info ok

构建

然后执行 node-gyp build会生成一个 build/Release 文件夹，其中包含了我们需要的文件 binding.node

$ node-gyp build    
gyp info it worked if it ends with ok
gyp info using node-gyp@10.0.1
gyp info using node@16.20.2 | darwin | arm64
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Nothing to be done for `all'.
gyp info ok

WARNING

我们上面的 addon.cc 是直接使用的 V8 的 API，所以例如通过 node16 环境构建出来的产物，很可能不能在 node18 使用

Use addons ​

有了生成了 binding.node 文件，我们就可以在 NodeJs 中使用 require 直接导入了

const addon = require('./build/Release/binding.node')
console.log(
    addon.hello() //=> world
)

遗憾的是，目前只原生支持使用 CJS 导入 .node 模块（issue here）。如果想使用 ESM 方式导入，有两种方法：

  1. 指定 experimental-specifier-resolution 参数
import addon from './build/Release/binding.node'

console.log(
    addon.hello() //=> world
)
$ node --experimental-specifier-resolution=node index.js
  1. 使用 module 模块的 createRequire 模拟 require
import { createRequire } from 'module';
const require = createRequire(import.meta.url);
const addon = require('./build/Release/binding.node')

console.log(
    addon.hello() //=> world
)

但是显然效果都不尽人意，目前最好的方法就是，创建 commonjs 风格同时兼容 CJS 和 ESM 的文件（index.js），例如

const addon = require('./build/Release/binding.node')

module.exports.addon = addon
import addon from './index.js'

// Good support
const addon = require('./index.js')

// Good support

Build with napi ​

让我们使用 napi 来构建一个跨 Node 版本的 addon 吧，我们会用到 nodejs/node-addon-api 这个库，它提供了更易用的 C++ API，使得在 C++ 中编写 NodeJs 插件更加方便

{
  "targets": [
    {
      "target_name": "binding",
      "cflags!": [ "-fno-exceptions" ],
      "cflags_cc!": [ "-fno-exceptions" ],
      "sources": [ "addon.cc" ],
      "include_dirs": [
        "<!@(node -p \"require('node-addon-api').include\")"
      ],
      'defines': [ 'NAPI_DISABLE_CPP_EXCEPTIONS' ],
    }
  ]
}
#include <napi.h>

Napi::String Method(const Napi::CallbackInfo& info) {
  Napi::Env env = info.Env();
  return Napi::String::New(env, "world");
}

Napi::Object Init(Napi::Env env, Napi::Object exports) {
  exports.Set(Napi::String::New(env, "hello"),
              Napi::Function::New(env, Method));
  return exports;
}

NODE_API_MODULE(hello, Init)
{
  "name": "hello_world",
  "version": "0.0.0",
  "description": "Node.js Addons with napi",
  "dependencies": {
    "node-addon-api": "^7.0.0"
  },
}
const addon = require('./build/Release/binding.node')

console.log(
    addon.hello()
)
$ node-gyp configure && node-gyp build 
...
$ node hello.js
world

node-pre-gyp ​

你可能注意到，在上面 Error stack example 的错误是 node-pre-gyp 发出的，而不是 node-gyp，那它又是什么呢？

node-pre-gyp 是一个 NodeJs 模块编译和发布工具，它的主要目的是简化和优化 NodeJs 模块的构建和部署过程

如前面所见，为了得到一个 .node 模块，我们需要经历执行 node-gyp configure/build 两个命令，实在过于麻烦

所以通常来说，我们会使用 node-pre-gyp 进行原生模块的构建，例如 canvas 这个 NPM 包。

Automattic/node-canvas/package.json
Lines 25 to 36 #a5b379b
25
26
27
28
29
30
31
32
33
34
35
36

TIP

推荐使用 @mapbox/node-pre-gyp 而不是 node-pre-gyp

最开始的那个错误其实就是在 MacOS 上执行 pnpm i canvas 抛出的，这个库经常有问题，解决方法就是：brew install pixman cairo pango

构建一直很难，并且还十分重要。但是 node-gpy 让开发者和使用者都十分困扰。NodeJs 作者 Ryan Dahl 在其 2018 「Design mistakes in Node」演讲中也指出了这一点，详情查看原 pdf 第十页 或者完整视频

Released under the MIT License.

Copyright © 2021-PRESENT Peter Roe