pymport

Use Python libraries from Node.js

Quickstart

Install pymport:

npm i pymport

Install numpy, pandas or whatever your favorite Python package is:

npx pympip3 install numpy

Start using from Node.js:

const { pymport, proxify } = require('pymport');
const np = proxify(pymport('numpy'));

const a = np.arange(15).reshape(3, 5);
const b = np.ones([2, 3], { dtype: np.int16 });

or

import { pymport, proxify } from 'pymport';
const np = proxify(pymport('numpy'));

const a = np.arange(15).reshape(3, 5);
const b = np.ones([2, 3], { dtype: np.int16 });

Then head to the wiki for the full documentation.

Or go for the quickstart by learning from the examples.

There is also a medium stort with a brief introduction.

Supported Versions

pymport is unit-tested on all combinations of:

except for the following combinations that are not supported:

• Node.js 16.x on Ubuntu 22.04: rebuilding from source against the system-installed Python is not possible due to Node.js containing a built-in OpenSSL 1.1 with exported symbols while the system-provided Python is built vs OpenSSL 3.0, upgrading Node.js or using the precompiled interpreter solves this issue

• Node.js 16.x and Python 3.12: node-gyp from Node.js 16.x does not work if Python 3.12 is installed

• Python 3.11.x on Windows: An upstream bug requires a specific workaround in this case #44

pymport vs PyScript/Pyodide

Although both projects provide similar functionality, they are completely unrelated with very different design goals:

• pymport goal is to allow using standard Python libraries without any modification in Node.js - it brings software such as numpy and pandas to Node.js
• Pyodide goal is to allow using Python code in a modified interpreter both in the browser and in Node.js - it allows scripting a web page in Python

You can check the wiki for a more detailed comparison.

Known Issues

The wiki has a list of some known and hard to fix issues: Known Issues

License

Copyright 2022-2024 Momtchil Momtchev momtchil@momtchev.com and contributors

Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[1.5.1] 2024-04-01

• Fix the build with recent Node.js versions after node_api_nogc_env in Node.js 18.20

[1.5.0] 2024-02-08

• Add Python 3.12 support and upgrade the built-in Python to 3.12.2
• Drop Node.js 14 support

[1.4.3] 2023-10-29

• Fix #104, do not mess up the loading of additional binary modules after pymport

[1.4.2] 2023-08-28

• Builtin Python updated to 3.12.2

[1.4.1] 2023-05-01

• Node.js 20 support
• Builtin Python updated to 3.10.11
• Fix [#78], a single Buffer argument is interpreted as kwargs

[1.4.0] 2023-01-22

• The new minimum supported versions of Node.js are now v12.22.0, v14.17.0, v16.0.0

New Features

• Allow limiting the depth of the recursion of toJS()
• Allow disabling of the generic Buffer protocol transformation to toJS()
• Add PyObject.prototype.map method compatible with Array.prototype.map
• Proxified functions are now also Proxy objects, resolves #58
• PyObject.item() now returns undefined instead of throwing an exception when an element cannot be retrieved by []
• Allow enabling of debug output via the environment (PYMPORT_DEBUG_sys vars)
• Improved bootstrap and shutdown allowing to load and unload the addon in worker_threads, partially resolves #69

Bug Fixes

• Fix #66, race condition can result in an abort when using pymport with worker_threads
• Fix #40, patch _sysconfigdata after installation
• Fix #70, random crash in a worker_thread
• Fix #69, pymport leaks TLS memory when used in a worker_thread

[1.3.1] 2023-01-07

• Fix #63, restore the executable bit of pympip3
• Fix #40, revert and lock setuptools for the builtin interpreter to 65.1.1 due to setuptools#3589
• Fix #60, proxified objects are not GCed
• Fix #48, do not consider falsy properties as being undefined
• Fix #50, proxified .toString() is not identical to PyObject.toString()

[1.3.0] 2022-12-22

New Features

• Builtin Python 3.10.9
• Multithreading safety
• callAsync method allowing asynchronous calling of Python functions
• On Linux and macOS, the builtin Python interpreter includes static versions of OpenSSL and libffi, this ensures better compatibility at the price of disabling the OpenSSL extensions supporting dedicated crypto hardware - this restriction does not apply if pymport is rebuilt from source
• Named ES6 exports
• Add PyObject.with implementing Python with
• Return the Python exception object and its constructor in PythonError.pythonValue and PythonError.pythonType

Bug Fixes

• Fix #27, a function as a single argument is considered a kwargs object
• Fix #33, PyObject.prototype.constr has wrong TypeScript type

[1.2.0] 2022-11-25

New Features

• Automatically proxified methods in pymport/proxified
• Proxified objects also proxify the input arguments of passed JS callbacks
• (BREAKING) Conversion of arguments in JS callbacks of Python code is now automatic only if it can be correctly deduced from the context, toJS() must be called in all other cases
• Implement Python type coercion
• Support Python sets

Bug Fixes

• Fix #25, type-ahead provokes a crash in Node.js 14.x REPL

[1.1.1] 2022-11-11

• Fix #14, toJS() converts Python bool to JS number
• Restore the JS function when converting a pymport.js_function back to JS
• Fix #17, PYTHONHOME is ignored
• In the version reporting, replace the "undefined" suffix with a "" when there is no version suffix

[1.1.0] 2022-11-08

New Features

• Converting JS functions to Python callables
• Expiring of function objects
• Converting of bytes and bytearray to Buffer and Buffer to bytearray
• Converting between TypedArray and array
• Convert Python iterators and generators to JavaScript Symbol.iterator
• Basic tracking of the memory held by Python objects referenced in JS by the V8 GC
• proxifyed object do not intercept and redirect calls to PyObject methods if the Python object has a method with the same name - ie. calling item() on a non-proxified PyObject invokes PyObject.item() but if the underlying Python object of a proxified object also has an item() function, calling item() on the proxified object will invoke the Python method
• profixy always return the same reference if called with a previously seen PyObject reference
• Support BigInt
• PyObject.slice accepts named arguments
• Return the Python traceback in pythonTrace and add a PythonError TypeScript type
• Do not allow PYTHONPATH to override PYTHONHOME when using the builtin
• Ignore the last argument of a Python function call if it is undefined - this allows to pass a last argument as a dictionary by calling fn(obj, undefined) instead of fn(obj) which will transform obj into named kw arguments
• Support PyObject.keys and PyObject.values on profixyed objects
• Support building against a prebuilt Python tree in a non-standard location
• (internals) New automatic reference counting

Bug Fixes

• Fix #6, proxified objects are wrongly caching values returned by getters
• Fix #8, Infinity is not recognized as a float by the automatic conversion
• Fix #11, npm install --build-from-source fails

[1.0.1] 2022-10-29

• Enable TypeScript strict mode
• Fix #1, always consume Python exceptions

[1.0.0] 2022-10-28

• First release