Installing Python Libraries during Cluster Bootstrapping

Clusters can be configured to install Python packages during bootstrapping. These packages will be available in the Python context when running the cluster.

Packages can be added to clusters as a list of strings. Each string must have pip requirements.txt file formatting. The accepted syntax can be found here. For example, the following list of packages could be added to the cluster metadata:

[
  "pandas == 2.3.1",
  "matplotlib",
  "urllib3 @ https://github.com/urllib3/urllib3/archive/refs/tags/1.26.8.zip"
]

The list of packages can be added to a cluster when it is created or altered.

Was this article helpful?
0 out of 0 found this helpful