MLECO-4065: Revising scripts to use Python3.9
Revising documentation and scripts to use Python3.9 explicitly
to remove the need for installing it system-wide that might
break a distro's desktop and pacakge manager utilities.
Change-Id: I683b55dd0243d0a726dc94eba2431005d4897c8c
Signed-off-by: Kshitij Sisodia <kshitij.sisodia@arm.com>
diff --git a/docs/sections/building.md b/docs/sections/building.md
index c7d69c8..7af3fdb 100644
--- a/docs/sections/building.md
+++ b/docs/sections/building.md
@@ -103,7 +103,9 @@
> **Note:** If you have an older version of Python installed (< 3.9) see the
> [Troubleshooting](./troubleshooting.md#how-to-update-python3-package-to-newer-version)
- > for instruction on how to update it.
+ > for instruction on how to install and use it.
+ > **Note:** This scenario might be true if you are using an Arm Virtual Hardware instance.
+ > See the troubleshooting link above on how to set up the environment in this case.
- The build system creates a Python virtual environment during the build process. Please make sure that Python virtual
environment module is installed by running:
@@ -112,12 +114,6 @@
python3 -m venv
```
- > **Note:** If you are using an Arm Virtual Hardware instance then Python virtual environment is not installed by default.
- > You will need to install it yourself by running the following command:
- > ```commandline
- > sudo apt install python3.8-venv
- > ```
-
- The build system uses external Python libraries during the building process. Please make sure that the latest pip and
libsndfile versions are installed.
@@ -362,7 +358,11 @@
Arm® *Ethos™-U65* NPU.
> **Note:** This script requires Python version 3.9 or higher. Please make sure all [build prerequisites](./building.md#build-prerequisites)
-> are satisfied.
+> are satisfied. If your environment points to system installed Python3 that is an older version than 3.9, choose the
+> required version explicitly after installing it:
+> ```sh
+> python3.9 ./set_up_default_resources.py
+> ```
>
> **Note:** This script also installs required version of CMake into the virtual environment, which can be used by activating it.
diff --git a/docs/sections/troubleshooting.md b/docs/sections/troubleshooting.md
index c8bb7b3..e6b921b 100644
--- a/docs/sections/troubleshooting.md
+++ b/docs/sections/troubleshooting.md
@@ -164,45 +164,36 @@
sudo apt-get install python3.9 python3.9-venv libpython3.9 libpython3.9-dev
```
-3. Update the `python3` alternatives (set as 1 your previous version displayed at step 1):
+3. Explicitly specify this Python when executing the set-up Python scripts. For example:
- ```commandline
- sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 1
- sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 2
- ```
-
-4. At the prompt, update the configuration by selecting Python3.9 as the chosen default alternative:
-
- ```commandline
- sudo update-alternatives --config python3
- ```
-
-5. Python3.9 is now set as default, you can check it by running:
-
- ```commandline
- python3 --version
- ```
-
- ```log
- Python 3.9.0
- ```
-
-> **Note:** After updating from Python3.6 to Python3.9 it may happen that the `gnome-terminal` or the relative
-> shortcuts don't work anymore.
-> If when opening it from XTerm with `gnome-terminal` the following error appear:
+ ```commandline
+ python3.9 ./set_up_default_resources.py
+ ```
+> **Note:**: We do not recommend updating the Python version system-wide as it might break
+> various desktop utilities for your OS distribution. However, if you are using a container
+> or any other sandboxed environment, you can choose to update it using the following steps.
>
-> ```log
-> Traceback (most recent call last):
-> File "/usr/bin/gnome-terminal", line 9, in <module>
-> from gi.repository import GLib, Gio
-> File "/usr/lib/python3/dist-packages/gi/__init__.py", line 42, in <module>
-> from . import _gi
-> ImportError: cannot import name '_gi' from partially initialized module 'gi' (most likely due to a circular import)
-> (/usr/lib/python3/dist-packages/gi/> __init__.py)
-> ```
+> * Update the `python3` alternatives (set as 1 your previous version displayed at step 1):
>
-> Modify the `gnome-terminal` script located in `/usr/bin/` and changing the environment (first line of the script)
-> from `#!/usr/bin/python3` to `#!/usr/bin/python3.6`.
+> ```commandline
+> sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 1
+> sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 2
+> ```
+>
+> * At the prompt, update the configuration by selecting Python3.9 as the chosen default alternative:
+>
+> ```commandline
+> sudo update-alternatives --config python3
+> ```
+> * Python3.9 is now set as default, you can check it by running:
+>
+> ```commandline
+> python3 --version
+> ```
+>
+> ```log
+> Python 3.9.0
+> ```
Next section of the documentation: [Appendix](appendix.md).
@@ -240,11 +231,11 @@
You can fix this error by installing Python virtual environment and removing the corrupted resources_downloaded folder.
```commandline
-sudo apt install python3.9-venv
rm -r resources_downloaded
```
-You can then try rebuilding again e.g.
+Follow the instructions to [update Python3 package to 3.9 version](./troubleshooting.md#how-to-update-python3-package-to-newer-version)
+before attempting a rebuild with:
```commandline
python3 ./build_default.py
diff --git a/scripts/cmake/source_gen_utils.cmake b/scripts/cmake/source_gen_utils.cmake
index 95b3ba4..6287cb6 100644
--- a/scripts/cmake/source_gen_utils.cmake
+++ b/scripts/cmake/source_gen_utils.cmake
@@ -284,14 +284,26 @@
return()
endif ()
+ # If environment is not found, find the required Python version
+ # and create it.
+ find_package(Python3 3.9
+ COMPONENTS Interpreter
+ REQUIRED)
+
+ if (NOT Python3_FOUND)
+ message(FATAL_ERROR "Required version of Python3 not found!")
+ else()
+ message(STATUS "Python3 (v${Python3_VERSION}) found: ${Python3_EXECUTABLE}")
+ endif()
+
message(STATUS "Configuring python environment at ${PYTHON}")
execute_process(
- COMMAND ${PY_EXEC} -m venv ${DEFAULT_VENV_DIR}
+ COMMAND ${Python3_EXECUTABLE} -m venv ${DEFAULT_VENV_DIR}
RESULT_VARIABLE return_code
)
if (NOT return_code STREQUAL "0")
- message(FATAL_ERROR "Failed to setup python3 environment. Return code: ${return_code}")
+ message(FATAL_ERROR "Failed to setup Python3 environment. Return code: ${return_code}")
endif ()
execute_process(
diff --git a/set_up_default_resources.py b/set_up_default_resources.py
index 740cfb8..dd702c7 100755
--- a/set_up_default_resources.py
+++ b/set_up_default_resources.py
@@ -23,6 +23,7 @@
import subprocess
import sys
import urllib.request
+import venv
from argparse import ArgumentParser
from argparse import ArgumentTypeError
from collections import namedtuple
@@ -382,18 +383,16 @@
metadata_dict = dict()
vela_version = "3.8.0"
- py3_major_version_minimum = 3 # Python >= 3.9 is required
- py3_minor_version_minimum = 9
+ py3_version_minimum = (3, 9)
# Is Python minimum requirement matched?
py3_version = sys.version_info
- if (
- py3_version.major < py3_major_version_minimum
- or py3_version.minor < py3_minor_version_minimum
- ):
+ if py3_version < py3_version_minimum:
raise Exception(
"ERROR: Python3.9+ is required, please see the documentation on how to update it."
)
+ else:
+ logging.info(f"Using Python version: {py3_version}")
setup_script_hash_verified = False
setup_script_hash = get_md5sum_for_file(Path(__file__).resolve())
@@ -433,15 +432,9 @@
env_activate = str(env_path / "bin" / "activate")
if not env_path.is_dir():
- os.chdir(download_dir)
- # Create the virtual environment.
- command = f"python3 -m venv {env_dirname}"
- call_command(command)
- commands = ["pip install --upgrade pip", "pip install --upgrade setuptools"]
- for c in commands:
- command = f"{env_python} -m {c}"
- call_command(command)
- os.chdir(current_file_dir)
+ # Create the virtual environment using current interpreter's venv
+ # (not necessarily the system's Python3)
+ venv.create(env_dir=env_path, with_pip=True, upgrade_deps=True)
# 1.3 Install additional requirements first, if a valid file has been provided
if additional_requirements_file and os.path.isfile(additional_requirements_file):