The current system python in the docker baseimage is py3.9, while the
targetpython and hostpython built by p4a are py3.8, and this was causing
linker issues in the pyqt5 build.
It is also cleaner IMHO to have p4a handle what is needed for a recipe
instead of assuming it is available in the system. (and I think this is
how other existing recipes work)
fixes https://github.com/spesmilo/electrum/issues/8016
This allows more freedom than using releases from PyPI.
(atm there is no released version that fixes https://github.com/pyinstaller/pyinstaller/pull/6701 )
Also, we now build the pyinstaller bootloader, just like in the windows build:
one fewer binary blob to trust.
see https://github.com/spesmilo/electrum/pull/7721#issuecomment-1072669548
-----
pyinstaller 4.2 failed (during its runtime) to create exes (worked with cpython 3.9.10, but not with cpython 3.9.11):
```
80572 INFO: Processing module hooks...
80573 INFO: Loading module hook 'hook-certifi.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
80618 INFO: Loading module hook 'hook-cryptography.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
82879 INFO: Loading module hook 'hook-dns.rdata.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
84147 INFO: Loading module hook 'hook-mnemonic.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
84207 INFO: Loading module hook 'hook-pycparser.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
84212 INFO: Loading module hook 'hook-usb1.py' from 'C:\\python3\\lib\\site-packages\\usb1\\__pyinstaller'...
84215 INFO: --- libusb1 pyinstaller hook ---
84226 INFO: Added libusb binaries: [('C:\\python3\\lib\\site-packages\\usb1\\libusb-1.0.dll', 'usb1')]
84228 INFO: Loading module hook 'hook-difflib.py' from 'C:\\python3\\lib\\site-packages\\PyInstaller\\hooks'...
84237 INFO: Excluding import of doctest from module difflib
84237 INFO: Loading module hook 'hook-distutils.py' from 'C:\\python3\\lib\\site-packages\\PyInstaller\\hooks'...
Unable to find "C:\python3\Include\pyconfig.h" when adding binary and data files.This would mean your Python installation doesn't
come with proper library files. This usually happens by missing development
package, or unsuitable build parameters of Python installation.
* On Debian/Ubuntu, you would need to install Python development packages
* apt-get install python3-dev
* apt-get install python-dev
* If you're building Python by yourself, please rebuild your Python with
`--enable-shared` (or, `--enable-framework` on Darwin)
🗯 ERROR: build-electrum-git failed
```
Looks like this might be fixed in pyinstaller 4.3 (we are using 4.2):
https://github.com/pyinstaller/pyinstaller/pull/5218
-----
While trying to update pyinstaller to have that fix, several issues found with versions 4.3-4.10,
now reported upstream and already fixed:
https://github.com/pyinstaller/pyinstaller/issues/6338https://github.com/pyinstaller/pyinstaller/issues/6339https://github.com/pyinstaller/pyinstaller/issues/6686
So the pyinstaller commit pinned here is from the stable "v4" branch, some commits after the 4.10 tag,
which has fixes for the above issues.
aiorpcx 0.20 changed the behaviour/API of TaskGroups.
When used as a context manager, TaskGroups no longer propagate
exceptions raised by their tasks. Instead, the calling code has
to explicitly check the results of tasks and decide whether to re-raise
any exceptions.
This is a significant change, and so this commit introduces "OldTaskGroup",
which should behave as the TaskGroup class of old aiorpcx. All existing
usages of TaskGroup are replaced with OldTaskGroup.
closes https://github.com/spesmilo/electrum/issues/7446
maybe fixes https://github.com/spesmilo/electrum/issues/7640
Looks like by default pip is ignoring the locally available setuptools and wheel,
and downloading the latest ones from the internet at build time...
https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/?highlight=no-build-isolation#disabling-build-isolationhttps://stackoverflow.com/a/62889268
> When making build requirements available, pip does so in an isolated environment. That is, pip does not install those requirements into the user’s site-packages, but rather installs them in a temporary directory which it adds to the user’s sys.path for the duration of the build. This ensures that build requirements are handled independently of the user’s runtime environment. For example, a project that needs a recent version of setuptools to build can still be installed, even if the user has an older version installed (and without silently replacing that version).
>
> In certain cases, projects (or redistributors) may have workflows that explicitly manage the build environment. For such workflows, build isolation can be problematic. If this is the case, pip provides a --no-build-isolation flag to disable build isolation. Users supplying this flag are responsible for ensuring the build environment is managed appropriately (including ensuring that all required build dependencies are installed).
If only it were that easy!
If we add the "--no-build-isolation" flag, it becomes our responsibility to install *all* build time deps,
hence we now have "requirements-build-makepackages.txt".
The <2.1 pin had been put there as dnspython 2.1 added "poetry" as a build time dep,
which pulled in a significant number of transitive dependencies, and it was also
causing issues with our appimage build.
dnspython 2.2.0 was released since, which no longer needs full poetry, only "poetry-core":
da279dec7e
related https://github.com/spesmilo/electrum/issues/7361
trezorlib requires "dataclasses ; python_version<'3.7'", and our min supported python version is 3.6,
so freeze_packages.sh pins down a version of "dataclasses". However, when creating binaries we
use newer versions of python (typically py3.9 atm), for which dataclasses is not available (it's a backport of py3.7 stuff).
Hmhm, what to do... short-term, I am manually removing the dataclasses pin, so it won't be installed in our binaries.
```
Collecting construct==2.10.67
Downloading construct-2.10.67.tar.gz (57 kB)
|████████████████████████████████| 57 kB 2.7 MB/s
Preparing metadata (setup.py) ... done
Requirement already satisfied: cryptography==36.0.1 in ./build/appimage/electrum.AppDir/usr/lib/python3.9/site-packages (from -r /opt/electrum/contrib/build-linux/appimage/../../../contrib/deterministic-build/requirements-hw.txt (line 74)) (36.0.1)
ERROR: Could not find a version that satisfies the requirement dataclasses==0.8 (from versions: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6)
ERROR: No matching distribution found for dataclasses==0.8
```
Sander van Grieken
SomberNight
ThomasV
Jamie C. Driver