Haven't checked if electrum-ecc compiles libsecp reproducibly.
For now let's just keep the old flow.
(but if we spent time on making that compilation reproducible,
the appimage and the macos builds could use it directly)
- repro builds to use fixed uid=1000 inside the container
- in case the file permissions leak into the binaries, they are still reproducible
- chown 1000:1000 fresh_clone
- repro builds to create fresh_clone dir outside git clone
- otherwise the local dev build would still interact with the fresh_clone dir
- due to e.g. recursive "find -exec touch",
- and even the "docker build" cmd itself would try to stat/read it
- see https://github.com/docker/for-linux/issues/380
- and "rm -rf fresh_clone" needs sudo if the host uid is not 1000
- this way the local dev build does not need sudo
to recap:
- local dev builds use the host userid inside the container, directly operate on the project dir
- does not need sudo
- repro builds create a fresh git clone, chown it to 1000, and use userid=1000 inside the container
- if the host userid is 1000, does not need sudo
- otherwise, needs sudo
closes https://github.com/spesmilo/electrum/issues/8261
reproducibility probably needs a hardcoded userid
Also, move the UID arg later in the dockerfiles, for better caching.
(if local dev build and repro build set different UIDs, the build caches
will diverge at that step)
The .dockerignore symlink in the project root dir is only picked up by the android build.
The android build has the project root as its build context for "docker build" --
the other builds have their own subdirectories as build context, e.g. contrib/build-linux/appimage.
note: python 3.9.x is now in source-only mode, so could not update to latest...
it is time to investigate upgrading to python 3.10 in the win and mac binaries
fixes https://github.com/spesmilo/electrum/issues/8011
> Upgraded from Appimage 4.1.5 to 4.3.2 on MX-Linux 19.
> The new version fails to start with:
>
> ```
> $ electrum
> qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
> This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may > fix this problem.
>
> Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl, xcb.
>
> Aborted (core dumped)
> ```
debian 10 only distributes libxcb-util0.
debian 11 distributes libxcb-util1.
our base image atm is debian 10, so we build the package from source.
This reverts commit 0c2a885c66.
Reverting for now due to reproducibility issues:
```
error: /Users/vagrant/Downloads/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocate: can't open file: /tmp/electrum_compare_dmg/signed_app/Electrum.app/Contents/MacOS/libffi.8.dylib (No such file or directory)
```
`Electrum.app/Contents/MacOS/libffi.8.dylib` is included when building on one machine, but missing on the other...
Will have to investigate later.
related https://github.com/spesmilo/electrum/pull/7918
4.1.5->4.2.1, the appimage grew 54M->66M. This change shrinks it back to 58M.
```
$ ls -lah
total 224M
drwxrwxr-x 5 user user 4.0K Mar 27 18:18 .
drwxrwxr-x 16 user user 4.0K Mar 23 16:02 ..
-rwxrw-r-- 1 user user 48M Dec 18 2020 electrum-4.0.9-x86_64.AppImage
-rwxrw-r-- 1 user user 54M Jan 19 14:25 electrum-4.1.5-x86_64.AppImage
-rwxr-xr-x 1 user user 58M Mar 27 18:12 electrum-4.2.1-dirty-x86_64.AppImage
-rwxrw-r-- 1 user user 66M Mar 27 15:00 electrum-4.2.1-x86_64.AppImage
```
I've used the great `ncdu` tool to investigate file sizes.
```
$ du squashfs-root-415/usr/lib/python3.7/ --max-depth=1 | sort -nr | head -n8
154608 squashfs-root-415/usr/lib/python3.7/
138864 squashfs-root-415/usr/lib/python3.7/site-packages
4720 squashfs-root-415/usr/lib/python3.7/lib-dynload
1744 squashfs-root-415/usr/lib/python3.7/encodings
664 squashfs-root-415/usr/lib/python3.7/pydoc_data
460 squashfs-root-415/usr/lib/python3.7/distutils
460 squashfs-root-415/usr/lib/python3.7/asyncio
436 squashfs-root-415/usr/lib/python3.7/email
$ du squashfs-root-421/usr/lib/python3.9/ --max-depth=1 | sort -nr | head -n8
194088 squashfs-root-421/usr/lib/python3.9/
143512 squashfs-root-421/usr/lib/python3.9/site-packages
33824 squashfs-root-421/usr/lib/python3.9/config-3.9-x86_64-linux-gnu
5244 squashfs-root-421/usr/lib/python3.9/lib-dynload
1720 squashfs-root-421/usr/lib/python3.9/encodings
696 squashfs-root-421/usr/lib/python3.9/pydoc_data
520 squashfs-root-421/usr/lib/python3.9/asyncio
464 squashfs-root-421/usr/lib/python3.9/distutils
```
We should delete `usr/lib/python3.9/config-3.9-x86_64-linux-gnu/` (which is 33M unpacked)
With py3.7 (electrum 4.1.5), this folder was named `config-3.7m-x86_64-linux-gnu`,
presumably because the default config to compile py3.7 was `--with-pymalloc`,
but maybe it is not for py3.9... ? not sure. (see https://peps.python.org/pep-3149/ )
This is bumping the python versions bundled inside our binaries.
For macOS and AppImage, from 3.9.10 to 3.9.11.
For Android, from 3.8.12 to 3.8.13.
Windows is left untouched as I am having issues with the wine build when using 3.9.11.
(see https://github.com/spesmilo/electrum/pull/7721#issuecomment-1071901116 )
This includes two logically separate changes:
- on the host, try not to require sudo when running the build scripts
- namely when interacting with the docker daemon, this requires
the unix user on the host to be part of the `docker` group
- this solves part of https://github.com/spesmilo/electrum/issues/7602
- while running inside the docker containers, do not run as root
- this means that e.g. files created in mounted folders should
no longer be owned by root on the host
- there is some code duplication involved here - not sure
how it could be deduped.
zebra-lucky
SomberNight
Sander van Grieken
gruve-p
gruve-p
Michel van Kessel