Skip to content

slapgrid-sr: do not rebootstrap unnecessarily

Bryton Lacquement requested to merge marker into master

Currently, bootstrapBuildout is called unconditionally, and since Software Releases use slapos.reboostrap, we end up with the following behaviour in the best case:

Processing software releases...
Installing software release /srv/slapgrid/slappart9/srv/testnode/aai/software.cfg...
Generated script '/srv/slapgrid/slappart9/srv/testnode/aai/soft/08f8010370c519147fe23fed3170be9e/bin/buildout'.
slapos.rebootstrap: Make sure that the section 'python2.7' won't be reinstalled after rebootstrap.
Unused options for buildout: 'allowed-eggs-from-site-packages' 'exec-sitecustomize' 'include-site-packages' 'unzip'.
Updating xz-utils.
Updating patch.
...
Updating gcc.
Updating python2.7.
Stripping binaries ...
Done.
slapos.rebootstrap: 
************ REBOOTSTRAP: IMPORTANT NOTICE ************
bin/buildout is being reinstalled right now, as new python:
  /srv/slapgrid/slappart9/srv/testnode/aai/soft/08f8010370c519147fe23fed3170be9e/parts/python2.7/bin/python2.7
is available, and buildout is using another python:
  /opt/slapgrid/b9f9e0967ab8491759399f306d65239a/parts/python2.7/bin/python2.7
Buildout will be restarted automatically to have this change applied.
************ REBOOTSTRAP: IMPORTANT NOTICE ************

Unused options for buildout: 'allowed-eggs-from-site-packages' 'exec-sitecustomize' 'include-site-packages' 'unzip'.
Updating xz-utils.
Updating patch.
...
Updating gcc.
Updating python2.7.
Updating autoconf.
...

Which means that bin/buildout always runs twice: updating parts is usually fast but loading extends and checking that binaries are stripped can take a while.

The idea of this commit is minimize the amount of work when bin/buildout already exists, in particular if it is already changed by slapos.reboostrap to use the built Python. We also hope this will avoid complete rebuilds when building a different version of Python:

Installing software release /srv/slapgrid/slappart1/srv/testnode/bsu/software.cfg...
Generated script '/srv/slapgrid/slappart1/srv/testnode/bsu/soft/90adf14823b5e2bc7dd89ccfbd9388df/bin/buildout'.
slapos.rebootstrap: Make sure that the section 'python3.5' won't be reinstalled after rebootstrap.
Uninstalling python3.5.
Uninstalling file.
Uninstalling file-msooxml.
Uninstalling gettext.
Uninstalling lunzip.
Uninstalling libxml2.
Uninstalling sqlite3.
Uninstalling openssl.
Uninstalling ca-certificates.
Uninstalling perl.
Uninstalling gdbm.
Uninstalling bzip2.
Uninstalling libffi.
Uninstalling libexpat.
Uninstalling readline.
Uninstalling ncurses.
Uninstalling zlib.
Uninstalling patch.
Uninstalling xz-utils.
Unused options for buildout: 'allowed-eggs-from-site-packages' 'exec-sitecustomize' 'include-site-packages' 'unzip'.
Installing xz-utils.
...
Stripping binaries ...
Done.
slapos.rebootstrap: 
************ REBOOTSTRAP: IMPORTANT NOTICE ************
bin/buildout is being reinstalled right now, as new python:
  /srv/slapgrid/slappart1/srv/testnode/bsu/soft/90adf14823b5e2bc7dd89ccfbd9388df/parts/python3.5/bin/python3.5
is available, and buildout is using another python:
  /opt/slapgrid/b9f9e0967ab8491759399f306d65239a/parts/python2.7/bin/python2.7
Buildout will be restarted automatically to have this change applied.
************ REBOOTSTRAP: IMPORTANT NOTICE ************

Uninstalling template.
...
Uninstalling m4.
Unused options for buildout: 'allowed-eggs-from-site-packages' 'exec-sitecustomize' 'include-site-packages' 'unzip'.
Updating xz-utils.
...
Updating python3.5.
Installing m4.
...

About the implementation, we could merely add a if not os.path.exists(...):. I fear cases where a previous run may have left bin/buildout in a non-working state, so I suggest the use of a marker to force bootstrap if a previous run failed whereas the existing bin/buildout was reused.

Merge request reports