Created attachment 373503 [details] [review]
cerbero: Implement --offline for package and build

This forces Cerbero to not attempt to update anything over the network,
especially git repositories. Very useful for reproducible builds.

The remaining bit is implementing it for bootstrap, which is much more
complicated.

I don't think it's related to reproducible builds. A reproducible build is just a build where all the versions has been fixated. It does not matter if you are online or offline. This should already be happening with the with the CI builds, since the only unfixed version should be the one of GStreamer repository, and we do fixate the versions there from the manifest when doing these builds. The bundle_source is also a much more powerful method to archive the build in a way that it can be reproduced, and also reproduced offline.

That being said, I'm fine if you want to add an offline mode, that prevents fetches, that seems generally useful, specially if you are editing the git repo and you don't want cerbero to mess them up.

The workflow is:

./cerbero-uninstalled fetch gstreamer-1.0

[copy local_sources to wherever you want]

./cerbero-uninstalled package --offline gstreamer-1.0

(In reply to Nicolas Dufresne (ndufresne) from comment #2)
> I don't think it's related to reproducible builds. A reproducible build is
> just a build where all the versions has been fixated. It does not matter if
> you are online or offline. 

It's related to reproducible builds in practice because if you build cerbero master that's going to pull in latest git. Fetch will update the git repo when building even if the local_sources exist.

This also happens when building the stable branches like 1.14. The only time this doesn't happen is when you build a specific tag like 1.14.2.

bundle-source is fine, but the git repos will *still* be updated when you build, afaik. So this is the other part that's needed to make bundle-source work as expected.

(In reply to Nirbheek Chauhan from comment #3)
> The workflow is:
> 
> ./cerbero-uninstalled fetch gstreamer-1.0
> 

*fetch-package

Well, there is a bug in bundle-source, it should generate a manifest, while --offline is a nice workaround for this one, it's not a nice solution.

For reproducible builds, shouldn't we add something to generate a manifest.. But I see that as orthogonal to having a --offline

(In reply to Olivier Crête from comment #6)
> For reproducible builds, shouldn't we add something to generate a manifest..
> But I see that as orthogonal to having a --offline

I agree, we need a manifest and the ability to build a particular set of commits and tarball URLs.

--offline helps by removing all network access during the build. For instance, building on a buildbot without network access, or avoiding build failures because the git repository was temporarily unavailable, and we didn't even need to update because we already have the refs needed.

In any case, the manifest and bundle-source improvements are for another bug IMO, and it seems this feature is useful anyway.

Thanks for the comments!

Attachment 373503 [details] pushed as d8c0f9b - cerbero: Implement --offline for package and build