After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 669772 - Set up ostree.gnome.org
Set up ostree.gnome.org
Status: RESOLVED FIXED
Product: sysadmin
Classification: Infrastructure
Component: Other
unspecified
Other Linux
: Normal normal
: ---
Assigned To: GNOME Sysadmins
GNOME Sysadmins
Depends on:
Blocks:
 
 
Reported: 2012-02-09 19:58 UTC by Owen Taylor
Modified: 2013-03-10 13:30 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Owen Taylor 2012-02-09 19:58:08 UTC
http://mail.gnome.org/archives/gnome-infrastructure/2012-January/msg00056.html
[ continues in February ]
http://mail.gnome.org/archives/gnome-infrastructure/2012-February/msg00002.html

Set up VM
=========
Create a RHEL 6 VM on clutter:

 - name ostree
 - 4GB ram
 - 4 Virt CPUS
 - 16GB root partition
 - 250GB data partition mounted on /srv/ostree, ownership ostree:ostree

DNS
===
Give the machine a public IP as ostree.gnome.org

Package set
===========
Colin will follow up with a list - it needs development packages for building an initial bootstrap environment

Authentication
==============
Create a 'ostree' group in LDAP, add 'walters' to that group, allow ostree group to sudo to the ostree user on ostree.gnome.org.

Web serving
===========
TBD - VM needs apache and port 80 open, but exact configuration not determined.

HTTPS
=====
We'll want to get a certificate for the machine, since we need a secure place to start downloading images.
Comment 1 Colin Walters 2012-02-16 20:18:15 UTC
Anything else that's necessary here?  Anything I can do to help?
Comment 2 Owen Taylor 2012-02-16 23:40:35 UTC
The cobbler install of the VM was having issues when I was working on it last, need to dig into that some more.
Comment 3 Colin Walters 2012-02-17 17:12:48 UTC
For the initial package set, the equivalent of @development-tools from Fedora is key.  And the version control systems.  And Docbook ideally.

So something like:

@development-tools cvs hg git tar gzip unzip docbook-style-dsssl sed docbook-style-xsl docbook-dtds docbook-utils
Comment 4 Owen Taylor 2012-02-18 17:10:09 UTC
> Set up VM
> =========
> Create a RHEL 6 VM on clutter:

Done

> DNS
> ===
> Give the machine a public IP as ostree.gnome.org

Done

> Authentication
> ==============
> Create a 'ostree' group in LDAP, add 'walters' to that group, allow ostree
> group to sudo to the ostree user on ostree.gnome.org.

Done.

Working on the package set now. It doesn't seem like puppet handles yum groups very well. 'yum grouplist "Development Tools"' on RHEL-6 gives:

 Mandatory Packages:
   autoconf automake binutils bison flex gcc gcc-c++
   gettext libtool make patch pkgconfig redhat-rpm-config rpm-build

 Default Packages:
   byacc cscope ctags cvs diffstat doxygen elfutils gcc-gfortran
   git indent intltool patchutils rcs subversion swig systemtap

I'm going to exclude from that set:

 redhat-rpm-config rpm-build cscope ctags diffstat gcc-gfortran
 indent patchutils rcs swig systemtap

Leaving, plus what you listed the above:

 autoconf automake binutils bison byacc cvs docbook-dtds docbook-style-dsssl
 docbook-style-xsl docbook-utils doxygen elfutils flex gcc gcc-c++ gettext git 
 gzip hg intltool libtool make patch pkgconfig sed subversion tar unzip
Comment 5 Owen Taylor 2012-02-18 17:38:44 UTC
Packages and httpd installed, port 80 checked to be open to the outside world.

Remaining to be done:

 - One exact desired httpd configuration is known, set that up in puppet
 - Obtain SSL certificate for extensions.gnome.org, install on machine

Let me know if you have permission problems, are missing packages, etc.
Comment 6 Colin Walters 2012-02-20 02:25:07 UTC
Looks like /srv/ostree needs a:

chown ostree:ostree /srv/ostree

As far as httpd configuration, let's have it be:

DocumentRoot /srv/ostree/public_html

Instead maybe we could make a home directory for the user - it seems weird that there isn't one.   Or maybe we could just make it /srv/ostree?
Comment 7 Owen Taylor 2012-02-20 04:39:52 UTC
(In reply to comment #6)
> Looks like /srv/ostree needs a:
> 
> chown ostree:ostree /srv/ostree

Sorry, done now.
 
> As far as httpd configuration, let's have it be:
> 
> DocumentRoot /srv/ostree/public_html
> 
> Instead maybe we could make a home directory for the user - it seems weird that
> there isn't one.   Or maybe we could just make it /srv/ostree?

Well, presumably you wouldn't want ostree.gnome.org:~ostree anyways, so I think doing it via Apache user homedir configuration would be weird. Using / for a special user like ostree is sort of normal, but I can certainly change it to /srv/ostree if that's convenient.

I'll add the web server configuration tomorrow.
Comment 8 Colin Walters 2012-02-20 14:56:06 UTC
Ok, there's one final step that needs root privileges on the guest, and that's to install as setuid root:

http://git.gnome.org/browse/linux-user-chroot/

I'm fairly confident that this tool couldn't be used to compromise integrity or confidentiality on the machine.  It's been through some review on LKML by security people, and e.g. Serge E. Hallyn roughly approved:  https://lkml.org/lkml/2011/12/12/547

However it almost certainly would be easy enough to use the tool to create tens of thousands of mount table entries for a DoS.  But on the other hand, I'm not sure there has ever been a point where one could install a "default" Fedora/RHEL/Debian that hasn't had easy to use DoS vectors.
Comment 9 Colin Walters 2012-02-20 15:32:54 UTC
(In reply to comment #7)
> 
> Well, presumably you wouldn't want ostree.gnome.org:~ostree anyways, so I think
> doing it via Apache user homedir configuration would be weird. Using / for a
> special user like ostree is sort of normal, but I can certainly change it to
> /srv/ostree if that's convenient.

It turns out I don't have bash history if I don't have a writable home directory, and history is nice.
Comment 10 Colin Walters 2012-02-20 15:44:44 UTC
Ok, Yocto appears to want these Fedora packages:

diffstat texinfo texi2html chrpath

IIRC at least texi2html isn't available in RHEL, probably because it depends on TeX =(  But I may be able to get away without it.

The others I could install just as the ostree user in /srv/ostree/bin, but it'd be nicer to have them via packages for reproducibility.
Comment 11 Owen Taylor 2012-02-20 15:57:58 UTC
(In reply to comment #9)
> (In reply to comment #7)
> > 
> > Well, presumably you wouldn't want ostree.gnome.org:~ostree anyways, so I think
> > doing it via Apache user homedir configuration would be weird. Using / for a
> > special user like ostree is sort of normal, but I can certainly change it to
> > /srv/ostree if that's convenient.
> 
> It turns out I don't have bash history if I don't have a writable home
> directory, and history is nice.

Homedir switched to /srv/ostree

(In reply to comment #10)
> Ok, Yocto appears to want these Fedora packages:
> 
> diffstat texinfo texi2html chrpath
> 
> IIRC at least texi2html isn't available in RHEL, probably because it depends on
> TeX =(  But I may be able to get away without it.
> 
> The others I could install just as the ostree user in /srv/ostree/bin, but it'd
> be nicer to have them via packages for reproducibility.

Added to puppet configuration and installed. texi2html is part of the RHEL6 optional channel.
Comment 12 Owen Taylor 2012-02-20 18:43:08 UTC
linux-user-chroot packaged, built in mock, installed - I've packaged it so only users in the luchroot group can run the binary.

In puppet I've added ostree to luchroot

libxslt and screen installed on the system as requested on IRC

public_html/ exported as requested. I added a dummy index.html there to check to see if it was serving files correctly.

Remaining todo from this bug is HTTPS.
Comment 13 Colin Walters 2012-06-07 18:11:38 UTC
Could I get buildbot-master and buildbot-slave installed on ostree.gnome.org?

Looks like npmccallum has a personal repo here:

http://repos.fedorapeople.org/repos/npmccallum/buildbot/epel-buildbot.repo

Also, one thing I'm wondering is if it'd be possible to automate some of the bringup of the machine in puppet.  I have a fair amount of stuff that I've done manually at the command line that's only semi-documented.

How do you guys normally do this?  One idea I have is to start writing the kickstart/puppet recipies which bring up a ostree-test.gnome.org VM, iterate on them until it works, then replace the current server with that.

Or I guess right now the /srv/ostree virtual disk is separate, so in theory we should be able to destroy the current VM, but keep /srv/ostree and have things work.
Comment 14 Owen Taylor 2012-06-07 18:36:46 UTC
(In reply to comment #13)
> Could I get buildbot-master and buildbot-slave installed on ostree.gnome.org?
> 
> Looks like npmccallum has a personal repo here:
> 
> http://repos.fedorapeople.org/repos/npmccallum/buildbot/epel-buildbot.repo

Would you mind filing a new ticket for this?

> Also, one thing I'm wondering is if it'd be possible to automate some of the
> bringup of the machine in puppet.  I have a fair amount of stuff that I've done
> manually at the command line that's only semi-documented.
>
> How do you guys normally do this?  One idea I have is to start writing the
> kickstart/puppet recipies which bring up a ostree-test.gnome.org VM, iterate on
> them until it works, then replace the current server with that.

What we normally do is to make all our configuration changes through puppet -
so e.g., with the above, we'd add buildbot-master and buildbot-slave to the
ostree module in puppet rather than doing 'yum install'. We don't have any sort
of verification procedure, but in general, we've had good luck when setting up
a new VM and moving a service there that the service is 98% working.
 
> Or I guess right now the /srv/ostree virtual disk is separate, so in theory we
> should be able to destroy the current VM, but keep /srv/ostree and have things
> work.

That's the idea - that stuff that isn't part of the system configuration in puppet
is kept in a separate place. Note that /srv/ostree *is not backed up* as discussed.
If you feel that it's getting hard to recreate, please reconsider whether you want
some parts of it under backup.

Our puppet setup at this point is restricted to root-level configuration, so it would
be hard to use it for the details of ostree as long as its actively changing.
Comment 15 Colin Walters 2012-08-24 12:59:39 UTC
Could zlib-devel be installed on ostree.gnome.org?  Actually I'll need the packages specified here:

http://git.gnome.org/browse/gnome-ostree/tree/install/amazon-linux-cloud-init.sh
Comment 16 Andrea Veri 2012-08-24 15:44:28 UTC
Done. All packages should be installed.
Comment 17 Andreas Nilsson 2013-03-09 12:52:09 UTC
Mark bug as fixed?
Comment 18 Andrea Veri 2013-03-10 13:30:03 UTC
Yeah, we can mark this as resolved now. Colin, please add additional bugs or ping me directly on IRC if you need anything else to be added on your setup.