After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 794984 - We need hosting for Developer Portal
We need hosting for Developer Portal
Status: RESOLVED FIXED
Product: sysadmin
Classification: Infrastructure
Component: Other
unspecified
Other Linux
: Normal normal
: ---
Assigned To: GNOME Sysadmins
GNOME Sysadmins
Depends on:
Blocks:
 
 
Reported: 2018-04-04 18:18 UTC by Sri Ramkrishna
Modified: 2018-08-08 08:50 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Sri Ramkrishna 2018-04-04 18:18:22 UTC
We need hosting for a documentation portal that will eventually become the new developer.gnome.org.

The site will be django based.  Let me know if you have any questions and we can fill in the blanks.  We are using pre-existing documentation portal software from Ubuntu.
Comment 1 Jim Campbell 2018-04-04 18:33:36 UTC
This seems to be being done in secret, even though I've asked Sri questions about it on IRC - I got no replies. Why has this not been discussed with the documentation team?

I don't mean to put stop energy on it, but it is strange for a documentation portal to get no mention on the documentation team list and no mention in the documentation team IRC channel.

I know that developer documentation is sorely lacking, but I don't think that going around existing contributors / not involving them at all is the best way to get community participation, and I think it's counter to how initiatives should started and dveloped in GNOME.
Comment 2 Jim Campbell 2018-04-04 18:34:26 UTC
If there have been such discussions, and I've missed them, I apologize.  I don't recall them happening, though.
Comment 3 Sri Ramkrishna 2018-04-04 23:27:13 UTC
Well, it wasn't intended to be done in secret.  But I can see where you might get that impression.  The effort is mostly about trying to create a platform to pull all the various documentation in developer/HowDoI/pygobject.readthedocs.io and others and try to make something coherent.  The project came from about from my rant on documentation in GUADEC last year.

And you shouldn't apologize.  Calling me on it is perfectly fine.  I didn't go into the documentation team mailing list and I should have.  I had thought that I would start small and look at the problem before expanding, but perhaps that was the wrong way of going about it.

I'm happy to pull a reset and involve the documentation team with my humble apologies.
Comment 4 Petr Kovar 2018-04-05 14:05:48 UTC
Jim, thanks for bringing this up.

Sri,

With my docs lead hat on, I'd say that if you have resources covered, then by all means go ahead with your effort. We've been struggling with that part in our team, but since the current developer site is still technically a part of the docs team deliverables, I'd too really appreciate if you include the docs team in your future communication. We might be able to help you out, after all.
Comment 5 Jim Campbell 2018-04-05 14:34:36 UTC
Thank you, Sri and Petr. I was not aware that this had been discussed at Guadec (though still stand by my note that involving the docs team beyond that is a preferred way to go).

As long as Petr is okay with it while wearing a docs lead hat, then I give a go-ahead, too. If the docs team can be of assistance with structuring layout, providing feedback, etc., then please do include the team.
Comment 6 Sri Ramkrishna 2018-04-05 18:11:19 UTC
Thanks Petr, much appreciated.  I would be more than happy to include the documentation team in everything we are doing.  It seems from our conversation on DDL that to make this successful requires a cross section of people as well.

So let's set some concrete steps.  Right now we have a telegram channel if one of you want to join it I would be more than happy to add you.  We have had regular meetings every Wednesday at 18:00 GMT if you are interested in joining.

Finally, I will post minutes on the documentation mailing list in case those are burdensome and I can play liaison between the two.  Most likely, we're going to need advice and discussions especially when dealing with the state of the platform when it comes to tooling.

Again, I apologize for not giving the documentation team a heads up.  The team growth metatasized and we didn't know how serious it was going to get since it was only three people wanting to solve a problem and along the way we picked up others.  Secondly, it's hard to have a conversation when there is no concrete details.  We still probably can't answer all the questions without doing some experimentation and exploring.
Comment 7 Andrea Veri 2018-04-05 19:18:00 UTC
Got an URL for me for the software that we should be installing on the GNOME servers to start testing this out?
Comment 8 Michael Hall 2018-04-05 21:18:52 UTC
The code is in Gitlab: https://gitlab.gnome.org/Community/DeveloperPortal/

The instructions are for a development environment setup using virtualenv, but for production we'll want an apache/nginx/gunicorn server connected to it via wsgi, and a postgresql database.

Is there a preferred deployment configuration tool for GNOME servers?
Comment 9 Michael Hall 2018-04-06 15:21:42 UTC
Also, would be good to know if this is Openstack infra or not, because we can use Swift buckets for storing uploaded files (images and such) to make it easier to scale in the future.
Comment 10 Sri Ramkrishna 2018-04-18 18:12:58 UTC
Hi @av, any progress on this?  We'd like to get the server up and going so we can continue working on the project.  Thank you, Andrea!
Comment 11 Sri Ramkrishna 2018-04-18 18:14:57 UTC
BTW, it's called Developer Portal not documentation portal, because I am an idiot.
Comment 12 Andrea Veri 2018-04-20 12:38:54 UTC
Will have this done early next week. A quick overview of what my plan is and a question:

1. developerportal will run via Docker as there's no RHEL release with Python 3 support yet. Are you aware whether there's any volume that should be externally mounted as it's going to store persistent data?

2. why is postgresql a requirement? I see the app is django based and django comes with mysql support as well. Asking as we have a mysqld cluster setup already.

Thanks,
Comment 13 Michael Hall 2018-04-20 16:34:25 UTC
We will need to persist uploaded content somewhere, this can be a local filesystem or an object storage system like Swift if we're running on OpenStack infrastructure.

Postgresql has better native support for Unicode among other advantages. We could try it on Mysql if it's configured to store text in UTF-8.
Comment 14 Andrea Veri 2018-04-27 12:45:02 UTC
Instance has been landed at [1], the DNS entry has been moved to the service hosted on the GNOME Infrastructure (and not the requested at [2]). Please provide me the GPG key fingerprints for the root password to be mailed over.

Thanks,

[1] https://developerportal.gnome.org
[2] https://gitlab.gnome.org/Community/DeveloperPortal/issues/5
Comment 15 Andrea Veri 2018-05-21 15:55:45 UTC
Any update on whether this service is going to be used at all? There was some pressure on making this happen but then no one ever touched the service after it got provisioned.

Thanks,
Comment 16 André Klapper 2018-06-17 16:46:34 UTC
Sri, Michael? ^^
Comment 17 Sri Ramkrishna 2018-07-26 16:39:03 UTC
We definitely want to keep going on this.  We have now expanded our group and so we should be seeing activity going forward.
Comment 18 Michael Hall 2018-07-26 17:07:22 UTC
Hey,  sorry I don't get emails when new comments are added to this, so I had no idea you were waiting on me.

My GPG key is 85064EDD0F6C20C9089F302989D461C18981F572
Comment 19 Andrea Veri 2018-07-27 11:16:48 UTC
Michel, password has been sent encrypted to you. Is there anything else I can do for you or we can mark this as resolved?

On a side note we'll soon move to Openshift, developerportal is one of the target apps we'll move as well. Will provide access to the project so you can manage deployments yourself directly.
Comment 20 Michael Hall 2018-07-27 15:01:44 UTC
Not everything is loading properly, requests to https://developerportal.gnome.org/static/djangocms_text_ckeditor/css/cms.ckeditor.css (and other files under /static/djangocms_text_ckeditor) are returning a 404. Can you verify if "manage.py collectstatic" was run and if those files exist in the filesystem?
Comment 21 Andrea Veri 2018-07-30 14:15:52 UTC
Michael, should be fixed, I see we have a gnome_theme directory now which contains a templates directory, are you aware how can we switch to it to avoid the app breaking when the templates directory cannot be found on the project's root directory?
Comment 22 Michael Hall 2018-07-30 20:18:32 UTC
"gnome_theme" is a Django app, it should have a subfolder called "templates", just like the "devporal" Django app does. Both of these are part of the source code, you'd never have one without the other, or have them in different locations.

The "collectstatic" command should combine images, css and javascript from both apps, as well as Django itself, into a ./static/ folder at the same level as manage.py, and we need the contents of that folder served statically on https://developerportal.gnome.org/static/

Whatever you changed fixed the ckeditor static files, but the gnome_theme static files are still now loading. For example: https://developerportal.gnome.org/static/gnome_theme/css/frontend.css
Comment 23 Andrea Veri 2018-07-31 10:15:14 UTC
Michael,

the static files for gnome_theme are missing due to the fact there's no mention of gnome_theme on the INSTALLED_APPS section of the settings.py file. Would you want to commit a fix so I can pull the latest code and rebuild the container or should I?
Comment 24 Michael Hall 2018-08-01 13:49:54 UTC
Hmm, it sounds like we haven't deployed the right version of the code then, because it's been in there for a while: https://gitlab.gnome.org/Community/DeveloperPortal/blob/master/devportal/settings.py#L66

Can you verify the git repo/branch/revision that's being used?
Comment 25 Andrea Veri 2018-08-01 16:50:13 UTC
Michael, if you could merge [1] that would be lovely so that we don't land any hardcoded settings.py via a Dockerfile anymore and we avoid losing changes like the gnome_theme one and others.

Thanks!

[1] https://gitlab.gnome.org/Community/DeveloperPortal/merge_requests/3
Comment 26 Michael Hall 2018-08-02 13:20:32 UTC
Okay, merged.
Comment 27 Andrea Veri 2018-08-02 14:32:39 UTC
You should be good to go!
Comment 28 Michael Hall 2018-08-03 20:39:31 UTC
Thanks Andrea!

What is the process for pushing updates to this deployment? Is it enough to just get them into the master branch in gitlab and open a new issue asking you to pull, run any needed commands, and then reload apache?
Comment 29 Andrea Veri 2018-08-08 08:50:39 UTC
Michael,

that's about right. We've recently setup an Openshift deployment, your application is one of those that will be migrated there in order for you to have access to your own project/s and process updates without requiring any manual intervention from the Infrastructure Team. I'll let you know once the app has been migrated and provide you (and other parties involved) access to the new resources.