After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 780512 - Convert gtkdoc-rebase from Perl to Python
Convert gtkdoc-rebase from Perl to Python
Status: RESOLVED FIXED
Product: gtk-doc
Classification: Platform
Component: general
1.25
Other Linux
: Normal enhancement
: ---
Assigned To: gtk-doc maintainers
gtk-doc maintainers
Depends on:
Blocks:
 
 
Reported: 2017-03-24 20:41 UTC by Jussi Pakkanen
Modified: 2017-03-29 19:28 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Patch for gtkdoc-rebase (19.17 KB, patch)
2017-03-24 20:41 UTC, Jussi Pakkanen
none Details | Review
Updated patch (19.10 KB, patch)
2017-03-28 19:31 UTC, Jussi Pakkanen
none Details | Review
Version 3 (20.71 KB, patch)
2017-03-29 18:42 UTC, Jussi Pakkanen
committed Details | Review
Converted gtkdoc-rebase from Perl to Python. (20.78 KB, patch)
2017-03-29 19:28 UTC, Stefan Sauer (gstreamer, gtkdoc dev)
committed Details | Review

Description Jussi Pakkanen 2017-03-24 20:41:57 UTC
Created attachment 348672 [details] [review]
Patch for gtkdoc-rebase

This patch converts gtkdoc-rebase from Perl to Python. Seems to work on a quick glance but the tests do not pass. The reason for this is that the test runner tries to invoke the script as a Perl module rather than running it as an executable. I don't know how Perl testing frameworks do their thing so I don't really have a good grasp on how this should be fixed.
Comment 1 Stefan Sauer (gstreamer, gtkdoc dev) 2017-03-27 18:24:14 UTC
We need to rewrite the tests as python tests. I am okay if you just disable the test in the patch for now. Before we convert the scripts that use the common module, we should add a python module too, then we can move the scripts in and make them testable.
Comment 2 Stefan Sauer (gstreamer, gtkdoc dev) 2017-03-27 19:10:12 UTC
Review of attachment 348672 [details] [review]:

Thanks. A great start!

::: gtkdoc-rebase.in
@@ +50,3 @@
+parser.add_argument('--version', action='version', version='@VERSION@')
+parser.add_argument('--html-dir', dest='html_dir', default='')
+parser.add_argument('--other-dir', dest='other_dir', default='')

this can be given multiple times, needs to become an array/list, use: action="append"

@@ +53,3 @@
+parser.add_argument('--dest-dir', dest='dest_dir', default='')
+parser.add_argument('--online', action='store_true', default=False)
+parser.add_argument('--version', action='version', version='@VERSION@')

please use a mutex group for online/relative:
https://docs.python.org/3/library/argparse.html#mutual-exclusion

@@ +78,3 @@
+    dir = subprocess.check_call(['@PKG_CONFIG@', '--variable=prefix', 'glib-2.0'], universal_newlines=True)
+    dir = dir.strip()
+                log(options, "Prepending GNOME2_PATH directory:", dir)

nit, whitespace after ',' but maybe we just run autopep8 once we've converted those.

@@ +82,3 @@
+    other_dirs = [dir] + other_dirs
+
+

one # is enough

@@ +90,3 @@
+        RelativizeLocalMap(options.html_dir);
+
+    log(options, "Prepending GLib directory", dir)

lowercase html_dir

@@ +120,3 @@
+                have_index = True
+
+            if file.is_dir():

None. More pythonic
if onlinedir and file.name == "index.sgml":

@@ +135,3 @@
+''' % (dir, file.name))
+
+                # Prefer this location over possibly stale index.sgml

IO:Zlib -> gzip module

@@ +156,2 @@
             # Remove trailing non-directory component.
+            onlinedir = re.sub(r'''(.*/).*''', r'\1', trial)

onlinedir = re.sub(r'''(.*/).*''', r'\1', match.groups(1))

@@ +167,3 @@
+            break
+        match = re.match(r'''^<ONLINE\s+href\s*=\s*"([^"]+)"\s*>''', line)
+        if match is not None:

if match:

@@ +171,2 @@
             # Remove trailing non-directory component.
+            onlinedir = re.sub(r'''(.*/).*''', r'\1', trial)

also inline 'trial' var

@@ +177,3 @@
+    package = None
+
+    package = os.path.split(dir)[1]

not sure, but isn't this taking the last dir entry? Maybe?
package = os.path.split(dir)[-1]

@@ +181,3 @@
+        dir = dir[len(options.dest_dir)-1:]
+
+    package = None

if onlinedir:

@@ +200,3 @@
+    prefix = os.path.split(dir)
+    for package, dir in LocalMap.items():
+    else:

?

@@ +226,3 @@
+        contents = contents[match.end(0):]
+        match = re.search(regex, contents, flags=re.MULTILINE)
+    prefix = os.path.split(dir)

re.sub should be able to use a function too:
https://docs.python.org/3/library/re.html#re.sub

@@ +239,3 @@
+    origdir = 'INVALID'
+
+            log(options, "Relativizing local location of $package to " + dir)

if match:

@@ +252,3 @@
+                package = match.groups(1);
+
+

if package:

@@ +257,3 @@
+            elif package in LocalMap:
+              dir = LocalMap[package]
+    log(options, "Fixing file: " + filename)

os.path.join() ?

@@ +265,3 @@
+                tmp = Mapped[origdir]
+                tmp[1] = tmp[1] + 1
+    processed = ''

>>> foo = {}
>>> foo['X'] = ["str", 1]
>>> foo['X'][1] += 1
>>> foo['X']
['str', 2]

so you should be able to write
Mapped[origdir][1] += 1
Comment 3 Jussi Pakkanen 2017-03-28 19:31:20 UTC
Created attachment 348898 [details] [review]
Updated patch

Updated and Perl test disabled.
Comment 4 Stefan Sauer (gstreamer, gtkdoc dev) 2017-03-29 18:01:48 UTC
Review of attachment 348898 [details] [review]:

Thanks. Only two small comments.

::: gtkdoc-rebase.in
@@ +157,2 @@
             # Remove trailing non-directory component.
+            onlinedir = re.sub(r'(.*/).*', r'\1', trial)

inline trial like below.

::: tests/gtkdoc-rebase.t
@@ +26,3 @@
 
+# Disabled until transition to Python is complete.
+#require_ok ("gtkdoc-rebase");

just delete the file and remove from the tests.
Comment 5 Jussi Pakkanen 2017-03-29 18:42:43 UTC
Created attachment 348945 [details] [review]
Version 3
Comment 6 Stefan Sauer (gstreamer, gtkdoc dev) 2017-03-29 19:27:54 UTC
The following fix has been pushed:
d1198e5 Converted gtkdoc-rebase from Perl to Python.
Comment 7 Stefan Sauer (gstreamer, gtkdoc dev) 2017-03-29 19:28:03 UTC
Created attachment 348950 [details] [review]
Converted gtkdoc-rebase from Perl to Python.