After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 780789 - Convert gtkdoc-scan from Perl to Python
Convert gtkdoc-scan from Perl to Python
Status: RESOLVED FIXED
Product: gtk-doc
Classification: Platform
Component: general
unspecified
Other Linux
: Normal enhancement
: 1.26
Assigned To: Stefan Sauer (gstreamer, gtkdoc dev)
gtk-doc maintainers
Depends on:
Blocks:
 
 
Reported: 2017-03-31 21:31 UTC by Jussi Pakkanen
Modified: 2017-04-09 20:42 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Happy now, Bugzilla? (82.10 KB, patch)
2017-04-01 16:14 UTC, Jussi Pakkanen
none Details | Review
Listed things fixed, no other changes (79.88 KB, patch)
2017-04-03 17:03 UTC, Jussi Pakkanen
none Details | Review
Convert gtkdoc-scan to python (80.49 KB, patch)
2017-04-07 21:25 UTC, Stefan Sauer (gstreamer, gtkdoc dev)
none Details | Review
Convert gtkdoc-scan to python (81.20 KB, patch)
2017-04-09 20:06 UTC, Stefan Sauer (gstreamer, gtkdoc dev)
none Details | Review
Convert gtkdoc-scan to python (81.14 KB, patch)
2017-04-09 20:38 UTC, Stefan Sauer (gstreamer, gtkdoc dev)
committed Details | Review

Description Jussi Pakkanen 2017-03-31 21:31:38 UTC
DOES NOT WORK ON REAL WORLD PROJECTS, DOES NOT EVEN PASS TESTS.

Here is a very rough conversion of gtkdoc-scan to Python. It works only as much as the test suite does not throw any Python syntax errors. The output is wrong.

I figured I'd put this up, because there are probably people who are more efficient in debugging the behaviour.
Comment 1 Stefan Sauer (gstreamer, gtkdoc dev) 2017-04-01 14:48:54 UTC
ENOPATCH :)
Comment 2 Jussi Pakkanen 2017-04-01 16:14:07 UTC
Created attachment 349116 [details] [review]
Happy now, Bugzilla?
Comment 3 Stefan Sauer (gstreamer, gtkdoc dev) 2017-04-02 17:52:48 UTC
Review of attachment 349116 [details] [review]:

Can you break out a separate patch with common.py (+ the toplevel Makefile.am) changes? For gtkdoc-scan.in I did not made it to the end.

For the logging, please always use "... %s", string) to not run into trouble for strings that e.g. contain "%" chars.

::: gtkdoc-scan.in
@@ +64,3 @@
+parser.add_argument('--rebuild-types', dest='rebuild_types',
+                    action='store_true', default=False)
+parser.add_argument('--output-dir', dest='output_dir', default='.')

--version missing

@@ +87,3 @@
+    old_types = os.path.join(options.output_dir, options.module+".types")
+    new_types = os.path.join(options.output_dir, options.module+".types.new")
+    if options.module == '':

please run pep8 against the source, I'll add a pre-commit check later today.
I am fine with longer lines, but here e.g. please add whitespace around operators.

@@ +91,3 @@
     # If this is the very first run then we create the .types file automatically.
+    if not os.path.exists(sections_file) and not os.path.exists(old_types):
+        REBUILD_TYPES = 1

rebuild_types = True

@@ +96,3 @@
+    DECL = open(new_decl, 'w')
+    if options.rebuild_types:
+        TYPES = open(new_types)

open(new_types, 'w')

@@ +151,3 @@
 
+def ScanHeaders(source_dir, section_list, options):
+    logging.info("Scanning source directory: $source_dir")

logging.info("Scanning source directory: %s", source_dir)

@@ +227,2 @@
     # Don't scan headers twice
+    canonical_input_file = os.path.join(os.getcwd(), input_file)

os.path.realpath

@@ +228,3 @@
+    canonical_input_file = os.path.join(os.getcwd(), input_file)
+    if canonical_input_file in seen_headers:
+        logging.info("File already scanned: $input_file")

.. %s", input_file)

@@ +253,1 @@
+    logging.info("Scanning " + input_file)

%s", input_file)

@@ +260,3 @@
         # Skip to the end of the current comment.
+        if in_comment:
+            logging.info("Comment: " + line)

%s", line)

@@ +304,3 @@
+            m = re.search(r'^\s*#\s*(if*|define)', line)
+            if not (m  or in_declaration == "enum"):
+                logging.info("Found deprecation annotation (decl: '%s'): %s", (in_declaration, line))

logging.info("Found deprecation annotation (decl: '%s'): %s", in_declaration, line)

@@ +318,1 @@
+        if in_declaration:

if not in_declaration:

@@ +319,2 @@
             # Skip top-level comments.
+            m = re.search(r'^\s*/\*%')

m = re.search(r'^\s*/\*%', line)

@@ +320,3 @@
+            m = re.search(r'^\s*/\*%')
+            if m:
+                if re.search(r'\*/'):

re.search(r'\*/', line):

@@ +321,3 @@
+            if m:
+                if re.search(r'\*/'):
+                    logging.info("Found one-line comment: $_")

logging.info("Found one-line comment: %s", line)

@@ +325,3 @@
+                    in_comment = 1
+                    doc_comment = m.group(0)
+                    logging.info("Found start of comment: " + doc_comment)

%s", doc_comment)

@@ +328,3 @@
+                continue
+
+                    in_comment = 1

logging.info("0: %s", line)

::: gtkdoc/common.py
@@ +25,3 @@
+# These are functions used by several of the gtk-doc Perl scripts.
+# We'll move more of the common routines here eventually, though they need to
+# stop using global variables first.

remove this comment

@@ +39,3 @@
+#                $make_backup - 1 if a backup of the old file should be kept.
+#                        It will have the .bak suffix added to the file name.
+#############################################################################

Can you turn this into a python docstring?

@@ +43,3 @@
+def UpdateFileIfChanged(old_file, new_file, make_backup):
+
+    ##("Comparing $old_file with $new_file...");

logging.info()

@@ +61,3 @@
+        os.rename(old_file, backupname)
+    if os.path.exists(old_file):
+        os.unlink(old_file)

if os.path.exists(old_file):
  if make_backup:
    backupname = old_file + '.bak'
    if os.path.exists(backupname):
      os.unlink(backupname)
    os.rename(old_file, backupname)
  else:
    os.unlink(old_file)
Comment 4 Stefan Sauer (gstreamer, gtkdoc dev) 2017-04-02 20:01:00 UTC
I've extraced the common.py part, modified it and added a test for it.

commit 6e5bb3ca8d95e6af189574522338ea48072b7dfa
Author: Stefan Sauer <ensonic@users.sf.net>
Date:   Sun Apr 2 21:58:16 2017 +0200

    common.py: start porting common.py from perl
    
    Add a first function with a test.
Comment 5 Jussi Pakkanen 2017-04-03 17:03:33 UTC
Created attachment 349190 [details] [review]
Listed things fixed, no other changes
Comment 6 Stefan Sauer (gstreamer, gtkdoc dev) 2017-04-07 21:25:06 UTC
Created attachment 349506 [details] [review]
Convert gtkdoc-scan to python

Almost there, mostly whitespace related changes.
Comment 7 Stefan Sauer (gstreamer, gtkdoc dev) 2017-04-09 20:06:53 UTC
Created attachment 349566 [details] [review]
Convert gtkdoc-scan to python
Comment 8 Stefan Sauer (gstreamer, gtkdoc dev) 2017-04-09 20:38:26 UTC
Created attachment 349567 [details] [review]
Convert gtkdoc-scan to python