After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 742349 - Implement logic in Python instead of using CPython's API
Implement logic in Python instead of using CPython's API
Status: RESOLVED FIXED
Product: libpeas
Classification: Platform
Component: python
git master
Other Linux
: Normal normal
: ---
Assigned To: libpeas-maint
libpeas-maint
Depends on:
Blocks:
 
 
Reported: 2015-01-05 01:23 UTC by Garrett Regier
Modified: 2015-01-20 09:25 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Shorten the internal Python filename (5.21 KB, patch)
2015-01-05 15:45 UTC, Garrett Regier
none Details | Review
Move internal Python code to another file (14.75 KB, patch)
2015-01-05 15:45 UTC, Garrett Regier
none Details | Review
Remove overly verbose warnings in the Python loader (3.21 KB, patch)
2015-01-05 15:47 UTC, Garrett Regier
none Details | Review
Use Python to implement the plugin loader's logic (35.43 KB, patch)
2015-01-05 15:47 UTC, Garrett Regier
none Details | Review
Add verbose Python plugin loader warnings (10.63 KB, patch)
2015-01-05 15:50 UTC, Garrett Regier
none Details | Review
Compile internal Python source during build (7.47 KB, patch)
2015-01-18 14:16 UTC, Garrett Regier
committed Details | Review
Shorten Python plugin loader filenames (7.96 KB, patch)
2015-01-18 14:17 UTC, Garrett Regier
committed Details | Review
Move internal Python code to another file (13.53 KB, patch)
2015-01-18 14:17 UTC, Garrett Regier
committed Details | Review
Remove overly verbose warnings in the Python loader (3.20 KB, patch)
2015-01-18 14:18 UTC, Garrett Regier
needs-work Details | Review
Use Python to implement the plugin loader's logic (35.53 KB, patch)
2015-01-18 14:18 UTC, Garrett Regier
none Details | Review
Add verbose Python plugin loader warnings (10.89 KB, patch)
2015-01-18 14:19 UTC, Garrett Regier
none Details | Review
Use Python to implement the plugin loader's logic v3 (35.70 KB, patch)
2015-01-20 08:17 UTC, Garrett Regier
committed Details | Review
Add verbose Python plugin loader warnings v3 (5.13 KB, patch)
2015-01-20 08:21 UTC, Garrett Regier
accepted-commit_now Details | Review
Add verbose Python plugin loader warnings v4 (3.74 KB, patch)
2015-01-20 09:14 UTC, Garrett Regier
committed Details | Review
Do not reveal Python loader internals in tracebacks (2.53 KB, patch)
2015-01-20 09:15 UTC, Garrett Regier
committed Details | Review

Description Garrett Regier 2015-01-05 01:23:53 UTC
Instead of implementing all of the loader's logic using CPython's API we should implement it in Python itself using internal.py.
Comment 1 Garrett Regier 2015-01-05 15:45:39 UTC
Created attachment 293829 [details] [review]
Shorten the internal Python filename

Using peas-plugin-loader-python-internal.py is just too long.

---

This is useful for coming patches.
Comment 2 Garrett Regier 2015-01-05 15:45:59 UTC
Created attachment 293830 [details] [review]
Move internal Python code to another file

All code that deals with internal.py is now in peas-python-internal.c instead of cluttering the plugin loader code.
Comment 3 Garrett Regier 2015-01-05 15:47:17 UTC
Created attachment 293833 [details] [review]
Remove overly verbose warnings in the Python loader

Instead of very specific and still confusing messages we use g_return_*() to give us an idea about what went wrong while initializing. This only affects the internal Python initialization code.
Comment 4 Garrett Regier 2015-01-05 15:47:49 UTC
Created attachment 293834 [details] [review]
Use Python to implement the plugin loader's logic

This allows us to avoid the CPython API and have a more understandable implementation.
Comment 5 Garrett Regier 2015-01-05 15:50:23 UTC
Created attachment 293835 [details] [review]
Add verbose Python plugin loader warnings

Supports formatting Python objects and automatically includes the current Python exception in the message if there is one. This is great for the test suite which
fails on the first uncaught warning, thus the Python exception is normally not printed.

---

This sucks, but there isn't an easier way to turn a Python exception into a string.
Comment 6 Garrett Regier 2015-01-18 14:16:57 UTC
Created attachment 294783 [details] [review]
Compile internal Python source during build

Otherwise syntax errors aren't noticed until the loader is initialized at runtime.
Comment 7 Garrett Regier 2015-01-18 14:17:20 UTC
Created attachment 294784 [details] [review]
Shorten Python plugin loader filenames

Using "peas-plugin-loader-python" as a prefix is just too long.
Comment 8 Garrett Regier 2015-01-18 14:17:50 UTC
Created attachment 294785 [details] [review]
Move internal Python code to another file

All code that deals with internal.py is now in peas-python-internal.c instead of cluttering the plugin loader code.
Comment 9 Garrett Regier 2015-01-18 14:18:19 UTC
Created attachment 294786 [details] [review]
Remove overly verbose warnings in the Python loader

Instead of very specific and still confusing messages we use g_return_*() to give us an idea about what went wrong while initializing. This only affects the internal Python initialization code.
Comment 10 Garrett Regier 2015-01-18 14:18:45 UTC
Created attachment 294787 [details] [review]
Use Python to implement the plugin loader's logic

This allows us to avoid the CPython API and have a more understandable implementation.
Comment 11 Garrett Regier 2015-01-18 14:19:21 UTC
Created attachment 294788 [details] [review]
Add verbose Python plugin loader warnings

Supports formatting Python objects and automatically includes the current Python exception in the message if there is one. This is great for the test suite which fails on the first uncaught warning, thus the Python exception is normally not printed.
Comment 12 Garrett Regier 2015-01-20 08:17:09 UTC
Created attachment 294952 [details] [review]
Use Python to implement the plugin loader's logic v3

This allows us to avoid the CPython API and have a more understandable implementation.

----

Just a few tweaks for the next patch.
Comment 13 Garrett Regier 2015-01-20 08:21:36 UTC
Created attachment 294953 [details] [review]
Add verbose Python plugin loader warnings v3

Use a Python hook to implement the call logic. This allows use to have verbose error messages and include the exception traceback.

This also adds format_plugin_exception() which removes all internal Python frames from the traceback.


----

This is a complete rework of the original patch, all of the work is now done in Python. This is of course the goal of this patch set and avoids adding ~200 lines of C to do an incomplete version of this path.
Comment 14 Ignacio Casal Quinteiro (nacho) 2015-01-20 08:53:25 UTC
Review of attachment 294783 [details] [review]:

Looks good, see the note though.

::: loaders/python/peas-python-compile.py
@@ +4,3 @@
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU Library General Public License as published by

old license? you should use the Lesser one?
Comment 15 Ignacio Casal Quinteiro (nacho) 2015-01-20 08:54:12 UTC
Review of attachment 294784 [details] [review]:

Sure
Comment 16 Ignacio Casal Quinteiro (nacho) 2015-01-20 08:59:00 UTC
Review of attachment 294785 [details] [review]:

Looks good. About the license, maybe make another commit changing all the licenses?

::: loaders/python/peas-plugin-loader-python.c
@@ +620,3 @@
     {
       state = PyGILState_Ensure ();
+      g_clear_pointer (&priv->internal,

checking already for priv->internal != NULL, just call the free method directly? or you want to safe the extra line to set it to NULL?

::: loaders/python/peas-python-internal.c
@@ +6,3 @@
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU Library General Public License as published by

lesser?

@@ +17,3 @@
+ *  You should have received a copy of the GNU Library General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.

also this is not used anymore afaik, it should point to the web page or something if I recall correct
Comment 17 Ignacio Casal Quinteiro (nacho) 2015-01-20 09:00:47 UTC
Review of attachment 294786 [details] [review]:

See the comment.

::: loaders/python/peas-python-internal.c
@@ -123,3 @@
-                 "failed to run internal Python code");
-
-      Py_DECREF (globals);

you are removing this decref, is this right? if yes please put it in a different commit
Comment 18 Ignacio Casal Quinteiro (nacho) 2015-01-20 09:06:19 UTC
Review of attachment 294952 [details] [review]:

Overlooked at it and seems fine to me. Cool stuff.
Comment 19 Ignacio Casal Quinteiro (nacho) 2015-01-20 09:08:32 UTC
Review of attachment 294953 [details] [review]:

This should be split in 2 patches but ok
Comment 20 Garrett Regier 2015-01-20 09:14:00 UTC
(In reply to comment #14)
> Review of attachment 294783 [details] [review]:
> 
> Looks good, see the note though.
> 
> ::: loaders/python/peas-python-compile.py
> @@ +4,3 @@
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU Library General Public License as published by
> 
> old license? you should use the Lesser one?

Huh, seems when the license was updated we used 2.0 not 2.1, will update in another bug.


(In reply to comment #16)
> Review of attachment 294785 [details] [review]:
> 
> Looks good. About the license, maybe make another commit changing all the
> licenses?
> 
> ::: loaders/python/peas-plugin-loader-python.c
> @@ +620,3 @@
>      {
>        state = PyGILState_Ensure ();
> +      g_clear_pointer (&priv->internal,
> 
> checking already for priv->internal != NULL, just call the free method
> directly? or you want to safe the extra line to set it to NULL?

Fixed.


(In reply to comment #17)
> Review of attachment 294786 [details] [review]:
> 
> See the comment.
> 
> ::: loaders/python/peas-python-internal.c
> @@ -123,3 @@
> -                 "failed to run internal Python code");
> -
> -      Py_DECREF (globals);
> 
> you are removing this decref, is this right? if yes please put it in a
> different commit

Nope, just fallout from trying to split all of the changes. Fixed.

(In reply to comment #19)
> Review of attachment 294953 [details] [review]:
> 
> This should be split in 2 patches but ok

Done.
Comment 21 Garrett Regier 2015-01-20 09:14:49 UTC
Created attachment 294960 [details] [review]
Add verbose Python plugin loader warnings v4

Use a Python hook to implement the call logic. This allows us to have verbose error messages and include the exception traceback.
Comment 22 Garrett Regier 2015-01-20 09:15:11 UTC
Created attachment 294961 [details] [review]
Do not reveal Python loader internals in tracebacks

This adds format_plugin_exception() which removes all internal Python frames from the traceback.
Comment 23 Ignacio Casal Quinteiro (nacho) 2015-01-20 09:17:58 UTC
Review of attachment 294960 [details] [review]:

Looks good.
Comment 24 Ignacio Casal Quinteiro (nacho) 2015-01-20 09:19:07 UTC
Review of attachment 294961 [details] [review]:

OK
Comment 25 Garrett Regier 2015-01-20 09:25:07 UTC
This problem has been fixed in the development version. The fix will be available in the next major software release. Thank you for your bug report.