GNOME Bugzilla – Bug 668136
No language definition for Python 3000
Last modified: 2012-02-04 20:43:54 UTC
Created attachment 205485 [details] Python 3 Language Definition for GtkSourceView Python 2 and 3 are very different. Strings are Unicode: - a string without a prefix behaves like u"hello" did in Python 2, u"" is gone. - 2's old binary strings ("hello") are replaced with bytes literals (b"hello") New number litrals: - 0o770 for octal (also works in 2.7, but Python 3 removes the old 0777 syntax) - 0b1110 for binary (also works in 2.7) New and removed keywords, functions and objects. These are the most pressing: - print and exec are now functions, not keywords - there's a new keyword "nonlocal" - the built-in function xrange has been removed So the differences are fairly major, and even though Python 3 doesn't have its own mimetype, I think they merit an additional language definition.
Hi Stefano! thanks for the file... I think we should try to share as much as possible between the two files by using cross-lang context incusion. This may also mean to split some of the contexts in python 2 so that for instance you move xrange in a "deprecated-keyword" context and than you can include the "keyword" context
Well, the more definitions they share, the more of them will be wrong for either language. As a Python nutter, I see a strong case for them being considered separate languages in GtkSourceView: A Python 2 interpreter doesn't run 3.x programs (and vice versa!) - they do however share a file extension and mimetype. Linux distributions therefore ship with 2.x as the default, "env python", and offer 3.x as a separate package - with a plan to 'eventually' switch. GtkSourceView should use the same strategy. Now I'm not even slightly familiar with sourceview's development practices, so feel free to tell me if I'm completely wrong. But really think they should be separate, so that developers can optionally enable Py3k in their editor. When distributions switch to Py3k as the default (a good number of years into the future), GtkSourceView can switch it round, making Python 3 the default highlighting with a second optional 2.x language definition.
I did not explain myself well. I agree they should be separate lang files. but for instance in your lang file you have <context id="module-handler" style-ref="module-handler"> <keyword>import</keyword> <keyword>from</keyword> <keyword>as</keyword> </context> this is an exact duplication of what is in python.lang and can be replaced with <context ref="python:module-handler /> In order to not duplicate all the context definitions that are shared but only the different ones.
I didn't notice any of that, fantastic. I think I have now duplicated as little as possible, you might want to have a look at these to check. I'll attach a diff for python.lang and a new python3.lang. By the way, and just in case you know, there's a <context id="format"> at the top of the file, which doesn't seem to do anything. Is that only because the styles don't implement it? I'm asking because there is a whole new formatting language in 3.x as well (the old formatting is deprecated but supported). Instead of ("%.2f" % s) you can now do ("{:.2f".format(s)). http://docs.python.org/release/3.1.3/library/string.html#format-string-syntax During editing these files, I've noticed that a lot of python.lang is still at Python version 2.5 (or something nearby). Is there a convention (lowest common subset?) as to what version of the language GtkSourceView implements? And thanks a million for your help so far.
Created attachment 205520 [details] [review] Patch for python.lang, separates objects to be removed in Python 3
Created attachment 205521 [details] New python3.lang (includes much of python.lang)
The patches look good from a cursory look (I have not tested yet). Thank you! Some more comments: - we should probably drop mime and globs for the time being otherwise you could randomly get py2 or py3 highlighting. We should pick a default behavior and for now I think 2 is still more common - the styles should probably map to the corrisponding python styles instead of the def ones, so that if in a style scheme you change the color of py strings it applies also to py3 - I am not sure I understand the problem with the "format" context, as far as I can see it is used inside strings and by default it uses the "character" color, which should be different from the string color so the format should get highlighted - with regard to being stuck at 2.5 it is only due to the fact that nobody submitted updates. Patches welcome! :-)
Created attachment 205523 [details] [review] Patch for python.lang, separates objects to be removed in Python 3, updates to 2.7
Created attachment 205524 [details] New python3.lang (includes much of python.lang, no globs and mimetypes)
The changes from 2.5 to >= 2.6 just meant adding two builtin functions, bin and format. So instead of opening a new bug I've just added these in the attached diff. Mimetypes and globs are gone too. It's worth considering adding a glob *.py3, because it's in common use. But it is non-standard and not endorsed by python's devs. Perhaps if and when someone else files a bug for that. If you've got a second, check that I haven't messed up any of the style references. Everything seems to be working great.
Sorry for the delay Stefano, I was away for some days. Pacthes look great to me, but since I am lazy I will ask for a final step :-) Would you mind attaching the patches created against gtksourceview master generating them with git format-patch? that way you will also appear as the author of the commit etc. See https://live.gnome.org/GnomeLove/SubmittingPatches You could also make some last nitpick changes: - in the py3 lang file I agree on adding the *py3 glob - for the py3 lang file please add it to Makefile.am and to po/POTFILES.in so that it will be installed and translated
I fixed up those things myself and committed. http://git.gnome.org/browse/gtksourceview/commit/?id=067f28a211661926a5dbd01aacae8153f1e62468 http://git.gnome.org/browse/gtksourceview/commit/?id=5dfc410de89977c7d9e53bf1aa096552f82d57b5 Thanks again for your contribution!
Heya, I was a bit busy, glad this was resolved so quickly. Thanks!