GNOME Bugzilla – Bug 700410
Add support for the Julia language
Last modified: 2013-05-28 20:40:15 UTC
Created attachment 244353 [details] [review] Patch adding Julia support Julia is a relatively new language aimed for scientific computing, similar to Matlab and R, but with focus on performance while keeping the flexibility of a hight level, dynamic language. See http://julialang.org for more details.
Following our conversation at ask ubuntu, I know you tried to used matlab as template so I'll be short about the things you should have followed from there: 1) do list the styles being used in <styles> 2) do have the julia context at the end, with only references to other contexts, even when defined on the same file 3) for things where it looks like octave or matlab, do reference those files 4) why the very non-standard method you're using for the opeartors? Also, are the stuff inside types, really only types? There's things there that look like functions.
I actually wrote it completely from scratch, using the tutorial and the reference doc, because to be honest using existing files was a bit daunting (the spec is a little more complex than it looks at first sight). I only tried modifying the matlab file as a debugging method, because the Julia one didn't work right away, but I understand why I should have reused stuff from there. I'll work on that. I also just realized that I submitted an old, non-working version of the syntax file. I'll reupload with working syntax and will also take a look at other files to see what can be standardized about the operators. As for the types, they are indeed (afaik) only types. I used the list produced by https://github.com/JuliaLang/julia/blob/master/examples/typetree.jl (and cross-checked against other syntax highlighting files at https://github.com/JuliaLang/julia/tree/master/contrib). But to be sure, can you give some examples of what entries look like functions? ps - for reference, the AskUbuntu conversation mentioned is this one: http://askubuntu.com/questions/295559/how-to-add-a-new-language-definition-for-syntax-highlighting-in-gedit
When rewriting it based on other files, be careful which one you choose. Not all follow of them follow what is now recommended. You should be fine following Octave's and Matlab lang files. I don't know Julia, so if they say they're types I believe. But things such as Rest, Set, VersionNumber, Wait, or Zip sound a bit like functions. I'm not saying you're wrong, just wanted to point it out in case it was a mistake. And since you didn't have a list of Julia functions to be highlighted, I thought maybe you mixed them up and this was the list of functions.
Created attachment 244684 [details] [review] Patch adding Julia support Thanks for the remarks. I've reworked the .lang file, and this version should be more conforming to the current standards. Let me know if there are other changes that need to be made.
Regarding the types and functions: the new patch does add some builtin functions, but only a very limited set. The full list can be seen in the exports.jl file (https://github.com/JuliaLang/julia/blob/master/base/exports.jl), but since that list is still evolving, I though it would be a better idea to let them be added to julia.lang at a later point, when they stabilize. I included a TODO comment in the .lang file explaining this. Also, to confirm: the type list is indeed comprised of types only. This can be checked by running, for example, "typeof(Zip)", which returns "DataType". I actually made sure to clarify some entries that didn't match between the types listed in exports.jl and the output of typetree.jl: see https://github.com/JuliaLang/julia/issues/3132 and https://github.com/JuliaLang/julia/issues/3133)
Since I don't know the language, I can't comment on the exactness of the highlight but you as the Julia user are probably more apt to anyway. Some points: 1) why not including def:shebang? 2) why have a julia context for macro that only includes octave:function-handle, rather than just inlcuding it directly into the <context id="julia">? 3) should the im in numbers be highlighted as such? Or should it be as math-constant? When one does 5im, it actually means 5*im, multiply 5 by the constant im, not the number 5im. Would you agree with it? If so, then you could use the def:hexadecimal/decimal/etc contexts?
So I asked in #julia and they told me that 5im and 5pi should be highlighted the same way. With `im` and `pi` both as constants then.
1) Good question. I couldn't find a shebang in any of the .jl scripts neither in the development code branch nor in the release (or even the packaged .deb for Ubuntu 13.04), which does define a preset installation path. Then again, I don't see why shebangs would be incompatible, so I'll add the context. 2) That was partly for semantic reasons (so the id of the context and corresponding style had the proper name), and to allow a separate styling, since the "function-handle" style in octave maps to "def:type", while the Julia macros were made to map to "def:keyword" instead. I can add a comment explaining this to the file. Would that seem ok to you? 3) Yes, that makes sense. I based my number regex in the other highlighting files at https://github.com/JuliaLang/julia/tree/master/contrib, but having im being highlighted separately makes more sense and is simpler. I'll change this too. I'll submit an updated patch later today. In the meantime, if you happen to see this, please let me know your thoughts about point 2.
Btw (not that it matters, just for completeness sake), the only reference I found to the use of shebangs with Julia files was this: https://groups.google.com/forum/#!msg/julia-dev/7Kcu1Kbopmc/TArQKaMhadsJ
2) you should be able to change the style in the include of the main julia block. Something like this should work (untested): <context id="julia"> <include> <context ref="octave:function-handle" style-ref="macro"/>
One last thing. You may want to set classes for the different contexts. I think the only classes are: no-spell-check comment string
Created attachment 244879 [details] [review] Patch adding Julia support Ok, here is the patch with the suggested modifications. I couldn't use def.lang's contexts for numbers, since those would both fail to catch valid Julia numbers (such as 03 and 2pi), and catch strings that aren't considered numbers in Julia (such as 0Xff or 9L) As for the classes, I was a little unsure how to use them, since https://developer.gnome.org/gtksourceview/stable/lang-reference.html doesn't mention them at all (nor does the tutorial), but I tried to apply them following octave.lang's example. Can you confirm they are used correctly in this patch?
Created attachment 244880 [details] [review] Patch adding Julia support (actually attach the correct updated patch this time)
What's the backticks in Julia really for? The name external commands (and the fact that perl uses backticks for this), suggests me it's to make system calls. Is that correct? If so, I don't think you should disable the no-spell-check (or mark it as string). Using the def regexp for numbers, it should catch 03 (as octal). It won't catch them as decimal, I'm unsure if it that could be considered a bug there. And the reason it won't catch 2pi is that it defines a number as followed by \w or dot. But if we split this and use the negative lookahead in the context for the number rather than the definition for the regexp. Or if we use \b instead like Octave does. I'm unsure what's the best here. There's shouldn't be that many different syntax for numbers. Maybe you should find one that is like Juli'as and ref that one instead? How does it compare to Octave's for example?
I will add a reference to http://docs.julialang.org/en/latest/manual/running-external-programs/ to the "external command" context, to make it clear. I'll also remove the classes as you suggested. As for reusing contexts from other lang files, I found that several others contain definitions for backtick-delimited "strings" (d.lang, go.lang, perl.lang, php.lang, python.lang, ruby.lang and sh.lang), but those contexts either contain extra stuff, or are declared with the string class. I don't see how I can reuse any of these for Julia. As for the number formats, there doesn't seem to be any opportunity for reusing other languages' patterns. For octal, for instance, most language files ignore the "0o" prefix, and those that include it also support a capital "O", while in Julia only the lowercase "o" is valid syntax. Furthermore, several languages include the f or l suffixes, which don't exist in Julia. Similar limitations exist for the other number formats (it took me a while, but I did sift through all the current definitions!). Meanwhile, I did notice something I overlooked: While Julia supports having a number directly prepended to a variable or constant (e.g. "3x"), the reverse isn't true: a number suffixed to a name becomes part of that name. Thus, I updated the regex to properly handle this case.
Created attachment 244969 [details] [review] Patch adding Julia support Updated patch with changes mentioned in previous comment
Review of attachment 244969 [details] [review]: This patch looks good to me. I'd approve it.
Review of attachment 244969 [details] [review]: Looks good to me too, a couple of nitpicks below, once they are fixed feel free to push ::: data/language-specs/julia.lang @@ +25,3 @@ + + <metadata> + <property name="mimetypes" >text/x-julia;application/x-julia</property> I do not think these mime types are registered in shared-mime-info. In that case I'd prefer to just omit them, this metadata is not mandatory. @@ +40,3 @@ + <style id="number" _name="Number" map-to="def:number" /> + <style id="boolean" _name="Boolean" map-to="def:boolean" /> + <style id="math-constant" _name="Math constant" map-to="def:constant" /> these should probably be capitalized (Math Constant, etc)
I pushed the patch and then another commit with the suggested changes. See: https://git.gnome.org/browse/gtksourceview/commit/?id=b6e29a6ec2799d6d9f5b2723935f8f7bc3111116 https://git.gnome.org/browse/gtksourceview/commit/?id=baaa2f6cbb8825af4c821ff1684654e2f4bfdb60 This problem has been fixed in the development version. The fix will be available in the next major software release. Thank you for your bug report.
Review of attachment 244969 [details] [review]: pushed