After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 745045 - Automatic unit tests
Automatic unit tests
Status: RESOLVED OBSOLETE
Product: vte
Classification: Core
Component: general
unspecified
Other All
: Normal enhancement
: ---
Assigned To: VTE Maintainers
VTE Maintainers
Depends on: vteparser
Blocks:
 
 
Reported: 2015-02-23 19:09 UTC by George Nachman
Modified: 2021-06-10 15:00 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description George Nachman 2015-02-23 19:09:31 UTC
This is somewhat related to bug 4993.

I got tired of trying to figure out why iTerm2 (the terminal emulator I maintain) wasn't passing VTTest, so I wrote my own set of unit tests which are much more fine-grained and don't require user interaction to run. The tests found a number of bugs in xterm, and I believe provide significant value beyond what VTTest provides, although it's not a complete replacement.

The esctest module may be found here:
https://github.com/gnachman/iTerm2/tree/master/tests/esctest

In order to get it up and running for VTE, you'll need to add support for DECRQCRA. Then run the test with --stop-on-failure and see how far you get. When it dies, add an annotation like:

@knownBug(terminal="vte", reason="Why it failed")

Then run it again. Once all the known bugs are documented, it will run to completion. When changes to vte are made, you can re-run the tests to verify that no new bugs were introduced.
Comment 1 Christian Persch 2018-03-04 22:55:14 UTC
I've implemented DECRQCRA and a test mode for the vte-2.91 test app (since DECRQCRA isn't desirable to be enabled by default in a production environment). 

To get esctest to run I've had to make surprisingly few changes, however I need to run vte-2.91 with --encoding="ISO-8859-1" (and --test-mode) to get the 8-bit tests to pass (not enough python knowledge here to change esctest to process utf-8 instead of latin1).

As you note, DECRQCRA is quite limited in that (if you pass a non-1x1 rect) it cannot detect if the character order is correct, and it can never detect character attributes.  How about we define our own RQCRA (perhaps CSI ? P...P * y) with a checksum algo that can do that?
Comment 2 George Nachman 2018-03-05 18:32:36 UTC
Yay, someone is using esctest! :)

> To get esctest to run I've had to make surprisingly few changes, however I need to run vte-2.91 with --encoding="ISO-8859-1" (and --test-mode) to get the 8-bit tests to pass (not enough python knowledge here to change esctest to process utf-8 instead of latin1).

I imagine all the failures have to do with 8-bit controls (for CSI, etc.). I don't imagine they stand a chance of working in a UTF-8 environment. I suggest adding annotations like this:

```
@optionRequired("vte", option=escargs.VTE_LATIN_1, allowPassWithoutOption=True)
```

You'll need to define `VTE_LATIN_1` at the top of escargs.py and to the list of accepted options in this line:

```
parser.add_argument("--options",
                    help="Space-separated options that are enabled.",
                    nargs="+",
                    choices=[ XTERM_WINOPS_ENABLED, DISABLE_WIDE_CHARS ])
```


> As you note, DECRQCRA is quite limited in that (if you pass a non-1x1 rect) it cannot detect if the character order is correct, and it can never detect character attributes.  How about we define our own RQCRA (perhaps CSI ? P...P * y) with a checksum algo that can do that?

I believe esctest ends up doing only 1x1 queries so it can pinpoint where there's a problem, so that ends up not being a big deal except for speed—but the speed is fine.

I would love a way to query for attributes. For reference, DECRQCRA is:

```
CSI Pid ; Pp; Pt;Pl;Pb;Pr * y
```

Pid is an identifier (useful for matching requests and responses) which we should keep. Pp is no longer sensible. So I would propose:

```
CSI ? Pid ; Pt;Pl;Pb;Pr * y
```

DECRQCRA's response looks like:

```
DCS Pid ! ~ D..D ST
```

Let's follow their lead and respond with:

```
DCS ? Pid ! ~ D..D ST
```

I'm not sure there's a good precedent for this response, but adding a ? ought to work.

At a minimum it would be nice to get an SGR-style number out. So `D..D` could contain a semicolon-delimited list of values. For SGR codes, they'd be like `1m` (bold). Other attributes would need to be decided on, like 24-bit colors from OSC 4. Not sure what the best course is to take so they can be represented in a way that doesn't conflict with other attributes we might like to add later.
Comment 3 Christian Persch 2018-08-20 18:09:39 UTC
xterm 334 has a new REPORTSGR function (CSI Pt;Pl;Pb;Pr # |) that reports SGR attributes that are common to all cells in the rectangular area. Maybe that's enough? (Not sure what it does with colours, esp. 24-bit colours.)
Comment 4 George Nachman 2018-08-21 06:35:29 UTC
I've added support for XTREPORTSGR to esctest in this commit:

https://github.com/gnachman/esctest/commit/74aa865452b7d249a6e0429a51e5927f21496d6c

Someone who has more free time than me is invited to write some tests in sgr.py. I put a basic test for bold in.

iTerm2 supports 24-bit color using the 38:2:1:r:g:b and 48:2:1:r:g:b SGR codes.
Comment 5 GNOME Infrastructure Team 2021-06-10 15:00:20 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to GNOME's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/vte/-/issues/2171.