After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 673034 - GBufferedInputStream, GBufferedOutputStream and GDataOutputStream should implement GSeekable
GBufferedInputStream, GBufferedOutputStream and GDataOutputStream should impl...
Status: RESOLVED FIXED
Product: glib
Classification: Platform
Component: gio
unspecified
Other Linux
: Normal enhancement
: ---
Assigned To: gtkdev
gtkdev
Depends on:
Blocks:
 
 
Reported: 2012-03-29 00:41 UTC by Maciej (Matthew) Piechotka
Modified: 2012-04-23 08:58 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
0001-Make-GBufferedInputStream-implement-GSeekable.patch (5.50 KB, patch)
2012-03-29 00:43 UTC, Maciej (Matthew) Piechotka
needs-work Details | Review
0002-Make-GBufferedOutputStream-implement-GSeekable.patch (5.84 KB, patch)
2012-03-29 00:43 UTC, Maciej (Matthew) Piechotka
needs-work Details | Review
0003-Make-GDataOutputStream-implement-GSeekable.patch (4.53 KB, patch)
2012-03-29 00:44 UTC, Maciej (Matthew) Piechotka
needs-work Details | Review
0001-Make-GBufferedInputStream-implement-GSeekable.patch (5.49 KB, patch)
2012-03-29 22:29 UTC, Maciej (Matthew) Piechotka
none Details | Review
0002-Make-GBufferedOutputStream-implement-GSeekable.patch (5.52 KB, patch)
2012-03-29 22:29 UTC, Maciej (Matthew) Piechotka
none Details | Review
0003-Make-GDataOutputStream-implement-GSeekable.patch (8.98 KB, patch)
2012-03-29 22:30 UTC, Maciej (Matthew) Piechotka
none Details | Review
0001-Make-GBufferedInputStream-implement-GSeekable.patch (9.49 KB, patch)
2012-04-10 05:44 UTC, Maciej (Matthew) Piechotka
none Details | Review
0002-Make-GBufferedOutputStream-implement-GSeekable.patch (7.13 KB, patch)
2012-04-10 05:44 UTC, Maciej (Matthew) Piechotka
none Details | Review
0003-Make-GDataOutputStream-implement-GSeekable.patch (12.87 KB, patch)
2012-04-10 05:45 UTC, Maciej (Matthew) Piechotka
none Details | Review
0001-Make-GBufferedInputStream-implement-GSeekable.patch (9.49 KB, patch)
2012-04-23 05:16 UTC, Maciej (Matthew) Piechotka
none Details | Review
0002-Make-GBufferedOutputStream-implement-GSeekable.patch (12.38 KB, patch)
2012-04-23 05:17 UTC, Maciej (Matthew) Piechotka
none Details | Review
0003-Make-GDataOutputStream-implement-GSeekable.patch (12.87 KB, patch)
2012-04-23 05:17 UTC, Maciej (Matthew) Piechotka
none Details | Review

Description Maciej (Matthew) Piechotka 2012-03-29 00:41:52 UTC
GBufferedInputStream, GBufferedOutputStream and GDataOutputStream should implement GSeekable as they provide thin layer over underlaying stream.
Comment 1 Maciej (Matthew) Piechotka 2012-03-29 00:43:06 UTC
Created attachment 210831 [details] [review]
0001-Make-GBufferedInputStream-implement-GSeekable.patch
Comment 2 Maciej (Matthew) Piechotka 2012-03-29 00:43:49 UTC
Created attachment 210832 [details] [review]
0002-Make-GBufferedOutputStream-implement-GSeekable.patch
Comment 3 Maciej (Matthew) Piechotka 2012-03-29 00:44:18 UTC
Created attachment 210833 [details] [review]
0003-Make-GDataOutputStream-implement-GSeekable.patch
Comment 4 Maciej (Matthew) Piechotka 2012-03-29 00:45:00 UTC
Those patches don't include tests (yet). However the feedback is welcome.
Comment 5 Dan Winship 2012-03-29 12:22:51 UTC
Review of attachment 210831 [details] [review]:

We don't have a machine-readable way to mark an interface implementation as being new in a particular release, but you should at least add something like "As of glib 2.34, #GBufferedInputStream implements #GSeekable" to the main docs at the top of the file.

::: gio/gbufferedinputstream.c
@@ +887,3 @@
+  base_offset = g_seekable_tell (base_stream_seekable);
+
+  return base_offset - available;

That's wrong; the GBufferedInputStream's position is the same as its underlying stream's position. What portion of the stream happens to be currently buffered is irrelevant.

@@ +922,3 @@
+      if (offset <= priv->end - priv->pos && offset >= -priv->pos)
+	{
+	  priv->pos += offset;

likewise, this (and the rest of g_buffered_input_stream_seek) is mixing up the global stream position with the position within the buffer.

The buffering should have no visible effect on tell/seek operations: a series of reads, skips, tells, and seeks on the GBufferedInputStream should return exactly the same results as doing it on its base_stream.
Comment 6 Dan Winship 2012-03-29 12:23:26 UTC
Review of attachment 210832 [details] [review]:

same issues
Comment 7 Dan Winship 2012-03-29 12:24:19 UTC
Review of attachment 210833 [details] [review]:

Why does GDataOutputStream need its own implementation? It ought to be able to just inherit GBufferedOuputStream's.
Comment 8 Maciej (Matthew) Piechotka 2012-03-29 15:44:26 UTC
(In reply to comment #5)
> Review of attachment 210831 [details] [review]:
> 
> We don't have a machine-readable way to mark an interface implementation as
> being new in a particular release, but you should at least add something like
> "As of glib 2.34, #GBufferedInputStream implements #GSeekable" to the main docs
> at the top of the file.
> 
> ::: gio/gbufferedinputstream.c
> @@ +887,3 @@
> +  base_offset = g_seekable_tell (base_stream_seekable);
> +
> +  return base_offset - available;
> 
> That's wrong; the GBufferedInputStream's position is the same as its underlying
> stream's position. What portion of the stream happens to be currently buffered
> is irrelevant.
> 

From what I understand it should return logical position of stream. For example if we have a buffer of size 4096, and reads of 2048:

+---------------+-------------------+--------------------+-------------------+
| After n reads | Base stream tell  | Position in buffer | Outer stream tell |
+---------------+-------------------+--------------------+-------------------+
|             0 |                 0 |                  0 |                 0 |
|             1 |              4096 |               2048 |              2048 |
|             2 |              4096 |               4096 |              4096 |
|             3 |              8192 |               2048 |              6144 |
|             4 |              8192 |               4096 |              8192 |
|           ... |               ... |                ... |               ... |
+---------------+-------------------+--------------------+-------------------+

Those position are not in sync - see http://git.gnome.org/browse/glib/tree/gio/gbufferedinputstream.c#n780 - if the data is in buffer then there is no call to underlaying stream (if there was what would be the point of buffering?).

> @@ +922,3 @@
> +      if (offset <= priv->end - priv->pos && offset >= -priv->pos)
> +    {
> +      priv->pos += offset;
> 
> likewise, this (and the rest of g_buffered_input_stream_seek) is mixing up the
> global stream position with the position within the buffer.
> 
> The buffering should have no visible effect on tell/seek operations: a series
> of reads, skips, tells, and seeks on the GBufferedInputStream should return
> exactly the same results as doing it on its base_stream.

Why? My believe was that GBufferedInputStream should be transparent in a sense that it should not be visible expect delay of some effects.

For example such code should pass (code is in pseudo-Vala):

var file_input = File.read ();
var input = new DataInputStream(file_input);
assert (input.tell () == 0);
input.read_int16 (NULL);
assert (input.tell () == 2);
// It is known that:
// 2 <= file_input.tell () <= input.buffer_size

EDIT: It seems that read_int16 reads only 2 elements but the generic read fills full buffer:

using GLib;

void main() {
	var file_input = File.new_for_path ("test.txt").read ();
	var input = new DataInputStream(file_input);
	stdout.printf("Offset: %d\n", (int)file_input.tell ());
	uint8[] buffer = new uint8[1024];
	ssize_t read = input.read (buffer);
	stdout.printf("Read: %d\n", (int)read);
	stdout.printf("Offset: %d\n", (int)file_input.tell ());
}

Output:
Offset: 0
Read: 1024
Offset: 4096

(In reply to comment #6)
> Review of attachment 210832 [details] [review]:
> 
> same issues

Same comments.

(In reply to comment #7)
> Review of attachment 210833 [details] [review]:
> 
> Why does GDataOutputStream need its own implementation? It ought to be able to
> just inherit GBufferedOuputStream's.

Well - GDataOuputStream does not inherit from GBufferedOutputStream.

In general the GSeekable might not be well-defined for GFilter{In,Out}putStream (conversions, buffering etc. would need to have special handling).
Comment 9 Alexander Larsson 2012-03-29 16:44:13 UTC
I agree with Maciej, the current position should return the "offset" where the next read operation starts returning data from, not what the underlying stream is positioned at.
Comment 10 Alexander Larsson 2012-03-29 17:07:57 UTC
Review of attachment 210832 [details] [review]:

::: gio/gbufferedoutputstream.c
@@ +600,3 @@
+	{
+	  memset (priv->buffer + priv->pos, 0, offset);
+	  priv->pos += offset;

This doesn't seem right. A seek ahead does not necessarily clear the skipped area (although that is what happens if you're at the end of a file *and* you then write something there). The stream might be replacing existing content, and the seek should not cause the skipped data to be cleared.

@@ +606,3 @@
+
+  flushed = flush_buffer (bstream, cancellable, error);
+  g_return_val_if_fail (flushed, FALSE);

This isn't right. g_return_val_if_fail should only be used to verify programmer assumtions. These checks get compiled out if you disable debug in your build.

@@ +637,3 @@
+
+  flushed = flush_buffer (bstream, cancellable, error);
+  g_return_val_if_fail (flushed, FALSE);

same here
Comment 11 Dan Winship 2012-03-29 17:17:31 UTC
sorry, i misread the patch. i thought it was always returning the current position within the internal buffer, completely ignoring the position in the underlying stream.
Comment 12 Maciej (Matthew) Piechotka 2012-03-29 18:34:04 UTC
(In reply to comment #10)
> Review of attachment 210832 [details] [review]:
> 
> ::: gio/gbufferedoutputstream.c
> @@ +600,3 @@
> +    {
> +      memset (priv->buffer + priv->pos, 0, offset);
> +      priv->pos += offset;
> 
> This doesn't seem right. A seek ahead does not necessarily clear the skipped
> area (although that is what happens if you're at the end of a file *and* you
> then write something there). The stream might be replacing existing content,
> and the seek should not cause the skipped data to be cleared.
> 

I see. Then I will flush the buffer explicitly. There might be a problem with seek backwards as not the whole area will be flushed.

> @@ +606,3 @@
> +
> +  flushed = flush_buffer (bstream, cancellable, error);
> +  g_return_val_if_fail (flushed, FALSE);
> 
> This isn't right. g_return_val_if_fail should only be used to verify programmer
> assumtions. These checks get compiled out if you disable debug in your build.
> 

That's what I mean by rusty C/GObject :(. I'll post fixes to it today (and nearly ready tests for GDataOutputStream).

Regards
Comment 13 Maciej (Matthew) Piechotka 2012-03-29 22:29:19 UTC
Created attachment 210904 [details] [review]
0001-Make-GBufferedInputStream-implement-GSeekable.patch
Comment 14 Maciej (Matthew) Piechotka 2012-03-29 22:29:52 UTC
Created attachment 210905 [details] [review]
0002-Make-GBufferedOutputStream-implement-GSeekable.patch
Comment 15 Maciej (Matthew) Piechotka 2012-03-29 22:30:36 UTC
Created attachment 210906 [details] [review]
0003-Make-GDataOutputStream-implement-GSeekable.patch
Comment 16 Maciej (Matthew) Piechotka 2012-03-29 22:33:03 UTC
Changes:

 - Remove g_return_val_if_fail when appropiate
 - Bug causing infinite recursion in GDataOutputStream.seek fixed
 - Add tests for seek (but not truncate) for GDataOutputStream
Comment 17 Alexander Larsson 2012-03-30 06:48:18 UTC
These all look good to me, but can't go in until we've branched, current master is still the stable 2.32 release.
Comment 18 Maciej (Matthew) Piechotka 2012-03-30 07:15:04 UTC
(In reply to comment #17)
> These all look good to me,

They are still not finished (tests may reveal some small bugs).

> but can't go in until we've branched, current master
> is still the stable 2.32 release.

That's what I expected.
Comment 19 Colin Walters 2012-03-30 12:49:22 UTC
(In reply to comment #18)
> (In reply to comment #17)
> > These all look good to me,
> 
> They are still not finished (tests may reveal some small bugs).

Yeah, these shouldn't go in without test cases.
Comment 20 Maciej (Matthew) Piechotka 2012-04-10 05:44:26 UTC
Created attachment 211692 [details] [review]
0001-Make-GBufferedInputStream-implement-GSeekable.patch
Comment 21 Maciej (Matthew) Piechotka 2012-04-10 05:44:52 UTC
Created attachment 211693 [details] [review]
0002-Make-GBufferedOutputStream-implement-GSeekable.patch
Comment 22 Maciej (Matthew) Piechotka 2012-04-10 05:45:41 UTC
Created attachment 211694 [details] [review]
0003-Make-GDataOutputStream-implement-GSeekable.patch
Comment 23 Maciej (Matthew) Piechotka 2012-04-10 05:46:35 UTC
Update of patches. Nearly all tests are added.
Comment 24 Maciej (Matthew) Piechotka 2012-04-23 05:16:28 UTC
Created attachment 212582 [details] [review]
0001-Make-GBufferedInputStream-implement-GSeekable.patch
Comment 25 Maciej (Matthew) Piechotka 2012-04-23 05:17:00 UTC
Created attachment 212583 [details] [review]
0002-Make-GBufferedOutputStream-implement-GSeekable.patch
Comment 26 Maciej (Matthew) Piechotka 2012-04-23 05:17:28 UTC
Created attachment 212584 [details] [review]
0003-Make-GDataOutputStream-implement-GSeekable.patch
Comment 27 Maciej (Matthew) Piechotka 2012-04-23 05:18:03 UTC
Should be OK.
Comment 28 Alexander Larsson 2012-04-23 08:23:15 UTC
Review of attachment 212582 [details] [review]:

::: gio/gbufferedinputstream.c
@@ +904,3 @@
+
+  base_stream = G_FILTER_INPUT_STREAM (seekable)->base_stream;
+  g_return_val_if_fail (G_IS_SEEKABLE (base_stream), 0);

Should not rely on g_return_val_if_fail here, also, should return G_IO_ERROR_NOT_SUPPORTED error.
Comment 29 Alexander Larsson 2012-04-23 08:24:26 UTC
Review of attachment 212583 [details] [review]:

::: gio/gbufferedoutputstream.c
@@ +542,3 @@
+
+  base_stream = G_FILTER_OUTPUT_STREAM (seekable)->base_stream;
+  g_return_val_if_fail (G_IS_SEEKABLE (base_stream), 0);

Shouldn't rely on g_return_val_if_fail here, must to a non-debug-only check.

@@ +573,3 @@
+
+  base_stream = G_FILTER_OUTPUT_STREAM (seekable)->base_stream;
+  g_return_val_if_fail (G_IS_SEEKABLE (base_stream), FALSE);

Same here. Also, should return G_IO_ERROR_NOT_SUPPORTED error.

@@ +604,3 @@
+  bstream = G_BUFFERED_OUTPUT_STREAM (seekable);
+  base_stream = G_FILTER_OUTPUT_STREAM (seekable)->base_stream;
+  g_return_val_if_fail (G_IS_SEEKABLE (base_stream), FALSE);

Same here.
Comment 30 Alexander Larsson 2012-04-23 08:27:36 UTC
Review of attachment 212584 [details] [review]:

::: gio/gdataoutputstream.c
@@ +562,3 @@
+
+  base_stream = G_FILTER_OUTPUT_STREAM (seekable)->base_stream;
+  g_return_val_if_fail (G_IS_SEEKABLE (base_stream), FALSE);

same here

@@ +586,3 @@
+
+  base_stream = G_FILTER_OUTPUT_STREAM (seekable)->base_stream;
+  g_return_val_if_fail (G_IS_SEEKABLE (base_stream), FALSE);

same here.
Comment 31 Alexander Larsson 2012-04-23 08:51:05 UTC
Review of attachment 212583 [details] [review]:

::: gio/tests/buffered-output-stream.c
@@ +237,3 @@
+  const gchar buffer[] = "abcdefghijklmnopqrstuvwxyz";
+
+  len = 8;

Len is undefined

@@ +243,3 @@
+  base_stream = G_MEMORY_OUTPUT_STREAM (g_memory_output_stream_new (stream_data, len, g_realloc, g_free));
+  stream = g_buffered_output_stream_new_sized (G_OUTPUT_STREAM (base_stream), 8);
+  seekable = G_SEEKABLE (stream);

No actual test here

@@ +256,3 @@
   g_test_add_func ("/buffered-output-stream/grow", test_grow);
+  g_test_add_func ("/buffered-output-stream/seek", test_seek);
+  g_test_add_func ("/buffered-output-stream/seek", test_truncate);

Should be "/buffered-output-stream/truncate" here
Comment 32 Alexander Larsson 2012-04-23 08:58:16 UTC
Pushed patches with above fixes to master.