While I don't think this is codified anywhere, it seems that the more
common behavior for Search windows is to keep them open after the first
search is done and to make subsequent searches search onwards from the last
found position (see search in Gedit or Epiphany for example).

Ideally I think that the buttons in the search window should additionally
be changed to convey that fact (see Epiphany as opposed to Gedit for example).