After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 506594 - [interleave] doesn't work with channels > 1
[interleave] doesn't work with channels > 1
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
git master
Other All
: Normal normal
: 0.10.8
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2007-12-31 16:43 UTC by Petri Rokka
Modified: 2008-05-26 10:28 UTC
See Also:
GNOME target: ---
GNOME version: 2.19/2.20



Description Petri Rokka 2007-12-31 16:43:45 UTC
Steps to reproduce:
1. run the appended program 

Stack trace:
  • #0 PyFrame_New
    at ../Objects/frameobject.c line 559
  • #1 PyEval_EvalCodeEx
    at ../Python/ceval.c line 2626
  • #2 function_call
    at ../Objects/funcobject.c line 517
  • #3 PyObject_Call
    at ../Objects/abstract.c line 1860
  • #4 instancemethod_call
    at ../Objects/classobject.c line 2509
  • #5 PyObject_Call
    at ../Objects/abstract.c line 1860
  • #6 PyEval_CallObjectWithKeywords
    at ../Python/ceval.c line 3433
  • #7 PyObject_CallObject
    at ../Objects/abstract.c line 1851
  • #8 ??
    from /var/lib/python-support/python2.5/gtk-2.0/gobject/_gobject.so
  • #9 ??
  • #10 ??
  • #11 ??
  • #12 ??
  • #13 ??
  • #14 ??


Other information:
import sys, os, os.path, time
import gobject
import pygst
pygst.require("0.10")
import gst

class InitError(Exception): pass
class MusicPlayerError(Exception): pass
class AudioControlError(Exception): pass

class GStreamerPlayer:
	## \brief Basic constructor
	##
	## Constructor initializes the class.
	def __init__(self):
		self._player = None
		self._songDict = {'title': '', 'artist': ''}
	
	## \brief create GST player bin
	def _createPlayerBin(self):
		try:
			self._createPlayerBinCore()
		except Exception, e:
			print e
		
		
	def _createPlayerBinCore(self):
		print "*** AUDIOSLAVE: GStreamerPlayer._createPlayerBin() INITIALIZING PLAYER BIN"
		
		self._numChannels = 2
		
		self._player = gst.parse_launch("gnomevfssrc name=filesrc ! decodebin name=decoder ! audioconvert ! deinterleave name=deinterleave")
		 
		interleave = gst.element_factory_make("interleave", "interleave")
		self._player.add(interleave)
		
		audioconv = gst.element_factory_make("audioconvert", "audioconvert")
		self._player.add(audioconv)
		
		audiosink = gst.element_factory_make("alsasink", "alsasink")
		self._player.add(audiosink)
		
		gst.element_link_many( interleave, audioconv, audiosink )
		
		self.deinterleave = self._player.get_by_name("deinterleave")
		self.deinterleave.connect("pad-added", self._deinterleave_pad_added)
		
		channels = ['left','right']	
		multiplier = 1
		
		self.interleave_sink = interleave_sink = {}
		self.tee = tee = {}
		self.queue = queue = {}
		self.adder = adder = {}
		
		for d in [interleave_sink, tee, adder]:
			for ch in channels:
				d[ch] = None
		
		for ch in channels:
			
			print 'configuring %s channel..' % ch
			
			interleave_sink[ch] = interleave.get_request_pad( 'sink%d' % channels.index(ch) )
			
			queue[ch] = gst.element_factory_make("queue", "%s_queue" % ch)
			assert( queue[ch] != None )
			self._player.add( queue[ch] )
				
			tee[ch] = gst.element_factory_make("tee", "%s_tee" % ch)
			assert( tee[ch] != None )
			self._player.add( tee[ch] )
			
			gst.element_link_many( queue[ch], tee[ch])
			
			adder[ch] = gst.element_factory_make("adder", "%s_adder" % ch)
			assert( adder[ch] != None )
			self._player.add( adder[ch] )
			
			adder[ch].get_pad('src').link( interleave_sink[ch] )
		
			for i in range(multiplier):
					
				print 'configuring multipath %d..' % i
				
				src = tee[ch].get_request_pad('src%d' % i)
				assert( src != None )
							
				print src
				
				queueM = gst.element_factory_make("queue", "%s_queue%d" % (ch,i))
				assert( queueM != None )
				self._player.add( queueM )
				queueM_sink = queueM.get_pad('sink')
				queueM_src = queueM.get_pad('src')
					
				convert = gst.element_factory_make("audioconvert", "%s_audioconvert%d" % (ch,i) )
				self._player.add(convert)		
				convert_sink = convert.get_pad('sink')
				convert_src = convert.get_pad('src')
								
				src.link( queueM_sink )
				queueM_src.link( convert_sink )
				
				last_src = convert_src

				adder_sink = adder[ch].get_request_pad('sink%d' % i)
				last_src.link( adder_sink )
		
		
		
		#ladspa-buttlow-iir name=ladspa-buttlow-iir ! ladspa-butthigh-iir name=ladspa-butthigh-iir ! ladspa-delay-5s name=ladspa-delay-5s Delay=0.0 Dry-Wet-Balance=1.0 ! audioamplify name=audioamplify
		
		# END SUPER DUPER
		
		
		#self._player = gst.parse_launch("gnomevfssrc name=filesrc ! decodebin name=decoder ! audioconvert ! ladspa-buttlow-iir name=ladspa-buttlow-iir ! ladspa-butthigh-iir name=ladspa-butthigh-iir ! ladspa-delay-5s name=ladspa-delay-5s Delay=0.0 Dry-Wet-Balance=1.0 ! audioamplify name=audioamplify ! audioconvert ! alsasink")
			
			
			
		#self._player = gst.parse_launch("gnomevfssrc name=filesrc ! decodebin name=decoder ! audioconvert ! alsasink")
		
	def _deinterleave_pad_added(self, element, pad):
		number = int( pad.get_name()[3:] )
		print 'channel%d' % number ,
		
		if number == 0:
			sink = self.queue['left'].get_pad('sink')
			pad.link( sink )
			print ' linked to %s' % sink
			
		if number == 1:
			sink = self.queue['right'].get_pad('sink')
			pad.link( sink )
			print ' linked to %s' % sink
			
		#if number == 1:
		#	print ' linked to fakesink'
		#	pad.link( self.fakesink)
		#	pad.link( sink )
	
	
		
	## \brief Uninitialize GStreamer
	def _uninitialize(self):
		print "*** AUDIOSLAVE: GStreamerPlayer._uninitialize() called"
		try:
			if self._player != None:
				print "GStreamerPlayer._uninitialize() UNINITIALIZING"
				self._player.set_state(gst.STATE_NULL)
				self._player = None
		except Exception, e:
			raise InitError, 'Could not uninitialize GStreamer'
	
	## \brief Initialize GStreamer
	def _initialize(self):
		
		print "*** AUDIOSLAVE: GStreamerPlayer._initialize() called"
		
		try:
			self._uninitialize()
			
			
			
			self._createPlayerBin()	
			
			bus = self._player.get_bus()
			bus.add_signal_watch()
			bus.connect('message', self._on_bus_message)
			
			self._gst_time_format = gst.Format(gst.FORMAT_TIME)
		except Exception, e:
			raise InitError, 'Could not initialize GStreamer'
	
	## \brief Handle GStreamer bus messages
	def _on_bus_message(self, bus, message):
		type = message.type
		if type == gst.MESSAGE_EOS:
			self._player.set_state(gst.STATE_NULL)
		elif type == gst.MESSAGE_ERROR:
			self._player.set_state(gst.STATE_NULL)
		elif type == gst.MESSAGE_TAG:
			tagList = message.parse_tag()
			for tag in tagList.keys():
				self._songDict[tag] = tagList[tag]
			
				
	def _seekTo(self, pos):
		try:
			#self._player.set_state(gst.STATE_PAUSED)
			self._player.seek_simple(self._gst_time_format, gst.SEEK_FLAG_FLUSH, pos * 1000000000)
			#self._player.set_state(gst.STATE_PLAYING)
		except Exception, e:
			raise MusicPlayerError, 'Could not seekTo %d' % pos
		
	
	def _convert_ns(self, time_int):
		time_int = time_int / 1000000000
		time_str = ""
		if time_int >= 3600:
			_hours = time_int / 3600
			time_int = time_int - (_hours * 3600)
			time_str = str(_hours) + ":"
		if time_int >= 600:
			_mins = time_int / 60
			time_int = time_int - (_mins * 60)
			time_str = time_str + str(_mins) + ":"
		elif time_int >= 60:
			_mins = time_int / 60
			time_int = time_int - (_mins * 60)
			time_str = time_str + "0" + str(_mins) + ":"
		else:
			time_str = time_str + "00:"
		if time_int > 9:
			time_str = time_str + str(time_int)
		else:
			time_str = time_str + "0" + str(time_int)
			
		return time_str

	
	## \brief Load music file from URI
	def loadURI(self, uri):
		
		print "*** AUDIOSLAVE: GStreamerPlayer.loadURI('%s') called" % uri
		
		self._initialize()
		
		try:
			print "*** AUDIOSLAVE: GStreamerPlayer.loadURI('%s') LOADING URI" % uri
			self._player.get_by_name('filesrc').set_property('location', uri)
			#self._player.set_property('location', uri)
			self._player.set_state(gst.STATE_PAUSED)
		except Exception, e:
			# TODO: save exception to error log?
			raise MusicPlayerError, "Could not load song from uri: %s" % uri
	
	
	## \brief Unload music file
	def unload(self):
		print '*** AUDIOSLAVE: GStreamerPlayer.unload called'
		try:
			self._player.set_state(gst.STATE_NULL)
		except Exception, e:
			# TODO: save exception to error log?
			raise MusicPlayerError, "Could not unload the song"
		
	
	## \brief Play music or pause if playing or unpause if paused
	def play(self):
		
		print '*** AUDIOSLAVE: GStreamerPlayer.play() called'
		
		self.seekTo(0)
		
		try:
			states = self._player.get_state()
						
			found = False
			for state in states:
				print 'state:', state
				if gst.State(state) == gst.STATE_PLAYING:
					print 'GStreamerPlayer.play: CHANGING TO PAUSED'
					self._player.set_state(gst.STATE_PAUSED)	
					found = True
					
			if not found:
				print 'GStreamerPlayer.play: CHANGING TO PLAYING'
				self._player.set_state(gst.STATE_PLAYING)
				
		except Exception, e:
			# TODO: save exception to error log?
			
			print e
			
			raise MusicPlayerError, "Could not start playback of the song"
		
	
	## \brief Stop music
	def stop(self):
		print '*** AUDIOSLAVE: GStreamerPlayer.stop called'
		try:
			self._player.set_state(gst.STATE_PAUSED)
		except Exception, e:
			# TODO: save exception to error log?
			
			raise MusicPlayerError, "Could not pause the song"
		
		self.seekTo(0)

	
	
	## \brief Pause music
	def pause(self):
		print '*** AUDIOSLAVE: GStreamerPlayer.pause called'
		try:
			self._player.set_state(gst.STATE_PAUSED)
		except Exception, e:
			# TODO: save exception to error log?
			
			raise MusicPlayerError, "Could not stop the song"

	
	
	## \brief Unpause music
	def unpause(self):
		print '*** AUDIOSLAVE: GStreamerPlayer.unpause called'
		try:
			self._player.set_state(gst.STATE_PLAYING)
		except Exception, e:
			# TODO: save exception to error log?
			
			raise MusicPlayerError, "Could not unpause the song"

	
	
	## \brief Seek to position
	def seekTo(self, pos):
		print '*** AUDIOSLAVE: GStreamerPlayer.seekTo(%d) called' % pos
		self._seekTo(pos)
	
	## \brief Get output audio channel count
	def getAudioChannelCount(self):
		print '*** AUDIOSLAVE: GStreamerPlayer.getAudioChannelCount() called'
		return 0
	
	## \brief Get equalizer band count
	def getEqualizerBandCount(self):
		print '*** AUDIOSLAVE: GStreamerPlayer.getEqualizerBandCount() called'
		return 0
	
	## \brief Get equalizer band frequency
	def getEqualizerBandFrequency(self):
		print '*** AUDIOSLAVE: GStreamerPlayer.getEqualizerBandFrequency() called'
		return 0
	
	## \brief Get music position
	def getMusicPosition(self):
		print '*** AUDIOSLAVE: GStreamerPlayer.getMusicPosition() called'
		try:
			positionNs = self._player.query_position(self._gst_time_format, None)[0]
			
			if positionNs == None:
				raise Exception, 'Got nothing'
			
			if positionNs != 0:
				return positionNs / 1000000000
			else:
				return 0
		except Exception, e:
			raise MusicPlayerError, 'Could not get music position'
	
	
	## \brief Get music length
	def getMusicLength(self):
		print '*** AUDIOSLAVE: GStreamerPlayer.getMusicLength() called'
		try:
			durationNs = self._player.query_duration(self._gst_time_format, None)[0]
			
			if durationNs == None:
				raise Exception, 'Got nothing'
			
			if durationNs != 0:
				return durationNs  / 1000000000
			else:
				return 0
		except Exception, e:
			raise MusicPlayerError, 'Could not get music length'
	
	## \brief Set channel volume gain
	def setChannelVolumeGain(self, channel, gain):
		print '*** AUDIOSLAVE: GStreamerPlayer.setChannelVolumeGain(%d, %f) called' % (channel, gain)
		try:
			self._player.get_by_name('audioamplify').set_property('amplification', gain)
		except:
			raise AudioControlError, 'Could not set lowpass frequency'
	
	
	## \brief Set channel lowpass filter frequency
	def setChannelLowPassFrequency(self, channel, frequency):
		print '*** AUDIOSLAVE: GStreamerPlayer.setChannelLowPassFrequency(%d, %f) called' % (channel, frequency)
		try:
			self._player.get_by_name('ladspa-buttlow-iir').set_property('Cutoff-Frequency', frequency)
		except:
			raise AudioControlError, 'Could not set lowpass frequency'
		
	
	
	## \brief Set channel highpass filter frequency
	def setChannelHighPassFrequency(self, channel, frequency):
		print '*** AUDIOSLAVE: GStreamerPlayer.setChannelHighPassFrequency(%d, %f) called' % (channel, frequency)
		try:
			self._player.get_by_name('ladspa-butthigh-iir').set_property('Cutoff-Frequency', frequency)
		except:
			raise AudioControlError, 'Could not set highpass frequency'
	
	
	## \brief Set channel delay filter length
	def setChannelDelayLength(self, channel, length):
		print '*** AUDIOSLAVE: GStreamerPlayer.setChannelDelayLength(%d, %f) called' % (channel, length)
		
		try:
			self._player.get_by_name('ladspa-delay-5s').set_property('Delay', length)
		except Exception, e:
			print e
			raise AudioControlError, 'Could not set delay length'
	
	## \brief Set channel lowpass filter state
	def setChannelLowPassState(self, channel, state):
		print '*** AUDIOSLAVE: GStreamerPlayer.setChannelLowPassState(%d, %d) called' % (channel, state)


	## \brief Set channel highpass filter state
	def setChannelHighPassState(self, channel, state):
		print '*** AUDIOSLAVE: GStreamerPlayer.setChannelHighPassState(%d, %d) called' % (channel, state)
	
	
	## \brief Set channel delay filter state
	def setChannelDelayState(self, channel, state):
		pass
	
	
	## \brief Set equalizer band gain
	def setEqualizerBandGain(self, band, gain):
		print '*** AUDIOSLAVE: GStreamerPlayer.setEqualizerBandGain(%d, %f) called' % (band, gain)
	
	
## running the program
#
# gstreamerPlayer = GStreamerPlayer()
# gstreamerPlayer.loadURI('file:///song.mp3')
# gstreamerPlayer.play()
#
Comment 1 Edward Hervey 2008-01-02 11:45:26 UTC
Add "gobject.threads_init()" just below "import gobject" and try again.
Comment 2 Petri Rokka 2008-01-02 12:04:12 UTC
Does not crash with the latest gstreamer  and py-gst CVS versions after adding  gobject.threads_init() to the begining.
Comment 3 Petri Rokka 2008-01-04 01:39:58 UTC
It still doesn't work.

The code tries to split a stereo stream from the music file to 2 mono streams with 
'deinterleave'-plugin and the further copy again those two mono channels with 'tee'-plugin  'multiplier' times. 

Then i would make operations on all invididual mono streams and finally add them together with 'adder'-plugin (or without) and finally make all back to a multichannel stream to output with alsa.

With one channel it does work.

With one than more channel in the channels list it just doesn't work. It doesn't crash but i don't hear a sound or get an indication of it working other ways either.

You should remove comments of those last rows of the code (gstreamerPlayer*) and try different amount of channels in the 'channels'-list and different values to the 'multiplier'-parameter.
Comment 4 Tim-Philipp Müller 2008-01-04 15:29:26 UTC
Example pipeline:

  gst-launch-0.10 -v interleave name=i ! fakesink   audiotestsrc ! audioconvert ! audio/x-raw-float,channels=2 ! deinterleave name=d  d. ! queue ! i.  d. ! queue ! i. 

will lead to interleave returning FLOW_ERROR due to

  interleave.c:501:gst_interleave_pen_buffer: Pad i:sink0 already has penned buffer

(which is due to interleave being fundamentally broken, at least in push mode; it should use GstCollectPads or GstMuxer or so).

I'm marking this as a blocker, because if this is not fixed in time for the release the element should be disabled for the release.
Comment 5 Jan Schmidt 2008-02-07 01:32:53 UTC
seems unlikely this is going to get fixed in time unless someone steps up in the next day...
Comment 6 Jan Schmidt 2008-02-08 03:39:48 UTC
Also, for the record, the pipeline in comment #4 is not the way interleave is (currently) intended to be used. It's expecting to be connected directly to the deinterleave like this:

 gst-launch-0.10 -v interleave name=i ! fakesink   audiotestsrc ! audioconvert ! audio/x-raw-float,channels=2 ! deinterleave name=d  d. ! i.  d. ! i. 

It's the queues that are busting it.
Comment 7 Tim-Philipp Müller 2008-02-08 09:27:25 UTC
> Also, for the record, the pipeline in comment #4 is not the way interleave is
> (currently) intended to be used. It's expecting to be connected directly to the
> deinterleave like this ... it's the queues that are busting it.

That's just not really particularly useful though, is it? Besides being different from all other muxer-type elements we have (adder, videomixer, muxers etc.), it limits the sort of effects you can plug a lot, since it will only work if you get equal-sized buffers on each stream at any time. And if you actually have a deinterleave upstream. "Only works in very special circumstances if you're really lucky" just isn't good enough IMO. I still consider this broken.
Comment 8 Jan Schmidt 2008-02-08 09:56:34 UTC
I *said* "for the record" :)

It's like "no offense, but... <something really offensive>" - one of those things one can say to get off scot-free
Comment 9 Jan Schmidt 2008-02-08 23:47:28 UTC
Actually, why do you think this is a blocker?

The element is already in -bad, where we make no guarantees about the quality, and it *does* have a case where it works, just a very specific one.
Comment 10 Jan Schmidt 2008-02-09 00:08:08 UTC
I've decided to leave the element active. Even though it doesn't work well, there is at least one scenario where it does function, and that's about all that's expected of an element that's in -bad.
Comment 11 Sebastian Dröge (slomo) 2008-05-06 10:15:15 UTC
I'll take a closer look at deinterleave and interleave in the next days... seems that interleave needs more or less a complete rewrite ;)
Comment 12 Petri Rokka 2008-05-06 10:30:37 UTC
I have been waiting for this to happen :-). Thank you.
Comment 13 Sebastian Dröge (slomo) 2008-05-26 10:28:41 UTC
Ok, this should be fixed in CVS now with the following change:

2008-05-26  Sebastian Dröge  <slomo@circular-chaos.org>

        * gst/interleave/deinterleave.c:
        Add another example launch line.

        * gst/interleave/interleave.c: (interleave_24),
        (gst_interleave_finalize), (gst_interleave_base_init),
        (gst_interleave_class_init), (gst_interleave_init),
        (gst_interleave_request_new_pad), (gst_interleave_release_pad),
        (gst_interleave_change_state), (__remove_channels),
        (__set_channels), (gst_interleave_sink_getcaps),
        (gst_interleave_set_process_function),
        (gst_interleave_sink_setcaps), (gst_interleave_sink_event),
        (gst_interleave_src_query_duration), (gst_interleave_src_query),
        (forward_event_func), (forward_event), (gst_interleave_src_event),
        (gst_interleave_collected):
        * gst/interleave/interleave.h:
        Major rewrite of interleave using GstCollectpads. This new version
        also supports almost all raw audio formats and has better caps
        negotiation. Fixes bug #506594.

        Also update docs and add some more examples.

        * tests/check/elements/interleave.c: (interleave_chain_func),
        (GST_START_TEST), (src_handoff_float32), (sink_handoff_float32),
        (interleave_suite):
        Add some more extensive unit tests for interleave.