Timeline handling sounds

[m1gu3l]

Hi everybody. I've written a small class extending TimelineLite, so that I could add Sound objects to tween queue.

 

package pl.m1gu3l.gs 
{
import com.greensock.TimelineLite;
import com.greensock.TweenLite;
import flash.media.Sound;
import flash.media.SoundChannel;

/**
 * ...
 * @author michal.przybys
 */

public class SoundTimeline extends TimelineLite {

	private var currentSound:Sound;
	private var currentChannel:SoundChannel;
	private var pausedPosition:Number;

	/**
	 * @param vars optionally pass in special properties like useFrames, onComplete, onCompleteParams, onUpdate, onUpdateParams, 
	 * onStart, onStartParams, tweens, align, stagger, delay, reversed, and/or autoRemoveChildren.
	 */
	public function SoundTimeline(vars:Object = null) {
		if (vars == null) vars = { paused: true }
		super(vars);
	}

	/**
	 * Converts preloaded Sound to TweenLite and appends it to timeline
	 * 
	 * @param preloadedSound preloaded Sound
	 * @param onComplete Function to be fired after Sound completes
	 * @param onCompleteParams an Array of parameters for onComplete function
	 */
	public function appendSound(preloadedSound:Sound, onComplete:Function = null, onCompleteParams:Array = null):void {
		this.append(soundToTween(preloadedSound, onComplete, onCompleteParams));
	}

	/**
	 * Converts preloaded Sound to TweenLite and prepends it to timeline
	 * 
	 * @param preloadedSound preloaded Sound
	 * @param onComplete Function to be fired after Sound completes
	 * @param onCompleteParams an Array of parameters for onComplete function
	 */
	public function prependSound(preloadedSound:Sound, onComplete:Function = null, onCompleteParams:Array = null):void {
		this.prepend(soundToTween(preloadedSound, onComplete, onCompleteParams));
	}

	/**
	 * Converts an Array of preloaded Sound objects to TweenLite and appends it to timeline
	 * 
	 * @param preloadedSound an Array of preloaded Sound objects
	 * @param onComplete Function to be fired after last of passed Sound objects completes
	 * @param onCompleteParams an Array of parameters for onComplete function
	 */
	public function appendMultipleSounds(preloadedSounds:Array, onComplete:Function = null, onCompleteParams:Array = null):void {
		for (var i:int = 0; i < preloadedSounds.length ; i++) {
			var tween:TweenLite = (i == preloadedSounds.length - 1) ? soundToTween(preloadedSounds[i], onComplete, onCompleteParams) : soundToTween(preloadedSounds[i]);
			this.append(tween);
		}
	}

	/**
	 * Converts an Array of preloaded Sound objects to TweenLite and prepends it to timeline
	 * 
	 * @param preloadedSound an Array of preloaded Sound objects
	 * @param onComplete Function to be fired after last of passed Sound objects completes
	 * @param onCompleteParams an Array of parameters for onComplete function
	 */
	public function prependMultipleSounds(preloadedSounds:Array, onComplete:Function = null, onCompleteParams:Array = null):void {
		for (var i:int = preloadedSounds.length - 1; i >= 0; i--) {
			var tween:TweenLite = (i == 0) ? soundToTween(preloadedSounds[i], onComplete, onCompleteParams) : soundToTween(preloadedSounds[i]);
			this.prepend(tween);
		}
	}

	/**
	 * Converts Function to TweenLite and appends it to timeline
	 * 
	 * @param f Function to be fired
	 * @param params an Array of parameters for passed function
	 */
	public function appendFunction(f:Function, params:Array = null):void {
		this.append(TweenLite.delayedCall(0, f, params));
	}

	/**
	 * Converts Function to TweenLite and prepends it to timeline
	 * 
	 * @param f Function to be fired
	 * @param params an Array of parameters for passed function
	 */
	public function prependFunction(f:Function, params:Array = null):void {
		this.prepend(TweenLite.delayedCall(0, f, params));
	}

	override public function stop():void {
		super.stop();
		if (currentChannel != null) currentChannel.stop();
		currentChannel = null;
		currentSound = null;
	}

	override public function pause():void {
		super.pause();
		if (currentChannel != null) {
			pausedPosition = currentChannel.position;
			currentChannel.stop();
		}
	}

	override public function resume():void {
		super.resume();
		if (currentSound != null) currentChannel = currentSound.play(pausedPosition);
	}

	/**
	 * Converts preloaded Sound to TweenLite and returns it
	 * 
	 * @param preloadedSound preloaded Sound
	 * @param onComplete Function to be fired after Sound completes
	 * @param onCompleteParams an Array of parameters for onComplete function
	 * @return com.greensock.TweenLite
	 */
	public function soundToTween(preloadedSound:Sound, onComplete:Function = null, onCompleteParams:Array = null):TweenLite {
		if (preloadedSound == null) throw new Error('sound passed is null');
		if (preloadedSound.bytesLoaded < preloadedSound.bytesTotal || preloadedSound.bytesTotal == 0) throw new Error('sound not preloaded');

		return new TweenLite(this, preloadedSound.length / 1000, {
			onStart: function() {
				currentSound = preloadedSound; currentChannel = preloadedSound.play();
				},
			onComplete: function() {
				currentSound = null; currentChannel = null;
				(onComplete != null) ? onComplete.call(this, onCompleteParams) : null;
				}
			} );
	}
}
}

 

As you can see it requires all passed Sound objects to be already prealoaded, which is fine if you know what you wanna play.

[m1gu3l]

If I want to build a timeline with f.e. text-to-speech sounds based on user actions, it would be nice if I could skip the preloading part and let the timeline handle it. To achieve this I thought I could just change the duration of child tween as the Sound object finishes preloading (let's say for now all Sound objects are still preloading during the creation of timeline, but are already preloaded when timeline starts) - and it doesn't work so well :/

 

package pl.m1gu3l.gs
{
  import com.greensock.TimelineLite;
  import com.greensock.TweenLite;
  import flash.display.Sprite;
  import flash.events.Event;
  import flash.media.Sound;
  import flash.media.SoundChannel;
  import flash.utils.Dictionary;

  /**
   * ...
   * @author michal.przybys
   */

  public class SoundTimeline extends TimelineLite {

      private var currentSound:Sound;
      private var currentChannel:SoundChannel;
      private var pausedPosition:Number;

      private var dict:Dictionary;

      /**
       * @param vars optionally pass in special properties like useFrames, onComplete, onCompleteParams, onUpdate, onUpdateParams,
       * onStart, onStartParams, tweens, align, stagger, delay, reversed, and/or autoRemoveChildren.
       */
      public function SoundTimeline(vars:Object = null) {
          if (vars == null) vars = { paused: true }
          dict = new Dictionary(true);
          super(vars);
      }

      /**
       * Converts Sound to TweenLite and appends it to timeline
       *
       * @param sound Sound
       * @param onComplete Function to be fired after Sound completes
       * @param onCompleteParams an Array of parameters for onComplete function
       */
      public function appendSound(sound:Sound, onComplete:Function = null, onCompleteParams:Array = null):void {
          this.append(soundToTween(sound, onComplete, onCompleteParams));
      }

      /**
       * Converts Sound to TweenLite and prepends it to timeline
       *
       * @param sound Sound
       * @param onComplete Function to be fired after Sound completes
       * @param onCompleteParams an Array of parameters for onComplete function
       */
      public function prependSound(sound:Sound, onComplete:Function = null, onCompleteParams:Array = null):void {
          this.prepend(soundToTween(sound, onComplete, onCompleteParams));
      }

      /**
       * Converts an Array of Sound objects to TweenLite and appends it to timeline
       *
       * @param sound an Array of Sound objects
       * @param onComplete Function to be fired after last of passed Sound objects completes
       * @param onCompleteParams an Array of parameters for onComplete function
       */
      public function appendMultipleSounds(sounds:Array, onComplete:Function = null, onCompleteParams:Array = null):void {
          for (var i:int = 0; i < sounds.length ; i++) {
              var tween:TweenLite = (i == sounds.length - 1) ? soundToTween(sounds[i], onComplete, onCompleteParams) : soundToTween(sounds[i]);
              this.append(tween);
          }
      }

      /**
       * Converts an Array of Sound objects to TweenLite and prepends it to timeline
       *
       * @param sound an Array of Sound objects
       * @param onComplete Function to be fired after last of passed Sound objects completes
       * @param onCompleteParams an Array of parameters for onComplete function
       */
      public function prependMultipleSounds(sounds:Array, onComplete:Function = null, onCompleteParams:Array = null):void {
          for (var i:int = sounds.length - 1; i >= 0; i--) {
              var tween:TweenLite = (i == 0) ? soundToTween(sounds[i], onComplete, onCompleteParams) : soundToTween(sounds[i]);
              this.prepend(tween);
          }
      }

      /**
       * Converts Function to TweenLite and appends it to timeline
       *
       * @param f Function to be fired
       * @param params an Array of parameters for passed function
       */
      public function appendFunction(f:Function, params:Array = null):void {
          this.append(TweenLite.delayedCall(0.1, f, params));
      }

      /**
       * Converts Function to TweenLite and prepends it to timeline
       *
       * @param f Function to be fired
       * @param params an Array of parameters for passed function
       */
      public function prependFunction(f:Function, params:Array = null):void {
          this.prepend(TweenLite.delayedCall(0.1, f, params));
      }

      override public function stop():void {
          super.stop();
          if (currentChannel != null) currentChannel.stop();
          currentChannel = null;
          currentSound = null;
      }

      override public function pause():void {
          super.pause();
          if (currentChannel != null) {
              pausedPosition = currentChannel.position;
              currentChannel.stop();
          }
      }

      override public function resume():void {
          super.resume();
          if (currentSound != null) currentChannel = currentSound.play(pausedPosition);
      }

      /**
       * Converts Sound to TweenLite and returns it
       *
       * @param sound Sound
       * @param onComplete Function to be fired after Sound completes
       * @param onCompleteParams an Array of parameters for onComplete function
       * @return com.greensock.TweenLite
       */
      public function soundToTween(sound:Sound, onComplete:Function = null, onCompleteParams:Array = null):TweenLite {
          if (sound == null) throw new Error('sound passed is null');

          var time:uint;

          if (!isPreloaded(sound)) {
              time = 1;
              sound.addEventListener(Event.COMPLETE, setDurationAfterLoaded);
          } else {
              time = sound.length / 1000;
          }

          var tl:TweenLite = new TweenLite(this, time, {
              onStart: function() {
                  currentSound = sound;
                  currentChannel = currentSound.play();
                  trace('start');
              },
              onComplete: function() {
                  trace('complete');
                  currentSound = null; currentChannel = null;
                  (onComplete != null) ? onComplete.call(this, onCompleteParams) : null;
                  }
              } );

          dict[sound] = tl;

          return tl;
      }

      private function setDurationAfterLoaded(e:Event):void {
          trace('setduration', dict[e.target].duration, totalDuration);
          dict[e.target].duration = e.target.length / 1000;
          totalDuration += e.target.length / 1000 - 1;
          trace('to', dict[e.target].duration, totalDuration);
      }

      private function isPreloaded(sound:Sound):Boolean {
          return sound.bytesTotal != 0 ? (sound.bytesLoaded >= sound.bytesTotal) : false;
      }
  }
}

 

any suggestions?

[Carl]

hey, that looks really impressive. I would love to see some sample implementations.

 

I'm assuming you can easily append a SoundTimeline into a regular TimelineLite.

 

Can a SoundTimeline be reversed?

 

What happens if a TimelineLite containining a SoundTimeline gets reversed?

 

very curious.

 

I totally commend you on you efforts and creativity! I'd imagine something like this has a lot of potential.

 

Carl

 

EDIT: I did not see your second post asking for suggestions until after posting this. I unfortunately do not have any suggestions.

[m1gu3l]

It has a TimelineLite as a superclass so it basically has all the stuff implemented by Jack, you can nest and nest something inside it and so on...

it can be reversed, but it won't play sounds while so - but feel free to alter the class and implement this.

 

I attached a sample Flash CS5/Flash Develop project

[m1gu3l]

The main problem is:

* if sound is not preloaded, it has no duration

* if it has no duration, i need to set some predefined duration for tween

* when sound preloads, i need to alter preset duration of tween to duration of sound

* it doesn't work

[Carl]

thanks for posting the file and answering my questions. I will check it out.

 

 

I'm sure GreenSock will pop in eventually. I'm not cut out to help with the finer details of TimelineLite.

 

Best,

 

Carl

[GreenSock]

I'm beyond swamped at the moment, so I don't have time to look at this in depth yet, but I'll offer a few things:

 

1) Thanks for sharing! It's great to see the effort you put into this.

 

2) Sound just doesn't work well with tweens because of several factors. What happens if you alter the timeline's timeScale to slow it down or speed it up? As you said, reversing also doesn't work. And what if I tween the currentTime with an ease? See the problem(s)?

 

3) It might be better to think about creating a TweenPlugin that would handle playing a sound. That seems like it would be a cleaner solution (maybe - I haven't tried it myself and it will still have the same challenges as your current implementation, but having it as a tween rather than a timeline seems more appropriate).

 

I hope this isn't discouraging - again, I appreciate the effort you put into this and it could certainly be useful in certain situations.

[m1gu3l]

Oj, I am completely aware it's gonna have drawbacks, I don't think anyone would want timeScale to alter Sound (however it is possible, via SampleDataEvent), it should also be possible to cover other problems, but it wasn't the goal.

I am looking for just easy, rapid way of content generation

 

Good idea with making this as a plugin, thanks.

[m1gu3l]

Working (99%) version.

 

package pl.m1gu3l.gs 
{
import com.greensock.easing.Linear;
import com.greensock.TimelineLite;
import com.greensock.TweenLite;
import flash.events.Event;
import flash.media.Sound;
import flash.media.SoundChannel;
import flash.utils.Dictionary;

/**
 * ...
 * @author michal.przybys
 */

public class SoundTimeline extends TimelineLite {

	private var currentSound:Sound;
	private var currentChannel:SoundChannel;
	private var pausedPosition:Number;

	private var dict:Dictionary;

	/**
	 * @param vars optionally pass in special properties like useFrames, onComplete, onCompleteParams, onUpdate, onUpdateParams, 
	 * onStart, onStartParams, tweens, align, stagger, delay, reversed, and/or autoRemoveChildren.
	 */
	public function SoundTimeline(vars:Object = null) {
		if (vars == null) vars = { paused: true }
		dict = new Dictionary(true);

		super(vars);
	}

	/**
	 * Converts Sound to TweenLite and appends it to timeline
	 * 
	 * @param sound Sound
	 * @param onComplete Function to be fired after Sound completes
	 * @param onCompleteParams an Array of parameters for onComplete function
	 */
	public function appendSound(sound:Sound, onComplete:Function = null, onCompleteParams:Array = null):void {
		this.append(soundToTween(sound, onComplete, onCompleteParams));
	}

	/**
	 * Converts Sound to TweenLite and prepends it to timeline
	 * 
	 * @param sound Sound
	 * @param onComplete Function to be fired after Sound completes
	 * @param onCompleteParams an Array of parameters for onComplete function
	 */
	public function prependSound(sound:Sound, onComplete:Function = null, onCompleteParams:Array = null):void {
		this.prepend(soundToTween(sound, onComplete, onCompleteParams));
	}

	/**
	 * Converts an Array of Sound objects to TweenLite and appends it to timeline
	 * 
	 * @param sound an Array of Sound objects
	 * @param onComplete Function to be fired after last of passed Sound objects completes
	 * @param onCompleteParams an Array of parameters for onComplete function
	 */
	public function appendMultipleSounds(sounds:Array, onComplete:Function = null, onCompleteParams:Array = null):void {
		for (var i:int = 0; i < sounds.length ; i++) {
			var tween:TweenLite = (i == sounds.length - 1) ? soundToTween(sounds[i], onComplete, onCompleteParams) : soundToTween(sounds[i]);
			this.append(tween);
		}
	}

	/**
	 * Converts an Array of Sound objects to TweenLite and prepends it to timeline
	 * 
	 * @param sound an Array of Sound objects
	 * @param onComplete Function to be fired after last of passed Sound objects completes
	 * @param onCompleteParams an Array of parameters for onComplete function
	 */
	public function prependMultipleSounds(sounds:Array, onComplete:Function = null, onCompleteParams:Array = null):void {
		for (var i:int = sounds.length - 1; i >= 0; i--) {
			var tween:TweenLite = (i == 0) ? soundToTween(sounds[i], onComplete, onCompleteParams) : soundToTween(sounds[i]);
			this.prepend(tween);
		}
	}

	/**
	 * Converts Function to TweenLite and appends it to timeline
	 * 
	 * @param f Function to be fired
	 * @param params an Array of parameters for passed function
	 */
	public function appendFunction(f:Function, params:Array = null):void {
		this.append(TweenLite.delayedCall(0.1, f, params));
	}

	/**
	 * Converts Function to TweenLite and prepends it to timeline
	 * 
	 * @param f Function to be fired
	 * @param params an Array of parameters for passed function
	 */
	public function prependFunction(f:Function, params:Array = null):void {
		this.prepend(TweenLite.delayedCall(0.1, f, params));
	}

	override public function stop():void {
		super.stop();
		if (currentChannel != null) currentChannel.stop();
		currentChannel = null;
		currentSound = null;
	}

	override public function pause():void {
		super.pause();
		if (currentChannel != null) {
			pausedPosition = currentChannel.position;
			currentChannel.stop();
		}
	}

	override public function resume():void {
		super.resume();
		if (currentSound != null) currentChannel = currentSound.play(pausedPosition);
	}

	/**
	 * Converts Sound to TweenLite and returns it
	 * 
	 * @param sound Sound
	 * @param onComplete Function to be fired after Sound completes
	 * @param onCompleteParams an Array of parameters for onComplete function
	 * @return com.greensock.TweenLite
	 */
	public function soundToTween(sound:Sound, onComplete:Function = null, onCompleteParams:Array = null):TweenLite {
		if (sound == null) throw new Error('sound passed is null');

		var time:Number;

		if (!isPreloaded(sound)) {
			time = 2;
			sound.addEventListener(Event.COMPLETE, setDurationAfterLoaded);
		} else {
			time = sound.length * 0.001;
		}

		var tl:TweenLite = new TweenLite(this, time, {
			ease: Linear.easeNone,
			onStart: function() {
				currentSound = sound;
				if (!isPreloaded(sound)) {
					pauseForLoading(sound);
				} else {
					currentChannel = currentSound.play();
				}
			},
			onComplete: function() {
				currentSound = null; currentChannel = null;
				(onComplete != null) ? onComplete.call(this, onCompleteParams) : null;
				}
			} );

		dict[sound] = tl;

		return tl;
	}

	private function pauseForLoading(sound:Sound):void {
		sound.addEventListener(Event.COMPLETE, resumeAfterLoaded);
		super.pause();
	}

	private function resumeAfterLoaded(e:Event):void {
		e.target.removeEventListener(Event.COMPLETE, resumeAfterLoaded);
		super.resume();
		currentChannel = currentSound.play();
	}

	private function setDurationAfterLoaded(e:Event):void {
		e.target.removeEventListener(Event.COMPLETE, setDurationAfterLoaded);
		var time:Number = e.target.length * 0.001;
		var tween:TweenLite = dict[e.target];
		tween.duration = time;
		shiftChildren(time, true, tween.startTime + 0.1);
	}

	private function isPreloaded(sound:Sound):Boolean {
		return sound.bytesTotal != 0 ? (sound.bytesLoaded >= sound.bytesTotal) : false;
	}

}

}

 

the remaining not working 1% throws null pointer error @ get totalDuration() of TimelineLite, when preceding tween is really short

 

override public function get totalDuration():Number {
		if (this.cacheIsDirty) {
			var max:Number = 0, end:Number, tween:TweenCore = (this.gc) ? _endCaps[0] : _firstChild, prevStart:Number = -Infinity, next:TweenCore;
			while (tween) {
				next = tween.nextNode; //record it here in case the tween changes position in the sequence...

				if (tween.cachedStartTime < prevStart) { //in case one of the tweens shifted out of order, it needs to be re-inserted into the correct position in the sequence
					this.insert(tween, tween.cachedStartTime - tween.delay);
					// ----- prevNode is null --------------------------------------------------
					prevStart = tween.prevNode.cachedStartTime;
					// after changing to the following, it works 100%
					// prevStart = tween.prevNode ? tween.prevNode.cachedStartTime : 0;
				} else {
					prevStart = tween.cachedStartTime;
				}
				if (tween.cachedStartTime < 0) { //children aren't allowed to have negative startTimes, so adjust here if one is found.
					max -= tween.cachedStartTime;
					this.shiftChildren(-tween.cachedStartTime, false, -9999999999);
				}
				end = tween.cachedStartTime + (tween.totalDuration / tween.cachedTimeScale);
				if (end > max) {
					max = end;
				}

				tween = next;
			}
			this.cachedDuration = this.cachedTotalDuration = max;
			this.cacheIsDirty = false;
		}
		return this.cachedTotalDuration;
	}

[m1gu3l]

TweenPlugin handling sound would be problematic, and I believe it can't be done for not-preloaded Sounds within current greensock architecture, because:

* still preloading Sound object has no length or incorrect length

* you can pause tween and wait for Sound object to load, but you can't pause a tween's parent (Timeline)

* you can change tween duration, but you can't shiftChildren of tween's parent (Timeline)

 

Correct me, if I am wrong, because I would love to do this

[GreenSock]

TweenPlugin handling sound would be problematic, and I believe it can't be done for not-preloaded Sounds within current greensock architecture, because:

* still preloading Sound object has no length or incorrect length

* you can pause tween and wait for Sound object to load, but you can't pause a tween's parent (Timeline)

* you can change tween duration, but you can't shiftChildren of tween's parent (Timeline)

 

Correct me, if I am wrong, because I would love to do this

1) It's true that preloading Sound doesn't have an accurate length, although you can use an algorithm to guesstimate it (I do this in my MP3Loader class in LoaderMax).

 

2) I don't understand why you'd pause a tween like that or why you'd want to pause the tween's parent timeline. In my opinion, if the associated Sound isn't loaded, it the tween just doesn't play at that point in time, but as soon as it's loaded, it picks up wherever it needs to. I think it would be dangerous/unintuitive to have a tween's "paused" state change based on anything except the developer setting it directly. Otherwise, what if it's supposed to be paused initially and then the sound loads? You'd need to track that additional data. And unpausing a tween affects its startTime in its parent timeline - that would be undesirable in this scenario.

 

3) Why would you need to shiftChildren() in the tween's parent?

[m1gu3l]

1) It's true that preloading Sound doesn't have an accurate length, although you can use an algorithm to guesstimate it (I do this in my MP3Loader class in LoaderMax).

 

2) I don't understand why you'd pause a tween like that or why you'd want to pause the tween's parent timeline. In my opinion, if the associated Sound isn't loaded, it the tween just doesn't play at that point in time, but as soon as it's loaded, it picks up wherever it needs to. I think it would be dangerous/unintuitive to have a tween's "paused" state change based on anything except the developer setting it directly. Otherwise, what if it's supposed to be paused initially and then the sound loads? You'd need to track that additional data. And unpausing a tween affects its startTime in its parent timeline - that would be undesirable in this scenario.

 

3) Why would you need to shiftChildren() in the tween's parent?

 

1. true, you can, but only if sound is already preloading when you check.

2. the goal was to get something like this (---animation---###sounds###---animation---###sound###---animation---) in an easy way - if i don't pause the timeline and wait for sound to preload and just play it, i could loose sync on low-bandwidth connection. i totally agree it's not a good code, but it works for this one purpose. about pausing, i think i could hack around by overriding paused getter and setter and using some additional variable to define whether it was paused by user or by itself.

3. because sometimes Sound is neither preloaded nor preloading, i pass predefined length (2s ) to the tween, just in case, and when it loads i alter it to correct length - and to sync with next tweens in timeline i need to alter their startTime params.