Character Lip Sync with text content

  • sazakir
  • Novice
  • Novice
  • sazakir
  • Posts: 23
  • Loc: India

Post 3+ Months Ago

I have a movie with an animation character, a text box. The text is dynamic, ie. its fetched from the db. I want to sync the lip movement of the ani character, as if its reading the text. A voice file will be played in the background.

Can anyone provide sample code for this ?
  • MYz160
  • Born
  • Born
  • User avatar
  • Posts: 3

Post 3+ Months Ago

You're fighting a losing battle, having a character realistically lip synch on the fly from text extracted from a db is damm difficult. Believe me, companies do this.

What it boils down to are distinguishing phenomes (basic speech pronounciation fragments) from the text, and then getting the character to respond to the phenomes by moving lips accordingly and speaking.

Recommended program flow

Plain text -----> Phenomic and letter-based verbal differences ----->
Speech Engine |___> Lip synch engine

The difficulties you may face are
1) the smooth transition between the hundreds of dual phenome combinations visually and spoken
2)Distinguishing phenomes from the text. Humans do not actually read text the same way, by looking at letters, and phenomes, and prounouncing them. Letters and pronunciations are different in every sentence spoken verbally, which machines find excruciatingly difficult to perform. These variations needs to be detected in the Phenomic and letter-based verbal difference engine

Recommended reading:

Scientific American
June 2005 Conversational computers page 40

Half life 2 also uses a rudimentary (though excellent) text to phenome engine, although the major difference here is that the labour is saved only in character lip animation, the voices are, however pre-recorded

Have fun

Post Information

  • Total Posts in this topic: 2 posts
  • Users browsing this forum: No registered users and 11 guests
  • You cannot post new topics in this forum
  • You cannot reply to topics in this forum
  • You cannot edit your posts in this forum
  • You cannot delete your posts in this forum
  • You cannot post attachments in this forum

© 1998-2017. Ozzu® is a registered trademark of Unmelted, LLC.