Red9 TalkBack LipSyncing solutions for Facial

Red9 TalkBack is our audio driven lip-syncing service we offer to clients. Developed over the last 5 years this uses state of the art algorithms and articulation processing to realistically drive any facial rig, purely from an audio file. 

This is an off-line process, allowing us to achieve far better results that standard engine driven systems but at the expense of being a non real-time solution. Perfect for mid-range cutscenes or projects that can't run to the expense of full performance capture. 

Initially developed to help us test our own Red9 Facial Rigs, TalkBack has matured into a huge system that we're now offering clients. We've already run through a number of high end game projects (all awaiting release) as well as several VFX projects. The system is also flexible enough to also produce accurate results on cartoon data and the more we process, the better the learning algorithms get. 

For more information please contact us to see how we could help streamline your production! 

info@red9consultancy.com

Red9 ProPack: Animation ReMapping

It's been a while since I undated this Blog, mainly because this will get rolled into the Red9 Consultancy website shortly. However, I wanted to add this new post as this is a really big workflow feature that we've been working on for ProPack and one that other studios really should take a serious look all.

The Animation Re-Targetter (Pro_to_BND ) is a huge part of the Red9 ProPack pipelines and workflows. Designed over the last 5 years to offer clients the ability to remap, on mass, any animation library to a fresh rig and bounce that data over to various formats.

 If you're working with us all our Red9 PuppetRigs come with the base BND binder file, if not you can still create these manually within the toolsets. We go through some of the extreme mapping added to the systems, including the new Spine re-fit solver, giving far more consistent data, perfect if you're moving data between MoBu and Maya consistently.

This is the absolute backbone of a modern pipeline, giving you the ability to simply reuse animation data from any source in your project.

As always please drop me a line if you'd like to take the ProPack for a test drive!

info@red9consultancy.com

thanks

Mark

Red9 ProPack continues to grow from strength to strength

With GDC in full swing we thought it a good time to go through some of the recent features of our Red9 ProPack pipeline. ProPack is available in both nodelocked and floating studio license, supporting multiple projects with full Perforce backend support.

ProPack continues to grow from strength to strength, we've been working really hard with some key clients recently on some major new features and we're hearing from more and more studios looking for solid solutions for their animation needs. We can generally take a client from having no rig or pipeline to fully setup within a matter of weeks and that also includes facial rigs and data!




ART Rig Support:
Firstly the latest build of ProPack includes full integration for the Unreal ART Rig, allowing you to hook ProPack systems directly to the rig as if it were a native Red9 Puppet. This is a big advantage to anybody wanting a complex workflow over and above the base systems of the Unreal Art Toolkit.

ART Rig - Read More




Animation ReDirector:
The Re-Director has always been a big part of the ProPack but in the latest builds this is now an interactive tool, allowing for instant feedback and re-pathing of your complex animation data. 

Animation ReDirector - Read More




Rig Manager: 
This bridges the gap between any external Maya rigs and our ProPack metadata systems, allowing for fast binding to the underlying node setups and exposing all the tools with a few clicks.  This is particularly powerful when bound to facial rigs as it also then exposes all of our imageplane and camera management specifically aimed at facial animators.

Rig Manager - Read More




Red9 PuppetRig: 
Our PuppetRig continues to develop along with the toolsets that manage it. This latest version includes a new iterative FK>IK spine fitting system, designed to resolve the issue of poorly fitting ik spline fit to fk data. The iterative fit works a little like a secondary ik solver, trying to best fit the systems.

Red9 PuppetRig updates - Read More




Dynamic Timecode: 
Finally we'd like to introduce you to our dynamic timecode systems, designed to allow for seamless syncing of data from performance capture to final sequence, managing body, facial and audio data inside Maya, none destructive and fully integrated into the entire ProPack pipeline. This system was built alongside CloudImperium games to help them handle their massive moCap needs..

Dynamic Timecode - Read More


More to come including our powerful custom exporter systems aimed at taming your outputs to engine!

For more information don't forget to contact us!!

Red9 ProPack: Project and Perforce Handling

Red9 Consultancy Link

In this demo we go through our new Project Handlers, expose newly upgraded Perforce Integration and go through our r9File class object for all TDs out there wanting an easy life.

 This is a hugely important concept for anybody running ProPack, especially within a large studio environment, if you’ve got a demo build of ProPack please ensure you watch at least the first 5 minutes of this video as its a CRUCIAL concept within the pipeline!

 The Project Handlers manage all aspects of our systems, anything that requires a variable, a path, a class binding or a settings file they all inspect this system, allowing dependencies to be dynamically changed by simply switching the project mount.

 More importantly the system now has deep Perforce integration, managing both server and workspace connection per project. Perforce is also exposed within the Red9 Browser, allowing you to seamlessly browser through your full P4 repository directly within Maya with options to checkout, revert, sync, add etc ..

 Finally we go through the r9File handler, a high level file object that binds all of the project systems together, exposing both P4 handling and base OS management in a really simple python object for all TDs to use.

Please connect us for more details

Red9 ProPack: upgraded FK>IK matching systems and ParentSpace management added to ProPack

Delighted to be able to announce a massive upgrade in the way that we deal with FK>IK systems and ParentSpaces within the Red9 ProPack pipeline. Key to this is a new Spine matching algorithm that we've hooked up to the Red9 PuppetRig to give us a really solid match between FK and IK spine systems, something that always been very difficult to achieve.

Also added are upgrades to the Limb matching to deal with FK twist when switching over to IK, new toe rotate extraction and full management of our Auto-Clavicle setup withing the rig. This allows clients to animate with the solver on but always bake it back down to clean extracted rotate data.

 
 

On top of all of this we've added ParentSpace matching to the systems so now you really can mess with the world space of the rig and always push systems around to match over time.

All of the new features also include a new SmartBake setup such that data can be matched to current keys within the system rather than per frame baked data, perfect for clients hand-keying animation rather than moCap data.

As always check-out our other updates on the news feed of our website or on our Vimeo Channel

thanks

Mark

Red9 ProPack : HealthManager

The Red9 Health Manager concept is a massive part of the Red9 ProPack Pipeline, designed to allow TD’s to sanity check files in a neat, clear API with full UI support. Write your own simple compounds either using our default tests or write your own from the ground up and use those to validate your files before pushing them out to production.

I can’t stress enough just how much this has saved our lives over the last few years, allowing us to ensure all our assets are solid before we send them out to clients. This demo first goes through the initial concepts and UI, then we cover some of the more complex tests. Finally we go through how to write your own tests and Validation classes.

Note: if you’re running blendshapes and having fbx index issues watch on!!




As always, please checkout the Red9 website for more details, and don't forget to subscribe to the new YouTube channel!

thanks

Mark

For those of you who don't follow the Red9 website news section I highly recommend you add it to your RSS feed. The 2 links below are new demos that go into much more detail than ever before about the whole Red9 ProPack and PuppetRig workflows.

Demo 1 : Red9 PuppetRig

This new series of videos are designed to give you a wider overview of the Red9 ProPack and what it’s like to work with us on a professional level to help streamline and speed-up your workflows.

This first video goes through our Red9 PuppetRig solution, the new DagMenu system and dynamic CharacterPicker. Crucially it also shows we use an abstract layer to communicate with all the tools and the API, meaning that none of our systems are tied directly to the rig solution itself.

Whilst we’d love everybody to use the Puppet Rig we’re more than aware that this is an impracticality for many studios who have invested time into their own rigging systems, this is a neat way to simply add a layer of MetaNodes to fool our systems into thinking they’re dealing with a native Puppet Rig.

Demo 2 : Red9 - Managing Animation Deliveries

In this second video in the series we show you just how easy it is to map animation data over to the Red9 Puppet Rig, on-mass, using the Red9 Browser and the “Pro:To_BND” functionality. This hopefully gives you some insight to the power of the Browser when used with the in-built Extensions Manager to add custom functionality, allowing TD’s to expose functions directly to the Animators.

We go through the AnimBinder setup itself, a file that accompanies all Red9 Puppet deliveries. This is ideal for layering fast modifications to any initial character remapping, something only exposed to clients running the Puppet. The Pro Binder setup also natively supports HIK under the hood, allowing you to remap data from any fbx source, including files from MotionBuilder.

Finally we show some of the tricks unique to ProPack to deal with that initial stage of animation clean-up, filtering and re-directing.

Lots of goodies in these rwo videos and more to come

thanks

Mark

Red9 ProPack - Animation Systems updates now available to Clients

It's been a while since I posted on the blog and I thought it about time I did a few new updates to show what's now available in the Red9 ProPack to Clients taking Red9 Support.

Firstly don't worry, StudioPack continues to get a lot of back-end updates and if you go grab latest from GitHub you'll see we've recently added all the ProPack codebase to the API docs so you can see just how much we've extended the pipelines for Pro.

So where's all the development going... well as you'll see below we've been concentrating most of our efforts on the bigger picture, creating the kind of animation tools that keep us on top of the game, offering clients that extra edge and taking the stress out of a modern pipeline.




Red9 Pro : Animation Re-Director : 
I did this first video a little while ago but never added it to the blog. The re-director is a lossless animation re-direction setup allowing you to dynamically re-path animation data in Maya. This make NO connections ot your rig at all, it's all done by caching the anim data and re-applying it rather than making any constraints etc.





Red9 Pro : Animation Format Part1: 
We've spent a huge amount of time in developing a new animation format that does what ATOM should have done in the first place and that is to make animation data easily transferable between rigs, flexible and powerful enough to integrate into other systems.




Red9 Pro : Animation Format Part2 : TIMECODE: 
This one is a huge deal for anybody doing performance capture and managing cutscenes. We embed timecode into the Maya rig allowing us to reload and re-build scenes from multiple r9Anim files by referencing a timecode pointer. This will even allow us to sync animation data based on the internal timecode of a piece of audio.

Many thanks


Mark

Red9 ProPack - First Sneak Peak - Animation ReDirector

We'll we thought it was about time that we started to show you what we've been up to with the Red9 ProPack. This first demo is the Animation ReDirector, a powerful tool for doing just that, taking a current animation and non-destructively and dynamically re-directing the animation. Hope you like it, plenty more to come over the next few months as we gear up to release the new tools pack, we just need to work out the licensing options and the back-end of the code!

 cheers

 Mark

Red9 Language Packs!

Wow, it's been a while since I posted up, hopefully over the next few months we'll be showing a lot more of what we've been up to both with the pro-pack and the consultancy business. We we're recently involved in a number of TV commercials which we should be able to show very soon, one a full performance facial setup for a Toyota commercial so watch this space!

In the mean time, I wanted to talk about language packs and how much interest there may be out there for us shipping Red9 StudioPack in multiple languages?

We were talking to some Chinese clients earlier today and the topic came up again so, I've been busy switching all the menu text over to a mapper file that we can easily replace and get translated. It's basically just a Python module with all the text in it that we import on boot, all we'd have to do to change the language is change the binding.

So, who out there would like to see this, and if so, what languages, would you be interested in helping us translate the mapping file?

thanks

Mark

Progress!!!

I thought it about time I did an update just so it doesn't seem like I've abandoned Red9!

Absolutely nothing could be further from the truth, the Studio Pack is getting some huge upgrades in preparation for the Pro-Pack, lots of extra support for transfer methods to take data between rigs, new animation file format, extensions to the pose format and a ton of other features and tweaks.

So what have we been up to?? In integrating the Pro-pack it's taking time restructuring the codebase, internally we now have 3 different modules running, the main StutioPack, the new Pro-Pack and our own internal systems which now include our new procedural rigging system and facial setup. We're deep in production with a few exciting clients and the new Red9Consultancy business is really starting to take off, thanks to all the support to everybody out there for spreading the news about the StudioPack ;)

Once we've finished the two projects we're working on we're hoping to update the main web-site so everybody can see what we're up to, and hopefully choose Red9 for their consultancy work, or rigging and facial solutions.

If you haven't already been over to the new website go for it:

red9consultancy.com

thanks

Mark

More Audio goodness

I've just added extra functionality to the Audio handlers in Red9 that hopefully you'll all find useful. 

Firstly I've expanded on the conversion tools to handling timecode, we now have the full complement:

milliseconds_to_Timecode(milliseconds, smpte=True, framerate=None)
milliseconds_to_frame(milliseconds, framerate=None)
timecode_to_milliseconds(timecode, smpte=True, framerate=None)
timecode_to_frame(timecode, smpte=True, framerate=None)
frame_to_timecode(frame, smpte=True, framerate=None)
frame_to_milliseconds(frame, framerate=None)

We're using these internally to process timecode around our MoCap deliveries, converting time to the relevant units as and when required.

I've also added a new function to the Red9_Sound menu in the Trax Editor, 'Sync_BWavs' which is designed to automatically sync multiple BWav files to their internal timecode. BWav's are Braodcast Wav's used when recording audio with timecode as it embeds the data into the wav's header, very useful in production.

The new function processes either the selected audio nodes in Trax, or ALL wavs in the scene, depending on selection. It then inspects the internal BWav's timecode, and if found and valid it will shift the audio nodes in Maya to match their original timecode's, re-syncing them to the original shoots timelines. 

I've also done a few fixes to the AudioNode class, adding the setters that were missing in some of the properties.

cheers

Mark

New AnimUI functions....

So a few updates. I've been working on the back-end to the AnimUI a fair bit recently and have exposed a few new options that should be useful to you all.

First one is the matchMethod - this is the code that takes 2 hierarchies and matches them, node for node to them process correctly. This is used by all the anim code calls, copyAttrs/Keys, poseLoad etc so it's a really low level feature. Now this has been there for a while but it wasn't actually hooked up correctly...ooops! There's been a flag to control the matchMethod but it was missing an option, and that in turn has caused a few of you issues.

The new option is 'index' and that turns OFF all the matching code and just takes the 2 hierarchies that the filter found, and zips them together in the order they were found. Why? well this is what Maya native does on all hierarchies, absolutely no intelligence just one to one mapping. It will now allow things like this to match:

AlphaRig_WristCtrl == BetaRig_WristCtrl

previously this would never match as there's no naming match there, other than me allowing you to enter a search and replace to the UI. With the index option it will match, as that node was found as the 4th child of the hierarchy in both rigs. This also means that the Pose's stored for Alpha_Rig will load correctly to Beta_Rig ;)

So what else, well I've added sort logic to the poseUI. You can do a number of new things in there. Firstly the searchBox takes comma separated searches, so you can add "walk,run,idle' and it will show those as a filter. You'll also notice there's 2 new icons, sort by name, and sort by date. This has been getting on my nerves as my facial library at work is 80+ shapes and if I add a new one I have a lot of scrolling to do!! So sort by date is a godsend.

These are in the GitHub branch and will shortly go into another live release, after testing ;)

cheers

Mark

Red9 Sound and Audio handler updates

Finally got round to doing a quick demo of the new AudioNode handler in the Red9 StudioPack. This one shows how you can use the 'compile' function in the sound menu to pre-compile multiple sound nodes in Maya so that you get a complete soundtrack to any playblasts. Anybody who's ever tried animating to multiple sound nodes in Maya, particularly when animating dialog will know how frustrating it is when you then try and playbladt that scene, and only get 1 sound track included!!

I also go through the new inspect code, capable of extracting metaData info from wav files including full support for Broadcast wav's. This is really useful to gain access to thiongs like internal timecode data. There's also extensions capable of extracting metaData from any media format, Mov, avi etc but for that you have to download ffmeg which I use in the inspect call.

I'm really keen to hear from those running MacOs or Linux, does the compiler work for you?

thanks (don't forget the donate button...cough...)

Mark

Red9 goes all GIT!

Well this has been asked for by many of you for a while so I'm finally sorting it out, pushing Red9 repository up onto GitHub. I've always used a private SVN server but this should give people more exposure to the changes happening on a daily basis.

 https://github.com/markj3d/Red9_StudioPack

Still getting by head round GIT so bear with me

Mark

New Pose Saver tools and features

Well it's been a while since I did a demo so here's one that goes through the upgrades to the poseSaver in v1.41.

New PoseBlending: Pose Blending is a new feature that allows you to mix in a percentage of any pose to the current state of your controllers. When you RMB>PoseBlend you get a new slider UI that controls the mix. Note that when this slider is launched the current state of the rig is CACHED against it's current state so please be aware of that. I had to do this to get the slider to react fast enough to make it worth doing.

New MaintainParents: Another big update to the poseLoader. This one allows the pose code to 'hold' or maintain a given set of attributes during pose load. This not only returns the given attrs back to their current state prior to loading the pose, but it also recalculates the pose at the same time. This means that even if a pose is stored with all the controllers in one parent space and your current controller is in a different space, the original stored pose will still be reached, but it'll be recalculated in the current space ;)

This relies on the 'relative space' flag and is only available in this mode as I use this mechanism to do the psace compensation.

Any comments, suggestions or bugs let me know

Mark

Red9 StudioPack - Time and Key Management

Hi all, thought I'd do a new demo to go through some of the new time management tools in the latest build v1.41 which I'll push out sometime over Christmas. Hopefully this all makes sense although there's so much more that I could expose here, just a case of not going to  mad with complex UIs and too many checkboxes.

Anyway, as always all feedback is very welcome

cheers

Mark

Red9 Studio Pack v1.40 Released

Well at long last I have a build I'm happy with so here it is, Red9 Studio Pack v1.40. This has been a long time coming and there's a number of posts already which go through the main updates.

If you've not already seen it pop onto Vimeo and watch the update v1.40 video. As usual the download link is off the Google Drive link below. Seems like Google have been playing with GDrive and as always it's totally un-obvious how to actually download a zip file! Open the link, when it pops up there's now a download arrow bottom right next to the open button.

http://red9-consultancy.blogspot.co.uk/search/label/Download

At some point I'll get round to doing a proper Red9 web site with download links. I also keep meaning to move to a GIT repository to give you all access to all the commits.

Anyway as always, any issues or feedback mail me. If you'd like to get involved or are using the pack in production I'd love to hear from you. There's a lot of stuff under the hood which can easily be exploited as I am at work.

cheers

Mark

Red9 StudioPack v1.40 - update video :)

Well it's nearly time for another release of the pack, this one will be a big one as there's been a hell of a lot of development behind the scenes to optimize and streamline the workflows. Thought I'd do an update video as it's far easier than trying to write it all down!

Highlights of v1.40:

• New - 'Use Priority as SnapList' in the hierarchy tab. This is THE biggest upgrade for a while, see the video as this is a massive optimization and gets over a ton of issues.
• New Presets in the hierarchy for Stewart, TSM and Andy rigs - these need testing.
• Object and CharacterSets are now handled like any other hierarchy in the filters.
• Red9 HUD integration in the channelBox has been upgraded and now supports both simple Hud elements and sliders.
• Pose Relative upgraded to support the new Snap methods
• SnapTransforms now upgraded to support the new Snap methods
• Track N Stabilize upgraded, now supports separated trans and rots in the tracker. 
• SnapTransforms plugin now takes tran and rot flags
• More functionality in the InteractiveCurve UI
• MirrorUI upgraded, now allows a pre-clear when loading mirrorMaps.

Hopefully I'll be pushing this pack out in the next week or so

cheers

Mark

Interactive FilterCurve update Vimeo

Just a quick update to the Interactive FilterCurve Tool in the StudioPack based on a ton of feedback from a good friend of mine running it in production, cheers Santi ;)

Resample has been added to the tool so that you can resample, interactively over time, effectively baking your data down interactively. The Filter now also has a clamp to frame option so you end with whole frames rather than subframes.

Thanks for all the support

Mark